Ruby 1.8 brought us a couple changes that made many kinds of metaprogramming easier, plus a whole bunch of library additions that made Ruby feel more "grown up". Without seeking external libraries, one could write Ruby to solve many problems developers face in commonplace jobs. I wasn't around for Ruby 1.6, but I've been thinking of Ruby 1.8 as a transition from "better Perl or Java" to "better Smalltalk".

Ruby 1.9 brought us features that make some functional programming idioms easier. Lambdas, i.e. anonymous functions, require less syntax and are better defined. Enumerators make it possible to use features of Enumerable, itself a very functional-esque feature, in more places. Symbol-to-proc makes it easier to pass methods around as blocks, another FP-esque practice. I might say that Ruby 1.9 is the "better MatzLisp" version of Ruby.

Ruby 2.0 is bringing us features that, on the surface, make it easier for Rails to extend the Ruby language via ActiveSupport. I think that's too shallow of a reading. The new tools in Ruby 2.0 (excepting the highly-controversial refinements) make it easier to cleanly add functionality to Ruby's core objects and library. Reducing the cost of extending the core make it possible for more libraries and applications to judiciously make high-leverage additions to the lower levels of Ruby. That seems like a pretty good thing.

I can't find a source for this, but I could have sworn I once read that all programming is language design. It was probably related to Lisp, where you're arguably directly manipulating the AST much of the time. If the changes in Ruby 2.0 can take us closer to this level of program design, where we think more about building language up to the problem domain instead of objects and mechanism, sign me up.

How many design considerations are there in an almost trivial method? Let's look at two of them. Consider this code:

def publish!
  self.update_attributes(created_at: Time.now)
end

If you've been studying OO design and the SOLID principles, using TDD as a practice to guide you towards those ideas, there's a missing piece here. The reference to Time is a dependency that should be injected. In Ruby, it's really easy for us to fix that:

def publish!(time=Time.now)
  self.update_attributes(created_at: time)
end

I suspect a lot of TDDers would instinctively write the above first, skipping the first version by force of habit. But, let's stop and think about what the drives us to want the second version.

The strength of the second version is that it is designed for test. If we need to test how this model behaves when it is published at night, or on a leap day, or the day before Arbor Day, injecting the time object makes that easier.

There are some other test-focused design direction this method could go. We could create our own object whose role is to hand out timestamps, which would allow us to reasonably stub out the time reference, instead of injecting it. I'll bet there are other approaches lurking out there as well.

I want to look at another set of design considerations. I could design this code for testability, which often leads me to code that follows the SOLID principles which often leads me to decoupled code that is easier to change later. To many people, that's a good thing.

However, there's another lens I can look through: API design. How does this method hold up as a piece of behavior that developers will leverage?

Strictly speaking, the TDD'd version is a more complicated API. Even adding one optional parameter to a method carries "mass". Consider documenting the parameter-less version:

Publishes the current post. The created_at timestamp is set to the current time. Returns the created_at timestamp.

For numbers sake, it's 40 words. More importantly, it reads linearly. Now let's look at the dependency-injected version:

Publishes the current post. By default, created_at is set to the current time. Optionally, callers may pass in a Time object, or any object that returns a Time object when sent the now message. The created_at column is set according to that Time value. Returns the value of the created_at timestamp.

This one is 54 words. That's not too many more, numerically, but notice that the explanation is no longer linear. There's a default, easy case where I don't care about the timestamp. Then there's a clever case where I do care about the timestamp. In real API documentation, I'd need to specify when and why I'd want to use that clever case and what it looks like.

There's some further potential trouble lurking in this API. What if a caller passes in the wrong kind of Time object? What if sending the now message raises an exception? Those are important parts of the API too, both from a behavior specification perspective and when considering the user experience of using this API in code and possibly troubleshooting it when things go wrong.

My point is, that optional argument is starting to look rather weighty. Adding the code is pretty trivial. The possible interactions with the optional argument and its support cost is where it gets expensive. Like many things, it's a trade-off.

I won't claim to know which of these is better. Honestly, I think it comes down to a subjective view on what's important: test design, or API design. This is where I can't make a bold-sounding prognosis. I believe that design, even of code, is about deciding what to leave out. Everyone has to decide what to leave out for themselves.

A lot of discussions on software design end up focusing on dependencies and coupling. In short, hell is dependencies and the couplings it produces. It's a tricky problem because its hard to look at some program text and see all of its dependencies; some of them require intelligence to recognize.

In Ruby, we don't have very good ways to declare a class's dependencies and no ways ways to declare its couplings. We can describe a project's dependencies with a Gemfile or a file's dependencies with requires. The trick is that these specifications often explode in complexity. Requiring Ruby's thread library brings in some thread-safe data structures like queues and condition variables. Requiring ActiveRecord brings in a world of dependencies and causes a number of behavioral changes to Ruby that some consider impolite.

In some tinkerings with Clojure this weekend, I was struck how the ns function is more effective at both declaring dependency and coupling and in restricting the possible distress those qualities may bring. Consider this snippet from my weekend project:

(ns hrq.routes

  (:use compojure.core

        hrq.core)

  (:require [compojure.route :as route]

            [compojure.handler :as handler]

            [ring.middleware.params :as params]))

I have pedantic quibbles with ns, but I like what's happening here. This file can only use the functions in compojure.core and hrq.core with no namespace qualifications. This file can only use the functions from compojure.route, compojure.handler and ring.middleware.params when they are qualified with the proper prefix. So now I have a very good idea of what code this particular file depends on and where I should look to find behavior that this file is subject to.

To a lesser extent, I have a good guess about what state this file depends on. If there are dynamically scoped variables (pardon me if those are the wrong Clojure/Lisp words) in the dependencies declared for this file, I would need to care about them. If those files are pure behavior (i.e. referentially transparent pure functions), I have nothing to worry about.

Clojure isn't perfect in this regard; it does allow mutations and state changes outside of functions. It's not strictly referentially transparent like Haskell is. The tradeoff is worthwhile, in my opinion. Admit some possible coupling in exchange for ease of building typical programs.

I'm not sure that Clojure is inherently superior to Ruby in this regard. It's possibly a momentary cultural advantage, a reaction by those who were burned by expansive, implicit dependencies in Ruby and other languages. That said, it's a good example of Clojure’s considered separation of concerns solving problems that are quite thorny in other languages.

In response to my hypermedia opinions, Mike Kelly said:

These two seem to conflict: “In my opinion, abstract container formats aren’t useful.” and “Just use JSON”. People normally talk about “generic” media types, but they don’t have to a “container” at all, they can simply add conventions for linking. Having conventions for this stuff is useful because it allows us to build tooling around it, if everyone reinvents the wheel in their own way then we can’t build re-usable code. For a similar reason, “specifying your own custom MIME types” is not a good idea – there’s also the time cost associated with doing that. If you use something like hal+json you avoid that cost, and can concentrate on establishing your API’s workflows via link relations.

The logic behind the madness goes like this: abstract containers aren’t solving a problem I currently have. Unfortunately, this means they create problems for me. In the end, I’m building an API to provide functionality, not as advocacy.

As of summer 2012, there are ideas like HAL, JSON collections, etc. and specifications of those ideas. There are very few implementations. As a service provider, I wouldn’t actually get any benefit out of using those formats. The convention, within my own API, that fields ending in _url are links is sufficient. I’d actually end up net negative, because I’d have to explain how e.g. HAL works and support client developers seeking to understand how to work with it. Anyone building to my API would likely end up having to write their own HAL code, so they don’t benefit much either.

I’ve decided to use JSON because providing an API that returns HTML and expects users to scrape it via selectors is extremely confusing to developers. Keep in mind, not everyone is savvy to the latest development trends. To them, an API means an HTTP service that returns XML or JSON. If I were to embrace HTML as a response type, I’m again stuck with explaining a new concept to client developers.

If it’s not obvious yet, one of my main principles in adopting hypermedia is to avoid educating developers on hypermedia as much as possible. I’m in the game of providing a useful API, not a system that shows off the possibilities of hypermedia and how deeply committed I am to its theories.

Finally, I’ve chosen to craft my own content types because I need some kind of contract with client developers that tells them what kind of data they can expect to see, plus some documentation that expresses that contract in a way that is easy for humans to understand. An RFC-style specification that states what a content type MUST/MAY/SHOULD include is exactly the right kind of abstraction. It allows me to update the specification with a version identifier and specify how each revision changes in terms of what data is available. Further, I found a content type the most tractable solution for specifying the input formats supported by PUT and POST endpoints. None of the abstract containers that existed as of August 2012 fit my needs for specifying links, structure, and how to submit data via POST/PUT/PATCH.

Honestly, link traversal and machine-to-machine interaction are not the pain I’m feeling. What I want is the simplest possible API that allows potential client developers to understand what they can do via our API and how to do it. Further, I want it to be possible, even if it’s not magical or easy, to change the API in the future so I’m not constricted by its current design. I feel no need to apply all of the hypermedia principles to make something useful; I can cherrypick some of the hypermedia principles and still achieves the goal of an API that is stable but not set in stone.

Through the Gowalla API, and now the Sifter API, I’ve worked with a couple systems one could reasonably call a hypermedia API. Since smart people are talking about them today, I feel compelled to throw in my two cents.

A URL-based API alone won’t prevent breakage, as many point out. It won’t even prevent people from figuring out how to extract IDs and continue to bang URL strings together. I don’t know why creating URL strings is the security blanket of many API client developers, but it seems you can’t take it away from a few of them.

That said, a URL-based API does make life easier for good actors. Client developers, especially the ones that can’t upgrade their deployed applications quickly, can sleep better knowing that the upstream API could change and their code won’t break. Service developers can rest easy knowing it’s not easy, but it is possible to change their URL structure if they need to.

A hypermedia API means you can’t skip on documentation. In fact, it means you probably need better documentation. You’ll need to explain what each link in the API responses means (is photos_url the URL for my photos, or my friends’ photos?) and what kind of data they can expect that URL to return (a photo object? a collection of photo objects? a user?) As I mentioned before, a lot of client developers don’t have their minds wired for hypermedia yet, so you’ll need a lot of examples for how they should build their clients.

In my opinion, abstract container formats aren’t useful. Squeezing your application’s data into someone else’s data model is not a great place to live. That said, I do think specifying your own custom MIME types is a promising idea. You specify each of the content types in your API, RFC-style, and then provide examples of that data. Should you need to change your response formats, you update your RFC-style specification, adding a new field as a “MAY provide” type feature.

The rub of this approach is that you can end up with an explosion of response-type handling code in your application. Again, it’s not that hypermedia principles make it easy, they just make it possible.

I’ve seen some hypermedia APIs expressed as hypertext, using HTML and cleverness. I don’t see the benefit of this. Just use JSON, and maybe specify your own response types.

The shortcoming in much of the hypermedia content out there is they focus on clients reading data and hand-wave over clients that need to create or update data. I can see why; I think that machine discoverable parameters are a tarpit. Trying to tell a client what fields the server will accept, what types are accepted, and then handle all the possible error flows is verbose and riddled with edge-cases. The way I’m solving this is to specify, in the documentation, which URLs clients can expect to POST/PUT/etc. data to and what content-type those URLs expect to receive. I’m pretty much doing the least hypermedia-ish thing possible, but I think that’s the correct choice right now.

The great thing about the principles of hypermedia APIs and its discourse if a focus on workflows. The biggest mistake I’ve made in building APIs is in exposing a database instead of a service. No one cares what your table structure or domain model looks like. They want to fetch some data, preferably indexed in a way that is immediately useful to them, and maybe write some data back. When you look through this lens, API design starts to look a lot like UI design. It’s fun!

You should read about hypermedia APIs, if only to challenge your thinking. It helped me a lot to do just that. Check out Steve Klabnik’s Designing Hypermedia APIs; it’s not perfect, but it makes a lot of these things easier to grasp.

In the end, I think designing, building, and supporting a hypermedia API is hard. You may be better off with an RPC-over-HTTP, or ID-based REST design. The good news is you can benefit from an incremental application of the principles of hypermedia APIs; you don’t have to go all in before you get something out of it.

How much easier would Haskell be if its vocabulary wasn’t so deeply rooted in abstract mathematics? How many more people would immediately understand Cassandra if it just adopted the vernacular of row-oriented databases instead of overriding it with column-oriented semantics? Wouldn’t the programming world be better if every language didn’t call itself object-oriented?

So many programmer-facing things could be better if they had better names for concepts. The power of good naming:

Characters are cheap, confusion is costly. Let’s not make things harder for the programmers who come after us. Remember, this is just as likely to be ourselves in a few months. Let’s avoid using a name like prj when project is only four characters more typing. Anything that reduces reading friction in our code is a good thing.

The cynical view is, we are actively confusing and inhibiting ourselves when we use names that are too short, not clear, or outright wrong. The optimistic view is that there’s a progression we can take from not knowing what something should be named, to giving it an acceptable name, to using the naming process to discover better structures.

If you ever hear me say something “needs better words”, it means that I think the idea is right but the labels are wrong. A philosophical dialog may ensue, where I struggle to discover what the essence of the idea is. The end goal is a word that is concise, has the right connotation, and whose meaning is obvious and accessible to as many audiences as possible.

I love naming things.

It’s handy to know a lot about programming langauges, patterns, “best” practices, and anecdotal experience in applying those ideas. But premature application of ideas can lead to its own bad experiences and new anecdotal evidence. How can you apply ideas without falling into the premature architecture/optimization trap?

Follow the smells.

You know what a bad method looks like. You know what slow code looks like and how to find it. You can tell when a class is doing too many things. Follow those smells.

Once you’ve smelled it, you’ve dealt it. Kidding!

Once you’ve found a culprit that is imminently making your life as a developer harder, you have to explain it to someone else. Tell them why it’s slow, badly coupled, or too much architecture. Once they’re convinced, then you can reach into that deep knowledge of languages, patterns, practices, and experience to fix the problem.

You commit the code, push it to production, and do a happy dance. You’re not done.

Now you need to share it. Write down what you did and share it with your team, lest the problem happen again. Show your co-workers how to find the problem elsewhere and quickly dispatch it.

If you do it this way, you’re helping yourself three times:

1. You fixed the original problem
2. You showed your colleagues how to fix the problem or how to avoid creating the problem
3. You showed your colleagues how to make measurable, if minor, progress in making your application better without the need for drastic projects

Follow the smells. Fix the smell. Put the fix in everyone else’s brain. Repeat.

When doing object-oriented programming and following SOLID principles, there is sometimes a concern that classes will proliferate and you end up with a Kingdom of Nouns.

I find it more concerning when there is a proliferation of concepts. Too many layers, too many patterns, too many frameworks. The best designs I’ve seen provide just one or two concepts that manage to tie everything together without breaking orthogonality.

The great thing about Rails and Sinatra was that they boiled previously conceptually heavy domains down to a simpler, better set of ideas. In doing so, they move you up the hierarchy of needs, so to speak, and you still end up inventing your own concepts and nouns. More on that soon!

A lot is made about how difficult it is to write multi-threaded programs. No doubt, it is harder than writing a CRUD application or your own testing library. On the other hand, it’s not as difficult as writing a database or 3D graphics engine. The point is, it’s worth learning how to do. Skipping the hubris and knowing your program will have bugs that require discipline to track down is an enabling step to learning to write multithreaded programs.

I haven’t seen much written about the experience of writing a concurrent program and how one designs classes and programs with the rules of concurrency in mind. So let’s look at what I’ve learned about designing threaded programs so far.

The headline is this: only allow objects in consistent states and don’t rely on changing state unless you have to. Let’s first look at a class that does not embody those principles at all.

class Rectangle
  attr_accessor :width, :height

  def orientation
    if width > height
      WIDE
    else
      TALL
    end
  end

  WIDE = "WIDE".freeze
  TALL = "TALL".freeze
end

Just for fun, mentally review that code. What are the shortcomings, what could go wrong, what would you advise the writer to change?

For our purposes, the first flaw is that new Rectangle objects are in an inconsistent state. If we create an object and immediately call orientation, bad things will happen. If you’re typing along at home:

begin
  r = Rectangle.new
  puts r.orientation
rescue
  puts "whoops, inconsistent"
end

The second flaw is that our object allows bad data. We should not be able to do this:

r.width = 100
r.height = -20
puts r.orientation

Alas, we can. The third flaw is that we could accidentally share this object across threads and end up messing up the state in one threads because of logic in another thread. This sort of bug is really difficult to figure out, so designing our objects so it can’t happen is highly desirable. We want to make this sort of code safe:

r.height = 150
puts r.orientation

When we modify width or height on a rectangle, we should get back an entirely new object.

Let’s go about fixing each of these flaws.

Encapsulate object state with Tell, Don’t Ask

The first flaw in our Rectangle class is that it isn’t guaranteed to exist in a consistent state. We go through contortions to make sure our databases are consistent; we should do the same with our Ruby objects too. When an object is created, it should be ready to go. It should not be possible to create a new object that is inconsistent.

Further, we can solve the second flaw by enforcing constraints on our objects. We use the “Tell, Don’t Ask” principle to ensure that when users of Rectangle change the object’s state, they don’t get direct access to the object’s state. Instead, they must pass through guards that protect our object’s state.

All of that sounds fancy, but it really couldn’t be simpler. You’re probably already writing your Ruby classes this way:

class Rectangle
  attr_reader :width, :height

  def initialize(width, height)
    @width, @height = width, height
  end

  def width=(w)
    raise "Negative dimensions are invalid" if w < 0
    @width = w
  end

  def height=(h)
    raise "Negative dimensions are invalid" if h < 0
    @height = h
  end

  def orientation
    if width > height
      WIDE
    else
      TALL
    end
  end

end

A lot of little things have changed in this class:

• The constructor now requires the width and height arguments. If you don’t know the width and height, you can’t create a valid rectangle, so why let anyone get confused and create a rectangle that doesn’t work? Our constructor now encodes and enforces this requirement.
• The width= and height= setters now enforce validation on the new values. If the constraints aren’t met, a rather blunt exception is raised. If everything is fine, the setters work just like they did in the old class.
• Because we’ve written our own setters, we use attr_reader instead of attr_accessor.

With just a bit of code, a little explicitness here and there, we’ve now got a Rectangle whose failure potential is far smaller than the naive version. This is simply good design. Why wouldn’t you want a class that is designed not to silently blow up in your face?

The crux of the biscuit for this article is that now we have an object with a narrower interface and an explicit interface. If we need to introduce a concurrency mechanism like locking or serialization (i.e. serial execution), we have some straight-forward places to do so. An explicit interface, specific messages an object responds to, opens up a world of good design consequences!

Lean towards immutability and value objects whenever possible

The third flaw in the naive Rectangle class is that it could accidentally be shared across threads, with possibly hard to detect consequences. We can get around that using a technique borrowed from Clojure and Erlang: immutable objects.

class Rectangle
  attr_reader :width, :height

  def initialize(width, height)
    validate_width(width)
    validate_height(height)
    @width, @height = width, height
  end

  def validate_width(w)
    raise "Negative dimensions are invalid" if w < 0
  end

  def validate_height(h)
    raise "Negative dimensions are invalid" if h < 0
  end

  def set_width(w)
    self.class.new(w, height)
  end

  def set_height(h)
    self.class.new(width, h)
  end

  def orientation
    if width > height
      WIDE
    else
      TALL
    end
  end

end

This version of Rectangle further extracts the validation logic into separate methods so we can call it from the constructor and from the setters. But, look more closely at the setters. They do something you don’t often see in Ruby code. Instead of changing self, these setters create an entirely new Rectangle instance with new dimensions.

The upside to this is, if you accidentally share an object across threads, any changes to the object will result in a new object owned by the thread that initiated the change. This means you don’t have to worry about locking around these Rectangles; in practice, sharing is, at worst, copying.

The downside to this side is you could end up with a proliferation of Rectangle objects in memory. This puts pressure on the Ruby GC, which might cause operational headaches further down the line. Clojure gets around this by using persistent data structures that are able to safely share their internal structures, reducing memory requirements. Hamster is one attempt at bringing such “persistent” data structures to Ruby.

Let’s think about object design some more. If you’ve read up on domain-driven design, you probably recognize that Rectangle is a value object. It doesn’t represent any particular rectangle. It binds a little bit of behavior to a domain concept our program uses.

That wasn’t so hard, now was it

I keep trying to tell people that, in some ways, writing multithreaded program is as simple as applying common object-oriented design principles. Build objects that are always in a sensible state, don’t allow twiddling that state without going through the object’s interface, use value objects when possible, and consider using immutable value objects if you’re starting from scratch.

Following these principles drastically reduces the number of states you have to think about and thus makes it easier to reason about how the program will run with multiple threads and how to protect data with whatever form of lock is appropriate.