2012
Intermediate variables, organizing OO, meeting Grinders half way
I work with Dave Copeland at LivingSocial, but not on the same team. Maybe someday I’ll fix that, but for now I learn a lot from his writings. Herein, a few things worth checking out yourself.
If you ever need to read my code, you’ll eventually come to suspect I have a particular dislike for intermediate variables. You’ll come to suspect this through finding lots of uses of inject and tap, two Ruby methods not everyone is on good terms with. You can imagine I’d side with Dave on the subject of Tap versus intermediate variables. You’d be right, but Dave says it so well, you should read his take on the joy of tap. He also shows how to annoy people with tap-like constructs in other languages. If you’re into combinators, Reg Braithwaite has written about tap in terms of Kestrels.
I’ve been learning a lot about how to think about organizing a non-trivial object-oriented system this year. Gary Bernhardt is doing some fantastic work explaining a hybrid imperative/object/functional system. If you don’t have time to dive into Gary’s entire backlog (it’s worth it to find the time), Dave covers some similar ground describing the only four types of classes in your OO system. Think of these as a post-hoc observation on how many systems seem to evolve; Record objects take root, Service objects reveal themselves (often intertwined amongst other objects), Builders are sprinkled throughout, and there are a few classes hanging out that you wish you’d made immutable. These are handy guides for thinking about and refactoring an existing design. That said, I think it would be overkill to start a design with these archetypes. Caveat: some developers will really dislike organizing a system this way; tread carefully.
I’ve written about the virtues of The Grinder. I know a lot of non-Grinders wish that the Grinder knew more about how to take the code they’ve made to work and improve it so that it is more malleable in the future. Making it Right: Technical Debt vs. Slop sets out a good mindset on how this can happen. Think before you type, write a test, make it work, and then tidy it up with future malleability in mind. From there, non-Grinders need to meet Grinders in the middle is in shrinking the feedback cycle. When tests are too much effort to write or take too long to run, Grinders fall back to their old habits. When making it right involves too many intermediate steps with nothing to show for it, Grinders move on to the next thing. When a non-Grinder learn to be less precious with our work, or a Grinder learns to take a moment to round off the sharp corners on their work, you end up with a much stronger team. Fight for it.
Why I'm down on hypermedia containers
In response to my hypermedia opinions, Mike Kelly said:
These two seem to conflict: “In my opinion, abstract container formats aren’t useful.” and “Just use JSON”. People normally talk about “generic” media types, but they don’t have to a “container” at all, they can simply add conventions for linking. Having conventions for this stuff is useful because it allows us to build tooling around it, if everyone reinvents the wheel in their own way then we can’t build re-usable code. For a similar reason, “specifying your own custom MIME types” is not a good idea – there’s also the time cost associated with doing that. If you use something like hal+json you avoid that cost, and can concentrate on establishing your API’s workflows via link relations.
The logic behind the madness goes like this: abstract containers aren’t solving a problem I currently have. Unfortunately, this means they create problems for me. In the end, I’m building an API to provide functionality, not as advocacy.
As of summer 2012, there are ideas like HAL, JSON collections, etc. and specifications of those ideas. There are very few implementations. As a service provider, I wouldn’t actually get any benefit out of using those formats. The convention, within my own API, that fields ending in _url are links is sufficient. I’d actually end up net negative, because I’d have to explain how e.g. HAL works and support client developers seeking to understand how to work with it. Anyone building to my API would likely end up having to write their own HAL code, so they don’t benefit much either.
I’ve decided to use JSON because providing an API that returns HTML and expects users to scrape it via selectors is extremely confusing to developers. Keep in mind, not everyone is savvy to the latest development trends. To them, an API means an HTTP service that returns XML or JSON. If I were to embrace HTML as a response type, I’m again stuck with explaining a new concept to client developers.
If it’s not obvious yet, one of my main principles in adopting hypermedia is to avoid educating developers on hypermedia as much as possible. I’m in the game of providing a useful API, not a system that shows off the possibilities of hypermedia and how deeply committed I am to its theories.
Finally, I’ve chosen to craft my own content types because I need some kind of contract with client developers that tells them what kind of data they can expect to see, plus some documentation that expresses that contract in a way that is easy for humans to understand. An RFC-style specification that states what a content type MUST/MAY/SHOULD include is exactly the right kind of abstraction. It allows me to update the specification with a version identifier and specify how each revision changes in terms of what data is available. Further, I found a content type the most tractable solution for specifying the input formats supported by PUT and POST endpoints. None of the abstract containers that existed as of August 2012 fit my needs for specifying links, structure, and how to submit data via POST/PUT/PATCH.
Honestly, link traversal and machine-to-machine interaction are not the pain I’m feeling. What I want is the simplest possible API that allows potential client developers to understand what they can do via our API and how to do it. Further, I want it to be possible, even if it’s not magical or easy, to change the API in the future so I’m not constricted by its current design. I feel no need to apply all of the hypermedia principles to make something useful; I can cherrypick some of the hypermedia principles and still achieves the goal of an API that is stable but not set in stone.
Hypermedia opinions
Through the Gowalla API, and now the Sifter API, I’ve worked with a couple systems one could reasonably call a hypermedia API. Since smart people are talking about them today, I feel compelled to throw in my two cents.
A URL-based API alone won’t prevent breakage, as many point out. It won’t even prevent people from figuring out how to extract IDs and continue to bang URL strings together. I don’t know why creating URL strings is the security blanket of many API client developers, but it seems you can’t take it away from a few of them.
That said, a URL-based API does make life easier for good actors. Client developers, especially the ones that can’t upgrade their deployed applications quickly, can sleep better knowing that the upstream API could change and their code won’t break. Service developers can rest easy knowing it’s not easy, but it is possible to change their URL structure if they need to.
A hypermedia API means you can’t skip on documentation. In fact, it means you probably need better documentation. You’ll need to explain what each link in the API responses means (is photos_url the URL for my photos, or my friends’ photos?) and what kind of data they can expect that URL to return (a photo object? a collection of photo objects? a user?) As I mentioned before, a lot of client developers don’t have their minds wired for hypermedia yet, so you’ll need a lot of examples for how they should build their clients.
In my opinion, abstract container formats aren’t useful. Squeezing your application’s data into someone else’s data model is not a great place to live. That said, I do think specifying your own custom MIME types is a promising idea. You specify each of the content types in your API, RFC-style, and then provide examples of that data. Should you need to change your response formats, you update your RFC-style specification, adding a new field as a “MAY provide” type feature.
The rub of this approach is that you can end up with an explosion of response-type handling code in your application. Again, it’s not that hypermedia principles make it easy, they just make it possible.
I’ve seen some hypermedia APIs expressed as hypertext, using HTML and cleverness. I don’t see the benefit of this. Just use JSON, and maybe specify your own response types.
The shortcoming in much of the hypermedia content out there is they focus on clients reading data and hand-wave over clients that need to create or update data. I can see why; I think that machine discoverable parameters are a tarpit. Trying to tell a client what fields the server will accept, what types are accepted, and then handle all the possible error flows is verbose and riddled with edge-cases. The way I’m solving this is to specify, in the documentation, which URLs clients can expect to POST/PUT/etc. data to and what content-type those URLs expect to receive. I’m pretty much doing the least hypermedia-ish thing possible, but I think that’s the correct choice right now.
The great thing about the principles of hypermedia APIs and its discourse if a focus on workflows. The biggest mistake I’ve made in building APIs is in exposing a database instead of a service. No one cares what your table structure or domain model looks like. They want to fetch some data, preferably indexed in a way that is immediately useful to them, and maybe write some data back. When you look through this lens, API design starts to look a lot like UI design. It’s fun!
You should read about hypermedia APIs, if only to challenge your thinking. It helped me a lot to do just that. Check out Steve Klabnik’s Designing Hypermedia APIs; it’s not perfect, but it makes a lot of these things easier to grasp.
In the end, I think designing, building, and supporting a hypermedia API is hard. You may be better off with an RPC-over-HTTP, or ID-based REST design. The good news is you can benefit from an incremental application of the principles of hypermedia APIs; you don’t have to go all in before you get something out of it.
How to Jerry Seinfeld
How Jerry Seinfeld writes a joke:
[youtube www.youtube.com/watch
Very different from how I approach it. But, I love knowing how much goes into his craft and the degree to which he is particular about how he does it.
Typing code examples, it's like biking
If you want to learn from a piece of code, you should type it out, instead of just reading it. The value of typing code:
Typing code may be like riding a bicycle. I’m surprised how much more detail I see the first time I ride my bicycle over a road I’ve driven on, mostly because I’m moving slower but also because there’s an additional muscular dimension to the experience.
I love this bicycle metaphor. The slowness of biking engages my brain in an entirely different way than running or driving. Even the mechanical sounds are more pleasant; the consistent whirr of the chain is so much more calming than the revving up and down of a gasoline engine.
The value of typing code holds very true for me; I usually get very little by simply reading code in books and articles. But when I take the time to type it in and actually try to run it, I struggle with it more (not all code examples are perfect) and get more out of it. You should give it a try.
Wherein I heart Code Climate
We’ve had Sifter’s repo hooked up to Code Climate for a couple months now and I’m really loving it. Garrett and I have both found it fun to kill duplication or refactor away complex code. A decent test suite enables this, but Code Climate is very much the compass that points you right to the trouble spots. The Code Climate blog is a great read too, consistently featuring thought-provoking ideas on how to think about making better code.
I love tools like flog and flay for quick smell detection. If you are like me and too lazy to configure CI and code metrics, Code Climate’s easy setup and awesome trending are well worth a look.
Hit it, don't quit
How to be good at anything. In short: do it, get feedback, study how to improve, repeat.
Something I’ve found, through crossfit, is that if I have any strong suit it’s not quitting. Seems trite now that I write it, but it occasionally helps to state the obvious.
Once I’ve decided an activity is worthwhile, I’m pretty good at sticking with it no matter how silly I feel doing it. I’m not the strongest, the fastest, or the best eater. I’m not the smartest, the funniest, or the most charming. But I’ve made progress in life by showing up every day and trying to do a little better than the previous day.
The bottom line: pick a few things to do well, do them, and don’t quit.
You can't solve technical debt, you can only hope to contain it
Technical Debt is an interesting phrase. We all have a sense of what it is instinctively, but we rarely want to think about it. If we think about it too hard, we feel somewhat oppressed by entropy. All systems tend to toward disorder and software systems are no different. The big question that we all face is what to do about it.
Systems tends toward disorder. Disorder is hard to reason about and risky to deal with, i.e. you’re likely to avoid dealing with it at all. But, most successful products have a system with more technical debt than you’d like at its center.
Increasingly, I think that the only way to confront technical debt and complexity is to contain it. Languages and tools only seem to help at the margin. Rigorously splitting large, complex, debt-filled systems into smaller, proportionally complex and debt-filled systems is the way to gain traction.
You can’t solve technical debt or essential complexity; you can only hope to contain it.
Needs better words
How much easier would Haskell be if its vocabulary wasn’t so deeply rooted in abstract mathematics? How many more people would immediately understand Cassandra if it just adopted the vernacular of row-oriented databases instead of overriding it with column-oriented semantics? Wouldn’t the programming world be better if every language didn’t call itself object-oriented?
So many programmer-facing things could be better if they had better names for concepts. The power of good naming:
Characters are cheap, confusion is costly. Let’s not make things harder for the programmers who come after us. Remember, this is just as likely to be ourselves in a few months. Let’s avoid using a name like prj when project is only four characters more typing. Anything that reduces reading friction in our code is a good thing.
The cynical view is, we are actively confusing and inhibiting ourselves when we use names that are too short, not clear, or outright wrong. The optimistic view is that there’s a progression we can take from not knowing what something should be named, to giving it an acceptable name, to using the naming process to discover better structures.
If you ever hear me say something “needs better words”, it means that I think the idea is right but the labels are wrong. A philosophical dialog may ensue, where I struggle to discover what the essence of the idea is. The end goal is a word that is concise, has the right connotation, and whose meaning is obvious and accessible to as many audiences as possible.
I love naming things.
The qualities of better code
What is 'better' code? Dave Copeland on the qualities readable, changeable code exhibits. Of the attributes he identifies, I think number of paths (ABC complexity) is the most important for reading code and fan-in/out is the most important thing to manage for easily changed code.
Get in my ears, you dissonant chord
Petrushka chord. Two major chords played a half-tone apart. So, it sounds good, except it sounds grating. It's a motif throughout Stravinsky's ballet Petrouchka. Ergo, like everything Stravinsky, get in my ears! Listen to it and learn more about the chord from the awesome "Feynman of classical music" (I just made that up) Leonard Bernstein.