Michael Feathers on how code grows
Festering Code Bases and Budding Code Bases:
Some teams produce what I call a festering code base. In a festering code base, the team changes the code primarily by adding code to existing methods and adding methods to existing classes. The results are predictable. Classes and methods grow malignantly, eventually becoming thousands of lines long.
Better teams produce budding code bases. Developers create new classes and methods and delegate work outward. Periodically, they collapse structure back into a simpler form, but the dominant trend is to grow the code by creating new structure.
I'd never put much thought into how code bases grow in the past. Feathers has some interesting ideas here about the characteristics of good and not-so-good growth and how languages and tools might promote good growth.
Incremental deployment at GitHub
Over the past year, I've read a lot about how teams are deploying their software. I've known for a while that Google has the ability to roll out new code to a small percentage of their servers and ramp up the breadth of deployment if they like how the software is behaving.
Lately, I'm starting to see more and more teams implement that sort of functionality. Rick Olson describes how GitHub implements it in How we deploy new features, and includes links to how Forrst and Flickr do it as well. At Velocity, Paul Hammond explained how to build an application-specific kind of version control into your app.
I'm a little surprised that few libraries have emerged for managing this. It would seem that, given all the excitement about continuous deployment, automated rollbacks, and incremental rollouts, someone would come up with something that they think is neat enough to share. I suspect that in fact, this is a really ugly, deeply application-specific sort of thing, no one likes to look at how they do it, and that's why there is plenty of talk about how to do it, but no libraries making it a simple thing.
The Cadence and Flow of Editing Programs
I figured out why my trists with other editors often end up back at TextMate. It sounds a bit like this:
Tap-tap-tap-tap-tap-tap; TAP; tap-tap-tap-tap-tap-tap-tap-tap-tap-tap-tap-tap; TAP; TAP; tap-tap-tap-tap-tap-tap; TAP.
When I’ve used vi and its descendants, it sounds like this:
Tap-tap-tap-tap-tap-tap; taptaptap; tap-tap-tap-tap-tap-tap; tapTAP TAP! tap-tap-tap-tap-tap-tap-tap-tap-tap-tap-tap-tap-tap-tap-tap-tap-tap-tap. tapTAPTAPtapTAP TAP!
And Emacs sounds like this:
Tap-tap-tap-tap-tap-tap; tapTAPtapTAP. tap-tap-tap-tap-tap-tap;tap-tap-tap-tap-tap-tap;tap-tap-tap-tap-tap-tap; tapTAP TAP; TAP TAPtapTAPtapTAPTAP. tapTAPtapTAP!
Lest you fear I’ve created some Ook-like language for describing shortcuts in any known editor, let me explain what’s going on here.
Cadence
Emacs is, at it’s core, a Lisp machine with a text editing language wrapped around it. Every interaction with Emacs invokes a function. Handily enough, the function that adds an “a” to the file you’re editing is bound to the a
key on your keyboard. Oddly enough, the function that writes the file you’re editing out to disk is bound to the combination of hitting control
and x
at the same time, followed by control
and s
at the same time. Getting them out of order matters. Control-s
followed by Control-x
does something entirely different.
So when you use Emacs, you type a bit, and then you run some command. Maybe you save the file, or switch to editing another file, or go to peruse a directory. So you tap for a while and then you stop tapping, move your hands every so slightly to mash the control, or alt keys and then tap some other key, usually emphatically. The most commonly used key combinations end up being hit even more emphatically. Sit in a room full of developers using Emacs, listen closely; every once in a while, you’ll here everyone save almost simultaneously and go back to a furry of lower-case tapping.
Vi is slightly different from Emacs in that it is built up from two Unix commands: one for editing single lines of text, and another for moving between said lines of text. Thus, the cadence of a vi user is slightly different. Staccato taps followed by a bang as they switch from line editing to line navigation; more staccato taps, this time oddly spaced as they move between lines and place the cursor to begin their next fury of editing; another burst of staccato text entry; a quick and emphatic tap to take them out of editing mode and then a quick but punctuated trio of taps as they invoke the command that saves the file out, a sequence of finger movements so ingrained in the vi users brain that it appears as more of a gesture than a triplet of discrete key presses.
Here’s a project idea for pranksters: stand in a room full of people using vi and Emacs, listen for the really emphatic taps, and trip the room’s breaker right before they all finish their emphatic save commands. Cackle as chaos ensues.
The space between the taps
A roomful of vi-users, Emacs-users, and TextMate users is a homogeneous mess of clackity-clackity to the untrained ear. Most accomplished programmers are touch typists, so what you’re likely to hear is an undifferentiated stream of rapid-fire tapping. But if you’ve used these editors enough, and wasted enough time thinking about the aesthetics they represent, you can hear the differences in the punctuation as commands are invoked by arcane combinations and sequences of keystrokes.
In Vi and Emacs, there is a concise sequence of keys you can mash to do a regular expression search, move down three lines, go to the second sentence on that line, and replace the word under the cursor with “bad-ass text editing programmer, do not offend”. It is, in part, this power that attracts, fascinates, and empowers their users.
TextMate can do this, sure. But there is very little in the way of support from the editor to do it. You mostly have to put your eye on the piece of text you wish to edit and use some primitive motion keystrokes to get the cursor where you want it. Then you use those same keystrokes to highlight the text to replace, this time holding down a modifier key, then you type in the text you want. TextMate, compared to its programmers editor brethren, is a language of grunts and chuffs next to the sophisticated Latin or French of vi and Emacs.
Flow
TextMate is unsophisticated next to the extensibility and conceptual unity of Emacs, or the pure practicality of vim. So why do I keep coming back to it?
It keeps me in flow.
This is a very personal answer. I’m not saying you can’t achieve a flow-state with vi or Emacs. I’m saying that while I like the idea of those editors, understand the aesthetic, and enjoy watching skilled operators using them, I get lost in the punctuation when I use them. I either forget what punctuation I should use in some text editing scenario, or I have a nagging doubt that there is some better punctuation I could be using instead.
If vi is about navigating lines and editing those lines; Emacs is about invoking Lisp functions on files containing text, then TextMate is about primitive but direct manipulation of the text in a file. There’s very little conceptual overhead. You don’t need to know how the editor is enhanced in order to understand how to operate it. You don’t need to know when to put yourself in different modes of operation to make things happen. You just think of what you want the text to look like, you move the cursor around and you type on the keyboard.
It ain’t much, but I (often) call it home.
Breaking My Habits For Editing Programs
I’m a Unix guy, by upbringing. My first formative experiences in software development were on an early, Linux 1.x version of Debian. I’d used Windows, but always came back to Linux. When OS X got good enough around 10.2, I switched to something that didn’t require so much tinkering, so I could make more useful stuff.
Software development on Unix has skewed towards focusing on tools, languages, and text editors for quite some time. IDEs and browsers on Unix are a messy, foreign thing (just like everything else in Unix). Thus, I’ve long favored the terminal-and-editor style of development.
I’ve decided that now is the time for me to try something different. I like text editors and directly manipulating text, but I can see why some people feel naked without an IDE. The ability to pop-up a level and make a more broad-stroked transformation to a program is appealing. Having code navigation and semantic awareness baked in has lots of potential.
I’ve probably said grumbly things about RubyMine in the past, but I think now is the time to give it a go. Worst thing that could happen is that I don’t like it and I go back to the infinite tinkering of Emacs or the 85% perfect experience of TextMate.
I’ll let you know how it goes.
I originally wrote that a few months ago, at the apex of my editor neurosis.
I did give RubyMine a try, and I like some parts of it. It’s code navigation is pretty nice, it does an admirable job of integrating with the unique ecosystem of tools that a Ruby developer uses to manage their environment, and it does an excellent job of grokking TDD with test/unit and RSpec. RubyMine is a step in the right direction. I suspect that if I had muscle memory for IntelliJ, it would be the way to go.
But, I have muscle memory for TextMate and Emacs, and I have an affinity for being close to my tools. RubyMine felt one step disconnected from both my muscle memory and my tools. That’s quite an accomplishment; most IDEs feel several steps removed the tools and seem to discourage developing finger-memory in favor of menu-memory. I’ll give RubyMine another try in a year, probably, see how it’s coming along. But in the mean time, it’s great to see that there is a vendor out there tackling the challenge that is tools for Ruby.
A rambling, regurgitated thought on process
Elevator pitch: I’ve found that if you want to divert a productive team into an hour or two of semi-fruitless banter, ask how the team should use Git, Pivotal Tracker, and Capistrano to manage incoming work, verify it, and deploy it to production. In reality, you should ignore all the corner-cases and figure out what will enable you to push really small chunks of work with great frequency.
Ed. What follows isn’t novel, but it was a useful change in perspective for me, so I decided to share.
I’ve been thinking a bit about software processes lately. Despite great variation in telling you how to do so, most processes seem to focus on to do more stuff faster.
Lately, the notion of doing less has a lot of interest. Lean startups are the new-new thing and Getting Real is the old new thing; both preach getting more done and delivering value by doing less and analyzing the results more.
There are two kinds of “do less” a software developer can engage in. In the past I’ve been a little too focused on how I can take on fewer responsibilities from other parties. Literally doing less by scoping down features, putting off decisions, and focusing on things that seem like they really matter. I sometimes feel like I’ve become too eager to do less, making myself something of a cranky coder/slacker. But I digress
Recently, I’ve been trying to tackle doing less in my habits of creating software. How can I write less code to implement a feature, not in the minimalist sense, but in the “how do I just get it to kinda work sense”? How can I take less time between starting something and getting some form of it out in the wild? How can I make my code less coupled so there are fewer changes to make when I decide it needs to do something else? How can I make this less coupled to data storage so that putting it out requires less deployment effort? How can I make changes that are less likely to cause long-term regressions? How can I make it less effort to rollback bad changes?
When I look through the lens of accomplishing more by doing less, a lot of popular software methodology seems like dead weight. Rather than trying to find a process that addresses every team member’s own scars and affections, both perceived and imaginary, it seems most useful to imagine the smallest ruleset that won’t result in uncontrollable entropy and put it into action. If something starts to hurt, imagine the simplest new rule and put it into play.
The goal, as stated above, is to get to the point where you make really tiny, maybe imperceptible changes, and push them really frequently. Everything that stands in the way is the enemy.
Rails' Next Top Model
Of all the new and reimagined code in Rails 3, ActiveModel and ActiveRelation rank amongst what I find the most interesting. I’m excited that they potentially lower the bar to implementing one’s own data layer. If you’ve got some custom backend or datastore, writing a nice API around it has previously been quite the endeavor. To get it working is one thing; to make it as pleasant to use for application programmers is another thing entirely. ActiveModel and ActiveRelation have been extracted from ActiveRecord and make the task of building one’s own model layer far easier.
My presentation for RailsConf 2010 focused on what ActiveModel and ActiveRelation provide and how one can use it to write cleaner code in domain models, how to make your models feel more like ActiveRecord objects, and how to use ActiveRelation to build your own persistence layers.
I hope you find the slides educational. Further, I’ve posted the examples on GitHub so you can play along at home. If you’re particularly interested in ActiveRelation, I hope you’ll find the examples useful as a starting point to using that library.
Make More Awesome
Over the past year, I’ve been trying to create more stuff, some of which I’d hope to turn out awesome. Largely this is an ongoing sort of thing. I try things, I learn a little, I keep at it. Some things I try work, others don’t. I try to make a habit out of the things that have worked out well. I make a note of things that seem to help me get out of a funk when I’m not making as much as I’d like or having trouble putting in the hours I think are necessary to make cool stuff.
I first distilled these ideas into a talk I did at RubyConf 2009 on having more fun while coding. But I didn’t realize it at the time; I was just sharing some ideas about how to have fun. For me, one of the ways to have more fun is to make more time to have fun. But that’s just the beginning of making more awesome. I found I had to make more time, and develop a bunch of other habits.
For Big Design 2010, I honed the ideas, habits, and tricks I’ve found useful to me into a presentation on how to get off the couch, start making more things, and make some of those things awesome. I hope you’ll find it useful and perhaps start making lots of awesome stuff.
A quick RVM rundown
(It so happens I’m presenting this at Dallas.rb tonight. Hopefully it can also be useful to those out in internetland too.)
RVM gives you three things:
- an easy way to use multiple versions of multiple Ruby VMs
- the ability to manage multiple indpendent sets of gems
- more sanity
First, let's install RVM:
gem install rvm
rvm-install
- follow the directions to integrate with your shell of choice
Now, let's install some Rubies:
rvm list known
will show us all the released Rubies that we can install (more on list)rvm list rubies
will show which Rubies we have locally installedrvm install ree-1.8.7
gives me the latest release of the 1.8.7 branch of Ruby Enterprise Editionrvm install jruby
will give me the default release for JRubyrvm use jruby
will switch to JRubyrvm use ree
will give me Ruby Enterprise Editionrvm use ruby-1.8.6
will give me an old and familiar friendrvm use system
will put me back wherever my operating system left me
The other trick that RVM gives us is the ability to switch between different sets of installed gems:
- Each Ruby VM (JRuby, Ruby 1.9, Ruby 1.8, REE) has its own set of gems. This is a fact of life, due to differing APIs and, you know, underlying languages.
rvm use ruby-1.9.1
gives you the default Ruby 1.9 gemsetrvm use ruby-1.9.1%acme
gives you the gemset for your work with Acme Corp (more on using gemsets)rvm use ruby-1.9.1%wayne
gives you the gemset for your work with Wayne Enterprisesrvm use ree%awesome
gives you the gemset for your awesome app- You can export and import gemsets. This can come in handy to bring new people onboard. No longer will they have to sheepishly install gems on their first day as they work through dependencies you long since forgot about.
Some other handy things to peruse:
I also promised you some extra sanity:
- RVM knows how to compile things, put Rubygems and rake in place, even apply patches and pull from specific tags. You can do more important things, like watch The View or read an eleven part series on pre-draft analysis for the Cowboys.
- RVM lets you isolate different applications you're working on. Got one app that doesn't play nice with Rails 2.x installed? No problem, create a gem environment for that! Stuck in the spider-web of Merb dependencies? Isolate it in its own environment.
- RVM makes multi-platform testing and benchmarking easy. You can easily run your test suite or performance gizmo on whatever Rubies you have installed.
- RVM makes it easy to tinker with esoteric patchlevels and implementations. For instance, feel free to tinker with MagLev or the mput branch of MRI.
A couple other things RVM tastes great with:
- Using homebrew to manage packages instead of MacPorts
- Not using
sudo
to install your gems - Managing your dotfiles on GitHub
The imperfection of our tools
I enjoy a well-crafted application. I place a high value on attention to detail, have opinions on what design elements make an application work, and try to empathize with the users of applications I’m involved in creating. Applications with a good aesthetic, a few novel but effective design decisions, and sensible workflow find themselves in my Mac’s dock. Those that don’t, do not.
The applications I observe fellow creators using to create often don’t fit into their environment. They don’t fit into the native look-and-feel. They ignore important idioms. Their metaphors are imperfect, the conceptual edges left unfinished.
In part I notice this because as creators we tend to live in a few different applications, and time reveals most shortcomings. But in part, I notice this because the applications are in fact flawed. Flawed to the point, that you would think given my opening words, that I would refuse to use them. And indeed, I refuse to use many of the applications that others find completely acceptable for making the same kinds of things I do.
Increasingly, it seems the applications that people who create things live in offer a disjoint user experience. I’m thinking of visual people living in Photoshop or Illustrator or developers living in Emacs or Terminal.app. We use these applications because they best allow us to make what we want and get in our way only a little bit. But, it’s a tenuous relationship at best.
What’s this say about what we’re doing and the boundaries that we operate along? Would we accept the same kinds of shortcomings in say, a calendar application or a clock widget, if those were central to our workflow? That is, is there something about the creative process that leads us to accept sub-perfect tools? Is it inevitable that someone seeking to make new things will find their tools imperfect? Is the quest for ever-more perfect tools part of how we grow as makers?
I hate closing with a bunch of questions, but this piece is but an imperfect tool for discovering an idea.
Ed. Closing could use some work.
Goodbye, gutbombs
Last March my wife and I joined a gym, started working out with a trainer, started trying to eat better, and set out to improve our health. Amazingly, we’ve stuck with it (after two previous failed attempts in years past) and are both in much better shape than we’ve been in for quite some time.
One of my personal reasons for doing this was what I’d been hearing about the correlation between working out, eating better, and brain function. Lots of people who read way more into this than I do had been saying that if you eat better and exercise more, your brain will work better.
I’ve noticed this first hand. The day after my first serious run, my mind was in overdrive. I had lots of great ideas, I worked through them quickly, and I didn’t procrastinate when it came to exploring or realizing them.
Today, I had the opposite experience. I went out for a rather large Tex-Mex lunch. Lots of starch. I got home and took a nap, as is often my wont. Usually I wake up ready to get back to work after my naps. But today was different. My brain was thoroughly sluggish. My body’s energy was going towards digestion, not thought.
I guess this is something of a break-up letter for me. You see, I’ve long enjoyed the large, starchy lunch. But, I’m not sure I can put up with it anymore. If its a choice between starchy, tasty lunches and a high-functioning brain, I’m going to have to choose my brain.
Sorry, lunch-time gutbombs. We had a good run, but I’m going to have to quit you for a while.
Give attribute_mapper a try
(For the impatient: skip directly to the `attribute_mapper` gem.)
In the past couple months, I’ve worked on two different projects that needed something like an enumeration, but in their data model. Given the ActiveRecord hammer, they opted to represent the enumeration as a has-many relationship and use a separate table to represent the actual enumeration values.
To a man with an ORM, everything looks like a model
So, their code ended up looking something like this:
class Post < ActiveRecord::Base belongs_to :status end class Status < ActiveRecord::Base has_many :tickets end
From there, the statuses
table is populated either from a migration or by seeding the data. Either way, they end up with something like this:
# Supposing statuses has a name column Status.create(:name => 'draft') Status.create(:name => 'reviewed') Status.create(:name => 'published')
With that in place, they can fiddle with posts as such:
post.status = Status.find_by_name('draft') post.status.name # => 'draft'
It gets the job done, sure. But, it adds a join to a lot of queries and abuses ActiveRecord. Luckily…
I happen to know of a better way
If what you really need is an enumeration, there’s no reason to throw in another table. You can just store the enumeration values as integers in a database column and then map those back to human-friendly labels in your code.
Before I started at FiveRuns, Marcel Molina and Bruce Williams wrote a plugin that does just this. I extracted it and here we are. It’s called attribute_mapper
, and it goes a little something like this:
class Post { :draft => 1, :reviewed => 2, :published => 3 } end
See, no extra table, no need to populate the table, and no extra model. Now, fiddling with posts goes like this:
post.status = :draft post.status # => :draft post.read_attribute(:status) # => 1
Further, we can poke the enumeration directly like so:
Post.statuses # => { :draft => 1, :reviewed => 2, :published => 3 } Post.statuses.keys # => [:draft, :reviewed, :published]
Pretty handy, friend.
Hey, that looks familiar
If you’ve read Advanced Rails Recipes, you may find this eerily familiar. In fact, recipe #61, “Look Up Constant Data Efficiently” tackles a similar problem. And in fact, I’m migrating a project away from that approach. Well, partially. I’m leaving two models in place where the “constant” model, Status
in this case, has actual code on it; that sorta makes sense, though I’m hoping to find a better way.
But, if you don’t need real behavior on your constants, attribute_mapper
is ready to make your domain model slightly simpler.
Just For Fun
This year was my fourth RubyConf. I’ve always come away from RubyConf energized and inspired. But, I’ve yet to follow through on that in a way I found satisfying. I have a feeling I’m not alone in that camp.
This was the first year I’ve given a presentation at RubyConf. At first, I had intended to use this watershed-for-me opportunity to ask whether Ruby was still fun. There’s been a number of “drama moments” since my first RubyConf; I thought it might be worth getting back to my early days of coding with Ruby, when I was exploring and having a great time turning my brain inside out.
As I started researching, it turned out that there are a lot of people having fun with Ruby. Some are doing things like writing games, making music or just tinkering with languages. Others are doing things that only some of us consider fun. Things like hacking on serious virtual machines, garbage collection, and asynchronous IO frameworks.
So, back to my talk. I saw my failure to harness the motivation what I’d seen at previous years at RubyConf as an opportunity to figure out ways to line up some tactics to make sure that after the conference, I was able to create awesome things, contribute them back to the community, and enjoy every minute of it.
Thus, I came up with a sort of “hierarchy of open source developer needs”. At the bottom is enjoyment; there’s little sense doing open source work if you’re not having fun. Once you’re having fun, you probably want to figure out how to find more time for making codes. Once you’re making more codes, you want to figure out how to get people interested in using your stuff. I’ve taken these three needs and identified several tactics that help me when I find myself in a rut or unable to produce. Call them patterns, practices, whatever; for me, they’re just tricks I resort to when the code isn’t flowing like I want to.
The talk I ended up with is equal parts highlighting people in the Ruby community that are having fun and highlight ways to enjoy making things and contributing it back to whatever community you happen to be part of. I hope that I avoided sounding too much like a productivity guru and kept it interesting for the super-technical RubyConf crowd.
If all of this sounds interesting you, grab the slides (which are slightly truncated, no thanks to Keynote) or watch the recording from the conference itself.
I wrote the proposal for this talk right after Why disappeared himself. His way of approaching code is what inspired me to write a talk about getting back to coding for fun. “Just for Fun” starts with a tribute to Why the Lucky Stiff. The sense of fun and playfulness that Why had is important to the Ruby community. I’ve tried to highlight some of his most interesting playful pieces. And in the end, I can’t say “thanks” enough. Why has inspired me a lot and I’m glad I got to meet him, experience him and learn through his works.
Even if you don’t take a look at my presentation, I strongly urge you to give a look at some of Why’s works and let them inspire you. My favorites are Potion and Camping.
Some other things I mentioned in my talk as interesting or fun:
- Greg Borenstein’s code, writings and tumblings
- Project Euler
- Marc-Andre Cournoyer’s codes and book
- Philip Kromer’s Wukung