2010
Bundler, not as bad as they say
Of all the new moving parts in Rails 3, the one I see the most grousing over is Bundler. This is not surprising, as its a big part of how your application works and it's right up front in the process of porting or building Rails 3 apps.Bundler: As Simple as What You Did Before:
Bundler has a lot of advanced features, and it’s definitely possible to model fairly complex workflows. However, we designed the simple case to be extremely simple, and to usually be even less work than what you did before. The problem often comes when trying to handle a slightly off-the-path problem, and using a much more complex solution than you need to. This can make everything much more complicated than it needs to be.
I haven't run into anything with Bundler that I couldn't solve with a little critical thinking and maybe a little searching. On the other hand, Bundler has made getting dependencies straight amongst team members and deploying them to production servers far easier than it was before. I'm very glad that while it's not strictly part of the scope of Rails, that Bundler is now part of it.
Language and brains, an update
Does Your Language Shape How You Think? An update on the current thinking around the Sapir-Whorf hypothesis, the one about language and the words therein shaping the thoughts you can and cannot form.
Using Rails 3.0's notification system
How to use Rails 3.0's new notification system to inject custom log events. Ever wondered what the notification/subscription stuff in Rails 3 is? Wonder no more! I just used this to add performance logging around some Cassandra stuff in our Rails 3 app. Once you get the hang of it, this is really rad stuff.
Computers should do the boring bits
Future-proofing, Uniform Access, and Masquerades:
Boring work should be a cardinal sin in programming: it indicates something that the computer should be doing but isn’t.
An ode to Hashie
I was building an API wrapper this weekend. As is common when writing these sorts of things, I found myself needing something that takes semi-structured data (hashes parsed from JSON) and yields Ruby objects that are easy to work with. I've always found myself hacking these sorts of things together on a somewhat ad-hoc basis. It's a fun, but a bit of a yak-shave.
This time around, I decided to see if the state of the art has advanced in this realm. Luckily, I reviewed Wynn Netherland's slides from Lone Star Ruby Conference and found exactly what I needed.
Where have you been all my life?
Intridea's Hashie is a library built on the notion of making hash-like data structures act a little more like objects and a little easier to work with. I have literally wanted something like this for years!
Suppose you have a hash like the following:
hash = {
"name" => "Adam",
"age" => 31,
"url" => "http://therealadam.com"
}
Coding up an object to store that isn't too hard, but writing the code that pulls values out of the Hash and tucks them away in the right attribute on the object gets tedious quickly. Hashie's Dash class makes this trivial.
class User >Hashie::Dashie property :name property :age property :url end
Its even more delightful to use:
user = User.new(hash) user.name # => "Adam"
Tons of boilerplate code, eliminated. My life is instantly better.
A great use of inheritance
It's been pointed out that ActiveRecord's use of inheritance is somewhat specious. To argue that "user is-a ActiveRecord::Base" takes a bit of hand-waving. So lately, you'll find lots of libraries insinuate themselves into classes as a mixin, rather than as a parent class. This is a little bit of you-say-potato-I-say-potato, but whatever.
In Hashie's case, I think that inheritance is being used correctly. All of the classes that Hashie provides (Mash, Dash, Trash and Clash) inherit from Hash. So the is-a relationship holds.
Sugary data structures taste great
While I'm going on about inheritance, here's how I used to create these sorts of wrapper classes:
User = Struct.new(:name, :age, :url)
For creating simple objects that just need to hold onto some data, I really like this approach. If they end up needing data, it can easily grow up:
class User < Struct.new(:name, :age, :url) # Behavior goes here end
I like what Hashie is doing even more though. Its enhancing a core class in a largely unobtrusive way, and doing so from the confines of a library that only those who need it can pull from.
I'd love to see more libraries like this that add extra sass to Ruby core library. An Array that pages values out to disk on an LRU-basis perhaps, or a bloom-filter based Set, perhaps?
I'm excited about languages like Erlang, Haskell, Scala, and Clojure and what they can bring to the adventurous developer. Despite that, I feel strongly that Ruby still has plenty of really nifty tricks up its sleeve.
Examining software principles
There are too many good things to say about the Design Principles Behind Smalltalk. A few of my favorites:
Scope: The design of a language for using computers must deal with internal models, external media, and the interaction between these in both the human and the computer.
This one is really obvious until you get to the last four words. The human and the computer. Luckily we're starting to take for granted the primacy of human communication in programming lately (mostly), but when Smalltalk was created, I'm sure its designers received no shortage of grief when they steered towards humane optimizations.
Uniform metaphor: A language should be designed around a powerful metaphor that can be uniformly applied in all areas.
Smalltalk is largely objects and messages. Lisp is largely lists and functions. Erlang is largely pattern matching, functions, and actors. These aren't perfect languages, but once you deeply understand, really grasp the core concepts, you have the whole language at your command.
Operating System: An operating system is a collection of things that don't fit into a language. There shouldn't be one.
The first sentence is a great principle when considering what should go in the core of a system and what should go in the surrounding ecosystem of libraries. The second sentence is wonderfully bold, in that it cuts against what nearly every successful system has done since Smalltalk was prominent and in that it contradicts the first sentence. I'm not sure what practical use to make of this principle; its density of intrigue is that keeps me coming back to it.
Natural Selection: Languages and systems that are of sound design will persist, to be supplanted only by better ones.
I stopped worrying about what might supplant Ruby a long time ago. Someday, it will happen. And when it does, whatever succeeds Ruby will have to be really awesome to fill its shoes. I'm looking forward to seeing what that is. But the same goes for any technology; they are often replaced with something wholly awesomer than the incumbent.
I've never done it, but it seems like it would be intriguing and vastly informative to sit down with one of the systems I work on daily and try to extract these principles post-hoc. What values and principles are embedded in the system? What does that say about the team and why the system is the way it is? What principles are enablers and what bad habits should the team work to correct?
Making the complicated seem simple
Don Norman, Simplicity Is Not the Answer:
We want devices that do a lot, but that do not confuse, do not lead to frustration. Ahah! This is not about simplicity: it is about frustration. The entire debate is being framed incorrectly. Features is not the same as capability. Simplicity is not the same as usability. Simplicity is not the answer.
Norman goes on to explain how you can take a confusing mass of features and turn it into something less frustrating:
- Modularize into understandable clusters
- Map clearly from actions to results
- Model the ideas and actions cohesively
The article is about interaction design, but it fits just as well in designing programming languages and software.
Michael Feathers on how code grows
Festering Code Bases and Budding Code Bases:
Some teams produce what I call a festering code base. In a festering code base, the team changes the code primarily by adding code to existing methods and adding methods to existing classes. The results are predictable. Classes and methods grow malignantly, eventually becoming thousands of lines long.
Better teams produce budding code bases. Developers create new classes and methods and delegate work outward. Periodically, they collapse structure back into a simpler form, but the dominant trend is to grow the code by creating new structure.
I'd never put much thought into how code bases grow in the past. Feathers has some interesting ideas here about the characteristics of good and not-so-good growth and how languages and tools might promote good growth.
Incremental deployment at GitHub
Over the past year, I've read a lot about how teams are deploying their software. I've known for a while that Google has the ability to roll out new code to a small percentage of their servers and ramp up the breadth of deployment if they like how the software is behaving.
Lately, I'm starting to see more and more teams implement that sort of functionality. Rick Olson describes how GitHub implements it in How we deploy new features, and includes links to how Forrst and Flickr do it as well. At Velocity, Paul Hammond explained how to build an application-specific kind of version control into your app.
I'm a little surprised that few libraries have emerged for managing this. It would seem that, given all the excitement about continuous deployment, automated rollbacks, and incremental rollouts, someone would come up with something that they think is neat enough to share. I suspect that in fact, this is a really ugly, deeply application-specific sort of thing, no one likes to look at how they do it, and that's why there is plenty of talk about how to do it, but no libraries making it a simple thing.
The Cadence and Flow of Editing Programs
I figured out why my trists with other editors often end up back at TextMate. It sounds a bit like this:
Tap-tap-tap-tap-tap-tap; TAP; tap-tap-tap-tap-tap-tap-tap-tap-tap-tap-tap-tap; TAP; TAP; tap-tap-tap-tap-tap-tap; TAP.
When I’ve used vi and its descendants, it sounds like this:
Tap-tap-tap-tap-tap-tap; taptaptap; tap-tap-tap-tap-tap-tap; tapTAP TAP! tap-tap-tap-tap-tap-tap-tap-tap-tap-tap-tap-tap-tap-tap-tap-tap-tap-tap. tapTAPTAPtapTAP TAP!
And Emacs sounds like this:
Tap-tap-tap-tap-tap-tap; tapTAPtapTAP. tap-tap-tap-tap-tap-tap;tap-tap-tap-tap-tap-tap;tap-tap-tap-tap-tap-tap; tapTAP TAP; TAP TAPtapTAPtapTAPTAP. tapTAPtapTAP!
Lest you fear I’ve created some Ook-like language for describing shortcuts in any known editor, let me explain what’s going on here.
Cadence
Emacs is, at it’s core, a Lisp machine with a text editing language wrapped around it. Every interaction with Emacs invokes a function. Handily enough, the function that adds an “a” to the file you’re editing is bound to the a key on your keyboard. Oddly enough, the function that writes the file you’re editing out to disk is bound to the combination of hitting control and x at the same time, followed by control and s at the same time. Getting them out of order matters. Control-s followed by Control-x does something entirely different.
So when you use Emacs, you type a bit, and then you run some command. Maybe you save the file, or switch to editing another file, or go to peruse a directory. So you tap for a while and then you stop tapping, move your hands every so slightly to mash the control, or alt keys and then tap some other key, usually emphatically. The most commonly used key combinations end up being hit even more emphatically. Sit in a room full of developers using Emacs, listen closely; every once in a while, you’ll here everyone save almost simultaneously and go back to a furry of lower-case tapping.
Vi is slightly different from Emacs in that it is built up from two Unix commands: one for editing single lines of text, and another for moving between said lines of text. Thus, the cadence of a vi user is slightly different. Staccato taps followed by a bang as they switch from line editing to line navigation; more staccato taps, this time oddly spaced as they move between lines and place the cursor to begin their next fury of editing; another burst of staccato text entry; a quick and emphatic tap to take them out of editing mode and then a quick but punctuated trio of taps as they invoke the command that saves the file out, a sequence of finger movements so ingrained in the vi users brain that it appears as more of a gesture than a triplet of discrete key presses.
Here’s a project idea for pranksters: stand in a room full of people using vi and Emacs, listen for the really emphatic taps, and trip the room’s breaker right before they all finish their emphatic save commands. Cackle as chaos ensues.
The space between the taps
A roomful of vi-users, Emacs-users, and TextMate users is a homogeneous mess of clackity-clackity to the untrained ear. Most accomplished programmers are touch typists, so what you’re likely to hear is an undifferentiated stream of rapid-fire tapping. But if you’ve used these editors enough, and wasted enough time thinking about the aesthetics they represent, you can hear the differences in the punctuation as commands are invoked by arcane combinations and sequences of keystrokes.
In Vi and Emacs, there is a concise sequence of keys you can mash to do a regular expression search, move down three lines, go to the second sentence on that line, and replace the word under the cursor with “bad-ass text editing programmer, do not offend”. It is, in part, this power that attracts, fascinates, and empowers their users.
TextMate can do this, sure. But there is very little in the way of support from the editor to do it. You mostly have to put your eye on the piece of text you wish to edit and use some primitive motion keystrokes to get the cursor where you want it. Then you use those same keystrokes to highlight the text to replace, this time holding down a modifier key, then you type in the text you want. TextMate, compared to its programmers editor brethren, is a language of grunts and chuffs next to the sophisticated Latin or French of vi and Emacs.
Flow
TextMate is unsophisticated next to the extensibility and conceptual unity of Emacs, or the pure practicality of vim. So why do I keep coming back to it?
It keeps me in flow.
This is a very personal answer. I’m not saying you can’t achieve a flow-state with vi or Emacs. I’m saying that while I like the idea of those editors, understand the aesthetic, and enjoy watching skilled operators using them, I get lost in the punctuation when I use them. I either forget what punctuation I should use in some text editing scenario, or I have a nagging doubt that there is some better punctuation I could be using instead.
If vi is about navigating lines and editing those lines; Emacs is about invoking Lisp functions on files containing text, then TextMate is about primitive but direct manipulation of the text in a file. There’s very little conceptual overhead. You don’t need to know how the editor is enhanced in order to understand how to operate it. You don’t need to know when to put yourself in different modes of operation to make things happen. You just think of what you want the text to look like, you move the cursor around and you type on the keyboard.
It ain’t much, but I (often) call it home.
Breaking My Habits For Editing Programs
I’m a Unix guy, by upbringing. My first formative experiences in software development were on an early, Linux 1.x version of Debian. I’d used Windows, but always came back to Linux. When OS X got good enough around 10.2, I switched to something that didn’t require so much tinkering, so I could make more useful stuff.
Software development on Unix has skewed towards focusing on tools, languages, and text editors for quite some time. IDEs and browsers on Unix are a messy, foreign thing (just like everything else in Unix). Thus, I’ve long favored the terminal-and-editor style of development.
I’ve decided that now is the time for me to try something different. I like text editors and directly manipulating text, but I can see why some people feel naked without an IDE. The ability to pop-up a level and make a more broad-stroked transformation to a program is appealing. Having code navigation and semantic awareness baked in has lots of potential.
I’ve probably said grumbly things about RubyMine in the past, but I think now is the time to give it a go. Worst thing that could happen is that I don’t like it and I go back to the infinite tinkering of Emacs or the 85% perfect experience of TextMate.
I’ll let you know how it goes.
I originally wrote that a few months ago, at the apex of my editor neurosis.
I did give RubyMine a try, and I like some parts of it. It’s code navigation is pretty nice, it does an admirable job of integrating with the unique ecosystem of tools that a Ruby developer uses to manage their environment, and it does an excellent job of grokking TDD with test/unit and RSpec. RubyMine is a step in the right direction. I suspect that if I had muscle memory for IntelliJ, it would be the way to go.
But, I have muscle memory for TextMate and Emacs, and I have an affinity for being close to my tools. RubyMine felt one step disconnected from both my muscle memory and my tools. That’s quite an accomplishment; most IDEs feel several steps removed the tools and seem to discourage developing finger-memory in favor of menu-memory. I’ll give RubyMine another try in a year, probably, see how it’s coming along. But in the mean time, it’s great to see that there is a vendor out there tackling the challenge that is tools for Ruby.
A rambling, regurgitated thought on process
Elevator pitch: I’ve found that if you want to divert a productive team into an hour or two of semi-fruitless banter, ask how the team should use Git, Pivotal Tracker, and Capistrano to manage incoming work, verify it, and deploy it to production. In reality, you should ignore all the corner-cases and figure out what will enable you to push really small chunks of work with great frequency.
Ed. What follows isn’t novel, but it was a useful change in perspective for me, so I decided to share.
I’ve been thinking a bit about software processes lately. Despite great variation in telling you how to do so, most processes seem to focus on to do more stuff faster.
Lately, the notion of doing less has a lot of interest. Lean startups are the new-new thing and Getting Real is the old new thing; both preach getting more done and delivering value by doing less and analyzing the results more.
There are two kinds of “do less” a software developer can engage in. In the past I’ve been a little too focused on how I can take on fewer responsibilities from other parties. Literally doing less by scoping down features, putting off decisions, and focusing on things that seem like they really matter. I sometimes feel like I’ve become too eager to do less, making myself something of a cranky coder/slacker. But I digress
Recently, I’ve been trying to tackle doing less in my habits of creating software. How can I write less code to implement a feature, not in the minimalist sense, but in the “how do I just get it to kinda work sense”? How can I take less time between starting something and getting some form of it out in the wild? How can I make my code less coupled so there are fewer changes to make when I decide it needs to do something else? How can I make this less coupled to data storage so that putting it out requires less deployment effort? How can I make changes that are less likely to cause long-term regressions? How can I make it less effort to rollback bad changes?
When I look through the lens of accomplishing more by doing less, a lot of popular software methodology seems like dead weight. Rather than trying to find a process that addresses every team member’s own scars and affections, both perceived and imaginary, it seems most useful to imagine the smallest ruleset that won’t result in uncontrollable entropy and put it into action. If something starts to hurt, imagine the simplest new rule and put it into play.
The goal, as stated above, is to get to the point where you make really tiny, maybe imperceptible changes, and push them really frequently. Everything that stands in the way is the enemy.
Rails' Next Top Model
Of all the new and reimagined code in Rails 3, ActiveModel and ActiveRelation rank amongst what I find the most interesting. I’m excited that they potentially lower the bar to implementing one’s own data layer. If you’ve got some custom backend or datastore, writing a nice API around it has previously been quite the endeavor. To get it working is one thing; to make it as pleasant to use for application programmers is another thing entirely. ActiveModel and ActiveRelation have been extracted from ActiveRecord and make the task of building one’s own model layer far easier.
My presentation for RailsConf 2010 focused on what ActiveModel and ActiveRelation provide and how one can use it to write cleaner code in domain models, how to make your models feel more like ActiveRecord objects, and how to use ActiveRelation to build your own persistence layers.
I hope you find the slides educational. Further, I’ve posted the examples on GitHub so you can play along at home. If you’re particularly interested in ActiveRelation, I hope you’ll find the examples useful as a starting point to using that library.
Make More Awesome
Over the past year, I’ve been trying to create more stuff, some of which I’d hope to turn out awesome. Largely this is an ongoing sort of thing. I try things, I learn a little, I keep at it. Some things I try work, others don’t. I try to make a habit out of the things that have worked out well. I make a note of things that seem to help me get out of a funk when I’m not making as much as I’d like or having trouble putting in the hours I think are necessary to make cool stuff.
I first distilled these ideas into a talk I did at RubyConf 2009 on having more fun while coding. But I didn’t realize it at the time; I was just sharing some ideas about how to have fun. For me, one of the ways to have more fun is to make more time to have fun. But that’s just the beginning of making more awesome. I found I had to make more time, and develop a bunch of other habits.
For Big Design 2010, I honed the ideas, habits, and tricks I’ve found useful to me into a presentation on how to get off the couch, start making more things, and make some of those things awesome. I hope you’ll find it useful and perhaps start making lots of awesome stuff.