A conversation between fictional engineers in a fictional world
A hypothetical conversation that may have occurred between two non-existent engineers working on the second Death Star in the completely fictional Star Wars universe.
Engineer #1: Hey Bob, I was perusing the blueprints for this “Second Deathstar” this morning. Pretty impressive stuff.
Engineer #2: Thanks Hank. I’m pretty proud of it.
Engineer #1: And you should be! Had one question though. There was something in the request-for-proposals that mentioned some flaw in the previous one where a snub fighter could drop a torpedo through a vent and blow the whole thing up, yeah?
Engineer #2: Yep! Don’t you feel bad for the poor schmuck who made that decision?
Engineer #1: Haha, that’s a good one Bob. So you fixed that right?
Engineer #2: Oh, definitely. All the exhaust ports are small enough the only thing falling in there is a grain of sand.
Engineer #1: Nice thinking! So, my real question is, what’s this giant opening you can fly a large freighter through? And why does it lead right to the station’s giant fusion reactor that sits in a room big enough to fly in circles in said large freighter?
Engineer #2: Oh, that? Well, the passage from that room to the surface is where I’m going to run all the pipes and wiring that I forget about until the last second. I figure once I’m done patching everything together, no pilot would be able to fly through there, even in a snub fighter.
Engineer #1: And the giant room?
Engineer #2: Oh, you know clients. Always deciding they want something really impressive at the last minute. I figured I’d just leave a little extra room in case they come up with something at the last minute.
Engineer #1: Haha, right again Bob. Clients are such idiots.
Simple Ruby pleasures
I think I first discovered the joy of take and drop in my journeys through Haskell. But it appears that, since 2008 at least, we have had the pleasure of using them in Ruby too.
Need the first or last N elements from an Enumerable. Easy!
[sourcecode language=“ruby” light=“true”] ary = (1..100).to_a ary.take(5) # => [1, 2, 3, 4, 5] ary.drop(95) # => [96, 97, 98, 99, 100]
range = (1..100) range.take(5) # => [1, 2, 3, 4, 5] range.drop(95) # => [96, 97, 98, 99, 100]
hsh = {:foo => 1, :bar => 2, :baz => 3} hsh.take(1) # => [[:bar, 2]] hsh.drop(2) # => [[:foo, 1]] [/sourcecode]
The real magic is when you use take along with other Enumerable goodies like select and map. Here’s one of my personal favorites amongst the code I wrote in 2010:
[sourcecode language=“ruby” light=“true” highlight=“12,13,14”] class QueryTracer < ActiveSupport::LogSubscriber
ACCEPT = %r{^(app|config|lib)}.freeze FRAMES = 5 THRESHOLD = 300 # In ms
def sql(event) return unless event.duration > THRESHOLD callers = Rails. backtrace_cleaner. clean(caller). select { |f| f =~ ACCEPT }. take(FRAMES). map { |f| f.split(":").take(2).join(":") }. join(" | ")
# Shamelessly stolen from ActiveRecord::LogSubscriber
warning = color("SLOW QUERY", RED, true)
name = '%s (%.1fms)' % [event.payload[:name], event.duration]
sql = event.payload[:sql].squeeze(' ')
warn " #{warning}"
warn " #{name} #{sql}"
warn " Trace: #{callers}"
end
end
QueryTracer.attach_to :active_record [/sourcecode]
This little ditty is awesome because:
- It's super-practical. Drop this in your Rails 3 app, tail your production log, see the slow queries, go to the method in your app calling it, and fix it. Easy.
- It only activates itself when it's needed. Queries that execute quickly return immediately.
- No framework spelunking required. Rails 3's notification system handles all of it. Rails' backtace cleaner gizmo even makes the backtraces much nicer to read.
- It chains methods to make something that reads like a nice, concise functional program.
Enumerable joy, read up on each_cons.
Bundler, not as bad as they say
Of all the new moving parts in Rails 3, the one I see the most grousing over is Bundler. This is not surprising, as its a big part of how your application works and it's right up front in the process of porting or building Rails 3 apps.Bundler: As Simple as What You Did Before:
Bundler has a lot of advanced features, and it’s definitely possible to model fairly complex workflows. However, we designed the simple case to be extremely simple, and to usually be even less work than what you did before. The problem often comes when trying to handle a slightly off-the-path problem, and using a much more complex solution than you need to. This can make everything much more complicated than it needs to be.
I haven't run into anything with Bundler that I couldn't solve with a little critical thinking and maybe a little searching. On the other hand, Bundler has made getting dependencies straight amongst team members and deploying them to production servers far easier than it was before. I'm very glad that while it's not strictly part of the scope of Rails, that Bundler is now part of it.
Language and brains, an update
Does Your Language Shape How You Think? An update on the current thinking around the Sapir-Whorf hypothesis, the one about language and the words therein shaping the thoughts you can and cannot form.
Using Rails 3.0's notification system
How to use Rails 3.0's new notification system to inject custom log events. Ever wondered what the notification/subscription stuff in Rails 3 is? Wonder no more! I just used this to add performance logging around some Cassandra stuff in our Rails 3 app. Once you get the hang of it, this is really rad stuff.
Computers should do the boring bits
Future-proofing, Uniform Access, and Masquerades:
Boring work should be a cardinal sin in programming: it indicates something that the computer should be doing but isn’t.
An ode to Hashie
I was building an API wrapper this weekend. As is common when writing these sorts of things, I found myself needing something that takes semi-structured data (hashes parsed from JSON) and yields Ruby objects that are easy to work with. I've always found myself hacking these sorts of things together on a somewhat ad-hoc basis. It's a fun, but a bit of a yak-shave.
This time around, I decided to see if the state of the art has advanced in this realm. Luckily, I reviewed Wynn Netherland's slides from Lone Star Ruby Conference and found exactly what I needed.
Where have you been all my life?
Intridea's Hashie is a library built on the notion of making hash-like data structures act a little more like objects and a little easier to work with. I have literally wanted something like this for years!
Suppose you have a hash like the following:
hash = {
"name" => "Adam",
"age" => 31,
"url" => "http://therealadam.com"
}
Coding up an object to store that isn't too hard, but writing the code that pulls values out of the Hash and tucks them away in the right attribute on the object gets tedious quickly. Hashie's Dash class makes this trivial.
class User >Hashie::Dashie property :name property :age property :url end
Its even more delightful to use:
user = User.new(hash) user.name # => "Adam"
Tons of boilerplate code, eliminated. My life is instantly better.
A great use of inheritance
It's been pointed out that ActiveRecord's use of inheritance is somewhat specious. To argue that "user is-a ActiveRecord::Base" takes a bit of hand-waving. So lately, you'll find lots of libraries insinuate themselves into classes as a mixin, rather than as a parent class. This is a little bit of you-say-potato-I-say-potato, but whatever.
In Hashie's case, I think that inheritance is being used correctly. All of the classes that Hashie provides (Mash, Dash, Trash and Clash) inherit from Hash. So the is-a relationship holds.
Sugary data structures taste great
While I'm going on about inheritance, here's how I used to create these sorts of wrapper classes:
User = Struct.new(:name, :age, :url)
For creating simple objects that just need to hold onto some data, I really like this approach. If they end up needing data, it can easily grow up:
class User < Struct.new(:name, :age, :url) # Behavior goes here end
I like what Hashie is doing even more though. Its enhancing a core class in a largely unobtrusive way, and doing so from the confines of a library that only those who need it can pull from.
I'd love to see more libraries like this that add extra sass to Ruby core library. An Array that pages values out to disk on an LRU-basis perhaps, or a bloom-filter based Set, perhaps?
I'm excited about languages like Erlang, Haskell, Scala, and Clojure and what they can bring to the adventurous developer. Despite that, I feel strongly that Ruby still has plenty of really nifty tricks up its sleeve.
Examining software principles
There are too many good things to say about the Design Principles Behind Smalltalk. A few of my favorites:
Scope: The design of a language for using computers must deal with internal models, external media, and the interaction between these in both the human and the computer.
This one is really obvious until you get to the last four words. The human and the computer. Luckily we're starting to take for granted the primacy of human communication in programming lately (mostly), but when Smalltalk was created, I'm sure its designers received no shortage of grief when they steered towards humane optimizations.
Uniform metaphor: A language should be designed around a powerful metaphor that can be uniformly applied in all areas.
Smalltalk is largely objects and messages. Lisp is largely lists and functions. Erlang is largely pattern matching, functions, and actors. These aren't perfect languages, but once you deeply understand, really grasp the core concepts, you have the whole language at your command.
Operating System: An operating system is a collection of things that don't fit into a language. There shouldn't be one.
The first sentence is a great principle when considering what should go in the core of a system and what should go in the surrounding ecosystem of libraries. The second sentence is wonderfully bold, in that it cuts against what nearly every successful system has done since Smalltalk was prominent and in that it contradicts the first sentence. I'm not sure what practical use to make of this principle; its density of intrigue is that keeps me coming back to it.
Natural Selection: Languages and systems that are of sound design will persist, to be supplanted only by better ones.
I stopped worrying about what might supplant Ruby a long time ago. Someday, it will happen. And when it does, whatever succeeds Ruby will have to be really awesome to fill its shoes. I'm looking forward to seeing what that is. But the same goes for any technology; they are often replaced with something wholly awesomer than the incumbent.
I've never done it, but it seems like it would be intriguing and vastly informative to sit down with one of the systems I work on daily and try to extract these principles post-hoc. What values and principles are embedded in the system? What does that say about the team and why the system is the way it is? What principles are enablers and what bad habits should the team work to correct?
Making the complicated seem simple
Don Norman, Simplicity Is Not the Answer:
We want devices that do a lot, but that do not confuse, do not lead to frustration. Ahah! This is not about simplicity: it is about frustration. The entire debate is being framed incorrectly. Features is not the same as capability. Simplicity is not the same as usability. Simplicity is not the answer.
Norman goes on to explain how you can take a confusing mass of features and turn it into something less frustrating:
- Modularize into understandable clusters
- Map clearly from actions to results
- Model the ideas and actions cohesively
The article is about interaction design, but it fits just as well in designing programming languages and software.
Michael Feathers on how code grows
Festering Code Bases and Budding Code Bases:
Some teams produce what I call a festering code base. In a festering code base, the team changes the code primarily by adding code to existing methods and adding methods to existing classes. The results are predictable. Classes and methods grow malignantly, eventually becoming thousands of lines long.
Better teams produce budding code bases. Developers create new classes and methods and delegate work outward. Periodically, they collapse structure back into a simpler form, but the dominant trend is to grow the code by creating new structure.
I'd never put much thought into how code bases grow in the past. Feathers has some interesting ideas here about the characteristics of good and not-so-good growth and how languages and tools might promote good growth.
Incremental deployment at GitHub
Over the past year, I've read a lot about how teams are deploying their software. I've known for a while that Google has the ability to roll out new code to a small percentage of their servers and ramp up the breadth of deployment if they like how the software is behaving.
Lately, I'm starting to see more and more teams implement that sort of functionality. Rick Olson describes how GitHub implements it in How we deploy new features, and includes links to how Forrst and Flickr do it as well. At Velocity, Paul Hammond explained how to build an application-specific kind of version control into your app.
I'm a little surprised that few libraries have emerged for managing this. It would seem that, given all the excitement about continuous deployment, automated rollbacks, and incremental rollouts, someone would come up with something that they think is neat enough to share. I suspect that in fact, this is a really ugly, deeply application-specific sort of thing, no one likes to look at how they do it, and that's why there is plenty of talk about how to do it, but no libraries making it a simple thing.
The Cadence and Flow of Editing Programs
I figured out why my trists with other editors often end up back at TextMate. It sounds a bit like this:
Tap-tap-tap-tap-tap-tap; TAP; tap-tap-tap-tap-tap-tap-tap-tap-tap-tap-tap-tap; TAP; TAP; tap-tap-tap-tap-tap-tap; TAP.
When I’ve used vi and its descendants, it sounds like this:
Tap-tap-tap-tap-tap-tap; taptaptap; tap-tap-tap-tap-tap-tap; tapTAP TAP! tap-tap-tap-tap-tap-tap-tap-tap-tap-tap-tap-tap-tap-tap-tap-tap-tap-tap. tapTAPTAPtapTAP TAP!
And Emacs sounds like this:
Tap-tap-tap-tap-tap-tap; tapTAPtapTAP. tap-tap-tap-tap-tap-tap;tap-tap-tap-tap-tap-tap;tap-tap-tap-tap-tap-tap; tapTAP TAP; TAP TAPtapTAPtapTAPTAP. tapTAPtapTAP!
Lest you fear I’ve created some Ook-like language for describing shortcuts in any known editor, let me explain what’s going on here.
Cadence
Emacs is, at it’s core, a Lisp machine with a text editing language wrapped around it. Every interaction with Emacs invokes a function. Handily enough, the function that adds an “a” to the file you’re editing is bound to the a key on your keyboard. Oddly enough, the function that writes the file you’re editing out to disk is bound to the combination of hitting control and x at the same time, followed by control and s at the same time. Getting them out of order matters. Control-s followed by Control-x does something entirely different.
So when you use Emacs, you type a bit, and then you run some command. Maybe you save the file, or switch to editing another file, or go to peruse a directory. So you tap for a while and then you stop tapping, move your hands every so slightly to mash the control, or alt keys and then tap some other key, usually emphatically. The most commonly used key combinations end up being hit even more emphatically. Sit in a room full of developers using Emacs, listen closely; every once in a while, you’ll here everyone save almost simultaneously and go back to a furry of lower-case tapping.
Vi is slightly different from Emacs in that it is built up from two Unix commands: one for editing single lines of text, and another for moving between said lines of text. Thus, the cadence of a vi user is slightly different. Staccato taps followed by a bang as they switch from line editing to line navigation; more staccato taps, this time oddly spaced as they move between lines and place the cursor to begin their next fury of editing; another burst of staccato text entry; a quick and emphatic tap to take them out of editing mode and then a quick but punctuated trio of taps as they invoke the command that saves the file out, a sequence of finger movements so ingrained in the vi users brain that it appears as more of a gesture than a triplet of discrete key presses.
Here’s a project idea for pranksters: stand in a room full of people using vi and Emacs, listen for the really emphatic taps, and trip the room’s breaker right before they all finish their emphatic save commands. Cackle as chaos ensues.
The space between the taps
A roomful of vi-users, Emacs-users, and TextMate users is a homogeneous mess of clackity-clackity to the untrained ear. Most accomplished programmers are touch typists, so what you’re likely to hear is an undifferentiated stream of rapid-fire tapping. But if you’ve used these editors enough, and wasted enough time thinking about the aesthetics they represent, you can hear the differences in the punctuation as commands are invoked by arcane combinations and sequences of keystrokes.
In Vi and Emacs, there is a concise sequence of keys you can mash to do a regular expression search, move down three lines, go to the second sentence on that line, and replace the word under the cursor with “bad-ass text editing programmer, do not offend”. It is, in part, this power that attracts, fascinates, and empowers their users.
TextMate can do this, sure. But there is very little in the way of support from the editor to do it. You mostly have to put your eye on the piece of text you wish to edit and use some primitive motion keystrokes to get the cursor where you want it. Then you use those same keystrokes to highlight the text to replace, this time holding down a modifier key, then you type in the text you want. TextMate, compared to its programmers editor brethren, is a language of grunts and chuffs next to the sophisticated Latin or French of vi and Emacs.
Flow
TextMate is unsophisticated next to the extensibility and conceptual unity of Emacs, or the pure practicality of vim. So why do I keep coming back to it?
It keeps me in flow.
This is a very personal answer. I’m not saying you can’t achieve a flow-state with vi or Emacs. I’m saying that while I like the idea of those editors, understand the aesthetic, and enjoy watching skilled operators using them, I get lost in the punctuation when I use them. I either forget what punctuation I should use in some text editing scenario, or I have a nagging doubt that there is some better punctuation I could be using instead.
If vi is about navigating lines and editing those lines; Emacs is about invoking Lisp functions on files containing text, then TextMate is about primitive but direct manipulation of the text in a file. There’s very little conceptual overhead. You don’t need to know how the editor is enhanced in order to understand how to operate it. You don’t need to know when to put yourself in different modes of operation to make things happen. You just think of what you want the text to look like, you move the cursor around and you type on the keyboard.
It ain’t much, but I (often) call it home.