Learning from a dropped refactoring

You don’t have to deploy every bit of code you write. In fact, it’s pretty healthy if you throw some of the code you write before you even think about committing it.

Case in point: I sat down to refactor some Sifter code this weekend. I wanted to experiment with an idea about how Rails apps should be organized. I moved some code around, stepped back, and decided the idea behind my refactoring wasn’t sufficiently well cooked and stashed it instead of committing it.

I think that one can follow SOLID principles without radically decoupling your app from Rails and especially ActiveRecord. Instead, you let AR objects do what they’re good at: representing data. You push behavior, as much as possible, out into objects. What you should end up with is a handful of models that encapsulate the tables in your database and a bunch of classes that encapsulate the logic and behavior of your application; whether these classes live in app/models or somewhere else is a matter of personal taste.

My goal is to extract an object that encapsulates user authentication behavior; a class that is mostly concerned with conditions and telling other objects to do things. I extracted a class focused on “remember me” session behavior. It looked something like this:

class UserAuthentication < Struct.new(:user)

  def remember_token?
    user.remember_expires_at.present? && (Time.now.utc < user.remember_expires_at)
  end

  def remember_me(offset=10.years)
    time = offset.from_now.utc
    user.touch_remember_token(time) unless remember_token?
  end

  def forget_me
    user.clear_remember_token!
  end
  
end

Then, for compatibility, I added this to the model:

  def authentication
    UserAuthentication.new(self)
  end

  delegate :remember_token?, :remember_me, :forget_me, to: :authentication

The principles I’m following are thus:

  • Don’t expose AR’s API to collaborators. Therefore, UserAuthentication must call methods on User rather than directly update attributes and save records.
  • Encapsulate behavior in non-model classes. Therefore, User shouldn’t know when or how to manipulate “remember me” data, only expose an API that handles the mechanics.

The result: now my extracted class, UserAuthentication has meaning, but doesn’t really own any behavior. It’s a bit envious of the User model. That “envy” hints that its behavior really should live in the model.

Further, using delegate means the User model’s API surface area isn’t actually reduced. Granted, this is a stopgap measure. I should really hunt down all invocations of the delegated method and convert them to use UserAuthentication instead.

At this point, it feels like my refactoring isn’t making forward progress. I’ve rearranged things a bit, but I don’t think I’ve made the code that much better.

As I alluded earlier, I decided to stash this refactoring for later contemplation. Maybe next time I should start from the callers of the APIs I want to refactor and drive the changes from there. Perhaps my conjecture about decoupling ActiveRecord data from application behavior needs tuning.

I beat the “ship lots of stuff!” drum a lot, but I’m happy with this result. Learning to strike the balance between shipping the first thing you type out and the thing your future self won’t hate is an undervalued skill. It takes practice, and that’s just what I did here. Plus, I parleyed it into a blog post. Everyone wins!

What I wish I’d known about rewrites

I can’t say enough good things about How to Survive a Ground-Up Rewrite Without Losing Your Sanity. Having been party to a few projects like this, a lot of this advice rings true to me. Let me quote you the good parts!

Burndown

You must identify the business value of the rewrite:

The key to fixing the “developers will cry less” thing is to identify, specifically, what the current, crappy system is holding you back from doing. E.g. are you not able to pass a security audit? Does the website routinely fall over in a way that customers notice? Is there some sexy new feature you just can’t add because the system is too hard to work with? Identifying that kind of specific problem both means you’re talking about something observable by the rest of the business, and also that you’re in a position to make smart tradeoffs when things blow up (as they will).

The danger of unicorn rewrites:

For the Unhappy Rewrite, the biz value wasn’t perfectly clear. And, thus, as often happens in that case, everyone assumed that, in the bright, shiny world of the New System, all their own personal pet peeves would be addressed. The new system would be faster! It would scale better! The front end would be beautiful and clever and new! It would bring our customers coffee in bed and read them the paper.

Delivering value incrementally is of the greatest importance:

Over my career, I’ve come to place a really strong value on figuring out how to break big changes into small, safe, value-generating pieces. It’s a sort of meta-design — designing the process of gradual, safe change.

But “big bang” incremental delivery is accidental waterfall:

False incrementalism is breaking a large change up into a set of small steps, but where none of those steps generate any value on their own. E.g. you first write an entire new back end (but don’t hook it up to anything), and then write an entire new front end (but don’t launch it, because the back end doesn’t have the legacy data yet), and then migrate all the legacy data. It’s only after all of those steps are finished that you have anything of any value at all.

Always keep failure on the table:

If a 3-month rewrite is economically rational, but a 13-month one is a giant loss, you’ll generate a lot value by realizing which of those two you’re actually facing.

I really wish I’d thought of applying “The Shrink Ray”, an idea borrowed from Kellan Elliot-McCrea:

We have a pattern we call shrink ray. It’s a graph of how much the old system is still in place. Most of these run as cron jobs that grep the codebase for a key signature. Sometimes usage is from wire monitoring of a component. Sometimes there are leaderboards. There is always a party when it goes to zero. A big party.

Engineer your migration scripts as idempotent, repeatable machines. You’re going to run them a lot:

Basically, treat your migration code as a first class citizen. It will save you a lot of time in the long run.

Finally, you should fear rewrites, but developing the skill to pull them off is huge:

I want to wrap up by flipping this all around — if you learn to approach your rewrites with this kind of ferocious, incremental discipline, you can tackle incredibly hard problems without fear.

Whenever I talk to people with monolithic applications, slow-running test suites, and an urge to do something drastic, I want to mind-meld the ideas above into their brains. You can turn code around, but it takes time, patience, and a culture of relentless improvement and quality to make it happen.

Look up every once in a while!

Sometimes, I feel conditioned never to look beyond the first ten feet of the earth. Watch where you’re going, don’t run into things, avoid being eaten by bears. Modern life!

A Texas sunset

I see stuff like this out my office window every day. Be jealous.

When I remind myself to look up, there’s so much great stuff. Trees, antennae, water towers, buildings. Airplanes, birds, superheroes. Never mind the visual pollution of smoke, contrails, and billboards. Nifty things, natural and man-made.

Clouds in particular are nifty. They’re almost always changing, even if you look at the same patch of sky. They have pleasing shapes, and just a little bit of texture. Simple pleasure, clouds are.

And sunsets! Hooo boy, those are great. I thought they were overrated for a long time, but boy was I wrong. Colors, dynamics, fading off into darkness. I’m pretty sure sunsets invented the word “poetic”.

Ed. This originally appeared in my Internet Todo List for Enthusiastic Thinkers. It’s an email thing you can subscribe to. When you do, good things come to you, often via email. It’s free, and it bears no shilling for other people.

Exemplary documentation: size and purpose

There’s a lot to say about programmer-focused software documentation. It’s more crucial than many developers think, so it is often neglected. Even when its not neglected, it’s often an after-thought. I’ve noticed there are three kinds of documentation I’m interested in.

When I first come across some software, I want short and focused examples of how I can use it for my own purposes. I’m not looking for a lot of theory or exposition; just show me the benefit. If I can’t quickly see how the software works and makes my life easier, I’m very likely to discard it. In other words, I want shorter, “tweet-sized” documentation that sells me on the sizzle right away.

rbenv's README

rbenv‘s old README is a good example. I can see from the screenshot what using rbenv looks like. The bullet points make it easy to know the specifics of what this software is about.

If I come back to some software, I often want to learn the whole thing in one sitting. I want a longer document that I can read through in a serial fashion to learn most or all of the concepts and details about using the code. It should cover the domain ideas of the software, the individual APIs, and how it all works together to make something. To continue the metaphor, I want a well-written, “Instapaper-length” document worthy of reading in a comfy chair.

Backbone.js homepage

The Backbone.js homepage is great at serving as a long-form read. It serves as a reference document and guide at the same time.

After I start using something, I will often want to return to it to remember how to do specific things or to figure out if a task is possible at all. This is when I lean most on traditional API documentation. One to three paragraphs, easily searched are the ideal here. Kind of like the “Tumblr-post” of documentation.

I’ve yet to find all three of these qualities in the documentation for a single piece of software. Finding that software has done a really good job at one of them is delight enough. I can’t imagine how excited the world, at large, would be if something were to have all three. There would be a lot of rejoicing.

The Third Shift

In the days of industrial labor, many factories ran three shifts per day. Three eight-hour shifts per day keeps a factory fully utilized and some business major’s spreadsheets happy. Luckily, for many of us, knowledge/thinking oriented businesses don’t usually follow this paradigm. We’re not (often) pressured to pick up a double shift, possibly freeing time to do useful things that we don’t get paid for.

For the ambitious (possible euphemism), this opens up an interesting opportunity: allocating the second shift to one’s own projects. Writing that great book you’ve got inside you, penciling a comic, running your Etsy business on the side, or bootstrapping that web app you’re dreaming about all make a great fit for a second shift. Find time before or after your day job, and then aim for the sky.

I found it easy to take this logic to the next level and think, well if two shifts works and I can make progress on _two_ things, three shifts might work and then I can do _three_ things! Wake up early, do something awesome. Work the nine to five, do awesome things. Take a couple hours in the evening, do even more awesome things. Seems good, right?

Unfortunately, the third shift is a bandaid over too many projects and lead me to do lower quality work across the board.

I need more physical rest and mental space than working on three things affords. Turning down an extra hour of sleep or the bleeping of an alarm clock is a hard bargain. One side project, as it turns out, is plenty.

That said, the third shift _is_ useful as a “turbo button” that I only press when I really mean it and used only for short-term projects that are important to whatever awesome thing I’m trying to do. A couple weeks waking up early to bang out a presentation or longer-form article are good. Sustaining that for a series of projects doesn’t work for me.

In short: ambition is great, but striking a balance with mental and physical rest is better.

A newsletter

So I did this thing where I wrote a newsletter. I’m going to do it again. The first iteration of this publication was a bit like a written late-night variety show. I wrote about interesting articles, or things that interested me. Each “episode” almost always closed with some kind of musically awesome thing I’d found on the internet.

The next iteration of this newsletter will be like a hand-delivered transmogrification of this weblog. I’ll include links to the articles I thought were most special or had a surprising reception. I’ll occasionally write “commentary tracks” on how an article came to be. Each edition will almost certainly end with a musical or pop culture find, because what fun is running a newsletter if I can’t annoy you with my pop culture tastes?

My hope is that you’ll find this interesting. You can take a look at what I’ve written previously and subscribe to this free internet email newsletter at your discretion. I think you might like it.

Web design for busy programmers

Here it is: I’m somewhere between horribly afraid and way-too-smart to seriously attempt front-end web work. Browsers are not the software whose bugs I am interested in knowing about.

That said, putting information on the web that doesn’t look like utter dross is a kind of required literacy in our field. While bravely dipping my toes back into the front-end waters, I recently found some great tricks. Rediscovered, probably, but I’m not sure where the idea originally came from.

Most important: design in greyscale. Color is hard and can lead to tinkering. My goal is to get in and out of the front-end bits quickly, so tinkering is the enemy. Greyscale is one dimensional, greatly simplifying matters. Give important information higher contrast and less important information or “chrome” less contrast. Now you’re done thinking about color.

Almost as important: use a fixed-with font. As a programmer, you look at them everyday, so it’s a touchpoint of comfort. Pick a font you don’t use in your editor all day, just so you can stare at something different for a while. Copy and paste a “font stack” from the aptly named fontstacks. Make important things big and unimportant things small. Now you’re done thinking about type.

The key to avoiding browser dragons, it seems, is to skip horizontal layout, i.e. pull quotes, text wrapped around images, etc. It’s pretty easy to use CSS if you only run things down the left side of the page. All the depth and despair of CSS is in trying to get things to appear off the left margin. So don’t do that. Leave it to people who know how browsers work and how to manage their gnarly bugs. Now you’re done thinking about layout.

It’s tempting to think you should make your code examples look really nice. Don’t worry about it; highlighting code is of marginal value. You’ll never be satisfied with how it looks. The human mind is capable of reading code without a rainbow spectrum of colors. Spend time on writing about the code, not on polishing the colors and how its highlighted.

With all of those things out of the way, your way is clear to think about the really important things. What do you need to say, how do you structure the message, what do you leave out, how do you organize all the information? That’s the essence of publishing on the web, not the accidental complexity of making things look interesting.

The gift and the curse of green-field projects

The “green field” in software is a gift and a curse.

On the bright side, you have an opportunity to use the new-shiny. Past wrongs can be righted. You can move quickly, without worry about why some code exists, how it works, or whether you should care about it. Life is good.

Fresh-cut grass

Does anything smell better than grass you didn’t have to cut yourself? Courtesy KENSR199

The peril I’ve found is that green field projects are by their nature isolated. They don’t have a deployment or monitoring story. They don’t spring forth fully integrated with other critical systems. The project probably hasn’t proven itself as useful yet.

Letting a green-field live in isolation too long is the root of lots of problems. I’ve experienced scope creep, confused expectations, and declining morale that all could have been avoided had I brought a green-field project “into the fold” sooner. But the whole point of a green field is that you don’t integrate too soon, lest you spin your wheels on legacy things.

Green field projects are fun and an often welcome change of pace from working within an existing system. However, succeeding on a green field project is just as hard, or harder, than succeeding with a legacy system. It’s a different set of trade-offs that each developer has to get good at.

Hypermedia chicken, web browser egg

A lot of the hypermedia philosophy is centered around the idea that API clients should work a lot like web browsers and plain-old Hypertext Markup Language. They should follow hyperlinks, leverage media types, cache data when they can, and intelligently take advantage of the meaning of hyperlinks whenever possible.

Good APIs require good services *and* good clients

The problem with that is, web browsers are way more capable than HTTP clients developers are using to build API clients with.

Here are some things web browsers have become pretty good at:

  • GET and POST requests
  • Following links and redirects
  • Discovering data structures via HTML forms and submitting data using that schema
  • Using headers to negotiate content types
  • Caching data when possible and expiring those caches
  • Handling streaming data and chunked responses
  • A bunch of stuff I’m probably forgetting

Here are some things the HTTP client in your typical standard library are good at:

  • GET, POST, PUT, and DELETE requests
  • Following redirects (if you manage to set the right options)

What’s at play here is a chicken and egg problem. Client developers can’t build on hypermedia principles until they are working at the level of hypermedia abstractions. Arming them only with HTTP requests and the ability to choose their encoding and schema poison is too low level.

Protocols like HAL or Collection+JSON could light a path to solving this problem. Rather than dealing in pure data, services and consumers work with data annotated with hypermedia-like semantics for traversing data structures and creating new data. If these protocols can gain traction, it’s “simply” a matter of getting HTTP clients into widespread use (read: standard libraries in stable releases of your-favorite-programming-language) that are as good at HTTP as web browsers are. At that point, API providers and API consumers could start using hypermedia principles to build APIs for those who aren’t interested in the mechanics of hypermedia.

Until then, it seems to me that putting hypermedia principles front-and-center is suboptimal. It’s akin to telling someone who wants to use your website that they need to understand MVC patterns first. It’s only going to discourage and confuse them.

How to understand Saturday Night Live

Saturday Night Live is a changing thing. It’s not new like it was in the seventies, it’s not a powerhouse like it was in the nineties, it may not be the training camp for NBC sitcoms anymore. Despite that, its still a big dog in the worlds of comedy and pop culture. Every time I hear or read “SNL was better when…”, I cringe a little. As far as I can tell, this isn’t true.

Everyone’s got their favorite cast. Ferrell/Shannon/Sanz/Oteri, Fey/Poehler/Rudolph, Hartman/Carvey/Myers. It seems largely to depend on whenever you started watching SNL or when you were a teenager or in college. So when someone says “SNL isn’t relevant any more”, I mentally substitute “I liked SNL better with the cast I watched first.”

Manhattan Skyline

Over the years of watching and reading about SNL, I’ve come to understand that the show is very much about the people on camera, but Lorne Michaels is the show. The only time the show has been in consistent decline was when Michaels wasn’t around in the early eighties. For the past thirty years, claiming SNL was on the down seems to be more of a sport than a rational argument.

Since Michaels’ return, the show is subject to fractal cycles. Each night, some skits kill and some skits bomb. Generally, the front of the show is better than the back; if you stay tuned after “Weekend Update”, you should count yourself a long-time fan, willing to see some weird and/or flat skits, or asleep on the couch.

If you zoom out to look at how a season flows, you’ll again find shows that are really great and some that aren’t. My theory is that this entirely depends on the quality of the host. A mediocre host seems to bring middling material out of the writers and performers. A good or high-profile host seems to bring good-but-not-great material and pretty good performances. One of the darling hosts, like Alec Baldwin and the more recent Jon Hamm, brings the A-game material from the writers and performers play up to the occasion.

Interestingly, musical guests can bring a certain electricity too. Paul Rudd is a capable host, but pairing him with Paul McCartney led to a show that was pretty electric. I defy you to watch that episode and tell me SNL just isn’t as good as it used to be.

Zooming out to look at successive seasons, you see the same sort of up-and-down. Will Ferrell’s first season was good, but not great. He definitely left at his sketch comedy peak, and the show was briefly weaker for it. But right on his heels came the Fey/Poehler/Rudolph powerhouse. There’s an ebb and flow as casts come together, hammer out a few good seasons, and then move on to other stages.

That’s how I understand SNL. Perhaps I’m seeing cognitive biases towards the show through my own cognitive biases. I think it’s still a relevant benchmark of American pop culture.