Refactor the cow paths

Ron Jeffries, Refactoring — Not on the backlog!

Simples! We take the next feature that we are asked to build, and instead of detouring around all the weeds and bushes, we take the time to clear a path through some of them. Maybe we detour around others. We improve the code where we work, and ignore the code where we don't have to work. We get a nice clean path for some of our work. Odds are, we'll visit this place again: that's how software development works.

Check out his drawings, telling the story of a project evolving from a clear lawn to one overwhelmed with brush. Once your project is overwhelmed with code slowing you down, don’t burn it down. Jeffries says we should instead use whatever work is next to do enabling refactorings to make the project work happens. Since locality is such a strong force in software, it’s likely that refactoring will help the next bit of project work. Repeat several times and a new golden path emerges through your software.

In other words, don’t reach for a new master plan when the effort to change your software goes up. Pave the cow paths through whatever work you’re doing!

A few qualities of mature developers

What is technical leadership? Per Mature Developers, it’s a lot of things. My favorites:

So one of the first and most important qualities of mature developers is they’re more often than not paying attention to what is going on around them. They’re deliberately taking their time to observe before proceeding (put succinctly as STOP; Stop, Take a breath, Observe, Proceed).

It is so hard for me to do the stop and breath part.

Sharing the [technical] vision with other involved parties not only serves as a perfect opportunity for practicing one’s skills to explain deeply technical terms and circumstances with non-technical people. It also serves the purpose to validate the vision in terms of relevance to business value and other aspects.

Assessing and understanding risks better puts them into a position where it’s also more likely they’ll actually take risks. Risks which, without the knowledge about business value and the bigger context, may look too big to be worthwhile. But not for mature developers who are able to see beyond the obvious risks and include more aspects into their judgement.

Managing risk, but not overmanaging it: also very difficult.

Previously: Thoughts on “Being a Senior Engineer”.

A few folks suggested I try lazy enumerables to make my extremely chained style practical. I was curious about the actual costs of my style, so it’s time for lies and microbenchmarks! Turns out naively chaining a bunch of maps together isn’t very costly, so go with that to start.

Lazy came in much slower than consolidating the logic in one loop or chaining them without lazy. I thought, I must not have used lazy properly. Turns out, I’m probably showing that laziness isn’t well suited to iterating over collections without an early termination clause (e.g. a take, first, or find) and that for small collections (like an 87-line /etc/passwd), the cost of the lazy plumbing can noticeably outweigh the work done inside the loops. Thanks to Rein Heinrich for talking me to the bottom line!

One idea per line

Lately, I’m doing a weird thing when writing Ruby code. I’m trying to only put one idea or action per line. I’m not sure about it yet.

Here’s what a method to fetch item-y things might look like:

def fetch_items(options={})
  limit = options.fetch(:limit)
  timestamp = options.fetch(:timestamp)
  paged_helper = PagedHelper
  client = OurHttpClient

  responses = paged_helper.
    new(limit, timestamp).
    fetch_pages { |params| client.get(params) }

  responses.
    map { |r| JSON.parse(r) }.
    map { |h| ItemCollection.new(h) }.
    map { |ic| ic.items }.
    flatten
end

For the sake of comparison, here’s how I may have written that method a couple years ago:

def fetch_items(options={})
  helper = PagedHelper.new(limit, timestamp)
  responses = helper.fetch_pages { |params| OurHttpClient.get(params) }
  
  responses.map { |r| ItemCollection.new(JSON.parse(r)).items }.flatten
end

I like that the pace of reading the first example is even. You don’t arrive upon some monster line of code that does a multiple things. You don’t have to unpack what’s happening in a situation where you’re calling f(g(h(some_args))). It makes moving lines of code around much simpler because each one is only dependent on what comes before, and not what happens inside. It’s a little easier to write a three-part method, which I really like.

But still, I hesitate. My methods end up about 50% longer. Breaking up the Enumerable transformations into multiple loops instead of one loop doing a bunch of work is probably pretty slow. I have to come up with a lot of names (which is, I think a net good), some of which end up a little redundant.

I’ll let you know how it goes. It may not even survive code review, who knows!

Programming is easier when you know how to stop solving 100 problems with 1 fancy thing and solve 100 problems with 20 plain things.

Ember is probably leading the JavaScript framework pack by supporting releases with security patches for slight more than a year. By comparison, there’s a cottage industry of garages restoring and updating old Porsche sports cars then selling them for ridiculous prices. The USAF (the same one, curiously, that is spending $1.5 trillion on a useless jet, somehow) is going to use their largest strategic bomber, the B-52, for one hundred years.

I’m always thinking about Greg Borenstein’s words when it comes to technology churn:

The constant churn of web technologies hobbles the creation of timeless learning materials and continuity of knowledge across generations.

We should try harder on this.

Code that resists

Kellan Elliott-McCrea, on the way towards an understanding of technical debt, catalogs the ways we end up with code that resists our efforts to change it:

Therefore the second common meaning of “technical debt” is the features of the codebase we encounter in our work that make it resist change. Examples of features that can make a codebase resist change include: poor modularization, poor documentation or poor test coverage. Just as easily though an abundance of modularization (and complexity) or an abundance documentation, and tests encoding the now the incorrect old behavior can apply a strong downward pressure on change.

A little discussed and poorly understood design goal for code is disposability. Given change, what design patterns can we follow that allow us to quickly expunge incorrect behavior from our codebase? Interestingly it is a much more tractable metric for measuring as opposed to more popular criteria like “elegance”. (a post for another day)

Put that in your thinker. Does something like Strategy or Adapter let you throw out whole classes when they prove unnecessary? Or is that so only when you luck out and chose the exact right axes of disposability? Does a microservice really let you discard codebases wholesale? Can maps and functions free you from intertwingled state and behavior or does it move the resistance somewhere else?

Grumpy, opinionated answers: possibly! Even more possibly! Meh. Very meh.

Three part method

I find methods/functions decomposed into three parts really satisfying. Consider a typical xUnit test:

def test_grants_new_role
  # setup
  user = make_user
  new_role = make_new_role
  
  # behavior under test
  user.add_role(new_role)
  
  # assert results
  assert_equal [new_role], user.roles
end

Lately I’ve been structuring Rails controller similarly:

def create
  # Extract inputs/parameters from HTTP request
  person_params = params.require(:person).permit(:name, :age)

  # Invoke behavior encapsulated in a Plain(ish) Ruby object somewhere
  user = UserService.create_user(person_params)
  
  # Check the result and make some HTTP output
  if user.persisted?
    redirect_to user_path(user.id)
  else
    @user = user
    render :new
  end
end

Clojure even has the let form which encourages this style:

; annotated from clj-http
; https://github.com/dakrone/clj-http/blob/master/src/clj_http/util.clj
(defn gzip
  "Returns a gzip'd version of the given byte array."
  [b]
  (when b
    ; set the table
    (let [baos (ByteArrayOutputStream.)
          gos  (GZIPOutputStream. baos)]
     
      ; do the work and clean up
      (IOUtils/copy (ByteArrayInputStream. b) gos)
      (.close gos)

      ; produce a result
      (.toByteArray baos))))

I don’t think there’s anything inherently wrong if a method or function isn’t organized this way. But when I read code structured this way, it feels less like a bunch of random logic and more like a cohesive unit that someone put time into thinking through how someone might try to understand it later. The Rule of Three rules everything around us.

The future of programming is design, teaching, and empathy

The Future Programming Manifesto starts with this header:

Inessential complexity is the root of all evil

OK, I’m on board!

We should measure complexity as the cumulative cognitive effort to learn a technology from novice all the way to expert. One simple surrogate measure is the size of the documentation.

Perhaps we could describe the complexity of a technology in “bookshelves”? For example, in my second internship I met a CleearCase administrator whose office bookcase had one shelf devoted to SunOS, one shelf to Oracle, and the final shelf dedicated to ClearCase itself. How many bookcases for Ruby, Rails, JS, CSS, a database, and all the other stuff you need to know to put a CRUD app in your browser (not even deploy it to the web!)

  • Maintaining compatibility increases complexity.
  • Technical debt increases complexity.
  • Most R&D is incremental: it adds features and tools and layers. Simplification requires that we throw things away.
  • Computer Science rejects simplification as a result because it is subjective.
  • The Curse of Knowledge: experts are blind to the complexity they have laboriously mastered.
  • Rewarding programmers for their ability to handle complexity selects for those who love it.
  • Our gold-rush economy encourages greed and haste.

A weird thing about programmer is that those that rant endlessly about someone else’s complexity, layers, and haste are almost completely blind to the complexity, layers, and haste they make in an effort to set the world just so.

We should work for end-users disenfranchised by lack of programming expertise. We should concentrate on their modest but ubiquitous needs rather than the high-end specialized problems addressed by most R&D. We should take inspiration from end-user tools like spreadsheets and HyperCard. We should avoid the trap of designing for ourselves.

What if more of programming was accessible as data manipulation (cf. spreadsheets, data files, JSX templates) instead of as logic and behavior (i.e. almost every programming language)?

We are doing Design: using experience and judgement to make complex tradeoffs in order to satisfy qualitative human needs.

This reminds me of Developer Experience. “Developer experience” is a weird word right now, but it’s becoming table stakes for success. It’s a design discipline. It’s considering the form and function of code. It’s the opposite of attempting to learn C ;)

Long story short: we’re gonna need more empathy, more design skills, and more teaching skills to reach the next level of great programming languages and tools.