On music, mostly

You know how sometimes, everything is clicking and you've just got it? Some people call it flow. On Thursday, I was in a quiping flow. You may have witnessed it on Twitter. I thought it would be fun to try and weave it into a coherent narrative, so here we go.

Sara Flemming started a new blog about digging into the technical mysteries she comes across as she works. It is, brilliantly, titled Visiting All The Turtles.

Upon seeing a press photo of Adele, I had an epiphany. As an SAT analogy, Achilles Heel is probably about like Adele's eyelashes. All of her singing powers come from those lashes.

There's not many ways to connect Brian Wilson and Axl Rose, except that they both worked on an album for more than a decade, managed to finish it, and missed the moment when it would have been a huge deal. That said, Brian Wilson's album Smile is way better and about as genius as you'd expect. It's better to be a follow-up to Pet Sounds than a follow-up to Use Your Illusion, even though that's my favorite Guns 'n Roses album (haters?).

It's easier to draw a connection between Igor Stravinsky and Brian Wilson. Listen to the former's ballets or the latter's albums (not the surf songs) and you'll always find something strange going on. A flute trill where it doesn't make sense, a honking bass clarinet, a bizarre harmony. It's fantastic.

As you can tell, Brian Wilson is a kind of my jam lately. It would be a shame if that guy doesn't have the opportunity to make all the music that is bouncing around in his head.

I've never actually been around someone “vaping”, but I'm pretty sure I don't like it based on the name alone. Because that word will never get mispronounced or misheard in a booze joint. Great job, tobacco industry!

On contemporary indie/rock music. No value judgement, just an observation of the way it is:

A one. A two. A one two three four.

hits play on drum machine

Semi-related: I am so glad 7-string guitars are (mostly) no longer a thing.

Tinkering with coffee plus condensed milk has brought my iced coffee game way up. I highly recommend it, if you have the means. Just be prepared to stir, a lot.

To wrap it up, on some other music I've enjoyed and thought a bit about lately:

Ben Folds taps into pathos. Bruce Springsteen taps into the American Dream. Dave Grohl taps into the part of us that just wants to turn it up.

Listen to suit.

20% of programming is duh

Sometimes I think 20% of programming is staring at a problem for thirty minutes and thinking there must be a really simple solution right in front of me and then, eureka and then, duh.

Note that it sometimes takes multiple sittings to reach that simple solution. Last night, for example, no eureka. Step away, try again, repeat. Brains are weird.

Oh, the complexities you'll know

Carried complexity is the bane of your application. When you add something to software, you incur the cost of doing the work plus the risk that the change will break something in already deployed versions of the software. But you also possibly incur the cost of understanding that addition later, probably in the context of other additions, and then trying to figure out what’s broken, slow, or needs to change to add this other new thing.

Software developers are most adept at identifying and resisting complexity when it comes from the product or business guys. Adding this feature will take a few days, fixing this bug will take a few hours, generating that report will take a week. This change might sound good for getting that new customer, but the way it interacts with the other features will confuse lots of people in the long run.

You’ve probably thought or said all of these things in the past.

It’s important to realize we put it upon ourselves too. Are there two ways of solving the same problem in your project, team, or organization? Are you migrating from one framework/queue/architecture to another? Those are complexities that you are carrying around in your head. They aren’t inherently bad, but they are costly, and it is important to weigh the pros and cons before you commit to them. When you choose to incur complexity, make sure it’s buying you something and not just shuffling the bits around a little bit.

Confidence despite evolving systems

Facing risk by instrumenting the hell out of it:

Software development is a complex system existing as it does at the intersection of people, systems, good intentions, confused and changing goals, and overly literal state machines. Past behavior isn’t always an indication of future behavior, and humans are terrible at reasoning about complex systems. As such we’re unlikely to know or have good visibility into whether we’ve reached a steady state and our hypotheses are likely to be wrong. In this uncertain and complex environment we initiate change only when the cost of not making a change overcomes the fear of making it.
First in a series on how Etsy writes, deploys, and operates changing software without The Fear. Thanks for writing this stuff down, Kellan!

Problems as ever-changing mazes

Problems, puzzles, startups as dynamic mazes:

just running to the entrance of (say) the “movies/music/filesharing/P2P” maze or the “photosharing” maze without any sense for the history of the industry, the players in the maze, the casualties of the past, and the technologies that are likely to move walls and change assumptions
I love this idea about thinking of solving systems as though they were an ever-changing maze, with history (fallen players) embedded within the system. Doubly so when you extend the metaphor to solutions that route around one problem to brazenly take on another problem. If this had a further extension to football playcalling, it would be perfect.

Refactor for value over cleanliness

Practice Responsible Refactoring:

When cleaning up the code enables you to work faster for a task you aren’t dreaming up but actually have at hand, refactoring is the way to go.
Dave Copeland makes the point that refactoring without a value-added change (feature, improvement, bug fix, optimization, etc.) is a losing proposition. By the numbers, he's absolutely right. Further, I've found that probably half of the refactorings I'm convinced are necessary aren't as simple or useful as I thought once I get an hour into them. Despite all that, keep doing therapeutic refactorings for practice and to keep your spirits up.

Finishing software ain't easy

When I start work on a project, whether for personal or professional purposes, I have a sense that I need to devote myself to it. That I should figure out how to make it the best it can be, I should only commit the best codes to it; it should be an exemplary piece of work.

On the one hand, this is taking pride in my work, and that’s good! On the other hand, this is ownership, and that’s a little problematic.

The problem with ownership is it leads to irrational decisions. There’s always one more improvement I could make, because this is the thing I own. There is little bit of scope I could add, because I can make this thing really good (at least in my head). There’s one more bug to fix (inexplicably, in my head).

What I’m finding all this points to is, finishing is hard. It’s easy to start a project. It’s challenging to get it out there, and get people to use it. It’s really hard to get it to the point where it’s running itself, you can delegate its operation to others, or it doesn’t need further work.

So that’s a thing I’m trying to figure out. Developing.

Improv perspectives on changing code

In the last improv class I took, we spent a lot of time focusing on four kinds of scenes that appear in improv with astonishing frequency:

  • Straight/absurd: A character has a strange perspective on the world, another points out the absurdities in what they're saying and encourages them to say even more absurd things.
  • Peas in a pod: Two characters who are very similar in demeanor, perspective, or motivation interact with each other.
  • Alternate reality: Two characters inhabit a world notably different from ours; maybe gravity is no longer a thing or it's entirely normal to wear ketchup as formal wear.
  • Real: Two players interact with each other mostly as themselves, bringing their own personalities and perspectives to the scene.
I noticed that, when faced with a puzzle to solve, such as code to write, these kinds of perspectives often pop up too:
  • Peas in a pod: take some code that already exists in an app, clone it somewhere else and make it do something slightly different. Extract the boilerplate and ship it.
  • Real: the code around the functionality I need to change, improve, or add to is already just fine (I probably wrote it or have an awesome team); I just code like I code.
  • Straight/absurd: the code I'm working on has good parts and bad parts; if at all possible I make my changes in the good parts or figure out how to make a new good part for my changes to live in.
  • Alternate reality: the code I'm working with is utterly bizarre and strange; I have to make lots of tactical decisions about how to make progress while bringing some level of sanity to it.
See also: Novels, Yes And Improv Comedy.

The simple problem inside the complex one

A sophisticated solution to a complex problem is fun to find. Its even fun when someone else finds the solution and thoroughly writes it down.

Despite the thrill of recognition for tackling a big complex problem, I find it's more practical to seek the simple problem lurking in and around the big problem. Not all tricky problems are complex. Some are presented in a complex way, some are complex because of restrictions that are easily worked around, and some are complex because of an adjoining social problem. Find the simple problem, or the social problem, and solve that instead. It often works for me.

My favorite kind of solution is one where the solver has taken a problem with a big surface area, found the core of the problem that is 80% of what people care about having solved, and then solved that tiny subproblem. If you want to see masters of this approach, check out the works of Blake Mizerany and Ryan Smith.

Technology that's not a startup

Here’s a nice story on technology that isn’t startups: Unhappy truckers and other algorithmic problems. Logistic networks are a technology, just like smartphones. They make our world way better, but they do so invisibly and at a slower pace than the churn of mobile apps, web frameworks, and startups. But they’re still solving problems, moving the needle. They’re just, possibly, less obsessed with technology tribalism and fashion. Some days, that seems like a pretty useful space to find oneself in.

I don't have time to not teach

It wasn’t long ago that other developers not knowing the things I know was really frustrating. “How could they not know this?!” I thought that I didn’t have time nor should I be expected to train other developers. If I could learn all this stuff, they can too. (Older, wiser Adam: younger, more naive Adam was very wrong!)

At some point, my perspective on this did a complete turnaround. Now I’m eager to teach other developers things they don’t know. The major benefit is, now they know the things I know! A side benefit is now I know the thing better because I had to teach it.

I was completely wrong when I thought I didn’t have time to teach. Turns out, I don’t have time to not teach other developers the things I think are important.

Overtime means your business is hurting

Overtime is Morphine, Ernie Miller:

A developer who is truly concerned about the health of his or her company also must be careful to ensure the "patient" isn't developing an unhealthy dependency on their heroics, allowing the company to limp along without experiencing the pain that should accompany unwise choices. Pain is how we learn to avoid repeating mistakes.
I've seen too many developers put in a heroic effort, only to repeat it the next day, sometimes without sleeping in. That's "killing the patient", to extend Ernie's metaphor. It's not the natural state of a business to notice the human cost it might have. The people inside the business have to assess that cost and do something about controlling it.

If your business, or the one you work at, requires regular heroics, consider that it is a broken system. Luckily, software developers are well equipped with mental routines for diagnosing and patching broken systems. Time to hack the organization.

Scala and Clojure in terms of city building

The Scala folks are building newer, better cities on top of older cities, which is how things really work (e.g. Paris, Rome, Boston, etc.). The Clojure folks are building Dyson spheres in space, which is a little ambitious and maybe not entirely realistic.

That said, sometimes wouldn’t you rather live in a sci-fi fantasy world rather than on top of layer after layer of archaeology?

Tools for software in the large

When software becomes successful, software often becomes large. More features, more support systems, more infrastructure, more people, etc. Therefore, software “in the large” seems like a good problem to solve: how can you work at the same pace with one person, one application, and one server when you reach a hundred people, ten applications and a thousand servers?

Right now, I think the only tool we have for reducing the overhead of large systems and organizations is language. Frameworks and libraries can help make larger applications plausible, but they don’t resist the forces that make a large application hard to work with. Tooling and process, like source control or CI, only help us keep large software going; they don’t slow us down on the path to producing huge software.

Better programming languages make it possible to produce software that does more things in a more concise statement. To some extent better “API language” makes that possible too, but it relies on the power of the host language.

Am I overlooking other tools that make software in the large a manageable endeavor?

Less beautiful code, more code that works in production

Developers should care and feed for their systems, especially when they’re in production. Alerting, logging, and metrics are the tools you need. Further, you need to use actual production errors as a feedback loop to push you to write more fault tolerant code.

I’ve struggled with that part in the past. It’s tempting to think I can just code along, building beautiful code. Beautiful code is entirely different from production code. Handling errors, timeouts, partial failures, logging, instrumentation: it’s not exactly design pattern stuff. To wit:

Fault-tolerant code is ugly. It requires conditionals. It won’t look like code in programming books, screencasts, or blog entries. This is the way it has to be, I’m sorry.
Production is all that matters. Highly intelligent words from Dave Copeland. The beauty of code, as it appears in my text editor, is not as important as how it works when it runs on servers. New mantra!

Uncertainty, feedback, confidence, a happy ending

Yesterday, I wanted to setup a quick feedback loop for writing some production code. But, I wasn’t entirely sure what that code should look like. Further, I hadn’t worked in the code base for several months, so I wasn’t sure how the new code would need to interact with other objects.

This is a scenario that often gets me in trouble. I end up spending too much time contemplating how things might be without the feedback of what’s possible and plausible.

What I ended up doing:

  • Start a new test case for the code. I hardly know what it should do, so I ended up with it "should do things".
  • Write the kind of code I would run in a REPL to get started writing the production code. This was a service client, so I did things like setup a connection, invoke a request, and handle a response.
  • Write a test assertion about what I think the side-effect or result of the production code should be. For this service client, I wanted to make sure a useful thing was returned.
  • Run the test, see it fail, learn some things about the REPL-y code I wrote. Since I'm writing Ruby, my feedback rate goes way up here. Running a single test, which I can do with a keystroke in my editor, is way faster than switching over to an IRB, reloading code, and finding the right incantation to run again.
  • Iterate on the code in the test case until I have something that passes. Think, type, feedback. Observe, orient, decide, act. Coding is fun.
  • Once I have passing code, start rearranging so its closer to what I want as actual production code. Replace dummy values with real ones. Figure out how to remove any cheats I may have put in place for expediency.
  • Once I know what the messages are, extract them into helper methods on the test case class. Now I'm starting to do actual design. After each change, I run the tests. If I break the test, I go back and fix it before I proceed. This is Legitimate Refactoring. Feels good.
  • Once I know what the objects are, I go ahead and create their file in the production code, move the code out of the test and into the production code, and make sure the original test still passes. Now I switch back to a more typical test-driven design cycle; write a test case, watch it fail, make it pass, refactor, repeat.
This is sort of like the Saff Squeeze, but for exploratory coding rather than defect isolation. It kept me from spinning my wheels on design and organization before I even knew what the forces are that affect said design and organization. It also makes a new, strange, or messy code base less intimidating; I can incrementally integrate with the existing code as I'm ready to dive further into it.

Give this a try some time. It’s a great confidence booster.

Use what you got

How Shopify scales Rails was one of my favorite talks at Big Ruby. Therein, John Duff talks through what Shopify’s Rails stack looks like and why it works for them. What impressed me most was that Shopify has been running basically the same stack since they started. Ruby, Rails, MySQL, Memcached. They added Redis in for queuing at one point.

I’m so easily tempted and fascinated by The New Shiny, it’s refreshing to read about a shop getting by with The Old Trusted instead. There’s a lot to be said for using solid, well-known tools for as much as you can and working with the strengths and weaknesses those tools afford you. It’s a lot closer to craftsmanship than the temptation of science fiction New Shiny tools, and less likely to turn your guts inside out when something goes sideways.

When I complain instead of solve

Pet peeve 74: whenever I slip and focus on complaining about who and what instead of thinking about how and why to solve the problem. This is doubly frustrating because I enjoy solving puzzles (i.e. problems) more than I enjoy annoying people.

It's so, so easy to kvetch. It feels good. But, it's more productive to figure out why the problem happened and how to solve it (or break the problem down to solvable subproblems).

Being human: it's tricky.

My Rite of Spring overfloweth

Today’s the hundredth anniversary of premier of hometown favorite Igor Stravinsky’s Rite of Spring. The National Public Radios are all over this. NPR Music and WQXR are collecting a bunch of articles they’ve done in the run-up to the anniversary. My favorites: a visualization of the score and a list of essential recordings. The latter features this amazing image of Stravinsky, which I’ve already posted twice today and will continue posting it until it just doesn’t feel right anymore.

[caption id="" align=“aligncenter” width=“312”]Igor Stravinsky in a giant coat, a hat, and sunglasses Indeed, Igor Stravinsky will be on that suit and tie.[/caption]

If you're playing along today, WQXR is streaming twenty-four hours of nothing but the Rite. Highly recommended, if you have the means.

Entropy and anti-entropy on your codebase

Entropy and Evolution of a Codebase goes well with Your Application is on Fire:

If you imagine the modules in a codebase like cells in a Game of Life automaton, you could see cells fading from healthy blue to sickly red, then transmitting their disease elsewhere. Very occasionally, you’ll see a cell or cluster of cells brightening to health as a developer restructures that area. Mostly, the codebase will decay until it must be discarded or rewritten.
Michael Nygard reflects on changes that increase and decrease the entropy of an application, what tends to introduce those changes, arrives at this wonderful metaphor for the thinking about the health of an application over time, and then offers several ways to manage entropy in a codebase. This is one that will stick in your head for a while.

If you haven’t read Michael Nygard’s Release It, you’re missing out on a lot of great stories and useful ideas on how to maintain production software.