Refactor to modules, for great good
Got a class or model that’s getting a little too fat? Refactor to Modules. I’ve done this a few times lately, and I’ve always liked the results. Easier to test, easier to understand, smaller files. As long as you’ve got a tool like ctags to help you navigate between methods, there’s no indirection penalty either.
That said, I’ve seen code that is overmodule’d. But, that almost always goes along with odd callback structures that obscure the flow-of-control. As long as you stick to Ruby’s method lookup semantics, it’s smooth sailing.
ZeroMQ inproc implies one context
I’ve been tinkering with ZeroMQ a bit lately. Abstracting sockets like this is a great idea. However, the Ruby library, like sockets in general, is a bit light on guidance and the error messages aren’t of the form “Hey dumbie, you do it in this order!”
Here’s something that tripped me up today. ZeroMQ puts everything into a context. If you’re doing in-process communication (e.g. between two threads in Ruby 1.9), you need to share that context.
Doing it right:
# Create a context for all in-process communication
>> ctx = ZMQ::Context.new
# Set up a request socket (think of this as the client)
>> req = ctx.socket(ZMQ::REQ)
# Set up a reply socket (think of this as the server)
>> rep = ctx.socket(ZMQ::REP)
# Like a server, the reply socket binds
>> rep.bind('inproc://127.0.0.1')
# Like a client, the request socket connects
>> req.connect('inproc://127.0.0.1')
# ZeroMQ only knows about strings
>> req.send('1')
=> true
# Reply/server side got the message
>> p rep.recv
"1"
=> "1"
# Reply/server side sends response
>> rep.send("urf!")
=> true
# Request/client side got the response
>> req.recv
=> "urf!"
Doing it wrong:
# Create a second context
>> ctx2 = ZMQ::Context.new(2)
# Create another client
>> req2 = ctx2.socket(ZMQ::REQ)
# Attempt to connect to a reply socket, but it doesn't
# exist in this context
>> req2.connect('inproc://127.0.0.1')
RuntimeError: Connection refused
from (irb):16:in `connect'
from (irb):16
from /Users/adam/.rvm/rubies/ruby-1.9.2-p180/bin/irb:16:in `'
I believe what is happening here is that each ZMQ::Context gets a thread pool to manage message traffic. In the case of in-process messages, the threads only know about each other within the confines of a context.
And now you know, roughly speaking.
Booting your project, no longer a giant pain
So your app has a few dependencies. A database here, a queue there, maybe a cache. Running all that stuff before you start coding is a pain. Shutting it all down can prove even more tedious.
Out of nowhere, I find two solutions to this problem. takeup seems more streamlined; clone a script, write a YAML config. foreman is a gem that defines a Procfile format for defining your project’s dependencies. Both handle all the particulars of starting your app up, shutting it down, etc.
I haven’t tried either of these because, of course, they came out the same week I bite the bullet and write a shell script to automate it on my projects. But I’m very pleased that folks are scratching this itch and hope I’ll have no choice but to start using one when it reaches critical goodness.
Ruby's roots in AWK
AWK-ward Ruby. One man Unix wrecking squad Ryan Tomayko reflects on aspects of Ruby that arguably grew from AWK more than Perl. Great archaeology, but also a good gateway drug to understanding how awk is a useful tool. Only recently have I started to really grok awk, but it’s super handy for ad-hoc data munging in the shell.
Humankind's genius turned upon itself
When We Tested Nuclear Bombs. An absolutely fantastic collection of photos from the nuclear test program. Beautiful to look at, terrifying to contemplate the ramifications in context. It’s harrowing to think that one of science’s greatest achievements could undo so much of science’s achievement.
Burpess and other intense workouts
But when pressed, he suggested one of the foundations of old-fashioned calisthenics: the burpee, in which you drop to the ground, kick your feet out behind you, pull your feet back in and leap up as high as you can. “It builds muscles. It builds endurance.” He paused. “But it’s hard to imagine most people enjoying” an all-burpees program, “or sticking with it for long.”I'm having trouble deciding whether I should say good things about burpees. I only do a handful at a time, usually as part of a series of movements. They're not so bad if you start with just a few and work up from there.
Burpees aside, it’s interesting to see opinions on what the most useful exercise movements are. I’m really glad I don’t need to start doing butterflies though.
Don't complain, make things better
notes on “an empathetic plan”:
Worse is when the the people doing the complaining also make software or web sites or iPhone applications themselves. As visible leaders of the web, I think there are a lot of folks who could do a favor to younger, less experienced people by setting an example of critiquing to raise up rather than critiquing to tear down.Set agreement to maximum. If you’re complaining on Twitter just to make yourself feel better, keep in mind that some of us are keeping score.If you’re a well known web or app developer who complains a lot on Twitter about other people’s projects, I am very likely talking about you. You and I both know that there are many reasons why something works a certain way or why something in the backend would affect the way something works on the front-end.
Don’t waste your time griping and bringing other people down. Spend your time making better things.
Perfection isn't sustainable
When an interesting person is momentarily not-interesting, I wait patiently. When a perfect organization, the boring one that's constantly using its policies to dumb things down, is imperfect, I get annoyed. Because perfect has to be perfect all the time.More and more, I think perfection is the biggest enemy of those who want to ship awesome things. Iteration can lead to moments of perfection, but perfection is not sustainable over time.
Using Conway's Law for the power of good
Michael Feathers isn’t so quick to place negative connotations on Conway’s Law. Perhaps it’s not so much that organizations don’t communicate well, the traditional reading of Conway’s Law. Maybe as organizations grow, people tend to only communicate frequently with a few people and those interactions end up defining the API layers.
I’ve been thinking about this a bit lately. It’s possible there’s something to be said about using Conway’s Law to your advantage when building service-based shearing layers. Some parts of your application should evolve quickly, others require more stability. Some iterate based on user and conversion testing, others iterate as TDD or BDD projects. You can discover these layers by observing team interactions and using Conway’s Law to define where the APIs belong.
Hell is other people's concurrency
The first rule of evented programming is, don’t block the event loop! Mathias Meyer’s great intro to Ruby’s EventMachine library. Non-blocking IO is so hot right now. But remember, it’s just a tool on your concurrency utili-belt. Remember to reach for coroutines, threads, actors, and STMs too.
Bloom, a language with time travel
Bloom, a language for disordered (whut!) distributed programming with powerful consistency analysis and concise, familiar syntax (the prototype is built on Ruby):
Traditional languages like Java and C are based on the von Neumann model, where a program counter steps through individual instructions in order. Distributed systems don’t work like that. Much of the pain in traditional distributed programming comes from this mismatch: programmers are expected to bridge from an ordered programming model into a disordered reality that executes their code. Bloom was designed to match–and exploit–the disorderly reality of distributed systems. Bloom programmers write programs made up of unordered collections of statements, and are given constructs to impose order when needed.Interested to see how languages will push the assumption that time proceeds from earlier to later as one reads down a source file.
The rules of the yak shave
Yak shaves. They’re great fun. Like most things, yak shaving is more fun when you have some rules to guide you away from the un-fun parts:
- always have a goal, know when you’re done
- timebox it
- work on a branch so you can switch to real work if you need to
- make smaller commits than usual so you can unwind if you should go awry
- don’t worry about writing tests if you don’t know what you’re doing
- if you aren’t sure where you are going, write a test harness and iterate on that
- have a pair or buddy to talk through what you’re trying to do and how to get there
- bail out if you are starting to burn out, face diminishing returns, or think of a better way to shave they yak
Linux screenshot nostalgia
Anyone else remember uploading screenshots of their super awesome, tweaked out Linux hacker desktops?
The joy of logs
But a better conceptual model is to treat logs as time-ordered streams: there is no beginning or end, but rather an ongoing, collated collection of events which we may wish to view in realtime as they happen (e.g. via tail -f or heroku logs --tail) or which we may wish to search in some time window (e.g. via grep or Splunk).Work on an app with a couple dozen servers, a handful of databases, and several moving parts and you start to realize that logs are one of your best friends. They're useful for troubleshooting, performance monitoring, and just knowing how your application works in reality, under real traffic.
I’ve tinkered with building deeper APIs for logging within applications and services, but I think Adam Wiggins is on the right path here (not the first time either). Logging should be as simple as possible in applications. All the smarts for aggregating, searching, and extracting interesting information should happen after the data is collected. Using standard out instead of files is a fantastic idea too.
I'm Corgi-internet famous
OCD: Obsessive Corgi Disorder, Kitty putting the moves on my man. Also on Men and their dogs. I’m corgi-internet famous! Photo by Courtney.
Noel Rappin's Advice on TDD
Testing Advice in Eleven Steps. My favorite:
At any given moment, the next test has some chance of costing you time in the short term. The problem is it’s nearly impossible to tell which tests will cost the time. Play the odds, write the test. Over the long haul, the chance that the tests are really the bottleneck are, in my experience, quite small.
All eleven are pretty handy for those like myself who still feel like they have a lot to learn about TDD, BDD, et. al.
Organizing and decoding problems
My favorite sort of problem involves the interactions between individuals, groups of people, and mechanical rules generated by individuals and groups. Speculating on how the rules were shaped by the experiences of the individuals and groups is a fun game to play when presented with a curious set of circumstances. Conversely, my least favorite problems are those generated by the institutional brain damage of certain kinds of groups. Software and systems shaped by regulations and the particulars of monetary exchange are tedious at best.
I find it quite amusing when answers aren’t “clean” and technological systems aren’t the best solution. Often, no amount of analysis and brilliant coding can make an improvement. What is needed is to understand what people do, why they do it, and persuade them to do otherwise.
This brings me to economics, specifically behavioral economics. If you take away all the math, economics is largely about how people behave in aggregate. It’s a useful tool in understanding how to work with systems that involve people. But there’s more to economics than explaining how people interact in markets.
Russ Roberts, in the process of explaining how economics is not useful as a mechanism for answering questions like “did the stimulus work?” or “when will the housing market recover?”, gets to what really interests me about economics and finance:
"Is economics a science because it is like Darwinian biology? Darwinian biology is very different from the physical sciences. Like economics it is a very useful way to organize your thinking about complex phenomena. But it is not a predictive or very precise science or whatever you want to call it."
The crux of the biscuit, for me, is right in the middle. Economics is a useful way to organize an intricate and interconnected problem and figure out how to reason about it. In casually studying economics and finance over the couple years, I’ve come to form a better mental model about how the world works.
Its possible that this mental modeling is what I do when I’m coding. I’m poking how some code works, looking at its inner workings, trying to understand how it interacts with the code around it. I sift through revision logs to see how it got to where it is today, talk to others on the team about the code and why it ended up one way and not another. I’m organizing and modeling the code in my head, building up a model that describes .
Lately, I’ve been describing my work as tinkering with lines of code until numbers appear on the screen in the right order. This is invariably greeted with a comforting-to-me response like “I could never do that”. But I really enjoy this. I’m not just debugging my code; I’m sharpening the way the program is organized in my head. Once I’ve got my head around it, I decode that organization into words, code, drawings, etc.
With programs, there is some system of rules, forces, and interactions which describe how the code works or doesn’t. Economics, too, describes a system of rules, forces, and interactions that predicts how a puzzle of human beings operate. Organizing and decoding these technical and social puzzles is great fun.