Adam Keys is Thinking

Polyglot persistence:

I think that many of the NoSQL crowd either fail to either recognize, or to properly describe that their preferred databases don’t replace applications like MySQL and Postgres, just as Ruby doesn’t replace Java. Instead, the explosion of these new options for persistence just work better for some domains (and worse for others).

Ben Scofield’s spot-on here. One of the many transitions we’re undergoing is from “I’ll use MySQL most of my career” to “I’ll tinker with a different database for every project over a couple years and then pick and choose the best as time goes on”.

A programming language zoo, a week of FP heaven, rewriting PHP with Haskell and a game for kids of any age to learn the untyped lambda calculus. Did I ever mention it’s a good time to have a fascination with functional programming languages?

(With due apologies to the creators of New Math, the writers of 30 Rock and the lovely iPod people.)

Why I do Time Tracking

At one point, I tracked every context switch during my work day. I kept a legal pad next to my mouse and I would write down the time whenever I switched between coding, emailing, surfing, talking to someone in my cube or walking around to find someone I needed to talk to.

People thought I was odd. They’d ask what I was jotting down when I noted my context switch, so I briefly explained it. “Okaaaayyyyyy?”, they’d say. An unintended side-effect was that people did end up bugging me about random things less. In fact, it was way more effective at that than putting up a sign that said “unless something is on fire, email me”. So it goes.

I never ended up running any aggregate numbers on the data. But, it did give me an idea about the frequency and cost of my context switches. I wish there was a better way to track this sort of thing passively. I’m sure it would still shock me how much time I go in and out of focus.

If you’ve never tried this sort of experiment, it’s worth running for a week or two. You’ll almost certainly notice something worth trying to cut out by glancing over the list of things you’re spending time or wasting focus on.

Fred in depth.

Still crazy after all these years:

Politicians should tone down the rhetoric. Protesters should read some history before making Hitler comparisons. Talk-show hosts should stop pretending that paranoid nitwits are asking reasonable questions.

The Economist does well to explain the insanity that is propagated by American political media. Reading articles like this help me stay sane. Also: ignoring media with deadlines shorter than a week, and consuming as much constructive satire as possible.

Almost five years ago, we were witness to the reinvention of web frameworks. A couple upstarts named Django and Rails appeared at almost the same time, espousing many of the same values. In the typical Gandhi-cycle, they were first ignored, the incumbents fought them, and then they achieved victory over the incumbents. Today, any framework that’s used on new projects is likely to have more than a hint of Django and Rails in it.

Today, we’re seeing the same thing for databases. Something is going on and things are changing. Barring an unprecedented departure of fashion from software development, I’ll look back five years from now and write about some other shift in the development landscape.

But, all shifts like this need a name. Right now, the best we have is NoSQL. The problem with that name is that it only defines what it is not. That makes it confrontational and not amazingly particular to what it includes or excludes.

Damien Katz, the creator of CouchDB, has noted the need for a better name for this storage revolution. Allow me to proffer mine:

Post-relational

What we’re seeing its the end of the assumption that valuable data should go in some kind of relational database. The end of the assumption that SQL and ACID are the only tools for solving our problems. The end of the viability of master/slave scaling. The end of weaving the relational model through our application code.

We’re seeing an explosion in the ideas about how one should store important data. We’re looking at data to see if it’s even worth persisting. We’re experimenting with new semantics around structure, consistency and concurrency.

In the same way that post-modernism is about reconsidering the ways of the past in art and architecture, post-relational is a chance for software developers to reconsider our own ways. Just as post-modernism didn’t invalidate the entire history of art, post-relational won’t invalidate the usefulness of relational databases.

However, it’s likely that those working in some domains will decide that non-relational databases better fit their needs. That’s progress in our field. It’s fun to watch.

Say it with me: post-relational databases, post-relational storage, post-relational thinking, or simply, post-relational.

Remember when you first started programming? Those early days when you'd take some code out of a book or article, type it out, and then try to make it print silly things or draw funny pictures?

The thing I remember about those days was the temptation to blame the compiler for all the ills of my code. Something doesn't work right and you can't figure out why? Blame the compiler! Of course, this was never right. The compiler is very rarely incorrect.

My goal today is to bring back the joy of blaming the compiler. Of course, Ruby doesn't _have_ a compiler (yet!), so we have to play tricks. But that's part of the fun!

Let's blame the compiler.

See, all you have to do is extend BlameTheCompiler. And then you'll find yourself with one chance in every three executions to say "Hmm, I'm sure I defined that method. Something must be wrong with Ruby." Just like those early days of programming, you're half right. The other half is that you've got a little prankster running in your application.

It's a fun parlour trick. I would not, however, recommend sending code including BlameTheCompiler in as a bug report to the JRuby or Rubinius folks. They wouldn't find it as funny as you or I.

People with two cellphones worry me.

(More six word stories. Also, an article as such in Wired.)

I promised you the details on how we built Tag Better, so here we go. This is what I used to build the back-end bits. You’ll have to pester Chris or Alex to get the front-end details.

h2. Sinatra

Technically, it’s a Rails Rumble. Read between the lines of the rules and you’ll see it’s really a Rack Rumble. And so I went with my favorite for prototyping, Sinatra.

Happily, Sinatra had my back the whole time. I never came across anything that stumped me. Further, I didn’t pay any taxes for ceremony I don’t need.

Verdict: perfect tool for the job.

h2. Passenger

I hadn’t used Passenger much before this weekend. I’m pretty happy spooling up app processes in a terminal and watching the logs scroll past. localhost:3000 is my friend.

However, I’m an outlier in this regard. My teammates aren’t as interested in lower-level bits as I am, so I figured that using Passenger is the best bet to help them get the app up and running locally.

The benefit that I didn’t realize we’d get from this is running the same stack locally as on the production server. Besides some virtual host wrangling that Passenger Pane saved me from locally, getting the app up and running was painless.

Verdict: I am quite likely to keep tolerate Apache for that Passenger goodness, especially when I am the operations guy.

h2. Sprinkle + passenger_stack

The moment that I realized we’d have to set up our own server instance was one of brief, abject terror. I knew this could easily expand to fill a lot more of my time than I wanted. Luckily, I was wrong.

Ben Schwartz’s passenger_stack helped me get our Linode slice up far faster than I would have been able to by hand. I cloned his repo, tweaked it to our needs (disabled MySQL, eventually added a CouchDB recipe) and ran it on our server. Several minutes later, we had a working server. Pretty awesome.

passenger_stack uses Sprinkle, which isn’t getting as much play in the server configuration space as Puppet and Chef. Sprinkle does seem really well suited to standing up apps on a few servers. We might want to step up to something heftier once we had more servers, but Sprinkle and passenger_stack are simple to understand and don’t require any supporting infrastructure to use.

Verdict: Not too primitive, not too involved; just right.

h2. CouchDB

When I’m building any app that relies on an API as its primary data source, caching API response data is forefront on my mind. Serving the data locally, rather than making a request every time, means the app feels more responsive. An added benefit is not upsetting the upstream data provider.

I’ve built apps like this that use MySQL as a cache and it just never felt right. I’ve been tinkering with CouchDB and Tokyo Cabinet/Tyrant lately. I decided to go with CouchDB for this one because of the excellent CouchDBX, which makes it easier for those who don’t even know what Erlang is to use CouchDB.

CouchDB ended up working pretty well. While we haven’t really leaned into it, it didn’t present any challenges while I was developing. Using CouchRest with Sinatra worked just fine.

Verdict: It just worked, which is exactly what I needed.

h2. Skipping traditional TDD

OK, so maybe only Jared Diamond would consider this a technology. But skipping the writing of tests to drive my design was pretty helpful. Consider Kent Beck’s flight metaphor. Doing a Rails Rumble is just like the taxi-ing phase. Or a minimum valuable product. Either way, you want to make a small investment towards validating an idea.

Notice I said traditional TDD. To tell the truth, I did write a sniff test script after I had the basic app working. But it wasn’t an xUnit-style test. It’s just a shell script that bangs on the app with fixed parameters. I do have to manually inspect it to make sure nothing is blowing up. What I’m really automating here is the pain of typing out Curl commands.

Verdict: worked great for the original purposes, but I’ll probably add a proper test suite as one of the first post-contest enhancements

So that’s what I think helped make our project go off pretty well. Really, what they did was help me get stuff done and then get out of the way. Isn’t that the best kind of tool?

Yesterday and today, I worked with Alex Bischoff and Chris Griego on a Rails Rumble project. In less than forty-eight hours, we set out to build a web application, using Ruby, from the ground up. While we didn’t boil the ocean, we did come up with something interesting.

Tag Better is the groundwork for an app that enhances one of our favorite, but often neglected, sites: del.icio.us. Personally, I’ve tried other bookmarking tools such as a plain text file, EagleFiler and pinboard.in (which I think has an excellent chance of eclipsing Delicious). However, I’ve found myself back on Delicious, even after a hiatus.

Given Yahoo’s unsteady stewardship of the site, we figured it’s time to take matters in our own hands. Using their API, it’s entirely possible for us to build enhancements on top of Delicious. We decided to do just that.

h2. Triage and tidying

We started out from a gem of an idea that Chris had. When you’re trying to better organize your tags and bundles in Delicious, there are two modes you might want to operate under. The first couple times, you need to do triage. Get stuff roughly into order, and start creating a filing system. Once you’ve got everything under control, you switch to tidying. Now you just need to keep things clean or maybe switch things around that aren’t working well.

We tackled triage first. In specific, we wanted a way to filter down our list of tags, select a bundle, and add some of the filtered tags to that bundle.

h2. Try it out

To get started with Tag Better, you’ll first need to log in with your Delicious username and password (more on that in a moment.)

Once you’re logged in, you’ll see your tag bundles on the left and all your tags on the right. If you click a bundle, the tags that are included in that bundle are highlighted. Clicking a tag toggles its inclusion in the bundle. Changes in Tag Better are immediately reflected in Delicious.

That’s the core interaction. You can also logout, of course. And, if you feel the need, we offer a way to remove your bundle and tag data from our system’s caches.

We were just four short characters away from getting live filtering of your tags via the search box on the top right. Alex and I even got it working, but keyboard cat had already played us out at that point. I promise we’ll add it as soon as judging is over!

A couple other caveats: you might want to create a test bundle (in the Delicious UI) to play with, rather than one you’ve already organized. I don’t think there are problems in the code, but it never hurt to show caution. Also, IE support is probably “interesting”, at this point.

h2. Trust

Even though we need your Delicious username and password to edit bundles on your behalf, we have made sure to never store authentication information on our side. Your authentication data is stored in a cookie on your machine; if you don’t like that, you should probably skip out on using Tag Better as a hosted app. If you delete that cookie, we won’t operate on your data. Further, we’ve added the ability for you to remove your bundle and tag data from our system. That’s the only data that we save on our disks.

I mentioned trust and your authentication data earlier. It’s an important part of an application like this. We don’t want to screw your bookmarks up, and you don’t want us to do skeezy things with your bookmarks. Unfortunately, Delicious does not yet offer a delegated authentication API like Flickr and Twitter do. So, we’ve got two options.

On the one hand, you could trust us. This is up to you. I’d like to think that if you’ve met Alex, Chris or I, then you’re happy to use the application knowing we’ve got more amusing things to do than soil your bookmarks. Further, when the competition is over, we’ll open the source repository up to the public. If you’re a code-review sort of person, go nuts.

On the other hand, you could run it yourself. Again, once the competition is over, we’re going to release the code. Look over the readme, install the dependencies, and run it on your own hardware. Sleep safely at night knowing your password is safely ensconced on your personal machine.

Tomorrow I’ll talk a bit about the technology we used to make Tag Better. Until then, I hope you’ll give it a try. If you have ideas for feedback, leave a comment here!

Chris Wanstrath:

Now we can all stop obsessing about who we think he was and instead focus on who he actually was – a prolific and inspiring hacker.

I met Why The Lucky Stiff for the first time at SXSW 2006. By some odd cosmic occurrence, he was performing during the Interactive schedule. I’m not sure the programming committee realized it would be a performance and not a panel. But there they were. All new material. Attempted audience interaction, subterfuged by the shoddy conference WiFi[1]. It was great.

After The Thirsty Cups performed, I introduced myself. “Hi, I’m Adam, glad to meet you”, etc. Then I came with my wildcard. Until I’d met him and shook his hand, I’d harbored a theory that Why was really an A-List celebrity, probably one from Ocean’s 11, that was using an alter-ego identity to share his love of Ruby. He suggested that I was, in fact, Brad Pitt. Which may be true. But I like to think that he enjoyed my theory.

One of my life-long pleasures has been making people laugh. Along those lines, a high watermark in that endeavor was when Why posted my zany Star Trek/Ruby sketch to Red Handed. I felt I had arrived; he praised my Picard impersonation!

Life is good.

I’d somewhat forgotten this, but a lack of seriousness, in the “I write weighty code, you should take me seriously” sort of way, is my best guess as to what Why’s schtick is. Lots of people have made a big deal about how unprofessional it is for him to just yank his code off the net, or about how his code was messy, stuff like this. They’re missing the point. I think Why just wanted to have fun, with code, and spread the fun, with code.

To an extent we, the community of developers who care about code and software, have forgotten how to have fun with what we do. Personally, I’m going to get right back on that.

More: Rick Olson sent me a picture from SXSW 2006, and Damon Clinkscales sent along some videos he took of the performance. Thanks, guys!

fn1. Even though SXSW was less than a thousand geeks at this time.

Cannonball Adderley:

It's called "Fun". F-U-N, fun. That's something you can do, when everything is mellow.

Here’s to mellow times. Seemed appropriate for a Friday.

The trick about good art is that it has some mystery, an unknown. The problem is that if you get too close to the art, you risk unraveling the mystery. If you deconstruct it, engage it, or study it, the unknown becomes known. Thus, if I really enjoy a song (in particular), movie, etc. I stay away from taking it apart to see how it works. I’d rather enjoy it for a long time.

I have this problem where I over-listen to an album. It started in my teenage years. I learned all the bass-lines for Pearl Jam’s Ten. After that, I couldn’t listen to the album for ten years; I knew all the secrets, all the interesting bits. Rewind a year ago, and The Who’s Live At Leeds was my jam. Now, I can’t listen to it.

But I’ve been very rigorous about listening to Bruce Springteen’s Born To Run. It is such a perfect piece that I only allow myself to listen to it once a month[1]. No more. Similarly, I won’t let myself learn to play any of the songs on the guitar. I want to maintain that mystery.

I wonder if there is other art like this. Could you get overexposed to a Mondrian painting or a Hemingway book? Even with works that are more popular in their sensibilities, is it possible? Is there such a thing as too much Starry Night or Ghostbusters?[2]

The bottom line: enjoy good art, but take care not to over-enjoy it.

fn1. I even feel like I’m cheating if I listen to Born To Run in anything but album-form. To hear “Thunder Road” or “Jungleland” by itself feels incomplete, like I’m missing something.

fn2. Yes, I just put these on the same level, even though I’m not much of a van Gogh aficionado.

Two unrelated and great songs, one title.

“Testify” by Parliament

“Testify” by Stevie Ray Vaughn

Enjoy!

Tips for working alone:

The other thing I’m doing is bringing back my practice of writing “daily pages”… ~750 words a day to myself, that sort of help me get everything out in the open in my head and so that I can find a bit of clarity in my typically scrambled thoughts.

I do this too. I find it’s pretty handy for focusing my thoughts and bringing ideas out.

Some other things I’ve done working from home the past year or so:

• Put as many distractions as possible in another room. Get the TV, Xbox, etc. out of your workspace.
• Keep it clean. Nothing is more distracting, to me, than mess and clutter.
• Take a nap! I’ve become a regular nap taker lately, right after lunch; it’s noticeable the improvement it brings.
• Use Twitter, IRC, Campfire, IM, etc. as your watercooler. Standard disclaimers apply - make sure your work/socializing blend is right.
• Get out every once in a while. I try to get out to socialize with other geeks or get lunch with my wife at least twice a week.

And remember, working from home isn’t for everyone. If it’s not for you, look into your local coworking space. (Dallas folks: give Cohabitat a try, it’s great.)

Logan Hicks has some really great subterranean photography going on. (Via Infrastructurist)

And if you like that, you’d probably also like some Russian submarines, their interiors, and their underground bases.

While we’re dwelling on intriguing/depressing concrete structures: one dude’s home built into a former missile silo and the Oak Ridge plant where the materials for the Manhattan Project were refined.

The Book Stalker - Rands figures you out by your bookshelf:

Where’s your bookshelf? It’s this awkward moment whenever I first walk into your home. Where is it? Everyone has one. It might not be huge. It might be hidden in a closet, but in decades of meeting new people, I’ve never failed in finding one and when I do I consume it.

Here’s mine from almost two years ago (plus more):

I’ve since expanded to two shelves and look forward to the day when I can devote a whole wall to just reading. As I often tell myself as I sit down to start, “reading is the best”.

Free Parking Isn’t Free. Turns out those parking lots, while sometimes handy, are actually pretty gnarly, if your goal is to build a nice place to live:

Throughout the 1940s and 50s, as automobile use became prolific in the United States, parking became a problem, congesting streets and overflowing into neighbors' lots. In response, most municipalities instituted off-street parking minimums requiring developers to provide all the parking that the residences or shops would need on-site. This seemingly sensible notion has created a cascade of problems. It encourages sprawl by spreading buildings apart to make room for more parking (requirements usually demand more area for parking than the building it supports). It also weakens urban design, as urban buildings are torn down to make room for desolate surface lots, and hulking parking garages sprouted in downtown areas. It discourages revitalization of existing historic buildings, since developers have trouble meeting modern parking requirements in neighborhoods that were built before auto dominance. And the requirements drive up the cost of development: parking spaces can cost between $10,000 and $50,000 – typically more than the cost of the car that occupies it. High parking requirements can raise the price of homes and apartments by $50,000 to $100,000, a serious challenge to affordability.

When I have more money that I know what to do with, I’m going to start buying up parking lots and turning them into parks. It’ll be my little way of sticking it to people who drive over-large cars.

The Menubar Challenge - everybody, clear out your menubars! It’s one of my secret productivity weapons, I highly recommend it. Also, read everything on Minimal Mac; it’s the best.

Here’s my current attempt to use as little as possible in my menubar:

LittleSnapper normally isn’t running, so that doesn’t count. If I could, I would run DropBox and FastScripts without menubar icons. I’m still not sure I like having a clock visible at all times, but at least analog clocks are classy-lookin'. I’d love to remove the battery icon, but it appears doing so disables the “your battery is tapped” warnings, leading to spontaneous laptop sleeping.