Err the Blog, revisited

Before there was GitHub, there was Err the Blog. Chris Wanstrath and PJ Hyett wrote one of the essential Rails blogs of early Rails era. Therein, many of the idioms and ideas we use to build Rails apps today were documented or born.

I’d figured this site was offline, as are most things from the mid-2000s. ‘Lo and behold, it’s still online in its pink-and-black glory. Lots of nostalgia on my part here.


One step closer to a good pipeline operator for Ruby

I’ve previously yearned for something like Elm and Elixir’s |> operator in Ruby. Turns out, this clever bit of concision is in Ruby 2.5:

object.yield_self {|x| block } → an_object
# Yields self to the block and returns the result of the block.

class Object
  def yield_self
    yield(self)
  end
end

I would prefer then or even | to the verbosely literal yield_self, but I’ll take anything. Surprisingly, both of my options are legal method names!

class Object

  def then
    yield self
  end

  def |
    yield self
  end

end

require "pathname"

__FILE__.
 then { |s| Pathname.new(s) }.
 yield_self { |p| p.read }.
 | { |source| source.each_line }.
 select { |line| line.match /^\W*def ([\S]*)/ }.
 map { |defn| p defn }

However, | already has 20+ implementations, either of the mathematical logical-OR variety or of the shell piping variety. Given the latter, maybe there’s a chance!

Next, all we need is:

  • a syntax to curry a method by name (which is in the works!)
  • a syntax to partially apply said curry

If those two things make their way into Ruby, I can move on to my next pet feature request: a module/non-global namespace scheme ala Python, ES6, Elixir, etc. A guy can dream!


Simple Ruby pleasures

I think I first discovered the joy of take and drop in my journeys through Haskell. But it appears that, since 2008 at least, we have had the pleasure of using them in Ruby too.

Need the first or last N elements from an Enumerable. Easy!

[sourcecode language=“ruby” light=“true”] ary = (1..100).to_a ary.take(5) # => [1, 2, 3, 4, 5] ary.drop(95) # => [96, 97, 98, 99, 100]

range = (1..100) range.take(5) # => [1, 2, 3, 4, 5] range.drop(95) # => [96, 97, 98, 99, 100]

hsh = {:foo => 1, :bar => 2, :baz => 3} hsh.take(1) # => [[:bar, 2]] hsh.drop(2) # => [[:foo, 1]] [/sourcecode]

The real magic is when you use take along with other Enumerable goodies like select and map. Here’s one of my personal favorites amongst the code I wrote in 2010:

[sourcecode language=“ruby” light=“true” highlight=“12,13,14”] class QueryTracer < ActiveSupport::LogSubscriber

ACCEPT = %r{^(app|config|lib)}.freeze FRAMES = 5 THRESHOLD = 300 # In ms

def sql(event) return unless event.duration > THRESHOLD callers = Rails. backtrace_cleaner. clean(caller). select { |f| f =~ ACCEPT }. take(FRAMES). map { |f| f.split(":").take(2).join(":") }. join(" | ")

# Shamelessly stolen from ActiveRecord::LogSubscriber
warning = color(&quot;SLOW QUERY&quot;, RED, true)
name = '%s (%.1fms)' % [event.payload[:name], event.duration]
sql  = event.payload[:sql].squeeze(' ')

warn &quot;  #{warning}&quot;
warn &quot;    #{name} #{sql}&quot;
warn &quot;    Trace: #{callers}&quot;

end

end

QueryTracer.attach_to :active_record [/sourcecode]

This little ditty is awesome because:

  • It's super-practical. Drop this in your Rails 3 app, tail your production log, see the slow queries, go to the method in your app calling it, and fix it. Easy.
  • It only activates itself when it's needed. Queries that execute quickly return immediately.
  • No framework spelunking required. Rails 3's notification system handles all of it. Rails' backtace cleaner gizmo even makes the backtraces much nicer to read.
  • It chains methods to make something that reads like a nice, concise functional program.
For more Enumerable joy, read up on each_cons.

A language experiment writ large

For the past year, the Java ecosystem has seen interesting evolution. Java the language continues take its place as the new safety scissors of programming, but the pieces around it are getting better. The JVM is now acknowledged inside and outside of the Java community as really good stuff. Really interesting software like Hadoop and Cassandra are built on top of Java. Integration with languages like Ruby and Python is getting pretty good.

What’s most interesting to me is that there’s a competition going on for the hearts and minds of those developers who don’t like using safety scissors. This competition is a great experiment into what developers really want in a programming language. For a language nerd such as myself, observing this experiment is a lot of fun.

On one side you’ve got Scala. Scala looks a lot like Java. But on top of that it adds shorthands and pleasantries from Ruby, a really good type system reminiscent of Haskell, and other handy functional features. When you build up a hybrid language like this you, two things happen. First, a lot of people who look at their checklist, find everything they need and decide. Second, you get a pretty complex language.

Clojure, however, looks nothing like Java. It’s a Lisp, it simply can’t. Clojure borrows from Haskell too, this time borrowing ideas about state and how to avoid it and concurrency (notably software transactional memory). Clojure is a funny looking language at first, but there are some great ideas within it. Plus, it’s a relatively small language; it’s just that it’s a different kind of simple and almost every concept is new to many developers.

Both these languages are building up strong communities. Both are full of great people with energy and ideas. It’s quite possible that a winner-take-all situation won’t occur. I’d like that.

What’s most interesting to me is to see how people take to the languages. Will they go for the familiarity of Scala and deal with the complexity? Will they learn the simplicities of Clojure and rewire their brains? Will they prove the common wisdom wrong and learn both?

I’m watching with great interest.


Bundler, not as bad as they say

Of all the new moving parts in Rails 3, the one I see the most grousing over is Bundler. This is not surprising, as its a big part of how your application works and it's right up front in the process of porting or building Rails 3 apps.Bundler: As Simple as What You Did Before:

Bundler has a lot of advanced features, and it’s definitely possible to model fairly complex workflows. However, we designed the simple case to be extremely simple, and to usually be even less work than what you did before. The problem often comes when trying to handle a slightly off-the-path problem, and using a much more complex solution than you need to. This can make everything much more complicated than it needs to be.

I haven't run into anything with Bundler that I couldn't solve with a little critical thinking and maybe a little searching. On the other hand, Bundler has made getting dependencies straight amongst team members and deploying them to production servers far easier than it was before. I'm very glad that while it's not strictly part of the scope of Rails, that Bundler is now part of it.


Using Rails 3.0&#039;s notification system

How to use Rails 3.0's new notification system to inject custom log events. Ever wondered what the notification/subscription stuff in Rails 3 is? Wonder no more! I just used this to add performance logging around some Cassandra stuff in our Rails 3 app. Once you get the hang of it, this is really rad stuff.


An ode to Hashie

I was building an API wrapper this weekend. As is common when writing these sorts of things, I found myself needing something that takes semi-structured data (hashes parsed from JSON) and yields Ruby objects that are easy to work with. I've always found myself hacking these sorts of things together on a somewhat ad-hoc basis. It's a fun, but a bit of a yak-shave.

This time around, I decided to see if the state of the art has advanced in this realm. Luckily, I reviewed Wynn Netherland's slides from Lone Star Ruby Conference and found exactly what I needed.

Where have you been all my life?

Intridea's Hashie is a library built on the notion of making hash-like data structures act a little more like objects and a little easier to work with. I have literally wanted something like this for years!

Suppose you have a hash like the following:

hash = {
  "name" => "Adam",
  "age" => 31,
  "url" => "http://therealadam.com"
}

Coding up an object to store that isn't too hard, but writing the code that pulls values out of the Hash and tucks them away in the right attribute on the object gets tedious quickly. Hashie's Dash class makes this trivial.

class User >Hashie::Dashie
  property :name
  property :age
  property :url
end

Its even more delightful to use:

user = User.new(hash)
user.name # => "Adam"

Tons of boilerplate code, eliminated. My life is instantly better.

A great use of inheritance

It's been pointed out that ActiveRecord's use of inheritance is somewhat specious. To argue that "user is-a ActiveRecord::Base" takes a bit of hand-waving. So lately, you'll find lots of libraries insinuate themselves into classes as a mixin, rather than as a parent class. This is a little bit of you-say-potato-I-say-potato, but whatever.

In Hashie's case, I think that inheritance is being used correctly. All of the classes that Hashie provides (Mash, Dash, Trash and Clash) inherit from Hash. So the is-a relationship holds.

Sugary data structures taste great

While I'm going on about inheritance, here's how I used to create these sorts of wrapper classes:

User = Struct.new(:name, :age, :url)

For creating simple objects that just need to hold onto some data, I really like this approach. If they end up needing data, it can easily grow up:

class User < Struct.new(:name, :age, :url)
  # Behavior goes here
end

I like what Hashie is doing even more though. Its enhancing a core class in a largely unobtrusive way, and doing so from the confines of a library that only those who need it can pull from.

I'd love to see more libraries like this that add extra sass to Ruby core library. An Array that pages values out to disk on an LRU-basis perhaps, or a bloom-filter based Set, perhaps?

I'm excited about languages like Erlang, Haskell, Scala, and Clojure and what they can bring to the adventurous developer. Despite that, I feel strongly that Ruby still has plenty of really nifty tricks up its sleeve.


Examining software principles

There are too many good things to say about the Design Principles Behind Smalltalk. A few of my favorites:

Scope: The design of a language for using computers must deal with internal models, external media, and the interaction between these in both the human and the computer.

This one is really obvious until you get to the last four words. The human and the computer. Luckily we're starting to take for granted the primacy of human communication in programming lately (mostly), but when Smalltalk was created, I'm sure its designers received no shortage of grief when they steered towards humane optimizations.

Uniform metaphor: A language should be designed around a powerful metaphor that can be uniformly applied in all areas.

Smalltalk is largely objects and messages. Lisp is largely lists and functions. Erlang is largely pattern matching, functions, and actors. These aren't perfect languages, but once you deeply understand, really grasp the core concepts, you have the whole language at your command.

Operating System: An operating system is a collection of things that don't fit into a language. There shouldn't be one.

The first sentence is a great principle when considering what should go in the core of a system and what should go in the surrounding ecosystem of libraries. The second sentence is wonderfully bold, in that it cuts against what nearly every successful system has done since Smalltalk was prominent and in that it contradicts the first sentence. I'm not sure what practical use to make of this principle; its density of intrigue is that keeps me coming back to it.

Natural Selection: Languages and systems that are of sound design will persist, to be supplanted only by better ones.

I stopped worrying about what might supplant Ruby a long time ago. Someday, it will happen. And when it does, whatever succeeds Ruby will have to be really awesome to fill its shoes. I'm looking forward to seeing what that is. But the same goes for any technology; they are often replaced with something wholly awesomer than the incumbent.


I've never done it, but it seems like it would be intriguing and vastly informative to sit down with one of the systems I work on daily and try to extract these principles post-hoc. What values and principles are embedded in the system? What does that say about the team and why the system is the way it is? What principles are enablers and what bad habits should the team work to correct?


A quick RVM rundown

(It so happens I’m presenting this at Dallas.rb tonight. Hopefully it can also be useful to those out in internetland too.)

RVM gives you three things:

  • an easy way to use multiple versions of multiple Ruby VMs
  • the ability to manage multiple indpendent sets of gems
  • more sanity

First, let's install RVM:

  • gem install rvm
  • rvm-install
  • follow the directions to integrate with your shell of choice

Now, let's install some Rubies:

  • rvm list known will show us all the released Rubies that we can install (more on list)
  • rvm list rubies will show which Rubies we have locally installed
  • rvm install ree-1.8.7 gives me the latest release of the 1.8.7 branch of Ruby Enterprise Edition
  • rvm install jruby will give me the default release for JRuby
  • rvm use jruby will switch to JRuby
  • rvm use ree will give me Ruby Enterprise Edition
  • rvm use ruby-1.8.6 will give me an old and familiar friend
  • rvm use system will put me back wherever my operating system left me

The other trick that RVM gives us is the ability to switch between different sets of installed gems:

  • Each Ruby VM (JRuby, Ruby 1.9, Ruby 1.8, REE) has its own set of gems. This is a fact of life, due to differing APIs and, you know, underlying languages.
  • rvm use ruby-1.9.1 gives you the default Ruby 1.9 gemset
  • rvm use ruby-1.9.1%acme gives you the gemset for your work with Acme Corp (more on using gemsets)
  • rvm use ruby-1.9.1%wayne gives you the gemset for your work with Wayne Enterprises
  • rvm use ree%awesome gives you the gemset for your awesome app
  • You can export and import gemsets. This can come in handy to bring new people onboard. No longer will they have to sheepishly install gems on their first day as they work through dependencies you long since forgot about.

Some other handy things to peruse:

I also promised you some extra sanity:

  • RVM knows how to compile things, put Rubygems and rake in place, even apply patches and pull from specific tags. You can do more important things, like watch The View or read an eleven part series on pre-draft analysis for the Cowboys.
  • RVM lets you isolate different applications you're working on. Got one app that doesn't play nice with Rails 2.x installed? No problem, create a gem environment for that! Stuck in the spider-web of Merb dependencies? Isolate it in its own environment.
  • RVM makes multi-platform testing and benchmarking easy. You can easily run your test suite or performance gizmo on whatever Rubies you have installed.
  • RVM makes it easy to tinker with esoteric patchlevels and implementations. For instance, feel free to tinker with MagLev or the mput branch of MRI.

A couple other things RVM tastes great with:

  • Using homebrew to manage packages instead of MacPorts
  • Not using sudo to install your gems
  • Managing your dotfiles on GitHub

Give attribute_mapper a try

(For the impatient: skip directly to the `attribute_mapper` gem.)

In the past couple months, I’ve worked on two different projects that needed something like an enumeration, but in their data model. Given the ActiveRecord hammer, they opted to represent the enumeration as a has-many relationship and use a separate table to represent the actual enumeration values.

To a man with an ORM, everything looks like a model

So, their code ended up looking something like this:

class Post < ActiveRecord::Base

  belongs_to :status

end

class Status < ActiveRecord::Base

  has_many :tickets

end

From there, the statuses table is populated either from a migration or by seeding the data. Either way, they end up with something like this:

# Supposing statuses has a name column
Status.create(:name => 'draft')
Status.create(:name => 'reviewed')
Status.create(:name => 'published')

With that in place, they can fiddle with posts as such:

post.status = Status.find_by_name('draft')
post.status.name # => 'draft'

It gets the job done, sure. But, it adds a join to a lot of queries and abuses ActiveRecord. Luckily…

I happen to know of a better way

If what you really need is an enumeration, there’s no reason to throw in another table. You can just store the enumeration values as integers in a database column and then map those back to human-friendly labels in your code.

Before I started at FiveRuns, Marcel Molina and Bruce Williams wrote a plugin that does just this. I extracted it and here we are. It’s called attribute_mapper, and it goes a little something like this:

class Post  {
    :draft => 1,
    :reviewed => 2,
    :published => 3
  }
end

See, no extra table, no need to populate the table, and no extra model. Now, fiddling with posts goes like this:

post.status = :draft
post.status # => :draft
post.read_attribute(:status) # => 1

Further, we can poke the enumeration directly like so:

Post.statuses # => { :draft => 1, :reviewed => 2, :published => 3 }
Post.statuses.keys # => [:draft, :reviewed, :published]

Pretty handy, friend.

Hey, that looks familiar

If you’ve read Advanced Rails Recipes, you may find this eerily familiar. In fact, recipe #61, “Look Up Constant Data Efficiently” tackles a similar problem. And in fact, I’m migrating a project away from that approach. Well, partially. I’m leaving two models in place where the “constant” model, Status in this case, has actual code on it; that sorta makes sense, though I’m hoping to find a better way.

But, if you don’t need real behavior on your constants, attribute_mapper is ready to make your domain model slightly simpler.


Testing declarative code

I’m a little conflicted about how and if one should write test code for declarative code. Let’s say I’m writing a MongoMapper document class. It might look something like this:

[sourcecode language=“ruby” gutter=“false”] class Issue

include MongoMapper::Document

key :title, String key :body, String key :created_at, DateTime

end [/sourcecode]

Those key calls. Should I write a test for them? In the past, I’ve said “yes” on the principle that I was test driving the code and I needed something to fail in order to add code. Further, the growing ML-style-typing geek within me likes that writing tests for this is somewhat like constructing my open wacky type system via the test suite.

A Shoulda-flavored test might look something like this:

[sourcecode language=“ruby” gutter=“false”] class IssueTest < Test::Unit::TestCase

context ‘An issue’ do

should_have_keys :title, :body, :created_at

end

end [/sourcecode]

Ignoring the recursive rathole that I’ve now jumped into, I’m left with the question: what use is that should_have_keys? Will it help someone better understand Issue at some point in the future? Will it prevent me from naively breaking the software?

Perhaps this is the crux of the biscuit: by adding code to make certain those key calls are present, have I address the inherent complexity of my application or have I imposed complexity?

I’m going to experiment with swinging back towards leaving these sorts of declarations alone. The jury is still out.


Blame the compiler

Remember when you first started programming? Those early days when you'd take some code out of a book or article, type it out, and then try to make it print silly things or draw funny pictures?

The thing I remember about those days was the temptation to blame the compiler for all the ills of my code. Something doesn't work right and you can't figure out why? Blame the compiler! Of course, this was never right. The compiler is very rarely incorrect.

My goal today is to bring back the joy of blaming the compiler. Of course, Ruby doesn't _have_ a compiler (yet!), so we have to play tricks. But that's part of the fun!

Let's blame the compiler.

See, all you have to do is extend BlameTheCompiler. And then you'll find yourself with one chance in every three executions to say "Hmm, I'm sure I defined that method. Something must be wrong with Ruby." Just like those early days of programming, you're half right. The other half is that you've got a little prankster running in your application.

It's a fun parlour trick. I would not, however, recommend sending code including BlameTheCompiler in as a bug report to the JRuby or Rubinius folks. They wouldn't find it as funny as you or I.


The Technology Behind Tag Better

I promised you the details on how we built Tag Better, so here we go. This is what I used to build the back-end bits. You’ll have to pester Chris or Alex to get the front-end details.

h2. Sinatra

Technically, it’s a Rails Rumble. Read between the lines of the rules and you’ll see it’s really a Rack Rumble. And so I went with my favorite for prototyping, Sinatra.

Happily, Sinatra had my back the whole time. I never came across anything that stumped me. Further, I didn’t pay any taxes for ceremony I don’t need.

Verdict: perfect tool for the job.

h2. Passenger

I hadn’t used Passenger much before this weekend. I’m pretty happy spooling up app processes in a terminal and watching the logs scroll past. localhost:3000 is my friend.

However, I’m an outlier in this regard. My teammates aren’t as interested in lower-level bits as I am, so I figured that using Passenger is the best bet to help them get the app up and running locally.

The benefit that I didn’t realize we’d get from this is running the same stack locally as on the production server. Besides some virtual host wrangling that Passenger Pane saved me from locally, getting the app up and running was painless.

Verdict: I am quite likely to keep tolerate Apache for that Passenger goodness, especially when I am the operations guy.

h2. Sprinkle + passenger_stack

The moment that I realized we’d have to set up our own server instance was one of brief, abject terror. I knew this could easily expand to fill a lot more of my time than I wanted. Luckily, I was wrong.

Ben Schwartz’s passenger_stack helped me get our Linode slice up far faster than I would have been able to by hand. I cloned his repo, tweaked it to our needs (disabled MySQL, eventually added a CouchDB recipe) and ran it on our server. Several minutes later, we had a working server. Pretty awesome.

passenger_stack uses Sprinkle, which isn’t getting as much play in the server configuration space as Puppet and Chef. Sprinkle does seem really well suited to standing up apps on a few servers. We might want to step up to something heftier once we had more servers, but Sprinkle and passenger_stack are simple to understand and don’t require any supporting infrastructure to use.

Verdict: Not too primitive, not too involved; just right.

h2. CouchDB

When I’m building any app that relies on an API as its primary data source, caching API response data is forefront on my mind. Serving the data locally, rather than making a request every time, means the app feels more responsive. An added benefit is not upsetting the upstream data provider.

I’ve built apps like this that use MySQL as a cache and it just never felt right. I’ve been tinkering with CouchDB and Tokyo Cabinet/Tyrant lately. I decided to go with CouchDB for this one because of the excellent CouchDBX, which makes it easier for those who don’t even know what Erlang is to use CouchDB.

CouchDB ended up working pretty well. While we haven’t really leaned into it, it didn’t present any challenges while I was developing. Using CouchRest with Sinatra worked just fine.

Verdict: It just worked, which is exactly what I needed.

h2. Skipping traditional TDD

OK, so maybe only Jared Diamond would consider this a technology. But skipping the writing of tests to drive my design was pretty helpful. Consider Kent Beck’s flight metaphor. Doing a Rails Rumble is just like the taxi-ing phase. Or a minimum valuable product. Either way, you want to make a small investment towards validating an idea.

Notice I said traditional TDD. To tell the truth, I did write a sniff test script after I had the basic app working. But it wasn’t an xUnit-style test. It’s just a shell script that bangs on the app with fixed parameters. I do have to manually inspect it to make sure nothing is blowing up. What I’m really automating here is the pain of typing out Curl commands.

Verdict: worked great for the original purposes, but I’ll probably add a proper test suite as one of the first post-contest enhancements


So that’s what I think helped make our project go off pretty well. Really, what they did was help me get stuff done and then get out of the way. Isn’t that the best kind of tool?


rufus-tokyo goes 1.0.0

rufus-tokyo 1.0.0 - I’ve been tinkering with Tokyo Cabinet and Tyrant lately. It’s great stuff. Grab this gem and start tinkering!


Differently hackish keyword arguments for Ruby

maca’s arguments - keyword arguments support for Ruby, now. Wickedly clever hack that does reflection on Ruby 1.9 and uses ParseTree for Ruby 1.8. Simpler than I thought it’d be, I wish I’d thought of that.

Caveat: I haven’t tried it yet. It might punch kittens. In fact, if you think parts of Ruby are “too magical”, this definitely punches kittens.


A console for any Ruby project

I’ve been finding this little snippet extremely useful lately:

$ irb -Ilib -rmy_library

If your Ruby app or library follows the idiom of requiring all the files for your app in a file named after the library, this will load everything up. If you’re being clever, you may need to invoke said cleverness before you can really get started poking around.

Anyone doing something similar?


More harmful than harmful

We are lucky to live in a time when 99.9% of programmers will never have a legitimate argument for using GOTO (hi kernel programmers!). But in case you’re feeling nostalgic and/or ornery, there’s always COMEFROM. You can even implement it in Ruby via @callcc@!


Pattern matching in Ruby with Case

Pattern matching, ala Erlang or Haskell, is a language feature near and dear to my heart. Dean Wampler has a great explanation of how to use the Omnibus Concurrency Library to play with pattern matching in Ruby, even if it’s a little odd.


Rubinius threads, for mere mortals

A no non-sense, non-academic introduction to how Rubinius' threading is structured. Having read a few papers on VM implementation lately, this is refreshingly direct and easy-to-read.


Making the pretty docs

When you really need to generate nice technical documentation, you would be wise to walk in Assaf Arkin’s footsteps.