When I sought to learn Ruby, it was for three reasons. I’d heard of this cool thing called blocks, and that they had a lot of great use cases. I read there was this thing called metaprogramming and it was easier and more practical than learning Lisp. Plus, I knew several smart, nice people who were doing Ruby so it was probably a good thing to pay attention to. As it turns out, I will never go back to a language without the first and last. I can’t live without blocks, and I can’t live without smart, kind, fun people.
Metaprogramming requires a little more nuance. I understand metaprogramming well enough to get clever with it, and I understand it well enough to mostly understand what other people’s metaprogramming does. I still struggle with the nomenclature (eigenclass, metaclass, class Class?) and I often fall back to trial and error or brute-force tinkering to get things working.
On the other hand, I think I’ve come far enough that I can start to smell out when metaprogramming is done in good taste. See, every language has a feature that is terribly abused because it’s the cool, clever thing in the language: operator overloading in Scala, monadic everything in Haskell, XML in Java, and metaprogramming in Ruby.
Adam’s Handy Guide to Metaprogramming
This guide won’t teach you how to metaprogram, but it will teach you when to metaprogram.
I want you to think twice the next time you reach for the metaprogramming hammer. It’s a great tool for building developer-friendly APIs, little languages, and using code as data. But often, it’s a step too far. Normal, everyday programming will do you just fine.
There are two principles at work here.
Don’t metaprogram when you can just program
Exhaust all your all tricks before you reach for metaprogramming. Use Ruby’s mixins and method delegation to compose a class. Dip into your Gang of Four book and see if there isn’t a pattern that solves your problem.
Lots of metaprogramming is in support of callback-oriented programming. Think “before”/”after”/”around” hooks. You can do this by defining extension points in the public API for your class and mixing other modules into the class that implement logic around those public methods.
Another common form is configuring an object or framework. Think about things that declare models, connections, or queries. Use method chaining to build or configure an object that acts as a parameter list for another method or object.
Use the weakest form of metaprogramming possible
Once you’ve exhausted your patterns and static Ruby tricks, it’s time to play a game: how little metaprogramming can you do and get the job done?
Various forms of metaprogramming are weaker or stronger than others. The weaker ones are harder to screw up and less likely to require a deep understanding of Ruby. The stronger ones have trade-offs that require careful application and possibly need a lot of explanation to newcomers to your codebase.
Now, I will present to you a partial ordering of metaprogramming forms, in order of weak to strong. We can bicker on their specific placement, but I’m pretty certain that the first one is far better to use frequently than the last.
- Blocks – I hesitate to call this a form of metaprogramming. But, it is sometimes abused, and it is sometimes smart to use blocks instead of tricks further down this list. That said, if you find yourself needing more than one block parameter to a method, you should consider a parameter object that holds those blocks instead.
- Dynamic message send on a static object – You set a symbol on an object and later it will send that symbol as a method selector to an object that doesn’t change at runtime. This is weak because the only thing that varies is the method that gets called. On the other hand, you could have just used a block.
- Dynamic message send on a dynamic object – You set a symbol and a receiver object, at some point they are combined into a method call. This is stronger than the previous form because you’ve got two points of variability, which means two things to hunt down and two more things to hold in your brain.
Class.new– I love this method so much. But, it’s a source of potential hurt when trying to understand a new piece of code. Classes magically poofing into existence at runtime makes code harder to read and navigate with simple tools. At the very least, have the civility to assign classes created this way to a constant so they feel like a normal class. Downsides, err, aside, I love this method so much, having it around is way better than not.define_method– I like this method a lot too. Again, it’s way better to have it around than not. It’s got two modes of use, one gnarly and one not-so-bad. If you look at how its used in Rails, you’ll see a lot of instances where its passed a string of code, sometimes with interpolations inside said string. This is the gnarly form; unfortunately, it’s also faster on MRI and maybe other runtimes. There is another form, where you pass a block todefine_methodand the block becomes the body of the newly defined method. This one is far easier to read. Don’t even ask me the differences in how variables are bound in that block; Evan Phoenix and Wilson Bilkovich tried to explain it to me once and I just stared at them like a yokel.class_eval– We’re getting into the big guns of metaprogramming now. The trick withclass_evalis that its tricky to understand exactly which class (the metaclass or the class itself) the parameters toclass_evalapply to. The upside is that’s mostly a write-time problem. It’s easy to look at code that usesclass_evaland figure out what it intends to do. Just don’t put that stuff in front of me in an interview and expect me to tell you where the methods land without typing the damn thing into IRB.instance_eval– Same tricks asclass_eval. This may have simpler semantics, but I always find myself falling back to tinkering with IRB, your mileage may vary. The one really tricky thing you can do withinstance_eval(and theclass <<some_objtrick) is put methods on specific instances of an object. Another thing that’s better to have around than not, but always gives me pause when I see it or think I should use it.method_missing– Behold, the easiest form of metaprogramming to grasp and thus the most widely abused. Don’t feel like typing out methods to delegate or want to build an API that’s easy to use but impossible to document?method_missingthat stuff! Builder objects are a legitimate use ofmethod_missing. Everything else requires deep zen to justify. Remember: friends don’t let friends write objects that indiscriminately swallow messages.eval– You almost certainly don’t need this; almost everything else is better off as a weaker form of metaprogramming. If I see this, I expect that you’re doing something really, really clever and therefore have a well-written justification and a note from your parents.
Bonus principle!
At some point you will accidentally type “meatprogram” instead of “metaprogram”. Cherish that moment!
It’s OK to write a few more lines of code if they’re simple, concise, and easy to test. Use delegation, decorators, adapters, etc. before you metaprogram. Exhaust your GoF tricks. Read up on SOLID principles and understand how they change how you program and give you much of the flexibility that metaprogramming provides without all the trickery. When you do resort to trickery, use the simplest trickery you can. Document it, test it, and have someone review it.
When it comes to metaprogramming, it’s not about how much of the language you use. It’s about what the next person to see the code whispers under their breath. Don’t let your present self make future enemies.
Adam Keys is typing 
“I read there was this thing called metaprogramming and it was easier and more practical than learning Lisp.”
Having spent a good number of years working with Ruby doing all sorts of metaprogramming nonsense and then moving to work in Clojure I’d like to contest this statement. For me, learning Clojure was easier and more practical than trying to wrap my head around the warped labyrinth of a rabbit hole that is metaprogramming in Ruby. That’s despite reading (and loving) Why’s Poignant Guide a number of enjoyable times…
@sam but _why_ is Clojure easier to metaprogram?
@adam, There is no difference between metaprogramming and programming with lisps – they are one and the same thing. This is not the case in Ruby where they are different things.
TL;DR
Programming is about manipulating the language’s human readable form (the text that is parsed). Metaprogramming is about manipulating the language’s computer readable form (the AST that is generated by parsing the text). Having a large difference between the text and the AST tends to make metaprogramming more difficult and less powerful.
With lisps, the text is a direct representation of the AST – they are essentially the same thing. So writing text in your language is the same as modifying the AST.
As the AST is essentially a basic data structure (a tree) and that data structure is a standard data structure in the language, this means that the basic simple methods you use to manipulate data-structures (i.e. regular programming) also work when manipulating the programming language itself. There is no difference in how you might work with a list of numbers or a list of programming statements.
Unfortunately with Ruby, there’s quite a large difference between the AST and the language – in fact there is no standard AST, so the AST created by JRuby is different from MRI is different from Rubinius. This means the AST tends not to be exposed, and you are limited to manipulating an AST API which provides that impedance mismatch and by its very nature is less powerful than being able to manipulate and generate the AST directly.
I can agree to the extent that AST manipulation is far easier and natural in Lisp. But this is apples to oranges; you don’t metaprogram in Ruby by manipulating the AST. You tinker with objects and methods, just like you do in “natural” Ruby. So that’s a push.
Further, there _is_ a difference between programming and metaprogramming in Lisp. Clojure even has special forms for it! If a skilled Lisp developer were to come across a program that was all macros, I wager they would proceed with great caution, just as a Ruby developer who found lots of DSL action would spend time figuring out how they work.
Let us not presume that we should use something just because we understand it.
I wasn’t attempting to make a fully concrete and watertight case, only broadly highlight my initial point which was that in my experience metaprogramming in Ruby was harder, more complicated and less practical than learning lisp.
Comparing apples to oranges was exactly my point – lisps can work directly with the AST because the text representing lisps *is* the AST. Ruby settles for a less powerful “objects and methods” API. This was a specific design decision by Matz. I’m not arguing against or in favour of this decision – I’m just saying that for me, this API was harder to learn and master than lisp was – by quite a long way.
Also, you are right, there is a subtle difference between programming and metaprogramming in lisp, but it’s not macros (although they are a useful and crazy powerful tool for serious metaprogramming unmatched in Ruby). The only real difference is the quote:
(+ 1 2) ;=> programming
‘(+ 1 2) ;=> metaprogramming
As you can see, the first is a basic summation of 1 and 2, the second is the datastructure which represents the summation of 1 and 2. This is just a basic list and may be manipulated as a normal list and then may be evaled to produce a result. The only difference is the quote, which indicates that the statement is meta – the real point is that the form of the data structure representing the summation is the same as the summation itself this is what I meant when I said “one and the same thing”
Finally, I never argued in favour of using something just because we understand it. I don’t believe that one bit – you should use what you believe is the best fit for the problem in hand. I also totally agree with the general sentiment of your post – unnecessary complexity is to be avoided wherever possible. This is especially the case when you consider programming as a form of communication between programmers which is how I tend to see the world.
We violently agree. Thanks for the challenge of thinking about how this applies to Lisp. :)
And, thank-you for the insightful post :-)
Well said.
Don’t forget that you can add methods to a particular instance with an ordinary module and instance.extend(OrdinaryModule) — no need to reach for instance_eval.
It’s hard to draw the line as to where exactly ‘metaprogramming’ begins, but at any rate I’d consider an ordinary module to an instance with #extend to be pretty low on the list of ‘strength’ (with ‘weak’ being more desirable, as you say).
And, worth pointing out that if you must do #method_missing, please please please fix #respond_to? to match, and don’t just swallow any method calls, call super on any method calls you didn’t mean to catch (which will result in a raised MethodMissing, unless the superclass wants to do something with it!).
I agree with Sam..I love ruby (it’s my favorite language) but metaprograming with ruby feels really tricky..it’s not natural and you need learn a set the new methods and ways to do things…clojure is different because it’s really consistent and flexible:
in any language you’ve code and data…and a wall between both..in clojure or lisp the data is code and the code is data…so you can use these in a flexible way and since the first time you really learn lisp..you learn how work it…i feel than learn this concept is similar to blocks in ruby (lambdas in any functional lang): when you hear about them don’t seems to be so special but when you begin ruby, you feel its power and the syntax becomes familiar and self explicit…
2 and really important…clojure is (Though seems different) a language with a really short syntax and it’s really consistent, you can do a lot with a few words and they always works like you expected…I feel than this is a big different between ruby because you wrote a lot different methods than in ruby you need learn for do MP…you need know the difference between them and know how write it…the way like you use class_eval is different to how you use eval…is a syntax really different…with clojure instead you need learn 2 things: defmacro command and the ” ‘ ” (ok there are other few symbols like ~ or ` but the idea is the same, convert your code to data or your data to code using it)
greetings from Italy, long live to ruby and clojure and by the way…sam aaron is a badass (the clojure creator said it)..he has a really interesting project for do music using clojure named Overtone…..
Can you show a real world use for Class.new.
Sure, that’s a good topic for a follow-up post.