It's not your fault if your tools confuse you

I. Pet Peeve #43: Complaining About Frameworks

The whole point of a framework is that you trade one or more axes of freedom in how you structure your program so that you can move faster writing the meaningful (unique) part of that program. To wit, the first definition I learned of a framework goes something like:

A framework is a library that takes over the main() function of your program and provides a higher-level entry point for calling your code.

Most programs would have a boring main() anyway, so this is a great tradeoff in most cases.

And yet:

  • Some programs aren’t boring.
  • Some programs demand more control.
  • Some developers crave choice and accept the burden of picking each library and choosing how to wire them together.
  • There’s a bit of a hero complex about rolling your own framework from libraries.

Where we end up is that I notice people complaining about frameworks. I think most of these are proxy complaints about someone else choosing a framework they have to live with. To their credit, some complainers are actively trying to make a better thing too. Kudos to them. To the idle complainers: cut it out.

2. I complain about a framework

RSpec and minitest are both frameworks in the sense that your entry point is a special class with special methods. RSpec is more frameworky in that you don’t actually write classes or methods. You use a language-y DSL to define test cases that share some aspect of scope. You can go pretty crazy in this manner. Or you can not go crazy, RSpec accomodates either style.

Lately I’ve been wrestling with test cases written using at least three kinds of RSpec scope and it’s driving me a little crazy. They look something like this:

describe "POST /something" do
  before do
    @thingy = somethings(:alices)
    post :something, thingy: thingy
  end

  it { expects(assigns[:something].attributes).to eq(@thingy) }
  it { is_expected.to render(:created) }
  it { is_expected.to respond_with(:json) }
end

If scoping is about answering the question “what methods are available to me on this line of code”, this code seems to have four kinds of scope:

  1. Inside the describe block, we can call RSpec methods
  2. Inside the before block we can call RSpec expectations and rspec-rails controller methods (but that is implicit!)
  3. Inside the it blocks we can call RSpec expectations
  4. When we call is_expected, we can reference…I’m not sure at all

It is at this point I try and take my own medicine and make a constructive observation rather than yelling “this sucks and someone should feel bad about it!” into the wind (i.e. Twitter).

3. What’s an RSpec for?

I didn’t like RSpec at first (circa 2008), then I really liked it (circa 2010), and now I’m sort of ambivalent. The question of what RSpec exists to solve has evolved too. At first it was “hey, BDD!” and then it was a less underscore-y way to build tests and now I think it’s a tool for writing tests in a style that some prefer and some don’t. In short, the Ruby community is figuring this out and kind of storming through the awkward parts.

That awkward part is how I think my example test came to be so unclear. Without doing a deep archaeological dig on the code, I’m guessing this code had three phases:

  1. Originally written according to the RSpec vogue at the time, which was to use it one-liners as forcing function on the constraint that tests should have only one assertion (Which in retrospect I think is a rule about the functionality of the method under test and not a guideline about how to write tests. This isn’t the first time developers have adhered to the letter of the rule and missed the spirit entirely).
  2. Use the amazingly great transpec tool to translate RSpec 2.x style to RSpec 3.x style without having to spend months carefully transitioning a large test suite from one API to another. Mostly this works out great, but you get some awkward translation, like the is_expected part.
  3. Use the newer RSpec 3.x style expect syntax for assertions, introducing two ways to say the same thing with the intended long-term benefit of using the clearer expect style everywhere.

I don’t blame RSpec for any of these phases. You can easily swap out the names of libraries and concepts for any other language or library and find a similar story buried in any chunk of code that’s been around for more than a year and worked on by more than one person. It’s a thing that happens to code. I’m not even sure people should feel bad about it. Mindful of cleaning it up over time, yes. Throw it all in the bin and start over, no.

My first temptation is to say that using it one-liners is a smell. They are nice to scan through but tricky to write and trickier still to change. But I can see where a series of well-intentioned code changes compresses many structurally similar test cases down to nearly-declarative (nearly!) one-liners without much duplicate typing. I can imagine a high-functioning team writing their own matchers, carefully using one-liners, and succeeding. So this one is a word of warning and not a smell.

The real smell, I think, is that its really easy to have very different scopes adjacent to each other in an RSpec test file. Further, not all scope-introducing constructs look the same! describe/context introduce one kind of scope, it introdues another kind of scope, let/subject/before introduce three similar but different kinds of scope, and expects/is_expected look the same but have different scopes as well.

Even smellier is that I’m making this list from an empirical understanding and not by examining the implementation or experimenting with reduced test cases.

What are you gonna do about it?

I’m probably going to leave that code alone. Wait for a muse to strike at the same time I need to make wholesale changes.

People who use RSpec should feel fine about themselves. People who contributed to RSpec should feel great about themselves. People who struggle with figuring out scope in RSpec should take solace that the best of us find this stuff confusing and frustrating at times. Developers not in one of these camps should take my advice, globals are bad but a bunch of weird scoping is not great either. Everyone else can smile and nod.

Adam Keys @therealadam