On instrumenting Rails, custom aggregators, bespoke dashboards, and reinventing the wheel; 37signals documents their own metrics infrastructure. They’re doing some cool things here:
- a StatsD fork that stores to Redis; for most people, this is way more sensible than the effort involved in installing Graphite, let alone maintaining it
- storing aggregated metrics to flat files; it’s super-tempting to overbuild this part, but if flat files work for you, run with it
- leaning on ActiveSupport notifications for instrumentation; I’ve tinkered with this a little and it’s awesome, I highly recommend it if you have the means
- building a custom reporting app on top of their metric data; anything is better than the Graphite reporting app
More like this, please.
One could take issue with them rolling this all on their own, rather than relying on existing services. If 37signals were a fresh new shop, one would have a point. Building out metrics infrastructure, even today with awesome tools like StatsD, can turn into an epic time sink. However, once you’ve been around for several years and thought enough about what you need to measure and act on your application, rolling your own metrics kit starts to make a lot of sense. It’s fun too!
Of course, the important part is they’re measuring things and operating based on evidence. Whether you roll your own metrics infrastructure or use something off the shelf like Librato or NewRelic, operating on hard data is the coolest thing of all.
