First, read all of this excellent distillation of distributed systems by Jay Kreps,  The Log: What every software engineer should know about real-time data’s unifying abstraction. Now, consider this.

There’s a moment, when you’re building services and web APIs, when you think you’ve pretty much got it under control. You’ve got an endpoint for every query, a resource for every workflow. All the use-cases seem to be under control. And then, the question appears:

“How can I get access to all the updates to all the data? You know, for [REASONS].”

For APIs exposed to external developers over the web, this is where you’d reach for web hooks or PubSubHubbub. It’s not the best solution, but it works. If you’re building an internal system, you could  use the same approaches, or…you could build a log.

No not that kind of log. An event log, like LinkedIn did with Kafka for their internal systems. Every time your data model changes, every create, update, or delete, you drop an event with all the metadata related to the change. The event goes into some kind of single-producer, multiple consumer queue. Then all the clients that want to know about all the changes to all the things can read events off the queue and do whatever it is they need to for those important REASONS.

If you find this intriguing, this is a lot like replication in database systems. Definitely read LinkedIn’s article on this, and definitely read up on how your database of choice handles replication. And if you’ve built this before and have a good answer to initially populating “replicas” of a database, let me know; I haven’t come up with anything better than “just rsync it”.

Brent Simmons is keeping a diary as he works through implementing sync for Vesper, an iOS note-taking app. Building this sort of thing isn’t easy; cf. it took Cultured Code multiple years to implement it for Things. Thus it’s pretty neat that Simmons is breaking it down into understandable chunks:

• Syncing Tags, on reasoning about identity in your data model
• Immutability, Deleting, and Calculated Properties, on tackling deleted data, aka the hardest kind of write in a distributed system
• Sync tokens and efficiency, on converting your system into an event log
• Merging Notes, on sync semantics and dealing with clocks

If you think you need to implement a synchronization system for your application, try to find a shortcut so that you don’t. If you can’t find a shortcut, you could do worse than starting with these notes.

So, Adam from a few years ago, you think you can build a distributed system?

Designing a concurrent system, one that runs across multiple processors using shared memory threads, is difficult because someone can move your cheese. One moment you’ve got food on your plate, and the next moment it’s on someone else’s plate.

Designing distributed systems, one that runs across multiple computers over networks that occasionally do weird things, is difficult because you can’t assume that the message will get through. One moment you’re telling your friend about an awesome new album and the next moment they don’t even know who you are anymore.

In both scenarios, you can’t assume the simplest case. You have to discard Occam’s razor; it’s entirely possible a perfectly devious alien is tinkering with your system, swallowing messages or preventing threads from running. For every operation in your system, you have to ask yourself what could go wrong and how will my system deal with that?

The first jarring thing about these sorts of problems is that you’re probably not using a framework. There are libraries to help you with logging, instrumentation, storage, coordination, and low-level primitives. There aren’t a lot of well-curated collections of opinions expressed as code. There’s no Rails, no Play, no Django, no Cocoa. There’s hardly even a Sinatra, a Celery, a JDBC.

That means you’re going to be designing, for real. Drawing on whiteboard, building prototypes and proofs of concepts. Getting feedback from as many smart people as you can corner. Questioning your assumptions, reading everything you can find on the topic. Looking for tradeoffs that decrease risk and simplify your problem domain.

Accept that you’re unlikely to ever get it entirely right. Those who are really good at it are more like Scotty than Spock. They’re surrounded by levers and measurements, tweaking and rebuilding the system as they go. It’s a fun puzzle!

rd.io/x/QFWiK0E…

Look, Prince, I’m not sure if I can agree with you regarding “Breakfast Can Wait”. Cereal? Toast? English Muffins? Sure, those things can wait. Migas? Pancakes? Bacon? These things cannot wait. And don’t get me started on eggs; wait too long and delicious eggs can go rubbery and gross all too quickly.

In Cocoa, you can poke inside object graphs (and more!) using dotted strings:

 NSDictionary *bestOfBeatles = @{@"paul": @{@"beatles": @"Hey Jude", @"solo": @"Jet"}};

NSString *solo = [bestOfBeatles valueForKeyPath:@"paul.solo"];
NSLog(@"Paul's Best solo song is: %@", solo);

Hey, did you notice that Objective-C sprouted some really handy literals for arrays and dictionaries lately? One could reasonably extrapolate that Apple has an interest in not breaking people’s fingers when they build for iOS and OS X.

There are libraries, e.g. Hashie, that make uniform access into object graphs, like you might get from an API response, as close to idiomatic Ruby as you probably want. Key paths, as implemented in Cocoa’s Key-Value Coding APIs, takes a different approach, using strings to define paths to traverse an object graph. I don’t really like “string programming”, but it’s an interesting approach, so let’s see where it takes us. Let’s add something like “key paths” to Ruby!

Hark, an implementation

Enumerable is an awesome thing about Ruby. Let’s build key path traversal on top of that. The goal is something close to the Objective-C snippet above:

 best_beatles = KVC.new({"paul" => {"beatles" => "Hey, Jude", "solo" => "Jet"}})
solo = best_beatles.for_path("paul.solo")
puts "Paul's best solo song is: #{solo}"

Turns out this isn’t hard to implement, even with a little error handling thrown in:

require 'delegate'

class KVC < SimpleDelegator

  def for_path(path)
    segments = path.split('.')

    segments.inject(self) do |value, s|
      if !value.respond_to?(:has_key?)
        raise ArgumentError.new("Expected #{value} to respond to has_key?")
      end

      if value.has_key?(s)
        value[s]
      else
        raise KeyError.new("Missing key #{s}")
      end
    end
  end

end

OK, first off, did you realize that SimpleDelegator makes it ridiculously easy to wrap objects with extra behavior? It sure does! Second, you could easily write this with a while loop or even a simple each, but why have local variables hanging around when you could use inject instead?![1] Third, this has more conditionals than I’d like, but it fits within 80 columns[2] so I’ll take it.

The pros and cons of using key paths

I’ve got this code, based on a whim. I’m not even sure it’s a good idea. Maybe it is!

• It’s simpler to specify a traversal into a deep object graph with a string than with a bunch of key lookups or method chains. It’s slightly clearer too.
• “Selector APIs”, like XPath or jQuery’s extensions to CSS, are an idea many developers have already wrapped their head around. I wouldn’t want to go anywhere near the complexity of those APIs, but an opinionated set of operations on an object graph might not get too complex.

Maybe it sucks?

• Enumerable is pretty great, why wouldn’t you use that?
• Programming by banging strings together is terrible.

Judgement

I can’t really tell you, definitively, if you should try this or not. I’ve barely even used KVC in Cocoa! It’s a nifty idea, though, and it’s pretty rad that you can implement something like it in Ruby so trivially.

Now you know!