Folks are ruffled that LLMs, even without tool use, are pretty good at coding interview challenges. How will we even know who is good at coding anymore? (Current hypothesis: by collaborating with them as early as possible.)

Friend, I am here to tell you that LLMs are not the problem. The issue is we were evaluating programmers on speed and competency through the tedious parts of programming. If we ever looked for what makes a programmer effective, it was accidentally. 🫠

In retrospect, we had mediocre insight into what activities are effective use of a programmer’s time. At best, we were using a metric from decades ago, when knowledge of algorithms was rare and solving arcane puzzles was more valuable. The past twenty years of human knowledge accumulation, into reusable libraries, on the web and then distilled into language models, has obviated much of that work.

That’s not even counting all the tedium we didn’t even realize we created. Generating projects with their dependencies and libraries all lined up just the right way (looking at you, C and JavaScript ecosystems), upgrading those libraries, configuring our development environments, accreting unruly towers of software dependencies that grew an industry (software supply chain management), even the seemly simple task of “commit my work to the correct branch and fix things up when I realize I was on the wrong branch when I committed”. 🙃 All of that tedium is required toil for successfully developing software, but none of it contributes to solving problems for people.

The wrong insight here is that developers will be replaced by LLMs that can quickly implement a linked list. (And much more, if you supervise them well enough.) Maybe the 10x developers were really only good at tedium or algorithm tests. 🌶️

A better insight is that Leetcode and similar schemes were probably never that great. But, they were easier (and perhaps better on average🤷‍♂️) than developing a well-considered coding exercise yourself. Silver lining: it’s an object lesson that you don’t need the best product to succeed in the market.

The best insight: way more of software development is tedium than we previously guessed. And, it’s within our grasp to automate those bits away. If we can reclaim the parts of our brains that remember arcane shell commands or know how to rebase our way out of a sticky git conundrum, all the better.