There Is So Much Left if You Have the Depth

On Friday, Simon Willison commented a blog article titled “I’m sad about AI”. Simon acknowledges that many software developers have felt the existential crisis caused by LLMs writing code of decent quality in much less time. But many managed to escape this crisis. As Simon observed:

Once you come to terms with the idea that translating an exact specification into decent code isn’t a unique skill any more, you can start looking at the larger set of problems that you face as a software engineer and realize that there is so much left, and your existing skill and experience mean you can master these new tools, provide value, and execute at a level far greater than anyone who is just getting started building software using agents without any of your depth.

(Source)

This comment has two parts that both resonate with me. First, “there is so much left.” This is so true, and it reminds me on a [quote] from Albert A. Michelson (sometimes wrongly contributed to William Thompson):

There is nothing new to be discovered in physics now. All that remains is more and more precise measurement.

(The actual quote is a bit longer.)

As Michelson, many developers today believe that LLMs are slowly taking over software development. With every release, they get better, until there’s nothing left to do for human developers.

I don’t see this happen anytime soon.

While LLMs seem to get better and better at coding at an insane speed, the way they’re constructed prevents them from ever understanding what they’re doing. In the 1980s, John Searle argued that computer programs can never achieve consciousness. He constructed the following thought experiment:

Imagine a program that is able to take Chinese characters as input and provide a meaningful output to that input, again in Chinese characters. (The program apparently was written by an English person in an English programming language, or else the story had no point.) We would not know if the program behaves intentionally, or even consciously.

Now change the scenario: John Searle sits in a room at a computer running this program. Someone slips in sheets of paper with Chinese characters under the door, and Searle, without knowing what these characters mean, passes them to the program, which returns other Chinese characters. Searle writes them on a paper and slips them back out under the door. At no point does Searle get an idea of what these characters mean. He is just following a recurring set of manual steps.

Now Searle argues that if he is not able to understand the conversation, the program, which essentially also just executes steps mechanically, isn’t able to understand it, either.

And this brings me back to Simon Willison’s conclusion that there’ll be still so much left. An LLM can mechanically write a program based on a description given to it, but, as the bit matrix calculator it is, it cannot understand the intention behind a decision a developer makes; after all, an LLM is merely trained to generate the most probable answer from a huge set of data it was trained on. So there is, and will be, plenty of work for humans to do in the endeavor to bring an idea to a finished program. This work will just be at a different level as we’re used to today.

The second part of Simon’s comment is even more important, and true: Depth of knowledge and skills is what sets experienced developers apart from the non-programmers who vibe-codes apps without ever looking at the code because they cannot make any sense of it. When your knowledge of programming languages and concepts is deep, your programming skills are battle-tested, and your mind is open and curious, you’re in the best position to steer an LLM to generate the code you need without having to blindly accept its outputs.

If you know the fundamentals of software development and learned and mastered at least one programming language (and Go would be an excellent choice), LLMs turn from an (perceived) existential threat to nothing but new, interesting tools under your toolbelt.