Recent advances in large language models and their ability to generate code have the AI field buzzing with excitement. And made many in the software industry nervous. The advances in code generation and assistance have been trickling out over the past decade, but with recent advances in LLMs with ChatGPT, Claude, (and we’re told eventually Bard) they are much more capable than before. In this post, we’ll explore the different levels of autonomy and mastery that AI has achieved in code generation. We’ll also theorize the skill sets that are most likely to be displaced by these new technologies in the near future.
Levels of code generation autonomy
First, let’s define what we mean by “Code Generation”. I like to think about AI code generation similarly to how autonomous vehicles are classified. There are six levels of autonomy when considering autonomous vehicles (level 0 being no automation), and I think we can also define six levels of autonomy when it comes to code generation.
Level 0: No automation
Human programmers write all code.
Level 1: Code assistance
AI can suggest snippets, autocomplete code, catch errors as programmers type. But humans write the actual logic and program flow.
Level 2: Function/module generation
AI can generate full functions, classes or modules when prompted with comments specifying the goal and parameters. But humans stitch these components together into a full program.
Level 3: Script/program generation
AI can generate simple scripts or programs from a high-level prompt. But the generated code often requires optimization, debugging and may have limited functionality. Human oversight and input is still essential.
Level 4: General program generation
AI can generate complex programs and systems with minimal human input. The generated programs can match human-level performance on some tasks but may lack robustness or have narrow capabilities. Human review and oversight remains necessary.
Level 5: Autonomous programming
AI achieves human-level programming ability, able to develop highly complex, scalable and robust software systems with self-supervised learning and little need for human input or oversight. This level remains largely speculative and a long-term goal of AI.
Where are we today?
Depending on the tooling used, some programmers today are still squarely in level 0. These developers are missing out on a tremendous amount of value and productivity boosters that basic IDEs (interactive developer environments) offer. With the advent of powerful IDEs such as VSCode, Eclipse, Jetbrains, and others most developers will experience Level 1. All IDEs offer code completion, syntax highlighting and basic debugging assistance. The release of Github Copilot and other more capable tools pushed us into the realm of Level 2, where these tools can suggest more complete sections of code. The LLMs mentioned above are also able to generate snippets of code for use, such as entire function implementations and scripts.
However, it wasn’t until recently with the release of GPT-4 and OpenAI Plugins that we started seeing the potential for Level 3. At first, it was just a neat trick to be able to ask ChatGPT or Claude to generate a function implementation for you, but with plugins that allow you to integrate that functionality into an existing tool (such as an IDE) we are definitely in the space of being able to interactively code in ways never before seen.
Accelerated advances
With the release of these more powerful LLMs, you are going to see rapidly increasing capabilities and tooling available that will move us into Level 3 of Code Generation Automation. There are an ever-increasing number of demos available showing early-adopters pushing the limits of these LLMs by using ChatGPT to generate entire programs. The original release video of GPT-4 showcased creating a simple web application from scratch using a hand-drawn image. This doesn’t (in my opinion) put us squarely into the third level of autonomy (yet), where entire programs are generated, but we’re definitely knocking at the door. Expect to see these tools widely available well before the end of 2023.
If you’re in the software industry and have dedicated development teams, stay on top of these advances in generational AI so your teams can benefit from the increased productivity they enable.
Where does that leave programmers?
Even starting to venture into Level 3 autonomy for code generation leaves plenty of room for skilled software developers. That being said, the key word there is skilled. As tooling becomes available that allows anyone to generate basic code and programs, low-skilled developers will quickly find themselves needing to up their game or find another profession. When a company can ask a tool to generate simple applications for them, including the ability to tweak and iterate on them, they won’t want to pay a self-taught programmer who spends most of their time crawling StackOverflow.
Developers who “specialize” in simple programming tasks are going to eventually find it harder to stay relevant. There will be other opportunities that come with these tools, and perhaps we’ll explore some in a future blog post, but with a new emerging technology comes a lot of unknowns, and what new positions will be needed to curate these tools is one of them.
Summary
As a developer, I have a full appreciation for the complexity and artistry that comes with good software development. I think we’re a ways off from completely displacing it or even coming close to automating away the complex parts of it. However, the accelerated advancement of these technologies shows that it WILL happen, and in order to stay relevant developers need to do two things:
-
Stay up to date on the advancements of these technologies and learn how to use them to 10x your own effectiveness
-
Learn what these technologies currently struggle with and will continue to struggle with and increase your knowledge in these areas.
And of course, as always, be a perpetual learner!
