Where's the Shovelware? The AI Coding Claims Conundrum
In recent years, AI coding tools like GitHub Copilot, Claude Code, and Cursor have promised extraordinary productivity gains for developers. These tools claim to make coding faster, easier, and more efficient, with taglines like “Built to make you extraordinarily productive” and “Build Better Software Faster.” However, despite these bold claims and widespread adoption, there is a noticeable lack of evidence that these tools are actually leading to a significant increase in software output.
In his article “Where’s the Shovelware? Why AI Coding Claims Don’t Add Up”, Mike Judge argues that the AI coding revolution is more hype than substance. He points out that if these tools were truly making developers as productive as claimed, we should be seeing a flood of new software—apps, games, websites, and more. Yet, the data shows no such surge. Charts tracking new software releases across various sectors remain flat, with no indication of the exponential growth one would expect if AI coding tools were living up to their promises.
Judge’s argument is compelling. If AI coding tools were genuinely making developers 10x more productive, as some claim, the world would be drowning in shovelware. The fact that this isn’t happening suggests that these tools may not be as transformative as advertised. This disconnect between claims and reality has serious implications for the tech industry, as companies are making decisions based on the assumption that AI coding tools are dramatically increasing productivity.
Key Takeaways from Mike Judge’s Analysis
Productivity Claims vs. Reality:
- AI coding tools claim to make developers significantly more productive, with some developers even claiming a 10x increase in output.
- However, studies like the METR study show that developers often overestimate their productivity gains from AI tools. In some cases, AI tools may actually slow developers down.
Lack of Shovelware:
- If AI coding tools were as effective as claimed, there would be a noticeable increase in the number of new software releases.
- Data from sources like Statista, Verisign, and SteamDB show no significant increase in new software releases post-2022/2023, despite widespread AI tool adoption.
Impact on Developers:
- The pressure to adopt AI coding tools is leading to job insecurity, with developers feeling compelled to use these tools even if they don’t find them helpful.
- The focus on AI tools may be distracting from other important aspects of software development, such as code quality and continuous improvement.
Personal Reflections on AI Coding Tools
While AI coding tools haven’t turned me into a 10x developer, they have given me the confidence to take on more complex coding tasks. As Simon Willison notes, only a small portion of a programmer’s time is spent on hands-on coding, which may explain why we’re not seeing a flood of new software. AI tools can speed up the coding process, but they don’t address the other aspects of software development, like planning, testing, and maintenance.
That said, the lack of a noticeable increase in software output is concerning. It suggests that the benefits of AI coding tools may be overstated, and that the industry’s focus on these tools could be misplaced. As Judge argues, developers should trust their gut and stick with what works for them, rather than feeling pressured to adopt tools that may not actually improve their productivity.
Conclusion
The AI coding revolution may not be living up to its promises, but that doesn’t mean these tools are useless. They can still be valuable for certain tasks and for certain developers. However, the industry should approach these tools with a healthy dose of skepticism and focus on objective measures of productivity, rather than hype and bold claims.
