Dumb AI and the software revolution

Posted: 2025-10-21

This is part of a series I’m writing on generative AI.

Introduction

Anyone who has worked with today’s generative AI models has experienced two conflicting truths: they are frustratingly dumb, and they are astonishingly powerful. The critics are right: our models are stochastic parrots, prone to bizarre mistakes and nonsensical hallucinations. And yet, they are already poised to have a massive impact on technical fields like software engineering.

The key is to stop waiting for a genius in the machine. Instead, we can harness a network of bumbling, lightning-fast collaborators, imposing adequate structure and guardrails. The result will be an explosion of custom software, solving niche problems we never thought were economical to tackle.

The frustrating stupidity of AI 🐒

“Two things are infinite: the universe and human stupidity; and I’m not sure about the universe.” – Albert Einstein

Let’s be upfront: using current-generation generative AI feels like working with extremely junior programmers. They are incredibly fast and eager to please, but they are also lazy and careless, and have the attention span of a goldfish. Their stupidity can be maddening. Sometimes they appear only barely more capable than simple Markov chains.

There’s no shortage of problems:

They get overwhelmed easily. As a conversation grows, the quality plummets.
They are wildly unpredictable. Contexts that worked perfectly ten times might inexplicably fail on the eleventh.
They can’t tell fact from fantasy. I’ve seen them invent and then diligently follow their own non-sensical instructions, as if I had issued them myself.
They get stuch in loops. I’ve watched them pointlessly read the same file (that they are supposed to analyze) again and again.

This is not the dawn of a superintelligence, or at least not yet. Rather, it is the arrival of a workforce that can be quite baffling.

Yet tremendously useful

And yet… this army of clumsy coders works at the speed of light. And you can scale it instantly (as long as you feet the bill). That changes everything.

I am optimistic that we can deliver good-quality software through this large pool of mediocre programmers. We don’t need a smarter model; rather organization: we need to build systems that impose structure, guide this new workforce, and ruthlessly discard the garbage while retaining the occasional diamonds.

The infinite monkey is no longer hitting keys at random –just somewhat stupidly. But at this speed and with instant scaling, the difference is monumental. With the right guidance, it will produce the works of Shakespeare.

I’m not talking about shipping low-quality code. Bad code is bad, regardless of how its written. Instead, I’m suggesting that we can orchestrate AI to produce high-quality maintainable code by using validation tools we already have. Does it software compile? Do all tests pass? Does it meet our explicit quality criteria? These mechanisms are not perfect, but they suffice.

From abstract ideas to concrete code

“The mere formulation of a problem is far more essential than its solution.” – Albert Einstein

We can think of large intellectual works, including software, as trees of ideas. They start from the most abstract high-level concepts and build all the way to small leaves of implementation details.

Asking models to produce the entire tree is a recipe for disaster. So many “I gave AI a large problem; it failed miserably” stories amount to exactly that.

But these models are already remarkably capable of tackling the smaller problems at the leaves of the tree. The details are still critical. Mess them up and the rocket explodes. And setting up the guardrails and external structure is far from trivial.

But it can be done! If we decompose the problem and set up guardrails adequately, AI can reliably solve a lot of these smaller tasks. In Agentic AI: Recipes for reliable generative workflows I explore ideas for imposing this structure, offering various approaches to deal with the aformentioned problems (e.g., suggestions on problem decomposition, validation, avoiding information overload, etc.).

The magic of technical excellence happens in the interplay between high-level ideas and low-level details. A high-level idea may seem excellent at first sight, but turn out to be infeasible or require significant adjustments upon closer inspection. AI accelerates this dialogue, allowing this iterative process to play out much faster.

A new frontier for software

This fundamentally changes the role of software engineers. Our mental energy can be focused on the abstract, high-level concepts at the top of the tree of ideas. We can leave the tedious details to our AI assistants and, thanks to their speed, refine our artifacts much faster.

As AI accelerates software delivery, the ROI equation changes drastically. Niche problems, where a software solution was once too expensive for the expected returns, suddenly become worthwhile.

This will likely lead to an explosion in the long tail of software: custom artifacts for specific needs and general software that is far more adaptable to niche problems.

Generative AI today isn’t an AGI; it’s a force multiplier for the human intellect, a bumbling worker that, with the right guidance and organization, will help us build the future – one leaf at a time.