Stop Generating, Start Thinking

Source: Hacker News

by Sophie Koonin
8 February 2026

Tags: ai, engineering

Throughout my career, I feel like I’ve done a pretty decent job of staying on top of new developments in the industry: attending conferences, following (and later befriending!) some of the very smart people writing the specs, being the one sharing news on Slack about exciting new features of CSS or JS with my colleagues. The joys of working on an internal tool where you only need to worry about the latest Chrome, and playing with anchor positioning in a production app while it’s still experimental!

It’s very unsettling, then, to find myself feeling like I’m in danger of being left behind—like I’m missing something. As much as I don’t like it, so many people have started going so hard on LLM‑generated code in a way that I just can’t wrap my head around.

I’ve been using Copilot—and more recently Claude—as a sort of “spicy autocomplete” and occasional debugging assistant for some time, but any time I try to get it to do anything remotely clever, it completely shits the bed. Don’t get me wrong, I know that a large part of this is me holding it wrong, but I find it hard to justify the value of investing so much of my time perfecting the art of asking a machine to write what I could do perfectly well in less time than it takes to hone the prompt.

You’ve got to give it enough context—but not too much, or it gets overloaded. You’re supposed to craft lengthy prompts that massage the AI assistant’s apparently fragile ego by telling it “you are an expert in distributed systems” as if it were an insecure, mediocre software developer.

Or I could just write the damn code in less time than all of this takes to get working.

As I see more and more people generating code instead of writing it, I wonder why engineers are so ready and willing to do away with one of the good bits of our jobs (coding) and leave themselves with the boring bit (reviews).

Perhaps people enjoy writing role‑play instructions for computers; I don’t know. But I find it dangerous that people will willingly—and proudly—pump their products full of generated code.

I’ll share a couple of the arguments I’ve encountered when I’ve expressed concern about this.

Reference: “Spicy autocomplete” – The Guardian

“This is the Industrial Revolution of our time! It’s like mechanisation all over again.”

Parallels with the Industrial Revolution

The Industrial Revolution contributed heavily to climate change—see the OER Project analysis.
Today, the energy consumption of AI data centres is staggering; the IEA report on AI‑driven demand highlights the scale.

• Not all of this electricity comes from fossil fuels, which is an improvement over the 19th‑century model.
• Nevertheless, we continue to waste massive resources on frivolous outputs—e.g., the viral “shrimp Jesus” images that were quickly banned by Meta. See the Business Insider story.

The Fast‑Fashion Analogy

Mechanisation made goods cheaper and more accessible, but it also drove a race to the bottom in quality. Modern examples include:

• SHEIN – you can buy a highly flammable pair of trousers for less than a cup of coffee.
• Off‑shoring of factories to less‑developed economies, where low wages and weak labour rights boost profits.

Generated code behaves much like fast‑fashion:

• It may look acceptable at first glance but often falls apart under scrutiny.
• It frequently copies existing designs, as reported by the Sunday Post on high‑street copycats.
• The environmental impact is significant.

Key Difference: Determinism vs. Non‑Determinism

• Mechanisation replaced human effort with machines that performed the same task reliably. A codemod or script that generates boiler‑plate code is a good example—if something goes wrong, a human can inspect the machine and diagnose the issue.
• LLM output is non‑deterministic and its inner workings are opaque. A mechanised process that produces different results each time, often peppered with hallucinations, offers little practical utility.

In short, while AI’s rise mirrors the Industrial Revolution’s scale and resource demands, its non‑deterministic nature and environmental footprint pose distinct challenges that must be addressed.

“LLMs are just another layer of abstraction, like higher‑level programming languages were to assembly.”

It’s true that writing Java or Go means I never have to bother learning assembly. The closest I get to anything resembling assembly is knitting patterns.

The way we write software has evolved in terms of what we need to think about (depending on your language of choice):

• I don’t have to think about garbage collection or memory allocation because the runtime does it for me.
• I still have to think about writing efficient code that makes sense architecturally in the wider context of our existing systems.
• I must consider how the software I’m building will affect critical paths and reason about maintainability vs. speed of delivery.
• When building for the web, we have to think about browser support, accessibility, security, and performance.

Where LLMs do the most damage

The biggest problem I’ve seen is engineers outsourcing the thinking that should go into software development. LLMs can’t reason about system architecture because they cannot reason—they do not think. If we stop thinking and the model doesn’t think, then nobody is thinking. Nothing good can come from software that no one has thoughtfully designed.

In the wake of the Horizon scandal—where innocent Post Office staff went to prison because of bugs in Post Office software that led management to think they’d been stealing money—we need to think about our software more than ever. We need accountability in our software.

Thirteen people killed themselves as a direct result of those bugs in the Post Office system.

Our terrible code is the problem

“But human developers today write inaccessible, unperformant, JavaScript‑heavy code! What’s the difference?”

Exactly. LLMs are trained (often without our explicit consent) on all of our shitty code, and we’ve taught them that this is what they should be outputting. Consequently, they:

1. Repeat human mistakes.
2. Are then re‑trained on the same flawed output, creating a feedback loop sometimes called human‑centipede epistemology (see Maggie Appleton’s article on the AI Dark Forest).

We don’t write good enough code as humans to deserve something that writes the same stuff faster.

And if you think we’ve done all right so far, we haven’t:

• Users of assistive technology often struggle with inaccessible interfaces.
• People in regions with terrible internet connections (or those relying on mobile data in any UK city) face performance issues.
• Racial discrimination persists in facial‑recognition software and even in mundane devices like hand dryers.
• The Post Office staff suffered the consequences of buggy software.

Instead of learning, improving, and building better software, we’ve outsourced our mistakes to an unthinking algorithm.

Four eyes good, two eyes bad

Jessica Rose and Eda Eren gave a brilliant talk at FFConf 2024 about the danger of AI coding assistants making us lose our skills. One slide stood out:

Key points

• Reviewing a PR written by a human carries a level of trust because a colleague has reasoned about the code.
• Open‑source maintainers are already seeing a deluge of low‑quality PRs generated by LLMs.
• Even if an LLM generates the code, the contributor remains accountable for what they commit.
• The reviewer also holds accountability, so there are still two pairs of eyes on the change.

The risk of “single‑eye” workflows

Some companies showcase tools like Claude that generate PRs from a Slack chat. The workflow looks like this:

1. A developer asks Claude to make a change.
2. Claude auto‑generates the code and opens a PR.
3. A reviewer approves the PR.

If the reviewer is the only person looking at the change, we’ve lost one pair of eyes and reduced shared context within the team.

Reviewing PRs isn’t just about catching bugs; it’s about sharing understanding of the code and the rationale behind changes. Teams that skip PRs altogether and commit directly to the main branch can only make it work at scale if engineers pair constantly, preserving shared context.

Takeaways

• LLMs are tools, not replacements for human reasoning.
• Accountability must stay with the human who writes, reviews, and ships code.
• Maintain multiple layers of review (the “four eyes” principle) to preserve shared understanding.
• Invest in better coding practices now; otherwise, we’ll only accelerate the propagation of our current mistakes.

I’m Not Anti‑Progress, I’m Anti‑Hype

I want to make it clear that I’m not “anti‑LLM”. My objection is to branding it as artificial intelligence—it isn’t intelligent, it’s just a form of machine learning. “Generative AI” is essentially a very sophisticated Markov chain that people expect far too much from.

I don’t begrudge using generative AI for quick prototypes. If you need to throw together a wireframe or an interactive demo, it makes a lot of sense. My concern is when people think they can “vibe code” their way to production‑ready software or hand off the actual thinking behind the code.

Mikayla Maki had a particularly good take on working with agents: keep the human in the loop, treat them like an external contributor you don’t trust. Only use agents for tasks you already know how to do, because it’s vital that you understand them.
— On Programming with Agents (Zed.dev)

I will continue using my spicy autocomplete, but I’m not outsourcing my thinking any time soon.

• Stop generating.
• Start understanding.
• Remember why we enjoyed doing this in the first place.