The harness was supposed to fix AI coding.
Give the model constraints. Give it checks. Give it a structure to operate within, and it will produce coherent code instead of a mess.
That was the proposal. Then it got critiqued. Then the critiques got answered. Every round produced a better harness. None of them asked whether a harness was the right thing to build.
The illusion, stated first
Here’s what the harness appears to do: it fits the model to your problem.
You define the structure. You set the constraints. You write the spec. The model works inside what you built. The problem shapes the tool.
Here’s what it actually does: it fits your problem to the model.
Every prompt is written for the model’s strengths. Every constraint is added because the model failed there. Every spec is adjusted until the model produces the right output. The shape of what you build isn’t determined by the problem. It’s determined by what the model can accept.
That’s the illusion. The work looks like design. It’s compatibility.
The refinements don’t escape it
First proposal: constraints around the model. The harness grows one patch per failure. Each patch is fitted to the failure it was built for, so none of them generalize. With N constraints, the search space is 2^N. There’s no model to converge on.
Second proposal: bound the harness. Give each component its own scoped harness, so token cost stays flat. That solves the cost. It doesn’t solve the coupling. A patch in one component’s harness affects others in ways nobody models, and every refactor calls for a re-fit.
Third proposal: common core with plugins. The core absorbs the choices that are genuinely general. What remains is M choices per component — the patches for that component’s specific failures. Smaller, but still exponential. And M is exactly the set the core couldn’t cover. The core removed the easy part. The hard part is what’s left.
Fourth proposal: keep the spec instead of the patch. A spec is only adequate if you know what it covers and how the model reads it. Nobody knows the second. So the spec is tested against one model and trusted against the next — the same bet the patch makes.
Prompt, patch, spec, harness. Each is fitted to one model. Each is a model of the model, not of the system.
Each refinement is a better harness. None of them is a design.
Why the illusion holds
Because it feels like design.
You’re defining components. You’re writing specifications. You’re setting up structure. All of that looks like engineering — and in a normal project, it would be.
But the shape isn’t determined by the problem. It’s determined by the model’s limitations. You’re not designing the software. You’re designing a representation of the software that this particular model can work with.
And when the model changes, the representation has to be rebuilt — because it was fitted to the old one.
The cost
This changes what kind of work software development is.
Before the harness, people built software with tools. A compiler, a framework, an IDE — general tools that serve every project. The work was building the software.
With the harness, people build software while building the tools that build it. And unlike a compiler, a harness is built for one component, one model, one failure history. It doesn’t serve every project. It serves this one.
So it doesn’t amortize. It defers.
A model upgrade by itself costs nothing. The harness sits there. Nobody runs it.
The cost lands when a component is modified. At that point, the model has to be run again — to regenerate code, or to write tests for the change. And the harness has to work against the current model, not the one it was fitted to.
The bill doesn’t arrive at upgrade time. It arrives when you next touch the code — and it arrives along the dependency graph.
That’s the cost of the illusion: not that the harness fails, but that it turns tooling into a deferred project expense. The illusion is cheapest for software nobody touches, and most expensive for software under active development — which is where AI was supposed to help most.
What the illusion hides
The object to fix isn’t the harness. It’s the structure the harness was supposed to describe.
Components, boundaries, dependencies. Designed before the model runs.
Because then the harness isn’t accumulated from failures. It’s derived from the design. And a derived constraint doesn’t expire when the model changes — because it was never about the model.
A description of the system that never mentions the model. Not “the model should do X.” But “this component has this boundary, and anything within it obeys these rules.”
That claim is about the system. It survives a model upgrade because it never depended on the model in the first place.
The correct proposition
So the answer isn’t “stop building harnesses.” The constraints have to work against the model you’re using. Fixing the harness is necessary.
But it isn’t the root fix. Every constraint you add is fitted to one model, and it expires when that model changes. The root fix is the structure of the system — designed before the model runs, so that the constraints can be derived from it rather than accumulated against it.
Fix the harness. But know that you’re fixing the part that expires.
The part that lasts is the structure.
The point
Everything you build around AI is a model of the AI.
Prompt, patch, spec, harness. Each is a claim about how the model behaves. Each is overfitted to the model it was written against. And each expires when the model changes, without telling you it has.
The only artifact that survives is a description of the system. Components, boundaries, dependencies. Written before the model runs, not fitted to it after.
Harness is an illusion. It looks like you’re shaping the tool to fit the problem. You’re shaping the problem to fit the tool — and the tool changes every year.
Fixing the harness is a local fix for a local problem. The structure is the root fix, because it’s the only thing that doesn’t expire when the model does.
Related: