Krosoft | The Agent Bottleneck Is the Harness

Executive Summary

The most important shift in today’s AI discourse was not a new model benchmark. It was a clearer consensus that agent performance is increasingly determined by the harness around the model, the interface through which people collaborate with it, and the controls that keep it legible when things go wrong. In other words: the bottleneck is moving from raw model IQ to workflow design.

The clearest evidence came from AI Tinkerers / Post-Training’s writeup on Reef, Coral Bricks’ open-source agent harness for finance research. The claim is striking not because it promises magical autonomy, but because it attributes a large evaluation jump to system structure rather than model switching. On Vals AI Finance Agent v2, the team reports an 82.6% pass rate for its full harness versus 44.87% for the reference harness, with a retrieval-only variant landing at 49.8%. Their argument is that most of the gain came from skill structure, planner/specialist separation, typed tool bindings, lazy skill loading, and runtime constraints—not from simply stuffing more instructions into the prompt or swapping in a better base model. You can read the full piece here: https://post-training.aitinkerers.org/p/how-to-write-a-winning-agent-harness-for-your-domain

That same theme showed up from a different angle in Ted Johnson’s AI Engineer talk, “The Prompt Is Still a Punch Card.” His point was that current AI interaction still behaves like batch computing: users package intent into a prompt, wait, inspect output, then repair. If that framing holds, prompt engineering is less a mature interface and more a workaround for an outdated protocol. Johnson’s design direction is toward systems that handle interruption, turn-taking, ambiguity repair, and timing better, so the human is not forced to do so much packaging work up front. Source: https://www.youtube.com/watch?v=hVJOnuhFmTA

Anthropic’s redeployment note for Fable 5 made the same underlying lesson harder to ignore. The company restored global access after a short export-control disruption, but the important detail was the operational tradeoff: a new classifier that blocks the reported bypass in more than 99% of tested cases also increases false positives on benign coding and debugging requests. That is a reminder that frontier models do not arrive as stable, neutral commodities. Access rules, safeguards, and product behavior can shift quickly, which makes portability, routing discipline, and explicit approval layers more valuable than ever. Source: https://www.anthropic.com/news/redeploying-fable-5

What Happened

Across the strongest items, the discourse converged on one idea: useful agents are being built less by adding prompt instructions and more by engineering the surrounding operating system for the model.

The Reef article is the most concrete example. It describes a prompt-heavy baseline that had grown beyond 2,500 lines of instructions and 50-plus overlapping tools, then argues that moving conventions into code and narrowing skill dispatch cut cost by roughly 10x while improving pass rates. That is a notable reinforcement of a developing canon in AI engineering: once models are competent enough, orchestration quality starts to dominate day-to-day outcomes.

Johnson’s talk complements that argument by moving one level higher. If the prompt is still a “punch card,” then even a well-harnessed system is still constrained by a crude collaboration protocol. The implication is that the next round of agent progress may come as much from interface design as from model releases.

Simon Willison’s short note on Geoffrey Litt’s “understand to participate” framing added the human counterweight. The warning is that if agents generate larger changes than their operators can still mentally track, the result is cognitive debt rather than leverage. The payoff from agents rises only if the user remains fluent enough to direct and extend the work. Source: https://simonwillison.net/2026/Jul/2/understand-to-participate/

Why It Matters

This is one of the clearest recent days for a recurring pattern in AI discourse: the practical frontier is shifting from model selection to system design.

That does not mean models no longer matter. It means model improvements are no longer the whole story for practitioners. A stronger model inside a weak harness can still be slow, expensive, hard to steer, and fragile under policy changes. A merely strong-enough model inside a disciplined harness may outperform it on real work because the system knows how to route tasks, enforce constraints, preserve state, and keep the human in the loop.

Anthropic’s Fable note sharpens the governance side of that argument. If deployment conditions and safeguards can change overnight, then builders should treat model choice as a replaceable dependency, not the foundation of the workflow itself.

Workflow Implications

For hands-on builders, the practical takeaway is concrete.

First, audit any agent stack that keeps getting “fixed” by adding more instructions. If performance degrades as prompts grow, the problem may be harness shape: too many tools, unclear routing, weak constraints, or conventions that belong in code rather than prose.

Second, treat interaction design as a real engineering surface. Measure where users are forced to package context, resolve ambiguity, and recover from timing mistakes manually. Those are not just UX annoyances; they are part of the capability ceiling.

Third, optimize for participation, not mere delegation. Preserve visible task state, approvals for consequential actions, and enough transparency that a human can still understand and redirect the work without starting from scratch.

The Agent Bottleneck Is the Harness

Executive Summary

What Happened

Why It Matters

Workflow Implications

Further Reading