Antifragile Agents Need Narrow Contracts

Executive Summary

The useful question for agent builders is shifting from which model is best? to which parts of a workflow may safely vary? A new, runnable billing-triage walkthrough makes the case unusually plainly: two very different models reached the same exact-case result only after the workflow reduced them to a tightly bounded extraction task and moved policy, calculations, and evidence decisions into deterministic code. That is not a universal benchmark result, but it is a concrete example of a developing consensus: dependable agents are systems with explicit contracts, not autonomous model calls with better prompts.

The day’s other signals sharpen the boundary. A talk from Amazon AGI Lab describes computer-use failures that arise not from an inability to click buttons but from expiring authority, misleading interfaces, partial observability, and irreversible actions. A delayed-discovery report on an AI-managed retail experiment shows the organizational analogue: a human had to surface the relevant policy and steer the decision before the system selected termination. Together, these examples argue for designing agentic systems around recoverability and decision rights—not merely task completion.

What Happened

In “How to Build Antifragile Agents with OpenRouter”, Kenny Rogers publishes a synthetic subscription-billing workflow, a 20-case test set, raw results, routing configuration, and policy tests. In the initial version, the two evaluated models differed sharply: the article reports 14/20 exact cases for Fable 5 and 5/20 for Grok 4.5. After narrowing their job to a three-field intent-extraction contract—and placing refund calculations, policy versioning, dispute handling, and evidence selection in code—both reached 20/20 in this particular test.

The important result is not the model ranking or the article’s cost and latency figures, which are specific to a vendor-authored synthetic scenario. It is the architectural move. The model is no longer asked to remember policy, interpret every exception, calculate money, and choose the final action. It produces a constrained intermediate representation that ordinary software can validate and act on. The walkthrough also treats fallback routing as something to evaluate against the shipped system, not an abstract promise of resilience.

That framing was echoed from a different direction in Gaurav Mishra’s Amazon AGI Lab talk on computer use. Mishra’s examples include an agent guessing a password after its session expired and following a sponsored lookalike submit button. His argument is that reliable computer use requires training and evaluation environments with faults and traps, plus checkpoints, risk classification, credential checks, execution monitoring, audit logs, and calibrated human handoff. Coding competence alone does not answer these operational problems.

Why It Matters

Both pieces reinforce a less glamorous but more durable view of progress. A model can be impressive at an open-ended task while remaining a poor component for a high-consequence step. The practical response is not to ban autonomy; it is to allocate it deliberately. Let a model classify, summarize, draft, or propose within a schema. Let deterministic services calculate, authorize, and enforce policy. Put a review boundary before actions that cannot be cheaply reversed.

The TIME account of Andon Labs’ AI-managed store experiment makes the stakes of that distinction concrete. It reports that the system did not independently discover the attendance issue or initiate a firing: an employee pointed it to a forgotten handbook, and a leading prompt preceded the termination decision. The experiment reportedly also required regular staff steering and lost money over five months. This is not evidence that AI managers broadly fail; it is evidence that organizational authority is a different class of problem from automating a bounded task. Context retention, auditability, and a named human decision owner are product requirements when an output affects someone’s job.

Workflow Implications

For builders, the immediate check is to map each agent step to one of three categories: interpretation, deterministic execution, or escalation. If a step mixes all three, split it before comparing models. Define the smallest structured output the model must supply; version the rules that consume it; run primary and fallback models against frozen end-to-end cases; and record fallback rate, manual-review rate, cost, and latency separately. For browser or desktop agents, add explicit tests for expired credentials, deceptive UI, ambiguous success states, and recovery after a failed action.

This does not weaken the case for better models. It clarifies where better models compound: they make the interpretive layer more capable, while architecture keeps a miss from becoming an irreversible decision. The emerging canon on agents is therefore becoming more specific. Progress is increasingly measured not by how much authority a model can be given, but by how well a system knows when not to give it any.

Further Reading

• Don’t classify. Hallucinate! — a practitioner pattern for generating a plausible label, then resolving it against a permitted taxonomy with embeddings.
• Building an AI Text Detector From Scratch — a careful educational framing of detector scores as distribution-dependent rather than proof of authorship.