Agent Reliability Is Becoming a System Design Problem
Executive Summary
The most useful signal today is not a new model claim but a small, unusually inspectable account of making an agent workflow less dependent on one. In “How to Build Antifragile Agents with OpenRouter”, an OpenRouter developer-relations author publishes a billing-triage example with code, synthetic cases, a holdout set, policy tests, and routing traces. Its conclusion is modest but important: reliability came less from choosing the “best” model than from relocating consequential decisions into deterministic software, evaluating models on the actual task, and making fallback behavior a tested part of the system.
This is one implementation, on a small synthetic suite—not a general model ranking or proof that every agent should use the same architecture. But it reinforces a developing practical canon: model capability is increasingly one component of an operational design whose real quality is determined by constraints, evaluation, observability, and recovery paths.
What Happened
The article’s task is deliberately concrete: classify a billing request, apply the appropriate policy, calculate a monetary outcome, and respond. The reported design separates the fuzzy portion—extracting intent from a request—from the decisions that ought to be stable: policy selection and money calculation. Those latter steps move into TypeScript rather than remaining model outputs.
That separation changes the routing question. Instead of treating a fallback as an emergency switch after a provider problem, the author qualifies both models against the deployed workflow, records the fallback order outside the application, and traces which model actually served each request. On the stated 20-case evaluation—including eight holdout cases—the two tested models moved from imperfect end-to-end performance to 20/20 exact results after the deterministic logic was introduced. The less expensive model then became the evaluated default, with the other retained as fallback.
The result should be read narrowly. Synthetic cases can miss production variation, the sample is small, and the author has a vendor affiliation. The value is the published shape of the experiment: readers can inspect the tests and reproduce or challenge the claim rather than taking an agent-reliability story on faith.
Why It Matters
Agent discourse often oscillates between two unhelpful questions: which frontier model wins, and whether agents can be trusted at all. This example points to a more actionable question: which parts of a workflow can safely remain probabilistic, and which must become explicit, testable machinery?
That is not an argument to minimize models. Intent extraction, interpretation, and open-ended drafting are precisely where they can add value. It is an argument against asking them to be an unversioned policy engine, calculator, and control plane at once. Once a workflow’s hard decisions are represented as ordinary code and tests, model selection can become a cost-and-quality decision made on local evidence rather than an act of reputation management.
It also complicates the usual “more capable model equals more reliable agent” story. Better models may broaden what can stay fuzzy, but they do not remove the need to define failure, verify high-stakes outputs, or know what happened when a request took an alternate path. Reliability is an architectural property, not a model attribute.
Workflow Implications
For builders, the immediate exercise is to take one agent flow that can create a commitment—money movement, permission changes, customer promises, data mutation, or production deployment—and draw its boundary between interpretation and decision. Put the latter behind deterministic functions where possible. Then build a compact evaluation set with at least a holdout slice, run every candidate and fallback through it, and log the model and route used in production.
The goal is not to manufacture elaborate routing for a simple assistant. It is to prevent an apparently resilient system from being merely a single-model workflow with an untested escape hatch. Start where errors have a clear cost, and treat the fallback as something that must earn its place in tests.
Further Reading
- How to Build Antifragile Agents with OpenRouter — the code-and-evaluation-backed primary artifact behind this report.
- Turn off Claude Code’s Memory — a practitioner argument, based on personal projects, for keeping durable agent context in repository artifacts and executable checks rather than opaque memory; useful as a complementary perspective, not independent evidence.