Fallbacks Need a Job Description

Executive Summary

The useful question for an AI fallback is no longer “which other model can we call?” It is “what narrower, evaluated job can another model safely perform?” A new practitioner write-up from OpenRouter’s Kenny Rogers makes that case with a small but unusually concrete billing-agent experiment: a second model looked unreliable when asked to run the whole workflow, yet became viable once the workflow was split so the model extracted a few fields and ordinary code applied policy, money, and evidence rules.

That distinction also connects the day’s cluster of real-time video talks. Whether the system is a billing agent or an interactive video stream, the hard work is moving outward from base-model selection: define the contract, isolate deterministic decisions, preserve state, observe the actual system, and test the failures users will encounter. Model choice still matters. But the emerging practical canon is that a model is a component of a service, not the service’s reliability strategy.

What Happened

In “How to Build Antifragile Agents with OpenRouter,” Rogers compares two ways of constructing a synthetic billing agent. In the end-to-end version, the agent had to infer intent, select policies, calculate amounts, and produce an answer. Across 20 cases, Fable 5 produced 14 exact matches and Grok 4.5 produced five. Those are task-specific synthetic results—not a general model leaderboard—but they expose how much policy and arithmetic were being left to an uncertain component.

The revised design asks the model only to extract three intent fields under a structured-output contract. TypeScript then handles the policy, calculation, and evidence logic. Both models passed the 20 cases and an eight-case holdout in the reported experiment; the post also reports lower cost and latency for Grok on that suite. The important result is not that one provider won. It is that substitutability appeared after the task was decomposed.

The proposed operating discipline follows from that result: test each route against the same product-level contract before treating it as a fallback; pin the model version used in the evaluation; log which model actually served a request; and turn reviewed production failures into new cases. A fallback that has merely returned valid JSON is not necessarily a fallback that can safely make a customer-affecting decision.

Why It Matters

This is a corrective to the popular image of routing as a simple availability or price switch. A full-workflow agent entangles classification, judgment, calculation, tool use, and explanation. Replacing its model can change all of them at once. A constrained extractor has a smaller observable surface, clearer success criteria, and more room for conventional software to enforce invariants.

That does not mean every agent should be reduced to extraction. Some tasks genuinely require model judgment, and deterministic logic can encode bad policy just as reliably as good policy. It does mean teams should identify where an error is expensive or irreversible, then decide whether that step belongs in a model call at all. The right fallback plan may be a different prompt and provider; it may also be a simpler mode, a queue, or a request for human review.

The Bigger Story

The same systems framing appeared in four AI Engineer conference talks on real-time generative video. Krea emphasized training and infrastructure observability beyond headline GPU utilization, including tensor-core and interconnect health. Reactor and uRun described interactive continuous video as a serving problem with streaming sessions, state continuity, and proximity to GPUs—not simply batch generation at a higher frame rate. LemonSlice described the complementary model-side problems of causal generation, one-step denoising, and accumulated error in real-time avatars.

These are vendor presentations, so their performance claims should not be read as independent benchmarks. Their convergence is still telling: visual AI products are discovering that continuity and evaluation are first-class requirements. Reactor’s acknowledgement that real-time consistency often still needs human judgment is especially useful. A system can meet a latency target and still visibly drift, break character, or fail the interaction.

Workflow Implications

For an agent with a fallback path, run one exercise this week: take a high-consequence workflow and write its contract as inputs, permitted actions, deterministic checks, required evidence, and failure modes. Then evaluate every proposed route—including the primary—against the same held-out cases. Move calculations, authorization, and policy rules out of the model path where that improves auditability; retain model judgment only where it is genuinely needed.

For real-time systems, add continuity tests alongside throughput and latency dashboards. Measure state drift across a session, record the serving route and model version, and define the human review or graceful-degradation path before an interactive experience is exposed as autonomous.

Further Reading

• How to Build Antifragile Agents with OpenRouter — the billing-agent experiment, source code, and routing recommendations.
• Krea on production infrastructure for generative video — a substantial talk on training/inference operations and meaningful hardware metrics.
• Reactor on real-time video generation — a useful account of continuity, latency, and the remaining evaluation gap.