More Reasoning Is Not the Same as Better Work

Executive Summary

The clearest practical signal today is that the most capable setting is often the wrong production choice. In two very different environments—coding agents and live voice agents—operators are finding that extra reasoning can impose steep costs in latency and token use without producing a commensurate gain in task performance. The emerging discipline is not “use the smartest model available,” but match model effort to an observable job, its time budget, and its failure modes.

Evidence was unusually thin, so this is a narrower report than usual. The two substantive items are practitioner and vendor accounts rather than independent evaluations. Their numbers should not be generalized as universal benchmarks. What matters is that both point toward the same operational rule: intelligence has to be evaluated inside the system where it will be used.

The Cost of Maximum Reasoning

In a hands-on review of long-running coding-agent workflows, Theo Browne reports a strikingly weak return from an Opus 5.5 “Max” reasoning setting on his Skatebench test. Average response tokens rose from 338 to roughly 5,000, average response time from six to 50 seconds, and the slowest response from 31 to 600 seconds. Accuracy moved only from 78% to 79%.

That is one self-reported benchmark, not a definitive model comparison. But it illustrates a decision that benchmark leaderboards routinely obscure: a marginal capability gain can be a large product regression when it makes feedback loops slower, raises cost, or reduces the number of iterations a user can afford. Browne’s practical recommendation is to reserve maximum reasoning rather than make it the default, using High or X-High for routine coding work.

The more durable part of his account concerns control. Long-running agents performed better when given the full task, a testable definition of done, explicit stop conditions, permission for expected sensitive steps, and clear prohibitions. One remote setup ran for about 17 minutes and completed; another agent stopped after “phase zero” until told more forcefully to continue. The lesson is not merely to write longer prompts. It is to define an execution contract that lets the agent distinguish progress, completion, and a legitimate reason to stop.

Production Constraints Choose the Model

A Retell AI deployment guide makes the same argument from live voice systems. Retell says GPT-4.1 remains its most-used model across more than 100 million monthly calls because low latency, instruction following, and reliable function calls matter more in conversation than maximum benchmark reasoning. Reasoning models can pause long enough to damage turn-taking—the quality users experience directly.

Retell recommends evaluating real calls through containment, transfer rate, call duration, and customer satisfaction. It also estimates total voice-agent cost at about $0.09–$0.145 per minute in example configurations, with the language model sometimes costing less than the voice engine and telephony. Those are vendor claims from a company selling the surrounding platform, so they need independent confirmation. Still, the decomposition is useful: optimizing inference price alone can miss both the dominant cost and the dominant user experience constraint.

The Bigger Story

Together, these accounts reinforce a developing view of agents as engineered workflows rather than autonomous intelligence in a box. Model quality is conditional. The right amount of reasoning depends on whether the bottleneck is accuracy, response time, tool reliability, supervision, or iteration speed. More test-time compute is valuable only when it changes an outcome users care about.

This also revises the familiar advice to “use the strongest model for hard tasks.” Difficulty is only one axis. A production decision should ask: What failure does added reasoning prevent? How much latency and cost does it add? Can the gain be measured on the actual task? If those questions have no concrete answer, maximum reasoning is an expensive default rather than a safety margin.

Workflow Implications

For agent builders, the practical pattern is simple:

• Define observable completion and stop conditions before tuning model effort.
• Measure end-to-end outcomes, not model metrics in isolation.
• Escalate reasoning selectively for ambiguous or high-stakes steps.
• Keep fast feedback for routine work; iteration can be more valuable than deliberation.
• Test cross-model review where independent criticism matters, but measure whether it catches consequential errors.

Further Reading

• AI Tinkerers: agent intent, streaming memory, and sandboxing — a useful collection of concrete experiments in permission boundaries, default-deny isolation, and staged context. The reported efficiency and verification gains remain presenter claims, but the implementation patterns are worth preserving.