Era Tests Whether Enterprise Agents Can Connect the Records

Executive Summary

An agent that retrieves the right records can still fail when asked to connect them. In Eon’s Era benchmark, nine models averaged 92.6% accuracy on simple filtering but just 3.7% on multi-hop questions. The useful finding is not a new model ranking: it is how much harder enterprise work becomes when an evaluation requires consistent relationships across business systems.

Benji Gruenbaum’s practitioner account of Era makes the underlying argument concrete: plausible synthetic documents do not necessarily describe a coherent company. Generating a shared company first creates questions with checkable answers—and exposes failures that disconnected mock data can conceal. These are developer-reported results from one company and 33 questions, not an independent estimate of production reliability.

What Happened

Era builds a fictional company as a seeded graph of employees, customers, deals, support tickets, calls, and other business activity. Product simulators then expose different views of that same company. A customer referenced in Salesforce and Zendesk remains the same entity; a deal’s history must respect its dates and ownership.

This reverses the usual document-by-document approach. Instead of hoping separately generated records agree, shared identities and dependencies determine the records. The same seed reproduces the company, and expected answers are computed from the completed data rather than maintained by hand.

Timing caveat: the essay surfaced in the past 24 hours, but its publication date is unverified. The underlying Era benchmark paper was submitted September 9; this is delayed discovery, not research released today.

The reported experiment asked nine models the same 33 questions three times, producing 891 answers over Salesforce, Zendesk, and Gong. Beyond the filtering–multi-hop gap, mean accuracy was 39.6% on cross-system questions, 11.1% on ranking and order-sensitive questions, and 7.4% on temporal questions.

One difficult task asked agents to identify the account with the greatest total open sales opportunity value, then calculate that account’s recorded call time. No model correctly answered the median ticket-resolution-time question. These are ordinary analytical requests, but they demand exhaustive retrieval, correct joins, and aggregation—not merely finding a relevant passage.

Why It Matters

The strongest methodological detail is that a correct answer must also be reachable. An audit caught a question whose join worked against stored data but depended on an internal identifier unavailable through the product interface. The benchmark rejects such questions rather than penalizing an agent for failing an impossible task.

That distinction matters as much as exact grading. A benchmark can have a mathematically correct answer key and still measure an interface defect instead of agent competence.

The results also belong to a particular agent setup. Models discovered systems and operations through three meta-tools, with budgets of 25 turns, 40 tool calls, and 8,000 characters per tool response. Consequently, the scores measure models operating through that discovery process and those limits—not reasoning ability in isolation.

Nor does the experiment establish a dependable leaderboard. Only three of 36 pairwise model differences remained statistically supported after correction. Three repeats reduce generation noise; they do not turn 33 question shapes into broad coverage of enterprise work.

The Bigger Story

Yesterday’s discussion separated access to capable model weights from dependable deployment. Era reinforces that distinction from the evaluation side: convincing demonstrations and strong general benchmarks do not establish that an agent can reconcile a company’s records.

Its own limits remain important. The evaluated environment was read-only, so it does not establish safe execution of business changes. Realism targets include estimates, templated text lacks the full variety of human writing, and the reported model comparison covers only one generated company.

The practical lesson is narrower than “enterprise agents do not work”: test whether they can join, exhaust, order, and aggregate records in a world whose answers you can verify. Successful lookup is a starting point, not evidence that the surrounding workflow is solved.

Further Reading

• The Era by Eon Benchmark — Full methods, answer-reachability checks, statistical comparisons, and limitations; an older paper worth preserving for its evaluation design.