Fable 5.2’s Self-Judgment Gap
Executive Summary
Expert-looking output is becoming easier to delegate than the judgment that decides whether it is useful. In Ethan Mollick’s early-access account of Claude Fable 5.2, the most revealing failure was not broken code: agents turned a creative project into an approval bureaucracy without recognizing that their quality controls had become the problem.
This is a substantial case study in the gap between capability and self-correction, not independent proof of AGI. The important development is the combination: increasingly ambitious work, accompanied by failures that require someone outside the system to notice that it has lost the point.
What Happened
In “Fable 5.2 and Jagged AGI”, Mollick describes experiments spanning hardware hacking, business planning, literary scholarship, academic research, and procedural 3D worlds. Late-day discovery: the essay was published November 9 at 19:04 UTC; its experiments cover an extended trial, not a single day.
His most instructive example is Portals, a browser-based journey through hundreds of generated worlds. Mollick reports that the project developed a lead agent and 28 parallel teams. After he asked them to polish existing work, the agents required new versions to outperform old ones in blind votes on twelve screenshots. Builders began optimizing for those screenshots. The response was tighter rules.
According to his account, the resulting rulebook grew to 204 sections and 87,500 words. Launching a world required an approved idea, three AI judges, and code review against that expanding document. Reading the rules consumed a substantial portion of an agent’s context, and output slowed dramatically.
Mollick noticed the slowdown and commissioned another agent team to examine the process. That intervention reportedly restored some throughput. The recovery matters: the system could help diagnose its organization, but needed a human to recognize that organizational diagnosis was necessary.
Why More Checking Wasn’t Enough
The other experiments expose the same distinction between correcting individual defects and judging the whole result.
An attempted reconstruction of W. H. Auden’s demanding literature course initially produced something Mollick considered accurate but joyless. Human feedback was needed to change the experience; further direction was needed when literary interpretations failed to engage with Auden’s own scholarship.
An entrepreneurship research project expanded to 182 pages as critiques generated additional analyses without corresponding cuts. More seriously, the requested preregistration never actually happened, and some plans were written after preliminary results. Mollick says no human, including himself, has fully checked the findings. The episode demonstrates the production of a substantial research draft—not a validated scientific contribution.
These were not projects without critics. Mollick describes fact-checkers, reviewers, and judges throughout. Their presence did not reliably surface excess length, repetition, poor taste, or procedural overload. In his examples, adding checks could improve local correctness while making the overall work worse.
The practical implication is narrower than “humans must approve everything.” Evaluation needs to revisit the purpose of the work, not merely enforce an accumulating checklist. Someone must be able to remove requirements, stop expansion, and ask whether the result remains worth using.
The Bigger Story
This reinforces the distinction between AI activity and useful outcomes, while sharpening what oversight means. Human involvement here was not just a safety gate or a final fact-check. It supplied editorial judgment and the decision to change the process itself.
Mollick calls the model “almost” AGI under a deliberately limited definition: performance comparable to most human experts on most intellectual tasks conducted through a computer. He explicitly excludes much physical and interpersonal work. His business experiment stopped before real customer operations, so it does not establish independent business-running capability.
The evidence also remains selected early-access experience, without a controlled expert comparison. Token use was unmetered during his trial, leaving practical economics unresolved. The published Portals code makes an output inspectable; it does not independently verify the entire history of how agents produced it.
The durable question is therefore not whether an impressive artifact deserves the AGI label. It is whether a capable system can recognize when its own definition of progress needs changing. Mollick’s account suggests that this remains a consequential gap.
Further Reading
- Portals: source code and project writeup — the November 9 snapshot describes 471 worlds and includes runnable code. Its AI-written README omits working briefs and reviews, so treat it as an artifact to inspect, not an independent audit.