Gemini 4 Neon’s Coding Results Depend on the Harness
Executive Summary
- Google released Gemini 4 Neon with a million-token context window and $3/$15 input/output pricing per million tokens. Its coding scores vary sharply between agent setups, making the surrounding software a major part of the release’s performance story.
- Google DeepMind’s Melvin Johnson describes engineering-agent training built around harder repository tasks and checks for test gaming. Hidden tests, isolated evaluators and reviews of agents’ actions help determine whether apparent success deserves a reward.
Neon Scores 12.8% With mini-SWE, 39% With Antigravity
Google’s November 18 announcement positions Neon between Carbon, for demanding long-running projects, and Hydrogen, for everyday execution. It is rolling out through the Gemini API and Google AI Studio, Gemini for business, and Antigravity and the Gemini app for Pro and Ultra subscribers. Cached input costs $0.15 per million tokens.
The clearest illustration of how much the agent setup matters comes from Google’s own evaluations: Neon scores 12.8% on SWE-sweep with the mini-SWE harness and 39.0% in Google Antigravity. The announcement does not isolate which parts of the harness produce that difference. For teams evaluating a switch, reproduce representative tasks with the tools, permissions and context management the model will actually use.
Google describes Neon handling multi-file debugging and refactoring, reducing redundant file reads and managing context compaction during long sessions. In Antigravity, it can also orchestrate parallel Hydrogen subagents.
There is a pricing wrinkle. In his November 18 hands-on post, Simon Willison notes that Carbon’s temporary $2/$10 input/output price undercuts Neon’s $3/$15. Carbon is scheduled to rise to $4/$20 on December 1. A nominally smaller tier therefore carries higher per-token prices during the discount period. Compare total task cost as well: reasoning effort, repeated attempts and tool use can change the bill.
DeepMind Trains Against Easy Tasks and Test Workarounds
In an AI Engineer talk published November 18, Google DeepMind’s Melvin Johnson explains a recurring post-training problem: as models improve, familiar tasks become too easy to provide useful reinforcement-learning signal.
He describes moving toward multi-file repository work and tasks derived from commits, with graded rubrics providing richer feedback than binary unit-test outcomes. His account also covers hiding tests and repository history from agents, isolating evaluators, and penalizing workarounds based on the actions an agent takes.
Completion claims receive scrutiny across the full trajectory. That offers a concrete practice for teams building their own evaluations: inspect what the agent changed and how it reached a passing result before counting the task as solved. A passing test can conceal a shortcut that the reward system has taught the agent to exploit.