# Amazon’s AI Metrics Dilemma

- Date: 28 Oct 2026 (2026-10-28T17:06:45.000Z)
- Summary: Amazon’s updated engineering guidance says an AI tool’s failure to help is a finding, not a personal failing. But it still endorses adoption and usage metrics, exposing the tension between measuring activity and judging outcomes.
- Tags: `digest`, `ai-discourse`, `amazon`, `ai-adoption`, `software-engineering`, `productivity-metrics`, `engineering-management`

## Sources

1. [About Amazon - An update from Amazon engineering leadership on the role of the developer in the age of AI](https://www.aboutamazon.com/news/company-news/amazon-software-engineer-role-guideline-ai) (website)
2. [Simon Willison - A quote from Terk Terkowitz](https://simonwillison.net/2026/Oct/28/terk-terkowitz/) (website)

## Executive Summary

Engineers should be able to report that AI does not help their work without that finding becoming evidence of personal failure. That is the strongest principle in Amazon’s updated guidance for software developers—but it sits alongside continued approval of AI-adoption and usage metrics as imperfect performance indicators.

The consequential tension is inside the policy itself: Amazon wants engineers judged by customer and business outcomes, while acknowledging that those outcomes are harder to measure than tokens, tool adoption, or code throughput. Its [published engineering-leadership memo](https://www.aboutamazon.com/news/company-news/amazon-software-engineer-role-guideline-ai) offers a concrete test of outcome-focused AI management, not proof that the measurement problem has been solved.

## What Happened

[Simon Willison highlighted the memo on October 28](https://simonwillison.net/2026/Oct/28/terk-terkowitz/), quoting Terk Terkowitz, VP and Distinguished Engineer in Worldwide Amazon Stores:

> “If your work does not benefit from current AI tooling, that is a finding, not a failing.”

The full memo gives that sentence important context. Amazon engineering leaders revised the software development engineer role guideline around what they consider durable: impact on customers, businesses, and teams. They explicitly avoid defining engineering excellence through rapidly changing tools and techniques.

This is not permission to ignore AI. The memo urges developers to learn and apply it, says AI will be central to future engineering practice, and expects leaders to gain hands-on experience. A leader should be able to describe both a production task where the tools helped and one where they did not, and explain that judgment to the team.

The distinctive expectation is therefore informed experimentation rather than unconditional enthusiasm. An engineer reporting a poor fit is asked to document why, share the finding with peers, discuss it with a manager, and seek solutions.

## The Measurement Contradiction

Amazon’s memo openly names the weaknesses of common proxies. Token usage and tool adoption measure inputs; code changes and reviews measure throughput. Turning them into targets can increase costs and complexity without improving the outcomes that matter. Its language is unusually direct: “tokenmaxxing is at odds with frugality,” while measuring lines of code conflicts with simplification.

Yet Amazon does not discard these indicators. The memo says they correlate with useful outputs and endorses using them with care and judgment. It recommends investigating both low-adoption teams and heavy users, including whether the latter are seeing delivery improvements that match their usage.

That makes this a qualified defense of proxy metrics, not their abolition. The published text supplies no numerical results or study design with which to assess the claimed correlation. It should be read as leadership’s account of its observations, not independent evidence that more AI use causes better delivery.

The practical distinction is between a metric that prompts investigation and one that settles a performance judgment. The memo recognizes the danger of confusing them; the document alone cannot establish how consistently that distinction will survive in management practice.

## Why It Matters

The important contribution is the explicit legitimacy of negative findings. If every unsuccessful AI experiment is treated as resistance or incompetence, employees have an incentive to conceal failures or demonstrate activity instead of value. Recognizing a well-explained limitation as engineering judgment makes honest evaluation easier—provided managers honor it.

This reinforces a useful principle for assessing AI progress: capability, adoption, and dependable work are different things. A tool can be impressive and heavily used without improving a particular task. Conversely, a developer who rejects it for a specific job may be exercising exactly the judgment an organization needs.

The evidence here is one substantive leadership statement, not a demonstrated industry-wide shift or a measured productivity result. Its significance is narrower and concrete: Amazon has publicly articulated both the case against AI activity targets and the reasons it still uses activity indicators. Whether outcomes genuinely outrank those proxies is the unresolved test.