Amazon has told its engineers that counting how much they use AI is a poor way to measure their work. In a memo published Tuesday, the company said token usage, AI tool adoption and counts of code changes are "deeply flawed" ways to gauge how engineers work with AI. The memo came with an updated role guideline for software development engineers.
Terk Terkowitz, vice president and distinguished engineer for Worldwide Amazon Stores, signed the memo. Amazon first shared it with staff in September, CNBC reported.
“If your work does not benefit from current AI tooling, that is a finding, not a failing,” Terkowitz wrote, according to the memo.
Why is Amazon warning engineers against tokenmaxxing?
The memo targets a practice the company calls tokenmaxxing, which it describes as using AI as much as possible with little regard for output. Amazon has used input metrics, including token usage and AI tool adoption, and has also counted code changes and reviews. The memo warns that if teams treat those numbers as targets rather than measures, costs and complexity could rise.
The memo ties its position to Amazon’s leadership principles. It says tokenmaxxing is at odds with frugality, and that measuring lines of code works against the principle of inventing and simplifying.
The shift follows an earlier problem. Amazon shut down an internal leaderboard in May after finding that employees were tokenmaxxing to climb the rankings, CNBC reported. An Amazon spokesperson told CNBC the company has not capped employees’ token usage.
What AI usage metrics did Amazon track before the memo?
Business Insider reported that Amazon had tracked engineers’ AI usage, token consumption and lines of code, and had used leaderboards to rank some teams. According to the outlet, those programs created friction among some employees.
Does Amazon still use AI usage metrics?
The memo does not tell engineers to abandon the metrics altogether. It says they correlate well with the outcomes that matter, and that engineers should use them with care and good judgment.
Amazon calls its most AI-intensive groups frontier teams. The memo says those teams show higher productivity and delivery, while teams and individuals who are not adopting AI are falling behind. It asks managers to study the bottom 10% of teams by AI adoption to find out why, and to check whether the top 10% are delivering more to match.
Leaders are also expected to use the tools on production work themselves. They should be able to describe one task where AI helped and one where it did not.
“We are certain that whatever future best practices are, AI will be at the center of software engineering practice,” the memo states.
Marc Brooker, a vice president and distinguished engineer at Amazon Web Services, told CNBC that the company has looked at measures such as feature velocity, which covers how many launches teams make and how many successful changes reach production. “What we don’t want to see is wasted effort where we are going and spending those tokens solving problems that don’t matter,” Brooker said.
Other companies have also changed how they handle AI spending this year. Microsoft set division-level AI budgets in August, and Uber’s tech chief said the tokenmaxxing era is ending. Accenture capped staff spending on Claude at $29 a month this month, 404 Media reported.
Dieser Artikel wurde mit Unterstützung von KI verfasst.
News Factory APP - agentische News für besseres SEO & AEO.