The underwhelming results of AI performance metrics should surprise exactly no one

Somebody at Meta built an internal leaderboard called Claudeonomics. It ranked the company’s top 250 consumers of AI tokens and handed out titles like “Token Legend” and “Cache Wizard.” Engineers set agents running for hours to climb it. In one thirty-day stretch, employees on the dashboard burned through more than 60 trillion tokens. Meanwhile, a Disney employee reportedly interacted with an AI assistant 460,000 times in nine days. JPMorgan, KPMG, Amazon, and Accenture have all been reported to be tracking employee AI activity, in some cases folding “AI-driven impact” into formal performance reviews. The internet named the resulting behavior tokenmaxxing. People quickly learned how to make their numbers look good. They generated token usage with work that really didn’t require AI. They used agents and digital delegates to execute processes that might, or might not, be useful. The denouement came swiftly. Meta’s people took the leaderboard down once it went public. Amazon reportedly shut one down too, with some observers saying that it incentivized the employees to cheat. Sigh. It seems that every generation or so we have to re-learn the painful lessons of previous attempts to mea…