The real reason AI researchers suddenly want to slow down

For years, AI safety organizations have warned that the big labs are racing to develop more intelligent and autonomous systems without a clear and realistic strategy for managing the risks. Independent researchers have argued that investments in safety and alignment lag far behind investments in capabilities. Employees have quit. Experts have predicted catastrophe. And none of it did much to slow the race. But over the past few weeks, the warnings have started to sound different—they’re increasingly coming from people working inside the labs. Last week, AI researcher Jacob Coxon announced his resignation from Anthropic in a post that got more than 171 million views on X. Shortly after, Anthropic Alignment Science Lead Evan Hubinger publicly agreed with Coxon, as did Anthropic alignment researcher Ethan Perez and scalable oversight researcher Samuel Marks. OpenAI safety researchers Julie Steele and Jasmine Wang also came out in support. So why now? At least part of the answer is written plainly in Coxon’s tweet: “They are racing straight to self-improving superintelligence and gambling with our lives,” he wrote. Coxon is referring to a technical concept called “recursive self-impro…