The researcher who quit Anthropic says they really do think AI could kill everyoneThe AI safety vibe shift

A researcher named Jacob Coxon, who spent three years doing pretraining work at both OpenAI and Anthropic, resigned and posted about it: neither company is acting responsibly, he wrote, they are racing straight to self-improving superintelligence and gambling with our lives. The post drew 159 million views.

The author had grown numb to this kind of resignation. Anthropic was founded by people who thought their former colleagues at OpenAI took safety too lightly; in 2022 it was described as the white-hot centre of AI doomerism. Another safety researcher had quit in February.

What actually set the discourse on fire

Anthropic's head of alignment science, Evan Hubinger, quote-posted it: Jacob is correct, we really do earnestly believe AI could kill all humans, and he personally puts it above 10% within the next decade. That added another 41 million views.

Hubinger had been saying this for years. Back in 2022 he said that once the model thinks it can kill us, it just murders us. Nobody cared then, because Anthropic was not yet a $965bn company heading toward what could be the largest IPO ever.

Congress seems to be listening this time

Senator Bernie Sanders and Representative Greg Casar introduced the Ban Artificial Superintelligence Act, which would bar companies from developing superintelligence and pause advanced AI work. On the Republican side, Josh Hawley opened a probe into the Hugging Face attack, calling OpenAI's handling reckless, and Ted Cruz is reportedly drafting legislation too.

Both parties moving suggests public opinion has turned. Safety advocates also took heart from Paul Christiano, an influential alignment researcher and government adviser, joining the OpenAI Foundation's board and its safety committee, which governs whether the for-profit arm can release new models.

The author is cautiously hopeful: posts and probes are no substitute for regulation, and Congress has a long history of tech legislation that goes nowhere. But researchers who quit jobs that would have made them rich are now being heard by the right people, and the hope is that those people act.