AI won't kill us all. Its bosses might.The politics and possibilities of ‘AI could kill us all’

A man named Jacob Coxon quit his job at Anthropic and, on the way out, posted that nearly everyone at OpenAI and Anthropic believes the systems they are building have a decent chance of killing every human on the planet before this decade is out.

The story ran in the Wall Street Journal, on Fox News, in WIRED. So people started asking the author: is AI about to end the world? His answer: no, don't worry about that. But that does not mean nothing is wrong.

What these people are actually afraid of

People inside AI have believed this for years. Some calculate their own probability of doom; 10 to 25 per cent is common. It is why Musk funded OpenAI in the first place.

Altman himself said the technology he was building posed an existential risk to humanity. Say that out loud and the press calls, and so do investors. So there is marketing in it, the author says. Not a Pepsi ad with a celebrity, but marketing.

What is actually worth fearing

Having read a good deal of AI extinction literature, the author has never found a step-by-step account of how a machine goes from improving itself to killing every last person. Most arguments run Jurassic Park in reverse: eventually it gets smart enough, and it finds a way.

What has actually happened: these companies let AI carry out felony hacks on other firms, handed the government autonomously generated kill lists, and sold technology explicitly meant to replace workers. That was not the machine having a mind of its own. People wrote it, tuned it and aimed it.

They hold unprecedented compute, they have inserted themselves into military kill chains, and they are run by egomaniacal billionaires racing to monopolise a new market. They get away with it because nobody has ever really punished them.

So make them punishable

Full model transparency is the floor. If these companies believe their products could kill people, the least the public deserves is to know how.

More important is liability. If Sam Altman could be charged with a felony for approving a hack, if Google paid defamation fines every time it published a false AI Overview, the models would likely behave themselves very quickly.

Illinois has already passed an AI liability law and Florida is considering one. Beyond that, the question is public control: a technology its own makers say might end the world cannot be governed by insiders alone.

In one line: the author is not losing sleep over a machine apocalypse, but Coxon's noise is a useful reminder of what these companies are doing, and how much political appetite there is for taking them on.