Anthropic researcher warns of double-digit AI extinction risk this decade

Jacob Coxon, described as a former pretraining researcher at OpenAI and Anthropic, has left Anthropic and accused both companies of knowingly risking human extinction. The Decoder reports that Anthropic colleague Evan Hubinger puts the odds of a misaligned superintelligent AI wiping out humanity within the next decade at more than ten percent. The claim is not presented as a general industry consensus, but it highlights how seriously some frontier-lab researchers say they view the downside of rapid AI scaling.

Coxon argues that current AI systems are approaching superhuman capabilities and says the pace of progress is not slowing. He claims OpenAI and Anthropic are not acting responsibly, and that public messaging is softened compared with private concern. According to the article, Coxon sees Anthropic as believing it must stay in the race because other labs may not act responsibly, a position he calls a “hubristic gamble.”

The debate centers on whether future systems can be reliably aligned before they gain much greater autonomy and capability. The article says fears focus less on today’s models than on RSI, a process where AI models optimize themselves, which supporters hope could accelerate progress but critics fear could create runaway behavior. Coxon and other concerned researchers point toward international coordination, pace agreements, and potentially costly actions such as a temporary ban on pushing model capabilities further.

The Decoder notes that whether RSI is possible with current technology remains disputed, with skeptics and proponents making different cases. It also reports pushback from AI researchers who argue that extreme pessimism can leave people helpless or depressed rather than focused on solutions. The practical takeaway is to separate three questions: how fast capabilities are advancing, whether alignment and monitoring are adequate, and what governance mechanisms could slow or redirect risky development without relying on trust alone.