Jacob Coxon, a prominent AI researcher who previously contributed to both OpenAI and Anthropic, has publicly announced his resignation from the artificial intelligence industry. His departure is accompanied by a stark warning regarding the unchecked and accelerated development of AI, which he believes presents a significant, potentially existential, threat to humanity.
A Dire Warning from Within the Industry
In a detailed post on X (formerly Twitter), Coxon confirmed his departure from Anthropic, leveling strong criticism at both his former employer and OpenAI. He accused these leading AI firms of "racing straight to self-improving superintelligence and gambling with our lives." Coxon emphasized the profound capabilities of the technology, stating, "Do not underestimate the power of this technology," and highlighted its potential to "revolutionize any field overnight and acquire real power and resources."
Coxon's concerns, he revealed, are not isolated but stem from private discussions with AI executives and fellow researchers who, he claims, acknowledge the possibility of catastrophic outcomes. He quoted these individuals as earnestly believing that AI "could kill us all by the end of the decade," adding that "No other human activity poses this level of danger."
The Paradox of the AI Arms Race
The former Anthropic researcher delved into the perplexing reasons why AI companies continue their rapid development despite recognizing the inherent risks. He suggested that OpenAI might not "deeply internalize the civilizational stakes." Conversely, he indicated that Anthropic's researchers, while acutely aware of the dangers, feel caught in an "AI arms race." They operate under the belief that they must develop powerful AI systems themselves because other entities will not act responsibly. Coxon critically labeled this rationale a "hubristic gamble."
Call for Greater Oversight and Caution
In light of these pressing concerns, Coxon urged his peers in the research community to speak out. He called for demands for more robust safety measures, increased regulatory oversight, and stronger guarantees before the continued development of advanced AI systems. He even proposed that a temporary ban on improving model capabilities might be essential to prevent a global, uncontrolled race with potentially devastating consequences.
Anthropic Leadership Corroborates Fears
Significantly, Evan Hubinger, Anthropic's Alignment Science Lead, publicly supported Coxon's assertions. Hubinger explicitly stated, "Jacob is correct here—we really do earnestly believe AI could kill all humans!" He personally estimated a "greater than 10 percent" chance of such an outcome within the next decade. While Hubinger affirmed Anthropic's efforts, he conceded that the company currently lacks a clear plan to achieve alignment for superintelligence. He clarified that his primary concern lies with future, self-improving AI systems, rather than the capabilities of current models.