Anthropic researcher Jacob Coxon is quitting the AI industry because he believes the race to create AI "superintelligence" is out of control, and only the government can slow it down.
I resigned from Anthropic today. I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives. More thoughts below.
— Jacob Coxon (@hilbertspaess) September 9, 2026
"Do not underestimate the power of this technology," Coxon writes. "These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources."
He adds, "We have all witnessed the progress in each of these domains, and progress is not slowing."
For those who think the danger is overblown or just a gimmick, Coxon speaks directly to them.
"The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible - but I hear the same people express fear privately. No other human activity poses this level of danger."
Coxon believes that "safety trade-offs are inevitable when companies are competing against one another and Chinese upstarts," according to the Wall Street Journal.
"Jacob is correct here—we really do earnestly believe AI could kill all humans!" writes Evan Hubinger, a colleague of Coxon's. "I personally think it is (greater than) 10% within the next decade," he said. "I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to."
OpenAI has spent the past week unveiling capabilities so dramatic that its own leaders — and industry titans like Nvidia CEO Jensen Huang — openly invoked the arrival of artificial general intelligence, or AGI.
OpenAI defines the term as AI that can outperform humans across most economically valuable work.
GPT-6 Astra, released Thursday and heralded as a "generational leap" toward AGI, demonstrated major gains across science, coding, cybersecurity and professional work.
Zoom out: That burst of progress is fueling an existential debate inside OpenAI over how far — and how fast — the AI race can safely go.
"Currently I believe that no lab has solved alignment and monitoring to a sufficient degree to continue responsibly scaling at maximum speed for much longer," Pachocki wrote in a blog post titled "An Alien Mind."
Coxon left OpenAI earlier this year because he believed Anthropic was a more responsible corporate citizen regarding its model safety efforts. But "he now believes no company can responsibly develop AI that can outperform humans in a range of tasks, sometimes called artificial general intelligence, absent government intervention or a coordinated industry slowdown."
Last April, I wrote this:
From my perspective, it's scary, exhilarating, awe-inspiring, confusing, and very, very worrisome. It's not just the dangers involved in a worst-case scenario; it's the very definition of "The Great Unknown." AI's most brilliant and driven men and women are pushing us into a future that even they have no idea what it will be like.
Anyone who isn't a little worried or scared about that needs to stop deluding themselves and start thinking about AI.
I added, "the government has to step in and set hard and fast guardrails that everyone can follow." In April, it was a suggestion. Now, it's an imperative. Forget Congress. This issue needs an executive order that supersedes the politically motivated state regulations and the mushy ideas coming from Congress.
No, China won't slow down. But we can deal with them down the road. This is the here and now, and we've got to do something about it.
Editor's Note: President Trump is leading America into the "Golden Age" as Democrats desperately try to stop it.
Help us continue to report on President Trump's successes and combat Democrats' lies. Join PJ Media VIP and use promo code FIGHT to receive 60% off your membership.
Join the conversation as a VIP Member