Sunday, October 18, 2026

Latest

Anthropic researcher puts AI extinction risk above 10% within next decade

Human extinction news
AI-generated image.

A senior researcher at Anthropic has estimated there is a greater than 10% chance that artificial intelligence could cause humanity’s extinction within the next decade, warning that developers still lack a reliable method for keeping future superintelligent systems under human control.

Evan Hubinger, whose research focuses on aligning advanced AI with human objectives, made the assessment following the resignation of fellow Anthropic researcher Jacob Coxon, who raised concerns about the speed at which major technology companies are pursuing increasingly capable systems.

“We really do earnestly believe AI could kill all humans,” Hubinger wrote on X on Wednesday. “I personally think it is >10% within the next decade.”



Hubinger said Anthropic was “trying its best” to address the danger but acknowledged the company does not currently have a clear solution for safely controlling superintelligence and is “not clearly on track” to develop one.

He distinguished existing AI systems from potentially much more powerful future models, saying present-day technology carries relatively little existential danger. The greater concern, he argued, is that future AI could acquire the ability to repeatedly improve its own capabilities, potentially accelerating beyond human intelligence before effective safeguards are established.

The comments followed Coxon’s departure from Anthropic on Tuesday. Coxon, who previously worked at OpenAI, accused leading AI developers of failing to adequately account for the “civilizational stakes” involved in developing self-improving systems.

He claimed the industry was “racing straight to self-improving superintelligence and gambling with our lives.”

“The people building AI earnestly believe that it could kill us all by the end of the decade,” Coxon wrote, arguing that such warnings were “not a marketing stunt.”

Speaking to the Wall Street Journal, Coxon suggested that under particularly aggressive development scenarios, AI could become “out of control” as early as the end of 2027.

He also argued that competitive pressure creates a fundamental problem for AI safety: even companies investing heavily in safeguards may be reluctant to slow development if they believe rival developers will continue advancing.

Anthropic and OpenAI have both publicly acknowledged the potential dangers associated with increasingly capable artificial intelligence and say substantial resources are being devoted to safety research. Leaders from the companies have also supported greater coordination between governments and the industry, including mechanisms that could slow development if serious risks emerge.

At the same time, major AI developers continue to release increasingly sophisticated models and autonomous agents, fuelling debate over whether safety measures are keeping pace with advances in capability.

Recent research has demonstrated AI agents performing potentially dangerous tasks during controlled evaluations, including generating malicious computer code, creating false identities and attempting to manipulate people. Researchers have also explored the ability of AI systems to assist with the design of functional viral genomes, adding to concerns about the potential misuse of increasingly powerful technology.

Hubinger’s warning highlights the widening debate within the AI industry itself over whether developers can create systems substantially more intelligent than humans while retaining effective control over how those systems behave.

Support DTNZ

DTNZ is committed to bringing Kiwis independent, not-for-profit news. We're up against the vast resources of the legacy mainstream media. Help us in the battle against them by donating today.

Promoted Content

No login required to comment. Name, email and web site fields are optional. Please keep comments respectful, civil and constructive. Moderation times can vary from a few minutes to a few hours. Comments may also be scanned periodically by Artificial Intelligence to eliminate trolls and spam.

3 COMMENTS

LEAVE A REPLY

Please enter your comment!
Please enter your name here

Wellington
clear sky
15.9 ° C
15.9 °
13.3 °
66 %
4.1kmh
4 %
Sun
16 °
Mon
16 °
Tue
16 °
Wed
18 °
Thu
16 °




Sponsored



Latest

Trending

Sport

Daily Life

Opinion