AI giant says its own models could soon improve themselves — and now it wants a global pause

Anthropic, the artificial intelligence company behind Claude, warned in a blog post that its own systems could eventually become capable of improving themselves. It even said the world should be ready to hit pause before that happens.

Anthropic said its Claude models are on a path toward what it calls "recursive self-improvement." That means AI can make itself better without direct human intervention.

The company said in the blog that the threshold has not yet been crossed but argued it "could come sooner than most institutions are prepared for."

"We believe it would be good for the world to have the option to slow or temporarily pause frontier AI development to enable societal structures and alignment research to keep up with the advance of the technology," Anthropic wrote.

At the same time, the company acknowledged how difficult it would be to carry out. Anthropic said any meaningful slowdown would likely require several well-funded labs working at or near the frontier in multiple countries to agree to halt under the same terms.

It also said such efforts would be hard to verify. AI training runs are much easier to hide than missile silos.

Some critics question whether a pause would also conveniently protect the company's current advantage.

The firms building the most powerful AI systems are increasingly shaping the rules in real time. If Anthropic is right, governments and regulators may have less time than expected to put safeguards in place around tools that could affect warfare, jobs, and public trust.

AI also has a direct connection to the energy grid. 

These systems can help forecast demand for utilities and improve battery and grid performance. But training and operating new models can require enormous amounts of electricity and water. They can strain local infrastructure and drive up costs for communities if demand rises too quickly.

Anthropic believes it is a cautious company. But critics have pointed to reports about its tools being used in military and intelligence contexts, arguing that its safety language does not always line up with its business decisions.

Not everyone is convinced by the company's warning.

NYU Professor Emeritus Gary Marcus said on Substack, "To some degree, the new Anthropic blog is a bait and switch: Anthropic is trying to strike terror into everyone's hearts … but all they have really shown is just faster coding — entirely under human control, along with progress on some benchmarks."

Marcus added, "A faster coding tool will probably not end the world."

University College London professor Steven Murdoch was also skeptical, telling The Guardian, "Anthropic might give the impression of being warm and fuzzy, but their definition of AI safety is narrow. Supporting U.S. authorities in the development of offensive capabilities has never been something they have spoken against."

Anthropic said it plans more discussions with policymakers, researchers, civil society groups, and other AI companies in the months ahead.

Get TCD's free newsletters for easy tips, smart advice, and a chance to earn $5,000 toward home upgrades. To see more stories like this one, change your Google preferences here.