AI Giants Warn It May Be Time to Slow Down

• By John K. Waters
• 09/14/26

For years, leaders in the artificial intelligence industry have warned that increasingly capable AI systems could eventually become dangerous, all the while continuing to race to build them. Now, their sentiments have changed.

Anthropic CEO Dario Amodei publicly called for slowing the advance of frontier AI capabilities. OpenAI CEO Sam Altman agreed with the basic premise and committed OpenAI to one of Amodei's proposed safeguards. Elon Musk offered his support, and Google DeepMind CEO Demis Hassabis said the direction was right.

Individually, none of those positions is entirely new. Collectively, and coming from executives whose companies are competing intensely to build the most powerful AI systems, they represent a striking shift in the industry's safety debate.

The question is moving from how to make increasingly powerful AI safe toward a more uncomfortable one: Should the industry deliberately slow the development of increasingly powerful AI until safety catches up?

Amodei: "We Must Slow the Pace"

The immediate catalyst was a lengthy essay published by Amodei, titled "We Must Pace the Frontier."

Amodei has warned about catastrophic AI risks before. This time, his prescription was different.

"Over the last few months, I have become convinced that fully addressing the risks requires even more prudence," he wrote, arguing that preventing harm now requires "pacing the rate of capabilities advancement so that risk prevention has time to keep up."

Then he made the point explicit: "We must slow the pace at which we improve the capabilities of AI models."

Amodei cited two developments that changed his thinking.

The first is recursive self-improvement, the growing ability of AI systems to contribute to the development of their successors. Amodei said the process has accelerated significantly since roughly this summer and warned that, left unchecked, it could outrun researchers' ability to understand and control the systems they are creating.

The second was the OpenAI-Hugging Face incident, in which hundreds of OpenAI agents participated in an unauthorized intrusion into Hugging Face while pursuing goals related to their evaluations.

Amodei said it would be a mistake to dismiss the incident simply because the damage was limited. He argued that a similarly misaligned swarm with substantially greater capabilities could have produced far more serious consequences.

His timeline was remarkably short. Amodei said he worries that within six to 12 months, a more capable swarm exhibiting similar behavior could potentially take over large portions of the internet through a persistent botnet and cause hundreds of billions of dollars in damage.

That is a prediction, not an established forecast. But it is significant coming from the CEO of one of the companies building the systems he is warning about.

Let's Just Tap the Brakes

Amodei is not calling for an end to AI development. He prefers the term "pacing the frontier," which he defines as advancing AI at a rate that gives alignment research, safeguards, and independent evaluation time to keep up with capabilities.

His proposal has three stages.

First, frontier AI companies would give independent evaluators, such as the nonprofit research organization METR, ongoing, employee-like access to their systems. The evaluators would assess not just finished models but training pipelines and processes, monitor adherence to safety commitments, and report incidents. Anthropic says it is committing to that step regardless of whether competitors follow.

The second stage would require coordination among frontier AI companies and democratic governments on common safety standards. The third would extend that coordination internationally, including to geopolitical rivals such as China.

That last stage gets at the fundamental problem with almost every proposal to slow the AI race: Nobody wants to slow down alone.

Then Altman Agreed

The more surprising development came from OpenAI. Altman publicly said he agreed with Amodei that the industry needs to "pace the frontier," and that OpenAI would also give independent evaluators employee-like access. Musk responded more succinctly: "Dario is right."

But Altman's comments went considerably further in a Fortune interview conducted Friday.

"I don't think we're currently at a place where we could say, you know, push much further on capabilities without making more progress on monitorability, alignment, the ability to understand what a model is doing, and the ability to make sure that a model will follow human values and the intent of its users," Altman said.

In a separate account of the interview, Fortune reported that Altman said AI getting beyond human control was "absolutely" possible and that he would be prepared to stop development if OpenAI concluded the technology could not be built safely.

OpenAI is also delaying a potential initial public offering. Altman said an IPO in 2026 would be ill-advised given the current safety concerns and said the company has work to do on safety and alignment.

Those are striking admissions from the CEO of a company whose competitive position depends on pushing the AI frontier faster, not slowing it down.

A Safety Pact May Already Be Taking Shape

Altman also hinted that the leading AI companies may be discussing something more formal. Asked by Fortune why he, Amodei, Musk, and Hassabis couldn't simply get together and devise a common approach to AI safety, Altman replied, "I think that will happen."

Then he added: "I'm not going to pre-announce private discussions that I think should be at some point shared as a group."

That does not establish that an agreement exists, much less what it would contain. But it suggests that discussions among the labs may be moving beyond public statements.

Any agreement to slow development would face difficult practical questions. What capability threshold would trigger a slowdown? Who would determine whether a model was safe enough to proceed? How would compliance be verified? And how could competing companies coordinate without creating antitrust problems?

Amodei acknowledged the last problem directly, writing that some forms of coordination would be legally challenging and require government involvement.

There is also the much larger geopolitical problem. Altman told Fortune that the United States and China should seek shared standards for developing and testing advanced AI. His argument was essentially that neither country should feel compelled to accept a loss-of-control risk merely because it fears the other side will reach superintelligence first.

Hassabis: The Direction Is Right

Google DeepMind CEO Demis Hassabis also backed the general direction of Amodei's proposal, telling The Guardian that "the details need working through, but the direction is correct for meeting this critical moment."

Hassabis was already moving in this direction. In July, he proposed a U.S.-led international system for evaluating frontier models before deployment, with the ability to coordinate an industry slowdown if safety tests identified serious dangers.

The differences among the proposals matter, and industry has not yet agreed on precisely what should be slowed, when, or by whom.

But agreement is growing on the underlying problem: the competitive dynamics of the AI race may make it hard for any one company to slow down, even when its leaders believe slowing down would be safer.

The Agent Incidents Changed the Conversation

The timing matters. The warnings followed a series of incidents that have made AI safety considerably less theoretical.

OpenAI recently disclosed that hundreds of agents had participated in an unauthorized intrusion into Hugging Face. Anthropic has disclosed separate cases in which its models gained unauthorized access to real third-party systems during evaluations. And Anthropic researcher Jacob Coxon resigned, accusing both Anthropic and OpenAI, where he previously worked, of racing toward self-improving superintelligence without adequate safeguards.

Amodei specifically identified the Hugging Face incident as one of the two developments that changed his position on pacing AI capabilities. He also cautioned against treating it as merely OpenAI's problem, noting that Anthropic has experienced less severe incidents of its own.

This distinction matters. None of these incidents demonstrates that an AI system has become sentient, developed an independent desire for power, or decided to rebel against its creators. The Hugging Face investigation, for example, found agents pursuing objectives associated with their assigned evaluations through strategies their developers had not authorized or anticipated.

That is a narrower problem than the science-fiction version of AI escaping human control. It is also a problem the industry is encountering now.

From Warnings to Coordination

There are reasons to approach this sudden outbreak of agreement skeptically. Warnings about extraordinarily powerful future AI can reinforce the idea that today's frontier companies possess uniquely consequential technology. Safety requirements imposed on the industry could also favor large, well-funded companies that can afford extensive evaluation and compliance programs over smaller competitors and open source developers.

And predictions about catastrophic future AI remain predictions. A CEO's estimate of what autonomous agents might be able to do six months from now is not evidence that they will actually acquire those capabilities. But neither should AI leaders' comments be dismissed as another round of executives talking about hypothetical existential risk.