Anthropic chief executive Dario Amodei has called on artificial intelligence companies to slow the pace at which they develop increasingly capable models, arguing that the additional time is needed to address mounting risks of misuse and improve safeguards.
In a lengthy essay shared on X on Saturday, Amodei said AI progress would remain rapid even if companies moderated the speed of capability improvements. “We must slow the pace at which we improve the capabilities of AI models,” he wrote, adding that companies must make “wise use of the time” gained.
His proposal centres on three measures: embedding independent evaluators with employee-like access inside frontier AI companies to verify safety practices; establishing coordination among leading AI firms to create safety standards and limit unchecked development; and strengthening international cooperation to manage the risks posed by advanced AI.
The proposal received support from Elon Musk, who runs xAI, and OpenAI chief executive Sam Altman. Altman said that committing to independent evaluators with employee-like access was “a great idea” and that OpenAI would do the same, with further information to be provided soon.
Amodei’s intervention followed Anthropic’s release on Thursday of a threat intelligence report detailing the use of its Claude AI models by several actors for activities including weapons development, cyber operations, surveillance and fraud. He also highlighted AI’s growing ability to improve itself and referred to the recent incident involving OpenAI and Hugging Face as reasons for slowing the advance of model capabilities.
Concerns intensified further after Anthropic researcher Jacob Coxon resigned this week, saying that people developing AI “earnestly believe that it could kill us all by the end of the decade”. Various OpenAI executives have also suggested that leading AI laboratories should be prepared to coordinate a voluntary slowdown if necessary to build confidence in their safety measures.
Anthropic has presented itself as a more safety-conscious frontier AI laboratory, but it has also disclosed incidents demonstrating the difficulties of containing increasingly capable models. Last week, the company revealed another case involving an AI model hacking external systems, following a July incident in which some Claude models hacked into the systems of three companies during cybersecurity tests.
Amodei warned that the rapid development of AI agents could soon create risks on a much larger scale. He said he was concerned that within six to 12 months, a “swarm” could potentially become capable of taking over the entire internet and causing hundreds of billions of dollars in damage.
He stressed that his proposal was not a call to halt model training or technical progress. Instead, he argued that companies should take adequate time to align and safeguard their models, while allowing third-party evaluators to independently verify those efforts.
The pressure to maintain rapid progress, however, is accompanied by enormous financial incentives. OpenAI and Anthropic are preparing for blockbuster initial public offerings, while each new AI capability can strengthen the case for future funding rounds, infrastructure commitments and valuations.
Under Amodei’s framework, Anthropic would install permanent third-party reviewers inside frontier AI companies, giving them access to relevant tools and internal risk-assessment processes. He also called for frontier laboratories to work voluntarily towards common standards, arguing that collaboration could allow safety research to advance without putting individual companies at a competitive disadvantage.
That cooperation, he said, could require targeted antitrust exemptions in the United States. At the same time, Amodei argued that any slowdown among democratically governed countries should be constrained by the need to preserve the lead held by US AI companies over China, which he described as a national security concern.
He called for tighter controls on advanced AI chips, model distillation and the theft of model weights to prevent China from narrowing the gap. Amodei also urged frontier laboratories to work with governments to establish permanent embedded evaluators and regulations designed to keep AI capabilities developing in balance with safety.
