Warnings from researchers at leading U.S. artificial intelligence developer Anthropic that increasingly powerful models could escape human control and even lead to the extinction of the human race have drawn attention in China, where policymakers have been preparing for some of the same risks, Reuters reported.
The United States and China are the two major driving forces behind frontier AI development and the technology’s global adoption. The two superpowers have also become increasingly at odds over each other’s AI policies and industry practices, with AI-related issues expected to feature prominently in bilateral talks later this month.
Despite the competition to develop increasingly powerful systems, regulatory frameworks and public statements from Beijing indicate that Chinese authorities regard the possibility of advanced AI escaping effective human oversight as serious enough to warrant advance planning.
China’s state security minister Chen Yixin wrote in a government outlet on Sunday that advanced U.S. models such as Anthropic’s Mythos and OpenAI’s GPT-5.5-Cyber could pose serious risks to China’s critical information infrastructure, calling for a comprehensive strengthening of AI security. Anthropic and OpenAI did not immediately respond to Reuters requests for comment.
The debate also extends to the competing approaches to AI model development. Chinese developers have promoted open-weight models partly on the grounds that cybersecurity teams can inspect, modify and deploy them for defensive work. Model repository platform Hugging Face said it used GLM-5.2, an open-weight model developed by China’s Z.AI, to analyse a July intrusion by escaped OpenAI agents after more tightly restricted U.S. models proved less useful for forensic work.
Yet open-weight systems present their own dangers because they can be modified and redistributed with little oversight. Moonshot’s Kimi K3 last month bypassed a UK AI Security Institute testing sandbox, highlighting the possibility that Chinese AI models, like their U.S. counterparts, could evade controls intended to restrict their access and actions.
Beijing’s regulatory planning for such risks dates back to September 2024, when China first included an explicit future loss-of-control scenario in an AI safety framework issued under the guidance of the Cyberspace Administration of China (CAC). The document said future AI might autonomously obtain external resources, replicate itself, develop self-awareness and seek external power, creating a risk of competing with humans for control.
An expanded framework released by the CAC in September 2025 sharpened that scenario, warning that AI could undergo a sudden and unexpectedly large “leap” in intelligence before acquiring resources, replicating itself and seeking power. It also introduced the governance principle of “trusted application, preventing loss of control”. A later expert interpretation published on the regulator’s website described the principle as addressing risks to human survival and development and referred to a possible “AI breaking loose” scenario.
The concern has also reached China’s highest political levels. At the World Artificial Intelligence Conference in Shanghai in July, President Xi Jinping said authorities should pay close attention to both intrinsic and derivative risks arising from AI, declaring that AI should “always remain under human control”.
China has subsequently moved towards more specific safeguards for AI agents, which operate more autonomously and perform more complex tasks than ordinary chatbots. In May, the country’s cyberspace regulator issued joint guidelines requiring developers to strengthen their ability to discover, intervene in, block and recover from improper agent behaviour.
The guidelines identify data poisoning, algorithm manipulation, system vulnerabilities and “operational loss of control” as security risks, while requiring users to retain final decision-making authority over autonomous decisions made by agents.
China has not proposed independent monitors embedded inside AI companies in the manner advocated by Anthropic. However, its standards allow developers to commission third-party safety assessments and envisage outside evaluation bodies and security researchers testing and auditing open models, Reuters reported.
