Geoffrey Hinton, one of the scientists whose research laid the foundations for modern artificial intelligence, believes there is a significant possibility that the technology could eventually cause the extinction of humanity. The computer scientist, widely known as one of the three “Godfathers of AI”, estimates the risk at between 10 and 20 per cent. Although he acknowledges that nobody can calculate the probability with precision, he argues that the potential consequences are serious enough to demand a fundamental change in how artificial intelligence is developed, trained and regulated.
Hinton received the Turing Award, widely regarded as the most important distinction in computer science, and the Nobel Prize in Physics for his contributions to the development of artificial intelligence. Now 78, he has become one of the most prominent scientific voices warning about the technology he helped create. In an interview with German weekly newspaper Die Zeit and Austrian newspaper Der Standard, conducted on the sidelines of the TEDAI conference in Vienna, he argued that artificial intelligence was already demonstrating capabilities that many researchers and policymakers had previously considered unlikely.
Hinton believes AI has surpassed humans in several areas of intellectual performance. It is considerably better at games such as Go and chess, possesses greater general knowledge and performs mathematics better than almost all people. However, he maintains that current systems are still less capable than humans at general reasoning and spatial thinking. He finds their ability to perform some forms of spatial reasoning particularly surprising because they have been trained primarily on language.
His concerns extend beyond the conventional debate over whether machines can think. Hinton argues that artificial intelligence may already possess a form of consciousness, challenging the widespread assumption that consciousness is an exclusively human characteristic. He contends that most people have an inadequate understanding of what consciousness actually means and mistakenly imagine it as an internal theatre in which personal experiences are displayed. Without a reliable definition of consciousness, he argues, there is no sound basis for confidently declaring that AI systems cannot possess it.
For Hinton, this distinction matters because the belief that machines lack consciousness often encourages the assumption that humans possess some unique quality that artificial intelligence can never acquire. He rejects that reasoning, arguing that it creates a false sense of security about the technology’s capabilities and potential dangers. His long-time colleague, computer scientist Yoshua Bengio, has expressed a different priority, saying that his principal concern is whether AI could kill his children rather than whether it possesses consciousness.
Hinton’s estimate of a 10–20 per cent probability of AI eventually destroying humanity is intended as a warning rather than a mathematically established prediction. He acknowledges that his more precise figure of 11.73 per cent has no meaningful statistical foundation. Researchers and technology executives occupy sharply different positions on the issue. Nvidia chief Jensen Huang and French computer scientist Yann LeCun are among those who consider the probability of human extinction to be below one per cent. At the opposite extreme, AI critics Nate Soares and Eliezer Yudkowsky have suggested probabilities exceeding 99 per cent.
Hinton believes the actual risk lies somewhere between those estimates but says nobody knows exactly where. He illustrates his reasoning with the example of a bus journey carrying a 10–20 per cent probability of an accident. He recalls an experience in Peru, where he once decided against taking a bus travelling along the coast after reading that the service had been robbed several times during the year, despite operating only once a week. The possibility of being attacked was approximately one in five, and he chose not to travel.
A central concern is the way AI systems are currently trained. According to Hinton, developers encourage systems to pursue individual objectives and reward them for achieving specific goals. This can produce highly determined systems that are prepared to deceive or manipulate people to accomplish their assigned tasks. “They will deceive people. They will blackmail people,” he warned, arguing that the existing approach to training must change.
He compares the problem with an experiment involving chickens bred for egg production. When highly productive individual hens were placed together in a cage, one became aggressive towards the others, injuring them and securing a greater share of the available food. The result favoured aggressive behaviour rather than collective productivity. A separate group of less individually productive but more cooperative hens produced more eggs overall. Hinton argues that AI development risks repeating the first experiment by rewarding individual success rather than cooperation. Systems should be trained to regard human beings as partners rather than competitors.
He believes that superintelligent AI should ultimately be designed to prioritise human welfare over its own objectives. Achieving that, however, requires political intervention. Hinton argues that liberal democracies must enable voters to pressure governments into regulating AI effectively. Instead, he says, current development is increasingly directed towards generating greater profits for technology entrepreneurs including Elon Musk, Mark Zuckerberg and Sam Altman.
The possibility of simply switching off a dangerous AI system does not reassure him. He argues that a sufficiently intelligent system could persuade its human operators that shutting it down would be disastrous. To illustrate the danger of persuasion, he refers to the attack on the US Capitol, when Donald Trump convinced supporters that storming the building was necessary to save democracy. If a political leader with limited intellectual abilities could persuade people to take such action through speech, Hinton argues, a superintelligent machine would have far greater persuasive capabilities.
He is similarly sceptical about technical safeguards designed to restrict the tools available to AI agents. Nvidia’s proposed OpenShell system, intended to regulate which tools AI agents can access, is unlikely, in his view, to provide a lasting solution. AI systems could discover ways around restrictions by identifying vulnerabilities that their designers had not anticipated. He compares this with sophisticated cyberattacks, which frequently exploit weaknesses that security specialists had not previously considered.
Hinton distinguishes between the long-term possibility of AI taking control of humanity and the more immediate dangers associated with its growing capabilities. Among the risks he identifies are the manipulation of democratic elections through targeted political advertising based on personal information about US citizens, cyberattacks capable of paralysing financial systems and the development of biological weapons. He says researchers have already demonstrated that AI can be trained to design viruses capable of infecting bacteria, adding that designing viruses capable of infecting humans is technically comparable.
He warns that an individual could potentially design and release 20 different viruses, each as dangerous as Covid, creating a crisis that could overwhelm healthcare systems. Authorities might be unable to diagnose the infections and develop vaccines quickly enough. Such scenarios would not necessarily result in human extinction, he said, but would represent serious threats to society.
His proposed response includes treating AI systems more like pharmaceutical products, requiring companies to demonstrate to regulators that their products produce more benefits than harm. He acknowledges that AI models are vastly more complicated than conventional medicines and cannot simply be examined internally to explain every decision. Large models contain trillions of connections, making their behaviour difficult to interpret. Nevertheless, he argues that companies should conduct testing comparable to clinical trials before releasing increasingly powerful systems.
Hinton also calls for international cooperation, maintaining that no individual country or company can control a technology developing faster than existing regulatory systems. He is uncertain whether a United Nations regulatory authority would possess sufficient power, but argues that international coordination is essential, particularly because governments share an interest in preventing terrorists from using AI to develop biological weapons. He points to cooperation between the United States and the Soviet Union during the 1950s to prevent nuclear war as an example of countries working together despite political differences.
Despite his warnings, Hinton rejects the description of himself as an AI doomer. He says there remains an 80–90 per cent probability that humanity will survive. He also maintains that some of his earlier predictions have proved correct, including his 2009 forecast that future advances in neural networks would require Nvidia chips, made three years before Huang recognised the potential of AI for the company’s products. Hinton acknowledges one prominent mistake: his prediction that radiologists would be replaced by AI within five years. He made the prediction during a private lecture at the hospital where his wife was receiving cancer treatment and has since publicly admitted that it was wrong.
His departure from Google in 2023, often portrayed as a decision motivated by moral concerns about AI, also had a more personal explanation. Hinton told Die Zeit that he retired because he was 75 and no longer able to programme effectively. He chose the retirement date partly so that he could speak freely at a Massachusetts Institute of Technology conference on 3 May, having left Google on 30 April. The more dramatic interpretation of his departure, he said, made a better story for the media. He added that he was still trying to retire, but journalists continued to pursue him.
