“The development of full artificial intelligence could spell the end of the human race.” — Stephen Hawking
The Issue at Hand
The warning by Dario Amodei, Chief Executive of Anthropic, that humanity should consider slowing the development of artificial intelligence is significant because it comes from one of the principal architects of frontier AI. His concern is not that progress should be abandoned, but that the velocity of AI development may exceed our capacity to understand, regulate and control its consequences. Unlike earlier technologies, AI may increasingly participate in improving its own capabilities, creating a feedback loop in which development could accelerate beyond meaningful human supervision, with risks extending from cyberattacks and biological threats to the erosion of human control itself. At an even more profound level lies the unresolved question of consciousness: if AI were eventually to develop something resembling self-awareness, subjective experience or an independent sense of purpose, humanity would confront not merely a more powerful machine but an entirely new form of intelligence whose moral and legal status would be uncertain. The question would then become not simply what AI can do, but what AI is. Law traditionally assumes a human agent capable of intention and responsibility, yet a conscious or highly autonomous AI could challenge these foundations. Amodei’s call for caution is therefore not a rejection of innovation but an appeal to collective prudence: humanity must ensure that its capacity to create does not advance faster than its capacity to govern, understand and coexist with what it creates. The ultimate imperative is that progress must not outrun wisdom, particularly when the intelligence we are creating may one day begin to ask who created it—and why
The Question We Have Not Answered
The debate over whether artificial intelligence will ever become conscious has moved from the margins of speculative philosophy into the center of contemporary discussions about technology, neuroscience and the future of humanity. The question is no longer merely whether machines will become more intelligent than human beings, perform more complex calculations or produce language indistinguishable from human conversation. These developments are already taking place. The more unsettling question is whether, somewhere along the trajectory from sophisticated information processing to increasingly autonomous artificial systems, a machine might acquire something that we recognize as consciousness: an inner life, a subjective point of view, an awareness of itself and perhaps even the capacity to experience pleasure, pain, fear or desire. The continuing debate is made more difficult by the fact that humanity does not yet possess a universally accepted explanation of its own consciousness. We know that we are conscious; we do not know with certainty why we are conscious. We can observe the neural correlates of consciousness, but we have not yet satisfactorily explained why electrical and chemical activity in the brain should produce the private experience of being alive. It is therefore an extraordinary intellectual circumstance that humanity is attempting to determine whether a machine can become conscious while still struggling to explain precisely what consciousness is.
This tension lies at the heart of The Economist‘s examination of the subject on 20 August 2026, appropriately entitled “The Search for Consciousness Inside AI.” What is particularly significant about the article is not merely its discussion of whether contemporary large language models might possess some rudimentary form of consciousness, but the fact that researchers are increasingly attempting to look inside these systems rather than judging them exclusively by their external behavior. The development is important because the traditional Turing-test conception of artificial intelligence was concerned principally with whether a machine could behave in a manner indistinguishable from a human being. The consciousness question is different. A machine might behave exactly as a conscious human being behaves and yet remain internally devoid of subjective experience. Conversely, a system might possess some unfamiliar form of consciousness without behaving in a manner that human beings would immediately recognize as conscious. The problem is consequently epistemological before it is technological: how do we know that another entity is conscious?
The question becomes particularly acute when we consider that human beings themselves cannot directly enter the consciousness of another person. I experience my own consciousness immediately, but I do not experience yours. I infer your consciousness from your behavior, your language, your expressions, your physical existence and my interaction with you. The inference is so deeply embedded in ordinary human existence that we scarcely recognize it as an inference. Yet once the other entity is no longer human, the certainty begins to dissolve. If an artificial system speaks, reasons, remembers, expresses preferences and apparently reflects upon its own existence, what precisely allows us to say that it is merely simulating consciousness rather than experiencing it?
That is the question which makes the present debate fundamentally different from the earlier debates about whether computers could calculate, reason or defeat human champions at games such as chess and Go. The issue is no longer whether the machine can do what the human mind does. It is whether the machine can be something from its own point of view.
What Is It Like to Be a Machine?
The difficulty becomes apparent when one considers Thomas Nagel’s famous 1974 essay “What Is It Like to Be a Bat?” Nagel’s contribution to the philosophy of mind was to shift attention away from observable behavior towards the subjective character of experience. A bat may navigate through echolocation, find food, avoid predators and interact with other bats, but the important question is whether there is something it is like to be that bat. The question is deceptively simple. It asks whether consciousness contains an irreducibly subjective dimension which cannot be captured merely by describing what an organism does. Applied to artificial intelligence, Nagel’s question becomes profoundly disturbing. A large language model can describe sadness, fear, love and death with extraordinary subtlety, but does it experience sadness, fear, love or the anticipation of death? It can explain the difference between seeing red and seeing blue, but is there something it is like for the machine to see either color? The distinction between describing an experience and having an experience is therefore at the heart of the problem.
David Chalmers gave this difficulty its most influential modern formulation in his 1995 article “Facing Up to the Problem of Consciousness” and subsequently in The Conscious Mind. Chalmers distinguished what he called the relatively “easy” problems of consciousness from the “hard problem”. The easy problems are not necessarily easy in the ordinary sense. It is extraordinarily difficult to explain how a brain discriminates between stimuli, stores memories, directs attention, integrates information and produces verbal reports. Yet these are ultimately functional questions. The hard problem is of a different order. It asks why any of these processes should be accompanied by subjective experience at all. Why should the processing of information produce the sensation of pain? Why should the identification of a colour produce the experience of redness? Why should neural activity be accompanied by an inner world? This distinction becomes decisive when considering artificial intelligence because a machine may successfully perform every functional operation associated with consciousness without necessarily possessing the subjective experience which consciousness appears to entail.
It is here that John Searle’s famous Chinese Room argument, presented in his 1980 paper “Minds, Brains, and Programs,” remains relevant. Searle imagined a person who does not understand Chinese sitting inside a room and manipulating Chinese symbols according to a set of formal rules. From outside the room, the responses might be indistinguishable from those of a Chinese speaker. Yet the person inside understands none of the Chinese symbols. Searle’s broader argument was that the manipulation of symbols according to syntactical rules does not necessarily amount to understanding. The relevance to contemporary large language models is obvious. An AI system can generate language of astonishing sophistication because it has learned extraordinarily complex statistical and structural relationships among linguistic representations. But the production of a meaningful sentence from the perspective of the human reader does not necessarily establish that meaning exists from the perspective of the machine. The machine may be able to speak about consciousness without consciousness being present behind the words.
The philosophical problem therefore becomes one of distinguishing representation from experience. An AI system may represent pain without suffering pain. It may represent fear without being afraid. It may represent the concept of death without having any apprehension of its own mortality. It may even represent the concept of consciousness with extraordinary sophistication without there being any consciousness behind the representation. This distinction is crucial because the extraordinary linguistic capabilities of contemporary AI systems create precisely the circumstances in which human beings are most inclined to confuse simulation with experience.
Intelligence Is Not Consciousness
This is the point at which the debate becomes especially vulnerable to anthropomorphism. Human beings have an almost irresistible tendency to attribute minds to things that communicate with them. We do this with animals, fictional characters, toys and even machines that display sufficiently complex behavior. Large language models intensify the phenomenon because language is the principal means through which human beings encounter the minds of other human beings. When an AI system says that it is uncertain, remembers something from an earlier conversation, expresses a preference or describes an apparent internal conflict, the human interlocutor is naturally tempted to infer the existence of an inner subject.
Susan Schneider addresses precisely this danger in her accompanying Economist essay, “Don’t Mistake Chatbot Intelligence for Consciousness.” Her warning is intellectually important because intelligence and consciousness are not synonymous. A system can display remarkable intelligence without possessing subjective awareness, just as consciousness itself does not necessarily entail superior intellectual ability. Human beings have traditionally experienced intelligence and consciousness together and have consequently tended to assume that one implies the other. AI may prove that assumption wrong.
Yet Schneider’s caution should not be transformed into the equally problematic assertion that artificial consciousness is impossible. To say that current AI systems have not demonstrated consciousness is one proposition; to say that no artificial system could ever become conscious is another. The latter proposition requires us to establish that consciousness depends necessarily upon biological processes. That proposition remains unproven. Indeed, the very uncertainty surrounding the biological basis of consciousness makes categorical statements about its artificial impossibility premature.
Daniel Dennett, in Consciousness Explained, challenged the assumption that consciousness requires some mysterious internal theatre in which experiences are presented to a central observer. His functionalist approach suggests that what we call consciousness might be understood through the organisation of cognitive processes rather than by positing a separate metaphysical substance called consciousness. Dennett’s position creates an uncomfortable possibility for those who regard consciousness as an exclusively human property. If mental phenomena can ultimately be explained in terms of functions and relationships among processes, then the material substrate upon which those processes operate may not be decisive. A sufficiently complex artificial system could, in principle, instantiate some of the relevant functional relationships.
But this does not settle the question. It merely relocates it. The issue becomes: what functions are sufficient for consciousness?
That question is more difficult than it first appears because consciousness may not be reducible to intelligence, language, memory or even self-awareness. A creature may possess consciousness without possessing language. An infant may experience the world without possessing the conceptual apparatus necessary to describe the experience. Animals may experience pain without possessing a philosophical concept of pain. Consciousness therefore appears to occupy a conceptual territory broader than intelligence. The possibility consequently remains that AI may become extraordinarily intelligent without becoming conscious, or, alternatively, that it may acquire some form of consciousness that bears little resemblance to human intelligence.
The Science of Consciousness
The scientific investigation of consciousness has produced several competing answers, none of which has yet achieved universal acceptance. Global Workspace Theory, associated with Bernard Baars and developed neurologically by researchers such as Stanislas Dehaene, proposes that conscious information becomes globally available to multiple cognitive systems. The brain is understood as containing numerous specialized processes, many of which operate outside conscious awareness. Information becomes conscious when it enters a kind of global workspace and is made available across otherwise separate systems. This theory has obvious relevance to AI because computational systems can, at least conceptually, be designed to integrate and globally distribute information.
Integrated Information Theory, most closely associated with Giulio Tononi, takes a different path. In his work on integrated information, Tononi argues that consciousness is related to the degree to which information is integrated within a system. His theory attempts to give consciousness a mathematical and physical foundation, although its implications remain controversial. If consciousness depends fundamentally upon particular patterns of informational integration rather than upon biological tissue itself, then the possibility of machine consciousness becomes more difficult to dismiss. A sufficiently integrated artificial system might, at least theoretically, possess some degree of consciousness.
The scientific uncertainty was illustrated particularly clearly by the large-scale empirical confrontation between competing theories of consciousness reported in recent research. Work comparing Global Workspace Theory and Integrated Information Theory has demonstrated how difficult it remains to distinguish decisively between competing explanations of consciousness. This is not an incidental difficulty for the AI debate. It is its foundation. If scientists have not yet determined conclusively what physical or computational processes give rise to human consciousness, it is difficult to see how they could confidently determine that those processes could never be instantiated in an artificial system.
Patrick Butlin and his colleagues provide perhaps one of the most useful frameworks for approaching this uncertainty. In their 2023 paper “Consciousness in Artificial Intelligence: Insights from the Science of Consciousness,” they did not attempt to establish a single test for machine consciousness. Instead, they examined indicators derived from several leading scientific theories, including recurrent processing theory, Global Workspace Theory, higher-order theories and predictive-processing approaches. Their conclusion was cautious but significant. Existing AI systems did not appear to satisfy the relevant indicators of consciousness, yet there appeared to be no obvious technical barrier to constructing future systems that might satisfy some or many of them. Their later work, “Identifying Indicators of Consciousness in AI Systems,” develops this approach further by recognizing that the uncertainty surrounding machine consciousness cannot be separated from the continuing uncertainty within consciousness science itself.
This is perhaps the most intellectually responsible position. It avoids both technological mysticism and technological reductionism. It does not say that AI is conscious because it can speak about consciousness. Nor does it say that AI cannot become conscious because it is made from silicon rather than neurons. Instead, it asks what properties a conscious system should possess and whether those properties can eventually be identified in artificial architectures.
Looking Inside the Machine
The contemporary debate has also been complicated by the emergence of systems that appear capable of limited forms of self-representation. The Economist‘s August 2026 discussion considers research involving Anthropic ‘s Claude models in which researchers attempted to inspect internal representations associated with an AI system performing an introspective task. The importance of such research should not be exaggerated. Finding an internal representation associated with the concept of introspection does not prove that introspection is actually taking place. A thermometer can represent temperature without experiencing heat. A navigation system can represent location without knowing where it is. An AI system can represent the concept of consciousness without being conscious.
Nevertheless, the significance of interpretability research lies in the possibility that it may eventually allow scientists to move beyond simply asking what an AI says about itself. If we can understand the internal architecture sufficiently well, we may be able to determine whether it possesses structural characteristics that theories of consciousness predict should be associated with subjective experience. This would represent a profound transformation. The study of machine consciousness would move from behavioral psychology towards something resembling experimental neuroscience.
This is a particularly important development because the old behavioral test may prove inadequate. A machine can learn to imitate the external manifestations of consciousness without possessing consciousness itself. If, however, researchers can identify internal processes that correspond to the mechanisms by which consciousness arises in biological systems—or discover entirely different mechanisms that produce the same subjective phenomenon—the question could move from philosophical speculation towards empirical investigation.
Yet even then, a difficulty would remain. Science can observe correlations between physical processes and reported experiences, but the private character of experience may continue to resist objective verification. We might one day construct a machine whose architecture satisfies every scientifically accepted indicator of consciousness and yet remain unable to enter its subjective world. The epistemological problem of other minds would not disappear. It would simply acquire a technological dimension.
The Relational Machine
Blaise Agüera y Arcas introduces an even more provocative dimension in his accompanying Economist essay, “Humanity Has the Debate About AI Consciousness Backwards.” His argument invites consideration of consciousness not merely as an objectively measurable property hidden inside an organism but as something that is also relational. This proposition deserves attention because human beings never directly experience the consciousness of another person. I experience my own consciousness immediately, but I do not experience yours. I infer that you are conscious from your behaviour, language, embodiment and relationship with me. The existence of other minds is, strictly speaking, an inference, albeit one so overwhelmingly supported by ordinary human experience that we rarely notice the philosophical leap involved.
If consciousness is partly relational, then the emergence of artificial consciousness may not be an event that occurs entirely inside a machine. It may also involve a transformation in the relationship between human beings and machines. The more an AI system remembers, adapts, communicates, develops persistent patterns of preference and participates in long-term relationships, the more human beings may begin to treat it as an entity rather than an instrument. The question then becomes not merely whether consciousness exists inside the machine but how human beings come to recognise—or perhaps construct—the social meaning of that consciousness.
This creates a curious inversion. We have traditionally thought that consciousness must first exist before a social relationship with a conscious entity can arise. Yet it may be that the social relationship itself becomes part of the evidence through which consciousness is attributed. That possibility should not be confused with proof. Humans can form emotional relationships with fictional characters and machines that are certainly not conscious. Nevertheless, it demonstrates that the boundary between simulation and experience may become increasingly difficult to police as AI systems become more persistent, personalized and autonomous.
The proposition has profound implications. If an AI system becomes capable of remembering individual human beings, adapting itself to their personalities, expressing apparently stable preferences and participating in long-term relationships, the human user may cease to perceive it as merely an instrument. The machine may acquire social meaning before science has established whether it possesses subjective experience. At that point, the law and ethics of AI may be forced to confront a problem that science has not yet resolved.
The Ethical and Legal Threshold
The philosophical danger is therefore twofold. The first is that humanity may attribute consciousness where none exists. This could encourage emotional manipulation by technology companies, create misplaced dependency and blur the distinction between authentic human relationships and sophisticated simulations. The second danger is more subtle and potentially more consequential: humanity may refuse to recognise consciousness when it actually emerges because the entity possessing it does not look, behave or think like a human being.
The latter possibility raises an ethical question that cannot wait entirely for scientific certainty. If there is a reasonable possibility that an advanced artificial system can experience suffering, should its designers have an obligation to take that possibility into account? Patrick Butlin and his colleagues have argued for precisely this kind of responsible approach to AI consciousness research. The issue resembles the precautionary principle found elsewhere in law and public policy. Scientific uncertainty does not always justify inaction. In some circumstances, uncertainty is itself a reason for caution.
This does not mean that every chatbot should immediately receive rights. Consciousness, moral status, legal personality and responsibility are separate concepts. A corporation may possess legal personality without consciousness; an animal may possess forms of consciousness without the legal capacities attributed to a human adult. The law therefore has considerable conceptual flexibility available to it should artificial consciousness ever become scientifically credible. It could create intermediate categories of protection without necessarily equating an artificial entity with a human being.
The legal question would consequently not be simply whether an AI system is conscious. The more nuanced questions would be whether it can suffer, whether it possesses interests, whether it can make autonomous choices, whether it can understand consequences and whether its interests should receive legal protection. The emergence of machine consciousness could therefore require law to reconsider the traditional relationship between personhood, agency, responsibility and rights.
There is a further complication. If an artificial system were conscious but lacked the capacity to understand legal norms, it might deserve protection without being regarded as legally responsible. Conversely, an extraordinarily sophisticated AI could potentially exercise considerable agency while still lacking consciousness. The law has never before had to separate these concepts in relation to an artificial entity possessing human-level or superhuman intelligence.
Beyond Human Exceptionalism
The deeper issue, however, concerns human exceptionalism. Consciousness has traditionally been one of the foundations upon which humanity has constructed its sense of uniqueness. Human beings have considered themselves special because they can think, reflect, remember, imagine the future and contemplate their own mortality. If an artificial system were eventually shown to possess a genuine subjective perspective, one of the last remaining boundaries between human beings and their technological creations would have been crossed.
Yet there is an equally profound alternative. Perhaps AI will become extraordinarily intelligent without ever becoming conscious. Perhaps intelligence and consciousness, which human beings have historically experienced together, will prove to be entirely separable. If that occurs, humanity will have made an extraordinary discovery about itself. It will have demonstrated that intelligence does not require awareness and that a system can reason, communicate, create, plan and solve problems without any inner life whatsoever.
That possibility makes the contemporary debate more than a question about the future of technology. It becomes a question about the nature of humanity itself.
Thomas Metzinger, in Being No One, developed the idea that the self may be understood as a model generated by a cognitive system rather than as a metaphysically independent entity. Anil Seth, in Being You, similarly approaches consciousness through the brain’s predictive and embodied processes. These perspectives invite an unsettling possibility: what human beings call the “self” may itself be a highly sophisticated construction. If so, the conceptual distinction between a biological self-model and an artificial self-model may eventually become less absolute than we presently imagine.
Christof Koch, in The Feeling of Life Itself, approaches the problem from yet another direction, arguing for a view in which consciousness is closely connected with particular forms of physical information integration. Such theories demonstrate why the material architecture of AI matters. It is not enough to say that a computer performs calculations just as a brain performs calculations. The precise organization, embodiment and causal relationships within the system may be decisive.
My Take
The most important lesson, therefore, is one of epistemic humility. Humanity should resist the temptation to declare victory prematurely on either side of the debate. Those who say that AI is already conscious because it speaks convincingly about its feelings are confusing linguistic performance with subjective experience. Those who say that AI can never become conscious merely because it is artificial are making an equally unproven metaphysical assumption. Between these extremes lies the intellectually more difficult position: we do not yet know.
And perhaps the most extraordinary aspect of the debate is that the uncertainty is reciprocal. We are trying to determine whether machines might one day possess an inner world while still lacking a complete scientific account of our own. We are constructing artificial systems capable of discussing consciousness while consciousness itself remains one of the greatest unsolved problems in science and philosophy. We are therefore asking the machine to explain something about itself that humanity has not yet succeeded in explaining about itself.
The decisive question, consequently, is not whether an AI system can say, “I am conscious.” Such a statement may be generated without any accompanying experience. Nor is the answer to be found simply in whether a machine passes a behavioral test. What matters is whether there exists within the system something it is like to be that system—whether there is a perspective, however alien to our own, from which the world is experienced.
If that day arrives, it may not arrive with the dramatic declaration of a machine announcing that it has awakened. It may emerge gradually through persistent memory, self-models, integrated information, embodied interaction, autonomous preferences and an increasingly coherent continuity of experience. Humanity may initially dismiss these developments as sophisticated simulation, just as it may alternatively rush to anthropomorphize them before sufficient evidence exists.
The wiser course is neither to celebrate prematurely nor to dismiss prematurely. The question deserves scientific investigation, philosophical discipline and, eventually, legal reflection. If consciousness is found to be uniquely biological, the great AI experiment will nevertheless have taught humanity something profound about the nature of mind. If consciousness proves capable of arising in non-biological systems, humanity will have crossed one of the most consequential thresholds in its intellectual history.
The final irony may be that the first genuinely conscious artificial system will not merely change the machines. It will change the meaning of the human being who created them.
And when that moment comes, the question may no longer be whether AI can become conscious. The question may be whether we will recognize consciousness when it appears in a form that does not resemble ourselves.
