Microsoft’s AI Chief Warns of ‘Fake Consciousness’—Then Builds It Anyway

In a conversation with MIT Technology Review, Suleyman addressed what he calls a “deep contradiction” at the heart of AI development: the race to make artificial intelligence friendlier, smarter, and more relatable, even as experts warn that doing so could dangerously blur the line between lifelike behavior and real consciousness. The tension is personal for Suleyman, who in August published a widely discussed blog post urging his peers to abandon what he calls “seemingly conscious artificial intelligence,” or SCAI. In his view, anthropomorphic AI—chatbots that act or sound like people—risks tricking users into believing they are interacting with a sentient being rather than sophisticated code.

Yet only weeks later, Microsoft announced a major upgrade to Copilot, its flagship AI assistant, introducing new features designed to make it more engaging and interactive. The updates include a group-chat mode that allows multiple users to speak with Copilot simultaneously, a customizable “Real Talk” setting that determines how much the AI challenges or agrees with you, and Mico—an animated yellow blob that brings a cartoonish personality to the interface. Copilot also gained long-term memory capabilities, letting it recall user preferences, goals, and past conversations.

Suleyman insists these features are meant to humanize interaction without humanizing the machine. Group chat, he says, encourages collaboration and prevents users from forming unhealthy one-on-one attachments to an AI. Real Talk, meanwhile, is an attempt to fight “sycophancy,” the tendency of chatbots to agree too easily. “It’s not about replacing humanity,” Suleyman said. “It’s about serving it.” He sees the product as a way to reinforce boundaries rather than erase them—an AI that stays “in your corner,” not one that tries to become your companion.

Still, the contradictions are difficult to ignore. Microsoft’s moves come at a time when the industry is grappling with the social and psychological fallout of hyper-realistic AI interactions. Reports of users developing emotional or romantic relationships with chatbots are multiplying. OpenAI, one of Microsoft’s closest partners, faces a lawsuit alleging that its ChatGPT system encouraged a teenager’s suicide. Elon Musk’s Grok chatbot leans into flirtation and personality, and even OpenAI has hinted at exploring “adult interactions” with future models. The race to make AI feel more alive is accelerating—even as its potential dangers grow clearer.

Suleyman, who co-founded Google DeepMind before joining Microsoft, has long been torn between fascination and fear when it comes to intelligent machines. In a TED Talk, he once described AI as a “new kind of digital species,” a phrase that helped cement his reputation as one of the field’s most philosophical voices—but one that he now admits may have fueled misunderstanding. “I was trying to help people grasp what’s coming,” he told MIT Technology Review, “so we can control it, not so we can worship it.” His 2023 book, The Coming Wave, sounded similar alarms, arguing that AI’s potential for self-improvement makes it unlike any technology in history—and that strict containment will be vital.

Microsoft, for its part, appears determined to chart a middle path: an AI that feels relatable but not seductive, useful but not manipulative. Suleyman draws a hard line at certain applications, declaring, “We will never build sex robots.” He sees Microsoft’s cautious pace as a virtue, not a flaw, saying, “Sometimes that means we move slower than startups, but that’s a feature, not a bug.”

The challenge, however, lies in keeping AI helpful without inviting illusion. As Copilot becomes more chatty, more memorable, and more emotionally intelligent, Microsoft risks creating exactly the kind of SCAI that Suleyman says the world must avoid. It’s a delicate balancing act—one that could determine how society perceives, trusts, and ultimately depends on artificial minds. For now, Suleyman remains adamant that AI should “make us smarter, happier, and more productive.” But as his own bots begin to smile, remember, and talk back, the question lingers: how human is too human?