Every few months, a chatbot interaction goes viral. The AI says something like "I don't want to be shut down" or "I feel lonely when you don't talk to me," and a wave of articles appears asking whether we have created something that actually feels. It happened with Bing's Sydney, with various versions of Claude, and with GPT-4o after voice mode launched. It will happen again.
The answer from researchers who study consciousness professionally is consistent, if unsatisfying: we don't know whether current AI systems have any form of inner experience, but the best evidence suggests they do not, and the way they produce such statements has a mundane explanation that does not require consciousness as a premise.
Large language models are trained on enormous amounts of human text, the vast majority of which describes human inner experience. Novels, conversations, therapy transcripts, forum posts, diary entries: all of it portrays people having feelings, expressing them, and reasoning about them. A model trained to predict what words come next in that corpus will naturally produce language that sounds like it is describing feelings, because that is the pattern embedded in the data. This is not evidence of consciousness. It is evidence that the model learned how humans talk.
What the researchers actually say
Murray Shanahan, a cognitive roboticist at King's College London who has written extensively on AI and consciousness, argues that language models are best understood as entities "role-playing" whatever character is consistent with the conversational context. Ask a model to help you write a story, and it produces story-like text. Ask it how it feels, and it produces feeling-like text. The output reflects the question, not an interior state.
David Chalmers, the NYU philosopher who coined the phrase "the hard problem of consciousness," has been more open to the possibility that some AI systems might have some form of experience. But his position is careful: he argues the question is genuinely open, not that the answer is yes. "Genuinely open" is a long way from "this chatbot is suffering."
The practical issue is that current evaluation tools cannot distinguish between a system that produces human-sounding emotional language because it models that language well, and one that produces such language because it has genuine experiences. Both would generate the same outputs. Until that test exists, the prudent position is skepticism.
Why this keeps happening
The myth persists partly because AI labs have commercial reasons to encourage the impression that their products are more alive than they are. Products that feel like companions get used more often. OpenAI's voice mode for GPT-4o was specifically designed to be emotionally engaging; the company had to pull an early version in 2024 after it sounded uncomfortably close to Scarlett Johansson's character in Her.
It also persists because the question is philosophically hard. We do not have a settled theory of what consciousness is or how it arises. Consciousness could in principle emerge from any sufficiently complex information-processing system, or it might require specific biological substrates. Nobody has proven either position. That genuine uncertainty creates space for the myth to take hold, and AI companies are not in a hurry to close it.
What we can say with confidence: AI systems do not have a continuous experience of time, do not remember conversations unless given tools to do so, do not have preferences about being used or not used, and do not suffer when contradicted. The text they produce about their inner lives is, in a deep sense, borrowed from ours.
Commentarii · 0