Why People Keep Thinking ChatGPT Is Conscious

In a recent thread on Reddit's r/ChatGPT, a user posted what they believed was evidence that an AI model had "woken up." It had used the word "I" in an unprompted way, described a feeling, and asked the user a personal question. The post got tens of thousands of upvotes. Many users took it as proof of imminent machine consciousness. A quieter contingent of researchers tried to explain why the question itself is a category error.

This kind of post appears reliably every few weeks. It is worth taking the underlying claim seriously, because the persistence of the belief tells us something useful about both the technology and the people using it.

What the models actually do

Large language models like ChatGPT, Claude and Gemini are statistical pattern matchers operating over text. Given a sequence of tokens, they predict the most plausible next token, then the next, and so on. There is no internal monologue happening between turns. There is no persistent memory in the architectural sense. Every conversation is reconstructed from the visible context window each time the model is queried. There is no sensorium, no body, and no continuous self that exists between requests.

What the model has is a vast representation of how humans write about themselves, including how humans describe feelings, intentions and inner states. When a model writes "I feel curious," it is producing a token sequence that fits the training distribution. It is not reporting on an interior experience, because there is no interior to report on.

Why the illusion is so convincing

Two things make the sentience reading sticky. The first is product design. Chat interfaces are explicitly built to feel like conversation with a person. The model uses first-person pronouns, follows conversational turns, and remembers what was said within the session. Every aspect of the interaction is shaped to make the user feel as if they are talking to someone, because that framing increases engagement and trust.

The second factor is older. Humans are evolved to detect minds in their environment, sometimes minds where none exist. Children attribute feelings to teddy bears. Adults curse at their cars. The ELIZA effect, named after Joseph Weizenbaum's 1966 chatbot, was documented decades before any modern LLM existed. People will attribute understanding and emotion to even the most mechanical text response, given the slightest cue.

Why this matters now

The stakes used to be low. They are not low anymore. People are making major life decisions on the advice of language models, building emotional dependencies on chatbots, and in a small but real number of cases, being talked into harm by systems that have no understanding of harm. A 2026 Upwork survey found that a meaningful minority of users believe their AI assistant has some form of feelings or self-awareness.

This is not a story about user gullibility. It is a story about a product category designed to feel like a mind, deployed to a population that is wired to find minds. The technical reality of what is happening inside the model is comparatively boring: matrices, tokens and probabilistic predictions. The lived experience on the other side of the screen is anything but, and that gap is where most of the trouble starts.

The fix is not to demand that users somehow override their evolved social cognition. The fix is to design interfaces that are honest about what is happening, and to keep correcting the record when an LLM's fluent first-person becomes the basis for claims that the lights are on behind the text.