Can an AI Detector Really Prove a Human Wrote It?

A student is accused of cheating because software flagged an essay as machine-written. A job applicant is quietly passed over after a detector scored a cover letter as AI. Scenes like these have become common, and they all rest on the same comfortable belief: that a tool can look at a piece of text and reliably tell whether a human or a model produced it. The belief is widespread, and it is far less solid than the confident percentages on the screen imply.

The core problem is that these detectors do not read for meaning. Most work by measuring statistical properties of text, such as how predictable each word is given the words around it, a quantity often called perplexity. Human writing that happens to be clean, simple and predictable can look machine-made by that measure. So can writing by people who learned English as a second language.

The false positive problem

That last point is not hypothetical. A widely cited Stanford study fed 91 essays written by non-native English speakers into seven leading detectors and found an average false positive rate above 61 percent, with all seven tools wrongly flagging nearly a fifth of the essays as AI. The bias was structural. Simpler word choice read as machine output, so the people most likely to be falsely accused were those still building fluency in the language.

Vendor claims and independent testing also diverge sharply. Turnitin, one of the most widely deployed tools in education, has advertised a false positive rate under one percent. Independent analyses have put the real-world figure between 5 and 20 percent, several times higher. At the scale of a large university, even a few percent means many innocent students flagged every term.

The most telling admission came from inside the industry. OpenAI, the company behind ChatGPT, built its own tool to detect AI-written text and then shut it down in July 2023, citing a low rate of accuracy. Its classifier correctly identified only about a quarter of AI-written text and misflagged roughly one in eleven human passages. If the maker of the model could not reliably detect the model, a third-party tool promising near-certainty deserves real skepticism.

What this means in practice

None of this makes detectors useless, and it does not mean AI writing is undetectable in every case. A long, obviously templated block of text scored by a good tool is a reasonable signal worth a closer look. The mistake is treating a score as a verdict. A detector produces a probability shaped by the quirks of its training, not a fingerprint, and it comes with no guarantee about how often it will be wrong.

Institutions are slowly absorbing the lesson. Some universities have pulled detection tools from their plagiarism systems rather than defend decisions built on them. The wider software world is wrestling with a related question of provenance, as seen when the Rust project moved to set rules for AI-written contributions rather than trying to detect them after the fact. The honest position is uncomfortable but clear. For now, no tool can prove who wrote a given piece of text, and treating one as if it can does real harm to real people.

Sources: Walter Writes AI, Leap AI, CASRAI.