Here is a belief that has quietly hardened into policy in schools and offices: paste a piece of writing into an AI detector, read the percentage it spits back, and you will know whether a machine wrote it. Teachers fail students over these scores. Managers use them to screen applicants. The trouble is that the tools do not work anywhere near well enough to carry that weight.
Start with the company that arguably knows most about how AI writes. In 2023, OpenAI quietly retired its own detector. Its own numbers explain why: the tool caught just 26% of AI-written text while flagging 9% of genuinely human writing as machine-made. A coin toss with a bias problem is no foundation for an accusation.
The false positives fall on real people
The failures are not random. Stanford researchers found that detectors were close to perfect on essays by US-born eighth graders, then misclassified more than 61% of essays written by non-native English speakers as AI-generated. Plain, simple prose reads as too clean to these systems, which is precisely how many people write in a second language. One widely shared test flagged the United States Constitution as almost entirely AI-written.
Independent reviews have not been kinder. One study ran fourteen detection tools and found that not a single one reached 80% accuracy. Turnitin, whose detector is built into many university systems, acknowledges a sentence-level false-positive rate of around 4% and quietly suppresses any score below 20% because results down there cannot be trusted. UCLA declined to switch the feature on at all.
So what actually works?
Not much, if you want certainty. Detection after the fact is a losing game, because a short rephrase or a translation pass defeats most tools. The more promising path runs the other way: marking AI output at the source. Anthropic has started embedding invisible watermarks in text Claude writes, and platforms such as Apple Music are labelling AI-generated songs using data supplied by the creators. Provenance beats forensics.
So here is the verdict. The claim that an AI detector can reliably tell you whether text was written by a machine is false, and treating a detector score as proof is worse than useless when a person's grade or job is on the line. Use the tools, if at all, as one weak signal among many. Never as a ruling.
Commentarii · 0