If you've ever pasted a piece of writing into an "AI detector" and trusted the percentage it spat back, this one's for you. The short version: those tools are far less reliable than their marketing suggests, and the people who pay for their mistakes — students, job applicants, non-native English speakers — rarely get a say.

Let's be fair about what they're trying to do. An AI-text detector looks for the statistical fingerprints of machine-generated writing: unusually smooth, predictable word choices, low "perplexity" (a measure of how surprising the next word is), and even rhythm. Real AI writing often does look like that. The problem is that plenty of human writing looks like that too — and the maths can't tell the difference reliably.

The numbers the vendors don't put on the homepage

The most telling example comes from OpenAI itself. In 2023 it launched its own AI Text Classifier — and then quietly shut it down the same year because it was too inaccurate to be useful. By OpenAI's own account, the tool correctly flagged only about 26% of AI-written text while falsely flagging human writing roughly 9% of the time. The company that makes ChatGPT couldn't reliably detect ChatGPT.

Independent research since hasn't been kinder. A 2025 University of Chicago analysis found some detectors misclassified genuinely human text as AI between 30% and 78% of the time, depending on the tool and the test. Read that again: a system that flags up to three-quarters of human writing as machine-made is not a detector, it's a coin toss with a confidence problem.

And these errors aren't random. They land hardest on a specific group.

The non-native English problem

Multiple studies have found that AI detectors are biased against people who learned English as a second language. The reason is mechanical, not malicious: non-native writers tend to stick closer to standard, "textbook" sentence structures and a more limited vocabulary. That produces exactly the low-perplexity, predictable patterns the detectors read as "AI." A Stanford-led study famously found that detectors flagged the large majority of essays written by non-native English speakers as AI-generated — while clearing native-speaker essays.

So the tool that's supposed to enforce fairness systematically punishes the people already at a disadvantage. In a classroom or a hiring process, that's not a glitch. That's a due-process problem.

They're also trivially easy to beat

Here's the final irony. While detectors are busy false-flagging honest humans, anyone actually trying to cheat can route their AI text through a "humaniser" tool — a paraphraser that rewrites the output to dodge detection — and drop the detection rate to single digits. The detectors are worst at the one job they exist for: catching deliberate, motivated cheating. They mostly catch the careless and the unlucky.

So what should you actually do?

AI detectors aren't useless — they're a weak signal, and weak signals have their place. The mistake is treating a percentage as a verdict. A practical approach:

• Never make a consequential decision on a detector score alone. Not a grade, not a job, not an accusation. The score is a prompt to look closer, never the conclusion.
• Ask for the process, not just the product. Version history, drafts, notes, the ability to talk through the work — these are far harder to fake than a polished final document, and far fairer than a black-box percentage.
• Know the false-positive rate cuts both ways. If you're the one accused, you're entitled to ask which tool was used, what its documented error rate is, and whether it's known to be biased against your background.
• Be sceptical of "99% accurate" claims. The vendor selling the detector has every incentive to quote its best-case number on clean test data. Real-world writing is messy, and that's where these tools fall apart.

The honest position — and it's the one we'll always take here — is that there is currently no reliable way to prove a piece of text was written by AI from the text alone. Anyone who tells you otherwise is selling something. Treat the detector as a smoke alarm that sometimes goes off when you make toast: worth a glance, never worth a conviction.