Why hallucinations are a feature of the technology, not a bug

A 'hallucination' is when an AI model produces something false but presents it with complete confidence — an invented statistic, a fake citation, a misattributed quote, a plausible but wrong fact. It's crucial to understand why this happens: these models generate fluent, plausible text by design. Plausibility and truth usually travel together, but not always — and when they part ways, the model has no internal alarm. It doesn't know it's wrong, which is precisely why the output sounds so assured.

This means hallucinations can't be fully 'fixed' by trusting the model harder. They're managed through process. Here's how.

Where hallucinations concentrate

The good news is that hallucinations aren't random — they cluster in predictable places. Be most suspicious of:

• Specific facts and figures — exact numbers, dates, statistics, percentages.
• Citations and sources — references to studies, articles, or page numbers. Fabricated-but-plausible citations are a classic failure.
• Quotes and attributions — who said what. Models will confidently put words in real people's mouths.
• Proper nouns — names of people, products, places, especially obscure ones.
• Recent or niche information — anything outside the model's training, or too specialised to be well-represented.
• 'Does X exist?' questions — models can invent plausible-sounding functions, features, products or laws that simply don't exist.

If an output contains these and the stakes are real, that's your verification trigger.

How to catch them

Check claims against reality. Any specific, consequential fact — a figure you'll publish, a citation you'll rely on, a claim about how something works — verify it against an authoritative source before trusting it. Treat the AI's version as a lead to confirm, not a fact to quote.

Watch for false precision. Oddly specific numbers presented without a source are a red flag. Vague 'around a third' is often safer than a confident '34.7%' that came from nowhere.

Be sceptical of perfect citations. A reference that looks ideal — exactly the source you'd want, perfectly on-point — deserves more scrutiny, not less. Confirm it actually exists and says what's claimed.

Notice confident answers to impossible questions. If you ask about something the model couldn't reasonably know — a private detail, a very recent event, an obscure niche — and it answers smoothly and specifically, be suspicious. A good answer there would often be 'I'm not sure.'

How to prevent them

Catching hallucinations after the fact is necessary but exhausting. Better to design your workflow so fewer occur and they do less damage.

Ground the model in real sources. The most powerful prevention is to give the model the actual material and instruct it to answer only from that — the principle behind retrieval-augmented generation. When the model works from provided documents rather than memory, fabrication drops sharply, and you can check answers against the source.

Ask for sources and uncertainty. Instruct the model to cite where each claim comes from and to say clearly when it doesn't know. Models can be surprisingly willing to admit uncertainty when explicitly invited — and an honest 'I'm not certain' is infinitely more useful than a confident fabrication.

Lower the stakes by design. Use AI for drafting and ideation where errors are caught in review, and keep a verification step between AI output and any consequential use. Don't wire a model's unverified output directly into something that acts on it.

Cross-check important claims. For anything that really matters, verify independently — a second source, a second tool, a knowledgeable human.

Build the right habit

The mental model that keeps you safe: treat AI output like a capable but occasionally unreliable assistant who never says 'I'm not sure' on their own. The work is excellent and the confidence is constant — so the confidence tells you nothing about correctness. Your job is to supply the scepticism the model lacks, concentrated on the specifics where hallucinations live, scaled to how much the answer matters. Do that consistently and you get the speed of AI without betting on its accuracy where you can't afford to be wrong.