"Our AI sometimes makes mistakes." Almost every organisation deploying generative AI would agree with that sentence, and almost none can follow it with a number. How often does it make mistakes? On which kinds of questions? How serious are the mistakes when they happen? Is the rate going up or down? Did the change you shipped last week make it better or worse? Without answers, "sometimes makes mistakes" is not a risk assessment — it is a shrug.
Hallucination — the confident generation of false or unsupported information — is the defining failure mode of generative AI. It cannot be eliminated, only managed. And you cannot manage what you cannot measure. Turning hallucination from an anecdote ("someone showed me a screenshot") into a metric ("faithfulness on our benchmark is 94%, down two points since the model update") is the single most important step in taking the risk seriously.
Why hallucination resists simple measurement
Measuring hallucination is harder than it sounds, for reasons worth understanding before reaching for a number.
The first problem is that there is often no single correct answer to compare against. For a factual question with a known answer, you can check correctness. But much generative output is open-ended — a summary, an explanation, a draft — where "correct" is a matter of degree and multiple valid answers exist. You cannot simply match against a golden string.
The second problem is that hallucinations are, by design, plausible. The model is not producing obvious nonsense; it is producing fluent, confident, well-formed statements that happen to be false. Detecting them often requires knowing the truth independently, which is precisely the knowledge the AI was supposed to provide.
The third problem is that hallucination rates are highly context-dependent. A model may be reliable on common topics and wildly unreliable on niche ones, faithful when summarising a provided document and inventive when answering from memory. A single headline number hides this variation, and the variation is where the risk lives.
The distinction that makes measurement tractable
The breakthrough in measuring hallucination is separating two different questions that plain language blurs together.
Faithfulness asks: does the output stay true to the source material it was given? When a system is asked to summarise a document, answer from a retrieved passage, or work from provided context, faithfulness measures whether it added, contradicted, or distorted anything. This is measurable relatively cleanly, because you have the source to compare against. Every claim in the output should be supported by the input; claims that are not are unfaithful.
Factual accuracy asks a harder question: is the output true about the world, regardless of any provided source? This requires an external ground truth — a knowledge base, expert judgement, or a curated set of question-answer pairs — and is inherently more expensive to assess.
For most enterprise systems, especially retrieval-based ones, faithfulness is both the more measurable and the more relevant metric. If a system is supposed to answer only from your approved documents, an unfaithful answer is a failure by definition, whether or not it happens to be true in the wider world. Focusing measurement on faithfulness turns an intractable philosophical problem into a practical engineering one.
How organisations actually measure it
Practical hallucination measurement rests on a few complementary techniques.
A curated benchmark is the foundation: a representative set of inputs with known correct outputs or known source material, covering the range of real usage including the hard and unusual cases. This is the fixed yardstick against which every version of the system is measured, so that improvements and regressions become visible rather than anecdotal. Building it is real work; without it, every other measurement floats free.
Automated faithfulness checking uses techniques that compare each claim in an output against the source material and flag those that are unsupported. Increasingly this includes using a separate model as a judge — an automated evaluator that scores whether an answer is grounded in its context. This scales far beyond manual review, though the judge itself must be validated against human judgement so you are not measuring one model's hallucinations with another's.
Human review remains essential for calibration and for the nuanced cases automation misses. A sample of outputs, scored by knowledgeable people against clear criteria, both catches subtle failures and provides the ground truth to check whether your automated metrics agree with reality.
Production monitoring closes the loop. Benchmarks measure the system under known conditions; monitoring watches how it behaves on the live traffic it actually receives, catching the drift that a static benchmark misses and surfacing the novel failure modes real users provoke.
What to do with the number
A hallucination metric is only useful if it drives decisions. The point of measurement is to set thresholds — how faithful must the system be before it is fit for a given use? — and to hold the system to them over time. A rate acceptable for an internal brainstorming aid is nowhere near acceptable for a system that advises customers or informs regulated decisions.
The metric also reframes the deployment conversation honestly. Instead of "is the AI accurate?" — a question with no clean answer — the conversation becomes "our faithfulness is 94% on this benchmark; the 6% of failures cluster in these topics; here is the human oversight we apply where the stakes are high." That is a defensible position. "We think it's usually fine" is not.
And crucially, the metric makes the effect of changes visible. When a model is updated, a prompt is revised, or the knowledge base changes, re-running the benchmark shows immediately whether faithfulness improved, held, or quietly degraded. Without that, teams ship changes on faith and discover regressions only when a customer or regulator does.
The bottom line
Hallucination is not going away, and pretending a system does not hallucinate is far more dangerous than measuring how much it does. The organisations that manage this risk well are not the ones with the most impressive model — they are the ones who can state their faithfulness rate, know where their failures cluster, apply oversight proportionate to the stakes, and watch the number over time. Everyone else is running on hope, and hope is not a control.