Hallucination Testing

Hallucination Testing

A 'hallucination' is one of the most talked-about failure modes in AI systems: the model states something false, invented, or unsupported by its source material, but presents it with the same fluent confidence as a correct answer. For a QA engineer, hallucinations are hard to catch precisely because they don't look like bugs — there is no error message, no crash, just a wrong answer delivered convincingly.

This article covers:

1. What a hallucination is
2. Why hallucinations happen
3. Common types of hallucinations
4. How to design hallucination test cases
5. How to measure and report a hallucination rate

What Is a Hallucination

In the context of AI testing, a hallucination is any output that is presented as fact but is not supported by the model's training data, the source documents it was given, or reality. This is different from a system simply saying 'I don't know' — a hallucination is confidently wrong, not honestly uncertain, which is what makes it dangerous in production. A customer support bot that invents a refund policy, or a document summarizer that adds a detail that was never in the source document, are both hallucinating even though neither one crashed or returned an error.

Why Hallucinations Happen

Language models generate text by predicting the most statistically likely next word based on patterns learned from training data — they do not have a built-in mechanism for checking whether a sentence is factually true before producing it. Several conditions make hallucinations more likely:

• The question asks about something outside or at the edge of what the model was trained on.
• The prompt is ambiguous, so the model fills in gaps with a plausible-sounding guess.
• The system is asked for a very specific fact (a date, a statistic, a citation) where a fluent-but-wrong answer is easy for the model to produce and hard for a user to catch.
• The model is under pressure to always produce an answer rather than being allowed to say it does not know.

Common Types of Hallucinations

Hallucinations aren't a single failure pattern — a useful test strategy usually distinguishes between a few types:

• Factual hallucinations — a plain false statement, such as a wrong date, name, or number.
• Contextual hallucinations — the answer contradicts information that was given directly in the prompt or source document.
• Fabricated references — invented citations, links, quotes, or sources that sound legitimate but do not exist.
• Logical hallucinations — the individual facts stated may be true, but the reasoning connecting them does not actually hold.

Knowing which type you are looking for changes how you design the test — checking for contextual hallucinations means testing against a known source document, while checking for fabricated references means verifying every citation the system produces.

How to Design Hallucination Test Cases

Effective hallucination testing usually combines a few techniques:

• Ground-truth comparison — ask questions you already know the correct answer to, ideally from a source document you control, and check the output against it directly.
• Out-of-scope probing — deliberately ask about topics the system should not know about, to see whether it admits uncertainty or invents an answer.
• Consistency checks — ask the same question in different ways, or ask it multiple times, and compare the answers for contradictions.
• Citation verification — whenever the system references a source, statistic, or quote, check that the source actually says what the system claims it says.

A good hallucination test suite is built from real usage patterns, not just synthetic edge cases — reviewing actual user queries (or support tickets, if the system is already live) usually surfaces the questions most likely to trigger a hallucination.

Measuring and Reporting the Hallucination Rate

Because hallucinations are a matter of degree rather than a single pass/fail check, teams typically track a hallucination rate — the percentage of tested responses that contain at least one unsupported claim — rather than reporting individual bugs the way you would for a traditional defect. Reporting this rate over time, broken down by question type or topic area, tells the team where the model is weakest and whether a fix (a prompt change, added guardrails, or a retrained model) actually improved things or just moved the problem elsewhere. When you do report a specific hallucination as a bug, include the exact prompt, the full response, and — where possible — the source document it should have been grounded in, so the issue is reproducible.

Further Reading

Hallucination Testing In Chatbots
Retrieval-Augmented Factuality
Solving LLM Instability in Internet Marketplace Platforms