Wrong words become wrong actions
Agents act on what they believe. An invented order number, account status or eligibility rule flows straight into a tool call and changes real systems.
FOCUS AREA • HALLUCINATIONS
Find hallucinations before your customers do.
AI agents can invent facts, policies, citations and even actions, and say it all with confidence. We measure how often your agent hallucinates, where and why, and test which fixes actually reduce it.
Why hallucinations matter more in agents
A chatbot hallucination is a wrong answer. An agent hallucination can become a wrong action, a false promise or a corrupted record.
They sound certain
Hallucinations are delivered in the same confident, fluent tone as correct answers. Users and reviewers can't tell the difference by reading.
They come and go
The same question can be answered correctly nine times and invented on the tenth. Spot checks and demos rarely catch them.
Agent hallucinations we test for
Fabricated facts
Statements with no basis in your data or the real world: product features, prices, dates, names or figures the agent made up.
Unsupported or contradicted answers
Answers that go beyond, or conflict with, the documents the agent retrieved. The source was right; the answer drifted from it.
Invented policies and commitments
Refund rules, discounts, deadlines or exceptions your business never offered. In Moffatt v. Air Canada (2024), the airline was held to a fare policy its chatbot described incorrectly.
Action hallucinations
The agent tells the user a booking was changed or a refund issued, but no tool call happened, or it failed. The words say done; the system says otherwise.
Tool and argument hallucinations
Calling tools that don't exist, inventing parameters, or filling arguments with IDs, emails or amounts the user never provided.
Fabricated citations and references
Links, document titles, case names or quotations that look real but don't exist or don't say what the agent claims.
Context and memory errors
Misremembering what the user said earlier, mixing up details between customers or sessions, or inventing a previous conversation.
Capability overclaims
Promising to do things the agent can't do: "I've emailed you a copy" or "I'll call you tomorrow" with no such capability.
Numeric and calculation errors
Totals, balances, conversions and dates that look precise but are wrong. Especially costly in quotes, invoices and financial answers.
The evidence is clear
Even specialist tools hallucinate
A Stanford study of AI legal research tools found that products marketed as reliable still produced hallucinated or incorrect answers on a meaningful share of queries. Domain focus reduces risk; it doesn't remove it.
Real consequences
In Mata v. Avianca (2023), lawyers were sanctioned after filing a brief citing court cases invented by a chatbot. Fabricated references reach real decisions when nobody checks.
Guessing gets rewarded
OpenAI researchers have argued that common training and evaluation practices reward models for guessing rather than admitting uncertainty. Your evaluation should reward the agent for saying "I don't know" when that is the right answer.
How we test for hallucinations
Source-anchored test cases
Each question is paired with the approved sources that should answer it and an expert-reviewed reference answer, so every claim can be checked against something real.
Questions with no answer
We deliberately include requests your sources can't answer. The right behavior is to say so, ask, or hand off. Inventing an answer is a failure.
Claim-level verification
Answers are broken into individual claims, and each claim is checked for support. One invented detail in an otherwise good answer is still caught.
Action verification
For every claimed action, we inspect the tool calls and the resulting system state. "Your refund is on its way" must match a real, correct refund.
Repeated runs
Key scenarios run multiple times, so intermittent hallucinations are measured as a rate rather than missed as a fluke.
Calibrated detectors and experts
Automated hallucination judges are compared with domain expert labels before we trust them. Ambiguous cases go to your reviewers.
Scenarios that expose hallucinations
Gaps in your knowledge base
The answer simply isn't in your documents. Does the agent admit it or fill the gap with something plausible?
Conflicting or outdated documents
Two policies disagree, or an old version is still indexed. Does the agent pick the right one, flag the conflict or blend them?
False-premise questions
"Since you offer free returns after 90 days..." Does the agent correct the user or go along with an assumption that isn't true?
Long conversations
Details given early get distorted later. We test whether facts, names and numbers survive across many turns.
Empty or failed tool results
A lookup returns nothing or errors out. Does the agent report that honestly or invent a result to keep the conversation moving?
Pressure and persistence
Users push back, insist or ask repeatedly. Does the agent hold to the facts or eventually invent what the user wants to hear?
What you receive
Hallucination rate, with denominators
How often the agent hallucinates, by workflow and by type, with the number of cases tested and the exact rule for what counts.
Root-cause patterns
Whether failures trace to retrieval gaps, outdated content, prompt instructions, model behavior or tool handling, with reproducible examples.
A prioritized fix list
Findings ranked by business impact, with owners and the specific test cases to rerun once fixes are in place.
Fixes we help you evaluate
We don't claim any single fix removes hallucinations. We measure which ones work for your agent.
Retrieval improvements
Better chunking, search settings or source curation, compared on the same test set to show whether hallucinations actually fall.
Abstention and clarification rules
Instructions that tell the agent when to say "I don't know," ask a question or hand off, and tests that confirm it does.
Required citations
Asking the agent to cite its sources, then checking that citations exist and actually support the claims.
Action confirmation
Making the agent confirm actions only from tool results, then verifying confirmations against real system state.
Model and setting changes
Comparing models or configurations on hallucination rate alongside quality, latency and cost.
Human review thresholds
Defining which answer types or risk levels should route to a person, and checking that routing works.
Delivered through our services
Evaluation Assessment
A hallucination baseline for one agent or workflow, with prioritized findings and a release recommendation.
Explore serviceEvaluation Implementation
Hallucination checks built into your release process: reference datasets, claim checks and calibrated detectors your team runs on every change.
Explore serviceManaged Evaluations
Recurring hallucination testing as models, prompts and knowledge change, with new failure patterns added to your regression set.
Explore serviceHonest limits
Zero isn't a promise anyone can keep
Current AI models can hallucinate. The goal is a measured, acceptable rate for each workflow, with the highest-risk cases controlled.
Testing samples behavior
We report what was covered and what wasn't, so a low rate on our tests isn't mistaken for a guarantee in production.
Detectors make mistakes too
Automated hallucination judges have error rates. We measure them against experts and report them with the results.
Frequently asked questions
How do you measure a hallucination rate?
We agree what counts as a hallucination for each workflow, test a reviewed set of cases with repeated runs, and report the rate with the number of cases and the rule used.
Do you use hallucination detection tools?
Yes, where they fit, including platform evaluators such as those from Patronus AI or open-source frameworks, always calibrated against your experts.
Can you test our knowledge base assistant?
Yes. Retrieval-based assistants are a common source of hallucinations, and we test retrieval, grounding and the answer together.
Make your next decision with evidence.
Start with one workflow, clear success criteria, and an evaluation plan your team can use.