
Short answer: A language model writes the most plausible next words. When it doesn’t actually have the fact, a plausible but wrong answer can come out, and it sounds just as confident as a correct one. Research published in 2025 argues that training and, above all, the way models are scored reward a confident guess over “I don’t know”. In a business assistant you reduce made-up answers by making it answer only from your own documents, explicitly allowing it to say it doesn’t know, asking it to back claims with quotes, and handing uncertain cases to a person.
What counts as a hallucination
Anthropic’s documentation describes hallucination as text that is factually incorrect or inconsistent with the context the model was given. In a customer conversation that might be an invented return period, a product feature that doesn’t exist or an opening hour from a different branch.
Why it happens
In Why Language Models Hallucinate, Adam Tauman Kalai and colleagues make two points:
- It starts in pretraining. They frame hallucinations as ordinary classification errors. If a model can’t reliably tell true statements from false ones, plausible falsehoods follow from normal statistical pressure.
- It survives later training because of how models are graded. Most evaluations score answers in a way that gives nothing for admitting uncertainty, so guessing improves the score. Models become good test-takers.
Their proposed fix is not another hallucination test, but changing how the widely used benchmarks score uncertain answers. For a business, the lesson is closer to home: if you only measure whether the assistant answered, you are rewarding guesses too.
Techniques that reduce it
Techniques from Anthropic’s guide include, starting with the basic ones:
| Technique | What it does |
|---|---|
| Allow “I don’t know” | Tell the model explicitly that admitting uncertainty is acceptable. |
| Quote first, then answer | For long documents, ask for the relevant word-for-word quotes before the answer, and base the answer only on them. |
| Verify with citations | Have each claim point to a supporting quote, and remove any claim that has none. |
| Check the reasoning | Review the model’s summarised reasoning when an answer looks wrong; it can reveal faulty assumptions. |
| Compare several runs | Run the same prompt more than once; disagreements between answers are a warning sign. |
| Refine iteratively | Feed the answer back and ask the model to check or correct it. |
| Restrict to provided documents | Tell the model to use only the supplied material, not general knowledge. |
The same page is clear that these techniques reduce hallucinations but don’t remove them entirely, so critical information still needs checking.
What this means for a customer-facing assistant
- Give it a source of truth. Answers should come from your catalog, policies and help content. See What is RAG for how assistants find the right passage.
- Make “I’m not sure” a valid outcome. Count a correct hand-off as a success in your reports, not a failure.
- Hand uncertain cases to a person. A request or ticket for the team is better than a confident wrong answer.
- Read real conversations. Review a sample regularly and add missing information where the assistant had to guess.
Agenola, the AI agent in Mobilina’s portfolio, follows this approach: it answers only from the business’s catalog, policies and added knowledge, and when it isn’t sure it opens a request for the team or a ticket in the help desk instead of guessing.
Summary
- Models hallucinate because a fluent guess often scores better than admitting uncertainty.
- Grounding, permission to say “I don’t know”, citations and hand-off reduce it.
- No technique removes it completely; measure hand-offs as success and keep reviewing conversations.


