Why do AI models make things up, and how do you reduce it?

Author: Mobilina Team · Published:

Illustration comparing a confident guess with an honest answer that hands the question to the team.

Short answer: A language model writes the most plausible next words. When it doesn’t actually have the fact, a plausible but wrong answer can come out, and it sounds just as confident as a correct one. Research published in 2025 argues that training and, above all, the way models are scored reward a confident guess over “I don’t know”. In a business assistant you reduce made-up answers by making it answer only from your own documents, explicitly allowing it to say it doesn’t know, asking it to back claims with quotes, and handing uncertain cases to a person.

What counts as a hallucination

Anthropic’s documentation describes hallucination as text that is factually incorrect or inconsistent with the context the model was given. In a customer conversation that might be an invented return period, a product feature that doesn’t exist or an opening hour from a different branch.

Why it happens

In Why Language Models Hallucinate, Adam Tauman Kalai and colleagues make two points:

  1. It starts in pretraining. They frame hallucinations as ordinary classification errors. If a model can’t reliably tell true statements from false ones, plausible falsehoods follow from normal statistical pressure.
  2. It survives later training because of how models are graded. Most evaluations score answers in a way that gives nothing for admitting uncertainty, so guessing improves the score. Models become good test-takers.

Their proposed fix is not another hallucination test, but changing how the widely used benchmarks score uncertain answers. For a business, the lesson is closer to home: if you only measure whether the assistant answered, you are rewarding guesses too.

Techniques that reduce it

Techniques from Anthropic’s guide include, starting with the basic ones:

Technique What it does
Allow “I don’t know” Tell the model explicitly that admitting uncertainty is acceptable.
Quote first, then answer For long documents, ask for the relevant word-for-word quotes before the answer, and base the answer only on them.
Verify with citations Have each claim point to a supporting quote, and remove any claim that has none.
Check the reasoning Review the model’s summarised reasoning when an answer looks wrong; it can reveal faulty assumptions.
Compare several runs Run the same prompt more than once; disagreements between answers are a warning sign.
Refine iteratively Feed the answer back and ask the model to check or correct it.
Restrict to provided documents Tell the model to use only the supplied material, not general knowledge.

The same page is clear that these techniques reduce hallucinations but don’t remove them entirely, so critical information still needs checking.

What this means for a customer-facing assistant

  • Give it a source of truth. Answers should come from your catalog, policies and help content. See What is RAG for how assistants find the right passage.
  • Make “I’m not sure” a valid outcome. Count a correct hand-off as a success in your reports, not a failure.
  • Hand uncertain cases to a person. A request or ticket for the team is better than a confident wrong answer.
  • Read real conversations. Review a sample regularly and add missing information where the assistant had to guess.

Agenola, the AI agent in Mobilina’s portfolio, follows this approach: it answers only from the business’s catalog, policies and added knowledge, and when it isn’t sure it opens a request for the team or a ticket in the help desk instead of guessing.

Summary

  • Models hallucinate because a fluent guess often scores better than admitting uncertainty.
  • Grounding, permission to say “I don’t know”, citations and hand-off reduce it.
  • No technique removes it completely; measure hand-offs as success and keep reviewing conversations.

Sources

  1. Why Language Models Hallucinate (Kalai et al., 2025) – arXiv
  2. Reduce hallucinations – Anthropic documentation