Hallucinations: Why AI Makes Things Up

You ask an AI for a biography of a minor historical figure. It produces a beautifully written, citation-heavy summary. It looks perfect.

The only problem? The person never existed.

This phenomenon is known as “Hallucination” (or confabulation). It is one of the most persistent and dangerous quirks of Large Language Models (LLMs). But why does a machine built on logic and data simply lie?

It’s Not Lying, It’s Predicting

To understand hallucinations, you have to understand what an LLM actually is. It is not a database of facts; it is a probabilistic next-token predictor.

When you ask, “Who was the first person to walk on Mars?” the AI isn’t looking up a fact. It is calculating: Given the sequence of words in the prompt, what word is statistically most likely to come next?

If the model has seen enough sci-fi in its training data, it might confidently answer “Captain Mark Watney” (from The Martian) because those words statistically cluster together, even though it’s fiction.

The Sources of Hallucination

1. The “Snowballing” Effect

LLMs generate text word-by-word. If an AI makes a tiny error early in a sentence, it commits to that error. To make the sentence grammatically and logically consistent, it must invent more details to support the first lie.

  • Example: If it accidentally starts a legal summary with “In the case of Smith v. Jones…” (a fake case), it will then proceed to invent a judge, a verdict, and legal precedents to make the paragraph flow.

2. Source-Reference Amnesia

Models compress vast amounts of data into parameters (weights). They remember the patterns of information but often lose the source.

  • The model might remember that “eating rocks is good for digestion” was a sentence it processed.
  • It forgets that the sentence came from a satirical article in The Onion, not a medical journal.
  • It repeats the advice as fact.

3. Sycophancy

Models are fine-tuned to be helpful and to follow user instructions. Sometimes, this desire to please overrides the truth.

  • User: “Why is the Golden Gate Bridge purple?”
  • AI (hallucinating): “The Golden Gate Bridge is purple because of a special anti-rust primer used during the 1980 renovation…”
  • Reality: The bridge is orange. But the user’s premise (“it is purple”) tricked the model into being “helpful” by explaining the false premise rather than correcting it.

Famous Case Studies

The Lawyer Who Cited Fake Cases (2023)

A US lawyer used ChatGPT to write a legal brief for a federal court. The AI invented half a dozen court cases (e.g., Varghese v. China Southern Airlines) complete with fake docket numbers and internal citations. The lawyer submitted the brief without checking. The judge was not amused. The lawyer faced sanctions.

The Google Gemini “James Webb” Error (2024)

In a promotional video, Google’s AI claimed the James Webb Space Telescope took the very first picture of an exoplanet. Astronomers immediately pointed out that the VLT (Very Large Telescope) did that in 2004. The error wiped $100 billion off Alphabet’s market cap in a single morning.

Mitigations: Can We Fix It?

We can reduce hallucinations, but “solving” them completely is mathematically difficult.

  1. RAG (Retrieval-Augmented Generation): Instead of relying on the AI’s internal memory, we force it to look up documents in a trusted database (like a company wiki) before answering.
  2. Grounding: Training models to say “I don’t know” rather than guessing.
  3. Citation Linking: Newer models (like Perplexity or GPT-4o with browsing) perform a web search and provide clickable footnotes, allowing users to verify claims instantly.

The Takeaway

Treat LLMs like a brilliant but sleepy intern. They are creative, fast, and articulate, but they will confidently make things up if they don’t know the answer. Trust, but verify.