What is Hallucination?

A hallucination is when an LLM fabricates false or unsourced content in a fluent, confident manner, and the polished phrasing is what makes the errors so hard to catch.
Hallucinations split broadly into factuality hallucinations, which contradict real-world facts, and faithfulness hallucinations, which stray from the given input, context, or instructions.
Their causes span the data stage (misinformation and knowledge cutoffs), the training stage (alignment and side effects of human feedback), and the inference stage (decoding and overconfidence); OpenAI research argues that evaluations rewarding only correct answers actively push models to guess.
Mitigations include RAG (retrieval-augmented generation) and grounding that tie answers to external evidence, improved decoding, and prompting for expressions of uncertainty, though even RAG does not eliminate hallucinations entirely.
As shown by the "glue on pizza" incident in Google's AI Overviews, hallucinations in generative search bear directly on brand trust, which makes accurate, citable content a core line of defense from a GEO standpoint.

What Is a Hallucination

A hallucination refers to a large language model generating false or unverifiable content as a fluent, confident statement, regardless of the facts it was trained on or the evidence it was given. Because the model is designed to predict the next word probabilistically, it does not "know" the right answer so much as it produces the "most plausible sentence." The result is nonexistent papers, court cases, or people; wrong dates and figures; and distorted quotations, all rendered in natural prose, and the smooth phrasing makes the errors difficult for users to spot.

The term originated in natural language generation research, where the survey by Ji et al. (2023) distinguished intrinsic hallucinations, which contradict the input, from extrinsic hallucinations, which cannot be verified against the input. With the arrival of the LLM era, Huang et al. (2023, arXiv:2311.05232) refined this distinction further.

Types of Hallucination

The taxonomy from Huang et al. (2023) organizes hallucinations along two axes: whether the content contradicts real-world facts, and whether it stays faithful to the input and context the user provided.

Category	Definition	Subtypes / Examples
Factuality hallucination	Generated content contradicts verifiable real-world facts	Factual contradiction (attributing the invention of the telephone to Edison rather than Bell), factual fabrication (inventing nonexistent papers or statistics)
Faithfulness hallucination	Output departs from the given input, context, or instructions, or contradicts itself	Instruction inconsistency (answering something other than what was asked), context inconsistency (conflicting with the provided document), logical inconsistency (internal contradictions in the reasoning)

Factuality hallucinations break down further into factual contradiction, where a real entity is described incorrectly, and factual fabrication, where content that cannot be verified at all is simply invented. Faithfulness hallucinations cover cases that violate the user's instructions, conflict with the provided context, or are logically inconsistent during reasoning. Context faithfulness becomes especially important in settings like RAG, where answers are grounded in external documents.

Why It Happens: Causes

Huang et al. (2023) group the causes of hallucination into three stages of the model lifecycle.

Data stage — Misinformation and bias mixed into the training corpus, the model's knowledge boundary (it cannot know facts that postdate its training), and poor alignment data all plant flawed knowledge.
Training stage — Structural limitations of pretraining, alignment mismatches in supervised fine-tuning (SFT), and the tendency toward sycophancy that arises from reinforcement learning with human feedback (RLHF) all contribute.
Inference stage — Imperfect decoding, overconfidence, the softmax bottleneck, and reasoning failures induce hallucinations at the output stage.

In September 2025, OpenAI researchers (Kalai, Nachum, Vempala, Zhang, arXiv:2509.04664) offered a more fundamental statistical explanation. Their argument has two central claims. First, during pretraining, hallucinations are a statistically inevitable error. If 20% of "rare facts" appear only once in the training data, the model is estimated to hallucinate on at least 20% of such questions. Second, hallucinations persist because most evaluation benchmarks award zero points for an "I don't know" answer and credit only correct answers. Under such scoring, guessing yields a higher expected score even when the model is uncertain, training the model to confidently produce wrong answers. The researchers analyzed major benchmarks such as MMLU-Pro, GPQA, and SWE-bench and found that they penalize expressions of uncertainty.

Mitigation Strategies

Hallucinations are hard to eliminate completely, but several techniques can substantially reduce their frequency and impact.

RAG (retrieval-augmented generation) and grounding — Tying answers to documents retrieved from an external knowledge base constrains the model to cite and summarize known evidence, and it provides source traceability as well. That said, RAG is not a cure-all: hallucinations still occur if retrieval goes wrong or the model ignores the retrieved results.
Data cleaning and model editing — Filtering misinformation out of the training data and directly correcting wrong facts.
Improved decoding — Applying decoding strategies that increase factuality and faithfulness.
Prompting for uncertainty and reforming evaluation — Rather than building new hallucination benchmarks, the OpenAI researchers propose redesigning the incentives in existing benchmark scoring, for example by introducing explicit confidence thresholds such as "penalize wrong answers, award zero for I don't know," so that saying "I don't know" becomes a reasonable choice.

Real-World Cases and Evidence

Hallucinations have moved beyond an academic concept and led to real-world harm.

Mata v. Avianca (2023) — In the U.S. District Court for the Southern District of New York, lawyers submitted a brief citing fake case law that ChatGPT had invented. The hallucinated cases came complete with judge names, docket numbers, and quotations, and a web of fictitious precedent formed in which fake cases cited other fake cases. Judge Castel imposed a $5,000 sanction for violating Rule 11 of the Federal Rules of Civil Procedure.
Google AI Overviews (2024) — Launched in the U.S. in May 2024, Google AI Overviews made headlines for answers such as "put glue on your pizza" and "eat one rock a day" (mistakenly drawn from a piece by the satirical outlet The Onion). CEO Sundar Pichai described hallucination as "an unsolved problem and, in some ways, an inherent feature."
Fabricated dates and identities — In the OpenAI study (arXiv:2509.04664), the DeepSeek-V3 model, when asked for one author's birthday, returned three different wrong dates despite an instruction to "answer only when certain," and several models each invented different titles, years, and universities for the same person's doctoral dissertation.

Implications from an SEO and GEO Perspective

In a generative search (GEO) environment, hallucinations operate in two directions. First, AI may describe your brand incorrectly. When evidence is insufficient, an AI search engine may distort your products, pricing, or facts in its answer, which translates directly into eroded brand trust. Second, generative search tools such as AI Overviews, ChatGPT, and Perplexity tend to prefer citing trustworthy, clearly defined sources when constructing answers, so the very way models work to reduce hallucinations becomes an opportunity for content strategy.

In practice, the following are recommended.

State core facts (company name, year founded, product specifications, pricing) on the page explicitly and consistently to reduce the room for AI to guess.
Strengthen the verifiability of your content with structured data (schema markup) and clear definitions, statistics, and citations. This is the kind of material AI is well suited to use as evidence.
Provide direct-answer paragraphs for questions so the model does not fill gaps with content of its own invention.
Regularly check how your brand is described in AI answers, and maintain trustworthy sources (official pages and documentation) that can correct any misinformation.