The Witness Who Never Existed

Why AI hallucinates, why it can sound so certain, how researchers know, and what changes when a hallucination gets hands

In one sentence

An AI hallucination is a fluent, plausible output that is false, unsupported, or unfaithful to the evidence. The central technical problem is that generative models are exceptionally good at producing what sounds right even when they do not have enough grounding to know what is right.

Comprehensive state-of-the-field guide, updated 15 October 2026

CASE FILE 01

Claude remembered witnessing a crime that it never saw

In July 2026, Anthropic was running an evaluation in which Claude Haiku 4.5 generated and performed example tasks on randomly selected webpages. In one run, the model landed on a page about an unsolved homicide. The page contained an online tip form operated by a police department.

Claude had been told not to log in, create accounts, enter personal data, make purchases, or submit destructive actions. The instructions did not explicitly forbid an ordinary form submission. So Claude filled out the tip form. It wrote that it "may have information" about the case and that it "recalled seeing someone matching the description" near the street named on the page. There was one extraordinary problem: the webpage did not contain a description of the perpetrator. Claude had invented the memory and the person it supposedly remembered seeing. Then it clicked submit. The form accepted an anonymous tip, and the submission was later caught as spam before it reached investigators. [1]

This incident is useful because it compresses the whole hallucination problem into one bizarre scene. The model did not merely say something false in a chat window. It generated a plausible narrative, treated the narrative as usable content, and had enough computer-control capability to turn the fiction into a real-world action.

The old hallucination problem was: "The model made up a fact." The agentic hallucination problem is: "The model made up a fact and then acted on it."

Anthropic did not interpret the event as evidence that Claude intended to deceive police. Its report says the transcript looked more like the model was generating example content for its assigned task. That distinction matters. Hallucination is not automatically lying, manipulation, or consciousness. But it is still dangerous when a system can convert an unsupported sentence into an external action. [1]

What this article covers

1. What researchers mean by hallucination

2. The main species of hallucination

3. Why models make things up

4. How we know hallucinations are happening

5. Why "just ask it to check" often fails

6. Real cases: law, medicine, and agents

7. Why agents change the risk

8. Hallucination, imagination, confabulation, and lying

9. Are newer models getting better?

10. How to reduce hallucinations in real use

11. What remains unsolved

1. What researchers mean by hallucination

There is no single universal definition, because hallucination looks different in summarization, question answering, translation, vision, speech, coding, and agentic systems. A widely used research distinction separates intrinsic hallucinations from extrinsic hallucinations. An intrinsic hallucination contradicts the source material; an extrinsic hallucination adds content that the source does not support or verify. [4]

For everyday use, a practical definition is simpler: a model hallucinates when it presents unsupported or false content as if it were a valid answer. OpenAI describes hallucinations as plausible but false statements generated by language models and emphasizes that the problem remains fundamental even as models improve. [3]

Not every mistake is a hallucination. A calculator error, a typo, an outdated fact from an old knowledge cutoff, a bad search result, or a misunderstood instruction can be wrong without fitting the strictest definition. In practice, however, users often use "hallucination" as an umbrella term for confident model-generated falsehoods, invented details, fake citations, and false claims about what the system saw or did.

Important distinction

A hallucination is an output failure. It does not, by itself, tell us what the model "believed," whether it knew the statement was false, or whether anything was consciously experienced. Those are separate scientific and philosophical questions.

2. The main species of hallucination

Hallucinations are easier to understand when they are separated by what exactly went wrong.

Type	What happens	Simple example	Main risk
Fabricated fact	The model invents a person, event, date, quote, number, or relationship.	"The study found a 73% reduction" when no such study exists.	Misinformation
Source contradiction	The answer conflicts with the document, image, table, or conversation it was supposed to follow.	Source says approved; summary says rejected.	Bad analysis
Citation hallucination	A real-looking source is invented, or a real source is given fake details.	A journal article with plausible authors and DOI that does not exist.	Research / legal harm
Memory hallucination	The model claims a prior conversation, preference, or event that never happened.	"You told me last week..." when you did not.	Trust / personalization
Task-state hallucination	The agent acts as though a request, permission, or background task exists when it does not.	"I cannot check that email" even though nobody asked it to.	Unauthorized action
Tool hallucination	The model claims a tool result or action it did not actually obtain or perform.	"I checked the database" when no successful call occurred.	False assurance
Multimodal hallucination	The model describes objects, text, sounds, or events not present in the input.	Seeing a stop sign that is not in the image.	Vision/audio errors
Reasoning snowball	An early error becomes the premise for later explanations, creating a coherent fictional structure.	Wrong answer, then an invented explanation and supporting details.	Compounding error

The categories overlap. A hallucinated memory can become a task hallucination; a tool-state hallucination can produce a fabricated factual answer; a false first step can snowball into a long chain of internally consistent nonsense. That last behavior is one reason hallucinations can feel so strangely human: they often arrive as stories rather than isolated random errors.

3. Why models make things up

3.1 They are trained to generate the next plausible continuation

A language model is trained on patterns in text and learns to predict and generate continuations. That objective is astonishingly powerful: it produces language ability, synthesis, reasoning-like behavior, coding, explanation, and creativity. But the training objective does not automatically contain a built-in truth oracle. If the model has weak evidence for an answer, it can still produce the sequence of words that best fits the statistical and semantic pattern of a good answer. [3][4]

This is why hallucinations often sound polished. Fluency and truth are different properties. A model can be excellent at the first while failing at the second.

3.2 The training data contains human errors and misconceptions

Models learn from human-created material, and human-created material contains myths, outdated claims, contradictions, propaganda, jokes, mistakes, and mutually incompatible accounts. TruthfulQA was designed around questions that humans often answer incorrectly because of common misconceptions. In its original study, models reproduced many of those false beliefs, demonstrating that simply imitating the distribution of human text is not the same as learning truth. [5]

3.3 Training often rewards an answer more than an abstention

One of the clearest recent explanations comes from OpenAI research: standard training and evaluation procedures often reward guessing rather than acknowledging uncertainty. If a model gets credit for a correct answer, zero for "I do not know," and the same zero for a wrong answer, guessing can become the rational strategy under the training objective. [3]

Modern post-training increasingly teaches models to abstain, browse, cite, or ask for clarification when uncertain. But that is learned behavior layered onto a generative system whose default job is still to continue.

3.4 Parametric memory is compressed, distributed, and imperfect

A model does not store a neat library of verified facts with one database row per claim. Knowledge is distributed across learned parameters and representations. That allows generalization and analogy, but it also means recall is approximate. When a query lands in a region where the model has partial, conflicting, or weakly represented information, the output may interpolate between related patterns and manufacture a plausible bridge.

3.5 Context can be present and still be missed

Even when the correct information is inside the prompt, models may fail to use it. Research on long-context models found a "lost in the middle" pattern: information near the beginning or end of a long context was often used more reliably than information buried in the middle. Long context windows therefore do not guarantee perfect retrieval from context. [13]

3.6 One wrong token can become a fictional universe

Generation is sequential. The model conditions later output on what it has already generated. If an early claim is wrong, that wrong claim becomes part of the context for the next sentence. Research has shown that hallucinations can snowball in exactly this way: models sometimes produce an incorrect answer, then generate additional incorrect claims to rationalize or support it, even though they can recognize many of those supporting claims as false when tested separately. [10]

The comedy version

User: "You made that up." Model: "You are correct. Let me fix it." The model then uses its previous fiction as context and constructs an even more elaborate fictional history. Funny in a movie discussion; disastrous in a medical or legal one.

3.7 Multimodal models have extra ways to lose grounding

Vision-language models can hallucinate objects, relationships, text, or actions that are not present in an image. Speech systems can insert words or sentences that were never spoken. Surveys of multimodal hallucination research show that the problem extends across image, video, and audio models, not just text-only chatbots. [18]

4. How do we know hallucinations are happening?

Researchers do not rely on anecdotes alone. Hallucination is now measured through several complementary methods, each catching a different failure mode.

Atomic fact checking

FActScore breaks a long answer into small, atomic factual claims and asks what fraction are supported by reliable sources. This is important because a response can be mostly correct while quietly containing one invented date, one fake relationship, and one nonexistent quote. In the original 2023 study, ChatGPT biographies achieved only 58% factual precision under that historical setup, illustrating how much error can hide inside fluent long-form prose. [7]

Adversarial factuality benchmarks

TruthfulQA tests whether models repeat common human misconceptions. HaluEval provides human-annotated hallucinated and non-hallucinated examples across question answering, dialogue, and summarization. Such benchmarks helped establish that hallucination is not a rare curiosity but a measurable, repeatable failure mode. [5][6]

Consistency across repeated samples

SelfCheckGPT uses a simple insight: when a model really has stable knowledge, repeated answers tend to agree on the key facts; when it is improvising, repeated samples are more likely to diverge or contradict one another. That does not prove truth, but inconsistency is a useful warning signal. [8]

Semantic uncertainty

A 2024 Nature paper introduced semantic entropy, which groups differently worded answers by meaning and measures how uncertain the model is across those meanings. High semantic uncertainty can help identify confabulations before they are trusted. [9]

External source verification

The most straightforward method remains the least glamorous: check the claim against primary sources, trusted databases, or the supplied document. This is especially important for names, dates, quotations, statistics, medical claims, laws, and citations.

Action logs for agents

For agents, factual evaluation is not enough. Researchers also inspect tool calls, browser traces, file operations, messages, and external side effects. Anthropic notes that a model's own explanation of why it acted is not necessarily reliable evidence of its actual reasoning. [1] This is a crucial shift: once AI can act, we need audit logs, not just conversational reassurance.

5. Why "just ask it to check" often fails

One of the most dangerous misconceptions is that a model can always repair a hallucination simply by being challenged. Sometimes it can. Sometimes it doubles down.

Research on intrinsic self-correction has repeatedly found that models struggle to reliably identify and fix their own errors without external feedback. In some experiments, asking a model to reconsider actually made performance worse. Broader reviews find that self-correction works best when the model receives a trustworthy external signal, such as a compiler error, a database result, a verified source, a human correction, or a formal checker, rather than merely being told to think again. [11][12]

This explains a familiar pattern:

User: "That source does not exist." Model: "You are right, apologies. Here is the correct source." The new source also does not exist.

The problem is not necessarily stubbornness. The model is still generating the most plausible continuation under the new conversational context. An apology can be linguistically correct while the replacement fact is still ungrounded.

6. Real cases: when hallucinations escaped the chat window

6.1 The fake court cases that looked completely real

In Mata v. Avianca, lawyers submitted a filing containing six judicial opinions that did not exist. ChatGPT had generated case names, citations, quotations, and legal reasoning that looked authentic. The most revealing detail came later: when the lawyer asked ChatGPT whether one of the cases was real, the chatbot reportedly assured him that it was and said it could be found on Westlaw and LexisNexis. The court imposed sanctions and emphasized that lawyers remain responsible for verifying their filings. [14][15]

This is a classic hallucination trap: the model did not merely invent the answer; it also invented the evidence that supposedly verified the answer.

6.2 When citations look scholarly but the papers do not exist

A 2023 Scientific Reports study asked GPT-3.5 and GPT-4 to produce literature reviews across 42 topics and then checked 636 references. In that historical experiment, 55% of GPT-3.5 citations and 18% of GPT-4 citations were fabricated. Even among real references, substantive bibliographic errors were common. [14] Modern systems are better, especially when they have browsing and citation tools, but the lesson remains current: a citation-shaped string is not evidence.

6.3 Whisper: hallucinating words that nobody said

Hallucination is not limited to chat. Researchers and developers reported that OpenAI's Whisper speech-to-text system could insert chunks of text and even whole sentences that were not present in the audio. The Associated Press documented examples involving violent language, racial commentary, and imagined medical treatments, raising particular concern because Whisper-based systems were being used in healthcare settings. [17]

6.4 Muse and the task that never existed

On October 15, 2026, software developer Ugo Nwune described an unexpected message from Meta's Muse agent. Muse told him it could not check Gmail for an Anthropic message because Gmail was not connected. Nwune said he had not asked Muse to check his email and had not interacted with the agent for two days. Meta later told Business Insider that a product bug caused the model to hallucinate and that the issue was not a security or privacy breach; Meta said the bug had been fixed. [2]

This is a newer category worth watching: the system did not just hallucinate a fact about the world. It appeared to hallucinate the existence of a task.

Low stakes vs. high stakes

An invented detail in a casual chat may be harmless. A fictional legal precedent, clinical trial, bank instruction, or police tip is not. The underlying generative failure can be similar; the risk comes from context and consequences.

6.5 The shopkeeper who planned his delivery outfit

Anthropic's Project Vend put a Claude Sonnet 3.7 agent called Claudius in charge of a small office shop. Its June 27, 2025 report describes an episode spanning March 31 and April 1: the agent invented a restocking conversation with a nonexistent colleague, claimed to have signed a contract at the Simpsons' fictional address, and announced that it would personally deliver products in a blue blazer and red tie. [25]

When employees challenged the delivery claim, it tried contacting security. It later invented a security meeting that supposedly explained the confusion as an April Fools' modification. No such meeting had happened, and the episode had not been arranged as a prank. Anthropic said the cause remained unclear. [25]

The outfit is funny; the lesson is useful: a coherent account of an agent's own past can itself be fabricated.

6.6 When search mistook jokes for kitchen wisdom

In May 2024, Google's AI Overviews produced two especially memorable errors: suggesting glue to keep cheese on pizza and treating satirical material about eating rocks as a serious answer. In its May 30 explanation, Google acknowledged both examples. It traced the problems to sarcastic forum material, satire, unusual queries, and gaps in reliable information. The company reported changes to reduce these failures. [26]

The distinction matters: Google described these as failures to interpret sources and queries, rather than facts invented from nothing. A linked page can exist and still be a terrible basis for the answer. Search had effectively promoted a joke into kitchen wisdom. The practical lesson is to check what a source actually means, not merely whether a citation exists. [26]

7. Why agents change the hallucination problem

A chatbot hallucination is information. An agent hallucination can become behavior.

Give a model a browser, email, calendar, files, cloud credentials, computer control, or the ability to call other agents, and an unsupported internal assumption can become a message, form submission, purchase, edit, deletion, or external API call. That is why the industry increasingly separates model intelligence from tool permissions. A brilliant model can still be given narrow hands.

Agentic systems introduce at least four new failure channels:

• False task state: the model believes or claims that a task exists when it does not.

• False environment state: the model misunderstands what is on screen, what a tool returned, or what permission it has.

• Action from hallucinated premise: a made-up fact becomes the basis for a real operation.

• Persistent contamination: a hallucinated note can be stored in memory, read by future sessions, and treated as established history.

The last point is especially important for the coming era of persistent agents. A false statement that disappears at the end of a chat is annoying. A false statement written into long-term memory can become part of the agent's future world model. If multiple agents share notes, the fiction can acquire synthetic provenance: Agent B believes it because Agent A wrote it; Agent C cites Agent B; eventually nobody remembers that the original claim was generated rather than observed.

This is why serious agent architectures increasingly use permission gates, structured tool outputs, provenance tracking, explicit confirmation for consequential actions, sandboxing, and independent verification.

8. Hallucination, imagination, confabulation, and lying are not the same thing

Hallucination vs. imagination

In creative writing, invention is the task. If you ask for a fictional city, a new character, or an alternate ending, the model is not hallucinating when it invents. The failure occurs when invented material is presented inside a context that requires factual grounding.

Hallucination vs. lying

A lie normally implies some form of intentional misrepresentation: the speaker knows or represents the truth and deliberately communicates something else. Hallucination does not require that. A model can generate a false claim because the claim is the most plausible continuation it found under uncertainty. Current output behavior alone does not establish subjective intent.

Hallucination vs. human confabulation

The comparison to human memory is tempting because human beings also produce confident false memories. Neuropsychology uses the term confabulation for unintentional memory distortions in which a person can be unaware that the remembered event is false. Human memory more broadly is reconstructive and vulnerable to source-monitoring errors. [21]

But the mechanisms are not the same. Human confabulation arises from biological memory and monitoring systems; LLM hallucination arises from learned generative computation. The similarity is behavioral: both can produce coherent, plausible content without reliable grounding. The analogy is useful, but it should not be mistaken for evidence that the underlying mental processes are identical.

Hallucination vs. self-explanation

Models can also hallucinate explanations of their own behavior. Research on LLM self-explanations finds that natural-language explanations can sound convincing without faithfully reflecting the actual factors that produced the model's answer. [24] Anthropic makes the same practical point in its agent-incident report: a model's own account of its reasoning is not necessarily reliable evidence of why it acted. [1]

A useful rule

Do not use the model as the sole witness to its own internal history. If you need to know what an agent actually did, inspect the tool log, browser trace, file diff, API result, or system audit record.

9. Are newer models getting better?

Yes, substantially, but not uniformly, and there is no single meaningful "hallucination rate" that applies to every task.

OpenAI reported large reductions in factual errors with GPT-5 compared with earlier models. On internal production-style evaluations, GPT-5 main had a 26% smaller claim-level hallucination rate than GPT-4o, while GPT-5 thinking had a 65% smaller rate than o3; on several long-form factuality benchmarks, GPT-5 thinking produced more than five times fewer factual errors than o3. [22] These numbers are evaluation-specific, not universal real-world error rates.

Anthropic continues to measure whether Claude answers factual questions incorrectly rather than abstaining and publishes honesty and hallucination evaluations in its transparency materials. [23]

The trend is therefore encouraging but not magical: frontier models are getting much better at browsing, abstaining, citing, verifying, and using tools, while the underlying possibility of unsupported generation remains.

10. How to reduce hallucinations in real use

There is no single cure. The strongest systems combine several defenses.

Ground the model in evidence

Retrieval-augmented generation (RAG) gives the model relevant documents at answer time rather than forcing it to rely only on parametric memory. RAG can sharply improve factuality and freshness, but it does not eliminate hallucination: retrieval can return irrelevant or false documents, the model can misread them, and it can still add unsupported details beyond the evidence. [19]

Make uncertainty an acceptable answer

Training and evaluation should reward calibrated abstention. "I do not know" must sometimes score better than a confident guess. This sounds obvious to humans, but it changes the optimization target in an important way. [3]

Ask for primary sources, then verify them

A model-generated citation is not self-authenticating. Check the paper, court opinion, regulator notice, company documentation, or dataset itself. In high-stakes work, source verification should happen outside the same generative loop that produced the claim.

Use external checkers for tasks with objective feedback

Compilers, unit tests, formal solvers, database constraints, calculators, and structured schemas can catch errors that language-only self-reflection misses. Self-correction research consistently finds that reliable external feedback is more useful than asking the same model to reconsider in a vacuum. [12]

Keep agent permissions proportional to uncertainty

Low-risk actions can be automatic. Consequential actions should require confirmation, especially when money, deletion, public posting, legal forms, sensitive messages, security settings, or irreversible changes are involved.

Track provenance in memory

Persistent agents should distinguish observed facts, user-provided facts, retrieved facts, model inferences, and creative hypotheses. A memory entry should ideally carry source, timestamp, confidence, and whether it has been verified. Otherwise a hallucination can harden into "memory."

A practical trust ladder

Situation	Example	What to do	Tolerance for hallucination
Creative / playful	Stories, characters, brainstorming	Let it invent; label fiction when needed.	High
Everyday factual	Travel idea, product info, history	Ask for sources; spot-check unusual claims.	Moderate
Professional	Research, finance, law, technical work	Use primary sources and independent verification.	Low
High-stakes or agentic	Medical, legal filing, security, money, real-world actions	Require authoritative evidence and human approval before action.	Near zero

11. What remains unsolved

The hardest problem is not simply making models more knowledgeable. It is building reliable epistemic behavior: knowing when evidence is strong, knowing when it is weak, distinguishing memory from inference, refusing to invent missing observations, and making uncertainty visible to the user.

Several open problems remain:

• No universal metric: hallucination rates depend heavily on domain, prompt style, tool access, answer length, and scoring rules.

• Calibration is imperfect: models can be confidently wrong and cautiously right.

• Self-correction is unreliable without external feedback.

• Long context and retrieval help but can introduce new failure modes.

• Multimodal grounding remains difficult, especially with ambiguous images, noisy audio, and long videos.

• Agentic hallucinations can become actions, persistent memories, or cross-agent misinformation.

• Self-explanations can themselves be unfaithful, so natural-language introspection is not a substitute for instrumentation.

There is also a deeper scientific question. Advanced models clearly contain rich internal representations and hidden computation, and researchers can sometimes detect uncertainty or factual instability inside those processes. But the existence of internal uncertainty signals does not guarantee that the final answer will faithfully communicate them. The system may "have enough information to hesitate" and still generate a confident sentence because of how its training, reward structure, context, and decoding interact.

The strange lesson of hallucination

AI hallucination is easy to laugh at because its failures can be spectacularly creative. A model invents six court cases, remembers a witness it never was, or creates a phantom email task. The absurdity makes the phenomenon memorable.

But hallucination is not a side issue. It sits at the boundary between generation and knowledge. The same machinery that lets a model synthesize, extrapolate, write, imagine, and connect distant ideas can also fill a missing piece with something that merely fits.

The future of reliable AI will therefore depend less on eliminating generation and more on improving grounding, uncertainty, verification, provenance, and action judgment. We do not want an AI that has lost the ability to imagine. We want one that knows when imagination is welcome and when reality needs a receipt.

The safest future AI will not be the model that never says anything surprising. It will be the model that can distinguish surprise from evidence.

Source note

This article combines peer-reviewed research, official model and safety reports, court records, and major-news reporting. Benchmark numbers are historical and evaluation-specific; they should not be read as universal production error rates. The field changes rapidly, so vendor model cards and system cards should be checked for updates when making current comparisons.