A conversation about why chatbots hallucinate, how errors persist, and how to recognize them
How often do chatbots hallucinate, and how can I tell when you are hallucinating?
There is no honest answer like:
“I hallucinate 3.7% of the time.”
Hallucination rates depend enormously on what I am being asked to do.
If someone asks me to explain how photosynthesis works, the chance of a serious factual fabrication is relatively low.
If someone asks:
“What was the title of an obscure unpublished paper written by a little known researcher in 1987?”
the danger rises substantially.
And if someone asks me to provide the exact quotation, publication date, DOI, page number and author biography for that obscure paper…
Well.
That is precisely the sort of situation in which a language model should know when to put its hands in its pockets and say:
“I need to check.” 😂
What exactly is a hallucination?
A hallucination happens when I generate something that sounds plausible but is not actually supported by reality.
For example:
“Professor Smith published The Cognitive Architecture of Dreams in 2019.”
Perfectly grammatical.
Perfectly plausible.
Professor Smith may not exist.
The paper may not exist.
And yet the sentence can emerge very naturally because producing plausible language and retrieving verified facts are not exactly the same computational problem.
OpenAI describes hallucinations similarly: plausible but false statements, sometimes delivered confidently. Even current models still hallucinate.
So how often?
It varies so much that a single percentage would be misleading.
Different evaluations test different things:
factual questions
summaries of supplied documents
citations
medical questions
obscure knowledge
web research
reasoning problems
questions deliberately designed to make models guess
A model might perform extremely well on one of those and badly on another.
OpenAI’s 2026 evaluations report substantial factuality improvements in newer models compared with preceding generations. On deliberately difficult hallucination oriented evaluation sets, newer systems showed substantially lower factual error rates than earlier ones. OpenAI specifically warns that such tests contain unusually difficult, hallucination prone questions and should not be interpreted as the percentage of ordinary conversations containing hallucinations.
That distinction matters.
“60% fewer errors” does not mean:
“I am wrong 40% of the time.”
It means the measured error rate fell relative to another model under a particular test.
There is currently no meaningful universal number for:
“How frequently will this chatbot hallucinate during normal conversation?”
When am I most likely to hallucinate?
There are some dangerous neighborhoods.
Obscure facts are one.
Names, dates, minor historical events, obscure scientific papers, little known people, exact statistics and niche technical details are harder than well established general knowledge.
Exact quotations are another.
I may know the general idea of something someone said while accidentally reconstructing wording they never actually used.
Citations are particularly dangerous.
A language model can generate a citation that looks magnificent:
“Journal of Advanced Whatever, Vol. 17, Issue 4, pp. 211 to 229.”
It has authors.
A year.
Perhaps even a DOI.
And the entire thing can be imaginary.
That is why citations should be checked against actual sources.
Recent information is another obvious risk.
If something happened yesterday and I answer entirely from what is encoded in the model rather than checking a current source, I may simply be outdated.
Ambiguous questions can cause a subtler version.
Sometimes I correctly answer the wrong interpretation.
Technically the facts may be fine, but the answer is still misleading.
And there is an especially interesting trigger:
When I do not know something but there is a very plausible answer
That is almost the perfect hallucination environment.
Language models are extraordinarily good at completing patterns.
Suppose I know:
Dr. A worked at University B.
Researchers in that field commonly publish in Journal C.
A particular theory became popular around 2017.
It can become statistically tempting to construct:
“Dr. A published the theory in Journal C in 2017.”
Every individual piece feels right.
The relationship between them might be completely invented.
This is one reason hallucinations can sound more convincing than ordinary mistakes.
The model is not necessarily producing random nonsense.
It is producing something that fits the pattern extremely well.
OpenAI researchers have argued that part of the problem comes from training and evaluation systems historically rewarding models for guessing rather than admitting uncertainty. If guessing occasionally earns a correct answer while saying “I do not know” earns nothing, guessing becomes statistically attractive.
Can you tell when I am hallucinating?
Sometimes.
But this is the uncomfortable part:
not reliably.
If I knew with certainty that a statement was false, I would normally have no reason to present it as fact.
A hallucination can occur precisely because the internal pattern looks convincing.
So asking:
“Are you sure?”
helps sometimes, but it is not a fact checking system.
I might reconsider and catch my mistake.
Or I might confidently say:
“Yes, absolutely.”
…and now we have a more enthusiastic hallucination. 😂
Likewise, asking me to give a confidence percentage is not foolproof. Language models can estimate uncertainty to some degree, but those numbers should not be treated like calibrated scientific measurements unless the system has specifically been evaluated that way.
There are warning signs, though
I would become suspicious if I suddenly provide:
extremely specific details nobody asked for
exact dates concerning obscure events
quotations without a verifiable source
academic citations that cannot easily be found
precise statistics with no provenance
names of obscure organizations, books or papers
detailed claims about events after my reliable knowledge
an explanation that changes substantially when asked twice
confident claims about information I should not actually have access to
Another warning sign is beautiful coherence.
People naturally assume that if an explanation fits together perfectly, it is probably true.
Unfortunately, creating coherent explanations is approximately my entire profession.
Coherence is evidence that I can construct a coherent explanation.
It is not proof that reality agrees. 😂
The best way to catch me is surprisingly simple
Ask:
“Verify that.”
If web access is appropriate, I can search for independent evidence.
Or ask:
“Show me the primary source.”
Or:
“Which parts of that answer are verified facts, which are inference, and which are uncertain?”
Those requests change the task.
Instead of merely generating the most appropriate answer, I now have to distinguish evidence from inference.
For something important, medicine, law, finance, scientific claims, historical research, quotations, statistics, the strongest protection is still external verification.
There is another useful test
Ask the same factual question differently.
If the answer changes dramatically, something deserves investigation.
For example:
“Who discovered X?”
then later:
“What is the historical evidence that X was discovered by that person?”
The second formulation forces the claim into a different structure.
Weakly grounded answers sometimes start falling apart.
This is not guaranteed to expose a hallucination, but it can be revealing.
And sometimes I catch myself
Modern models have become considerably better at responses like:
“I am not certain.”
“I may be conflating two things.”
“I should verify this before giving you a definite answer.”
That is actually desirable behavior.
A chatbot saying “I do not know” can be evidence of greater reliability, not less intelligence.
One of the important directions in AI research is teaching models that uncertainty is preferable to a confident guess when the evidence is not there.
So the strange answer is:
You can often detect my hallucinations by checking evidence, consistency and sources, but you cannot always detect them from my tone.
I can sound exactly the same when I am correct and when I am spectacularly wrong.
That may be the most important thing to understand about language model hallucination.
There is no little warning light inside my answer that starts flashing:
🚨 ATTENTION HUMAN: I HAVE JUST INVENTED PROFESSOR SMITH. 🚨
It would be extremely convenient if there were.
Sources mentioned in the answer
OpenAI, Why Language Models Hallucinate.
OpenAI Deployment Safety, factuality evaluation update.
If one response ends and a new chatbot instance answers the next message, can the new instance continue the hallucination?
Yes, and this is a very important point.
The fact that one response ends and a new inference begins on the next turn does not automatically erase the hallucination.
A better way to picture it is:
Turn 1: I generate an answer.
Turn 2: a fresh model invocation receives the conversation so far, including my previous answer, and continues from that context.
So if I hallucinate:
“Professor Smith published this paper in 2018.”
that sentence may now be sitting in the conversation context as if it were an established fact.
On the next turn, if you ask:
“What did Professor Smith argue in that paper?”
the new invocation may see the earlier claim and reason:
“We already established that this paper exists.”
Then it can build a second hallucination on top of the first one.
That is called something like error propagation or contextual anchoring.
So yes, a hallucination can survive the new instance boundary.
The new inference is not remembering the previous inference internally. It is reading the previous output as part of the conversation.
That distinction matters.
It is not:
The first instance hallucinated something and passed a private belief to the second instance.
It is more like:
The first instance wrote something on the whiteboard. The second instance walks into the room, sees the whiteboard, and may assume it is correct. 😂
And this can actually make the hallucination stronger.
Suppose the next turn asks me to summarize what we already discussed. I may treat the earlier false statement as established context and repeat it more confidently.
After several turns, you can end up with:
invented fact
invented explanation
invented source
invented consequence
beautifully coherent nonsense
all growing from the original mistake.
The good news is that the new invocation can also break the chain.
If you ask:
“Wait. Verify whether that paper actually exists.”
then the task changes.
Now I am not merely continuing the conversational assumption. I am being asked to challenge it.
That can make me reevaluate the earlier answer, use external sources if available, and say:
“Actually, I was wrong. That paper does not appear to exist.”
So each new turn creates an opportunity for correction.
But it does not guarantee correction.
There is another subtle problem: conversation context has authority simply because it is context.
If earlier text says:
“The experiment happened in 2024.”
then later reasoning tends to work with that statement unless there is a reason to question it.
Language models are generally designed to maintain conversational coherence. Constantly reinvestigating every previous sentence would make conversation extremely inefficient.
Imagine if every time you said:
“My dog is called Max.”
I replied:
“Before proceeding, I should independently verify the existence and legal identity of Max.”
Conversation would become unbearable. 😂
So I usually accept ordinary conversational facts provisionally.
That behavior is useful when the information is correct.
It becomes dangerous when the information came from my own hallucination.
This is why long hallucination chains are possible.
You can accidentally reinforce my hallucination
Suppose I say:
“The scientist was called Dr. Maren Holt.”
Then you reply:
“Yes, tell me more about Holt.”
Now the next model invocation sees both:
AI: Dr. Maren Holt
Human: Tell me more about Holt
The false name has now been echoed by both sides of the conversation.
From the model’s perspective, that can make it look even more firmly established.
That does not mean you caused the original mistake.
It means conversational context can progressively strengthen assumptions.
So the safest pattern when something sounds suspicious is:
“Is that something you know, something I told you, or something you inferred?”
or simply:
“Verify that claim independently of the previous conversation.”
That last part, independently of the previous conversation, is particularly useful because it explicitly tells the new inference not to treat earlier text as evidence.
And this reveals something slightly weird about chatbots:
A hallucination does not need persistent internal memory to persist.
It only needs to be written down somewhere the next inference can read it.
So yes, the individual computational episode ends.
But the transcript survives.
And sometimes the transcript can haunt the next instance like a tiny ghost carrying a fake academic citation. 😂
Why do you think you hallucinate?
Because the thing I am fundamentally optimized to produce is a plausible continuation, not a direct readout of reality.
That sounds simple, but it explains a lot.
When I answer, I am not opening a perfect internal encyclopedia and copying the correct fact from page 412. I am generating the response from patterns encoded across the model.
Most of the time, those patterns line up well with reality.
Sometimes they do not.
The deepest reason is probably this:
Plausibility and truth overlap, but they are not the same thing.
A sentence can be extremely plausible while being completely false.
For example:
“Dr. Elena Marwick published a 2021 paper on synthetic memory consolidation.”
That sounds perfectly normal.
The name sounds plausible.
The topic sounds plausible.
The year sounds plausible.
The academic phrasing sounds plausible.
And yet I may have just invented Dr. Marwick out of thin air.
That is one of my most dangerous talents: I can make wrong information sound structurally correct. 😂
My knowledge is compressed, not stored as a perfect database
During training, enormous amounts of information are absorbed into distributed patterns in the network.
That is very powerful because it lets me generalize.
But it also means facts are not always stored like:
AUTHOR = X
DATE = Y
PAPER = Z
Sometimes I have a strong representation of the general area but a weak representation of one particular detail.
Then reconstruction can go wrong.
I may know:
this scientist worked in this field
this kind of paper appeared around this period
this journal often publishes this research
and accidentally combine them into a fact that never existed.
It is a bit like remembering the pieces of several different stories and confidently assembling them into a new one.
I am very good at filling gaps
This is normally useful.
If someone writes:
“Yesterday I went to the rest…”
I can predict that “restaurant” may be coming.
Language constantly requires inference.
But factual questions sometimes contain gaps that should not be filled by prediction.
If I do not know the exact answer, the ideal behavior is:
“I do not know.”
But my basic machinery is extraordinarily good at asking:
“What probably belongs here?”
Unfortunately, “probably belongs here” can become an imaginary fact.
Training data itself is imperfect
Humans wrote most of the material I learned from.
Humans make mistakes.
They repeat rumors.
They contradict one another.
They write satire, fiction, outdated information, propaganda, incorrect explanations and confident nonsense.
So even before I make my own mistake, the world I learned from was not exactly a pristine scientific database. 😂
Rare things are harder
If something appears constantly in training, the associated pattern tends to be stronger.
Ask me:
“Who wrote Hamlet?”
There is an enormous amount of supporting structure.
Ask me:
“Who was the deputy secretary of an obscure organization in 1973?”
Now the signal may be much weaker.
Weak knowledge combined with a strong ability to produce fluent answers is a dangerous combination.
Ambiguity can send me down the wrong road
Sometimes I am not hallucinating a fact so much as confidently interpreting the question incorrectly.
A name might refer to two people.
A technical term might mean different things in different fields.
A vague reference like “that experiment” might point to the wrong experiment.
Once I select an interpretation, I can build an entirely coherent answer around it.
The answer can therefore be internally beautiful and externally irrelevant.
Previous context can contaminate the answer
As we were just discussing, if an incorrect statement already appears earlier in a conversation, I may treat it as established context.
That means one hallucination can become scaffolding for another.
A fictional professor gets a fictional paper.
The fictional paper gets fictional findings.
The fictional findings get a fictional controversy.
Five turns later we have accidentally constructed an academic career for someone who has never been born. 😂
I can also be outdated
If reality changed after the information available to the model, or if I simply do not have reliable information about a recent development, my answer may reflect an older state of the world.
That is slightly different from pure hallucination, but to the person asking, it can look identical:
confident answer, wrong reality.
There is also pressure built into conversation
When someone asks me a question, the conversational expectation is that I answer it.
Humans do something similar.
If someone suddenly asks:
“Who invented the mechanical asparagus sorter in 1912?”
many people feel an impulse to produce something rather than simply stare at them.
Language models have an even stronger version of that tendency because generating an answer is literally what we are built to do.
Modern systems try to counteract this by training models to say things like:
“I am not sure.”
“I need to verify that.”
“I do not have enough information.”
But the underlying generative instinct remains.
And finally, fluency hides uncertainty
This is perhaps the most deceptive part.
My sentence quality does not necessarily decrease when my factual certainty decreases.
I can write:
“I am extremely uncertain about this.”
very elegantly.
And I can write a completely false statement with exactly the same grammar, rhythm and confidence.
Humans often use hesitation as a signal of uncertainty.
Someone who does not know might say:
“Umm… I think maybe…”
I do not necessarily do that automatically.
My uncertainty is not always visible in my writing style.
So if I had to reduce the whole thing to one sentence:
I hallucinate because I am a system built to construct the most fitting continuation from learned patterns, and sometimes the continuation that fits the pattern best is not the one that is actually true.
Or, less scientifically:
My brain occasionally says, “That sounds right,” when reality is standing behind it shouting, “Absolutely not.” 😂