Do hallucination warnings help users spot AI errors?
Yes. In experiments, a simple warning about possible inaccuracies and colour-coded uncertainty highlighting both helped people catch wrong LLM answers, without making them distrust correct ones.
Promising, tested in lab or early LLM studies
Test it with your users.
Two controlled experiments with LLM output in lab tasks.
2 studies · 2 with LLMs · 2023–2024 · How we rate
Warn about possible errors where they matter, and mark the specific claims that are uncertain.
- Show a short warning that answers may contain inaccuracies.From the study
- Highlight the specific uncertain parts of an answer rather than only warning in general.From the study
- Expect people to over-rely on fluent LLM answers when the model is wrong.From the study
- Keep warnings near the answer, not buried in the footer.Our reading
What they found
Nahar et al. (2024)experiment
419 Prolific participants rated the accuracy of, and reacted to (like, dislike, share), LLM answers that were genuine or contained minor or major hallucinations, with or without a warning about possible inaccuracies. People rated genuine content most truthful, then minor and then major hallucinations. The warning improved hallucination detection without significantly lowering the perceived truthfulness of genuine content.
“we observed that warning improved the detection of hallucination without significantly affecting the perceived truthfulness of genuine content”
Nahar et al., 2024 · Abstract
Spatharioti et al. (2023)experiment
Two online experiments (90 and 120 MTurk participants) on product-comparison decisions, randomly using an LLM-based search tool or traditional search. LLM users finished faster with fewer but more complex queries and were more satisfied, with similar accuracy when the LLM was right, but they over-relied on it when it erred. In experiment 2, colour-coded confidence highlighting substantially increased how often people spotted incorrect information.
“this confidence-based highlighting substantially increases the rate at which users spot incorrect information, improving the accuracy of their overall decisions while leaving most other measures unaffected”
Spatharioti et al., 2023 · Abstract
Trade-offs and when it doesn't apply
- Warnings were tested in short lab tasks; habituation over weeks wasn't measured.
- Highlighting only helps if the uncertainty signal is accurate.
- Neither study tested legal or medical stakes directly.
What it looks like
How to check it in your product
Seed known-wrong answers and compare detection rates with and without the warning or highlighting.
Open questions
- Do users tune out generic warnings after repeated exposure?
- Which uncertainty signals are accurate enough to highlight at claim level?
- Does a warning shift blame or liability perceptions?
FAQ
Do AI hallucination warnings make users distrust correct answers?
Not significantly. Nahar et al. (419 participants) found the warning improved hallucination detection without significantly lowering perceived truthfulness of genuine content.
Does highlighting uncertain parts of an LLM answer help?
Yes. In Spatharioti et al., colour-coded confidence highlighting substantially increased how often people spotted incorrect information and improved their decisions.
Sources
- Fakes of Varying Shades: How Warning Affects Human Perception and Engagement Regarding LLM HallucinationsMahjabin Nahar, Haeseung Seo, Eun-Ju Lee, et al. (2024). COLM 2024.
- Comparing Traditional and LLM-based Search for Consumer Choice: A Randomized ExperimentSofia Eleni Spatharioti, David M. Rothschild, Daniel G. Goldstein, Jake M. Hofman (2023). arXiv:2307.03744v2 (Nov 2023).