Should AI highlight the parts of its output it's unsure about?
Yes, if the highlights mark what's likely wrong: highlighting likely-to-be-edited code sped people up, but highlighting raw low-probability tokens didn't help.
Promising, tested in lab or early LLM studies
Test it with your users.
One small controlled study and two crowd experiments with LLM tools.
2 studies · 2 with LLMs · 2023–2024 · How we rate
Highlight the specific parts most likely to need fixing, not raw token probabilities.
- Base highlights on what users are likely to change, not generation probability.From the study
- Keep highlights granular, informative and not overwhelming.From the study
- Use confidence colour-coding on uncertain facts in answers.From the study
- Explain what a highlight means on hover.Our reading
What they found
Vasconcelos et al. (2024)experiment
Mixed-methods study with 30 programmers comparing code completions with no highlighting, highlighting the lowest-probability tokens, or highlighting tokens most likely to be edited. Highlighting likely-to-be-edited tokens led to faster task completion and more targeted edits and was preferred; highlighting by generation probability gave no benefit over no highlighting.
“highlighting tokens according to their probability of being generated does not provide any benefit over the baseline with no highlighting”
Vasconcelos et al., 2024 · Abstract
Spatharioti et al. (2023)experiment
Two online experiments (90 and 120 MTurk participants) on product-comparison decisions, randomly using an LLM-based search tool or traditional search. LLM users finished faster with fewer but more complex queries and were more satisfied, with similar accuracy when the LLM was right, but they over-relied on it when it erred. In experiment 2, colour-coded confidence highlighting substantially increased how often people spotted incorrect information.
“this confidence-based highlighting substantially increases the rate at which users spot incorrect information, improving the accuracy of their overall decisions while leaving most other measures unaffected”
Spatharioti et al., 2023 · Abstract
Trade-offs and when it doesn't apply
- 30 programmers in one study.
- Building an edit-likelihood model takes data.
- Too many highlights become noise.
What it looks like
How to check it in your product
Compare time-to-correct and missed errors with and without highlighting.
Open questions
- What signal best predicts errors in prose rather than code?
- How many highlights per answer is too many?
- Do highlights reduce checking of unhighlighted parts?
FAQ
Does highlighting low-probability tokens help programmers?
No. Vasconcelos et al. (30 programmers) found it gave no benefit over no highlighting.
What kind of highlighting helps?
Highlighting tokens most likely to be edited led to faster completion and more targeted edits.
Sources
- Generation Probabilities Are Not Enough: Uncertainty Highlighting in AI Code CompletionsHelena Vasconcelos, Gagan Bansal, Adam Fourney, Q. Vera Liao, Jennifer Wortman Vaughan (2024). ACM TOCHI (preprint, arXiv:2302.07248v3, Nov 2024).
- Comparing Traditional and LLM-based Search for Consumer Choice: A Randomized ExperimentSofia Eleni Spatharioti, David M. Rothschild, Daniel G. Goldstein, Jake M. Hofman (2023). arXiv:2307.03744v2 (Nov 2023).