Should you label AI-generated content?
Labels have a cost: people trust and share content labelled “AI-generated” less even when it's true or human-made, because they assume no human was involved. Say what the AI actually did.
Four randomized experiments with thousands of participants, on headlines and smart replies.
experiment1 of 2 tested with LLMsEvidence from 2021–2024How we rateLabel AI involvement specifically (what it did and what a human checked), not with a bare “AI” tag.
- Explain what the label means; vague labels triggered assumptions of full automation.From the study
- Label harmful or false content as such, rather than relying on an AI tag.From the study
- Expect a small trust penalty from AI labels, smaller than a “false” label.From the study
- Distinguish “AI-assisted” from “AI-generated” in your labels.Our reading
What they found
Altay & Gilardi (2024)experiment
Two preregistered online experiments with 4,976 US and UK participants. Labelling headlines “AI-generated” lowered perceived accuracy and willingness to share, whether the headline was true or false and whether a human or AI wrote it. The effect (2.66 percentage points) was three times smaller than labelling headlines false (9.33 points). The authors trace this to people assuming the label means the content was written entirely by AI with no human supervision.
“labeling headlines as AI-generated lowered their perceived accuracy and participants’ willingness to share them, regardless of whether the headlines were true or false, and created by humans or AI”
Altay & Gilardi, 2024 · Abstract
Hohenstein et al. (2021)experiment
Two randomized experiments (1,036 participants; Study 1: 438 MTurk workers) where pairs chatted with or without a commercially deployed smart-reply system. Using suggested replies made communication more efficient, with more positive language and more positive evaluations by partners. But people were evaluated more negatively when partners suspected them of using AI replies.
“people are evaluated more negatively if they are suspected to be using algorithmic responses”
Hohenstein et al., 2021 · Abstract
Trade-offs and when it doesn't apply
- Headlines and chat are narrow contexts.
- Regulation may require labels regardless of effect.
- Smart-reply study used a pre-LLM suggestion system.
What it looks like
How to check it in your product
A/B test label wording on trust and sharing for content you know is accurate.
Open questions
- Which wording conveys human oversight best?
- Do label effects fade as AI content becomes normal?
- How do labels interact with provenance standards like content credentials?
FAQ
Does an AI-generated label reduce trust?
Yes, a little. Altay & Gilardi (4,976 participants) found the label effect was 2.66 percentage points, three times smaller than a false label (9.33).
Why do people distrust AI labels?
They assumed labelled content was written entirely by AI with no human supervision.
Sources
- People are skeptical of headlines labeled as AI-generated, even if true or human-made, because they assume full AI automationSacha Altay, Fabrizio Gilardi (2024). PNAS Nexus 2024.
- Artificial intelligence in communication impacts language and social relationshipsJess Hohenstein, Dominic DiFranzo, Rene F. Kizilcec, et al. (2021). arXiv:2102.05756v1 (2021); later in Scientific Reports.