# Does writing with AI make everyone's content sound the same?

It can: essays written with an instruction-tuned model were more similar to each other, and an opinionated assistant pulled writing toward its stance.

Evidence: Limited (Early signal, treat it as a hypothesis) · LLM fit: Medium (Tested in lab or early LLM studies, likely carries over) (https://aiuxevidence.com/how-we-rate)
One small controlled experiment plus one large related experiment, both with LLMs.

## Design it like this

Offer varied suggestions and keep the user's own voice in charge.

- Offer several diverse suggestions rather than one “best” one. [our reading]
- Measure output diversity across users, not just quality. [from the study]
- Expect the effect to come from model text; encourage user-written content. [from the study]
- Audit stance in suggestions on contested topics. [from the study]

## What they found

### Padmakumar & He (2024), experiment

38 Upwork writers wrote 300 argumentative essays on 10 topics with GPT-3, with InstructGPT, or alone. Writing with InstructGPT (but not GPT-3) significantly reduced diversity: essays by different authors became more similar, with lower lexical and content diversity. The effect came mainly from the model's contributions; users' own text was unaffected.

> “writing with InstructGPT (but not the GPT3) results in a statistically significant reduction in diversity” (Abstract)

### Jakesch et al. (2023), experiment

1,506 Prolific participants wrote a post on whether social media is good for society, with no assistant or a language-model writing assistant set to argue for or against; 500 judges rated their writing. With a pro-social-media model, people were 2.04× as likely as the control group to argue it was good; with a critical model, 2.0× as likely to argue it was bad. Their later survey attitudes shifted too.

> “Using the opinionated language model affected the opinions expressed in participants’ writing and shifted their opinions in the subsequent attitude survey.” (Abstract)

## Trade-offs and when it doesn't apply

- 38 writers, argumentative essays only.
- Base GPT-3 didn't show the effect.
- Some homogeneity (consistent tone) can be desired.

## What it looks like

- Do: Three differently styled suggestions plus “write it yourself”.
- Don't: One polished paragraph auto-inserted for everyone.

## How to check it in your product

Compute similarity between outputs from different users with and without AI help.

## Open questions

- Do newer models increase or reduce homogenization?
- Does offering multiple options fix it?
- What does reduced diversity do to audiences over time?

## FAQ

### Does AI writing assistance reduce diversity?

Padmakumar & He found writing with InstructGPT (not GPT-3) significantly reduced diversity across 300 essays.

### Is it the users or the model?

Mainly the model's contributions; users' own text was unaffected.

## Related questions

- [Can AI writing assistants change what users believe?](https://aiuxevidence.com/e/nudging-opinions)
- [Should you label AI-generated content?](https://aiuxevidence.com/e/ai-labels)
- [Do AI reply suggestions change how people communicate?](https://aiuxevidence.com/e/smart-replies)

## Sources

1. Vishakh Padmakumar, He He (2024). Does Writing with Language Models Reduce Content Diversity?. ICLR 2024. https://arxiv.org/abs/2309.05196
2. Maurice Jakesch, Advait Bhat, Daniel Buschek, Lior Zalmanson, Mor Naaman (2023). Co-Writing with Opinionated Language Models Affects Users' Views. CHI 2023. https://arxiv.org/abs/2302.00560

Canonical: https://aiuxevidence.com/e/homogenization