# Do users overshare personal data with AI chatbots?

Often, yes. Real ChatGPT logs showed many kinds of personal data, users misunderstood how it's used, and most didn't know about the training opt-out.

Evidence: Limited (Early signal, treat it as a hypothesis) · LLM fit: High (Tested on LLM products) (https://aiuxevidence.com/how-we-rate)
Qualitative interviews and real ChatGPT logs.

## Design it like this

Make data use and privacy controls visible at the moment people share.

- Surface the training opt-out where people chat, not only in settings. [from the study]
- Correct wrong mental models with plain explanations of data use. [from the study]
- Detect and warn on obvious personal data before sending. [our reading]
- Avoid dark patterns that nudge sharing. [from the study]

## What they found

### Zhang et al. (2024), qualitative study

Analysed 200 real ChatGPT sessions (10,380 messages) from the ShareGPT dataset and interviewed 19 users. People disclosed many kinds of personal information, traded privacy for utility and convenience, held wrong mental models of how their data is used, and most hadn't heard of the opt-out from training. Human-like interactions encouraged more sensitive disclosures.

> “the human-like interactions encouraged more sensitive disclosures, which complicated users’ ability to navigate the trade-offs” (Abstract)

## Trade-offs and when it doesn't apply

- Qualitative (19 interviews) plus 200 shared sessions.
- ShareGPT users chose to share their chats publicly.
- Warnings can add friction to legitimate use.

## What it looks like

- Do: “This looks like a phone number. Remove it before sending?” with a link to data controls.
- Don't: Privacy controls hidden three menus deep.

## How to check it in your product

Measure how many users can correctly say whether their chats are used for training.

## Open questions

- Do inline privacy nudges reduce disclosures without hurting usefulness?
- How do memory features change disclosure behaviour?
- What do users expect deletion to mean?

## FAQ

### Do people share personal information with ChatGPT?

Yes. Zhang et al. found many kinds of personal information in 200 real sessions (10,380 messages).

### Do users know about privacy controls?

Most of the 19 interviewees hadn't heard of the option to opt out of training.

## Related questions

- [Does making an AI sound human increase user trust?](https://aiuxevidence.com/e/anthropomorphism)
- [Should you label AI-generated content?](https://aiuxevidence.com/e/ai-labels)
- [How should you onboard users to an AI feature?](https://aiuxevidence.com/e/onboarding)

## Sources

1. Zhiping Zhang, Michelle Jia, Hao-Ping (Hank) Lee, et al. (2024). “It's a Fair Game”, or Is It? Examining How Users Navigate Disclosure Risks and Benefits When Using LLM-Based Conversational Agents. CHI 2024. https://arxiv.org/abs/2309.11653

Canonical: https://aiuxevidence.com/e/privacy-disclosure