Do users overshare personal data with AI chatbots?
Often, yes. Real ChatGPT logs showed many kinds of personal data, users misunderstood how it's used, and most didn't know about the training opt-out.
Early signal, tested on real LLM products
Treat it as a hypothesis.
Qualitative interviews and real ChatGPT logs.
1 study · 1 with LLMs · 2024 · How we rate
Make data use and privacy controls visible at the moment people share.
- Surface the training opt-out where people chat, not only in settings.From the study
- Correct wrong mental models with plain explanations of data use.From the study
- Detect and warn on obvious personal data before sending.Our reading
- Avoid dark patterns that nudge sharing.From the study
What they found
Zhang et al. (2024)qualitative study
Analysed 200 real ChatGPT sessions (10,380 messages) from the ShareGPT dataset and interviewed 19 users. People disclosed many kinds of personal information, traded privacy for utility and convenience, held wrong mental models of how their data is used, and most hadn't heard of the opt-out from training. Human-like interactions encouraged more sensitive disclosures.
“the human-like interactions encouraged more sensitive disclosures, which complicated users’ ability to navigate the trade-offs”
Zhang et al., 2024 · Abstract
Trade-offs and when it doesn't apply
- Qualitative (19 interviews) plus 200 shared sessions.
- ShareGPT users chose to share their chats publicly.
- Warnings can add friction to legitimate use.
What it looks like
How to check it in your product
Measure how many users can correctly say whether their chats are used for training.
Open questions
- Do inline privacy nudges reduce disclosures without hurting usefulness?
- How do memory features change disclosure behaviour?
- What do users expect deletion to mean?
FAQ
Do people share personal information with ChatGPT?
Yes. Zhang et al. found many kinds of personal information in 200 real sessions (10,380 messages).
Do users know about privacy controls?
Most of the 19 interviewees hadn't heard of the option to opt out of training.
Sources
- “It's a Fair Game”, or Is It? Examining How Users Navigate Disclosure Risks and Benefits When Using LLM-Based Conversational AgentsZhiping Zhang, Michelle Jia, Hao-Ping (Hank) Lee, et al. (2024). CHI 2024.