# Every paper we cite

33 papers behind 25 design questions; 20 tested with LLMs.

- Fabrizio Dell'Acqua, Edward McFowland III, Ethan Mollick, et al. (2026). Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of Artificial Intelligence on Knowledge Worker Productivity and Quality. Organization Science (2026); first released as HBS Working Paper 24-013 (2023). https://doi.org/10.1287/orsc.2025.21838. Evidence type: field experiment. LLM-tested: yes. Supports: [Who benefits most from AI tools at work?](https://aiuxevidence.com/e/who-benefits); [How do you stop users from blindly trusting AI answers?](https://aiuxevidence.com/e/make-users-think)
- K. J. Kevin Feng, Kevin Pu, Matt Latzke, et al. (2026). Cocoa: Co-Planning and Co-Execution with AI Agents. CHI 2026 (arXiv:2412.10999v4). https://doi.org/10.1145/3772318.3791673. Evidence type: system evaluation. LLM-tested: yes. Supports: [How should users and AI agents share control of a task?](https://aiuxevidence.com/e/agent-handoff)
- Hao-Ping (Hank) Lee, Advait Sarkar, Lev Tankelevitch, et al. (2025). The Impact of Generative AI on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects From a Survey of Knowledge Workers. CHI 2025. https://doi.org/10.1145/3706598.3713778. Evidence type: survey. LLM-tested: yes. Supports: [Does relying on AI reduce users' critical thinking?](https://aiuxevidence.com/e/deskilling)
- Hussein Mozannar, Gagan Bansal, Cheng Tan, Adam Fourney, et al. (2025). Magentic-UI: Towards Human-in-the-loop Agentic Systems. arXiv:2507.22358 (Microsoft Research, 2025). https://arxiv.org/abs/2507.22358. Evidence type: system evaluation. LLM-tested: yes. Supports: [When should an AI agent ask for confirmation before acting?](https://aiuxevidence.com/e/ask-before-acting); [How should users and AI agents share control of a task?](https://aiuxevidence.com/e/agent-handoff)
- Chenglei Si, Navita Goyal, Sherry Tongshuang Wu, et al. (2024). Large Language Models Help Humans Verify Truthfulness—Except When They Are Convincingly Wrong. arXiv:2310.12558v2 (Apr 2024). https://arxiv.org/abs/2310.12558. Evidence type: experiment. LLM-tested: yes. Supports: [Should AI show its reasoning or explanations to users?](https://aiuxevidence.com/e/llm-explanations); [Is an AI chatbot better than traditional search for decisions?](https://aiuxevidence.com/e/chat-vs-search)
- Erik Brynjolfsson, Danielle Li, Lindsey Raymond (2024). Generative AI at Work. arXiv:2304.11771v2 (revised Nov 2024). https://arxiv.org/abs/2304.11771v2. Evidence type: field study. LLM-tested: yes. Supports: [Who benefits most from AI tools at work?](https://aiuxevidence.com/e/who-benefits)
- Helena Vasconcelos, Gagan Bansal, Adam Fourney, Q. Vera Liao, Jennifer Wortman Vaughan (2024). Generation Probabilities Are Not Enough: Uncertainty Highlighting in AI Code Completions. ACM TOCHI (preprint, arXiv:2302.07248v3, Nov 2024). https://doi.org/10.1145/3702320. Evidence type: experiment. LLM-tested: yes. Supports: [Do AI coding assistants make code less secure?](https://aiuxevidence.com/e/code-security); [Should AI highlight the parts of its output it's unsure about?](https://aiuxevidence.com/e/highlight-uncertainty)
- Hussein Mozannar, Gagan Bansal, Adam Fourney, Eric Horvitz (2024). Reading Between the Lines: Modeling User Behavior and Costs in AI-Assisted Programming. arXiv:2210.14306v5 (revised Apr 2024). https://arxiv.org/abs/2210.14306v5. Evidence type: observational study. LLM-tested: yes. Supports: [Do inline AI suggestions like Copilot cost users time?](https://aiuxevidence.com/e/inline-suggestions)
- Mahjabin Nahar, Haeseung Seo, Eun-Ju Lee, et al. (2024). Fakes of Varying Shades: How Warning Affects Human Perception and Engagement Regarding LLM Hallucinations. COLM 2024. https://arxiv.org/abs/2404.03745. Evidence type: experiment. LLM-tested: yes. Supports: [Do hallucination warnings help users spot AI errors?](https://aiuxevidence.com/e/hallucination-warnings)
- Michelle Cohn, Mahima Pushkarna, Gbolahan O. Olanubi, et al. (2024). Believing Anthropomorphism: Examining the Role of Anthropomorphic Cues on Trust in Large Language Models. CHI 2024 (author version, arXiv:2405.06079). https://arxiv.org/abs/2405.06079. Evidence type: experiment. LLM-tested: yes. Supports: [Does making an AI sound human increase user trust?](https://aiuxevidence.com/e/anthropomorphism)
- Sacha Altay, Fabrizio Gilardi (2024). People are skeptical of headlines labeled as AI-generated, even if true or human-made, because they assume full AI automation. PNAS Nexus 2024. https://doi.org/10.1093/pnasnexus/pgae403. Evidence type: experiment. LLM-tested: yes. Supports: [Should you label AI-generated content?](https://aiuxevidence.com/e/ai-labels)
- Sunnie S. Y. Kim, Q. Vera Liao, Mihaela Vorvoreanu, Stephanie Ballard, Jennifer Wortman Vaughan (2024). “I'm Not Sure, But...”: Examining the Impact of Large Language Models' Uncertainty Expression on User Reliance and Trust. FAccT 2024. https://arxiv.org/abs/2405.00623. Evidence type: experiment. LLM-tested: yes. Supports: [How should AI show confidence or uncertainty to users?](https://aiuxevidence.com/e/show-confidence)
- Vishakh Padmakumar, He He (2024). Does Writing with Language Models Reduce Content Diversity?. ICLR 2024. https://arxiv.org/abs/2309.05196. Evidence type: experiment. LLM-tested: yes. Supports: [Does writing with AI make everyone's content sound the same?](https://aiuxevidence.com/e/homogenization)
- Zhiping Zhang, Michelle Jia, Hao-Ping (Hank) Lee, et al. (2024). “It's a Fair Game”, or Is It? Examining How Users Navigate Disclosure Risks and Benefits When Using LLM-Based Conversational Agents. CHI 2024. https://arxiv.org/abs/2309.11653. Evidence type: qualitative study. LLM-tested: yes. Supports: [Do users overshare personal data with AI chatbots?](https://aiuxevidence.com/e/privacy-disclosure); [Does making an AI sound human increase user trust?](https://aiuxevidence.com/e/anthropomorphism)
- Helena Vasconcelos, Matthew Jörke, Madeleine Grunde-McLaughlin, et al. (2023). Explanations Can Reduce Overreliance on AI Systems During Decision-Making. CSCW 2023 (arXiv 2022). https://arxiv.org/abs/2212.06823. Evidence type: experiment. LLM-tested: no. Supports: [Do AI explanations help users trust AI appropriately?](https://aiuxevidence.com/e/explanations-trust)
- J.D. Zamfirescu-Pereira, Richmond Y. Wong, Bjoern Hartmann, Qian Yang (2023). Why Johnny Can't Prompt: How Non-AI Experts Try (and Fail) to Design LLM Prompts. CHI 2023. https://doi.org/10.1145/3544548.3581388. Evidence type: qualitative study. LLM-tested: yes. Supports: [Is a blank chat box good UX for AI products?](https://aiuxevidence.com/e/blank-prompt)
- Maurice Jakesch, Advait Bhat, Daniel Buschek, Lior Zalmanson, Mor Naaman (2023). Co-Writing with Opinionated Language Models Affects Users' Views. CHI 2023. https://arxiv.org/abs/2302.00560. Evidence type: experiment. LLM-tested: yes. Supports: [Can AI writing assistants change what users believe?](https://aiuxevidence.com/e/nudging-opinions); [Does writing with AI make everyone's content sound the same?](https://aiuxevidence.com/e/homogenization)
- Neil Perry, Megha Srivastava, Deepak Kumar, Dan Boneh (2023). Do Users Write More Insecure Code with AI Assistants?. CCS 2023. https://doi.org/10.1145/3576915.3623157. Evidence type: experiment. LLM-tested: yes. Supports: [Does relying on AI reduce users' critical thinking?](https://aiuxevidence.com/e/deskilling); [Do AI coding assistants make code less secure?](https://aiuxevidence.com/e/code-security)
- Nelson F. Liu, Tianyi Zhang, Percy Liang (2023). Evaluating Verifiability in Generative Search Engines. Findings of EMNLP 2023. https://arxiv.org/abs/2304.09848. Evidence type: audit. LLM-tested: yes. Supports: [Can you trust citations in AI search answers?](https://aiuxevidence.com/e/citations)
- Shakked Noy, Whitney Zhang (2023). Experimental evidence on the productivity effects of generative artificial intelligence. Science 381 (6654), 2023. https://doi.org/10.1126/science.adh2586. Evidence type: experiment. LLM-tested: yes. Supports: [Who benefits most from AI tools at work?](https://aiuxevidence.com/e/who-benefits)
- Sofia Eleni Spatharioti, David M. Rothschild, Daniel G. Goldstein, Jake M. Hofman (2023). Comparing Traditional and LLM-based Search for Consumer Choice: A Randomized Experiment. arXiv:2307.03744v2 (Nov 2023). https://arxiv.org/abs/2307.03744. Evidence type: experiment. LLM-tested: yes. Supports: [Do hallucination warnings help users spot AI errors?](https://aiuxevidence.com/e/hallucination-warnings); [Is an AI chatbot better than traditional search for decisions?](https://aiuxevidence.com/e/chat-vs-search); [Should AI highlight the parts of its output it's unsure about?](https://aiuxevidence.com/e/highlight-uncertainty)
- Gagan Bansal, Tongshuang Wu, Joyce Zhou, et al. (2021). Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance. CHI 2021 (arXiv 2020). https://arxiv.org/abs/2006.14779. Evidence type: experiment. LLM-tested: no. Supports: [Should AI show its reasoning or explanations to users?](https://aiuxevidence.com/e/llm-explanations); [Do AI explanations help users trust AI appropriately?](https://aiuxevidence.com/e/explanations-trust)
- Jess Hohenstein, Dominic DiFranzo, Rene F. Kizilcec, et al. (2021). Artificial intelligence in communication impacts language and social relationships. arXiv:2102.05756v1 (2021); later in Scientific Reports. https://arxiv.org/abs/2102.05756. Evidence type: experiment. LLM-tested: no. Supports: [Should you label AI-generated content?](https://aiuxevidence.com/e/ai-labels); [Do AI reply suggestions change how people communicate?](https://aiuxevidence.com/e/smart-replies)
- Zana Buçinca, Maja Barbara Malaya, Krzysztof Z. Gajos (2021). To Trust or to Think: Cognitive Forcing Functions Can Reduce Overreliance on AI in AI-assisted Decision-making. CSCW 2021. https://arxiv.org/abs/2102.09692. Evidence type: experiment. LLM-tested: no. Supports: [How do you stop users from blindly trusting AI answers?](https://aiuxevidence.com/e/make-users-think)
- Yunfeng Zhang, Q. Vera Liao, Rachel K. E. Bellamy (2020). Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Making. FAT* 2020. https://arxiv.org/abs/2001.02114. Evidence type: experiment. LLM-tested: no. Supports: [How should AI show confidence or uncertainty to users?](https://aiuxevidence.com/e/show-confidence)
- Carrie J. Cai, Samantha Winter, David Steiner, Lauren Wilcox, Michael Terry (2019). "Hello AI": Uncovering the Onboarding Needs of Medical Practitioners for Human-AI Collaborative Decision-Making. CSCW 2019. https://doi.org/10.1145/3359206. Evidence type: qualitative study. LLM-tested: no. Supports: [How should you onboard users to an AI feature?](https://aiuxevidence.com/e/onboarding)
- Rafal Kocielnik, Saleema Amershi, Paul N. Bennett (2019). Will You Accept an Imperfect AI? Exploring Designs for Adjusting End-user Expectations of AI Systems. CHI 2019. https://doi.org/10.1145/3290605.3300641. Evidence type: experiment. LLM-tested: no. Supports: [How do you set user expectations for an AI that makes mistakes?](https://aiuxevidence.com/e/set-expectations)
- Saleema Amershi, Dan Weld, Mihaela Vorvoreanu, et al. (2019). Guidelines for Human-AI Interaction. CHI 2019. https://doi.org/10.1145/3290605.3300233. Evidence type: guidelines. LLM-tested: no. Supports: [When should an AI agent ask for confirmation before acting?](https://aiuxevidence.com/e/ask-before-acting); [How do you set user expectations for an AI that makes mistakes?](https://aiuxevidence.com/e/set-expectations)
- Zahra Ashktorab, Mohit Jain, Q. Vera Liao, Justin D. Weisz (2019). Resilient Chatbots: Repair Strategy Preferences for Conversational Breakdowns. CHI 2019. https://doi.org/10.1145/3290605.3300484. Evidence type: experiment. LLM-tested: no. Supports: [How should a chatbot recover when it misunderstands?](https://aiuxevidence.com/e/chatbot-repair)
- Berkeley J. Dietvorst, Joseph P. Simmons, Cade Massey (2018). Overcoming Algorithm Aversion: People Will Use Imperfect Algorithms If They Can (Even Slightly) Modify Them. Management Science. https://doi.org/10.1287/mnsc.2016.2643. Evidence type: experiment. LLM-tested: no. Supports: [Should users be able to edit AI-generated output?](https://aiuxevidence.com/e/let-users-edit)
- Berkeley J. Dietvorst, Joseph P. Simmons, Cade Massey (2015). Algorithm aversion: People erroneously avoid algorithms after seeing them err. Journal of Experimental Psychology: General. https://doi.org/10.1037/xge0000033. Evidence type: experiment. LLM-tested: no. Supports: [Should users be able to edit AI-generated output?](https://aiuxevidence.com/e/let-users-edit)
- John D. Lee, Katrina A. See (2004). Trust in Automation: Designing for Appropriate Reliance. Human Factors. https://doi.org/10.1518/hfes.46.1.50_30392. Evidence type: review. LLM-tested: no. Supports: [How should AI show confidence or uncertainty to users?](https://aiuxevidence.com/e/show-confidence)
- Eric Horvitz (1999). Principles of mixed-initiative user interfaces. CHI 1999. https://doi.org/10.1145/302979.303030. Evidence type: principles. LLM-tested: no. Supports: [When should an AI agent ask for confirmation before acting?](https://aiuxevidence.com/e/ask-before-acting)