Every paper we cite

33 research papers behind the 25 design questions, 20 of them tested with large language models. Each one links to the questions it supports.

1. Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of Artificial Intelligence on Knowledge Worker Productivity and Quality

   Fabrizio Dell'Acqua, Edward McFowland III, Ethan Mollick, et al. · 2026 · Organization Science (2026); first released as HBS Working Paper 24-013 (2023)

   field experimentTested with LLMs

   Supports

   • Who benefits most from AI tools at work?
   • How do you stop users from blindly trusting AI answers?

2. Cocoa: Co-Planning and Co-Execution with AI Agents

   K. J. Kevin Feng, Kevin Pu, Matt Latzke, et al. · 2026 · CHI 2026 (arXiv:2412.10999v4)

   system evaluationTested with LLMs

   Supports

   • How should users and AI agents share control of a task?

3. The Impact of Generative AI on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects From a Survey of Knowledge Workers

   Hao-Ping (Hank) Lee, Advait Sarkar, Lev Tankelevitch, et al. · 2025 · CHI 2025

   surveyTested with LLMs

   Supports

   • Does relying on AI reduce users' critical thinking?

4. Magentic-UI: Towards Human-in-the-loop Agentic Systems

   Hussein Mozannar, Gagan Bansal, Cheng Tan, Adam Fourney, et al. · 2025 · arXiv:2507.22358 (Microsoft Research, 2025)

   system evaluationTested with LLMs

   Supports

   • When should an AI agent ask for confirmation before acting?
   • How should users and AI agents share control of a task?

5. Large Language Models Help Humans Verify Truthfulness—Except When They Are Convincingly Wrong

   Chenglei Si, Navita Goyal, Sherry Tongshuang Wu, et al. · 2024 · arXiv:2310.12558v2 (Apr 2024)

   experimentTested with LLMs

   Supports

   • Should AI show its reasoning or explanations to users?
   • Is an AI chatbot better than traditional search for decisions?

6. Generative AI at Work

   Erik Brynjolfsson, Danielle Li, Lindsey Raymond · 2024 · arXiv:2304.11771v2 (revised Nov 2024)

   field studyTested with LLMs

   Supports

   • Who benefits most from AI tools at work?

7. Generation Probabilities Are Not Enough: Uncertainty Highlighting in AI Code Completions

   Helena Vasconcelos, Gagan Bansal, Adam Fourney, Q. Vera Liao, Jennifer Wortman Vaughan · 2024 · ACM TOCHI (preprint, arXiv:2302.07248v3, Nov 2024)

   experimentTested with LLMs

   Supports

   • Do AI coding assistants make code less secure?
   • Should AI highlight the parts of its output it's unsure about?

8. Reading Between the Lines: Modeling User Behavior and Costs in AI-Assisted Programming

   Hussein Mozannar, Gagan Bansal, Adam Fourney, Eric Horvitz · 2024 · arXiv:2210.14306v5 (revised Apr 2024)

   observational studyTested with LLMs

   Supports

   • Do inline AI suggestions like Copilot cost users time?

9. Fakes of Varying Shades: How Warning Affects Human Perception and Engagement Regarding LLM Hallucinations

   Mahjabin Nahar, Haeseung Seo, Eun-Ju Lee, et al. · 2024 · COLM 2024

   experimentTested with LLMs

   Supports

   • Do hallucination warnings help users spot AI errors?

10. Believing Anthropomorphism: Examining the Role of Anthropomorphic Cues on Trust in Large Language Models

    Michelle Cohn, Mahima Pushkarna, Gbolahan O. Olanubi, et al. · 2024 · CHI 2024 (author version, arXiv:2405.06079)

    experimentTested with LLMs

    Supports

    • Does making an AI sound human increase user trust?

11. People are skeptical of headlines labeled as AI-generated, even if true or human-made, because they assume full AI automation

    Sacha Altay, Fabrizio Gilardi · 2024 · PNAS Nexus 2024

    experimentTested with LLMs

    Supports

    • Should you label AI-generated content?

12. “I'm Not Sure, But...”: Examining the Impact of Large Language Models' Uncertainty Expression on User Reliance and Trust

    Sunnie S. Y. Kim, Q. Vera Liao, Mihaela Vorvoreanu, Stephanie Ballard, Jennifer Wortman Vaughan · 2024 · FAccT 2024

    experimentTested with LLMs

    Supports

    • How should AI show confidence or uncertainty to users?

13. Does Writing with Language Models Reduce Content Diversity?

    Vishakh Padmakumar, He He · 2024 · ICLR 2024

    experimentTested with LLMs

    Supports

    • Does writing with AI make everyone's content sound the same?

14. “It's a Fair Game”, or Is It? Examining How Users Navigate Disclosure Risks and Benefits When Using LLM-Based Conversational Agents

    Zhiping Zhang, Michelle Jia, Hao-Ping (Hank) Lee, et al. · 2024 · CHI 2024

    qualitative studyTested with LLMs

    Supports

    • Do users overshare personal data with AI chatbots?
    • Does making an AI sound human increase user trust?

15. Explanations Can Reduce Overreliance on AI Systems During Decision-Making

    Helena Vasconcelos, Matthew Jörke, Madeleine Grunde-McLaughlin, et al. · 2023 · CSCW 2023 (arXiv 2022)

    experimentPre-LLM

    Supports

    • Do AI explanations help users trust AI appropriately?

16. Why Johnny Can't Prompt: How Non-AI Experts Try (and Fail) to Design LLM Prompts

    J.D. Zamfirescu-Pereira, Richmond Y. Wong, Bjoern Hartmann, Qian Yang · 2023 · CHI 2023

    qualitative studyTested with LLMs

    Supports

    • Is a blank chat box good UX for AI products?

17. Co-Writing with Opinionated Language Models Affects Users' Views

    Maurice Jakesch, Advait Bhat, Daniel Buschek, Lior Zalmanson, Mor Naaman · 2023 · CHI 2023

    experimentTested with LLMs

    Supports

    • Can AI writing assistants change what users believe?
    • Does writing with AI make everyone's content sound the same?

18. Do Users Write More Insecure Code with AI Assistants?

    Neil Perry, Megha Srivastava, Deepak Kumar, Dan Boneh · 2023 · CCS 2023

    experimentTested with LLMs

    Supports

    • Does relying on AI reduce users' critical thinking?
    • Do AI coding assistants make code less secure?

19. Evaluating Verifiability in Generative Search Engines

    Nelson F. Liu, Tianyi Zhang, Percy Liang · 2023 · Findings of EMNLP 2023

    auditTested with LLMs

    Supports

    • Can you trust citations in AI search answers?

20. Experimental evidence on the productivity effects of generative artificial intelligence

    Shakked Noy, Whitney Zhang · 2023 · Science 381 (6654), 2023

    experimentTested with LLMs

    Supports

    • Who benefits most from AI tools at work?

21. Comparing Traditional and LLM-based Search for Consumer Choice: A Randomized Experiment

    Sofia Eleni Spatharioti, David M. Rothschild, Daniel G. Goldstein, Jake M. Hofman · 2023 · arXiv:2307.03744v2 (Nov 2023)

    experimentTested with LLMs

    Supports

    • Do hallucination warnings help users spot AI errors?
    • Is an AI chatbot better than traditional search for decisions?
    • Should AI highlight the parts of its output it's unsure about?

22. Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance

    Gagan Bansal, Tongshuang Wu, Joyce Zhou, et al. · 2021 · CHI 2021 (arXiv 2020)

    experimentPre-LLM

    Supports

    • Should AI show its reasoning or explanations to users?
    • Do AI explanations help users trust AI appropriately?

23. Artificial intelligence in communication impacts language and social relationships

    Jess Hohenstein, Dominic DiFranzo, Rene F. Kizilcec, et al. · 2021 · arXiv:2102.05756v1 (2021); later in Scientific Reports

    experimentPre-LLM

    Supports

    • Should you label AI-generated content?
    • Do AI reply suggestions change how people communicate?

24. To Trust or to Think: Cognitive Forcing Functions Can Reduce Overreliance on AI in AI-assisted Decision-making

    Zana Buçinca, Maja Barbara Malaya, Krzysztof Z. Gajos · 2021 · CSCW 2021

    experimentPre-LLM

    Supports

    • How do you stop users from blindly trusting AI answers?

25. Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Making

    Yunfeng Zhang, Q. Vera Liao, Rachel K. E. Bellamy · 2020 · FAT* 2020

    experimentPre-LLM

    Supports

    • How should AI show confidence or uncertainty to users?

26. "Hello AI": Uncovering the Onboarding Needs of Medical Practitioners for Human-AI Collaborative Decision-Making

    Carrie J. Cai, Samantha Winter, David Steiner, Lauren Wilcox, Michael Terry · 2019 · CSCW 2019

    qualitative studyPre-LLM

    Supports

    • How should you onboard users to an AI feature?

27. Will You Accept an Imperfect AI? Exploring Designs for Adjusting End-user Expectations of AI Systems

    Rafal Kocielnik, Saleema Amershi, Paul N. Bennett · 2019 · CHI 2019

    experimentPre-LLM

    Supports

    • How do you set user expectations for an AI that makes mistakes?

28. Guidelines for Human-AI Interaction

    Saleema Amershi, Dan Weld, Mihaela Vorvoreanu, et al. · 2019 · CHI 2019

    guidelinesPre-LLM

    Supports

    • When should an AI agent ask for confirmation before acting?
    • How do you set user expectations for an AI that makes mistakes?

29. Resilient Chatbots: Repair Strategy Preferences for Conversational Breakdowns

    Zahra Ashktorab, Mohit Jain, Q. Vera Liao, Justin D. Weisz · 2019 · CHI 2019

    experimentPre-LLM

    Supports

    • How should a chatbot recover when it misunderstands?

30. Overcoming Algorithm Aversion: People Will Use Imperfect Algorithms If They Can (Even Slightly) Modify Them

    Berkeley J. Dietvorst, Joseph P. Simmons, Cade Massey · 2018 · Management Science

    experimentPre-LLM

    Supports

    • Should users be able to edit AI-generated output?

31. Algorithm aversion: People erroneously avoid algorithms after seeing them err

    Berkeley J. Dietvorst, Joseph P. Simmons, Cade Massey · 2015 · Journal of Experimental Psychology: General

    experimentPre-LLM

    Supports

    • Should users be able to edit AI-generated output?

32. Trust in Automation: Designing for Appropriate Reliance

    John D. Lee, Katrina A. See · 2004 · Human Factors

    reviewPre-LLM

    Supports

    • How should AI show confidence or uncertainty to users?

33. Principles of mixed-initiative user interfaces

    Eric Horvitz · 1999 · CHI 1999

    principlesPre-LLM

    Supports

    • When should an AI agent ask for confirmation before acting?

No papers match. Try another search.

Also available as JSON.