How should you set expectations for an imperfect AI?
Design it like this
State accuracy and typical failures plainly at first use, and pick the error type your users tolerate best.
What the research says
Telling users up front how accurate the AI is and how it can fail, and letting them tune it, made them more accepting of its mistakes. Which kind of error you tune toward (false alarms vs. misses) also changed how users felt.
“Make clear how well the system can do what it can do. Help the user understand how often the AI system may make mistakes.”
Amershi et al., 2019
Sources
- Will You Accept an Imperfect AI? Exploring Designs for Adjusting End-user Expectations of AI Systems2019 · CHI 2019 · Rafal Kocielnik, Saleema Amershi, Paul N. Bennett
- Guidelines for Human-AI Interaction2019 · CHI 2019 · Saleema Amershi, Dan Weld, Mihaela Vorvoreanu, et al.
“Make clear how well the system can do what it can do. Help the user understand how often the AI system may make mistakes.”
Guideline G2