Hong Shen|Haojian Jin|Ángel Alexander Cabrera|Adam Perer|Haiyi Zhu|Jason Hong
Designing Alternative Representations of Confusion Matrices to Support Non-Expert Public Understanding of Algorithm Performance
Carnegie Mellon University|Carnegie Mellon University|Carnegie Mellon University|Carnegie Mellon University|Carnegie Mellon University|Carnegie Mellon University
ensuring effective public understanding algorithmic decisions powered machine learning techniques become urgent task increasing deployment ai systems society work present concrete step toward goal redesigning confusion matrices binary classification support nonexperts understanding performance machine learning models interviews n7 survey n102 mapped two major sets challenges lay people understanding standard confusion matrices general terminologies matrix design identified three subchallenges regarding matrix design namely confusion direction reading data layered relations quantities involved conducted online experiment 483 participants evaluate effective series alternative representations target challenges context algorithm making recidivism predictions developed three levels questions evaluate users objective understanding assessed effectiveness alternatives accuracy answering questions completion time subjective understanding results suggest 1 contextualizing terminologies significantly improve users understanding 2 flow charts help point direction reading data useful improving objective understanding findings set stage developing intuitive generally understandable representations performance machine learning models