XEQ Scale for Evaluating XAI Experience Quality Grounded in Psychometric Theory
<|reference_start|>XEQ Scale for Evaluating XAI Experience Quality Grounded in Psychometric Theory: Explainable Artificial Intelligence (XAI) aims to improve the transparency of autonomous decision-making through explanations. Recent literature has emphasised users' need for holistic "multi-shot" explanations and the ability to personalise their engagement with XAI systems. We refer to this user-centred interaction as an XAI Experience. Despite advances in creating XAI experiences, evaluating them in a user-centred manner has remained challenging. To address this, we introduce the XAI Experience Quality (XEQ) Scale (pronounced "Seek" Scale), for evaluating the user-centred quality of XAI experiences. Furthermore, XEQ quantifies the quality of experiences across four evaluation dimensions: learning, utility, fulfilment and engagement. These contributions extend the state-of-the-art of XAI evaluation, moving beyond the one-dimensional metrics frequently developed to assess single-shot explanations. In this paper, we present the XEQ scale development and validation process, including content validation with XAI experts as well as discriminant and construct validation through a large-scale pilot study. Out pilot study results offer strong evidence that establishes the XEQ Scale as a comprehensive framework for evaluating user-centred XAI experiences.<|reference_end|>
@article{wijekoon2024xeq,
    title={XEQ Scale for Evaluating XAI Experience Quality Grounded in Psychometric
  Theory},
    author={Anjana Wijekoon, Nirmalie Wiratunga, David Corsar, Kyle Martin,
  Ikechukwu Nkisi-Orji, Belen D'iaz-Agudo, Derek Bridge},
    journal={arXiv preprint arXiv:2407.10662},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.10662},
    primaryClass={cs.AI cs.HC}
}