Abstract:Explainable AI (XAI) techniques aim to provide insights into predictive models and enhance user performance, yet they often fall short of these expectations. Conversational XAI assistants promise to overcome such limitations, but empirical evidence on their impact on objective performance measures remains limited. We propose an experimental design for evaluating explanation assistance through prediction accuracy, model understanding, and error identification. Using an explainable-by-design prediction model, we create conditions where users can outperform the model by identifying and compensating for systematic errors. We compare conversational assistance against Q&A-based assistance to assess which better supports users in working with model explanations. Preliminary results from testing our experimental design show that participants (N=42) in both treatments significantly outperformed the model but reveal no performance differences between assistance types and modest engagement overall. These findings inform refinements for our planned full study, including enhanced engagement interventions and investigation of the mechanisms driving improved predictions.
| Comments: | Accepted at Thirty-Fourth European Conference on Information Systems (ECIS 2026), Milan, Italy |
| Subjects: | Machine Learning (cs.LG); Human-Computer Interaction (cs.HC) |
| Cite as: | arXiv:2605.20439 [cs.LG] |
| (or arXiv:2605.20439v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2605.20439 arXiv-issued DOI via DataCite (pending registration) |
From: Julian Rosenberger [view email]
[v1]
Tue, 19 May 2026 19:47:17 UTC (1,218 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。