Evaluating the Reliability of Self-Explanations in Large Language Models
<|reference_start|>Evaluating the Reliability of Self-Explanations in Large Language Models: This paper investigates the reliability of explanations generated by large language models (LLMs) when prompted to explain their previous output. We evaluate two kinds of such self-explanations - extractive and counterfactual - using three state-of-the-art LLMs (2B to 8B parameters) on two different classification tasks (objective and subjective). Our findings reveal, that, while these self-explanations can correlate with human judgement, they do not fully and accurately follow the model's decision process, indicating a gap between perceived and actual model reasoning. We show that this gap can be bridged because prompting LLMs for counterfactual explanations can produce faithful, informative, and easy-to-verify results. These counterfactuals offer a promising alternative to traditional explainability methods (e.g. SHAP, LIME), provided that prompts are tailored to specific tasks and checked for validity.<|reference_end|>
@article{randl2024evaluating,
    title={Evaluating the Reliability of Self-Explanations in Large Language Models},
    author={Korbinian Randl, John Pavlopoulos, Aron Henriksson, and Tony Lindgren},
    journal={arXiv preprint arXiv:2407.14487},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.14487},
    primaryClass={cs.CL}
}