Evaluating Transparency of Machine Generated Fact Checking Explanations
<|reference_start|>Evaluating Transparency of Machine Generated Fact Checking Explanations: An important factor when it comes to generating fact-checking explanations is the selection of evidence: intuitively, high-quality explanations can only be generated given the right evidence. In this work, we investigate the impact of human-curated vs. machine-selected evidence for explanation generation using large language models. To assess the quality of explanations, we focus on transparency (whether an explanation cites sources properly) and utility (whether an explanation is helpful in clarifying a claim). Surprisingly, we found that large language models generate similar or higher quality explanations using machine-selected evidence, suggesting carefully curated evidence (by humans) may not be necessary. That said, even with the best model, the generated explanations are not always faithful to the sources, suggesting further room for improvement in explanation generation for fact-checking.<|reference_end|>
@article{xing2024evaluating,
    title={Evaluating Evidence Attribution in Generated Fact Checking Explanations},
    author={Rui Xing, Timothy Baldwin, Jey Han Lau},
    journal={arXiv preprint arXiv:2406.12645},
    year={2024},
    archivePrefix={arXiv},
    eprint={2406.12645},
    primaryClass={cs.CL cs.AI}
}