Identifying Factual Inconsistencies in Summaries: Grounding LLM Inference via Task Taxonomy
<|reference_start|>Identifying Factual Inconsistencies in Summaries: Grounding LLM Inference via Task Taxonomy: Factual inconsistencies pose a significant hurdle for the faithful summarization by generative models. While a major direction to enhance inconsistency detection is to derive stronger Natural Language Inference (NLI) models, we propose an orthogonal aspect that underscores the importance of incorporating task-specific taxonomy into the inference. To this end, we consolidate key error types of inconsistent facts in summaries, and incorporate them to facilitate both the zero-shot and supervised paradigms of LLMs. Extensive experiments on ten datasets of five distinct domains suggest that, zero-shot LLM inference could benefit from the explicit solution space depicted by the error type taxonomy, and achieves state-of-the-art performance overall, surpassing specialized non-LLM baselines, as well as recent LLM baselines. We further distill models that fuse the taxonomy into parameters through our designed prompt completions and supervised training strategies, efficiently substituting state-of-the-art zero-shot inference with much larger LLMs.<|reference_end|>
@article{xu2024identifying,
    title={Identifying Factual Inconsistencies in Summaries: Grounding LLM
  Inference via Task Taxonomy},
    author={Liyan Xu, Zhenlin Su, Mo Yu, Jin Xu, Jinho D. Choi, Jie Zhou, Fei Liu},
    journal={arXiv preprint arXiv:2402.12821},
    year={2024},
    archivePrefix={arXiv},
    eprint={2402.12821},
    primaryClass={cs.CL cs.LG}
}