Mining the Explainability and Generalization: Fact Verification Based on Self-Instruction
<|reference_start|>Mining the Explainability and Generalization: Fact Verification Based on Self-Instruction: Fact-checking based on commercial LLMs has become mainstream. Although these methods offer high explainability, it falls short in accuracy compared to traditional fine-tuning approaches, and data security is also a significant concern. In this paper, we propose a self-instruction based fine-tuning approach for fact-checking that balances accuracy and explainability. Our method consists of Data Augmentation and Improved DPO fine-tuning. The former starts by instructing the model to generate both positive and negative explanations based on claim-evidence pairs and labels, then sampling the dataset according to our customized difficulty standards. The latter employs our proposed improved DPO to fine-tune the model using the generated samples. We fine-tune the smallest-scale LLaMA-7B model and evaluate it on the challenging fact-checking datasets FEVEROUS and HOVER, utilizing four fine-tuning methods and three few-shot learning methods for comparison. The experiments demonstrate that our approach not only retains accuracy comparable to, or even surpassing, traditional fine-tuning methods, but also generates fluent explanation text. Moreover, it also exhibit high generalization performance. Our method is the first to leverage self-supervised learning for fact-checking and innovatively combines contrastive learning and improved DPO in fine-tuning LLMs, as shown in the experiments.<|reference_end|>
@article{lu2024mining,
    title={Mining the Explainability and Generalization: Fact Verification Based on
  Self-Instruction},
    author={Guangyao Lu and Yulin Liu},
    journal={arXiv preprint arXiv:2405.12579},
    year={2024},
    archivePrefix={arXiv},
    eprint={2405.12579},
    primaryClass={cs.CL}
}