Contrastive Learning to Improve Retrieval for Real-world Fact Checking
<|reference_start|>Contrastive Learning to Improve Retrieval for Real-world Fact Checking: Recent work on fact-checking addresses a realistic setting where models incorporate evidence retrieved from the web to decide the veracity of claims. A bottleneck in this pipeline is in retrieving relevant evidence: traditional methods may surface documents directly related to a claim, but fact-checking complex claims requires more inferences. For instance, a document about how a vaccine was developed is relevant to addressing claims about what it might contain, even if it does not address them directly. We present Contrastive Fact-Checking Reranker (CFR), an improved retriever for this setting. By leveraging the AVeriTeC dataset, which annotates subquestions for claims with human written answers from evidence documents, we fine-tune Contriever with a contrastive objective based on multiple training signals, including distillation from GPT-4, evaluating subquestion answers, and gold labels in the dataset. We evaluate our model on both retrieval and end-to-end veracity judgments about claims. On the AVeriTeC dataset, we find a 6\% improvement in veracity classification accuracy. We also show our gains can be transferred to FEVER, ClaimDecomp, HotpotQA, and a synthetic dataset requiring retrievers to make inferences.<|reference_end|>
@article{sriram2024contrastive,
    title={Contrastive Learning to Improve Retrieval for Real-world Fact Checking},
    author={Aniruddh Sriram, Fangyuan Xu, Eunsol Choi, Greg Durrett},
    journal={arXiv preprint arXiv:2410.04657},
    year={2024},
    archivePrefix={arXiv},
    eprint={2410.04657},
    primaryClass={cs.CL cs.AI cs.LG}
}