Surprising Efficacy of Fine-Tuned Transformers for Fact-Checking over Larger Language Models
<|reference_start|>Surprising Efficacy of Fine-Tuned Transformers for Fact-Checking over Larger Language Models: In this paper, we explore the challenges associated with establishing an end-to-end fact-checking pipeline in a real-world context, covering over 90 languages. Our real-world experimental benchmarks demonstrate that fine-tuning Transformer models specifically for fact-checking tasks, such as claim detection and veracity prediction, provide superior performance over large language models (LLMs) like GPT-4, GPT-3.5-Turbo, and Mistral-7b. However, we illustrate that LLMs excel in generative tasks such as question decomposition for evidence retrieval. Through extensive evaluation, we show the efficacy of fine-tuned models for fact-checking in a multilingual setting and complex claims that include numerical quantities.<|reference_end|>
@article{setty2024surprising,
    title={Surprising Efficacy of Fine-Tuned Transformers for Fact-Checking over
  Larger Language Models},
    author={Vinay Setty},
    journal={arXiv preprint arXiv:2402.12147},
    year={2024},
    archivePrefix={arXiv},
    eprint={2402.12147},
    primaryClass={cs.CL cs.AI}
}