FactFinders at CheckThat! 2024: Refining Check-worthy Statement Detection with LLMs through Data Pruning
<|reference_start|>FactFinders at CheckThat! 2024: Refining Check-worthy Statement Detection with LLMs through Data Pruning: The rapid dissemination of information through social media and the Internet has posed a significant challenge for fact-checking, among others in identifying check-worthy claims that fact-checkers should pay attention to, i.e. filtering claims needing fact-checking from a large pool of sentences. This challenge has stressed the need to focus on determining the priority of claims, specifically which claims are worth to be fact-checked. Despite advancements in this area in recent years, the application of large language models (LLMs), such as GPT, has only recently drawn attention in studies. However, many open-source LLMs remain underexplored. Therefore, this study investigates the application of eight prominent open-source LLMs with fine-tuning and prompt engineering to identify check-worthy statements from political transcriptions. Further, we propose a two-step data pruning approach to automatically identify high-quality training data instances for effective learning. The efficiency of our approach is demonstrated through evaluations on the English language dataset as part of the check-worthiness estimation task of CheckThat! 2024. Further, the experiments conducted with data pruning demonstrate that competitive performance can be achieved with only about 44\% of the training data. Our team ranked first in the check-worthiness estimation task in the English language.<|reference_end|>
@article{li2024factfinders,
    title={FactFinders at CheckThat! 2024: Refining Check-worthy Statement
  Detection with LLMs through Data Pruning},
    author={Yufeng Li, Rrubaa Panchendrarajan, Arkaitz Zubiaga},
    journal={arXiv preprint arXiv:2406.18297},
    year={2024},
    archivePrefix={arXiv},
    eprint={2406.18297},
    primaryClass={cs.CL}
}