Coarse-Tuning for Ad-hoc Document Retrieval Using Pre-trained Language Models
<|reference_start|>Coarse-Tuning for Ad-hoc Document Retrieval Using Pre-trained Language Models: Fine-tuning in information retrieval systems using pre-trained language models (PLM-based IR) requires learning query representations and query-document relations, in addition to downstream task-specific learning. This study introduces coarse-tuning as an intermediate learning stage that bridges pre-training and fine-tuning. By learning query representations and query-document relations in coarse-tuning, we aim to reduce the load of fine-tuning and improve the learning effect of downstream IR tasks. We propose Query-Document Pair Prediction (QDPP) for coarse-tuning, which predicts the appropriateness of query-document pairs. Evaluation experiments show that the proposed method significantly improves MRR and/or nDCG@5 in four ad-hoc document retrieval datasets. Furthermore, the results of the query prediction task suggested that coarse-tuning facilitated learning of query representation and query-document relations.<|reference_end|>
@article{keyaki2024coarse-tuning,
    title={Coarse-Tuning for Ad-hoc Document Retrieval Using Pre-trained Language
  Models},
    author={Atsushi Keyaki and Ribeka Keyaki},
    journal={arXiv preprint arXiv:2403.16915},
    year={2024},
    archivePrefix={arXiv},
    eprint={2403.16915},
    primaryClass={cs.IR cs.AI cs.CL cs.LG}
}