Adaptive Query Rewriting: Aligning Rewriters through Marginal Probability of Conversational Answers
<|reference_start|>Adaptive Query Rewriting: Aligning Rewriters through Marginal Probability of Conversational Answers: Query rewriting is a crucial technique for passage retrieval in open-domain conversational question answering (CQA). It decontexualizes conversational queries into self-contained questions suitable for off-the-shelf retrievers. Existing methods attempt to incorporate retriever's preference during the training of rewriting models. However, these approaches typically rely on extensive annotations such as in-domain rewrites and/or relevant passage labels, limiting the models' generalization and adaptation capabilities. In this paper, we introduce AdaQR ($\textbf{Ada}$ptive $\textbf{Q}$uery $\textbf{R}$ewriting), a framework for training query rewriting models with limited rewrite annotations from seed datasets and completely no passage label. Our approach begins by fine-tuning compact large language models using only ~$10\%$ of rewrite annotations from the seed dataset training split. The models are then utilized to generate rewrite candidates for each query instance. A novel approach is then proposed to assess retriever's preference for these candidates by the probability of answers conditioned on the conversational query by marginalizing the Top-$K$ passages. This serves as the reward for optimizing the rewriter further using Direct Preference Optimization (DPO), a process free of rewrite and retrieval annotations. Experimental results on four open-domain CQA datasets demonstrate that AdaQR not only enhances the in-domain capabilities of the rewriter with limited annotation requirement, but also adapts effectively to out-of-domain datasets.<|reference_end|>
@article{zhang2024adaptive,
    title={Adaptive Query Rewriting: Aligning Rewriters through Marginal
  Probability of Conversational Answers},
    author={Tianhua Zhang, Kun Li, Hongyin Luo, Xixin Wu, James Glass, Helen Meng},
    journal={arXiv preprint arXiv:2406.10991},
    year={2024},
    archivePrefix={arXiv},
    eprint={2406.10991},
    primaryClass={cs.CL}
}