Rank1: Test-Time Compute for Reranking in Information Retrieval
We introduce Rank1, the first reranking model trained to take advantage of
test-time compute. Rank1 demonstrates the applicability within retrieval of
using a reasoning language model (i.e. OpenAI's o1, Deepseek's R1, etc.) for
distillation in order to rapidly improve the performance of a smaller model. We
gather and open-source a dataset of more than 600,000 examples of R1 reasoning
traces from queries and passages in MS MARCO. Models trained on this dataset
show: (1) state-of-the-art performance on advanced reasoning and instruction
following datasets; (2) work remarkably well out of distribution due to the
ability to respond to user-input prompts; and (3) have explainable reasoning
chains that can be given to users or RAG-based systems. Further, we demonstrate
that quantized versions of these models retain strong performance while using
less compute/memory. Overall, Rank1 shows that test-time compute allows for a
fundamentally new type of explainable and performant reranker model for search.
67bf17b23f838c1e33ac7c4d
Orion Weller
67bf17b23f838c1e33ac7c4e
Kathryn Ricci
67bf17b23f838c1e33ac7c4f
Eugene Yang
67bf17b23f838c1e33ac7c50
Andrew Yates
67bf17b23f838c1e33ac7c51
Dawn Lawrie
67bf17b23f838c1e33ac7c52
Benjamin Van Durme
Rank1: Test-Time Compute for Reranking in Information Retrieval
We introduce Rank1, the first reranking model trained to take advantage of
test-time compute. Rank1 demonstrates the applicability within retrieval of
using a reasoning language model (i.e. OpenAI's o1, Deepseek's R1, etc.) for
distillation in order to rapidly improve the performance of a smaller model. We
gather and open-source a dataset of more than 600,000 examples of R1 reasoning
traces from queries and passages in MS MARCO. Models trained on this dataset
show: (1) state-of-the-art performance on advanced reasoning and instruction
following datasets; (2) work remarkably well out of distribution due to the
ability to respond to user-input prompts; and (3) have explainable reasoning
chains that can be given to users or RAG-based systems. Further, we demonstrate
that quantized versions of these models retain strong performance while using
less compute/memory. Overall, Rank1 shows that test-time compute allows for a
fundamentally new type of explainable and performant reranker model for search.
Rank1 is a re-ranking model using test-time compute and reasoning language models to enhance performance and explainability compared to smaller models, demonstrating state-of-the-art results on reasoning and instruction-following datasets.
reranking model test-time compute reasoning language model OpenAI's o1 Deepseek's R1 MS MARCO state-of-the-art performance re-ranking out of distribution explainable reasoning chains quantized models