title: Rank1: Test-Time Compute for Reranking in Information Retrieval
summary: We introduce Rank1, the first reranking model trained to take advantage of
  test-time compute. Rank1 demonstrates the applicability within retrieval of
  using a reasoning language model (i.e. OpenAI's o1, Deepseek's R1, etc.) for
  distillation in order to rapidly improve the performance of a smaller model. We
  gather and open-source a dataset of more than 600,000 examples of R1 reasoning
  traces from queries and passages in MS MARCO. Models trained on this dataset
  show: (1) state-of-the-art performance on advanced reasoning and instruction
  following datasets; (2) work remarkably well out of distribution due to the
  ability to respond to user-input prompts; and (3) have explainable reasoning
  chains that can be given to users or RAG-based systems. Further, we demonstrate
  that quantized versions of these models retain strong performance while using
  less compute/memory. Overall, Rank1 shows that test-time compute allows for a
  fundamentally new type of explainable and performant reranker model for search.
paper_authors._id: 67bf17b23f838c1e33ac7c4d
paper_authors.name: Orion Weller
paper_authors._id: 67bf17b23f838c1e33ac7c4e
paper_authors.name: Kathryn Ricci
paper_authors._id: 67bf17b23f838c1e33ac7c4f
paper_authors.name: Eugene Yang
paper_authors._id: 67bf17b23f838c1e33ac7c50
paper_authors.name: Andrew Yates
paper_authors._id: 67bf17b23f838c1e33ac7c51
paper_authors.name: Dawn Lawrie
paper_authors._id: 67bf17b23f838c1e33ac7c52
paper_authors.name: Benjamin Van Durme
paper_title: Rank1: Test-Time Compute for Reranking in Information Retrieval
paper_summary: We introduce Rank1, the first reranking model trained to take advantage of
  test-time compute. Rank1 demonstrates the applicability within retrieval of
  using a reasoning language model (i.e. OpenAI's o1, Deepseek's R1, etc.) for
  distillation in order to rapidly improve the performance of a smaller model. We
  gather and open-source a dataset of more than 600,000 examples of R1 reasoning
  traces from queries and passages in MS MARCO. Models trained on this dataset
  show: (1) state-of-the-art performance on advanced reasoning and instruction
  following datasets; (2) work remarkably well out of distribution due to the
  ability to respond to user-input prompts; and (3) have explainable reasoning
  chains that can be given to users or RAG-based systems. Further, we demonstrate
  that quantized versions of these models retain strong performance while using
  less compute/memory. Overall, Rank1 shows that test-time compute allows for a
  fundamentally new type of explainable and performant reranker model for search.
paper_ai_summary: Rank1 is a re-ranking model using test-time compute and reasoning language models to enhance performance and explainability compared to smaller models, demonstrating state-of-the-art results on reasoning and instruction-following datasets.
paper_ai_keywords: reranking model test-time compute reasoning language model OpenAI's o1 Deepseek's R1 MS MARCO state-of-the-art performance re-ranking out of distribution explainable reasoning chains quantized models