title: Distillation and Refinement of Reasoning in Small Language Models for
    Document Re-ranking
summary: We present a novel approach for training small language models for
  reasoning-intensive document ranking that combines knowledge distillation with
  reinforcement learning optimization. While existing methods often rely on
  expensive human annotations or large black-box language models, our methodology
  leverages web data and a teacher LLM to automatically generate high-quality
  training examples with relevance explanations. By framing document ranking as a
  reinforcement learning problem and incentivizing explicit reasoning
  capabilities, we train a compact 3B parameter language model that achieves
  state-of-the-art performance on the BRIGHT benchmark. Our model ranks third on
  the leaderboard while using substantially fewer parameters than other
  approaches, outperforming models that are over 20 times larger. Through
  extensive experiments, we demonstrate that generating explanations during
  inference, rather than directly predicting relevance scores, enables more
  effective reasoning with smaller language models. The self-supervised nature of
  our method offers a scalable and interpretable solution for modern information
  retrieval systems.
paper_authors._id: 67f4a21bb1e7ca3656ab6195
paper_authors.name: Chris Samarinas
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-09T07:37:46.626Z
paper_authors._id: 6216cfcd6a99db28e0b3155a
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1676512583831-6216cfcd6a99db28e0b3155a.jpeg
paper_authors.fullname: Chris Samarinas
paper_authors.type: user
paper_authors.user: algoprog
paper_authors._id: 67f4a21bb1e7ca3656ab6196
paper_authors.name: Hamed Zamani
paper_title: Distillation and Refinement of Reasoning in Small Language Models for
    Document Re-ranking
paper_summary: We present a novel approach for training small language models for
  reasoning-intensive document ranking that combines knowledge distillation with
  reinforcement learning optimization. While existing methods often rely on
  expensive human annotations or large black-box language models, our methodology
  leverages web data and a teacher LLM to automatically generate high-quality
  training examples with relevance explanations. By framing document ranking as a
  reinforcement learning problem and incentivizing explicit reasoning
  capabilities, we train a compact 3B parameter language model that achieves
  state-of-the-art performance on the BRIGHT benchmark. Our model ranks third on
  the leaderboard while using substantially fewer parameters than other
  approaches, outperforming models that are over 20 times larger. Through
  extensive experiments, we demonstrate that generating explanations during
  inference, rather than directly predicting relevance scores, enables more
  effective reasoning with smaller language models. The self-supervised nature of
  our method offers a scalable and interpretable solution for modern information
  retrieval systems.
paper_ai_summary: A novel approach combines knowledge distillation and reinforcement learning to train small, efficient language models for document ranking using web data and teacher LLMs, achieving state-of-the-art performance with fewer parameters.
paper_ai_keywords: knowledge distillation reinforcement learning document ranking web data teacher LLM BRIGHT benchmark state-of-the-art performance parameter efficiency relevance explanations self-supervised learning information retrieval systems