title: Sailing AI by the Stars: A Survey of Learning from Rewards in
    Post-Training and Test-Time Scaling of Large Language Models
summary: Recent developments in Large Language Models (LLMs) have shifted from
  pre-training scaling to post-training and test-time scaling. Across these
  developments, a key unified paradigm has arisen: Learning from Rewards, where
  reward signals act as the guiding stars to steer LLM behavior. It has
  underpinned a wide range of prevalent techniques, such as reinforcement
  learning (in RLHF, DPO, and GRPO), reward-guided decoding, and post-hoc
  correction. Crucially, this paradigm enables the transition from passive
  learning from static data to active learning from dynamic feedback. This endows
  LLMs with aligned preferences and deep reasoning capabilities. In this survey,
  we present a comprehensive overview of the paradigm of learning from rewards.
  We categorize and analyze the strategies under this paradigm across training,
  inference, and post-inference stages. We further discuss the benchmarks for
  reward models and the primary applications. Finally we highlight the challenges
  and future directions. We maintain a paper collection at
  https://github.com/bobxwu/learning-from-rewards-llm-papers.
paper_authors._id: 6821acfb2808328b91c0e365
paper_authors.name: Xiaobao Wu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-12T13:17:15.826Z
paper_authors._id: 64cb02869e30a46f7b80b355
paper_authors.avatarUrl: /avatars/81ce4ba78826b54f0e1b53eeaff87ee6.svg
paper_authors.fullname: Xiaobao Wu
paper_authors.type: user
paper_authors.user: bobxwu
paper_title: Sailing AI by the Stars: A Survey of Learning from Rewards in
    Post-Training and Test-Time Scaling of Large Language Models
paper_summary: Recent developments in Large Language Models (LLMs) have shifted from
  pre-training scaling to post-training and test-time scaling. Across these
  developments, a key unified paradigm has arisen: Learning from Rewards, where
  reward signals act as the guiding stars to steer LLM behavior. It has
  underpinned a wide range of prevalent techniques, such as reinforcement
  learning (in RLHF, DPO, and GRPO), reward-guided decoding, and post-hoc
  correction. Crucially, this paradigm enables the transition from passive
  learning from static data to active learning from dynamic feedback. This endows
  LLMs with aligned preferences and deep reasoning capabilities. In this survey,
  we present a comprehensive overview of the paradigm of learning from rewards.
  We categorize and analyze the strategies under this paradigm across training,
  inference, and post-inference stages. We further discuss the benchmarks for
  reward models and the primary applications. Finally we highlight the challenges
  and future directions. We maintain a paper collection at
  https://github.com/bobxwu/learning-from-rewards-llm-papers.
paper_ai_summary: A survey of the Learning from Rewards paradigm in Large Language Models, encompassing reinforcement learning techniques and reward-guided decoding strategies, enabling dynamic feedback and aligned preferences.
paper_ai_keywords: Learning from Rewards reinforcement learning RLHF DPO GRPO reward-guided decoding post-hoc correction active learning reward models dynamic feedback aligned preferences