title: Exploring the Effect of Reinforcement Learning on Video Understanding:
    Insights from SEED-Bench-R1
summary: Recent advancements in Chain of Thought (COT) generation have significantly
  improved the reasoning capabilities of Large Language Models (LLMs), with
  reinforcement learning (RL) emerging as an effective post-training approach.
  Multimodal Large Language Models (MLLMs) inherit this reasoning potential but
  remain underexplored in tasks requiring both perception and logical reasoning.
  To address this, we introduce SEED-Bench-R1, a benchmark designed to
  systematically evaluate post-training methods for MLLMs in video understanding.
  It includes intricate real-world videos and complex everyday planning tasks in
  the format of multiple-choice questions, requiring sophisticated perception and
  reasoning. SEED-Bench-R1 assesses generalization through a three-level
  hierarchy: in-distribution, cross-environment, and cross-environment-task
  scenarios, equipped with a large-scale training dataset with easily verifiable
  ground-truth answers. Using Qwen2-VL-Instruct-7B as a base model, we compare RL
  with supervised fine-tuning (SFT), demonstrating RL's data efficiency and
  superior performance on both in-distribution and out-of-distribution tasks,
  even outperforming SFT on general video understanding benchmarks like
  LongVideoBench. Our detailed analysis reveals that RL enhances visual
  perception but often produces less logically coherent reasoning chains. We
  identify key limitations such as inconsistent reasoning and overlooked visual
  cues, and suggest future improvements in base model reasoning, reward modeling,
  and RL robustness against noisy signals.
paper_authors._id: 67eca2b8351721d62aa537df
paper_authors.name: Yi Chen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-02T08:22:30.143Z
paper_authors._id: 60d045c4778bafd0fbcfa3f5
paper_authors.avatarUrl: /avatars/0cc0c2739c1934430ea09df7e9668c80.svg
paper_authors.fullname: Yi Chen
paper_authors.type: user
paper_authors.user: ChenYi99
paper_authors._id: 67eca2b8351721d62aa537e0
paper_authors.name: Yuying Ge
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-02T09:40:05.288Z
paper_authors._id: 6455cc8f654d8bccae50e4d4
paper_authors.avatarUrl: /avatars/506a9992e5bf52e06d37cc22e4b307c0.svg
paper_authors.fullname: Yuying Ge
paper_authors.type: user
paper_authors.user: tttoaster
paper_authors._id: 67eca2b8351721d62aa537e1
paper_authors.name: Rui Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-02T09:40:12.320Z
paper_authors._id: 62e0f1314db2175cd270ad08
paper_authors.avatarUrl: /avatars/1d3d6af6c63557f4abf0484e028fa942.svg
paper_authors.fullname: Rui Wang
paper_authors.type: user
paper_authors.user: ruiwang
paper_authors._id: 67eca2b8351721d62aa537e2
paper_authors.name: Yixiao Ge
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-02T09:40:19.876Z
paper_authors._id: 640e9762b03f4cd29f58d982
paper_authors.avatarUrl: /avatars/81da37d628163fe3e094b247c7c3a3b5.svg
paper_authors.fullname: Yixiao Ge
paper_authors.type: user
paper_authors.user: yxgeee
paper_authors._id: 67eca2b8351721d62aa537e3
paper_authors.name: Lu Qiu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-04T09:05:03.244Z
paper_authors._id: 672a037c19f1f942483f680c
paper_authors.avatarUrl: /avatars/a48464044e9eb11a2bc062be05d9aa9a.svg
paper_authors.fullname: qiulu
paper_authors.type: user
paper_authors.user: qiulu66
paper_authors._id: 67eca2b8351721d62aa537e4
paper_authors.name: Ying Shan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-02T09:40:26.551Z
paper_authors._id: 63ca3ddc04c979828310bfcb
paper_authors.avatarUrl: /avatars/615e0d8622950b4408b40d550f02a894.svg
paper_authors.fullname: Ying Shan
paper_authors.type: user
paper_authors.user: yshan2u
paper_authors._id: 67eca2b8351721d62aa537e5
paper_authors.name: Xihui Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-02T09:40:32.831Z
paper_authors._id: 65d5ec74cd05bc1eaa125040
paper_authors.avatarUrl: /avatars/2de1b1539a86452c2c89570eeb02f5ab.svg
paper_authors.fullname: Xihui Liu
paper_authors.type: user
paper_authors.user: XihuiLiu
paper_title: Exploring the Effect of Reinforcement Learning on Video Understanding:
    Insights from SEED-Bench-R1
paper_summary: Recent advancements in Chain of Thought (COT) generation have significantly
  improved the reasoning capabilities of Large Language Models (LLMs), with
  reinforcement learning (RL) emerging as an effective post-training approach.
  Multimodal Large Language Models (MLLMs) inherit this reasoning potential but
  remain underexplored in tasks requiring both perception and logical reasoning.
  To address this, we introduce SEED-Bench-R1, a benchmark designed to
  systematically evaluate post-training methods for MLLMs in video understanding.
  It includes intricate real-world videos and complex everyday planning tasks in
  the format of multiple-choice questions, requiring sophisticated perception and
  reasoning. SEED-Bench-R1 assesses generalization through a three-level
  hierarchy: in-distribution, cross-environment, and cross-environment-task
  scenarios, equipped with a large-scale training dataset with easily verifiable
  ground-truth answers. Using Qwen2-VL-Instruct-7B as a base model, we compare RL
  with supervised fine-tuning (SFT), demonstrating RL's data efficiency and
  superior performance on both in-distribution and out-of-distribution tasks,
  even outperforming SFT on general video understanding benchmarks like
  LongVideoBench. Our detailed analysis reveals that RL enhances visual
  perception but often produces less logically coherent reasoning chains. We
  identify key limitations such as inconsistent reasoning and overlooked visual
  cues, and suggest future improvements in base model reasoning, reward modeling,
  and RL robustness against noisy signals.
paper_ai_summary: A benchmark (SEED-Bench-R1) evaluates reinforcement learning versus supervised fine-tuning for multimodal large language models in video understanding, highlighting RL's data efficiency and superior performance but identifying limitations in logical coherence and visual cue processing.
paper_ai_keywords: Chain of Thought (COT) Large Language Models (LLMs) reinforcement learning (RL) Multimodal Large Language Models (MLLMs) video understanding multiple-choice questions Qwen2-VL-Instruct-7B LongVideoBench supervised fine-tuning (SFT) visual perception reasoning chains reward modeling RL robustness