title: Robot-R1: Reinforcement Learning for Enhanced Embodied Reasoning in
    Robotics
summary: Large Vision-Language Models (LVLMs) have recently shown great promise in
  advancing robotics by combining embodied reasoning with robot control. A common
  approach involves training on embodied reasoning tasks related to robot control
  using Supervised Fine-Tuning (SFT). However, SFT datasets are often
  heuristically constructed and not explicitly optimized for improving robot
  control. Furthermore, SFT often leads to issues such as catastrophic forgetting
  and reduced generalization performance. To address these limitations, we
  introduce Robot-R1, a novel framework that leverages reinforcement learning to
  enhance embodied reasoning specifically for robot control. Robot-R1 learns to
  predict the next keypoint state required for task completion, conditioned on
  the current scene image and environment metadata derived from expert
  demonstrations. Inspired by the DeepSeek-R1 learning approach, Robot-R1 samples
  reasoning-based responses and reinforces those that lead to more accurate
  predictions. Our experiments show that models trained with Robot-R1 outperform
  SFT methods on embodied reasoning tasks. Despite having only 7B parameters,
  Robot-R1 even surpasses GPT-4o on reasoning tasks related to low-level action
  control, such as spatial and primitive movement reasoning.
paper_authors._id: 683fd7d79d4fb703271ad9ac
paper_authors.name: Dongyoung Kim
paper_authors._id: 683fd7d79d4fb703271ad9ad
paper_authors.name: Sumin Park
paper_authors._id: 683fd7d79d4fb703271ad9ae
paper_authors.name: Huiwon Jang
paper_authors._id: 683fd7d79d4fb703271ad9af
paper_authors.name: Jinwoo Shin
paper_authors._id: 683fd7d79d4fb703271ad9b0
paper_authors.name: Jaehyung Kim
paper_authors._id: 683fd7d79d4fb703271ad9b1
paper_authors.name: Younggyo Seo
paper_title: Robot-R1: Reinforcement Learning for Enhanced Embodied Reasoning in
    Robotics
paper_summary: Large Vision-Language Models (LVLMs) have recently shown great promise in
  advancing robotics by combining embodied reasoning with robot control. A common
  approach involves training on embodied reasoning tasks related to robot control
  using Supervised Fine-Tuning (SFT). However, SFT datasets are often
  heuristically constructed and not explicitly optimized for improving robot
  control. Furthermore, SFT often leads to issues such as catastrophic forgetting
  and reduced generalization performance. To address these limitations, we
  introduce Robot-R1, a novel framework that leverages reinforcement learning to
  enhance embodied reasoning specifically for robot control. Robot-R1 learns to
  predict the next keypoint state required for task completion, conditioned on
  the current scene image and environment metadata derived from expert
  demonstrations. Inspired by the DeepSeek-R1 learning approach, Robot-R1 samples
  reasoning-based responses and reinforces those that lead to more accurate
  predictions. Our experiments show that models trained with Robot-R1 outperform
  SFT methods on embodied reasoning tasks. Despite having only 7B parameters,
  Robot-R1 even surpasses GPT-4o on reasoning tasks related to low-level action
  control, such as spatial and primitive movement reasoning.
paper_ai_summary: Robot-R1, a reinforcement learning framework, enhances embodied reasoning for robotics by predicting keypoint states, outperforming supervised fine-tuning methods and even surpassing GPT-4o in low-level action control tasks.
paper_ai_keywords: Large Vision-Language Models (LVLMs) embodied reasoning robot control Supervised Fine-Tuning (SFT) catastrophic forgetting generalization performance reinforcement learning keypoint state scene image environment metadata expert demonstrations DeepSeek-R1 reasoning-based responses parameter-efficient fine-tuning