title: Interactive Post-Training for Vision-Language-Action Models
summary: We introduce RIPT-VLA, a simple and scalable reinforcement-learning-based
  interactive post-training paradigm that fine-tunes pretrained
  Vision-Language-Action (VLA) models using only sparse binary success rewards.
  Existing VLA training pipelines rely heavily on offline expert demonstration
  data and supervised imitation, limiting their ability to adapt to new tasks and
  environments under low-data regimes. RIPT-VLA addresses this by enabling
  interactive post-training with a stable policy optimization algorithm based on
  dynamic rollout sampling and leave-one-out advantage estimation.
    RIPT-VLA has the following characteristics. First, it applies to various VLA
  models, resulting in an improvement on the lightweight QueST model by 21.2%,
  and the 7B OpenVLA-OFT model to an unprecedented 97.5% success rate. Second, it
  is computationally efficient and data-efficient: with only one demonstration,
  RIPT-VLA enables an unworkable SFT model (4%) to succeed with a 97% success
  rate within 15 iterations. Furthermore, we demonstrate that the policy learned
  by RIPT-VLA generalizes across different tasks and scenarios and is robust to
  the initial state context. These results highlight RIPT-VLA as a practical and
  effective paradigm for post-training VLA models through minimal supervision.
paper_authors._id: 6833f7847e0c637c71de0ec6
paper_authors.name: Shuhan Tan
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-26T08:09:10.881Z
paper_authors._id: 64b64debeb9a833e08d079fd
paper_authors.avatarUrl: /avatars/62ad6f5a8c1b69252e855ef26cc4e7c2.svg
paper_authors.fullname: Shuhan Tan
paper_authors.type: user
paper_authors.user: tanshh97
paper_authors._id: 6833f7847e0c637c71de0ec7
paper_authors.name: Kairan Dou
paper_authors._id: 6833f7847e0c637c71de0ec8
paper_authors.name: Yue Zhao
paper_authors._id: 6833f7847e0c637c71de0ec9
paper_authors.name: Philipp Krähenbühl
paper_title: Interactive Post-Training for Vision-Language-Action Models
paper_summary: We introduce RIPT-VLA, a simple and scalable reinforcement-learning-based
  interactive post-training paradigm that fine-tunes pretrained
  Vision-Language-Action (VLA) models using only sparse binary success rewards.
  Existing VLA training pipelines rely heavily on offline expert demonstration
  data and supervised imitation, limiting their ability to adapt to new tasks and
  environments under low-data regimes. RIPT-VLA addresses this by enabling
  interactive post-training with a stable policy optimization algorithm based on
  dynamic rollout sampling and leave-one-out advantage estimation.
    RIPT-VLA has the following characteristics. First, it applies to various VLA
  models, resulting in an improvement on the lightweight QueST model by 21.2%,
  and the 7B OpenVLA-OFT model to an unprecedented 97.5% success rate. Second, it
  is computationally efficient and data-efficient: with only one demonstration,
  RIPT-VLA enables an unworkable SFT model (4%) to succeed with a 97% success
  rate within 15 iterations. Furthermore, we demonstrate that the policy learned
  by RIPT-VLA generalizes across different tasks and scenarios and is robust to
  the initial state context. These results highlight RIPT-VLA as a practical and
  effective paradigm for post-training VLA models through minimal supervision.
paper_ai_summary: RIPT-VLA is a reinforcement learning-based interactive post-training paradigm that enhances pretrained Vision-Language-Action models using sparse binary success rewards, improving adaptability and generalization.
paper_ai_keywords: reinforcement-learning-based interactive post-training Vision-Language-Action (VLA) models sparse binary success rewards offline expert demonstration supervised imitation dynamic rollout sampling leave-one-out advantage estimation policy optimization lightweight QueST model OpenVLA-OFT model success rate computational efficiency data-efficient policy learned generalization initial state context