title: RLVR-World: Training World Models with Reinforcement Learning
summary: World models predict state transitions in response to actions and are
  increasingly developed across diverse modalities. However, standard training
  objectives such as maximum likelihood estimation (MLE) often misalign with
  task-specific goals of world models, i.e., transition prediction metrics like
  accuracy or perceptual quality. In this paper, we present RLVR-World, a unified
  framework that leverages reinforcement learning with verifiable rewards (RLVR)
  to directly optimize world models for such metrics. Despite formulating world
  modeling as autoregressive prediction of tokenized sequences, RLVR-World
  evaluates metrics of decoded predictions as verifiable rewards. We demonstrate
  substantial performance gains on both language- and video-based world models
  across domains, including text games, web navigation, and robot manipulation.
  Our work indicates that, beyond recent advances in reasoning language models,
  RLVR offers a promising post-training paradigm for enhancing the utility of
  generative models more broadly.
paper_authors._id: 682e8369b16c79c271b4db81
paper_authors.name: Jialong Wu
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2025-05-22T05:44:24.633Z
paper_authors._id: 643b866bff50448bcfc7d1d1
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/q12v-BatVKimoi8q-coi-.jpeg
paper_authors.fullname: Jialong Wu
paper_authors.type: user
paper_authors.user: manchery
paper_authors._id: 682e8369b16c79c271b4db82
paper_authors.name: Shaofeng Yin
paper_authors._id: 682e8369b16c79c271b4db83
paper_authors.name: Ningya Feng
paper_authors._id: 682e8369b16c79c271b4db84
paper_authors.name: Mingsheng Long
paper_title: RLVR-World: Training World Models with Reinforcement Learning
paper_summary: World models predict state transitions in response to actions and are
  increasingly developed across diverse modalities. However, standard training
  objectives such as maximum likelihood estimation (MLE) often misalign with
  task-specific goals of world models, i.e., transition prediction metrics like
  accuracy or perceptual quality. In this paper, we present RLVR-World, a unified
  framework that leverages reinforcement learning with verifiable rewards (RLVR)
  to directly optimize world models for such metrics. Despite formulating world
  modeling as autoregressive prediction of tokenized sequences, RLVR-World
  evaluates metrics of decoded predictions as verifiable rewards. We demonstrate
  substantial performance gains on both language- and video-based world models
  across domains, including text games, web navigation, and robot manipulation.
  Our work indicates that, beyond recent advances in reasoning language models,
  RLVR offers a promising post-training paradigm for enhancing the utility of
  generative models more broadly.
paper_ai_summary: RLVR-World uses reinforcement learning with verifiable rewards to optimize world models for task-specific metrics, achieving improved performance across language and video domains.
paper_ai_keywords: world models maximum likelihood estimation transition prediction reinforcement learning verifiable rewards autoregressive prediction tokenized sequences text games web navigation robot manipulation