title: ORV: 4D Occupancy-centric Robot Video Generation
summary: Acquiring real-world robotic simulation data through teleoperation is
  notoriously time-consuming and labor-intensive. Recently, action-driven
  generative models have gained widespread adoption in robot learning and
  simulation, as they eliminate safety concerns and reduce maintenance efforts.
  However, the action sequences used in these methods often result in limited
  control precision and poor generalization due to their globally coarse
  alignment. To address these limitations, we propose ORV, an Occupancy-centric
  Robot Video generation framework, which utilizes 4D semantic occupancy
  sequences as a fine-grained representation to provide more accurate semantic
  and geometric guidance for video generation. By leveraging occupancy-based
  representations, ORV enables seamless translation of simulation data into
  photorealistic robot videos, while ensuring high temporal consistency and
  precise controllability. Furthermore, our framework supports the simultaneous
  generation of multi-view videos of robot gripping operations - an important
  capability for downstream robotic learning tasks. Extensive experimental
  results demonstrate that ORV consistently outperforms existing baseline methods
  across various datasets and sub-tasks. Demo, Code and Model:
  https://orangesodahub.github.io/ORV
paper_authors._id: 683fee0a179d710da07d4352
paper_authors.name: Xiuyu Yang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-04T08:54:13.216Z
paper_authors._id: 634aab35dcf125e4dafc87b1
paper_authors.avatarUrl: /avatars/aa2db84fd423e9eefe3ef3167c9d3999.svg
paper_authors.fullname: YangXiuyu
paper_authors.type: user
paper_authors.user: gzzyyxy
paper_authors._id: 683fee0a179d710da07d4353
paper_authors.name: Bohan Li
paper_authors._id: 683fee0a179d710da07d4354
paper_authors.name: Shaocong Xu
paper_authors._id: 683fee0a179d710da07d4355
paper_authors.name: Nan Wang
paper_authors._id: 683fee0a179d710da07d4356
paper_authors.name: Chongjie Ye
paper_authors._id: 683fee0a179d710da07d4357
paper_authors.name: Zhaoxi Chen
paper_authors._id: 683fee0a179d710da07d4358
paper_authors.name: Minghan Qin
paper_authors._id: 683fee0a179d710da07d4359
paper_authors.name: Yikang Ding
paper_authors._id: 683fee0a179d710da07d435a
paper_authors.name: Xin Jin
paper_authors._id: 683fee0a179d710da07d435b
paper_authors.name: Hang Zhao
paper_authors._id: 683fee0a179d710da07d435c
paper_authors.name: Hao Zhao
paper_title: ORV: 4D Occupancy-centric Robot Video Generation
paper_summary: Acquiring real-world robotic simulation data through teleoperation is
  notoriously time-consuming and labor-intensive. Recently, action-driven
  generative models have gained widespread adoption in robot learning and
  simulation, as they eliminate safety concerns and reduce maintenance efforts.
  However, the action sequences used in these methods often result in limited
  control precision and poor generalization due to their globally coarse
  alignment. To address these limitations, we propose ORV, an Occupancy-centric
  Robot Video generation framework, which utilizes 4D semantic occupancy
  sequences as a fine-grained representation to provide more accurate semantic
  and geometric guidance for video generation. By leveraging occupancy-based
  representations, ORV enables seamless translation of simulation data into
  photorealistic robot videos, while ensuring high temporal consistency and
  precise controllability. Furthermore, our framework supports the simultaneous
  generation of multi-view videos of robot gripping operations - an important
  capability for downstream robotic learning tasks. Extensive experimental
  results demonstrate that ORV consistently outperforms existing baseline methods
  across various datasets and sub-tasks. Demo, Code and Model:
  https://orangesodahub.github.io/ORV
paper_ai_summary: ORV, an Occupancy-centric Robot Video generation framework, uses 4D semantic occupancy sequences to produce photorealistic, temporally consistent, and precisely controllable robot videos, enhancing existing methods.
paper_ai_keywords: action-driven generative models occupancy-centric 4D semantic occupancy sequences video generation photorealistic robot videos temporal consistency precise controllability multi-view videos