title: EnerVerse-AC: Envisioning Embodied Environments with Action Condition
summary: Robotic imitation learning has advanced from solving static tasks to
  addressing dynamic interaction scenarios, but testing and evaluation remain
  costly and challenging due to the need for real-time interaction with dynamic
  environments. We propose EnerVerse-AC (EVAC), an action-conditional world model
  that generates future visual observations based on an agent's predicted
  actions, enabling realistic and controllable robotic inference. Building on
  prior architectures, EVAC introduces a multi-level action-conditioning
  mechanism and ray map encoding for dynamic multi-view image generation while
  expanding training data with diverse failure trajectories to improve
  generalization. As both a data engine and evaluator, EVAC augments
  human-collected trajectories into diverse datasets and generates realistic,
  action-conditioned video observations for policy testing, eliminating the need
  for physical robots or complex simulations. This approach significantly reduces
  costs while maintaining high fidelity in robotic manipulation evaluation.
  Extensive experiments validate the effectiveness of our method. Code,
  checkpoints, and datasets can be found at
  <https://annaj2178.github.io/EnerverseAC.github.io>.
paper_authors._id: 6826b00c251d26fc0cd035cc
paper_authors.name: Yuxin Jiang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-16T08:10:30.068Z
paper_authors._id: 63c20105726f62e411fbe882
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/63c20105726f62e411fbe882/2UsU9O2psbDjJzz-sAmGH.jpeg
paper_authors.fullname: Yuxin Jiang
paper_authors.type: user
paper_authors.user: YuxinJiang
paper_authors._id: 6826b00c251d26fc0cd035cd
paper_authors.name: Shengcong Chen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-16T08:10:37.358Z
paper_authors._id: 6575f9aeca03b6c514fe6e5c
paper_authors.avatarUrl: /avatars/a6e9d428beaa124ee989d702b9bf4f85.svg
paper_authors.fullname: Shengcong Chen
paper_authors.type: user
paper_authors.user: Shengcong
paper_authors._id: 6826b00c251d26fc0cd035ce
paper_authors.name: Siyuan Huang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-16T08:10:50.217Z
paper_authors._id: 63c7a33121bd95f80ed74652
paper_authors.avatarUrl: /avatars/7dd59afea785a2bff0ec2b757abd474e.svg
paper_authors.fullname: Siyuan Huang
paper_authors.type: user
paper_authors.user: thuhsy
paper_authors._id: 6826b00c251d26fc0cd035cf
paper_authors.name: Liliang Chen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-16T08:10:55.980Z
paper_authors._id: 640b00555a9c21b95c6449b3
paper_authors.avatarUrl: /avatars/5fa43b956f3acc671f033e31b7ca76c5.svg
paper_authors.fullname: Liliang Chen
paper_authors.type: user
paper_authors.user: pathcn
paper_authors._id: 6826b00c251d26fc0cd035d0
paper_authors.name: Pengfei Zhou
paper_authors._id: 6826b00c251d26fc0cd035d1
paper_authors.name: Yue Liao
paper_authors._id: 6826b00c251d26fc0cd035d2
paper_authors.name: Xindong He
paper_authors._id: 6826b00c251d26fc0cd035d3
paper_authors.name: Chiming Liu
paper_authors._id: 6826b00c251d26fc0cd035d4
paper_authors.name: Hongsheng Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-16T08:11:29.857Z
paper_authors._id: 65c04e9c27a5fdca81abcbd9
paper_authors.avatarUrl: /avatars/12a155683c824fa23da4a9e2bed4f64e.svg
paper_authors.fullname: Hongsheng LI
paper_authors.type: user
paper_authors.user: hsli-cuhk
paper_authors._id: 6826b00c251d26fc0cd035d5
paper_authors.name: Maoqing Yao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-16T08:11:36.940Z
paper_authors._id: 67739bfa64e8b7438ae68eb4
paper_authors.avatarUrl: /avatars/15193bfbce487b2de4ce8c86bd18885a.svg
paper_authors.fullname: Maoqing Yao
paper_authors.type: user
paper_authors.user: AutobotZero
paper_authors._id: 6826b00c251d26fc0cd035d6
paper_authors.name: Guanghui Ren
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-16T08:11:44.407Z
paper_authors._id: 646ec9b135f55eb49e405faa
paper_authors.avatarUrl: /avatars/a17194be585d20e2a021e77a5a20e213.svg
paper_authors.fullname: Guanghui Ren
paper_authors.type: user
paper_authors.user: sundrops
paper_title: EnerVerse-AC: Envisioning Embodied Environments with Action Condition
paper_summary: Robotic imitation learning has advanced from solving static tasks to
  addressing dynamic interaction scenarios, but testing and evaluation remain
  costly and challenging due to the need for real-time interaction with dynamic
  environments. We propose EnerVerse-AC (EVAC), an action-conditional world model
  that generates future visual observations based on an agent's predicted
  actions, enabling realistic and controllable robotic inference. Building on
  prior architectures, EVAC introduces a multi-level action-conditioning
  mechanism and ray map encoding for dynamic multi-view image generation while
  expanding training data with diverse failure trajectories to improve
  generalization. As both a data engine and evaluator, EVAC augments
  human-collected trajectories into diverse datasets and generates realistic,
  action-conditioned video observations for policy testing, eliminating the need
  for physical robots or complex simulations. This approach significantly reduces
  costs while maintaining high fidelity in robotic manipulation evaluation.
  Extensive experiments validate the effectiveness of our method. Code,
  checkpoints, and datasets can be found at
  <https://annaj2178.github.io/EnerverseAC.github.io>.
paper_ai_summary: EnerVerse-AC, an action-conditional world model, enables realistic robotic inference and testing by simulating future actions and observations, thereby reducing costs and improving generalization in dynamic settings.
paper_ai_keywords: action-conditional world model robotic imitation learning dynamic multi-view image generation ray map encoding failure trajectories policy testing