EWMBench: Evaluating Scene, Motion, and Semantic Quality in Embodied
  World Models
Recent advances in creative AI have enabled the synthesis of high-fidelity
images and videos conditioned on language instructions. Building on these
developments, text-to-video diffusion models have evolved into embodied world
models (EWMs) capable of generating physically plausible scenes from language
commands, effectively bridging vision and action in embodied AI applications.
This work addresses the critical challenge of evaluating EWMs beyond general
perceptual metrics to ensure the generation of physically grounded and
action-consistent behaviors. We propose the Embodied World Model Benchmark
(EWMBench), a dedicated framework designed to evaluate EWMs based on three key
aspects: visual scene consistency, motion correctness, and semantic alignment.
Our approach leverages a meticulously curated dataset encompassing diverse
scenes and motion patterns, alongside a comprehensive multi-dimensional
evaluation toolkit, to assess and compare candidate models. The proposed
benchmark not only identifies the limitations of existing video generation
models in meeting the unique requirements of embodied tasks but also provides
valuable insights to guide future advancements in the field. The dataset and
evaluation tools are publicly available at
https://github.com/AgibotTech/EWMBench.
6826ae4611765454f5757d7c
Hu Yue
6826ae4611765454f5757d7d
Siyuan Huang
admin_assigned
2025-05-16T08:16:33.829Z
63c7a33121bd95f80ed74652
/avatars/7dd59afea785a2bff0ec2b757abd474e.svg
Siyuan Huang
user
thuhsy
6826ae4611765454f5757d7e
Yue Liao
6826ae4611765454f5757d7f
Shengcong Chen
admin_assigned
2025-05-16T08:16:15.416Z
6575f9aeca03b6c514fe6e5c
/avatars/a6e9d428beaa124ee989d702b9bf4f85.svg
Shengcong Chen
user
Shengcong
6826ae4611765454f5757d80
Pengfei Zhou
6826ae4611765454f5757d81
Liliang Chen
admin_assigned
2025-05-16T08:15:58.407Z
640b00555a9c21b95c6449b3
/avatars/5fa43b956f3acc671f033e31b7ca76c5.svg
Liliang Chen
user
pathcn
6826ae4611765454f5757d82
Maoqing Yao
admin_assigned
2025-05-16T08:15:51.039Z
67739bfa64e8b7438ae68eb4
/avatars/15193bfbce487b2de4ce8c86bd18885a.svg
Maoqing Yao
user
AutobotZero
6826ae4611765454f5757d83
Guanghui Ren
admin_assigned
2025-05-16T08:15:44.660Z
646ec9b135f55eb49e405faa
/avatars/a17194be585d20e2a021e77a5a20e213.svg
Guanghui Ren
user
sundrops
EWMBench: Evaluating Scene, Motion, and Semantic Quality in Embodied
  World Models
Recent advances in creative AI have enabled the synthesis of high-fidelity
images and videos conditioned on language instructions. Building on these
developments, text-to-video diffusion models have evolved into embodied world
models (EWMs) capable of generating physically plausible scenes from language
commands, effectively bridging vision and action in embodied AI applications.
This work addresses the critical challenge of evaluating EWMs beyond general
perceptual metrics to ensure the generation of physically grounded and
action-consistent behaviors. We propose the Embodied World Model Benchmark
(EWMBench), a dedicated framework designed to evaluate EWMs based on three key
aspects: visual scene consistency, motion correctness, and semantic alignment.
Our approach leverages a meticulously curated dataset encompassing diverse
scenes and motion patterns, alongside a comprehensive multi-dimensional
evaluation toolkit, to assess and compare candidate models. The proposed
benchmark not only identifies the limitations of existing video generation
models in meeting the unique requirements of embodied tasks but also provides
valuable insights to guide future advancements in the field. The dataset and
evaluation tools are publicly available at
https://github.com/AgibotTech/EWMBench.
A new benchmark framework evaluates text-to-video diffusion models in embodied AI for visual, motion, and semantic consistency, using a diverse dataset and evaluation toolkit.
text-to-video diffusion models embodied world models visual scene consistency motion correctness semantic alignment evaluation framework benchmark