title: ReVision: High-Quality, Low-Cost Video Generation with Explicit 3D
    Physics Modeling for Complex Motion and Interaction
summary: In recent years, video generation has seen significant advancements. However,
  challenges still persist in generating complex motions and interactions. To
  address these challenges, we introduce ReVision, a plug-and-play framework that
  explicitly integrates parameterized 3D physical knowledge into a pretrained
  conditional video generation model, significantly enhancing its ability to
  generate high-quality videos with complex motion and interactions.
  Specifically, ReVision consists of three stages. First, a video diffusion model
  is used to generate a coarse video. Next, we extract a set of 2D and 3D
  features from the coarse video to construct a 3D object-centric representation,
  which is then refined by our proposed parameterized physical prior model to
  produce an accurate 3D motion sequence. Finally, this refined motion sequence
  is fed back into the same video diffusion model as additional conditioning,
  enabling the generation of motion-consistent videos, even in scenarios
  involving complex actions and interactions. We validate the effectiveness of
  our approach on Stable Video Diffusion, where ReVision significantly improves
  motion fidelity and coherence. Remarkably, with only 1.5B parameters, it even
  outperforms a state-of-the-art video generation model with over 13B parameters
  on complex video generation by a substantial margin. Our results suggest that,
  by incorporating 3D physical knowledge, even a relatively small video diffusion
  model can generate complex motions and interactions with greater realism and
  controllability, offering a promising solution for physically plausible video
  generation.
paper_authors._id: 6812db1567abbb1d11fb4e9d
paper_authors.name: Qihao Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-05T08:14:06.173Z
paper_authors._id: 639f1e519f1f2baab2f00d22
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/639f1e519f1f2baab2f00d22/pFjd51WZuVZ3A11rItvmk.jpeg
paper_authors.fullname: Qihao Liu
paper_authors.type: user
paper_authors.user: QHL067
paper_authors._id: 6812db1567abbb1d11fb4e9e
paper_authors.name: Ju He
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-06T08:34:31.327Z
paper_authors._id: 661c9059bcd78151e5c06ea1
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/661c9059bcd78151e5c06ea1/27bfNo1LZeZQ77vWuAa10.png
paper_authors.fullname: Ju He
paper_authors.type: user
paper_authors.user: turkeyju
paper_authors._id: 6812db1567abbb1d11fb4e9f
paper_authors.name: Qihang Yu
paper_authors._id: 6812db1567abbb1d11fb4ea0
paper_authors.name: Liang-Chieh Chen
paper_authors._id: 6812db1567abbb1d11fb4ea1
paper_authors.name: Alan Yuille
paper_title: ReVision: High-Quality, Low-Cost Video Generation with Explicit 3D
    Physics Modeling for Complex Motion and Interaction
paper_summary: In recent years, video generation has seen significant advancements. However,
  challenges still persist in generating complex motions and interactions. To
  address these challenges, we introduce ReVision, a plug-and-play framework that
  explicitly integrates parameterized 3D physical knowledge into a pretrained
  conditional video generation model, significantly enhancing its ability to
  generate high-quality videos with complex motion and interactions.
  Specifically, ReVision consists of three stages. First, a video diffusion model
  is used to generate a coarse video. Next, we extract a set of 2D and 3D
  features from the coarse video to construct a 3D object-centric representation,
  which is then refined by our proposed parameterized physical prior model to
  produce an accurate 3D motion sequence. Finally, this refined motion sequence
  is fed back into the same video diffusion model as additional conditioning,
  enabling the generation of motion-consistent videos, even in scenarios
  involving complex actions and interactions. We validate the effectiveness of
  our approach on Stable Video Diffusion, where ReVision significantly improves
  motion fidelity and coherence. Remarkably, with only 1.5B parameters, it even
  outperforms a state-of-the-art video generation model with over 13B parameters
  on complex video generation by a substantial margin. Our results suggest that,
  by incorporating 3D physical knowledge, even a relatively small video diffusion
  model can generate complex motions and interactions with greater realism and
  controllability, offering a promising solution for physically plausible video
  generation.
paper_ai_summary: ReVision enhances video generation by integrating 3D physical priors into video diffusion models, improving motion fidelity and coherence with fewer parameters.
paper_ai_keywords: video diffusion model 3D object-centric representation parameterized physical prior model motion-consistent videos motion fidelity coherence Stable Video Diffusion