title: Force Prompting: Video Generation Models Can Learn and Generalize
    Physics-based Control Signals
summary: Recent advances in video generation models have sparked interest in world
  models capable of simulating realistic environments. While navigation has been
  well-explored, physically meaningful interactions that mimic real-world forces
  remain largely understudied. In this work, we investigate using physical forces
  as a control signal for video generation and propose force prompts which enable
  users to interact with images through both localized point forces, such as
  poking a plant, and global wind force fields, such as wind blowing on fabric.
  We demonstrate that these force prompts can enable videos to respond
  realistically to physical control signals by leveraging the visual and motion
  prior in the original pretrained model, without using any 3D asset or physics
  simulator at inference. The primary challenge of force prompting is the
  difficulty in obtaining high quality paired force-video training data, both in
  the real world due to the difficulty of obtaining force signals, and in
  synthetic data due to limitations in the visual quality and domain diversity of
  physics simulators. Our key finding is that video generation models can
  generalize remarkably well when adapted to follow physical force conditioning
  from videos synthesized by Blender, even with limited demonstrations of few
  objects. Our method can generate videos which simulate forces across diverse
  geometries, settings, and materials. We also try to understand the source of
  this generalization and perform ablations that reveal two key elements: visual
  diversity and the use of specific text keywords during training. Our approach
  is trained on only around 15k training examples for a single day on four A100
  GPUs, and outperforms existing methods on force adherence and physics realism,
  bringing world models closer to real-world physics interactions. We release all
  datasets, code, weights, and interactive video demos at our project page.
paper_authors._id: 6835dc82635029ffb93dffea
paper_authors.name: Nate Gillman
paper_authors._id: 6835dc82635029ffb93dffeb
paper_authors.name: Charles Herrmann
paper_authors._id: 6835dc82635029ffb93dffec
paper_authors.name: Michael Freeman
paper_authors._id: 6835dc82635029ffb93dffed
paper_authors.name: Daksh Aggarwal
paper_authors._id: 6835dc82635029ffb93dffee
paper_authors.name: Evan Luo
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-28T08:59:26.536Z
paper_authors._id: 68361bd243cbbe13e0e962cb
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/J9vNatiYgDvShTUZT3Zmu.png
paper_authors.fullname: Evan Luo
paper_authors.type: user
paper_authors.user: Evan-Luo-jpg
paper_authors._id: 6835dc82635029ffb93dffef
paper_authors.name: Deqing Sun
paper_authors._id: 6835dc82635029ffb93dfff0
paper_authors.name: Chen Sun
paper_title: Force Prompting: Video Generation Models Can Learn and Generalize
    Physics-based Control Signals
paper_summary: Recent advances in video generation models have sparked interest in world
  models capable of simulating realistic environments. While navigation has been
  well-explored, physically meaningful interactions that mimic real-world forces
  remain largely understudied. In this work, we investigate using physical forces
  as a control signal for video generation and propose force prompts which enable
  users to interact with images through both localized point forces, such as
  poking a plant, and global wind force fields, such as wind blowing on fabric.
  We demonstrate that these force prompts can enable videos to respond
  realistically to physical control signals by leveraging the visual and motion
  prior in the original pretrained model, without using any 3D asset or physics
  simulator at inference. The primary challenge of force prompting is the
  difficulty in obtaining high quality paired force-video training data, both in
  the real world due to the difficulty of obtaining force signals, and in
  synthetic data due to limitations in the visual quality and domain diversity of
  physics simulators. Our key finding is that video generation models can
  generalize remarkably well when adapted to follow physical force conditioning
  from videos synthesized by Blender, even with limited demonstrations of few
  objects. Our method can generate videos which simulate forces across diverse
  geometries, settings, and materials. We also try to understand the source of
  this generalization and perform ablations that reveal two key elements: visual
  diversity and the use of specific text keywords during training. Our approach
  is trained on only around 15k training examples for a single day on four A100
  GPUs, and outperforms existing methods on force adherence and physics realism,
  bringing world models closer to real-world physics interactions. We release all
  datasets, code, weights, and interactive video demos at our project page.
paper_ai_summary: Force prompts enable video generation models to simulate realistic physical interactions using pretrained models and force conditioning from Blender-generated videos.
paper_ai_keywords: force prompts video generation models physical forces control signal visual and motion prior force adherence physics realism Blender-generated videos visual diversity text keywords