title: Generative Image Dynamics
summary: We present an approach to modeling an image-space prior on scene dynamics.
  Our prior is learned from a collection of motion trajectories extracted from
  real video sequences containing natural, oscillating motion such as trees,
  flowers, candles, and clothes blowing in the wind. Given a single image, our
  trained model uses a frequency-coordinated diffusion sampling process to
  predict a per-pixel long-term motion representation in the Fourier domain,
  which we call a neural stochastic motion texture. This representation can be
  converted into dense motion trajectories that span an entire video. Along with
  an image-based rendering module, these trajectories can be used for a number of
  downstream applications, such as turning still images into seamlessly looping
  dynamic videos, or allowing users to realistically interact with objects in
  real pictures.
paper_authors._id: 6503f7e5dcfe8fd06a78a6a2
paper_authors.name: Zhengqi Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-15T09:15:06.802Z
paper_authors._id: 6384f232cafe4e4ac406d116
paper_authors.avatarUrl: /avatars/33453dad4609231ba956811798e0b357.svg
paper_authors.fullname: Zhengqi Li
paper_authors.type: user
paper_authors.user: lixx2938
paper_authors._id: 6503f7e5dcfe8fd06a78a6a3
paper_authors.name: Richard Tucker
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-15T09:15:14.466Z
paper_authors._id: 63894f08d68e37abd59fed5c
paper_authors.avatarUrl: /avatars/ea3ac13ab6289005b5b082aa88c00238.svg
paper_authors.fullname: Richard Tucker
paper_authors.type: user
paper_authors.user: reyet
paper_authors._id: 6503f7e5dcfe8fd06a78a6a4
paper_authors.name: Noah Snavely
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-15T09:15:21.449Z
paper_authors._id: 632b79ca0a9dc02985af9e0b
paper_authors.avatarUrl: /avatars/e2ca585398a3212b2956a14804d0ba67.svg
paper_authors.fullname: Noah Snavely
paper_authors.type: user
paper_authors.user: jimantha
paper_authors._id: 6503f7e5dcfe8fd06a78a6a5
paper_authors.name: Aleksander Holynski
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-15T09:15:28.222Z
paper_authors._id: 62f6e2792e53c2efd33faa92
paper_authors.avatarUrl: /avatars/85d94b4022577747b8d2d10a82c2f3c7.svg
paper_authors.fullname: Aleksander Holynski
paper_authors.type: user
paper_authors.user: holynski
paper_title: Generative Image Dynamics
paper_summary: We present an approach to modeling an image-space prior on scene dynamics.
  Our prior is learned from a collection of motion trajectories extracted from
  real video sequences containing natural, oscillating motion such as trees,
  flowers, candles, and clothes blowing in the wind. Given a single image, our
  trained model uses a frequency-coordinated diffusion sampling process to
  predict a per-pixel long-term motion representation in the Fourier domain,
  which we call a neural stochastic motion texture. This representation can be
  converted into dense motion trajectories that span an entire video. Along with
  an image-based rendering module, these trajectories can be used for a number of
  downstream applications, such as turning still images into seamlessly looping
  dynamic videos, or allowing users to realistically interact with objects in
  real pictures.
paper_ai_summary: A frequency-coordinated diffusion sampling process is used to predict long-term motion representations for still images, enabling dynamic video creation and interactive scene manipulation.
paper_ai_keywords: frequency-coordinated diffusion sampling process neural stochastic motion texture Fourier domain dense motion trajectories image-based rendering module dynamic videos interactive scene manipulation