title: SG-I2V: Self-Guided Trajectory Control in Image-to-Video Generation
summary: Methods for image-to-video generation have achieved impressive,
  photo-realistic quality. However, adjusting specific elements in generated
  videos, such as object motion or camera movement, is often a tedious process of
  trial and error, e.g., involving re-generating videos with different random
  seeds. Recent techniques address this issue by fine-tuning a pre-trained model
  to follow conditioning signals, such as bounding boxes or point trajectories.
  Yet, this fine-tuning procedure can be computationally expensive, and it
  requires datasets with annotated object motion, which can be difficult to
  procure. In this work, we introduce SG-I2V, a framework for controllable
  image-to-video generation that is self-guidedx2013offering
  zero-shot control by relying solely on the knowledge present in a pre-trained
  image-to-video diffusion model without the need for fine-tuning or external
  knowledge. Our zero-shot method outperforms unsupervised baselines while being
  competitive with supervised models in terms of visual quality and motion
  fidelity.
paper_authors._id: 672d851d14742348552868ae
paper_authors.name: Koichi Namekata
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-11-15T09:31:25.687Z
paper_authors._id: 65aec2b6101482afcc3126f2
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/65aec2b6101482afcc3126f2/IEKT6u1Upx2clt7qhhoUB.jpeg
paper_authors.fullname: Koichi Namekata
paper_authors.type: user
paper_authors.user: kmcode
paper_authors._id: 672d851d14742348552868af
paper_authors.name: Sherwin Bahmani
paper_authors._id: 672d851d14742348552868b0
paper_authors.name: Ziyi Wu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-06T09:09:59.064Z
paper_authors._id: 666bb5a310073a7de7ba0d8d
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/Z-yB3jbYIjhZQ9WwJhleq.jpeg
paper_authors.fullname: Ziyi Wu
paper_authors.type: user
paper_authors.user: Wuziyi616
paper_authors._id: 672d851d14742348552868b1
paper_authors.name: Yash Kant
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-13T08:26:49.298Z
paper_authors._id: 638546d0a179f856005ae310
paper_authors.avatarUrl: /avatars/f7f2fface336c8e168a1daaf9fd4d40c.svg
paper_authors.fullname: yashkant
paper_authors.type: user
paper_authors.user: yashkant
paper_authors._id: 672d851d14742348552868b2
paper_authors.name: Igor Gilitschenski
paper_authors._id: 672d851d14742348552868b3
paper_authors.name: David B. Lindell
paper_title: SG-I2V: Self-Guided Trajectory Control in Image-to-Video Generation
paper_summary: Methods for image-to-video generation have achieved impressive,
  photo-realistic quality. However, adjusting specific elements in generated
  videos, such as object motion or camera movement, is often a tedious process of
  trial and error, e.g., involving re-generating videos with different random
  seeds. Recent techniques address this issue by fine-tuning a pre-trained model
  to follow conditioning signals, such as bounding boxes or point trajectories.
  Yet, this fine-tuning procedure can be computationally expensive, and it
  requires datasets with annotated object motion, which can be difficult to
  procure. In this work, we introduce SG-I2V, a framework for controllable
  image-to-video generation that is self-guidedx2013offering
  zero-shot control by relying solely on the knowledge present in a pre-trained
  image-to-video diffusion model without the need for fine-tuning or external
  knowledge. Our zero-shot method outperforms unsupervised baselines while being
  competitive with supervised models in terms of visual quality and motion
  fidelity.
paper_ai_summary: SG-I2V is a self-guided framework for image-to-video generation that achieves zero-shot control and competitive visual quality and motion fidelity without fine-tuning or external knowledge.
paper_ai_keywords: image-to-video generation diffusion model zero-shot control visual quality motion fidelity