title: Phantom: Subject-consistent video generation via cross-modal alignment
summary: The continuous development of foundational models for video generation is
  evolving into various applications, with subject-consistent video generation
  still in the exploratory stage. We refer to this as Subject-to-Video, which
  extracts subject elements from reference images and generates
  subject-consistent video through textual instructions. We believe that the
  essence of subject-to-video lies in balancing the dual-modal prompts of text
  and image, thereby deeply and simultaneously aligning both text and visual
  content. To this end, we propose Phantom, a unified video generation framework
  for both single and multi-subject references. Building on existing
  text-to-video and image-to-video architectures, we redesign the joint
  text-image injection model and drive it to learn cross-modal alignment via
  text-image-video triplet data. In particular, we emphasize subject consistency
  in human generation, covering existing ID-preserving video generation while
  offering enhanced advantages. The project homepage is here
  https://phantom-video.github.io/Phantom/.
paper_authors._id: 67b40141ad717fe02e188c1a
paper_authors.name: Lijie Liu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-18T09:31:42.570Z
paper_authors._id: 63a950ac3453852ef5394178
paper_authors.avatarUrl: /avatars/48a5e537b10e2247a17e63502e3201a6.svg
paper_authors.fullname: Lijie Liu
paper_authors.type: user
paper_authors.user: liulj13
paper_authors._id: 67b40141ad717fe02e188c1b
paper_authors.name: Tianxiang Ma
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-19T09:45:00.117Z
paper_authors._id: 657ab4705e1c941f4c2f7877
paper_authors.avatarUrl: /avatars/c450f81f83dd0436ae120ab15616c4f7.svg
paper_authors.fullname: Tianxiang Ma
paper_authors.type: user
paper_authors.user: Grayson111
paper_authors._id: 67b40141ad717fe02e188c1c
paper_authors.name: Bingchuan Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-19T09:47:57.441Z
paper_authors._id: 63b415037af2e415f2599c18
paper_authors.avatarUrl: /avatars/4afbe7d6d05a702f1beeed9c53e78153.svg
paper_authors.fullname: Bingchuan Li
paper_authors.type: user
paper_authors.user: lbc402
paper_authors._id: 67b40141ad717fe02e188c1d
paper_authors.name: Zhuowei Chen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-19T09:47:50.995Z
paper_authors._id: 6304e2dabad6ce7fc0287d57
paper_authors.avatarUrl: /avatars/3fd4a9a62b0ef98db2573411463a9247.svg
paper_authors.fullname: Zhuowei_Chen
paper_authors.type: user
paper_authors.user: ZhuoweiChen
paper_authors._id: 67b40141ad717fe02e188c1e
paper_authors.name: Jiawei Liu
paper_authors._id: 67b40141ad717fe02e188c1f
paper_authors.name: Qian He
paper_authors._id: 67b40141ad717fe02e188c20
paper_authors.name: Xinglong Wu
paper_title: Phantom: Subject-consistent video generation via cross-modal alignment
paper_summary: The continuous development of foundational models for video generation is
  evolving into various applications, with subject-consistent video generation
  still in the exploratory stage. We refer to this as Subject-to-Video, which
  extracts subject elements from reference images and generates
  subject-consistent video through textual instructions. We believe that the
  essence of subject-to-video lies in balancing the dual-modal prompts of text
  and image, thereby deeply and simultaneously aligning both text and visual
  content. To this end, we propose Phantom, a unified video generation framework
  for both single and multi-subject references. Building on existing
  text-to-video and image-to-video architectures, we redesign the joint
  text-image injection model and drive it to learn cross-modal alignment via
  text-image-video triplet data. In particular, we emphasize subject consistency
  in human generation, covering existing ID-preserving video generation while
  offering enhanced advantages. The project homepage is here
  https://phantom-video.github.io/Phantom/.
paper_ai_summary: A unified video generation framework called Phantom integrates text and image inputs to produce subject-consistent videos, focusing on human generation and leveraging cross-modal alignment.
paper_ai_keywords: text-to-video image-to-video joint text-image injection model cross-modal alignment text-image-video triplet data subject consistency ID-preserving video generation