title: DreamRelation: Relation-Centric Video Customization
summary: Relational video customization refers to the creation of personalized videos
  that depict user-specified relations between two subjects, a crucial task for
  comprehending real-world visual content. While existing methods can personalize
  subject appearances and motions, they still struggle with complex relational
  video customization, where precise relational modeling and high generalization
  across subject categories are essential. The primary challenge arises from the
  intricate spatial arrangements, layout variations, and nuanced temporal
  dynamics inherent in relations; consequently, current models tend to
  overemphasize irrelevant visual details rather than capturing meaningful
  interactions. To address these challenges, we propose DreamRelation, a novel
  approach that personalizes relations through a small set of exemplar videos,
  leveraging two key components: Relational Decoupling Learning and Relational
  Dynamics Enhancement. First, in Relational Decoupling Learning, we disentangle
  relations from subject appearances using relation LoRA triplet and hybrid mask
  training strategy, ensuring better generalization across diverse relationships.
  Furthermore, we determine the optimal design of relation LoRA triplet by
  analyzing the distinct roles of the query, key, and value features within
  MM-DiT's attention mechanism, making DreamRelation the first relational video
  generation framework with explainable components. Second, in Relational
  Dynamics Enhancement, we introduce space-time relational contrastive loss,
  which prioritizes relational dynamics while minimizing the reliance on detailed
  subject appearances. Extensive experiments demonstrate that DreamRelation
  outperforms state-of-the-art methods in relational video customization. Code
  and models will be made publicly available.
paper_authors._id: 67cfb2efb77bc8e7d415f904
paper_authors.name: Yujie Wei
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-11T11:17:35.633Z
paper_authors._id: 637f70d6fab5db9101c3dfc8
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/637f70d6fab5db9101c3dfc8/NgkYNXWLDavLbrnCby2Fl.jpeg
paper_authors.fullname: Yujie Wei
paper_authors.type: user
paper_authors.user: weilllllls
paper_authors._id: 67cfb2efb77bc8e7d415f905
paper_authors.name: Shiwei Zhang
paper_authors._id: 67cfb2efb77bc8e7d415f906
paper_authors.name: Hangjie Yuan
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-11T08:21:32.780Z
paper_authors._id: 649d54b314afbb10ce2a9eeb
paper_authors.avatarUrl: /avatars/15c325d8c2273ff63569f23015e98486.svg
paper_authors.fullname: Hangjie Yuan
paper_authors.type: user
paper_authors.user: JacobYuan
paper_authors._id: 67cfb2efb77bc8e7d415f907
paper_authors.name: Biao Gong
paper_authors._id: 67cfb2efb77bc8e7d415f908
paper_authors.name: Longxiang Tang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-11T08:21:30.700Z
paper_authors._id: 6492a0d8d4ae24c933ace44d
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/DXIky2sdPwmiCOR9p-JBQ.png
paper_authors.fullname: Longxiang Tang
paper_authors.type: user
paper_authors.user: lloong
paper_authors._id: 67cfb2efb77bc8e7d415f909
paper_authors.name: Xiang Wang
paper_authors._id: 67cfb2efb77bc8e7d415f90a
paper_authors.name: Haonan Qiu
paper_authors._id: 67cfb2efb77bc8e7d415f90b
paper_authors.name: Hengjia Li
paper_authors._id: 67cfb2efb77bc8e7d415f90c
paper_authors.name: Shuai Tan
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-11T10:22:02.278Z
paper_authors._id: 6697b62a27bf71ffe289492f
paper_authors.avatarUrl: /avatars/f48095f1e2ca69836294343301be1700.svg
paper_authors.fullname: Shuai Tan
paper_authors.type: user
paper_authors.user: Shuaishuai0219
paper_authors._id: 67cfb2efb77bc8e7d415f90d
paper_authors.name: Yingya Zhang
paper_authors._id: 67cfb2efb77bc8e7d415f90e
paper_authors.name: Hongming Shan
paper_title: DreamRelation: Relation-Centric Video Customization
paper_summary: Relational video customization refers to the creation of personalized videos
  that depict user-specified relations between two subjects, a crucial task for
  comprehending real-world visual content. While existing methods can personalize
  subject appearances and motions, they still struggle with complex relational
  video customization, where precise relational modeling and high generalization
  across subject categories are essential. The primary challenge arises from the
  intricate spatial arrangements, layout variations, and nuanced temporal
  dynamics inherent in relations; consequently, current models tend to
  overemphasize irrelevant visual details rather than capturing meaningful
  interactions. To address these challenges, we propose DreamRelation, a novel
  approach that personalizes relations through a small set of exemplar videos,
  leveraging two key components: Relational Decoupling Learning and Relational
  Dynamics Enhancement. First, in Relational Decoupling Learning, we disentangle
  relations from subject appearances using relation LoRA triplet and hybrid mask
  training strategy, ensuring better generalization across diverse relationships.
  Furthermore, we determine the optimal design of relation LoRA triplet by
  analyzing the distinct roles of the query, key, and value features within
  MM-DiT's attention mechanism, making DreamRelation the first relational video
  generation framework with explainable components. Second, in Relational
  Dynamics Enhancement, we introduce space-time relational contrastive loss,
  which prioritizes relational dynamics while minimizing the reliance on detailed
  subject appearances. Extensive experiments demonstrate that DreamRelation
  outperforms state-of-the-art methods in relational video customization. Code
  and models will be made publicly available.
paper_ai_summary: DreamRelation, a novel framework, personalizes complex visual relationships across subjects through relational decoupling and dynamics enhancement, achieving state-of-the-art results in relational video customization.
paper_ai_keywords: Relational video customization Relational Decoupling Learning Relational Dynamics Enhancement relation LoRA triplet hybrid mask training MM-DiT's attention mechanism space-time relational contrastive loss