DreamRelation: Relation-Centric Video Customization
Relational video customization refers to the creation of personalized videos
that depict user-specified relations between two subjects, a crucial task for
comprehending real-world visual content. While existing methods can personalize
subject appearances and motions, they still struggle with complex relational
video customization, where precise relational modeling and high generalization
across subject categories are essential. The primary challenge arises from the
intricate spatial arrangements, layout variations, and nuanced temporal
dynamics inherent in relations; consequently, current models tend to
overemphasize irrelevant visual details rather than capturing meaningful
interactions. To address these challenges, we propose DreamRelation, a novel
approach that personalizes relations through a small set of exemplar videos,
leveraging two key components: Relational Decoupling Learning and Relational
Dynamics Enhancement. First, in Relational Decoupling Learning, we disentangle
relations from subject appearances using relation LoRA triplet and hybrid mask
training strategy, ensuring better generalization across diverse relationships.
Furthermore, we determine the optimal design of relation LoRA triplet by
analyzing the distinct roles of the query, key, and value features within
MM-DiT's attention mechanism, making DreamRelation the first relational video
generation framework with explainable components. Second, in Relational
Dynamics Enhancement, we introduce space-time relational contrastive loss,
which prioritizes relational dynamics while minimizing the reliance on detailed
subject appearances. Extensive experiments demonstrate that DreamRelation
outperforms state-of-the-art methods in relational video customization. Code
and models will be made publicly available.
67cfb2efb77bc8e7d415f904
Yujie Wei
admin_assigned
2025-03-11T11:17:35.633Z
637f70d6fab5db9101c3dfc8
https://cdn-avatars.huggingface.co/v1/production/uploads/637f70d6fab5db9101c3dfc8/NgkYNXWLDavLbrnCby2Fl.jpeg
Yujie Wei
user
weilllllls
67cfb2efb77bc8e7d415f905
Shiwei Zhang
67cfb2efb77bc8e7d415f906
Hangjie Yuan
claimed_verified
2025-03-11T08:21:32.780Z
649d54b314afbb10ce2a9eeb
/avatars/15c325d8c2273ff63569f23015e98486.svg
Hangjie Yuan
user
JacobYuan
67cfb2efb77bc8e7d415f907
Biao Gong
67cfb2efb77bc8e7d415f908
Longxiang Tang
claimed_verified
2025-03-11T08:21:30.700Z
6492a0d8d4ae24c933ace44d
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/DXIky2sdPwmiCOR9p-JBQ.png
Longxiang Tang
user
lloong
67cfb2efb77bc8e7d415f909
Xiang Wang
67cfb2efb77bc8e7d415f90a
Haonan Qiu
67cfb2efb77bc8e7d415f90b
Hengjia Li
67cfb2efb77bc8e7d415f90c
Shuai Tan
claimed_verified
2025-03-11T10:22:02.278Z
6697b62a27bf71ffe289492f
/avatars/f48095f1e2ca69836294343301be1700.svg
Shuai Tan
user
Shuaishuai0219
67cfb2efb77bc8e7d415f90d
Yingya Zhang
67cfb2efb77bc8e7d415f90e
Hongming Shan
DreamRelation: Relation-Centric Video Customization
Relational video customization refers to the creation of personalized videos
that depict user-specified relations between two subjects, a crucial task for
comprehending real-world visual content. While existing methods can personalize
subject appearances and motions, they still struggle with complex relational
video customization, where precise relational modeling and high generalization
across subject categories are essential. The primary challenge arises from the
intricate spatial arrangements, layout variations, and nuanced temporal
dynamics inherent in relations; consequently, current models tend to
overemphasize irrelevant visual details rather than capturing meaningful
interactions. To address these challenges, we propose DreamRelation, a novel
approach that personalizes relations through a small set of exemplar videos,
leveraging two key components: Relational Decoupling Learning and Relational
Dynamics Enhancement. First, in Relational Decoupling Learning, we disentangle
relations from subject appearances using relation LoRA triplet and hybrid mask
training strategy, ensuring better generalization across diverse relationships.
Furthermore, we determine the optimal design of relation LoRA triplet by
analyzing the distinct roles of the query, key, and value features within
MM-DiT's attention mechanism, making DreamRelation the first relational video
generation framework with explainable components. Second, in Relational
Dynamics Enhancement, we introduce space-time relational contrastive loss,
which prioritizes relational dynamics while minimizing the reliance on detailed
subject appearances. Extensive experiments demonstrate that DreamRelation
outperforms state-of-the-art methods in relational video customization. Code
and models will be made publicly available.
DreamRelation, a novel framework, personalizes complex visual relationships across subjects through relational decoupling and dynamics enhancement, achieving state-of-the-art results in relational video customization.
Relational video customization Relational Decoupling Learning Relational Dynamics Enhancement relation LoRA triplet hybrid mask training MM-DiT's attention mechanism space-time relational contrastive loss