title: MAGREF: Masked Guidance for Any-Reference Video Generation
summary: Video generation has made substantial strides with the emergence of deep
  generative models, especially diffusion-based approaches. However, video
  generation based on multiple reference subjects still faces significant
  challenges in maintaining multi-subject consistency and ensuring high
  generation quality. In this paper, we propose MAGREF, a unified framework for
  any-reference video generation that introduces masked guidance to enable
  coherent multi-subject video synthesis conditioned on diverse reference images
  and a textual prompt. Specifically, we propose (1) a region-aware dynamic
  masking mechanism that enables a single model to flexibly handle various
  subject inference, including humans, objects, and backgrounds, without
  architectural changes, and (2) a pixel-wise channel concatenation mechanism
  that operates on the channel dimension to better preserve appearance features.
  Our model delivers state-of-the-art video generation quality, generalizing from
  single-subject training to complex multi-subject scenarios with coherent
  synthesis and precise control over individual subjects, outperforming existing
  open-source and commercial baselines. To facilitate evaluation, we also
  introduce a comprehensive multi-subject video benchmark. Extensive experiments
  demonstrate the effectiveness of our approach, paving the way for scalable,
  controllable, and high-fidelity multi-subject video synthesis. Code and model
  can be found at: https://github.com/MAGREF-Video/MAGREF
paper_authors._id: 683917d0ecf59de6ef345e76
paper_authors.name: Yufan Deng
paper_authors._id: 683917d0ecf59de6ef345e77
paper_authors.name: Xun Guo
paper_authors._id: 683917d0ecf59de6ef345e78
paper_authors.name: Yuanyang Yin
paper_authors._id: 683917d0ecf59de6ef345e79
paper_authors.name: Jacob Zhiyuan Fang
paper_authors._id: 683917d0ecf59de6ef345e7a
paper_authors.name: Yiding Yang
paper_authors._id: 683917d0ecf59de6ef345e7b
paper_authors.name: Yizhi Wang
paper_authors._id: 683917d0ecf59de6ef345e7c
paper_authors.name: Shenghai Yuan
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-30T06:53:49.399Z
paper_authors._id: 63468720dd6d90d82ccf3450
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/63468720dd6d90d82ccf3450/tVBFlmZNz8FRMkOrDaDID.jpeg
paper_authors.fullname: YSH
paper_authors.type: user
paper_authors.user: BestWishYsh
paper_authors._id: 683917d0ecf59de6ef345e7d
paper_authors.name: Angtian Wang
paper_authors._id: 683917d0ecf59de6ef345e7e
paper_authors.name: Bo Liu
paper_authors._id: 683917d0ecf59de6ef345e7f
paper_authors.name: Haibin Huang
paper_authors._id: 683917d0ecf59de6ef345e80
paper_authors.name: Chongyang Ma
paper_title: MAGREF: Masked Guidance for Any-Reference Video Generation
paper_summary: Video generation has made substantial strides with the emergence of deep
  generative models, especially diffusion-based approaches. However, video
  generation based on multiple reference subjects still faces significant
  challenges in maintaining multi-subject consistency and ensuring high
  generation quality. In this paper, we propose MAGREF, a unified framework for
  any-reference video generation that introduces masked guidance to enable
  coherent multi-subject video synthesis conditioned on diverse reference images
  and a textual prompt. Specifically, we propose (1) a region-aware dynamic
  masking mechanism that enables a single model to flexibly handle various
  subject inference, including humans, objects, and backgrounds, without
  architectural changes, and (2) a pixel-wise channel concatenation mechanism
  that operates on the channel dimension to better preserve appearance features.
  Our model delivers state-of-the-art video generation quality, generalizing from
  single-subject training to complex multi-subject scenarios with coherent
  synthesis and precise control over individual subjects, outperforming existing
  open-source and commercial baselines. To facilitate evaluation, we also
  introduce a comprehensive multi-subject video benchmark. Extensive experiments
  demonstrate the effectiveness of our approach, paving the way for scalable,
  controllable, and high-fidelity multi-subject video synthesis. Code and model
  can be found at: https://github.com/MAGREF-Video/MAGREF
paper_ai_summary: MAGREF is a unified framework for video generation that uses masked guidance and dynamic masking for coherent multi-subject synthesis from diverse references and text prompts.
paper_ai_keywords: diffusion-based approaches multi-subject consistency unified framework masked guidance region-aware dynamic masking pixel-wise channel concatenation appearance features multi-subject video benchmark scalable controllable high-fidelity video synthesis