title: VideoGrain: Modulating Space-Time Attention for Multi-grained Video
    Editing
summary: Recent advancements in diffusion models have significantly improved video
  generation and editing capabilities. However, multi-grained video editing,
  which encompasses class-level, instance-level, and part-level modifications,
  remains a formidable challenge. The major difficulties in multi-grained editing
  include semantic misalignment of text-to-region control and feature coupling
  within the diffusion model. To address these difficulties, we present
  VideoGrain, a zero-shot approach that modulates space-time (cross- and self-)
  attention mechanisms to achieve fine-grained control over video content. We
  enhance text-to-region control by amplifying each local prompt's attention to
  its corresponding spatial-disentangled region while minimizing interactions
  with irrelevant areas in cross-attention. Additionally, we improve feature
  separation by increasing intra-region awareness and reducing inter-region
  interference in self-attention. Extensive experiments demonstrate our method
  achieves state-of-the-art performance in real-world scenarios. Our code, data,
  and demos are available at https://knightyxp.github.io/VideoGrain_project_page/
paper_authors._id: 67bd515c0417e7f92283d3b8
paper_authors.name: Xiangpeng Yang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-14T08:35:35.071Z
paper_authors._id: 6486df66373f79a52913e017
paper_authors.avatarUrl: /avatars/4741683fbbcec3a615d0a8df62bc6fec.svg
paper_authors.fullname: Xiangpeng Yang
paper_authors.type: user
paper_authors.user: XiangpengYang
paper_authors._id: 67bd515c0417e7f92283d3b9
paper_authors.name: Linchao Zhu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-25T15:16:07.044Z
paper_authors._id: 63521e1dfe367c0d9b155007
paper_authors.avatarUrl: /avatars/b22804fc63b507fd60191486b17cdf7c.svg
paper_authors.fullname: Linchao Zhu
paper_authors.type: user
paper_authors.user: ffmpbgrnn
paper_authors._id: 67bd515c0417e7f92283d3ba
paper_authors.name: Hehe Fan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-25T15:16:12.884Z
paper_authors._id: 64ad04020fb9b20dbabbd30e
paper_authors.avatarUrl: /avatars/a6bae4a3a4bcd6b54c33860fe14c7923.svg
paper_authors.fullname: Hehe Fan
paper_authors.type: user
paper_authors.user: hehefan
paper_authors._id: 67bd515c0417e7f92283d3bb
paper_authors.name: Yi Yang
paper_title: VideoGrain: Modulating Space-Time Attention for Multi-grained Video
    Editing
paper_summary: Recent advancements in diffusion models have significantly improved video
  generation and editing capabilities. However, multi-grained video editing,
  which encompasses class-level, instance-level, and part-level modifications,
  remains a formidable challenge. The major difficulties in multi-grained editing
  include semantic misalignment of text-to-region control and feature coupling
  within the diffusion model. To address these difficulties, we present
  VideoGrain, a zero-shot approach that modulates space-time (cross- and self-)
  attention mechanisms to achieve fine-grained control over video content. We
  enhance text-to-region control by amplifying each local prompt's attention to
  its corresponding spatial-disentangled region while minimizing interactions
  with irrelevant areas in cross-attention. Additionally, we improve feature
  separation by increasing intra-region awareness and reducing inter-region
  interference in self-attention. Extensive experiments demonstrate our method
  achieves state-of-the-art performance in real-world scenarios. Our code, data,
  and demos are available at https://knightyxp.github.io/VideoGrain_project_page/
paper_ai_keywords: diffusion models multi-grained video editing semantic misalignment text-to-region control feature coupling zero-shot approach space-time attention mechanisms cross-attention self-attention spatial-disentangled region intra-region awareness inter-region interference