title: MIVE: New Design and Benchmark for Multi-Instance Video Editing
summary: Recent AI-based video editing has enabled users to edit videos through simple
  text prompts, significantly simplifying the editing process. However, recent
  zero-shot video editing techniques primarily focus on global or single-object
  edits, which can lead to unintended changes in other parts of the video. When
  multiple objects require localized edits, existing methods face challenges,
  such as unfaithful editing, editing leakage, and lack of suitable evaluation
  datasets and metrics. To overcome these limitations, we propose a zero-shot
  Multi-Instance Video Editing
  framework, called MIVE. MIVE is a general-purpose mask-based framework, not
  dedicated to specific objects (e.g., people). MIVE introduces two key modules:
  (i) Disentangled Multi-instance Sampling (DMS) to prevent editing leakage and
  (ii) Instance-centric Probability Redistribution (IPR) to ensure precise
  localization and faithful editing. Additionally, we present our new MIVE
  Dataset featuring diverse video scenarios and introduce the Cross-Instance
  Accuracy (CIA) Score to evaluate editing leakage in multi-instance video
  editing tasks. Our extensive qualitative, quantitative, and user study
  evaluations demonstrate that MIVE significantly outperforms recent
  state-of-the-art methods in terms of editing faithfulness, accuracy, and
  leakage prevention, setting a new benchmark for multi-instance video editing.
  The project page is available at https://kaist-viclab.github.io/mive-site/
paper_authors._id: 67628a706562d8e0d34a6ed6
paper_authors.name: Samuel Teodoro
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-18T12:39:54.778Z
paper_authors._id: 669ffd8bc8cb5058061229cf
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/SIGrFQfuTTh1dMSh5um1P.png
paper_authors.fullname: Samuel Teodoro
paper_authors.type: user
paper_authors.user: TheRota
paper_authors._id: 67628a706562d8e0d34a6ed7
paper_authors.name: Agus Gunawan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-18T12:39:38.581Z
paper_authors._id: 63bbeda2141c7d395c467084
paper_authors.avatarUrl: /avatars/fcaa0e4a95796f2ac7000991f1aa19e5.svg
paper_authors.fullname: Agus Gunawan
paper_authors.type: user
paper_authors.user: agusgun
paper_authors._id: 67628a706562d8e0d34a6ed8
paper_authors.name: Soo Ye Kim
paper_authors._id: 67628a706562d8e0d34a6ed9
paper_authors.name: Jihyong Oh
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-18T12:38:29.484Z
paper_authors._id: 6576b99d58ce19fa1e33eb1d
paper_authors.avatarUrl: /avatars/b533e776aa3d95d722b46ef0cd381acd.svg
paper_authors.fullname: Jihyong Oh
paper_authors.type: user
paper_authors.user: ozbro
paper_authors._id: 67628a706562d8e0d34a6eda
paper_authors.name: Munchurl Kim
paper_title: MIVE: New Design and Benchmark for Multi-Instance Video Editing
paper_summary: Recent AI-based video editing has enabled users to edit videos through simple
  text prompts, significantly simplifying the editing process. However, recent
  zero-shot video editing techniques primarily focus on global or single-object
  edits, which can lead to unintended changes in other parts of the video. When
  multiple objects require localized edits, existing methods face challenges,
  such as unfaithful editing, editing leakage, and lack of suitable evaluation
  datasets and metrics. To overcome these limitations, we propose a zero-shot
  Multi-Instance Video Editing
  framework, called MIVE. MIVE is a general-purpose mask-based framework, not
  dedicated to specific objects (e.g., people). MIVE introduces two key modules:
  (i) Disentangled Multi-instance Sampling (DMS) to prevent editing leakage and
  (ii) Instance-centric Probability Redistribution (IPR) to ensure precise
  localization and faithful editing. Additionally, we present our new MIVE
  Dataset featuring diverse video scenarios and introduce the Cross-Instance
  Accuracy (CIA) Score to evaluate editing leakage in multi-instance video
  editing tasks. Our extensive qualitative, quantitative, and user study
  evaluations demonstrate that MIVE significantly outperforms recent
  state-of-the-art methods in terms of editing faithfulness, accuracy, and
  leakage prevention, setting a new benchmark for multi-instance video editing.
  The project page is available at https://kaist-viclab.github.io/mive-site/
paper_ai_summary: MIVE is a mask-based framework for zero-shot multi-instance video editing that addresses editing leakage and localization issues using Disentangled Multi-instance Sampling and Instance-centric Probability Redistribution, evaluated with a new dataset and CIA score.
paper_ai_keywords: zero-shot mask-based multi-instance editing Disentangled Multi-instance Sampling Instance-centric Probability Redistribution MIVE Dataset Cross-Instance Accuracy editing leakage localization