title: DiffuEraser: A Diffusion Model for Video Inpainting
summary: Recent video inpainting algorithms integrate flow-based pixel propagation
  with transformer-based generation to leverage optical flow for restoring
  textures and objects using information from neighboring frames, while
  completing masked regions through visual Transformers. However, these
  approaches often encounter blurring and temporal inconsistencies when dealing
  with large masks, highlighting the need for models with enhanced generative
  capabilities. Recently, diffusion models have emerged as a prominent technique
  in image and video generation due to their impressive performance. In this
  paper, we introduce DiffuEraser, a video inpainting model based on stable
  diffusion, designed to fill masked regions with greater details and more
  coherent structures. We incorporate prior information to provide initialization
  and weak conditioning,which helps mitigate noisy artifacts and suppress
  hallucinations. Additionally, to improve temporal consistency during
  long-sequence inference, we expand the temporal receptive fields of both the
  prior model and DiffuEraser, and further enhance consistency by leveraging the
  temporal smoothing property of Video Diffusion Models. Experimental results
  demonstrate that our proposed method outperforms state-of-the-art techniques in
  both content completeness and temporal consistency while maintaining acceptable
  efficiency.
paper_authors._id: 678e125d09dc6d3a311cc04e
paper_authors.name: Xiaowen Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-24T10:31:02.655Z
paper_authors._id: 6497b4464a3c31df8e4148d8
paper_authors.avatarUrl: /avatars/4397a380468e84bc7945fddd9a6d1066.svg
paper_authors.fullname: Xiaowen Li
paper_authors.type: user
paper_authors.user: asLKHFksasak
paper_authors._id: 678e125d09dc6d3a311cc04f
paper_authors.name: Haolan Xue
paper_authors._id: 678e125d09dc6d3a311cc050
paper_authors.name: Peiran Ren
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-24T10:30:47.443Z
paper_authors._id: 64b74a45f902508f0d786505
paper_authors.avatarUrl: /avatars/8bc5aaa011642827e12524c4f0a56927.svg
paper_authors.fullname: Peiran REN
paper_authors.type: user
paper_authors.user: lyraestar
paper_authors._id: 678e125d09dc6d3a311cc051
paper_authors.name: Liefeng Bo
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-24T10:30:55.550Z
paper_authors._id: 63d0cc736b985b0f25d0412c
paper_authors.avatarUrl: /avatars/3eb8c79f9a7c4c819038ea7b04e323dd.svg
paper_authors.fullname: Bo
paper_authors.type: user
paper_authors.user: Liefeng
paper_title: DiffuEraser: A Diffusion Model for Video Inpainting
paper_summary: Recent video inpainting algorithms integrate flow-based pixel propagation
  with transformer-based generation to leverage optical flow for restoring
  textures and objects using information from neighboring frames, while
  completing masked regions through visual Transformers. However, these
  approaches often encounter blurring and temporal inconsistencies when dealing
  with large masks, highlighting the need for models with enhanced generative
  capabilities. Recently, diffusion models have emerged as a prominent technique
  in image and video generation due to their impressive performance. In this
  paper, we introduce DiffuEraser, a video inpainting model based on stable
  diffusion, designed to fill masked regions with greater details and more
  coherent structures. We incorporate prior information to provide initialization
  and weak conditioning,which helps mitigate noisy artifacts and suppress
  hallucinations. Additionally, to improve temporal consistency during
  long-sequence inference, we expand the temporal receptive fields of both the
  prior model and DiffuEraser, and further enhance consistency by leveraging the
  temporal smoothing property of Video Diffusion Models. Experimental results
  demonstrate that our proposed method outperforms state-of-the-art techniques in
  both content completeness and temporal consistency while maintaining acceptable
  efficiency.
paper_ai_summary: DiffuEraser, a video inpainting model based on stable diffusion with enhanced generative and temporal consistency capabilities, outperforms state-of-the-art techniques.
paper_ai_keywords: flow-based pixel propagation transformer-based generation optical flow diffusion models stable diffusion visual Transformers prior information weak conditioning noisy artifacts hallucinations temporal consistency temporal receptive fields Video Diffusion Models long-sequence inference temporal smoothing property