title: PixelMan: Consistent Object Editing with Diffusion Models via Pixel
    Manipulation and Generation
summary: Recent research explores the potential of Diffusion Models (DMs) for
  consistent object editing, which aims to modify object position, size, and
  composition, etc., while preserving the consistency of objects and background
  without changing their texture and attributes. Current inference-time methods
  often rely on DDIM inversion, which inherently compromises efficiency and the
  achievable consistency of edited images. Recent methods also utilize energy
  guidance which iteratively updates the predicted noise and can drive the
  latents away from the original image, resulting in distortions. In this paper,
  we propose PixelMan, an inversion-free and training-free method for achieving
  consistent object editing via Pixel Manipulation and generation, where we
  directly create a duplicate copy of the source object at target location in the
  pixel space, and introduce an efficient sampling approach to iteratively
  harmonize the manipulated object into the target location and inpaint its
  original location, while ensuring image consistency by anchoring the edited
  image to be generated to the pixel-manipulated image as well as by introducing
  various consistency-preserving optimization techniques during inference.
  Experimental evaluations based on benchmark datasets as well as extensive
  visual comparisons show that in as few as 16 inference steps, PixelMan
  outperforms a range of state-of-the-art training-based and training-free
  methods (usually requiring 50 steps) on multiple consistent object editing
  tasks.
paper_authors._id: 6765152de07adde9c961fabb
paper_authors.name: Liyao Jiang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-20T08:32:26.476Z
paper_authors._id: 64ac49ccb7d86b40fd60a8dd
paper_authors.avatarUrl: /avatars/e9f5482cffdd1d5917523a496a3805f0.svg
paper_authors.fullname: Liyao Jiang
paper_authors.type: user
paper_authors.user: LiyaoJiang
paper_authors._id: 6765152de07adde9c961fabc
paper_authors.name: Negar Hassanpour
paper_authors._id: 6765152de07adde9c961fabd
paper_authors.name: Mohammad Salameh
paper_authors._id: 6765152de07adde9c961fabe
paper_authors.name: Mohammadreza Samadi
paper_authors._id: 6765152de07adde9c961fabf
paper_authors.name: Jiao He
paper_authors._id: 6765152de07adde9c961fac0
paper_authors.name: Fengyu Sun
paper_authors._id: 6765152de07adde9c961fac1
paper_authors.name: Di Niu
paper_title: PixelMan: Consistent Object Editing with Diffusion Models via Pixel
    Manipulation and Generation
paper_summary: Recent research explores the potential of Diffusion Models (DMs) for
  consistent object editing, which aims to modify object position, size, and
  composition, etc., while preserving the consistency of objects and background
  without changing their texture and attributes. Current inference-time methods
  often rely on DDIM inversion, which inherently compromises efficiency and the
  achievable consistency of edited images. Recent methods also utilize energy
  guidance which iteratively updates the predicted noise and can drive the
  latents away from the original image, resulting in distortions. In this paper,
  we propose PixelMan, an inversion-free and training-free method for achieving
  consistent object editing via Pixel Manipulation and generation, where we
  directly create a duplicate copy of the source object at target location in the
  pixel space, and introduce an efficient sampling approach to iteratively
  harmonize the manipulated object into the target location and inpaint its
  original location, while ensuring image consistency by anchoring the edited
  image to be generated to the pixel-manipulated image as well as by introducing
  various consistency-preserving optimization techniques during inference.
  Experimental evaluations based on benchmark datasets as well as extensive
  visual comparisons show that in as few as 16 inference steps, PixelMan
  outperforms a range of state-of-the-art training-based and training-free
  methods (usually requiring 50 steps) on multiple consistent object editing
  tasks.
paper_ai_keywords: Diffusion Models (DMs) consistent object editing DDIM inversion energy guidance image consistency PixelMan Pixel Manipulation pixel space sampling approach harmonize inpaint consistency-preserving optimization techniques