title: HD-Painter: High-Resolution and Prompt-Faithful Text-Guided Image
    Inpainting with Diffusion Models
summary: Recent progress in text-guided image inpainting, based on the unprecedented
  success of text-to-image diffusion models, has led to exceptionally realistic
  and visually plausible results. However, there is still significant potential
  for improvement in current text-to-image inpainting models, particularly in
  better aligning the inpainted area with user prompts and performing
  high-resolution inpainting. Therefore, in this paper we introduce HD-Painter, a
  completely training-free approach that accurately follows to prompts and
  coherently scales to high-resolution image inpainting. To this end, we design
  the Prompt-Aware Introverted Attention (PAIntA) layer enhancing self-attention
  scores by prompt information and resulting in better text alignment
  generations. To further improve the prompt coherence we introduce the
  Reweighting Attention Score Guidance (RASG) mechanism seamlessly integrating a
  post-hoc sampling strategy into general form of DDIM to prevent
  out-of-distribution latent shifts. Moreover, HD-Painter allows extension to
  larger scales by introducing a specialized super-resolution technique
  customized for inpainting, enabling the completion of missing regions in images
  of up to 2K resolution. Our experiments demonstrate that HD-Painter surpasses
  existing state-of-the-art approaches qualitatively and quantitatively,
  achieving an impressive generation accuracy improvement of 61.4% vs 51.9%. We
  will make the codes publicly available at:
  https://github.com/Picsart-AI-Research/HD-Painter
paper_authors._id: 65850292fde12510fc7f7b5f
paper_authors.name: Hayk Manukyan
paper_authors._id: 65850292fde12510fc7f7b60
paper_authors.name: Andranik Sargsyan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-22T11:10:40.706Z
paper_authors._id: 655e7dbf29e57ae2ebc3b825
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/655e7dbf29e57ae2ebc3b825/wmbHlasxyyfR3G0qbhTPt.png
paper_authors.fullname: Andranik Sargsyan
paper_authors.type: user
paper_authors.user: andraniksargsyan
paper_authors._id: 65850292fde12510fc7f7b61
paper_authors.name: Barsegh Atanyan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-22T11:19:22.467Z
paper_authors._id: 64a55d9f1adc23da7262915c
paper_authors.avatarUrl: /avatars/1ab617563fd90a116217645952e19dcd.svg
paper_authors.fullname: Barsegh Atanyan
paper_authors.type: user
paper_authors.user: Barsegh
paper_authors._id: 65850292fde12510fc7f7b62
paper_authors.name: Zhangyang Wang
paper_authors._id: 65850292fde12510fc7f7b63
paper_authors.name: Shant Navasardyan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-22T11:19:39.013Z
paper_authors._id: 641dadf816ef0e5a7275a5cd
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/641dadf816ef0e5a7275a5cd/YrieHABBK9VH82VxYPJnp.jpeg
paper_authors.fullname: Shant Navasardyan
paper_authors.type: user
paper_authors.user: navasardyanshant
paper_authors._id: 65850292fde12510fc7f7b64
paper_authors.name: Humphrey Shi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-12-22T11:19:46.775Z
paper_authors._id: 61e1188afc27c0f5e3641eb3
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/61e1188afc27c0f5e3641eb3/R4JFlWgYomZFvZ9CLSZfD.jpeg
paper_authors.fullname: Humphrey Shi
paper_authors.type: user
paper_authors.user: Humphrey
paper_title: HD-Painter: High-Resolution and Prompt-Faithful Text-Guided Image
    Inpainting with Diffusion Models
paper_summary: Recent progress in text-guided image inpainting, based on the unprecedented
  success of text-to-image diffusion models, has led to exceptionally realistic
  and visually plausible results. However, there is still significant potential
  for improvement in current text-to-image inpainting models, particularly in
  better aligning the inpainted area with user prompts and performing
  high-resolution inpainting. Therefore, in this paper we introduce HD-Painter, a
  completely training-free approach that accurately follows to prompts and
  coherently scales to high-resolution image inpainting. To this end, we design
  the Prompt-Aware Introverted Attention (PAIntA) layer enhancing self-attention
  scores by prompt information and resulting in better text alignment
  generations. To further improve the prompt coherence we introduce the
  Reweighting Attention Score Guidance (RASG) mechanism seamlessly integrating a
  post-hoc sampling strategy into general form of DDIM to prevent
  out-of-distribution latent shifts. Moreover, HD-Painter allows extension to
  larger scales by introducing a specialized super-resolution technique
  customized for inpainting, enabling the completion of missing regions in images
  of up to 2K resolution. Our experiments demonstrate that HD-Painter surpasses
  existing state-of-the-art approaches qualitatively and quantitatively,
  achieving an impressive generation accuracy improvement of 61.4% vs 51.9%. We
  will make the codes publicly available at:
  https://github.com/Picsart-AI-Research/HD-Painter
paper_ai_keywords: text-guided image inpainting text-to-image diffusion models inpainted area user prompts high-resolution inpainting HD-Painter Prompt-Aware Introverted Attention (PAIntA) layer self-attention scores text alignment Reweighting Attention Score Guidance (RASG) mechanism post-hoc sampling strategy DDIM super-resolution technique latent shifts generation accuracy state-of-the-art approaches