HD-Painter: High-Resolution and Prompt-Faithful Text-Guided Image
  Inpainting with Diffusion Models
Recent progress in text-guided image inpainting, based on the unprecedented
success of text-to-image diffusion models, has led to exceptionally realistic
and visually plausible results. However, there is still significant potential
for improvement in current text-to-image inpainting models, particularly in
better aligning the inpainted area with user prompts and performing
high-resolution inpainting. Therefore, in this paper we introduce HD-Painter, a
completely training-free approach that accurately follows to prompts and
coherently scales to high-resolution image inpainting. To this end, we design
the Prompt-Aware Introverted Attention (PAIntA) layer enhancing self-attention
scores by prompt information and resulting in better text alignment
generations. To further improve the prompt coherence we introduce the
Reweighting Attention Score Guidance (RASG) mechanism seamlessly integrating a
post-hoc sampling strategy into general form of DDIM to prevent
out-of-distribution latent shifts. Moreover, HD-Painter allows extension to
larger scales by introducing a specialized super-resolution technique
customized for inpainting, enabling the completion of missing regions in images
of up to 2K resolution. Our experiments demonstrate that HD-Painter surpasses
existing state-of-the-art approaches qualitatively and quantitatively,
achieving an impressive generation accuracy improvement of 61.4% vs 51.9%. We
will make the codes publicly available at:
https://github.com/Picsart-AI-Research/HD-Painter
65850292fde12510fc7f7b5f
Hayk Manukyan
65850292fde12510fc7f7b60
Andranik Sargsyan
admin_assigned
2023-12-22T11:10:40.706Z
655e7dbf29e57ae2ebc3b825
https://cdn-avatars.huggingface.co/v1/production/uploads/655e7dbf29e57ae2ebc3b825/wmbHlasxyyfR3G0qbhTPt.png
Andranik Sargsyan
user
andraniksargsyan
65850292fde12510fc7f7b61
Barsegh Atanyan
admin_assigned
2023-12-22T11:19:22.467Z
64a55d9f1adc23da7262915c
/avatars/1ab617563fd90a116217645952e19dcd.svg
Barsegh Atanyan
user
Barsegh
65850292fde12510fc7f7b62
Zhangyang Wang
65850292fde12510fc7f7b63
Shant Navasardyan
admin_assigned
2023-12-22T11:19:39.013Z
641dadf816ef0e5a7275a5cd
https://cdn-avatars.huggingface.co/v1/production/uploads/641dadf816ef0e5a7275a5cd/YrieHABBK9VH82VxYPJnp.jpeg
Shant Navasardyan
user
navasardyanshant
65850292fde12510fc7f7b64
Humphrey Shi
admin_assigned
2023-12-22T11:19:46.775Z
61e1188afc27c0f5e3641eb3
https://cdn-avatars.huggingface.co/v1/production/uploads/61e1188afc27c0f5e3641eb3/R4JFlWgYomZFvZ9CLSZfD.jpeg
Humphrey Shi
user
Humphrey
HD-Painter: High-Resolution and Prompt-Faithful Text-Guided Image
  Inpainting with Diffusion Models
Recent progress in text-guided image inpainting, based on the unprecedented
success of text-to-image diffusion models, has led to exceptionally realistic
and visually plausible results. However, there is still significant potential
for improvement in current text-to-image inpainting models, particularly in
better aligning the inpainted area with user prompts and performing
high-resolution inpainting. Therefore, in this paper we introduce HD-Painter, a
completely training-free approach that accurately follows to prompts and
coherently scales to high-resolution image inpainting. To this end, we design
the Prompt-Aware Introverted Attention (PAIntA) layer enhancing self-attention
scores by prompt information and resulting in better text alignment
generations. To further improve the prompt coherence we introduce the
Reweighting Attention Score Guidance (RASG) mechanism seamlessly integrating a
post-hoc sampling strategy into general form of DDIM to prevent
out-of-distribution latent shifts. Moreover, HD-Painter allows extension to
larger scales by introducing a specialized super-resolution technique
customized for inpainting, enabling the completion of missing regions in images
of up to 2K resolution. Our experiments demonstrate that HD-Painter surpasses
existing state-of-the-art approaches qualitatively and quantitatively,
achieving an impressive generation accuracy improvement of 61.4% vs 51.9%. We
will make the codes publicly available at:
https://github.com/Picsart-AI-Research/HD-Painter
HD-Painter, a training-free approach, enhances high-resolution text-guided image inpainting through Prompt-Aware Introverted Attention and Reweighting Attention Score Guidance, achieving superior accuracy.
Prompt-Aware Introverted Attention PAIntA Reweighting Attention Score Guidance RASG DDIM super-resolution text-guided image inpainting