  Image manipulation under the guidance of textual descriptions has recently
received a broad range of attention. In this study, we focus on the regional
editing of images with the guidance of given text prompts. Different from
current mask-based image editing methods, we propose a novel region-aware
diffusion model (RDM) for entity-level image editing, which could automatically
locate the region of interest and replace it following given text prompts. To
strike a balance between image fidelity and inference speed, we design the
intensive diffusion pipeline by combing latent space diffusion and enhanced
directional guidance. In addition, to preserve image content in non-edited
regions, we introduce regional-aware entity editing to modify the region of
interest and preserve the out-of-interest region. We validate the proposed RDM
beyond the baseline methods through extensive qualitative and quantitative
experiments. The results show that RDM outperforms the previous approaches in
terms of visual quality, overall harmonization, non-editing region content
preservation, and text-image semantic consistency. The codes are available at
https://github.com/haha-lisa/RDM-Region-Aware-Diffusion-Model.

Computer Vision and Pattern Recognition;Graphics;Multimedia