IIDM: Image-to-Image Diffusion Model for Semantic Image Synthesis
<|reference_start|>IIDM: Image-to-Image Diffusion Model for Semantic Image Synthesis: Semantic image synthesis aims to generate high-quality images given semantic conditions, i.e. segmentation masks and style reference images. Existing methods widely adopt generative adversarial networks (GANs). GANs take all conditional inputs and directly synthesize images in a single forward step. In this paper, semantic image synthesis is treated as an image denoising task and is handled with a novel image-to-image diffusion model (IIDM). Specifically, the style reference is first contaminated with random noise and then progressively denoised by IIDM, guided by segmentation masks. Moreover, three techniques, refinement, color-transfer and model ensembles, are proposed to further boost the generation quality. They are plug-in inference modules and do not require additional training. Extensive experiments show that our IIDM outperforms existing state-of-the-art methods by clear margins. Further analysis is provided via detailed demonstrations. We have implemented IIDM based on the Jittor framework; code is available at https://github.com/ader47/jittor-jieke-semantic_images_synthesis.<|reference_end|>
@article{liu2024iidm:,
    title={IIDM: Image-to-Image Diffusion Model for Semantic Image Synthesis},
    author={Feng Liu, Xiaobin Chang},
    journal={arXiv preprint arXiv:2403.13378},
    year={2024},
    archivePrefix={arXiv},
    eprint={2403.13378},
    primaryClass={cs.CV}
}