ComFusion: Personalized Subject Generation in Multiple Specific Scenes From Single Image
<|reference_start|>ComFusion: Personalized Subject Generation in Multiple Specific Scenes From Single Image: Recent advancements in personalizing text-to-image (T2I) diffusion models have shown the capability to generate images based on personalized visual concepts using a limited number of user-provided examples. However, these models often struggle with maintaining high visual fidelity, particularly in manipulating scenes as defined by textual inputs. Addressing this, we introduce ComFusion, a novel approach that leverages pretrained models generating composition of a few user-provided subject images and predefined-text scenes, effectively fusing visual-subject instances with textual-specific scenes, resulting in the generation of high-fidelity instances within diverse scenes. ComFusion integrates a class-scene prior preservation regularization, which leverages composites the subject class and scene-specific knowledge from pretrained models to enhance generation fidelity. Additionally, ComFusion uses coarse generated images, ensuring they align effectively with both the instance image and scene texts. Consequently, ComFusion maintains a delicate balance between capturing the essence of the subject and maintaining scene fidelity.Extensive evaluations of ComFusion against various baselines in T2I personalization have demonstrated its qualitative and quantitative superiority.<|reference_end|>
@article{hong2024comfusion:,
    title={ComFusion: Personalized Subject Generation in Multiple Specific Scenes
  From Single Image},
    author={Yan Hong, Jianfu Zhang},
    journal={arXiv preprint arXiv:2402.11849},
    year={2024},
    archivePrefix={arXiv},
    eprint={2402.11849},
    primaryClass={cs.CV}
}