Enhance Image-to-Image Generation with LLaVA-generated Prompts
<|reference_start|>Enhance Image-to-Image Generation with LLaVA-generated Prompts: This paper presents a novel approach to enhance image-to-image generation by leveraging the multimodal capabilities of the Large Language and Vision Assistant (LLaVA). We propose a framework where LLaVA analyzes input images and generates textual descriptions, hereinafter LLaVA-generated prompts. These prompts, along with the original image, are fed into the image-to-image generation pipeline. This enriched representation guides the generation process towards outputs that exhibit a stronger resemblance to the input image. Extensive experiments demonstrate the effectiveness of LLaVA-generated prompts in promoting image similarity. We observe a significant improvement in the visual coherence between the generated and input images compared to traditional methods. Future work will explore fine-tuning LLaVA prompts for increased control over the creative process. By providing more specific details within the prompts, we aim to achieve a delicate balance between faithfulness to the original image and artistic expression in the generated outputs.<|reference_end|>
@article{ding2024enhance,
    title={Enhance Image-to-Image Generation with LLaVA-generated Prompts},
    author={Zhicheng Ding, Panfeng Li, Qikai Yang, Siyang Li},
    journal={Proceedings of the 2024 5th International Conference on
  Information Science, Parallel and Distributed Systems (ISPDS), 2024, pp.
  77-81},
    year={2024},
    doi={10.1109/ISPDS62779.2024.10667513},
    archivePrefix={arXiv},
    eprint={2406.01956},
    primaryClass={cs.CV}
}