MIGC++: Advanced Multi-Instance Generation Controller for Image Synthesis
<|reference_start|>MIGC++: Advanced Multi-Instance Generation Controller for Image Synthesis: We introduce the Multi-Instance Generation (MIG) task, which focuses on generating multiple instances within a single image, each accurately placed at predefined positions with attributes such as category, color, and shape, strictly following user specifications. MIG faces three main challenges: avoiding attribute leakage between instances, supporting diverse instance descriptions, and maintaining consistency in iterative generation. To address attribute leakage, we propose the Multi-Instance Generation Controller (MIGC). MIGC generates multiple instances through a divide-and-conquer strategy, breaking down multi-instance shading into single-instance tasks with singular attributes, later integrated. To provide more types of instance descriptions, we developed MIGC++. MIGC++ allows attribute control through text \& images and position control through boxes \& masks. Lastly, we introduced the Consistent-MIG algorithm to enhance the iterative MIG ability of MIGC and MIGC++. This algorithm ensures consistency in unmodified regions during the addition, deletion, or modification of instances, and preserves the identity of instances when their attributes are changed. We introduce the COCO-MIG and Multimodal-MIG benchmarks to evaluate these methods. Extensive experiments on these benchmarks, along with the COCO-Position benchmark and DrawBench, demonstrate that our methods substantially outperform existing techniques, maintaining precise control over aspects including position, attribute, and quantity. Project page: https://github.com/limuloo/MIGC.<|reference_end|>
@article{zhou2024migc++:,
    title={MIGC++: Advanced Multi-Instance Generation Controller for Image
  Synthesis},
    author={Dewei Zhou, You Li, Fan Ma, Zongxin Yang, Yi Yang},
    journal={arXiv preprint arXiv:2407.02329},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.02329},
    primaryClass={cs.CV}
}