Guided Latent Slot Diffusion for Object-Centric Learning
<|reference_start|>Guided Latent Slot Diffusion for Object-Centric Learning: Slot attention aims to decompose an input image into a set of meaningful object files (slots). These latent object representations enable various downstream tasks. Yet, these slots often bind to object parts, not objects themselves, especially for real-world datasets. To address this, we introduce Guided Latent Slot Diffusion - GLASS, an object-centric model that uses generated captions as a guiding signal to better align slots with objects. Our key insight is to learn the slot-attention module in the space of generated images. This allows us to repurpose the pre-trained diffusion decoder model, which reconstructs the images from the slots, as a semantic mask generator based on the generated captions. GLASS learns an object-level representation suitable for multiple tasks simultaneously, e.g., segmentation, image generation, and property prediction, outperforming previous methods. For object discovery, GLASS achieves approx. a +35% and +10% relative improvement for mIoU over the previous state-of-the-art (SOTA) method on the VOC and COCO datasets, respectively, and establishes a new SOTA FID score for conditional image generation amongst slot-attention-based methods. For the segmentation task, GLASS surpasses SOTA weakly-supervised and language-based segmentation models, which were specifically designed for the task.<|reference_end|>
@article{singh2024guided,
    title={Guided Latent Slot Diffusion for Object-Centric Learning},
    author={Krishnakant Singh, Simone Schaub-Meyer, Stefan Roth},
    journal={arXiv preprint arXiv:2407.17929},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.17929},
    primaryClass={cs.CV cs.LG}
}