PartGLEE: A Foundation Model for Recognizing and Parsing Any Objects
<|reference_start|>PartGLEE: A Foundation Model for Recognizing and Parsing Any Objects: We present PartGLEE, a part-level foundation model for locating and identifying both objects and parts in images. Through a unified framework, PartGLEE accomplishes detection, segmentation, and grounding of instances at any granularity in the open world scenario. Specifically, we propose a Q-Former to construct the hierarchical relationship between objects and parts, parsing every object into corresponding semantic parts. By incorporating a large amount of object-level data, the hierarchical relationships can be extended, enabling PartGLEE to recognize a rich variety of parts. We conduct comprehensive studies to validate the effectiveness of our method, PartGLEE achieves the state-of-the-art performance across various part-level tasks and obtain competitive results on object-level tasks. The proposed PartGLEE significantly enhances hierarchical modeling capabilities and part-level perception over our previous GLEE model. Further analysis indicates that the hierarchical cognitive ability of PartGLEE is able to facilitate a detailed comprehension in images for mLLMs. The model and code will be released at https://provencestar.github.io/PartGLEE-Vision/ .<|reference_end|>
@article{li2024partglee:,
    title={PartGLEE: A Foundation Model for Recognizing and Parsing Any Objects},
    author={Junyi Li and Junfeng Wu and Weizhi Zhao and Song Bai and Xiang Bai},
    journal={arXiv preprint arXiv:2407.16696},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.16696},
    primaryClass={cs.CV}
}