OCC-MLLM-Alpha:Empowering Multi-modal Large Language Model for the Understanding of Occluded Objects with Self-Supervised Test-Time Learning
<|reference_start|>OCC-MLLM-Alpha:Empowering Multi-modal Large Language Model for the Understanding of Occluded Objects with Self-Supervised Test-Time Learning: There is a gap in the understanding of occluded objects in existing large-scale visual language multi-modal models. Current state-of-the-art multi-modal models fail to provide satisfactory results in describing occluded objects through universal visual encoders and supervised learning strategies. Therefore, we introduce a multi-modal large language framework and corresponding self-supervised learning strategy with support of 3D generation. We start our experiments comparing with the state-of-the-art models in the evaluation of a large-scale dataset SOMVideo [18]. The initial results demonstrate the improvement of 16.92% in comparison with the state-of-the-art VLM models.<|reference_end|>
@article{yang2024occ-mllm-alpha:empowering,
    title={OCC-MLLM-Alpha:Empowering Multi-modal Large Language Model for the
  Understanding of Occluded Objects with Self-Supervised Test-Time Learning},
    author={Shuxin Yang, Xinhan Di},
    journal={arXiv preprint arXiv:2410.01861},
    year={2024},
    archivePrefix={arXiv},
    eprint={2410.01861},
    primaryClass={cs.CV}
}