Multimodal Multilabel Classification by CLIP
<|reference_start|>Multimodal Multilabel Classification by CLIP: Multimodal multilabel classification (MMC) is a challenging task that aims to design a learning algorithm to handle two data sources, the image and text, and learn a comprehensive semantic feature presentation across the modalities. In this task, we review the extensive number of state-of-the-art approaches in MMC and leverage a novel technique that utilises the Contrastive Language-Image Pre-training (CLIP) as the feature extractor and fine-tune the model by exploring different classification heads, fusion methods and loss functions. Finally, our best result achieved more than 90% F_1 score in the public Kaggle competition leaderboard. This paper provides detailed descriptions of novel training methods and quantitative analysis through the experimental results.<|reference_end|>
@article{guo2024multimodal,
    title={Multimodal Multilabel Classification by CLIP},
    author={Yanming Guo},
    journal={arXiv preprint arXiv:2406.16141},
    year={2024},
    archivePrefix={arXiv},
    eprint={2406.16141},
    primaryClass={cs.CV}
}