SimO Loss: Anchor-Free Contrastive Loss for Fine-Grained Supervised Contrastive Learning
<|reference_start|>SimO Loss: Anchor-Free Contrastive Loss for Fine-Grained Supervised Contrastive Learning: We introduce a novel anchor-free contrastive learning (AFCL) method leveraging our proposed Similarity-Orthogonality (SimO) loss. Our approach minimizes a semi-metric discriminative loss function that simultaneously optimizes two key objectives: reducing the distance and orthogonality between embeddings of similar inputs while maximizing these metrics for dissimilar inputs, facilitating more fine-grained contrastive learning. The AFCL method, powered by SimO loss, creates a fiber bundle topological structure in the embedding space, forming class-specific, internally cohesive yet orthogonal neighborhoods. We validate the efficacy of our method on the CIFAR-10 dataset, providing visualizations that demonstrate the impact of SimO loss on the embedding space. Our results illustrate the formation of distinct, orthogonal class neighborhoods, showcasing the method's ability to create well-structured embeddings that balance class separation with intra-class variability. This work opens new avenues for understanding and leveraging the geometric properties of learned representations in various machine learning tasks.<|reference_end|>
@article{bouhsine2024simo,
    title={SimO Loss: Anchor-Free Contrastive Loss for Fine-Grained Supervised
  Contrastive Learning},
    author={Taha Bouhsine, Imad El Aaroussi, Atik Faysal, Wang Huaxia},
    journal={arXiv preprint arXiv:2410.05233},
    year={2024},
    archivePrefix={arXiv},
    eprint={2410.05233},
    primaryClass={cs.LG cs.AI cs.CV}
}