Boosting Open-Domain Continual Learning via Leveraging Intra-domain Category-aware Prototype
<|reference_start|>Boosting Open-Domain Continual Learning via Leveraging Intra-domain Category-aware Prototype: Despite recent progress in enhancing the efficacy of Open-Domain Continual Learning (ODCL) in Vision-Language Models (VLM), failing to (1) correctly identify the Task-ID of a test image and (2) use only the category set corresponding to the Task-ID, while preserving the knowledge related to each domain, cannot address the two primary challenges of ODCL: forgetting old knowledge and maintaining zero-shot capabilities, as well as the confusions caused by category-relatedness between domains. In this paper, we propose a simple yet effective solution: leveraging intra-domain category-aware prototypes for ODCL in CLIP (DPeCLIP), where the prototype is the key to bridging the above two processes. Concretely, we propose a training-free Task-ID discriminator method, by utilizing prototypes as classifiers for identifying Task-IDs. Furthermore, to maintain the knowledge corresponding to each domain, we incorporate intra-domain category-aware prototypes as domain prior prompts into the training process. Extensive experiments conducted on 11 different datasets demonstrate the effectiveness of our approach, achieving 2.37% and 1.14% average improvement in class-incremental and task-incremental settings, respectively.<|reference_end|>
@article{lu2024boosting,
    title={Boosting Open-Domain Continual Learning via Leveraging Intra-domain
  Category-aware Prototype},
    author={Yadong Lu, Shitian Zhao, Boxiang Yun, Dongsheng Jiang, Yin Li, Qingli
  Li, Yan Wang},
    journal={arXiv preprint arXiv:2408.09984},
    year={2024},
    archivePrefix={arXiv},
    eprint={2408.09984},
    primaryClass={cs.CV}
}