Learning Equi-angular Representations for Online Continual Learning
<|reference_start|>Learning Equi-angular Representations for Online Continual Learning: Online continual learning suffers from an underfitted solution due to insufficient training for prompt model update (e.g., single-epoch training). To address the challenge, we propose an efficient online continual learning method using the neural collapse phenomenon. In particular, we induce neural collapse to form a simplex equiangular tight frame (ETF) structure in the representation space so that the continuously learned model with a single epoch can better fit to the streamed data by proposing preparatory data training and residual correction in the representation space. With an extensive set of empirical validations using CIFAR-10/100, TinyImageNet, ImageNet-200, and ImageNet-1K, we show that our proposed method outperforms state-of-the-art methods by a noticeable margin in various online continual learning scenarios such as disjoint and Gaussian scheduled continuous (i.e., boundary-free) data setups.<|reference_end|>
@article{seo2024learning,
    title={Learning Equi-angular Representations for Online Continual Learning},
    author={Minhyuk Seo, Hyunseo Koh, Wonje Jeung, Minjae Lee, San Kim, Hankook
  Lee, Sungjun Cho, Sungik Choi, Hyunwoo Kim, Jonghyun Choi},
    journal={arXiv preprint arXiv:2404.01628},
    year={2024},
    archivePrefix={arXiv},
    eprint={2404.01628},
    primaryClass={cs.CV cs.AI cs.LG}
}