An Efficient Replay for Class-Incremental Learning with Pre-trained Models
<|reference_start|>An Efficient Replay for Class-Incremental Learning with Pre-trained Models: In general class-incremental learning, researchers typically use sample sets as a tool to avoid catastrophic forgetting during continuous learning. At the same time, researchers have also noted the differences between class-incremental learning and Oracle training and have attempted to make corrections. In recent years, researchers have begun to develop class-incremental learning algorithms utilizing pre-trained models, achieving significant results. This paper observes that in class-incremental learning, the steady state among the weight guided by each class center is disrupted, which is significantly correlated with catastrophic forgetting. Based on this, we propose a new method to overcoming forgetting . In some cases, by retaining only a single sample unit of each class in memory for replay and applying simple gradient constraints, very good results can be achieved. Experimental results indicate that under the condition of pre-trained models, our method can achieve competitive performance with very low computational cost and by simply using the cross-entropy loss.<|reference_end|>
@article{yin2024an,
    title={An Efficient Replay for Class-Incremental Learning with Pre-trained
  Models},
    author={Weimin Yin and Bin Chen adn Chunzhao Xie and Zhenhao Tan},
    journal={arXiv preprint arXiv:2408.08084},
    year={2024},
    archivePrefix={arXiv},
    eprint={2408.08084},
    primaryClass={cs.LG cs.AI}
}