title: Adapt before Continual Learning
summary: Continual Learning (CL) seeks to enable neural networks to incrementally
  acquire new knowledge (plasticity) while retaining existing knowledge
  (stability). While pre-trained models (PTMs) have become pivotal in CL,
  prevailing approaches freeze the PTM backbone to preserve stability, limiting
  their plasticity, particularly when encountering significant domain gaps in
  incremental tasks. Conversely, sequentially finetuning the entire PTM risks
  catastrophic forgetting of generalizable knowledge, exposing a critical
  stability-plasticity trade-off. To address this challenge, we propose Adapting
  PTMs before the core CL process (ACL), a novel framework that refines the PTM
  backbone through a plug-and-play adaptation phase before learning each new task
  with existing CL approaches (e.g., prompt tuning). ACL enhances plasticity by
  aligning embeddings with their original class prototypes while distancing them
  from others, theoretically and empirically shown to balance stability and
  plasticity. Extensive experiments demonstrate that ACL significantly improves
  CL performance across benchmarks and integrated methods, offering a versatile
  solution for PTM-based CL.
paper_authors._id: 6841396eee27975702b57e87
paper_authors.name: Aojun Lu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-05T07:26:29.156Z
paper_authors._id: 6759546743971eff5a12a087
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/esJm_83zW1R6NqWltof8P.png
paper_authors.fullname: Aojun Lu
paper_authors.type: user
paper_authors.user: Kurt1024
paper_authors._id: 6841396eee27975702b57e88
paper_authors.name: Tao Feng
paper_authors._id: 6841396eee27975702b57e89
paper_authors.name: Hangjie Yuan
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-05T07:26:31.411Z
paper_authors._id: 649d54b314afbb10ce2a9eeb
paper_authors.avatarUrl: /avatars/15c325d8c2273ff63569f23015e98486.svg
paper_authors.fullname: Hangjie Yuan
paper_authors.type: user
paper_authors.user: JacobYuan
paper_authors._id: 6841396eee27975702b57e8a
paper_authors.name: Chunhui Ding
paper_authors._id: 6841396eee27975702b57e8b
paper_authors.name: Yanan Sun
paper_title: Adapt before Continual Learning
paper_summary: Continual Learning (CL) seeks to enable neural networks to incrementally
  acquire new knowledge (plasticity) while retaining existing knowledge
  (stability). While pre-trained models (PTMs) have become pivotal in CL,
  prevailing approaches freeze the PTM backbone to preserve stability, limiting
  their plasticity, particularly when encountering significant domain gaps in
  incremental tasks. Conversely, sequentially finetuning the entire PTM risks
  catastrophic forgetting of generalizable knowledge, exposing a critical
  stability-plasticity trade-off. To address this challenge, we propose Adapting
  PTMs before the core CL process (ACL), a novel framework that refines the PTM
  backbone through a plug-and-play adaptation phase before learning each new task
  with existing CL approaches (e.g., prompt tuning). ACL enhances plasticity by
  aligning embeddings with their original class prototypes while distancing them
  from others, theoretically and empirically shown to balance stability and
  plasticity. Extensive experiments demonstrate that ACL significantly improves
  CL performance across benchmarks and integrated methods, offering a versatile
  solution for PTM-based CL.
paper_ai_summary: Adapting Pre-trained Models before the core CL process (ACL) improves Continual Learning by enhancing plasticity while maintaining stability.
paper_ai_keywords: Continual Learning CL Pre-trained models PTMs plasticity stability domain gaps catastrophic forgetting prompt tuning embeddings class prototypes