title: Rethinking the Stability-Plasticity Trade-off in Continual Learning from
    an Architectural Perspective
summary: The quest for Continual Learning (CL) seeks to empower neural networks with
  the ability to learn and adapt incrementally. Central to this pursuit is
  addressing the stability-plasticity dilemma, which involves striking a balance
  between two conflicting objectives: preserving previously learned knowledge and
  acquiring new knowledge. While numerous CL methods aim to achieve this
  trade-off, they often overlook the impact of network architecture on stability
  and plasticity, restricting the trade-off to the parameter level. In this
  paper, we delve into the conflict between stability and plasticity at the
  architectural level. We reveal that under an equal parameter constraint, deeper
  networks exhibit better plasticity, while wider networks are characterized by
  superior stability. To address this architectural-level dilemma, we introduce a
  novel framework denoted Dual-Arch, which serves as a plug-in component for CL.
  This framework leverages the complementary strengths of two distinct and
  independent networks: one dedicated to plasticity and the other to stability.
  Each network is designed with a specialized and lightweight architecture,
  tailored to its respective objective. Extensive experiments demonstrate that
  Dual-Arch enhances the performance of existing CL methods while being up to 87%
  more compact in terms of parameters.
paper_authors._id: 68415a1cce09e3eca94e02ef
paper_authors.name: Aojun Lu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-05T09:59:50.242Z
paper_authors._id: 6759546743971eff5a12a087
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/esJm_83zW1R6NqWltof8P.png
paper_authors.fullname: Aojun Lu
paper_authors.type: user
paper_authors.user: Kurt1024
paper_authors._id: 68415a1cce09e3eca94e02f0
paper_authors.name: Hangjie Yuan
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-05T09:59:52.907Z
paper_authors._id: 649d54b314afbb10ce2a9eeb
paper_authors.avatarUrl: /avatars/15c325d8c2273ff63569f23015e98486.svg
paper_authors.fullname: Hangjie Yuan
paper_authors.type: user
paper_authors.user: JacobYuan
paper_authors._id: 68415a1cce09e3eca94e02f1
paper_authors.name: Tao Feng
paper_authors._id: 68415a1cce09e3eca94e02f2
paper_authors.name: Yanan Sun
paper_title: Rethinking the Stability-Plasticity Trade-off in Continual Learning from
    an Architectural Perspective
paper_summary: The quest for Continual Learning (CL) seeks to empower neural networks with
  the ability to learn and adapt incrementally. Central to this pursuit is
  addressing the stability-plasticity dilemma, which involves striking a balance
  between two conflicting objectives: preserving previously learned knowledge and
  acquiring new knowledge. While numerous CL methods aim to achieve this
  trade-off, they often overlook the impact of network architecture on stability
  and plasticity, restricting the trade-off to the parameter level. In this
  paper, we delve into the conflict between stability and plasticity at the
  architectural level. We reveal that under an equal parameter constraint, deeper
  networks exhibit better plasticity, while wider networks are characterized by
  superior stability. To address this architectural-level dilemma, we introduce a
  novel framework denoted Dual-Arch, which serves as a plug-in component for CL.
  This framework leverages the complementary strengths of two distinct and
  independent networks: one dedicated to plasticity and the other to stability.
  Each network is designed with a specialized and lightweight architecture,
  tailored to its respective objective. Extensive experiments demonstrate that
  Dual-Arch enhances the performance of existing CL methods while being up to 87%
  more compact in terms of parameters.
paper_ai_summary: A novel framework, Dual-Arch, enhances Continual Learning by addressing the stability-plasticity dilemma at the architectural level using two specialized networks.
paper_ai_keywords: Continual Learning stability-plasticity dilemma deep networks wide networks Dual-Arch