Rethinking the Stability-Plasticity Trade-off in Continual Learning from
  an Architectural Perspective
The quest for Continual Learning (CL) seeks to empower neural networks with
the ability to learn and adapt incrementally. Central to this pursuit is
addressing the stability-plasticity dilemma, which involves striking a balance
between two conflicting objectives: preserving previously learned knowledge and
acquiring new knowledge. While numerous CL methods aim to achieve this
trade-off, they often overlook the impact of network architecture on stability
and plasticity, restricting the trade-off to the parameter level. In this
paper, we delve into the conflict between stability and plasticity at the
architectural level. We reveal that under an equal parameter constraint, deeper
networks exhibit better plasticity, while wider networks are characterized by
superior stability. To address this architectural-level dilemma, we introduce a
novel framework denoted Dual-Arch, which serves as a plug-in component for CL.
This framework leverages the complementary strengths of two distinct and
independent networks: one dedicated to plasticity and the other to stability.
Each network is designed with a specialized and lightweight architecture,
tailored to its respective objective. Extensive experiments demonstrate that
Dual-Arch enhances the performance of existing CL methods while being up to 87%
more compact in terms of parameters.
68415a1cce09e3eca94e02ef
Aojun Lu
claimed_verified
2025-06-05T09:59:50.242Z
6759546743971eff5a12a087
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/esJm_83zW1R6NqWltof8P.png
Aojun Lu
user
Kurt1024
68415a1cce09e3eca94e02f0
Hangjie Yuan
claimed_verified
2025-06-05T09:59:52.907Z
649d54b314afbb10ce2a9eeb
/avatars/15c325d8c2273ff63569f23015e98486.svg
Hangjie Yuan
user
JacobYuan
68415a1cce09e3eca94e02f1
Tao Feng
68415a1cce09e3eca94e02f2
Yanan Sun
Rethinking the Stability-Plasticity Trade-off in Continual Learning from
  an Architectural Perspective
The quest for Continual Learning (CL) seeks to empower neural networks with
the ability to learn and adapt incrementally. Central to this pursuit is
addressing the stability-plasticity dilemma, which involves striking a balance
between two conflicting objectives: preserving previously learned knowledge and
acquiring new knowledge. While numerous CL methods aim to achieve this
trade-off, they often overlook the impact of network architecture on stability
and plasticity, restricting the trade-off to the parameter level. In this
paper, we delve into the conflict between stability and plasticity at the
architectural level. We reveal that under an equal parameter constraint, deeper
networks exhibit better plasticity, while wider networks are characterized by
superior stability. To address this architectural-level dilemma, we introduce a
novel framework denoted Dual-Arch, which serves as a plug-in component for CL.
This framework leverages the complementary strengths of two distinct and
independent networks: one dedicated to plasticity and the other to stability.
Each network is designed with a specialized and lightweight architecture,
tailored to its respective objective. Extensive experiments demonstrate that
Dual-Arch enhances the performance of existing CL methods while being up to 87%
more compact in terms of parameters.
A novel framework, Dual-Arch, enhances Continual Learning by addressing the stability-plasticity dilemma at the architectural level using two specialized networks.
Continual Learning stability-plasticity dilemma deep networks wide networks Dual-Arch