Online Continual Learning Without the Storage Constraint
Online continual learning (OCL) research has primarily focused on mitigating
catastrophic forgetting with fixed and limited storage allocation throughout
the agent's lifetime. However, the growing affordability of data storage
highlights a broad range of applications that do not adhere to these
assumptions. In these cases, the primary concern lies in managing computational
expenditures rather than storage. In this paper, we target such settings,
investigating the online continual learning problem by relaxing storage
constraints and emphasizing fixed, limited economical budget. We provide a
simple algorithm that can compactly store and utilize the entirety of the
incoming data stream under tiny computational budgets using a kNN classifier
and universal pre-trained feature extractors. Our algorithm provides a
consistency property attractive to continual learning: It will never forget
past seen data. We set a new state of the art on two large-scale OCL datasets:
Continual LOCalization (CLOC), which has 39M images over 712 classes, and
Continual Google Landmarks V2 (CGLM), which has 580K images over 10,788 classes
-- beating methods under far higher computational budgets than ours in terms of
both reducing catastrophic forgetting of past data and quickly adapting to
rapidly changing data streams. We provide code to reproduce our results at
https://github.com/drimpossible/ACM.
64643f5aa5af935cfe7347a2
Ameya Prabhu
claimed_verified
2023-05-17T11:43:16.348Z
6464a0d41683d3c81f51924a
https://cdn-avatars.huggingface.co/v1/production/uploads/6464a0d41683d3c81f51924a/s7yYVwfUB4WOhVFJS6A6T.jpeg
Ameya Prabhu
user
AmeyaPrabhu
64643f5aa5af935cfe7347a3
Zhipeng Cai
claimed_verified
2023-05-19T10:28:12.390Z
6419fe794577db917f6d17fc
/avatars/ed10371a0082c5e02d32b76217200207.svg
zhipeng cai
user
zcai
64643f5aa5af935cfe7347a4
Puneet Dokania
64643f5aa5af935cfe7347a5
Philip Torr
64643f5aa5af935cfe7347a6
Vladlen Koltun
claimed_verified
2024-10-10T11:00:10.702Z
6707b1a9356f027ca986d6b1
/avatars/d32eae81b27dc4f7090196d410e96876.svg
Vladlen Koltun
user
vkoltun
64643f5aa5af935cfe7347a7
Ozan Sener
Online Continual Learning Without the Storage Constraint
Online continual learning (OCL) research has primarily focused on mitigating
catastrophic forgetting with fixed and limited storage allocation throughout
the agent's lifetime. However, the growing affordability of data storage
highlights a broad range of applications that do not adhere to these
assumptions. In these cases, the primary concern lies in managing computational
expenditures rather than storage. In this paper, we target such settings,
investigating the online continual learning problem by relaxing storage
constraints and emphasizing fixed, limited economical budget. We provide a
simple algorithm that can compactly store and utilize the entirety of the
incoming data stream under tiny computational budgets using a kNN classifier
and universal pre-trained feature extractors. Our algorithm provides a
consistency property attractive to continual learning: It will never forget
past seen data. We set a new state of the art on two large-scale OCL datasets:
Continual LOCalization (CLOC), which has 39M images over 712 classes, and
Continual Google Landmarks V2 (CGLM), which has 580K images over 10,788 classes
-- beating methods under far higher computational budgets than ours in terms of
both reducing catastrophic forgetting of past data and quickly adapting to
rapidly changing data streams. We provide code to reproduce our results at
https://github.com/drimpossible/ACM.
Research addresses online continual learning with focus on computational budgets, using a kNN classifier and pre-trained feature extractors to manage large datasets efficiently.
kNN classifier universal pre-trained feature extractors online continual learning catastrophic forgetting Continual LOCalization Continual Google Landmarks V2