title: Online Continual Learning Without the Storage Constraint
summary: Online continual learning (OCL) research has primarily focused on mitigating
  catastrophic forgetting with fixed and limited storage allocation throughout
  the agent's lifetime. However, the growing affordability of data storage
  highlights a broad range of applications that do not adhere to these
  assumptions. In these cases, the primary concern lies in managing computational
  expenditures rather than storage. In this paper, we target such settings,
  investigating the online continual learning problem by relaxing storage
  constraints and emphasizing fixed, limited economical budget. We provide a
  simple algorithm that can compactly store and utilize the entirety of the
  incoming data stream under tiny computational budgets using a kNN classifier
  and universal pre-trained feature extractors. Our algorithm provides a
  consistency property attractive to continual learning: It will never forget
  past seen data. We set a new state of the art on two large-scale OCL datasets:
  Continual LOCalization (CLOC), which has 39M images over 712 classes, and
  Continual Google Landmarks V2 (CGLM), which has 580K images over 10,788 classes
  -- beating methods under far higher computational budgets than ours in terms of
  both reducing catastrophic forgetting of past data and quickly adapting to
  rapidly changing data streams. We provide code to reproduce our results at
  https://github.com/drimpossible/ACM.
paper_authors._id: 64643f5aa5af935cfe7347a2
paper_authors.name: Ameya Prabhu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-05-17T11:43:16.348Z
paper_authors._id: 6464a0d41683d3c81f51924a
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6464a0d41683d3c81f51924a/s7yYVwfUB4WOhVFJS6A6T.jpeg
paper_authors.fullname: Ameya Prabhu
paper_authors.type: user
paper_authors.user: AmeyaPrabhu
paper_authors._id: 64643f5aa5af935cfe7347a3
paper_authors.name: Zhipeng Cai
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-05-19T10:28:12.390Z
paper_authors._id: 6419fe794577db917f6d17fc
paper_authors.avatarUrl: /avatars/ed10371a0082c5e02d32b76217200207.svg
paper_authors.fullname: zhipeng cai
paper_authors.type: user
paper_authors.user: zcai
paper_authors._id: 64643f5aa5af935cfe7347a4
paper_authors.name: Puneet Dokania
paper_authors._id: 64643f5aa5af935cfe7347a5
paper_authors.name: Philip Torr
paper_authors._id: 64643f5aa5af935cfe7347a6
paper_authors.name: Vladlen Koltun
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-10T11:00:10.702Z
paper_authors._id: 6707b1a9356f027ca986d6b1
paper_authors.avatarUrl: /avatars/d32eae81b27dc4f7090196d410e96876.svg
paper_authors.fullname: Vladlen Koltun
paper_authors.type: user
paper_authors.user: vkoltun
paper_authors._id: 64643f5aa5af935cfe7347a7
paper_authors.name: Ozan Sener
paper_title: Online Continual Learning Without the Storage Constraint
paper_summary: Online continual learning (OCL) research has primarily focused on mitigating
  catastrophic forgetting with fixed and limited storage allocation throughout
  the agent's lifetime. However, the growing affordability of data storage
  highlights a broad range of applications that do not adhere to these
  assumptions. In these cases, the primary concern lies in managing computational
  expenditures rather than storage. In this paper, we target such settings,
  investigating the online continual learning problem by relaxing storage
  constraints and emphasizing fixed, limited economical budget. We provide a
  simple algorithm that can compactly store and utilize the entirety of the
  incoming data stream under tiny computational budgets using a kNN classifier
  and universal pre-trained feature extractors. Our algorithm provides a
  consistency property attractive to continual learning: It will never forget
  past seen data. We set a new state of the art on two large-scale OCL datasets:
  Continual LOCalization (CLOC), which has 39M images over 712 classes, and
  Continual Google Landmarks V2 (CGLM), which has 580K images over 10,788 classes
  -- beating methods under far higher computational budgets than ours in terms of
  both reducing catastrophic forgetting of past data and quickly adapting to
  rapidly changing data streams. We provide code to reproduce our results at
  https://github.com/drimpossible/ACM.
paper_ai_keywords: continual learning (CL) online continual learning (OCL) catastrophic forgetting k-Nearest Neighbors (kNN) k-Nearest Neighbor (kNN) classifier universal pre-trained feature extractors Continual LOCalization (CLOC) Continual Google Landmarks V2 (CGLM)