title: Context-Aware Meta-Learning
summary: Large Language Models like ChatGPT demonstrate a remarkable capacity to learn
  new concepts during inference without any fine-tuning. However, visual models
  trained to detect new objects during inference have been unable to replicate
  this ability, and instead either perform poorly or require meta-training and/or
  fine-tuning on similar objects. In this work, we propose a meta-learning
  algorithm that emulates Large Language Models by learning new visual concepts
  during inference without fine-tuning. Our approach leverages a frozen
  pre-trained feature extractor, and analogous to in-context learning, recasts
  meta-learning as sequence modeling over datapoints with known labels and a test
  datapoint with an unknown label. On 8 out of 11 meta-learning benchmarks, our
  approach -- without meta-training or fine-tuning -- exceeds or matches the
  state-of-the-art algorithm, P>M>F, which is meta-trained on these benchmarks.
paper_authors._id: 652f4c88262ab88f123e6757
paper_authors.name: Christopher Fifty
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-10-19T08:10:23.862Z
paper_authors._id: 65304af3d107f378e1f5a869
paper_authors.avatarUrl: /avatars/fc222f5aee84d09df7906a15388cfc49.svg
paper_authors.fullname: Chris Fifty
paper_authors.type: user
paper_authors.user: cfifty
paper_authors._id: 652f4c88262ab88f123e6758
paper_authors.name: Dennis Duan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-10-18T08:36:12.234Z
paper_authors._id: 6445ddc07a7b94ddc2d6e73a
paper_authors.avatarUrl: /avatars/a293e9396fec0bd34da44e3816ae6d5c.svg
paper_authors.fullname: Dennis Duan
paper_authors.type: user
paper_authors.user: dduan97
paper_authors._id: 652f4c88262ab88f123e6759
paper_authors.name: Ronald G. Junkins
paper_authors._id: 652f4c88262ab88f123e675a
paper_authors.name: Ehsan Amid
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-10-19T08:10:26.656Z
paper_authors._id: 6530385d800ac851b9b06bac
paper_authors.avatarUrl: /avatars/b93a59e821deca7c7ca63989f15e98ac.svg
paper_authors.fullname: Ehsan Amid
paper_authors.type: user
paper_authors.user: ehsanamid
paper_authors._id: 652f4c88262ab88f123e675b
paper_authors.name: Jure Leskovec
paper_authors._id: 652f4c88262ab88f123e675c
paper_authors.name: Christopher Ré
paper_authors._id: 652f4c88262ab88f123e675d
paper_authors.name: Sebastian Thrun
paper_title: Context-Aware Meta-Learning
paper_summary: Large Language Models like ChatGPT demonstrate a remarkable capacity to learn
  new concepts during inference without any fine-tuning. However, visual models
  trained to detect new objects during inference have been unable to replicate
  this ability, and instead either perform poorly or require meta-training and/or
  fine-tuning on similar objects. In this work, we propose a meta-learning
  algorithm that emulates Large Language Models by learning new visual concepts
  during inference without fine-tuning. Our approach leverages a frozen
  pre-trained feature extractor, and analogous to in-context learning, recasts
  meta-learning as sequence modeling over datapoints with known labels and a test
  datapoint with an unknown label. On 8 out of 11 meta-learning benchmarks, our
  approach -- without meta-training or fine-tuning -- exceeds or matches the
  state-of-the-art algorithm, P>M>F, which is meta-trained on these benchmarks.
paper_ai_summary: A meta-learning algorithm learns new visual concepts during inference without fine-tuning, leveraging a pre-trained feature extractor and sequence modeling.
paper_ai_keywords: meta-learning Large Language Models pre-trained feature extractor in-context learning sequence modeling meta-training fine-tuning meta-learning benchmarks