title: Meta-Learning an In-Context Transformer Model of Human Higher Visual
    Cortex
summary: Understanding functional representations within higher visual cortex is a
  fundamental question in computational neuroscience. While artificial neural
  networks pretrained on large-scale datasets exhibit striking representational
  alignment with human neural responses, learning image-computable models of
  visual cortex relies on individual-level, large-scale fMRI datasets. The
  necessity for expensive, time-intensive, and often impractical data acquisition
  limits the generalizability of encoders to new subjects and stimuli. BraInCoRL
  uses in-context learning to predict voxelwise neural responses from few-shot
  examples without any additional finetuning for novel subjects and stimuli. We
  leverage a transformer architecture that can flexibly condition on a variable
  number of in-context image stimuli, learning an inductive bias over multiple
  subjects. During training, we explicitly optimize the model for in-context
  learning. By jointly conditioning on image features and voxel activations, our
  model learns to directly generate better performing voxelwise models of higher
  visual cortex. We demonstrate that BraInCoRL consistently outperforms existing
  voxelwise encoder designs in a low-data regime when evaluated on entirely novel
  images, while also exhibiting strong test-time scaling behavior. The model also
  generalizes to an entirely new visual fMRI dataset, which uses different
  subjects and fMRI data acquisition parameters. Further, BraInCoRL facilitates
  better interpretability of neural signals in higher visual cortex by attending
  to semantically relevant stimuli. Finally, we show that our framework enables
  interpretable mappings from natural language queries to voxel selectivity.
paper_authors._id: 68380974717461677df17514
paper_authors.name: Muquan Yu
paper_authors._id: 68380974717461677df17515
paper_authors.name: Mu Nan
paper_authors._id: 68380974717461677df17516
paper_authors.name: Hossein Adeli
paper_authors._id: 68380974717461677df17517
paper_authors.name: Jacob S. Prince
paper_authors._id: 68380974717461677df17518
paper_authors.name: John A. Pyles
paper_authors._id: 68380974717461677df17519
paper_authors.name: Leila Wehbe
paper_authors._id: 68380974717461677df1751a
paper_authors.name: Margaret M. Henderson
paper_authors._id: 68380974717461677df1751b
paper_authors.name: Michael J. Tarr
paper_authors._id: 68380974717461677df1751c
paper_authors.name: Andrew F. Luo
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2025-05-29T07:16:44.495Z
paper_authors._id: 64b6ce23dbbd1f2cdb624d56
paper_authors.avatarUrl: /avatars/022738ce8f76fa9545b3e363d7264b53.svg
paper_authors.fullname: Andrew Luo
paper_authors.type: user
paper_authors.user: aluo-x
paper_title: Meta-Learning an In-Context Transformer Model of Human Higher Visual
    Cortex
paper_summary: Understanding functional representations within higher visual cortex is a
  fundamental question in computational neuroscience. While artificial neural
  networks pretrained on large-scale datasets exhibit striking representational
  alignment with human neural responses, learning image-computable models of
  visual cortex relies on individual-level, large-scale fMRI datasets. The
  necessity for expensive, time-intensive, and often impractical data acquisition
  limits the generalizability of encoders to new subjects and stimuli. BraInCoRL
  uses in-context learning to predict voxelwise neural responses from few-shot
  examples without any additional finetuning for novel subjects and stimuli. We
  leverage a transformer architecture that can flexibly condition on a variable
  number of in-context image stimuli, learning an inductive bias over multiple
  subjects. During training, we explicitly optimize the model for in-context
  learning. By jointly conditioning on image features and voxel activations, our
  model learns to directly generate better performing voxelwise models of higher
  visual cortex. We demonstrate that BraInCoRL consistently outperforms existing
  voxelwise encoder designs in a low-data regime when evaluated on entirely novel
  images, while also exhibiting strong test-time scaling behavior. The model also
  generalizes to an entirely new visual fMRI dataset, which uses different
  subjects and fMRI data acquisition parameters. Further, BraInCoRL facilitates
  better interpretability of neural signals in higher visual cortex by attending
  to semantically relevant stimuli. Finally, we show that our framework enables
  interpretable mappings from natural language queries to voxel selectivity.
paper_ai_summary: BraInCoRL employs a transformer-based in-context learning approach to model higher visual cortex neural responses with few-shot examples, demonstrating superior performance and generalizability across new subjects, stimuli, and datasets.
paper_ai_keywords: functional representations higher visual cortex artificial neural networks fMRI datasets in-context learning transformer architecture inductive bias voxelwise neural responses image features voxel activations test-time scaling interpretability natural language queries voxel selectivity