title: Multimodal Neurons in Pretrained Text-Only Transformers
summary: Language models demonstrate remarkable capacity to generalize representations
  learned in one modality to downstream tasks in other modalities. Can we trace
  this ability to individual neurons? We study the case where a frozen text
  transformer is augmented with vision using a self-supervised visual encoder and
  a single linear projection learned on an image-to-text task. Outputs of the
  projection layer are not immediately decodable into language describing image
  content; instead, we find that translation between modalities occurs deeper
  within the transformer. We introduce a procedure for identifying "multimodal
  neurons" that convert visual representations into corresponding text, and
  decoding the concepts they inject into the model's residual stream. In a series
  of experiments, we show that multimodal neurons operate on specific visual
  concepts across inputs, and have a systematic causal effect on image
  captioning.
paper_authors._id: 64cc5f898a16b1748f21b910
paper_authors.name: Sarah Schwettmann
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-08-04T12:43:18.131Z
paper_authors._id: 648bbafe09f73fe180a5cf38
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/648bbafe09f73fe180a5cf38/BgNBoZequvWtp3ycBxy2M.png
paper_authors.fullname: Sarah Schwettmann
paper_authors.type: user
paper_authors.user: generish
paper_authors._id: 64cc5f898a16b1748f21b911
paper_authors.name: Neil Chowdhury
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-08-07T12:38:49.615Z
paper_authors._id: 63ec754359a46fd5f4b81a2f
paper_authors.avatarUrl: /avatars/aaa4a63b25bd03c366f855427ba9cad4.svg
paper_authors.fullname: Neil Chowdhury
paper_authors.type: user
paper_authors.user: nch0w
paper_authors._id: 64cc5f898a16b1748f21b912
paper_authors.name: Antonio Torralba
paper_title: Multimodal Neurons in Pretrained Text-Only Transformers
paper_summary: Language models demonstrate remarkable capacity to generalize representations
  learned in one modality to downstream tasks in other modalities. Can we trace
  this ability to individual neurons? We study the case where a frozen text
  transformer is augmented with vision using a self-supervised visual encoder and
  a single linear projection learned on an image-to-text task. Outputs of the
  projection layer are not immediately decodable into language describing image
  content; instead, we find that translation between modalities occurs deeper
  within the transformer. We introduce a procedure for identifying "multimodal
  neurons" that convert visual representations into corresponding text, and
  decoding the concepts they inject into the model's residual stream. In a series
  of experiments, we show that multimodal neurons operate on specific visual
  concepts across inputs, and have a systematic causal effect on image
  captioning.
paper_ai_summary: Frozen text transformers can incorporate visual information through self-supervised learning, with specific neurons facilitating translation between modalities and influencing image captioning.
paper_ai_keywords: text transformer self-supervised visual encoder linear projection multimodal neurons residual stream image captioning