title: A Multimodal Automated Interpretability Agent
summary: This paper describes MAIA, a Multimodal Automated Interpretability Agent.
  MAIA is a system that uses neural models to automate neural model understanding
  tasks like feature interpretation and failure mode discovery. It equips a
  pre-trained vision-language model with a set of tools that support iterative
  experimentation on subcomponents of other models to explain their behavior.
  These include tools commonly used by human interpretability researchers: for
  synthesizing and editing inputs, computing maximally activating exemplars from
  real-world datasets, and summarizing and describing experimental results.
  Interpretability experiments proposed by MAIA compose these tools to describe
  and explain system behavior. We evaluate applications of MAIA to computer
  vision models. We first characterize MAIA's ability to describe (neuron-level)
  features in learned representations of images. Across several trained models
  and a novel dataset of synthetic vision neurons with paired ground-truth
  descriptions, MAIA produces descriptions comparable to those generated by
  expert human experimenters. We then show that MAIA can aid in two additional
  interpretability tasks: reducing sensitivity to spurious features, and
  automatically identifying inputs likely to be mis-classified.
paper_authors._id: 66271ebbc25c3b1ac48eec87
paper_authors.name: Tamar Rott Shaham
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-04-23T08:13:59.030Z
paper_authors._id: 63914afeb6b839bb6143a6be
paper_authors.avatarUrl: /avatars/da25b555d105f4755c8187479469ca77.svg
paper_authors.fullname: Tamar Rott Shaham
paper_authors.type: user
paper_authors.user: tamarott
paper_authors._id: 66271ebbc25c3b1ac48eec88
paper_authors.name: Sarah Schwettmann
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-04-23T07:13:52.276Z
paper_authors._id: 648bbafe09f73fe180a5cf38
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/648bbafe09f73fe180a5cf38/BgNBoZequvWtp3ycBxy2M.png
paper_authors.fullname: Sarah Schwettmann
paper_authors.type: user
paper_authors.user: generish
paper_authors._id: 66271ebbc25c3b1ac48eec89
paper_authors.name: Franklin Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-04-23T08:16:02.993Z
paper_authors._id: 6537ec2f33b10aa38f3f6469
paper_authors.avatarUrl: /avatars/f25b2dfb2dc4f03b245761dccf3df2d7.svg
paper_authors.fullname: Franklin Wang
paper_authors.type: user
paper_authors.user: fxwang
paper_authors._id: 66271ebbc25c3b1ac48eec8a
paper_authors.name: Achyuta Rajaram
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-04-23T08:16:09.787Z
paper_authors._id: 649ee2b51f44b2c0301d6be7
paper_authors.avatarUrl: /avatars/b449f2b7b4545a034ce6f90cc2f5020d.svg
paper_authors.fullname: Achyuta Rajaram
paper_authors.type: user
paper_authors.user: flybamboo
paper_authors._id: 66271ebbc25c3b1ac48eec8b
paper_authors.name: Evan Hernandez
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-04-23T08:16:26.196Z
paper_authors._id: 644467a679e7797ce71d1980
paper_authors.avatarUrl: /avatars/a0946ddb7f15cbbea123cacde76eb1aa.svg
paper_authors.fullname: Evan Hernandez
paper_authors.type: user
paper_authors.user: evanhernandez
paper_authors._id: 66271ebbc25c3b1ac48eec8c
paper_authors.name: Jacob Andreas
paper_authors._id: 66271ebbc25c3b1ac48eec8d
paper_authors.name: Antonio Torralba
paper_title: A Multimodal Automated Interpretability Agent
paper_summary: This paper describes MAIA, a Multimodal Automated Interpretability Agent.
  MAIA is a system that uses neural models to automate neural model understanding
  tasks like feature interpretation and failure mode discovery. It equips a
  pre-trained vision-language model with a set of tools that support iterative
  experimentation on subcomponents of other models to explain their behavior.
  These include tools commonly used by human interpretability researchers: for
  synthesizing and editing inputs, computing maximally activating exemplars from
  real-world datasets, and summarizing and describing experimental results.
  Interpretability experiments proposed by MAIA compose these tools to describe
  and explain system behavior. We evaluate applications of MAIA to computer
  vision models. We first characterize MAIA's ability to describe (neuron-level)
  features in learned representations of images. Across several trained models
  and a novel dataset of synthetic vision neurons with paired ground-truth
  descriptions, MAIA produces descriptions comparable to those generated by
  expert human experimenters. We then show that MAIA can aid in two additional
  interpretability tasks: reducing sensitivity to spurious features, and
  automatically identifying inputs likely to be mis-classified.
paper_ai_summary: MAIA, a multimodal automated interpretability agent, uses neural models to perform feature interpretation and failure mode discovery for other models, demonstrating comparable results to human experimenters and aiding in reducing sensitivity to spurious features and identifying potential misclassifications.
paper_ai_keywords: neural models feature interpretation failure mode discovery vision-language model neuron-level features learned representations ground-truth descriptions spurious features mis-classified inputs