title: Large Multi-modal Models Can Interpret Features in Large Multi-modal
    Models
summary: Recent advances in Large Multimodal Models (LMMs) lead to significant
  breakthroughs in both academia and industry. One question that arises is how
  we, as humans, can understand their internal neural representations. This paper
  takes an initial step towards addressing this question by presenting a
  versatile framework to identify and interpret the semantics within LMMs.
  Specifically, 1) we first apply a Sparse Autoencoder(SAE) to disentangle the
  representations into human understandable features. 2) We then present an
  automatic interpretation framework to interpreted the open-semantic features
  learned in SAE by the LMMs themselves. We employ this framework to analyze the
  LLaVA-NeXT-8B model using the LLaVA-OV-72B model, demonstrating that these
  features can effectively steer the model's behavior. Our results contribute to
  a deeper understanding of why LMMs excel in specific tasks, including EQ tests,
  and illuminate the nature of their mistakes along with potential strategies for
  their rectification. These findings offer new insights into the internal
  mechanisms of LMMs and suggest parallels with the cognitive processes of the
  human brain.
paper_authors._id: 67444e5070ee90e92d3ba59d
paper_authors.name: Kaichen Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-11-25T20:38:58.758Z
paper_authors._id: 64bb77e786e7fb5b8a317a43
paper_authors.avatarUrl: /avatars/4c1720db323082e644babeb827c6f5d4.svg
paper_authors.fullname: kcz
paper_authors.type: user
paper_authors.user: kcz358
paper_authors._id: 67444e5070ee90e92d3ba59e
paper_authors.name: Yifei Shen
paper_authors._id: 67444e5070ee90e92d3ba59f
paper_authors.name: Bo Li
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-11-25T20:38:56.667Z
paper_authors._id: 62d3f7d84b0933c48f3cdd9c
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/62d3f7d84b0933c48f3cdd9c/rSoSyH0Td9fD9HVlyK7bh.jpeg
paper_authors.fullname: Bo Li
paper_authors.type: user
paper_authors.user: luodian
paper_authors._id: 67444e5070ee90e92d3ba5a0
paper_authors.name: Ziwei Liu
paper_title: Large Multi-modal Models Can Interpret Features in Large Multi-modal
    Models
paper_summary: Recent advances in Large Multimodal Models (LMMs) lead to significant
  breakthroughs in both academia and industry. One question that arises is how
  we, as humans, can understand their internal neural representations. This paper
  takes an initial step towards addressing this question by presenting a
  versatile framework to identify and interpret the semantics within LMMs.
  Specifically, 1) we first apply a Sparse Autoencoder(SAE) to disentangle the
  representations into human understandable features. 2) We then present an
  automatic interpretation framework to interpreted the open-semantic features
  learned in SAE by the LMMs themselves. We employ this framework to analyze the
  LLaVA-NeXT-8B model using the LLaVA-OV-72B model, demonstrating that these
  features can effectively steer the model's behavior. Our results contribute to
  a deeper understanding of why LMMs excel in specific tasks, including EQ tests,
  and illuminate the nature of their mistakes along with potential strategies for
  their rectification. These findings offer new insights into the internal
  mechanisms of LMMs and suggest parallels with the cognitive processes of the
  human brain.
paper_ai_keywords: Large Multimodal Models (LMMs) Sparse Encoder(SAE) human understandable features automatic interpretation framework open-semantic LLaVA-NeXT-8B LLaVA-OV-72B EQ tests internal mechanisms cognitive processes