Large Multi-modal Models Can Interpret Features in Large Multi-modal
  Models
Recent advances in Large Multimodal Models (LMMs) lead to significant
breakthroughs in both academia and industry. One question that arises is how
we, as humans, can understand their internal neural representations. This paper
takes an initial step towards addressing this question by presenting a
versatile framework to identify and interpret the semantics within LMMs.
Specifically, 1) we first apply a Sparse Autoencoder(SAE) to disentangle the
representations into human understandable features. 2) We then present an
automatic interpretation framework to interpreted the open-semantic features
learned in SAE by the LMMs themselves. We employ this framework to analyze the
LLaVA-NeXT-8B model using the LLaVA-OV-72B model, demonstrating that these
features can effectively steer the model's behavior. Our results contribute to
a deeper understanding of why LMMs excel in specific tasks, including EQ tests,
and illuminate the nature of their mistakes along with potential strategies for
their rectification. These findings offer new insights into the internal
mechanisms of LMMs and suggest parallels with the cognitive processes of the
human brain.
67444e5070ee90e92d3ba59d
Kaichen Zhang
claimed_verified
2024-11-25T20:38:58.758Z
64bb77e786e7fb5b8a317a43
https://cdn-avatars.huggingface.co/v1/production/uploads/64bb77e786e7fb5b8a317a43/J0jOrlZJ9gazdYaeSH2Bo.png
kcz
user
kcz358
67444e5070ee90e92d3ba59e
Yifei Shen
67444e5070ee90e92d3ba59f
Bo Li
claimed_verified
2024-11-25T20:38:56.667Z
62d3f7d84b0933c48f3cdd9c
https://cdn-avatars.huggingface.co/v1/production/uploads/62d3f7d84b0933c48f3cdd9c/rSoSyH0Td9fD9HVlyK7bh.jpeg
Bo Li
user
luodian
67444e5070ee90e92d3ba5a0
Ziwei Liu
Large Multi-modal Models Can Interpret Features in Large Multi-modal
  Models
Recent advances in Large Multimodal Models (LMMs) lead to significant
breakthroughs in both academia and industry. One question that arises is how
we, as humans, can understand their internal neural representations. This paper
takes an initial step towards addressing this question by presenting a
versatile framework to identify and interpret the semantics within LMMs.
Specifically, 1) we first apply a Sparse Autoencoder(SAE) to disentangle the
representations into human understandable features. 2) We then present an
automatic interpretation framework to interpreted the open-semantic features
learned in SAE by the LMMs themselves. We employ this framework to analyze the
LLaVA-NeXT-8B model using the LLaVA-OV-72B model, demonstrating that these
features can effectively steer the model's behavior. Our results contribute to
a deeper understanding of why LMMs excel in specific tasks, including EQ tests,
and illuminate the nature of their mistakes along with potential strategies for
their rectification. These findings offer new insights into the internal
mechanisms of LMMs and suggest parallels with the cognitive processes of the
human brain.
A framework using Sparse Autoencoders and automatic interpretation helps understand human-understandable features in Large Multimodal Models (LMMs), providing insights into model behavior and cognitive processes.
Sparse Autoencoder LMMs neural representations LLaVA-NeXT-8B LLaVA-OV-72B EQ tests cognitive processes