Understanding Information Storage and Transfer in Multi-modal Large Language Models
<|reference_start|>Understanding Information Storage and Transfer in Multi-modal Large Language Models: Understanding the mechanisms of information storage and transfer in Transformer-based models is important for driving model understanding progress. Recent work has studied these mechanisms for Large Language Models (LLMs), revealing insights on how information is stored in a model's parameters and how information flows to and from these parameters in response to specific prompts. However, these studies have not yet been extended to Multi-modal Large Language Models (MLLMs). Given their expanding capabilities and real-world use, we start by studying one aspect of these models -- how MLLMs process information in a factual visual question answering task. We use a constraint-based formulation which views a visual question as having a set of visual or textual constraints that the model's generated answer must satisfy to be correct (e.g. What movie directed by the director in this photo has won a Golden Globe?). Under this setting, we contribute i) a method that extends causal information tracing from pure language to the multi-modal setting, and ii) VQA-Constraints, a test-bed of 9.7K visual questions annotated with constraints. We use these tools to study two open-source MLLMs, LLaVa and multi-modal Phi-2. Our key findings show that these MLLMs rely on MLP and self-attention blocks in much earlier layers for information storage, compared to LLMs whose mid-layer MLPs are more important. We also show that a consistent small subset of visual tokens output by the vision encoder are responsible for transferring information from the image to these causal blocks. We validate these mechanisms by introducing MultEdit, a model-editing algorithm that can correct errors and insert new long-tailed information into MLLMs by targeting these causal blocks.<|reference_end|>
@article{basu2024understanding,
    title={Understanding Information Storage and Transfer in Multi-modal Large
  Language Models},
    author={Samyadeep Basu, Martin Grayson, Cecily Morrison, Besmira Nushi, Soheil
  Feizi, Daniela Massiceti},
    journal={arXiv preprint arXiv:2406.04236},
    year={2024},
    archivePrefix={arXiv},
    eprint={2406.04236},
    primaryClass={cs.CV}
}