title: Intriguing Properties of Large Language and Vision Models
summary: Recently, large language and vision models (LLVMs) have received significant
  attention and development efforts due to their remarkable generalization
  performance across a wide range of tasks requiring perception and cognitive
  abilities. A key factor behind their success is their simple architecture,
  which consists of a vision encoder, a projector, and a large language model
  (LLM). Despite their achievements in advanced reasoning tasks, their
  performance on fundamental perception-related tasks (e.g., MMVP) remains
  surprisingly low. This discrepancy raises the question of how LLVMs truly
  perceive images and exploit the advantages of the vision encoder. To address
  this, we systematically investigate this question regarding several aspects:
  permutation invariance, robustness, math reasoning, alignment preserving and
  importance, by evaluating the most common LLVM's families (i.e., LLaVA) across
  10 evaluation benchmarks. Our extensive experiments reveal several intriguing
  properties of current LLVMs: (1) they internally process the image in a global
  manner, even when the order of visual patch sequences is randomly permuted; (2)
  they are sometimes able to solve math problems without fully perceiving
  detailed numerical information; (3) the cross-modal alignment is overfitted to
  complex reasoning tasks, thereby, causing them to lose some of the original
  perceptual capabilities of their vision encoder; (4) the representation space
  in the lower layers (<25%) plays a crucial role in determining performance and
  enhancing visual understanding. Lastly, based on the above observations, we
  suggest potential future directions for building better LLVMs and constructing
  more challenging evaluation benchmarks.
paper_authors._id: 6705df47f6f1dfb1734d6ee4
paper_authors.name: Young-Jun Lee
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-09T07:37:29.247Z
paper_authors._id: 6434b6619bd5a84b5dcfa4de
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6434b6619bd5a84b5dcfa4de/h8Q6kPNjFNc03wmdboHzq.jpeg
paper_authors.fullname: Young-Jun Lee
paper_authors.type: user
paper_authors.user: passing2961
paper_authors._id: 6705df47f6f1dfb1734d6ee5
paper_authors.name: Byungsoo Ko
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-11T10:34:44.659Z
paper_authors._id: 60ff7a87aa025227d344b735
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1627359121375-60ff7a87aa025227d344b735.jpeg
paper_authors.fullname: Byungsoo Ko
paper_authors.type: user
paper_authors.user: kobiso
paper_authors._id: 6705df47f6f1dfb1734d6ee6
paper_authors.name: Han-Gyu Kim
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2024-10-09T01:41:28.430Z
paper_authors._id: 668644c2c22e4833a634f90f
paper_authors.avatarUrl: /avatars/dec3f3e8737379567e6cc0d1d27d5ec8.svg
paper_authors.fullname: Han-Gyu Kim
paper_authors.type: user
paper_authors.user: mkmiracle
paper_authors._id: 6705df47f6f1dfb1734d6ee7
paper_authors.name: Yechan Hwang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-14T07:54:02.769Z
paper_authors._id: 653032b83ecbe51d6a6f0498
paper_authors.avatarUrl: /avatars/3d435a420bf7024560ed5c5aa2e204d6.svg
paper_authors.fullname: Yechan Hwang
paper_authors.type: user
paper_authors.user: YYXYmint
paper_authors._id: 6705df47f6f1dfb1734d6ee8
paper_authors.name: Ho-Jin Choi
paper_title: Intriguing Properties of Large Language and Vision Models
paper_summary: Recently, large language and vision models (LLVMs) have received significant
  attention and development efforts due to their remarkable generalization
  performance across a wide range of tasks requiring perception and cognitive
  abilities. A key factor behind their success is their simple architecture,
  which consists of a vision encoder, a projector, and a large language model
  (LLM). Despite their achievements in advanced reasoning tasks, their
  performance on fundamental perception-related tasks (e.g., MMVP) remains
  surprisingly low. This discrepancy raises the question of how LLVMs truly
  perceive images and exploit the advantages of the vision encoder. To address
  this, we systematically investigate this question regarding several aspects:
  permutation invariance, robustness, math reasoning, alignment preserving and
  importance, by evaluating the most common LLVM's families (i.e., LLaVA) across
  10 evaluation benchmarks. Our extensive experiments reveal several intriguing
  properties of current LLVMs: (1) they internally process the image in a global
  manner, even when the order of visual patch sequences is randomly permuted; (2)
  they are sometimes able to solve math problems without fully perceiving
  detailed numerical information; (3) the cross-modal alignment is overfitted to
  complex reasoning tasks, thereby, causing them to lose some of the original
  perceptual capabilities of their vision encoder; (4) the representation space
  in the lower layers (<25%) plays a crucial role in determining performance and
  enhancing visual understanding. Lastly, based on the above observations, we
  suggest potential future directions for building better LLVMs and constructing
  more challenging evaluation benchmarks.
paper_ai_keywords: large language and vision models (LLVMs) vision encoder large language model (LLM) MMVP permutation invariance robustness math reasoning alignment preserving and importance evaluation benchmarks global manner visual patch sequences cross-modal alignment representation space