title: Efficient LLaMA-3.2-Vision by Trimming Cross-attended Visual Features
summary: Visual token reduction lowers inference costs caused by extensive image
  features in large vision-language models (LVLMs). Unlike relevant studies that
  prune tokens in self-attention-only LVLMs, our work uniquely addresses
  cross-attention-based models, which achieve superior performance. We identify
  that the key-value (KV) cache size for image tokens in cross-attention layers
  significantly exceeds that of text tokens in self-attention layers, posing a
  major compute bottleneck. To mitigate this issue, we exploit the sparse nature
  in cross-attention maps to selectively prune redundant visual features. Our
  Trimmed Llama effectively reduces KV cache demands without requiring additional
  training. By benefiting from 50%-reduced visual features, our model can reduce
  inference latency and memory usage while achieving benchmark parity.
paper_authors._id: 67eca9a501a4d1c29e4e70f1
paper_authors.name: Jewon Lee
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-02T12:22:32.687Z
paper_authors._id: 642d395784bf892b8fa91816
paper_authors.avatarUrl: /avatars/2db2202641ad7ed9aeaea7e8bfc3acca.svg
paper_authors.fullname: Jewon Lee
paper_authors.type: user
paper_authors.user: je1lee
paper_authors._id: 67eca9a501a4d1c29e4e70f2
paper_authors.name: Ki-Ung Song
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-02T12:22:38.520Z
paper_authors._id: 645f6407b5a2706bce578eed
paper_authors.avatarUrl: /avatars/8f576ebd8b7c89578e083ccabf6ebc33.svg
paper_authors.fullname: Ki-Ung song
paper_authors.type: user
paper_authors.user: sk851
paper_authors._id: 67eca9a501a4d1c29e4e70f3
paper_authors.name: Seungmin Yang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-02T12:22:46.031Z
paper_authors._id: 679cedca09f47bf5c1ca251f
paper_authors.avatarUrl: /avatars/aa8d6cc5f2993849b7f7c16d2cb1c8f6.svg
paper_authors.fullname: Seungmin Yang
paper_authors.type: user
paper_authors.user: ainoobchobo
paper_authors._id: 67eca9a501a4d1c29e4e70f4
paper_authors.name: Donguk Lim
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-02T12:22:55.363Z
paper_authors._id: 62e7773c870b23dac1b944e2
paper_authors.avatarUrl: /avatars/919ad4150b0b0aa58bc49651b4911c53.svg
paper_authors.fullname: Lim
paper_authors.type: user
paper_authors.user: DongUK
paper_authors._id: 67eca9a501a4d1c29e4e70f5
paper_authors.name: Jaeyeon Kim
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-02T12:23:09.888Z
paper_authors._id: 6548c84e10d47c1f771f183a
paper_authors.avatarUrl: /avatars/37d759063d870e2fe0df926efebff750.svg
paper_authors.fullname: Jaeyeon Kim
paper_authors.type: user
paper_authors.user: jaeyeonkim99
paper_authors._id: 67eca9a501a4d1c29e4e70f6
paper_authors.name: Wooksu Shin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-02T12:23:19.454Z
paper_authors._id: 67a0a0d48f047b67c3172e9e
paper_authors.avatarUrl: /avatars/fa2ee986071f3642c8c7cd6ff693dbfd.svg
paper_authors.fullname: Wooksu Shin
paper_authors.type: user
paper_authors.user: dnrtn1101
paper_authors._id: 67eca9a501a4d1c29e4e70f7
paper_authors.name: Bo-Kyeong Kim
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-02T12:23:24.647Z
paper_authors._id: 64374f10cd93f4c9a34c1239
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64374f10cd93f4c9a34c1239/tdkdcgl18rKXLvyTEhfnG.jpeg
paper_authors.fullname: Bo-Kyeong Kim
paper_authors.type: user
paper_authors.user: bokyeong1015
paper_authors._id: 67eca9a501a4d1c29e4e70f8
paper_authors.name: Yong Jae Lee
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-02T12:23:30.906Z
paper_authors._id: 649f41ee70a478f8b36b2984
paper_authors.avatarUrl: /avatars/1c9a76717a450ac4aeb62a1e823d2e4a.svg
paper_authors.fullname: Yong Jae Lee
paper_authors.type: user
paper_authors.user: yjlee0222
paper_authors._id: 67eca9a501a4d1c29e4e70f9
paper_authors.name: Tae-Ho Kim
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-04-02T12:23:44.160Z
paper_authors._id: 67a701f26b44b5a72af20c9e
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/67a701f26b44b5a72af20c9e/vriqmqgRG3f1mMIjj3tce.png
paper_authors.fullname: Tae-ho Kim
paper_authors.type: user
paper_authors.user: sabioguru
paper_title: Efficient LLaMA-3.2-Vision by Trimming Cross-attended Visual Features
paper_summary: Visual token reduction lowers inference costs caused by extensive image
  features in large vision-language models (LVLMs). Unlike relevant studies that
  prune tokens in self-attention-only LVLMs, our work uniquely addresses
  cross-attention-based models, which achieve superior performance. We identify
  that the key-value (KV) cache size for image tokens in cross-attention layers
  significantly exceeds that of text tokens in self-attention layers, posing a
  major compute bottleneck. To mitigate this issue, we exploit the sparse nature
  in cross-attention maps to selectively prune redundant visual features. Our
  Trimmed Llama effectively reduces KV cache demands without requiring additional
  training. By benefiting from 50%-reduced visual features, our model can reduce
  inference latency and memory usage while achieving benchmark parity.
paper_ai_summary: Visual token reduction in cross-attention-based models improves inference efficiency by pruning redundant features without additional training, maintaining benchmark performance.
paper_ai_keywords: key-value cache cross-attention visual features inference latency memory usage benchmark parity Trimmed Llama