title: Can LLMs Maintain Fundamental Abilities under KV Cache Compression?
summary: This paper investigates an under-explored challenge in large language models
  (LLMs): the impact of KV cache compression methods on LLMs' fundamental
  capabilities. While existing methods achieve impressive compression ratios on
  long-context benchmarks, their effects on core model capabilities remain
  understudied. We present a comprehensive empirical study evaluating prominent
  KV cache compression methods across diverse tasks, spanning world knowledge,
  commonsense reasoning, arithmetic reasoning, code generation, safety, and
  long-context understanding and generation.Our analysis reveals that KV cache
  compression methods exhibit task-specific performance degradation. Arithmetic
  reasoning tasks prove particularly sensitive to aggressive compression, with
  different methods showing performance drops of 17.4%-43.3%. Notably, the
  DeepSeek R1 Distill model exhibits more robust compression tolerance compared
  to instruction-tuned models, showing only 9.67%-25.53% performance
  degradation. Based on our analysis of attention patterns and cross-task
  compression performance, we propose ShotKV, a novel compression approach that
  distinctly handles prefill and decoding phases while maintaining shot-level
  semantic coherence. Empirical results show that ShotKV achieves 9%-18%
  performance improvements on long-context generation tasks under aggressive
  compression ratios.
paper_authors._id: 67a2e2a02dd2adbc88755a47
paper_authors.name: Xiang Liu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-05T10:12:48.427Z
paper_authors._id: 63024676056ec3a2a8714b24
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1661093436322-noauth.jpeg
paper_authors.fullname: Xiang Liu
paper_authors.type: user
paper_authors.user: Dominic789654
paper_authors._id: 67a2e2a02dd2adbc88755a48
paper_authors.name: Zhenheng Tang
paper_authors._id: 67a2e2a02dd2adbc88755a49
paper_authors.name: Hong Chen
paper_authors._id: 67a2e2a02dd2adbc88755a4a
paper_authors.name: Peijie Dong
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-02T07:54:02.147Z
paper_authors._id: 6395f845aec00abff778ad31
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6395f845aec00abff778ad31/bZkAlchSvqER1HgBKmcHI.jpeg
paper_authors.fullname: PeijieDong
paper_authors.type: user
paper_authors.user: pprp
paper_authors._id: 67a2e2a02dd2adbc88755a4b
paper_authors.name: Zeyu Li
paper_authors._id: 67a2e2a02dd2adbc88755a4c
paper_authors.name: Xiuze Zhou
paper_authors._id: 67a2e2a02dd2adbc88755a4d
paper_authors.name: Bo Li
paper_authors._id: 67a2e2a02dd2adbc88755a4e
paper_authors.name: Xuming Hu
paper_authors._id: 67a2e2a02dd2adbc88755a4f
paper_authors.name: Xiaowen Chu
paper_title: Can LLMs Maintain Fundamental Abilities under KV Cache Compression?
paper_summary: This paper investigates an under-explored challenge in large language models
  (LLMs): the impact of KV cache compression methods on LLMs' fundamental
  capabilities. While existing methods achieve impressive compression ratios on
  long-context benchmarks, their effects on core model capabilities remain
  understudied. We present a comprehensive empirical study evaluating prominent
  KV cache compression methods across diverse tasks, spanning world knowledge,
  commonsense reasoning, arithmetic reasoning, code generation, safety, and
  long-context understanding and generation.Our analysis reveals that KV cache
  compression methods exhibit task-specific performance degradation. Arithmetic
  reasoning tasks prove particularly sensitive to aggressive compression, with
  different methods showing performance drops of 17.4%-43.3%. Notably, the
  DeepSeek R1 Distill model exhibits more robust compression tolerance compared
  to instruction-tuned models, showing only 9.67%-25.53% performance
  degradation. Based on our analysis of attention patterns and cross-task
  compression performance, we propose ShotKV, a novel compression approach that
  distinctly handles prefill and decoding phases while maintaining shot-level
  semantic coherence. Empirical results show that ShotKV achieves 9%-18%
  performance improvements on long-context generation tasks under aggressive
  compression ratios.
paper_ai_summary: The study evaluates KV cache compression methods in LLMs and introduces ShotKV, which improves performance on long-context tasks under high compression ratios.
paper_ai_keywords: KV cache compression LLMs world knowledge commonsense reasoning arithmetic reasoning code generation safety long-context understanding attention patterns prefill decoding semantic coherence ShotKV