title: The Lottery LLM Hypothesis, Rethinking What Abilities Should LLM
    Compression Preserve?
summary: Motivated by reducing the computational and storage costs of LLMs, model
  compression and KV cache compression have attracted much attention from
  researchers. However, current methods predominantly emphasize maintaining the
  performance of compressed LLMs, as measured by perplexity or simple accuracy on
  tasks of common sense knowledge QA and basic arithmetic reasoning. In this
  blog, we present a brief review of recent advancements in LLMs related to
  retrieval-augmented generation, multi-step reasoning, external tools, and
  computational expressivity, all of which substantially enhance LLM performance.
  Then, we propose a lottery LLM hypothesis suggesting that for a given LLM and
  task, there exists a smaller lottery LLM capable of producing the same
  performance as the original LLM with the assistance of multi-step reasoning and
  external tools. Based on the review of current progress in LLMs, we discuss and
  summarize the essential capabilities that the lottery LLM and KV cache
  compression must possess, which are currently overlooked in existing methods.
paper_authors._id: 67beaec94a1d9d7e368a7840
paper_authors.name: Zhenheng Tang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-26T09:15:34.971Z
paper_authors._id: 66a4a319a1711696948b045c
paper_authors.avatarUrl: /avatars/1d92d57a949332cb8227697b9a0c2f39.svg
paper_authors.fullname: Zhenheng Tang
paper_authors.type: user
paper_authors.user: coolzhtang
paper_authors._id: 67beaec94a1d9d7e368a7841
paper_authors.name: Xiang Liu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-27T09:15:20.100Z
paper_authors._id: 63024676056ec3a2a8714b24
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1661093436322-noauth.jpeg
paper_authors.fullname: Xiang Liu
paper_authors.type: user
paper_authors.user: Dominic789654
paper_authors._id: 67beaec94a1d9d7e368a7842
paper_authors.name: Qian Wang
paper_authors._id: 67beaec94a1d9d7e368a7843
paper_authors.name: Peijie Dong
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-02T07:52:55.355Z
paper_authors._id: 6395f845aec00abff778ad31
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6395f845aec00abff778ad31/bZkAlchSvqER1HgBKmcHI.jpeg
paper_authors.fullname: PeijieDong
paper_authors.type: user
paper_authors.user: pprp
paper_authors._id: 67beaec94a1d9d7e368a7844
paper_authors.name: Bingsheng He
paper_authors._id: 67beaec94a1d9d7e368a7845
paper_authors.name: Xiaowen Chu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-26T09:15:41.884Z
paper_authors._id: 6676935fcd0b89a0115174b0
paper_authors.avatarUrl: /avatars/4caca1b672d29e787814f9a30bf20bcc.svg
paper_authors.fullname: Xiaowen Chu
paper_authors.type: user
paper_authors.user: wenxinsiju
paper_authors._id: 67beaec94a1d9d7e368a7846
paper_authors.name: Bo Li
paper_title: The Lottery LLM Hypothesis, Rethinking What Abilities Should LLM
    Compression Preserve?
paper_summary: Motivated by reducing the computational and storage costs of LLMs, model
  compression and KV cache compression have attracted much attention from
  researchers. However, current methods predominantly emphasize maintaining the
  performance of compressed LLMs, as measured by perplexity or simple accuracy on
  tasks of common sense knowledge QA and basic arithmetic reasoning. In this
  blog, we present a brief review of recent advancements in LLMs related to
  retrieval-augmented generation, multi-step reasoning, external tools, and
  computational expressivity, all of which substantially enhance LLM performance.
  Then, we propose a lottery LLM hypothesis suggesting that for a given LLM and
  task, there exists a smaller lottery LLM capable of producing the same
  performance as the original LLM with the assistance of multi-step reasoning and
  external tools. Based on the review of current progress in LLMs, we discuss and
  summarize the essential capabilities that the lottery LLM and KV cache
  compression must possess, which are currently overlooked in existing methods.
paper_ai_summary: A proposal for a lottery LLM hypothesis to reduce the size of LLMs with similar performance using multi-step reasoning and external tools, while identifying essential capabilities that current model and cache compression methods overlook.
paper_ai_keywords: LLM compression KV cache compression retrieval-augmented generation multi-step reasoning external tools computational expressivity lottery LLM hypothesis