title: Rethinking Compression: Reduced Order Modelling of Latent Features in
    Large Language Models
summary: Due to the substantial scale of Large Language Models (LLMs), the direct
  application of conventional compression methodologies proves impractical. The
  computational demands associated with even minimal gradient updates present
  challenges, particularly on consumer-grade hardware. This paper introduces an
  innovative approach for the parametric and practical compression of LLMs based
  on reduced order modelling, which entails low-rank decomposition within the
  feature space and re-parameterization in the weight space. Notably, this
  compression technique operates in a layer-wise manner, obviating the need for a
  GPU device and enabling the compression of billion-scale models within
  stringent constraints of both memory and time. Our method represents a
  significant advancement in model compression by leveraging matrix
  decomposition, demonstrating superior efficacy compared to the prevailing
  state-of-the-art structured pruning method.
paper_authors._id: 65791dea8628ec00e9129ce3
paper_authors.name: Arnav Chavan
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-12-13T08:32:56.799Z
paper_authors._id: 648979dd308f19bc7c42899f
paper_authors.avatarUrl: /avatars/6b7bde1d453f0328fcd5a86cfd9eb3b2.svg
paper_authors.fullname: Arnav Chavan
paper_authors.type: user
paper_authors.user: Arnav0400
paper_authors._id: 65791dea8628ec00e9129ce4
paper_authors.name: Nahush Lele
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-12-13T08:32:52.611Z
paper_authors._id: 65794432821f4394980cd5e3
paper_authors.avatarUrl: /avatars/be5896fe21672949b7ffade6dd06e812.svg
paper_authors.fullname: Nahush Lele
paper_authors.type: user
paper_authors.user: Nahush-26
paper_authors._id: 65791dea8628ec00e9129ce5
paper_authors.name: Deepak Gupta
paper_title: Rethinking Compression: Reduced Order Modelling of Latent Features in
    Large Language Models
paper_summary: Due to the substantial scale of Large Language Models (LLMs), the direct
  application of conventional compression methodologies proves impractical. The
  computational demands associated with even minimal gradient updates present
  challenges, particularly on consumer-grade hardware. This paper introduces an
  innovative approach for the parametric and practical compression of LLMs based
  on reduced order modelling, which entails low-rank decomposition within the
  feature space and re-parameterization in the weight space. Notably, this
  compression technique operates in a layer-wise manner, obviating the need for a
  GPU device and enabling the compression of billion-scale models within
  stringent constraints of both memory and time. Our method represents a
  significant advancement in model compression by leveraging matrix
  decomposition, demonstrating superior efficacy compared to the prevailing
  state-of-the-art structured pruning method.
paper_ai_summary: This paper presents a new compression technique for Large Language Models (LLMs) using reduced order modeling and low-rank decomposition that operates in a layer-wise manner without a GPU, achieving better results than structured pruning.
paper_ai_keywords: Large Language Models (LLMs) reduced order modeling low-rank decomposition weight space layer-wise compression matrix decomposition structured pruning