title: LLMZip: Lossless Text Compression using Large Language Models
summary: We provide new estimates of an asymptotic upper bound on the entropy of
  English using the large language model LLaMA-7B as a predictor for the next
  token given a window of past tokens. This estimate is significantly smaller
  than currently available estimates in cover1978convergent,
  lutati2023focus. A natural byproduct is an algorithm for lossless
  compression of English text which combines the prediction from the large
  language model with a lossless compression scheme. Preliminary results from
  limited experiments suggest that our scheme outperforms state-of-the-art text
  compression schemes such as BSC, ZPAQ, and paq8h.
paper_authors._id: 648146dd856901b0edbd9824
paper_authors.name: Chandra Shekhara Kaushik Valmeekam
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-06-09T15:32:25.325Z
paper_authors._id: 639533db00e782e0c0c95ab3
paper_authors.avatarUrl: /avatars/b7d4e2dbf6261d3442cd9646102c0f5d.svg
paper_authors.fullname: Kaushik Valmeekam
paper_authors.type: user
paper_authors.user: vcskaushik9
paper_authors._id: 648146dd856901b0edbd9825
paper_authors.name: Krishna Narayanan
paper_authors._id: 648146dd856901b0edbd9826
paper_authors.name: Dileep Kalathil
paper_authors._id: 648146dd856901b0edbd9827
paper_authors.name: Jean-Francois Chamberland
paper_authors._id: 648146dd856901b0edbd9828
paper_authors.name: Srinivas Shakkottai
paper_title: LLMZip: Lossless Text Compression using Large Language Models
paper_summary: We provide new estimates of an asymptotic upper bound on the entropy of
  English using the large language model LLaMA-7B as a predictor for the next
  token given a window of past tokens. This estimate is significantly smaller
  than currently available estimates in cover1978convergent,
  lutati2023focus. A natural byproduct is an algorithm for lossless
  compression of English text which combines the prediction from the large
  language model with a lossless compression scheme. Preliminary results from
  limited experiments suggest that our scheme outperforms state-of-the-art text
  compression schemes such as BSC, ZPAQ, and paq8h.
paper_ai_keywords: large language model LLaMA-7B next token past tokens entropy asymptotic upper bound lossless compression BSC ZPAQ paq8h