title: CMC-Bench: Towards a New Paradigm of Visual Signal Compression
summary: Ultra-low bitrate image compression is a challenging and demanding topic.
  With the development of Large Multimodal Models (LMMs), a Cross Modality
  Compression (CMC) paradigm of Image-Text-Image has emerged. Compared with
  traditional codecs, this semantic-level compression can reduce image data size
  to 0.1\% or even lower, which has strong potential applications. However, CMC
  has certain defects in consistency with the original image and perceptual
  quality. To address this problem, we introduce CMC-Bench, a benchmark of the
  cooperative performance of Image-to-Text (I2T) and Text-to-Image (T2I) models
  for image compression. This benchmark covers 18,000 and 40,000 images
  respectively to verify 6 mainstream I2T and 12 T2I models, including 160,000
  subjective preference scores annotated by human experts. At ultra-low bitrates,
  this paper proves that the combination of some I2T and T2I models has surpassed
  the most advanced visual signal codecs; meanwhile, it highlights where LMMs can
  be further optimized toward the compression task. We encourage LMM developers
  to participate in this test to promote the evolution of visual signal codec
  protocols.
paper_authors._id: 666c7348e0b2d43ff35e7f58
paper_authors.name: Chunyi Li
paper_authors._id: 666c7348e0b2d43ff35e7f59
paper_authors.name: Xiele Wu
paper_authors._id: 666c7348e0b2d43ff35e7f5a
paper_authors.name: Haoning Wu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-15T09:13:38.596Z
paper_authors._id: 632c7a0d1d303f5f9acf01b8
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/632c7a0d1d303f5f9acf01b8/T010IFuCp6UaOeIyWhbCk.jpeg
paper_authors.fullname: Haoning Wu
paper_authors.type: user
paper_authors.user: haoningwu
paper_authors._id: 666c7348e0b2d43ff35e7f5b
paper_authors.name: Donghui Feng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-15T09:13:45.052Z
paper_authors._id: 6498a0c309cea55085307172
paper_authors.avatarUrl: /avatars/32c4085ff511a4803f06b1247cebc1cd.svg
paper_authors.fullname: Donghui Feng
paper_authors.type: user
paper_authors.user: crazysteven
paper_authors._id: 666c7348e0b2d43ff35e7f5c
paper_authors.name: Zicheng Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-15T09:14:30.886Z
paper_authors._id: 6526cc6bab4f5d98382f5603
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6526cc6bab4f5d98382f5603/NekYYb61I5nt4au6gXsVK.jpeg
paper_authors.fullname: Zicheng Zhang
paper_authors.type: user
paper_authors.user: zhangzicheng
paper_authors._id: 666c7348e0b2d43ff35e7f5d
paper_authors.name: Guo Lu
paper_authors._id: 666c7348e0b2d43ff35e7f5e
paper_authors.name: Xiongkuo Min
paper_authors._id: 666c7348e0b2d43ff35e7f5f
paper_authors.name: Xiaohong Liu
paper_authors._id: 666c7348e0b2d43ff35e7f60
paper_authors.name: Guangtao Zhai
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-15T09:14:58.072Z
paper_authors._id: 65535b125413c1a54e6fb243
paper_authors.avatarUrl: /avatars/03bcf1d58865f5406aff49a415e78bdc.svg
paper_authors.fullname: Guangtao Zhai
paper_authors.type: user
paper_authors.user: GTZhai
paper_authors._id: 666c7348e0b2d43ff35e7f61
paper_authors.name: Weisi Lin
paper_title: CMC-Bench: Towards a New Paradigm of Visual Signal Compression
paper_summary: Ultra-low bitrate image compression is a challenging and demanding topic.
  With the development of Large Multimodal Models (LMMs), a Cross Modality
  Compression (CMC) paradigm of Image-Text-Image has emerged. Compared with
  traditional codecs, this semantic-level compression can reduce image data size
  to 0.1\% or even lower, which has strong potential applications. However, CMC
  has certain defects in consistency with the original image and perceptual
  quality. To address this problem, we introduce CMC-Bench, a benchmark of the
  cooperative performance of Image-to-Text (I2T) and Text-to-Image (T2I) models
  for image compression. This benchmark covers 18,000 and 40,000 images
  respectively to verify 6 mainstream I2T and 12 T2I models, including 160,000
  subjective preference scores annotated by human experts. At ultra-low bitrates,
  this paper proves that the combination of some I2T and T2I models has surpassed
  the most advanced visual signal codecs; meanwhile, it highlights where LMMs can
  be further optimized toward the compression task. We encourage LMM developers
  to participate in this test to promote the evolution of visual signal codec
  protocols.
paper_ai_keywords: Large Multimodal Models (LMMs) Cross-Modality Compression (CMC) Image-to-Text (I2T) Text-to-Image (T2I)