title: Learned Compression for Compressed Learning
summary: Modern sensors produce increasingly rich streams of high-resolution data. Due
  to resource constraints, machine learning systems discard the vast majority of
  this information via resolution reduction. Compressed-domain learning allows
  models to operate on compact latent representations, allowing higher effective
  resolution for the same budget. However, existing compression systems are not
  ideal for compressed learning. Linear transform coding and end-to-end learned
  compression systems reduce bitrate, but do not uniformly reduce dimensionality;
  thus, they do not meaningfully increase efficiency. Generative autoencoders
  reduce dimensionality, but their adversarial or perceptual objectives lead to
  significant information loss. To address these limitations, we introduce WaLLoC
  (Wavelet Learned Lossy Compression), a neural codec architecture that combines
  linear transform coding with nonlinear dimensionality-reducing autoencoders.
  WaLLoC sandwiches a shallow, asymmetric autoencoder and entropy bottleneck
  between an invertible wavelet packet transform. Across several key metrics,
  WaLLoC outperforms the autoencoders used in state-of-the-art latent diffusion
  models. WaLLoC does not require perceptual or adversarial losses to represent
  high-frequency detail, providing compatibility with modalities beyond RGB
  images and stereo audio. WaLLoC's encoder consists almost entirely of linear
  operations, making it exceptionally efficient and suitable for mobile
  computing, remote sensing, and learning directly from compressed data. We
  demonstrate WaLLoC's capability for compressed-domain learning across several
  tasks, including image classification, colorization, document understanding,
  and music source separation. Our code, experiments, and pre-trained audio and
  image codecs are available at https://ut-sysml.org/walloc
paper_authors._id: 675badf8b46c607ac7ab809d
paper_authors.name: Dan Jacobellis
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-13T15:22:11.962Z
paper_authors._id: 63213080d2d45f3151837eba
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/63213080d2d45f3151837eba/aBhKfY-0gZhKGQmb_Gwi2.png
paper_authors.fullname: Dan Jacobellis
paper_authors.type: user
paper_authors.user: danjacobellis
paper_authors._id: 675badf8b46c607ac7ab809e
paper_authors.name: Neeraja J. Yadwadkar
paper_title: Learned Compression for Compressed Learning
paper_summary: Modern sensors produce increasingly rich streams of high-resolution data. Due
  to resource constraints, machine learning systems discard the vast majority of
  this information via resolution reduction. Compressed-domain learning allows
  models to operate on compact latent representations, allowing higher effective
  resolution for the same budget. However, existing compression systems are not
  ideal for compressed learning. Linear transform coding and end-to-end learned
  compression systems reduce bitrate, but do not uniformly reduce dimensionality;
  thus, they do not meaningfully increase efficiency. Generative autoencoders
  reduce dimensionality, but their adversarial or perceptual objectives lead to
  significant information loss. To address these limitations, we introduce WaLLoC
  (Wavelet Learned Lossy Compression), a neural codec architecture that combines
  linear transform coding with nonlinear dimensionality-reducing autoencoders.
  WaLLoC sandwiches a shallow, asymmetric autoencoder and entropy bottleneck
  between an invertible wavelet packet transform. Across several key metrics,
  WaLLoC outperforms the autoencoders used in state-of-the-art latent diffusion
  models. WaLLoC does not require perceptual or adversarial losses to represent
  high-frequency detail, providing compatibility with modalities beyond RGB
  images and stereo audio. WaLLoC's encoder consists almost entirely of linear
  operations, making it exceptionally efficient and suitable for mobile
  computing, remote sensing, and learning directly from compressed data. We
  demonstrate WaLLoC's capability for compressed-domain learning across several
  tasks, including image classification, colorization, document understanding,
  and music source separation. Our code, experiments, and pre-trained audio and
  image codecs are available at https://ut-sysml.org/walloc
paper_ai_keywords: compressed-domain learning resolution reduction linear transformation coding non-linear dimensions-reducing autoencoders entropy bottleneck invertible wavelet packet transform WaLLoC (Wavelet Learned Lossy Compression) linear operations neural codec architecture compressed-domain learning perceptual loss adversarial loss high-frequency detail feature operations compressed-domain image classification colorization document understanding music source separation pretrained audio and image codecs