Learned Compression for Compressed Learning
Modern sensors produce increasingly rich streams of high-resolution data. Due
to resource constraints, machine learning systems discard the vast majority of
this information via resolution reduction. Compressed-domain learning allows
models to operate on compact latent representations, allowing higher effective
resolution for the same budget. However, existing compression systems are not
ideal for compressed learning. Linear transform coding and end-to-end learned
compression systems reduce bitrate, but do not uniformly reduce dimensionality;
thus, they do not meaningfully increase efficiency. Generative autoencoders
reduce dimensionality, but their adversarial or perceptual objectives lead to
significant information loss. To address these limitations, we introduce WaLLoC
(Wavelet Learned Lossy Compression), a neural codec architecture that combines
linear transform coding with nonlinear dimensionality-reducing autoencoders.
WaLLoC sandwiches a shallow, asymmetric autoencoder and entropy bottleneck
between an invertible wavelet packet transform. Across several key metrics,
WaLLoC outperforms the autoencoders used in state-of-the-art latent diffusion
models. WaLLoC does not require perceptual or adversarial losses to represent
high-frequency detail, providing compatibility with modalities beyond RGB
images and stereo audio. WaLLoC's encoder consists almost entirely of linear
operations, making it exceptionally efficient and suitable for mobile
computing, remote sensing, and learning directly from compressed data. We
demonstrate WaLLoC's capability for compressed-domain learning across several
tasks, including image classification, colorization, document understanding,
and music source separation. Our code, experiments, and pre-trained audio and
image codecs are available at https://ut-sysml.org/walloc
675badf8b46c607ac7ab809d
Dan Jacobellis
claimed_verified
2024-12-13T15:22:11.962Z
63213080d2d45f3151837eba
https://cdn-avatars.huggingface.co/v1/production/uploads/63213080d2d45f3151837eba/aBhKfY-0gZhKGQmb_Gwi2.png
Dan Jacobellis
user
danjacobellis
675badf8b46c607ac7ab809e
Neeraja J. Yadwadkar
Learned Compression for Compressed Learning
Modern sensors produce increasingly rich streams of high-resolution data. Due
to resource constraints, machine learning systems discard the vast majority of
this information via resolution reduction. Compressed-domain learning allows
models to operate on compact latent representations, allowing higher effective
resolution for the same budget. However, existing compression systems are not
ideal for compressed learning. Linear transform coding and end-to-end learned
compression systems reduce bitrate, but do not uniformly reduce dimensionality;
thus, they do not meaningfully increase efficiency. Generative autoencoders
reduce dimensionality, but their adversarial or perceptual objectives lead to
significant information loss. To address these limitations, we introduce WaLLoC
(Wavelet Learned Lossy Compression), a neural codec architecture that combines
linear transform coding with nonlinear dimensionality-reducing autoencoders.
WaLLoC sandwiches a shallow, asymmetric autoencoder and entropy bottleneck
between an invertible wavelet packet transform. Across several key metrics,
WaLLoC outperforms the autoencoders used in state-of-the-art latent diffusion
models. WaLLoC does not require perceptual or adversarial losses to represent
high-frequency detail, providing compatibility with modalities beyond RGB
images and stereo audio. WaLLoC's encoder consists almost entirely of linear
operations, making it exceptionally efficient and suitable for mobile
computing, remote sensing, and learning directly from compressed data. We
demonstrate WaLLoC's capability for compressed-domain learning across several
tasks, including image classification, colorization, document understanding,
and music source separation. Our code, experiments, and pre-trained audio and
image codecs are available at https://ut-sysml.org/walloc
WaLLoC, a neural codec combining linear transform coding with nonlinear autoencoders, improves compressed-domain learning efficiency and performance across various tasks.
WaLLoC Wavelet Learned Lossy Compression linear transform coding autoencoders entropy bottleneck wavelet packet transform latent diffusion models perceptual losses adversarial losses compressed-domain learning image classification colorization document understanding music source separation