title: Minimum Entropy Coupling with Bottleneck
summary: This paper investigates a novel lossy compression framework operating under
  logarithmic loss, designed to handle situations where the reconstruction
  distribution diverges from the source distribution. This framework is
  especially relevant for applications that require joint compression and
  retrieval, and in scenarios involving distributional shifts due to processing.
  We show that the proposed formulation extends the classical minimum entropy
  coupling framework by integrating a bottleneck, allowing for a controlled
  degree of stochasticity in the coupling. We explore the decomposition of the
  Minimum Entropy Coupling with Bottleneck (MEC-B) into two distinct optimization
  problems: Entropy-Bounded Information Maximization (EBIM) for the encoder, and
  Minimum Entropy Coupling (MEC) for the decoder. Through extensive analysis, we
  provide a greedy algorithm for EBIM with guaranteed performance, and
  characterize the optimal solution near functional mappings, yielding
  significant theoretical insights into the structural complexity of this
  problem. Furthermore, we illustrate the practical application of MEC-B through
  experiments in Markov Coding Games (MCGs) under rate limits. These games
  simulate a communication scenario within a Markov Decision Process, where an
  agent must transmit a compressed message from a sender to a receiver through
  its actions. Our experiments highlight the trade-offs between MDP rewards and
  receiver accuracy across various compression rates, showcasing the efficacy of
  our method compared to conventional compression baseline.
paper_authors._id: 6725296eccb533a2ed462bfa
paper_authors.name: M. Reza Ebrahimi
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-11-03T14:20:51.667Z
paper_authors._id: 648a1075e8bee5332919e0ab
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/648a1075e8bee5332919e0ab/KDzi904zGPuy_PL3gbUYE.jpeg
paper_authors.fullname: M.Reza Ebrahimi
paper_authors.type: user
paper_authors.user: mamaj92
paper_authors._id: 6725296eccb533a2ed462bfb
paper_authors.name: Jun Chen
paper_authors._id: 6725296eccb533a2ed462bfc
paper_authors.name: Ashish Khisti
paper_title: Minimum Entropy Coupling with Bottleneck
paper_summary: This paper investigates a novel lossy compression framework operating under
  logarithmic loss, designed to handle situations where the reconstruction
  distribution diverges from the source distribution. This framework is
  especially relevant for applications that require joint compression and
  retrieval, and in scenarios involving distributional shifts due to processing.
  We show that the proposed formulation extends the classical minimum entropy
  coupling framework by integrating a bottleneck, allowing for a controlled
  degree of stochasticity in the coupling. We explore the decomposition of the
  Minimum Entropy Coupling with Bottleneck (MEC-B) into two distinct optimization
  problems: Entropy-Bounded Information Maximization (EBIM) for the encoder, and
  Minimum Entropy Coupling (MEC) for the decoder. Through extensive analysis, we
  provide a greedy algorithm for EBIM with guaranteed performance, and
  characterize the optimal solution near functional mappings, yielding
  significant theoretical insights into the structural complexity of this
  problem. Furthermore, we illustrate the practical application of MEC-B through
  experiments in Markov Coding Games (MCGs) under rate limits. These games
  simulate a communication scenario within a Markov Decision Process, where an
  agent must transmit a compressed message from a sender to a receiver through
  its actions. Our experiments highlight the trade-offs between MDP rewards and
  receiver accuracy across various compression rates, showcasing the efficacy of
  our method compared to conventional compression baseline.
paper_ai_summary: A novel lossy compression framework with a bottleneck extension to minimum entropy coupling is proposed, providing theoretical insights and practical applications in Markov Coding Games.
paper_ai_keywords: logarithmic loss minimum entropy coupling bottleneck Entropy-Bounded Information Maximization Minimum Entropy Coupling Markov Coding Games