NECST: Neural Joint Source-Channel Coding
For reliable transmission across a noisy communication channel, classical results from information theory show that it is asymptotically optimal to separate out the source and channel coding processes. However, this decomposition can fall short in the finite bit-length regime, as it requires non-trivial tuning of hand-crafted codes and assumes infinite computational power for decoding. In this work, we propose Neural Error Correcting and Source Trimming (NECST) codes to jointly learn the encoding and decoding processes in an end-to-end fashion. By adding noise into the latent codes to simulate the channel during training, we learn to both compress and error-correct given a fixed bit-length and computational budget. We obtain codes that are not only competitive against several capacity-approaching channel codes, but also learn useful robust representations of the data for downstream tasks such as classification. Finally, we learn an extremely fast neural decoder, yielding almost an order of magnitude in speedup compared to standard decoding methods based on iterative belief propagation.
 This interesting paper tackles the problem of joint source-channel coding, by means of learning. From 100kft heights, especially given the choice of VIMCO gradient estimates, this is effectively a "let*s embed a source-channel-decoder simulator and differentiate through it", and find a solution that is better than source|channel factorized classic methods, or hand-tuned approaches. The method and results are good. The authors also show some interesting results about the representations learned, about how decoded samples (images) change smoothly when the (discrete) embedding (the-codes) changes over deltas of hamming_d()=1bit. This is very good results IHMO. One limitation of this method is the fixed-code-length. Jumping straight to my main main issue with this paper: no code was made available, at least not at this time. While the authors do provide an extensive appendix with hyper-parameter specs, usually in my experience when dealing with discrete / monte-carlo methods, it*s usually rather hard to reproduce results. I really strongly advise the authors to provide fully reproducible code for this paper, to help further research on this topic. Besides that I have three technical comments / request regarding this paper: 1// the choice of BSC channel - while this is the easiest most natural choice, and we should certainly have results on BSC, I am left wondering why the authors didn*t try other more complex / more realistic channels? The authors only mention this as potential area of future research in the last sentence of the conclusions. There are several reasons for this comment: first of all, it is well known that even classic joint source-channel coding methods do shine on complex channels, such fading/erasure channels and/or in general channels with correlated error sequences. Such channels are indeed key in modern wireless communications, and are easy to simulate. Given that more-complex channels could be introduced in the channel model p(y_hat|y) - it would not change the rest of the method - it would be particularly interesting to see what results this method achieve in these more complex environments. 2// I would like to hear more about the choice of VIMCO. Understood the authors statement to "preserve the hard discreteness" ~ that said methods like Gumbel-SM and several others also referenced in the paper ~ have been used successfully to solve for propagating gradients through discrete units. This is where, in my opinion, experiments comparing VIMCO approximation results to at least one other method could allow to decide / validate the best architecture. This is also because, in my previous experience, this type of networks with discrete units may be hard to train. I would like to hear from the authors about how stable the training was under different hyper-parameters, and perhaps see some convergence curves for the loss function(s). 3// it*s not 100% clear to me where the limitation of fixed code-length come into play from the architecture. Could the authors please point this out clearly? Thank you!
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
NECST: Neural Joint Source-Channel Coding
For reliable transmission across a noisy communication channel, classical results from information theory show that it is asymptotically optimal to separate out the source and channel coding processes. However, this decomposition can fall short in the finite bit-length regime, as it requires non-trivial tuning of hand-crafted codes and assumes infinite computational power for decoding. In this work, we propose Neural Error Correcting and Source Trimming (NECST) codes to jointly learn the encoding and decoding processes in an end-to-end fashion. By adding noise into the latent codes to simulate the channel during training, we learn to both compress and error-correct given a fixed bit-length and computational budget. We obtain codes that are not only competitive against several capacity-approaching channel codes, but also learn useful robust representations of the data for downstream tasks such as classification. Finally, we learn an extremely fast neural decoder, yielding almost an order of magnitude in speedup compared to standard decoding methods based on iterative belief propagation.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



 This interesting paper tackles the problem of joint source-channel coding, by means of learning. From 100kft heights, especially given the choice of VIMCO gradient estimates, this is effectively a "let*s embed a source-channel-decoder simulator and differentiate through it", and find a solution that is better than source|channel factorized classic methods, or hand-tuned approaches. The method and results are good. The authors also show some interesting results about the representations learned, about how decoded samples (images) change smoothly when the (discrete) embedding (the-codes) changes over deltas of hamming_d()=1bit. This is very good results IHMO. One limitation of this method is the fixed-code-length. Jumping straight to my main main issue with this paper: no code was made available, at least not at this time. While the authors do provide an extensive appendix with hyper-parameter specs, usually in my experience when dealing with discrete / monte-carlo methods, it*s usually rather hard to reproduce results. I really strongly advise the authors to provide fully reproducible code for this paper, to help further research on this topic. Besides that I have three technical comments / request regarding this paper: 1// the choice of BSC channel - while this is the easiest most natural choice, and we should certainly have results on BSC, I am left wondering why the authors didn*t try other more complex / more realistic channels? The authors only mention this as potential area of future research in the last sentence of the conclusions. There are several reasons for this comment: first of all, it is well known that even classic joint source-channel coding methods do shine on complex channels, such fading/erasure channels and/or in general channels with correlated error sequences. Such channels are indeed key in modern wireless communications, and are easy to simulate. Given that more-complex channels could be introduced in the channel model p(y_hat|y) - it would not change the rest of the method - it would be particularly interesting to see what results this method achieve in these more complex environments. 2// I would like to hear more about the choice of VIMCO. Understood the authors statement to "preserve the hard discreteness" ~ that said methods like Gumbel-SM and several others also referenced in the paper ~ have been used successfully to solve for propagating gradients through discrete units. This is where, in my opinion, experiments comparing VIMCO approximation results to at least one other method could allow to decide / validate the best architecture. This is also because, in my previous experience, this type of networks with discrete units may be hard to train. I would like to hear from the authors about how stable the training was under different hyper-parameters, and perhaps see some convergence curves for the loss function(s). 3// it*s not 100% clear to me where the limitation of fixed code-length come into play from the architecture. Could the authors please point this out clearly? Thank you!
assistant
 This interesting paper tackles the problem of joint source-channel coding, by means of learning. From 100kft heights, especially given the choice of VIMCO gradient estimates, this is effectively a "let*s embed a source-channel-decoder simulator and differentiate through it", and find a solution that is better than source|channel factorized classic methods, or hand-tuned approaches. The method and results are good. The authors also show some interesting results about the representations learned, about how decoded samples (images) change smoothly when the (discrete) embedding (the-codes) changes over deltas of hamming_d()=1bit. This is very good results IHMO. One limitation of this method is the fixed-code-length. Jumping straight to my main main issue with this paper: no code was made available, at least not at this time. While the authors do provide an extensive appendix with hyper-parameter specs, usually in my experience when dealing with discrete / monte-carlo methods, it*s usually rather hard to reproduce results. I really strongly advise the authors to provide fully reproducible code for this paper, to help further research on this topic. Besides that I have three technical comments / request regarding this paper: 1// the choice of BSC channel - while this is the easiest most natural choice, and we should certainly have results on BSC, I am left wondering why the authors didn*t try other more complex / more realistic channels? The authors only mention this as potential area of future research in the last sentence of the conclusions. There are several reasons for this comment: first of all, it is well known that even classic joint source-channel coding methods do shine on complex channels, such fading/erasure channels and/or in general channels with correlated error sequences. Such channels are indeed key in modern wireless communications, and are easy to simulate. Given that more-complex channels could be introduced in the channel model p(y_hat|y) - it would not change the rest of the method - it would be particularly interesting to see what results this method achieve in these more complex environments. 2// I would like to hear more about the choice of VIMCO. Understood the authors statement to "preserve the hard discreteness" ~ that said methods like Gumbel-SM and several others also referenced in the paper ~ have been used successfully to solve for propagating gradients through discrete units. This is where, in my opinion, experiments comparing VIMCO approximation results to at least one other method could allow to decide / validate the best architecture. This is also because, in my previous experience, this type of networks with discrete units may be hard to train. I would like to hear from the authors about how stable the training was under different hyper-parameters, and perhaps see some convergence curves for the loss function(s). 3// it*s not 100% clear to me where the limitation of fixed code-length come into play from the architecture. Could the authors please point this out clearly? Thank you!