ADAPTIVE ESTIMATORS SHOW INFORMATION COM- PRESSION IN DEEP NEURAL NETWORKS
To improve how neural networks function it is crucial to understand their learning process. The information bottleneck theory of deep learning proposes that neural networks achieve good generalization by compressing their representations to disregard information that is not relevant to the task. However, empirical evidence for this theory is conflicting, as compression was only observed when networks used saturating activation functions. In contrast, networks with non-saturating activation functions achieved comparable levels of task performance but did not show compression. In this paper we developed more robust mutual information estimation techniques, that adapt to hidden activity of neural networks and produce more sensitive measurements of activations from all functions, especially unbounded functions. Using these adaptive estimation techniques, we explored compression in networks with a range of different activation functions. With two improved methods of estimation, firstly, we show that saturation of the activation function is not required for compression, and the amount of compression varies between different activation functions. We also find that there is a large amount of variation in compression between different network initializations. Secondary, we see that L2 regularization leads to significantly increased compression, while preventing overfitting. Finally, we show that only compression of the last layer is positively correlated with generalization.
This paper has 3 principal contributions: it proposes a different way of measuring mutual information in a neural network, proposes a compression score to compare different models, then empirically analyses different activation functions and L2 weights.
This work seems like a welcome addition to the IB thread. To me the most interesting result is simply that activation functions aren't simply about gradient flow, and that they may each have properties that are more or less desirable depending on the domain they might be used on. The authors are careful in the wording of their conclusions, I think with reason; while these results are useful in that there seem to be consistently different behaviors coming from different hyperparameters, information planes show a relatively qualitative part of the picture.
Quantitatively, the proposed compression score is interesting, but as the authors say, simplistic. It seems to me that we care more about the converged models than the whole training trajectory; how does this score evolve with time?
I think an important part of discussion that lacks in this paper is a more in-depth take as to how these findings relate to the Zhang et al [1] memorization vs generalization paper and its follow ups. There seem to be many links to be drawn.
This work is overall a good contribution, but I'll have to agree with the authors' conclusion that more principled analysis methods are required to have a solid grasp of the training dynamics of DNNs. The writing of the paper is good, but the writing of the captions could be improved. (the hard page limit of ICLR is 10 pages and your paper has a lot of captions, so I think investing into a bit more text would be good)
Comments:
- It might be worth to re-explain what the information plane plots are in a figure caption, not just in the text (the text also doesn't really explain that each point is a moment in training, and each thread a different layer, this paper should be readable by someone who has never seen these plots before). 
- It's not clear what is going on in figure 5, I can guess but, again, this paper should be readable by anyone in the field. You mention different initializations, but which exactly? What makes you say that 5c has no compression but that 5a does compression first? It should be explained explicitly.
- I believe what you say about Figure 8, but the plots are so similar that it is hard to compare them visually. Maybe a different kind of superposition into a single plot would better illustrate the compression effect of L2?
- Typo in the x axis caption of figures 9.
- Figure 9a is not readable in greyscale (or by a colorblind person), consider using a different symbol for the softmax scatter (and adding this symbol to the legend).
- The first Schmidhuber citation of the paper seems a bit out of place. I think he himself would say that deep learning has been going on for much longer than since 2015. (in fact I think you could just remove the entire first paragraph, it is just unnecessary boilerplate)
- Why should there be a direct correlation between compression and generalization? For example, it is known that training DNNs with soft targets improves test accuracy in classification, or even forcing softness in both targets and representations [2] also improves test accuracy.
- I'm still personally not sold on binning as a strategy to evaluate MI. Did you perform experiments that show that the observed difference is consistent if more computation is done to approximate MI, and not just an artefact of max-entropy binning?
[1] Zhang et al (2016) https://arxiv.org/abs/1611.03530
[2] Verma et al (2018) https://arxiv.org/abs/1806.05236