Published as a conference paper at ICLR 2018 ON THE INFORMATION BOTTLENECK THEORY OF DEEP LEARNING
The practical successes of deep neural networks have not been matched by theoretical progress that satisfyingly explains their behavior. In this work, we study the information bottleneck (IB) theory of deep learning, which makes three specific claims: first, that deep networks undergo two distinct phases consisting of an initial fitting phase and a subsequent compression phase; second, that the compression phase is causally related to the excellent generalization performance of deep networks; and third, that the compression phase occurs due to the diffusion-like behavior of stochastic gradient descent. Here we show that none of these claims hold true in the general case. Through a combination of analytical results and simulation, we demonstrate that the information plane trajectory is predominantly a function of the neural nonlinearity employed: double-sided saturating nonlinearities like tanh yield a compression phase as neural activations enter the saturation regime, but linear activation functions and single-sided saturating nonlinearities like the widely used ReLU in fact do not. Moreover, we find that there is no evident causal connection between compression and generalization: networks that do not compress are still capable of generalization, and vice versa. Next, we show that the compression phase, when it exists, does not arise from stochasticity in training by demonstrating that we can replicate the IB findings using full batch gradient descent rather than stochastic gradient descent. Finally, we show that when an input domain consists of a subset of task-relevant and task-irrelevant information, hidden representations do compress the task-irrelevant information, although the overall information about the input may monotonically increase with training time, and that this compression happens concurrently with the fitting process rather than during a subsequent compression period.
The authors address the issue of whether the information bottleneck (IB) theory can provide insight into the working of deep networks. They show, using some counter-examples, that the previous understanding of IB theory and its application to deep networks is limited.
PROS: The paper is very well written and makes its points very clearly. To the extent of my knowledge, the content is original. Since it clearly elucidates the limitations of IB theory in its ability to analyse deep networks, I think it is a significant 
contribution worthy of acceptance. The experiments are also well designed and executed. 
CONS: On the downside, the limitations exposed are done so empirically, but the underlying theoretical causes are not explored (although this could be potentially because this is hard to do). Also, the paper exposes the limitations of another paper published in a non-peer reviewed location (arXiv) which potentially limits its applicability and significance.
Some detailed comments:
In section 2, the influence of binning on how the mutual information is calculated should be made clear. Since the comparison is between a bounded non-linearity and an unbounded one, it is not self-evident how the binning in the latter case should be done. A justification for the choice made for binning the relu case would be helpful.
In the same section, it is claimed that the dependence of the mutual information I(X; T) on the magnitude of the weights of the network explains why a tanh non-linearity shows the compression effect (non-monotonicity vs I(X; T)) in the information plane dynamics. But the claim that large weights are required for doing anything useful is unsubstantiated and would benefit from having citations to papaers that discuss this issue. If networks with small weights are able to learn most datasets, the arguments given in this section wouldn't be applicable in its entirety.
Additionally, figures that show the phase plane dynamics for other non-linearities e.g. relu+ or sigmoid, should be added, at 
least in the supplementary section. This is important to complete the overall picture of how the compression effect depends on having specific activation functions.
In section 3, a sentence or two should be added to describe what a "teacher-student setup" is, and how it is relevant/interesting.
Also in section 3, the cases where batch gradient descent is used and where stochastic gradient descent is used should be 
pointed out much more clearly. It is mentioned in the first line of page 7 that batch gradient descent is used, but it is not 
clear why SGD couldn't have been used to keep things consistent. This applies to figure 4 too. 
In section 4, it seems inconsistent that the comparison of SGD vs BGD is done using linear network as opposed to a relu network which is what's used in Section 2. At the least, a comparison using relu should be added to the supplementary section.
Minor comments 
The different figure styles using in Fig 4A and C that have the same quantities plotted makes it confusing.
An additional minor comment on the figures: some of the labels are hard to read on the manuscript.