Analyzing deep neural networks (DNNs) via information plane (IP) theory has gained tremendous attention recently as a tool to gain insight into, among others, their generalization ability. However, it is by no means obvious how to estimate mutual information (MI) between each hidden layer and the input/desired output, to construct the IP. For instance, hidden layers with many neurons require MI estimators with robustness towards the high dimensionality associated with such layers. MI estimators should also be able to naturally handle convolutional layers, while at the same time being computationally tractable to scale to large networks. None of the existing IP methods to date have been able to study truly deep Convolutional Neural Networks (CNNs), such as the e.g. VGG-16. In this paper, we propose an IP analysis using the new matrix-based Rényi's entropy coupled with tensor kernels over convolutional layers, leveraging the power of kernel methods to represent properties of the probability distribution independently of the dimensionality of the data. The obtained results shed new light on the previous literature concerning small-scale DNNs, however using a completely new approach. Importantly, the new framework enables us to provide the first comprehensive IP analysis of contemporary large-scale DNNs and CNNs, investigating the different training phases and providing new insights into the training dynamics of largescale neural networks.
This paper concerns the "information plane" view of visualizing and understanding neural net training. Kernel-based estimators of Renyi entropy-like quantities are applied to study moderate-sized DNNs. A straightforward extension to CNNs is presented.
Major comments:
I found this paper extremely hard to follow. I think a lot of my difficulty was a lack of familiarity with the papers this work was building on, but I also felt the main line of reasoning was unnecessarily opaque. I've tried to indicate where some clarifying wording might help readers such as myself.
In general I am very skeptical that reasonable entropy-like quantities can be estimated reliably for high-dimensional data using anything along the lines of kernel density estimation, especially based on a single minibatch or small collection of minibatches! The authors provide no experimental evidence that these estimates are even close to being accurate (for example on synthetic datasets where the true entropy / mutual information is known). Clearly the estimated quantities evolve during training, and that may be interesting in itself, but calling the estimated quantities "mutual information" seems like a leap that's not justified in the paper.
Throughout the paper, CNNs are referred to as having some special difficulty for entropy estimates because of the channel dimension. This is misleading. For DNNs the activations are vectors, and so a kernel defined over rank 1 tensors is needed. Even without the channel dimension, CNN activations would be rank 2 tensors, and so a kernel would need to be defined over rank 2 tensors. Going from rank 2 to rank 3 doesn't pose any special difficulties. Indeed the most obvious way of defining a kernel over higher rank tensors is to flatten them and use a kernel defined over vectors, which is exactly what the paper does in practice.
Minor comments:
In the introduction, it would be helpful to include some relevant citations after the sentence "Several works have demonstrated that one may unveil interesting properties... IP".
In section 3, the multivariate extension proposed by Yu et al. (2019) seems like an interesting side note (since it was used in a previous attempt to estimate information plane for CNNs), but it doesn't seem central to the paper, and I personally found it unnecessarily confusing to have it presented in the main text. What about moving sections 3.2 and 4.2 to the appendix for clarity?
In section 3.1, "over a finite set" is probably incorrect ("probability density function" implies a continuous space for X, as does the integral in (1)).
In section 3.1 (and the appendix), "The ones developed for Shannon" seems imprecise. "Certain approaches developed for estimating the Shannon entropy"?
It's not clear what "have the same functional form of the statistical quantity" means. Which statistical quantity? What aspects of the functional form are similar? Please elaborate in the paper.
I think "marginal distribution" is incorrect. It's representing a whole probability distribution, not just a marginal distribution. Which marginal would it be in any case?
Section 3.2 states "The matrix-based... entropy functional is not suitable for estimating... convolutional layer... as the output consists of C feature maps", but as discussed above, there is no special difficulty caused by the channel dimension. Even if the channel dimension were not present the difficulties would be the same. (Also, it seems like defining a kernel over the rank-3 tensors is an extremely natural / unsurprising thing to try given the set-up so far.)
In section 4.1, "can not include tensor data without modifications" seems misleading for a similar reason. One of the great things about kernels is that they can be defined on lots of objects, including things like rank 3 tensors!
Near (8), it would be very helpful to state explicitly what is done for the joint entropy term in (5). It sounds like this term, which involves the Hadamard product, in practice amounts to summing the Euclidean distances between x's and between y's, and it might be helpful to the new reader to point this out. (It also highlights that the method is easy to implement in practice).
The discussion in section 4.2 is only valid for the RBF kernel, but the first paragraph of that section makes it sound like it is true more generally.
At the bottom of section 4.2, if the proposed approach is equivalent to the multivariate approach, then how can one suffer from numerical instability while the other doesn't? Also, numerical stability makes it sound like an artifact of floating point arithmetic, whereas I think the point that's being made is more mathematical? Please clarify in the paper.
In "enabling training of complex neural networks", shouldn't "training" be "analysis"?
In section 5, under "increasing DNN size", I wasn't clear on the meaning of "deterministic" in "neither .... is deterministic for our proposed estimator". Random variables can be deterministic or not, but how can a mutual information be deterministic?
Under "effect of early stopping", isn't looking at the test set entropies (as is done elsewhere in the paper) much more relevant to overfitting than considering different "patience" values?
In the bibliography, two different papers are "Yu et al. (2019)".
In appendix A, "the same functional form of the statistical quantity", "marginal", etc don't seem quite correct, as mentioned above. Also the first equation should not have a comma between f(X) and g(Y) (which if I understand correctly are being multiplied).