SVCCA: Singular Vector Canonical Correlation Analysis for Deep Learning Dynamics and Interpretability
We propose a new technique, Singular Vector Canonical Correlation Analysis (SVCCA), a tool for quickly comparing two representations in a way that is both invariant to affine transform (allowing comparison between different layers and networks) and fast to compute (allowing more comparisons to be calculated than with previous methods). We deploy this tool to measure the intrinsic dimensionality of layers, showing in some cases needless over-parameterization; to probe learning dynamics throughout training, finding that networks converge to final representations from the bottom up; to show where class-specific information in networks is formed; and to suggest new training regimes that simultaneously save computation and overfit less.
The paper presents an analysis of deep networks based on a combination of SVD and CCA to test similarity between representations at different layers. The authors use this technique to analyze aspects of the deep neural network training procedure, for example, how the representation builds bottom-up throughout training, and suggest an improved training procedure based on this analysis where lower-layers are frozen for most the training.
The paper is well-written and contributions are clearly stated.
The authors characterize the representation of a neuron by the vector of its activations for each data point, and then represent each layer as a matrix composed of the activation vectors for each neuron. Abstracting each layer as a kernel, and applying kernel SVD / CCA (with the linear kernel as a special case) would seem a more natural way to describe the same problem.
In section 2.1, it is unclear why the authors perform CCA on two networks instead of PCA on a single network, except for the sake of expressing the analysis as a CCA problem. It would be good to have PCA as a baseline.
While the bottom-up emergence of representation is an interesting effect, it is unclear whether the effect is general or is a consequence of the convnet structure of the model, in particular, the smaller number of parameters in the early layers.
In Figure 4, it seems that the representations have relatively low correlation until the very end where they jump to 1. It suggests noise in the training procedure for the top layers. It would be interesting to test whether the frozen layers training strategy lead to higher correlations also for the higher layers.