It is commonly agreed that the use of relevant invariances as a good statistical bias is important in machine-learning. However, most approaches that explicitely incorporate invariances into a model architecture only make use of very simple transformations, such as translations and rotations. Hence, there is a need for methods to model and extract richer transformations that capture much higherlevel invariances. To that end, we introduce a tool allowing to parametrize the set of filters of a trained convolutional neural network with the latent space of a generative adversarial network. We then show that the method can capture highly non-linear invariances of the data by visualizing their effect in the data space.
Recent work on incorporating prior knowledge about invariances into neural networks suggests that the feature dimension in a stack of feature maps has some kind of group or manifold structure, similar to how the spatial axes form a plane. This paper proposes a method to uncover this structure from the filters of a trained ConvNet. The method uses an InfoGAN to learn the distribution of filters. By varying the latent variables of the GAN, one can traverse the manifold of filters. The effect of moving over the manifold can be visualized by optimizing an input image to produce the same activation profile when using a perturbed synthesized filter as when using an unperturbed synthesized filter.
The idea of empirically studying the manifold / topological / group structure in the space of filters is interesting. A priori, using a GAN to model a relatively small number of filters seems problematic due to overfitting, but the authors show that their InfoGAN approach seems to work well.
My main concerns are:
Controls
To generate the visualizations, two coordinates in the latent space are varied, and for each variation, a figure is produced. To figure out if the GAN is adding anything, it would be nice to see what would happen if you varied individual coordinates in the filter space ("x-space" of the GAN), or varied the magnitude of filters or filter planes. Since the visualizations are as much a function of the previous layers as they are a function of the filters in layer l which are modelled by the GAN, I would expect to see similar plots for these baselines.
Lack of new Insights
The visualizations produced in this paper are interesting to look at, but it is not clear what they tell us, other than "something non-trivial is going on in these networks". In fact, it is not even clear that the transformations being visualized are indeed non-linear in pixel space (note that even a 2D diffeomorphism, which is a non-linear map on R^2, is a linear operator on the space of *functions* on R^2, i.e. on the space of images). In any case, no attempt is made to analyze the results, or provide new insights into the computations performed by a trained ConvNet.
Interpretation
This is a minor point, but I would not say (as the paper does) that the method captures the invariances learned by the model, but rather that it aims to show the variability captured by the model. A ReLU net is only invariant to changes that are mapped to zero by the ReLU, or that end up in the kernel of one of the linear layers. The presented method does not consider this and hence does not analyze invariances.
Minor issues:
- In the last equation on page 2, the right-hand side is missing a "min max".