A deployable machine learning model relies on a good representation. A desirable criteria for a good representation is to be interpretable by humans. We propose a technique termed perceptual regularization that enables visualization of the latent representation. By visualizing the learned representation, we are also able to understand the attention of a model, obtaining visual evidence that supervised networks learn task-specific representations. Moreover our method provides a direct visualization of the effect that adversarial attacks have on the internal representation of a deep network. We show how this can be used to systematically perform latent space interpolation to modify semantic features.
Summary: This paper discusses the use of a reconstruction network which is fed an internal representation of a deep network and is trained to reconstruct the input. The classification and reconstruction networks are trained simultaneously with a combined loss function. Various experiments are carried out to test whether this approach can help investigate adversarial examples and the quality of learned features for transfer learning.
Review: The idea presented in this paper is simple and easy to understand and the paper includes experiments testing its effectiveness in various settings. However, there are various issues with the paper in its current form: First, the idea of reconstructing hidden activations is not new and the paper fails to cite future work. "Dynamic Routing Between Capsules" by Sabour et al. consider adding a similar reconstruction network to Capsule networks. "Detecting and Diagnosing Adversarial Images with Class-Conditional Capsule Reconstructions" considers the use of this reconstruction network for detecting adverarsarial examples (like this paper) and also includes an unconditional reconstruction network which is added to a convnet (like this paper). I imagine there are other examples, but those are ones I know of. Second, the experiments are limited - they only include simple datasets (SVHN, MNIST, CelebA) which are normalized so that the object appears in the center of the image. This makes reconstructions difficult. The experiments also lack various important details like the model architectures and other hyperparameters, and overall are mostly "proof of concept" experiments rather than exhaustively testing out the proposed method. Finally, some of the claims made in the paper (particularly in the introduction) are dubious, for example that adversarial examples are a security concern and that transfer learning has been unsuccessful because it can harm performance. Based on these issues, I won't recommend acceptance. I think if the authors better situated the novelty of their work, included more exhaustive and difficult experiments, and/or focused on one specific application area it would be a stronger submission.
Specific comments:
- "to be understandable" - can you be more specific? Internal representations of deep neural networks are not understandable whatsoever on their own, we usually use some post-hoc method (like your reconstruction network) to try to interpret them somehow.
- "[adversarial examples] raise many security concerns, such as the reliability of driverless cars, or the trustworthiness of facial recognition systems" The field is moving towards a consensus that adversarial examples do not pose a realistic security threat in any of the settings they have been tested in. The most significant description of this viewpoint is in "Motivating the Rules of the Game for Adversarial Example Research" by Gilmer et al.
- "but has made the transferability of features to new tasks difficult, sometimes even harming performance" This statement completely omits a huge body of work showing how effective transfer learning is, not only in computer vision (typically going from an ImageNet classifier to some downstream task with a small training set) but also in NLP (where transfer learning-based methods have state-of-the-art in virtually every benchmark). A good discussion of when transfer learning works and doesn't in CV is provided in "Do Better ImageNet Models Transfer Better?" by Kornblith et al.
- The variables X and Y (without mathcal) in the "(Perceptual regularization)" definition equation are not defined.
- The classification and reconstruction models are not described in Section 3. This is incredibly important. The quality of the reconstructions is likely extremely dependent on these architectural choices as they provide an implicit prior over the image space. This detail needs to be filled in. E.g. the pooling/downsampling in your classification network  will have a strong impact on the reconstruction quality, as will the size, depth, and configuration of the reconstruction network. Furthermore, I imagine the regularization parameter lambda is extremely important (as shown in section 4.2) but its value is not mentioned.
- The experiments are on SVHN, MNIST, and CelebA which are all "normalized" datasets, i.e. all of their constituent images are normalized so that the object of interest appears in the same position in the center of the image. This makes reconstruction much, much easier. Results should be presented on non-normalized datasets (e.g. CIFAR-10, ImageNet, etc) in order to be convincing.
- "But, as the network gets more complicated it becomes computationally difficult to decode the feature map." Can you quantify this claim or back it up with experiments?
- While the results on transferring to different CelebA classification tasks are interesting, it is much more common to pre-train on a diverse task like ImageNet than an extremely fine-grained task like classifying smiling or not smiling. I think these results would be more impressive if you followed modern experimental practice more closely.
- "The cost is that our method performs slightly poorer than the standard training method when S = T" I would not call a difference of upwards of 8% absolute small.