EXCESSIVE INVARIANCE CAUSES ADVERSARIAL VULNERABILITY
Despite their impressive performance, deep neural networks exhibit striking failures on out-of-distribution inputs. One core idea of adversarial example research is to reveal neural network errors under such distribution shifts. We decompose these errors into two complementary sources: sensitivity and invariance. We show deep networks are not only too sensitive to task-irrelevant changes of their input, as is well-known from -adversarial examples, but are also too invariant to a wide range of task-relevant changes, thus making vast regions in input space vulnerable to adversarial attacks. We show such excessive invariance occurs across various tasks and architecture types. On MNIST and ImageNet one can manipulate the class-specific content of almost any image without changing the hidden activations. We identify an insufficiency of the standard cross-entropy loss as a reason for these failures. Further, we extend this objective based on an informationtheoretic analysis so it encourages the model to consider all task-dependent features in its decision. This provides the first approach tailored explicitly to overcome excessive invariance and resulting vulnerabilities.
This paper studies a new perspective on why adversarial examples exist in machine learning -- instead of seeing adversarial examples as the result of a classifier being sensitive to changes in irrelevant information (aka nuisance), the authors see them as the result of a classifier being invariant to changes in relevant (aka semantic) information. They show how to efficiently find such adversarial examples in bijective networks. Moreover, they propose to modify the training objective so that the bijective networks could be more robust to such attacks.
Pros:
 -- clarity is good (except for a few places, e.g. no definition of F(x)_i in Definition 1; Page 6 "three ways forward" item 3: I(y;z_n|z_s) = I(y;z_s) should be I(y;z_n|z_s) = I(y;z_n).)
 -- the idea is original to the best of my knowledge
 -- the mathematical motivation is sound
 -- Figure 6 seems to show that the proposed defense works on MNIST (However, would you provide more details on how you interpolated z_n? Moreover, what do the images generated with z_s from one input and z_n from another input look like (in your method)?)
Cons:
 -- scope: as all the presented problems and solutions assume bijective mapping, I wonder how is it relevant to the traditional perspective of adversarial attack and defense? It seems to me that the contribution of this paper is identifying a problem of bijective networks and then proposing a solution, thus its significance is restricted.
 -- method: while the mathematical motivation is sound, I'm not sure if the proposed training objective can achieve that goal. To elaborate, I see problems with both terms added in the proposed loss function:
 (a.) for the objective of maximizing the cross entropy of the nuisance classifier, it is possible that I(y;z_n) is not reduced, but rather the information about y is encoded in a way that the nuisance classifier is not able to decode, similar to what happens in a one-way function (for example, see https://en.wikipedia.org/wiki/Cryptographic_hash_function ). In the MNIST experiments, the nuisance classifier is a three-layer MLP, which may be too weak and susceptible to information concealing.
 (b.) for the objective of maximizing the likelihood of a factorized model of p(z_s, z_n), I don't see how optimizing it would reduce I(z_s; z_n). In general, even if z_s and z_n are strongly correlated, one can still fit such a factorized model. This only ensures that I(Z_s; Z_n) = 0 for Z_s, Z_n *sampled from the model*, but does not necessarily reduce I(z_s; z_n) for z_s, z_n *used to train the model*. The discrepancy between p(Z_s, Z_n) and p(z_s, z_n) could be huge, in which case one has the model misspecification problem which is another topic.
 (c.) a side question: why is the MLE objective using likelihood rather than log likelihood? Since the two cross entropy losses are similar to log likelihood, I feel there is a mismatch here.
----------------------------------------
AFTER REBUTTAL:
Thanks for your reply to my comments. The new revision has improved clarity and provided new supporting evidences. I would like to raise my rating to 6.
That being said, (as you agreed) the link from the conceptual goal to the proposed objective has mostly empirical support. Therefore I hope it may encourage future investigation on when and why the proposed objective is successful in achieving the conceptual goal.