Deep neural networks are vulnerable to adversarial examples: input data that has been manipulated to cause dramatic model output errors. To defend against such attacks, we propose NeuralFingerprinting: a simple, yet effective method to detect adversarial examples that verifies whether model behavior is consistent with a set of fingerprints. These fingerprints are encoded into the model response during training and are inspired by the use of biometric and cryptographic signatures. In contrast to previous defenses, our method does not rely on knowledge of the adversary and can scale to large networks and input data. In this work, we 1) theoretically analyze NeuralFingerprinting for linear models and 2) show that NeuralFingerprinting significantly improves on state-of-the-art detection mechanisms for deep neural networks, by detecting the strongest known adversarial attacks with 98-100% AUC-ROC scores on the MNIST, CIFAR-10 and MiniImagenet (20 classes) datasets. In particular, we consider several threat models, including the most conservative one in which the attacker has full knowledge of the defender's strategy. In all settings, the detection accuracy of NeuralFingerprinting generalizes well to unseen test-data and is robust over a wide range of hyperparameters.
This paper proposes a new technique for detecting adversarial examples by introducing "fingerprints" into the landscape while training, and exploiting the fingerprints at test time to detect adversarial examples. The idea is novel and the paper is well-written, but concerns about gradient masking prevent me from recommending acceptance just yet.
Positives: 
The paper is extremely well-written, and the approach is clear and presented well. The authors also clearly put significant effort into the evaluation, and accurately/consistently describe threat models that they consider. The approach is also clearly novel, and is interesting. 
Concerns:
My biggest concern is that this detection mechanism masks gradients in its loss function. The two reasons I strongly believe this is the case are (a) Figure 5 and (b) the authors themselves state that their loss is highly non-convex and that no gradient-based method may be able to find a solution. This, however, does not guarantee robustness (see [1] for why such “unfriendly” landscapes can usually be circumvented)
Some concrete evaluation concerns and experiments that the authors can run to alleviate them:
- Figure 5 shows adversarial robustness even against eta = 0.25---at this value of epsilon, the attacker should be able to create realistic images of other classes (even without PGD), so this suggests that the loss is somehow making examples hard to find rather than removing them. The authors should address this issue.
- Showing that at a sufficiently high eta attacks start to succeed is also useful
- Running SPSA, CarliniL2-FP, and PGD for *many* more iterations and using *many* more steps for binary search (right now it looks like the binary search is looking in a space of size 10^6 with 10 steps, which only has a granularity of about 5k, which means you never see any value < 5k in a bisection search, which casts into doubt all of these results)
- The AUC should monotonically degrade with eta (this is another indication the attacks might not be running for long enough)
- The method does not seem to be specific to L-infinity constraint. To this end, a version of Figure 5 in the L2 case would be extremely useful in understanding the detection method.
I also apologize if some of these concerns about gradient masking seem unsubstantiated; that said, I tried to run the code given in the paper, but got several OOM and other errors (utils modules not found, and PyTorch deprecations), even on a machine with 8 12GB-memory GPUs. If the authors can provide instructions for running the code I will be happy to test it and alleviate some of my own concerns. I also tried to reimplement the approach, but did not manage to finish before the review deadline. If I am able to reimplement the approach I will update my review accordingly.
Some smaller comments on the paper:
A consolidated set of tables for attack parameters in an appendix is needed
- Page 4 last paragraph line 4 find the subset that “satisfies” instead of “satisfy”
- Page 5 paragraph 1 line 1 defender,for needs a space before for
- Page 5 paragraph right before theorem 1 last line Here, for detection needs a , after detection 
- Page 7 paragraph 1 line 2 (2 hidden layers the 2 should be written two
- Page 7 second last paragraph line 2 “is chosen” instead of “are chosen”
- Page 7 last paragraph line 2 “across attacks” needs a , after 
- Page 9 table 6 label line 2 “does not shown” should be “show” instead of “shown” 
- Page 9 last line “measure of robustness” remove "of"
[1] https://arxiv.org/pdf/1802.00420.pdf