This paper is concerned with the defense of deep models against adversarial attacks. We develop an adversarial detection method, which is inspired by the certificate defense approach, and captures the idea of separating class clusters in the embedding space to increase the margin. The resulting defense is intuitive, effective, scalable, and can be integrated into any given neural classification model. Our method demonstrates state-of-the-art (detection) performance under all threat models.
Summary
========
This paper proposes a defense against adversarial examples that detects perturbed inputs using kernel density estimation. The paper uses a combination of known (and often known to be broken) techniques, and does not provide a fully convincing evaluation.
I lean towards rejection of this paper.
Detailed comments
=================
The idea of increasing robustness by maximizing inter-class margins and minimizing intra-class variance is fairly natural, but the author's discussion of their approach (mainly in sections 1 and 2) is very hand-wavy and relies on a lot of general intuitions and unproven claims about neural networks.
For example, in the introduction, the authors claim:
"A trained deep classification model tends to organize instances into clusters in the embedding space, according to class labels. Classes with clusters in close proximity to one another, provide excellent opportunities for attackers to fool the model. This geometry explains the tendency of untargeted attacks to alter the label of a given image to a class adjacent in the embedding space as demonstrated in Figure 1a."
First, a t-SNE representation is just a 2D projection of high-dimensional data that is useful for visualization purposes, and one should be careful when extrapolating insights about the actual data from it. For example, distances in the 2D projection do not necessarily correspond directly to distances in the embedding space. 
The claim that untargeted attacks lead to a "nearby" cluster are hard to verify given just Figure 1. First, the colors of the labels between 1a and 1b do not seem to match (e.g., Dog is bright green in 1b but this color does not appear in 1a). If the other colors match, then this would seem to suggest that trucks (purple) often get altered to ships (orange). Yet, the two clusters are quite far apart in 1a. It seems hard to say something qualitative here. An actual experiment comparing distances in the embedding space and the tendency of untargeted attacks to move from one class to another would be helpful.
The color scheme in Figure 1b is also unclear. A color bar would help here at the very least.
These observations are then used to justify increasing cluster distance while minimizing cluster variance, but it would be nice to see a more formal argument relating these concepts to the embedding distance.
The technique proposed in Section 3.2. to reduce variance loss estimates each class' variance on each batch. Would  this still work for a dataset with a large number of classes (e.g., ImageNet)? For such a dataset, each class will be present less than once in expectation in each batch, which seems problematic.
The plots in Figure 2 don't give much of a sense of how the combination of the different proposed techniques is better than any individual technique. The evaluation compares PDM to RCE, but from Figure 2 one could guess that variance reduction alone (2c) performs very similarly to PDM (2e). An ablation study showing the contribution of each of the individual techniques would be helpful.
The evaluation section could be improved significantly. FGSM, JSMA, and to some extent BIM, are not recommended attacks for evaluating robustness. The gray-box and black-box threat model evaluations are also not the most interesting here. Instead, and following the recommendations of Carlini et al. (2019), the evaluation should:
- Propose an adaptive attack objective, tailored for the proposed defense in a white-box setting. The authors do this to some extent, by re-using the attack objective from Carlini & Wagner 2017, which targets KDE. It would still be good to provide additional explanations about how the hyperparameters for this attack were set.
- Optimize this objective using both gradient-based and gradient-free attacks
- As the proposed defense is attack-agnostic, I also suggest trying it out on rotation-translation attacks, as the worst-case attack can always be found by brute-force search
Other
=====
- The citations for adversarial training in the 2nd paragraph of the intro are unusual. Standard references here are for sure the first two below, and maybe some of the other three as is relevant to your work
    - Szegedy et al. 2013: "intriguing properties of neural networks"
    - Goodfellow et al. 2014: "Explaining and harnessing adversarial examples"
    - Kurakin et al. 2016: "Adversarial Machine Learning at Scale"
    - Madry et al. 2017: "Towards deep learning models resistant to adversarial attacks"
    - Tramer et al. 2017: "Ensemble Adversarial Training"
- The Taylor approximation in (1) does not seem to be well defined. The Jacobian of F is a matrix, so it isn't clear what evaluating that matrix at a point x means.
- The "greater yet similar" symbol (e.g., in equation (4)) should be defined formally.