We introduce a novel geometric perspective and unsupervised model augmentation framework for transforming traditional deep (convolutional) neural networks into adversarially robust classifiers. Class-conditional probability densities based on Bayesian nonparametric mixtures of factor analyzers (BNP-MFA) over the input space are used to design soft decision labels for feature to label isometry. Classconditional distributions over features are also learned using BNP-MFA to develop plug-in maximum a posterior (MAP) classifiers to replace the traditional multinomial logistic softmax classification layers. This novel unsupervised augmented framework, which we call geometrically robust networks (GRN), is applied to CIFAR-10, CIFAR-100, and to Radio-ML (a time series dataset for radio modulation recognition). We demonstrate the robustness of GRN models to adversarial attacks from fast gradient sign method, Carlini-Wagner, and projected gradient descent.
The paper is working on a robust classifier that consists of two stages. The first stage performs unsupervised conditional kernel density estimates (KDE) of the covariate vectors, and the second stage is feature extractions and classification. I appreciate the authors' efforts to clarify the intuition, but more technical details and experiments can be provided to support their arguments. My questions and comments are below.
1. Page 2. "this means the stochastic gradient descent training algorithm minimizing..." Is the problem because of SGD or the structure of NN? I think the reason might be the latter, consider logistic regression, which can be seen as a single-layer NN, does not suffer such a problem. 
2. I know the KDE part is from an existing paper, but more technical details can make the paper clearer and some statements are questionable. Specifically, what basis vectors are used for (3)? Is it really speedy and scalable (Page 4, Section 3.1) for BNP-MFA if using Gibbs sampling? Is it the reason why the experiments in Table 1 is incomplete?
3. For Eqn (7), how do you calculate \beta's to "scale the correct class label higher than incorrect classes for the cases...?"
4. Is the proposed model robust to all kinds of attacks, like gradient based noise, and outliers which locates far away from the corresponding cluster?
5. Can you provide some experiments to show the advantage over other approaches?[1]
I highly encourage the use of BNP KDE which has many advantages as stated in the paper. But the authors may have to solve the problem of scalability and show advantages over other approaches.
[1]Uličný, Matej, Jens Lundström, and Stefan Byttner. "Robustness of deep convolutional neural networks for image recognition." International Symposium on Intelligent Computing Systems. Springer, Cham, 2016.