Many techniques are developed to defend against adversarial examples at scale. So far, the most successful defenses generate adversarial examples during each training step and add them to the training data. Yet, this brings significant computational overhead. In this paper, we investigate defenses against adversarial attacks. First, we propose feature smoothing, a simple data augmentation method with little computational overhead. Essentially, feature smoothing trains a neural network on virtual training data as an interpolation of features from a pair of samples, with the new label remaining the same as the dominant data point. The intuition behind feature smoothing is to generate virtual data points as close as adversarial examples, and to avoid the computational burden of generating data during training. Our experiments on MNIST and CIFAR10 datasets explore different combinations of known regularization and data augmentation methods and show that feature smoothing with logit squeezing performs best for both adversarial and clean accuracy. Second, we propose an unified framework to understand the connections and differences among different efficient methods by analyzing the biases and variances of decision boundary. We show that under some symmetrical assumptions, label smoothing, logit squeezing, weight decay, mix up and feature smoothing all produce an unbiased estimation of the decision boundary with smaller estimated variance. All of those methods except weight decay are also stable when the assumptions no longer hold.
The paper proposes a feature smoothing technique, which generates virtual data points by interpolating the input space of two randomly sampled examples. The aim is to generate virtual training data points that are close to adversarial examples. Experimental results on both MNIST and Cifar10 datasets show that the proposed method augmented with other regularization techniques are robust to adversarial attacks and obtain higher accuracy when comparing with some testing baselines. Also, the paper presents some theoretical analyses showing that label smoothing, logit squeezing, weight decay, Mixup and feature smoothing all produce small estimated variance of the decision boundary when regularizing the networks. 
The paper is generally well written, and the experiments show promising results. Nevertheless, the proposed method is not very novel, and the method is not comprehensively evaluated with experiments.
Major remarks:
1.	The experiments show that feature smoothing has to combine with other regularizers in order to outperform other testing methods. In this sense the contribution of the feature smoothing along is not clear. For example, without integrating other regularizers, Mixup and feature smoothing obtain very close results for BlackBox-PGD, BlackBoxcw and Clean, as shown in Table 1. In addition, in the paper, the feature smoothing along is only validated on the MNIST (not even tested on Cifar10 in Table2). Consequently, it is difficult to evaluate the contribution of the proposed smoothing technique. 
2.	Experiments are conducted on datasets MNIST and Cifar10 with small number of target classes. Empirically, it would be useful to see how it performs on more complex data set such as Cifar100 or ImageNet.
3.	The argument for why the proposed feature smoothing method works is presented in Theorem4.3 in Section 4.2, but the theorem seems to rely on the assumption that one can add data around the true decision boundary. However, how we can generate samples near the true decision boundary and how we should chose the mixing ratio to attain this goal is not clear to me in the paper. In addition, how we can sure that the adding synthetic data from one class does not collide with manifolds of other classes as suggested in AdaMixup (Guo et al., MixUp as Locally Linear Out-Of-Manifold Regularization)? This is particular relevant if the proposed feature smoothing strategy prefers to create virtual samples close to the true decision boundary.
4.	At the end of page4, the authors claim that both feature smoothing and Mixup generate new data points that are closer to the true boundary. I wonder if the authors could further justify or show that either theoretically or experimentally. 
5.	The proposed method is similar to SMOTE (Chawla et al., SMOTE: Synthetic Minority Over-sampling Technique). In this sense, comparison with SMOTE would be very beneficial.
Minor remarks:
1.	In the paper Mixup, value 1 was carefully chosen as the mixing policy Alpha for Cifar10 (otherwise, underfitting can easily occur as shown in AdaMixUp), and it seems in the paper the authors used a very large value of 8 for Mixup’s Beta distribution, and I did not see the justification for that number in the paper.
2.	Typo in the second paragraph of page2: SHNV should be SVHN