It has been widely recognized that adversarial examples can be easily crafted to fool deep networks, which mainly root from the locally unreasonable behavior nearby input examples. Applying mixup in training provides an effective mechanism to improve generalization performance and model robustness against adversarial perturbations, which introduces the globally linear behavior in-between training examples. However, in previous work, the mixup-trained models only passively defend adversarial attacks in inference by directly classifying the inputs, where the induced global linearity is not well exploited. Namely, since the locality of the adversarial perturbations, it would be more efficient to actively break the locality via the globality of the model predictions. Inspired by simple geometric intuition, we develop an inference principle, named mixup inference (MI), for mixup-trained models. MI mixups the input with other random clean samples, which can shrink and transfer the equivalent perturbation if the input is adversarial. Our experiments on CIFAR-10 and CIFAR-100 demonstrate that MI can further improve the adversarial robustness for the models trained by mixup and its variants.
Notes: 
-Paper claims that adversarial examples stem from locally non-linear behavior.  However, wasn't this the exact opposite of the conclusion of "Explaining and Harnessing Adversarial Examples" (Goodfellow 2015)?  It is cited in this paper but I think the proper conclusion is the opposite of what is written here.  
-Linearity however may still be an important component since it simplifies the problem of adversarial robustness.  
-Many techniques try to introduce adversarial robustness through transformations during inference time.  
-Paper claims that adversarial training induces locally linear behavior - but I'm rather skeptical of this claim.  Think about something like KNN with k=1 and euclidean distance.  This should be L2-robust on the training set, yet very non-linear.  
-I understand the contents of Figure 1, but I'm not sure exactly what conclusion I should draw from it.  
-The novel procedure presented is "Mixup Inference".  This involves classifying using interpolations of test inputs.  
-Two mechanisms are proposed for why this could help.  One is that the magnitude of the perturbation will shrink after doing mixup (although the signal in the original image will also shrink, so I'm not sure if I like this argument).  The second argument is that the adversarial perturbation will have to appear with different random examples which will force the attack to be more "universal" to succeed.  This second argument I find much more compelling.  
-The notation $y_s \sim p_s(y_s)$ is rather abusive since the random variable and the same have the same name but this is common in machine learning.  
-The technique if I understand correctly (Algorithm 1) amounts to using an average of the prediction at the original point along with an average of the mixes going to all other points in the dataset.  
-The paper is a bit slow to explain what distribution lambda will come from - but it effects the algorithm a lot (especially if the distribution is symmetric or asymmetric).  
Comments: 
-There is another paper (Shimada 2019) that also uses interpolation at test time and should be cited here, although I admit that paper is written in a confusing way so the connection may not be immediately obvious: https://arxiv.org/pdf/1906.08412v1.pdf
-I have a suggestion for the organization of the paper that I think would improve it.  I would suggest to first introduce the method in a clear fashion (after motivating it), along with the equations.  Then, *after that*, clearly and separately introduce the analysis of the "optimal linear model": "a well mixup-trained model F can be denoted as a linear function H on the convex combinations of clean examples".  I think that would make the paper much clearer.  
-For example, how can we actually know what G_k() is unless we know the adversarial perturbation (which shouldn't generally be possible during inference)?  I found this discussion to be rather confusing (basically section 3.1) although admittedly it might be my own fault.  
-Why does mixup-inference hurt the clean accuracy by 10% (table 2)?  This seems like quite a lot to me.  Still the degree of robustness does seem impressive.  And I also believe that the obtained robustness of this technique along with AT is state of the art.  
-It might be nice to see examples of attacks on the resulting model, especially the one with the best robustness.  It's possible that Linf-bounded attacks against *this model* will be perceptible, which would support lowering the epsilon-attack-budget (which is actually a good thing for the research field as it's evidence that maybe this epsilon shouldn't be considered practically imperceptible).  
Review: 
This paper was interesting, because it has nice experimental results and seems like a good idea, but I feel like the paper needs to be improved.  The biggest issue is that the paper repeatedly claims that adversarial robustness can be improved by making networks more linear, yet I believe that this is the opposite of what prior work has found.  I also found the exposition of the idea to be confusing as it simultaneously introduces the idea and an analysis of the technique - I would much prefer if the technique were introduced first and the analysis under some optimal assumptions moved to a different section.