MixUp is a data augmentation scheme in which pairs of training samples and their corresponding labels are mixed using linear coefficients. Without label mixing, MixUp becomes a more conventional scheme: input samples are moved but their original labels are retained. Because samples are preferentially moved in the direction of other classes we refer to this method as directional adversarial training, or DAT. We show that under two mild conditions, MixUp asymptotically convergences to a subset of DAT. We define untied MixUp (UMixUp), a superset of MixUp wherein training labels are mixed with different linear coefficients to those of their corresponding samples. We show that under the same mild conditions, untied MixUp converges to the entire class of DAT schemes. Motivated by the understanding that UMixUp is both a generalization of MixUp and a form of adversarial training, we experiment with different datasets and loss functions to show that UMixUp provides improved performance over MixUp. In short, we present a novel interpretation of MixUp as belonging to a class highly analogous to adversarial training, and on this basis we introduce a simple generalization which outperforms MixUp.
This paper proposes a novel data augmentation method, untied MixUp (UMixUp), which is a general case of both MixUp and Directional Adversarial Traning (DAT). DAT is referred to in this paper as a scheme that only input feature vectors are mixed, while MixUp also incorporates their corresponding labels. The authors provide a theoretical discussion that both DAT and UMixUp converges to be equivalent to each other when the number of training samples becomes infinity. Experimental results on Cifar 10, Cifar 100, MNIST, and Fashion MNIST show quantitative comparisons among the baseline, MixUp, and UMixUp.
According to the author guideline,
> There will be a strict upper limit of 10 pages for the main text. Reviewers will be instructed to apply a higher standard to papers in excess of 8 pages.
The authors use nine pages. Therefore, the review should be more careful about its quality.
Currently, I have three major concerns that keep me from judging this paper acceptable in ICLR 2020.
First, the authors failed to cite two closely related papers below:
- Tokozume et al., LEARNING FROM BETWEEN-CLASS EXAMPLES FOR DEEP SOUND RECOGNITION. ICLR, 2018.
- Tokozume et al., Between-class Learning for Image Classification. CVPR, 2018.
The first one is published in the previous ICLR and mixing two samples belonging to different classes. The second one is an application to image classification using ImageNet dataset, which is larger than the dataset used in this paper.  What's more important is that both papers propose that the mixing ratio of two samples is not linearly but depending on the strength of their signals. Since UMixUp is also focusing on the mixing ratio between two training samples, Between-Class Learning should have been compared to the proposed method.
Secondly, the theoretical discussion is not so fascinating. Actually, both MixUp and UMixUp are shown to converge to DAT when the number of training samples tends to infinity. Data augmentation is, however, performed to remedy the lack of training samples in general. The discussion that the number of training samples is assumed to be large is the opposite situation. 
Thirdly, the experimental results show that the performance gain by UMixUp is relatively small in comparison to that of the original MixUp. There are no ablation studies using different values for alpha and beta, which are parameters for the policy of UMixUp. The authors reported that these values are defined using a heuristic search. Thus, we cannot see if the performance is sensitive to the parameter selection.
I lean to reject this paper because of these concerns. I'm looking forward to seeing the revised version in another conference.