Learning to Pivot with Adversarial Networks
Several techniques for domain adaptation have been proposed to account for differences in the distribution of the data used for training and testing. The majority of this work focuses on a binary domain label. Similar problems occur in a scientific context where there may be a continuous family of plausible data generation processes associated to the presence of systematic uncertainties. Robust inference is possible if it is based on a pivot -a quantity whose distribution does not depend on the unknown values of the nuisance parameters that parametrize this family of data generation processes. In this work, we introduce and derive theoretical results for a training procedure based on adversarial networks for enforcing the pivotal property (or, equivalently, fairness with respect to continuous attributes) on a predictive model. The method includes a hyperparameter to control the tradeoff between accuracy and robustness. We demonstrate the effectiveness of this approach with a toy example and examples from particle physics.
The paper proposes an adversarial framework for training a classifier that is robust to systematic uncertainties. The paper studies a scenario where the data generation process of interest is governed by an unknown random variable Z where the random variable defines a family of data generation distribution (i.e. a different distribution is incurred by a realization of the random variable). It then applies the proposed method to train a classifier whose performance is independent of the random variable. The algorithm was tested on a synthetic dataset and a high energy physics example. No comparison with prior arts were drawn.
Paper strength
The paper extends the generative adversarial framework to a robust classification problem against systematic noise in the data generation mechanism. 
The paper shows that the optimal classifier of the proposed framework is a classifier whose output is independent to the input variable. 
Paper weakness
The paper fails to motivate the problem. Due to the difference of the members in the family of data distribution, samples from some member is naturally easier to classify than members from the other distribution (For example, consider the case that z is very large in Section 5.1). In other words, it is common to have classification outputs correlates with the unknown variable. A classifier would do better for samples from some members and do worse for samples from the other members. By adding the proposed adversary, which encourages the output to be independent with Z. It basically forces the classifier to do worse for the easy cases for matching the performance for the hard cases. The experiment results reports in the paper also suggest this is what happens. The overall accuracy is traded-off for consistency. It is unclear why it is advantageous to do so. A more reasonable approach would be to improve the performance across the members. 
No comparison with prior works. The paper does not report comparison with baseline approaches that also consider the existence of Z when training the classifier. The only baseline model is a plane classifier model that ignores the existence of Z.
Experiment details are missing. Many details such as the data format in the section 5.2 is missing. It is difficult to understand the experiment setting.
The discussions on whether the optimal and pivotal classifier is achievable is very vague. The results that the training curves shown in the paper are away from the optimal values do not mean that the optimal and pivotal classifier is not achievable. The achievability might be just limited by the training of the network capacity. Hence, the statement is not solid.