Model-Powered Conditional Independence Test
We consider the problem of non-parametric Conditional Independence testing (CI testing) for continuous random variables. Given i.i.d samples from the joint distribution f (x, y, z) of continuous random vectors X, Y and Z, we determine whether X ? ? Y |Z. We approach this by converting the conditional independence test into a classification problem. This allows us to harness very powerful classifiers like gradient-boosted trees and deep neural networks. These models can handle complex probability distributions and allow us to perform significantly better compared to the prior state of the art, for high-dimensional CI testing. The main technical challenge in the classification problem is the need for samples from the conditional product distribution f CI (x, y, z) = f (x|z)f (y|z)f (z) -the joint distribution if and only if X ? ? Y |Z. -when given access only to i.i.d. samples from the true joint distribution f (x, y, z). To tackle this problem we propose a novel nearest neighbor bootstrap procedure and theoretically show that our generated samples are indeed close to f CI in terms of total variational distance. We then develop theoretical results regarding the generalization bounds for classification for our problem, which translate into error bounds for CI testing. We provide a novel analysis of Rademacher type classification bounds in the presence of non-i.i.d nearindependent samples. We empirically validate the performance of our algorithm on simulated and real datasets and show performance gains over previous methods.
The authors propose a general test to test conditional independence between three continuous variables. The authors don't have strong parametric assumptions -- rather, they propose a method that uses samples from the joint distribution to create samples that are close to the conditional product distribution in total variation distance. Then, they feed these samples to a binary classifier such as a deep neural net, and after training it on samples from each of the two distributions, use the predictive error of the net to either accept or reject the null hypothesis of conditional independence. They compare their proposed method to those existing in literature, both with descriptions and with experimental results on synthetic and real data.  
Conditional independence testing is an important tool for modern machine learning, especially in the domain of causal inference, where we are frequently curious about claims of unconfoundedness. The proposed method appears straightforward and powerful enough to be used for many causal applications.
Additionally, the paper is well-written. While the details of the theoretical results and proofs can be hard to follow, the authors do a good job of summarizing the results and making the methodology clear to practitioners. 
To me, the most impressive part of the paper is the theoretical results. The proposed method is a somewhat straightforward extension of the method for testing independence in the paper "Revisiting Classifier Two-Sample Tests"  (https://arxiv.org/pdf/1610.06545.pdf), as the authors acknowledge (they modify the test to include a third variable Z, and after matching neighbors of Z, the methods are similar). However, the guarantees of the theoretical results are impressive. 
A few more comments:
-Is there a discrete analog? The bootstrap becomes even more powerful when Z is discrete, right?
-How sensitive is the test on the classification method used? Is it possible that a few changed hyperparameters can lead to different acceptance and rejection results? This would be useful to see alongside the simulations
-Section 1.1 of Main Contributions is redundant -- the authors summarize their approach multiple times, and the extent appears unnecessary. 
-Writing out Algorithm 2 seems unnecessary — this process can be described more succinctly, and has been described already in the paper
-Slight typos — should be “classifier” in line 157, “graphs” in 280