B-tests: Low Variance Kernel Two-Sample Tests
A family of maximum mean discrepancy (MMD) kernel two-sample tests is introduced. Members of the test family are called Block-tests or B-tests, since the test statistic is an average over MMDs computed on subsets of the samples. The choice of block size allows control over the tradeoff between test power and computation time. In this respect, the B-test family combines favorable properties of previously proposed MMD two-sample tests: B-tests are more powerful than a linear time test where blocks are just pairs of samples, yet they are more computationally efficient than a quadratic time test where a single large block incorporating all the samples is used to compute a U-statistic. A further important advantage of the B-tests is their asymptotically Normal null distribution: this is by contrast with the U-statistic, which is degenerate under the null hypothesis, and for which estimates of the null distribution are computationally demanding. Recent results on kernel selection for hypothesis testing transfer seamlessly to the B-tests, yielding a means to optimize test power via kernel choice.
The authors present an interesting contribution of a 
      block splitted MMD based two-sample test. The authors give a very good and 
      clear motivation in the introduction. It is proposed and described with 
      respect to the two extremes of MMD_u and MMD_l. A recommendation for the 
      block size B is given. Besides the theoretical contribution, the method is 
      convincingly illustrated on several data sets. The work can be promising 
      e.g. towards big data. On the other hand, with respect to the 
      bootstrap, also block based methods have been proposed: A scalable 
      bootstrap for massive data. A. Kleiner, A. Talwalkar, P. Sarkar and M. I. 
      Jordan. arXiv:1112.5016, 2012. It would be good to position the 
      work also with respect to these results. When reading the abstract 
      it is not immediately clear what the authors mean by B-test. I would 
      suggest to already explain in this early stage that B stands for block 
      splitting. Currently, one expects from the name B-test in the title and 
      abstract that is is a completely new approach, which is not really the 
      case. typo: Komogorov should be Kolmogorov When taking 
      sigma = 1, isn't a suitable scaling of the data assumed here? please 
      explain. Eq 13: shouldn't this be rounded to the nearest integer? 
      The choice sigma \in {-15, ..., 10} is unclear to me, what do you 
      mean and why negative values? Is this choice crucial or not? 
      Please explain in more detail what you mean by multiple kernels. 
      The authors have carefully answered to the questions. I have 
      updated my score.