B test: A Non parametric, Low Variance Kernel Two sample Test
We propose a family of maximum mean discrepancy (MMD) kernel two sample tests that have low sample complexity and are consistent. The test has a hyperparameter that allows one to control the tradeoff between sample complexity and computational time. Our family of tests, which we denote as B tests, is both computationally and statistically efficient, combining favorable properties of previously proposed MMD two sample tests. It does so by better leveraging samples to produce low variance estimates in the finite sample case, while avoiding a quadratic number of kernel evaluations and complex null hypothesis approximation as would be required by tests relying on one sample U statistics. The B test uses a smaller than quadratic number of kernel evaluations and avoids completely the computational burden of complex null hypothesis approximation while maintaining consistency and probabilistically conservative thresholds on Type I error. Finally, recent results of combining multiple kernels transfer seamlessly to our hypothesis test, allowing a further increase in discriminative power and decrease in sample complexity.
NIPS