Principal Differences Analysis: Interpretable Characterization of Differences between Distributions
We introduce principal differences analysis (PDA) for analyzing differences between high-dimensional distributions. The method operates by finding the projection that maximizes the Wasserstein divergence between the resulting univariate populations. Relying on the Cramer-Wold device, it requires no assumptions about the form of the underlying distributions, nor the nature of their inter-class differences. A sparse variant of the method is introduced to identify features responsible for the differences. We provide algorithms for both the original minimax formulation as well as its semidefinite relaxation. In addition to deriving some convergence results, we illustrate how the approach may be applied to identify differences between cell populations in the somatosensory cortex and hippocampus as manifested by single cell RNA-seq. Our broader framework extends beyond the specific choice of Wasserstein divergence.
The proposed approach is motivated by the "Cramer-Wold device", which ensures the existence of a linear projection that differentiates two distributions. The authors apply the Wasserstein metric directly on samples from both distributions, and show favorable theoretical properties of such an approach under reasonable assumptions (such as bounded domain variables). The analysis is extended to distributions that differ only in a few dimensions, by considering sparse projections. The resulting computational problem turns out to be non-convex. The authors propose a semidefinite program relaxation and further pre-processing to estimate a solution. Empirical evaluation is provided on simulated and real datasets, including an application to bioinformatics.The paper is well written and clearly lays out the main ideas. The paper dresses a problem of interest to the community using a novel technique and sufficient theoretical and empirical validation. Minor issues: - I would encourage the authors to include more description and legends in each figure so they are more self contained and simplify interpretation. - the term "Cramer-Wold device" seems to be fairly non-standard outside of the reference [10]. Would the authors prefer the more standard "Cramer-Wold theorem"?Suggestions for future work: The presented theoretical results are concerned with the sample complexity as compared to the optimal linear projection. It would also be useful to characterize the relationship between the induced divergences and more standard divergences computed directly between the high dimensional distributions, as this would help clarify the convergence in distribution implied by the Cramer Wold Theorem.