Practical Methods for Graph Two-Sample Testing
Hypothesis testing for graphs has been an important tool in applied research fields for more than two decades, and still remains a challenging problem as one often needs to draw inference from few replicates of large graphs. Recent studies in statistics and learning theory have provided some theoretical insights about such high-dimensional graph testing problems, but the practicality of the developed theoretical methods remains an open question. In this paper, we consider the problem of two-sample testing of large graphs. We demonstrate the practical merits and limitations of existing theoretical tests and their bootstrapped variants. We also propose two new tests based on asymptotic distributions. We show that these tests are computationally less expensive and, in some cases, more reliable than the existing methods.
This paper studies the problem of two-sample testing of large graphs under the inhomogeneous Erdos Renyi model. This model is pretty generic, and assumes that an undirected edge (ij) is in the graph with probability P_{ij} independently of all other edges. Most generically the parameter matrix P could be anything symmetric (zero diagonal), but common models are stochastic block model or mixed membership stochastic block model, which both result in P being low rank. Suppose there were two random graph distributions, parameterized by matrices P and Q, and the goal is to test whether P = Q or not (the null hypothesis being that they are equal).              They assume that the graphs are vertex-aligned, which helps as it reduces the problem of searching over permutations to align the graphs. If one has many samples from each distribution, and the number of samples goes to infinity, then there are kernel based tests constructed based on comparing the graph adjacency or Laplacian matrices. This paper focuses on the setting where there are only a few samples from each distribution, or even possibly only one sample from each distribution.               When the number of samples per distribution is greater than 1, then the paper uses test statistics previously introduced by Ghoshdastidar, one which looks at the spectral norm of differences between the mean adjacency matrix of the two distributions, and the other which looks at correlation of edges in the average adjacency matrices of the two distributions. The limitation of previous tests was that the theoretical thresholds used in the test are too large (for moderate graph sizes), so they estimate threshold by bootstrapping, which can be quite expensive. The contribution of this paper is to provide an asymptotic analysis of these test statistics as the number of vertices goes to infinity, showing that the test statistic is asymptotically dominated by a standard normal random variable. This suggests an easy to compute threshold to use for the hypothesis test which does not require bootstrapping samples to compute. The asymptotics are as the number of vertices goes to infinity rather than the number of samples going to infinity, but the separation condition between P and Q that the test can distinguish between decreases as the number of samples per distribution increases.               When there is only one sample per distribution, most previous methods are limited to assuming that P and Q are low rank. The authors propose a test based on asymptotic distribution of eigenvalues of a matrix related to the difference between adjacency matrices (but each entry scaled by a term related to the variances of that edge). Basically they show that if one knew the parameter matrices P and Q, then the extreme eigenvalues of matrix C defined in eq (9) follow the Tracy-Widom law, thus the spectral radius of C is a useful test statistic. Since in practice P and Q are not known, the authors then propose to estimate P and Q using community detection algorithms such as spectral clustering, where P and Q are approximated by stochastic block models. I think this last part is more heuristic as the authors did not mention whether the theoretical result in spectral clustering are tight enough to provide analogous results as Theorem 4 in the setting where P and Q are estimated.              I enjoyed reading the paper and found the results quite interesting and clearly presented.              Comment about organization:       I feel that since the formal problem statement pertains to the entire paper (m > 1 and m=1), it shouldn't be a subsection within the m \to \infty section; maybe make the problem statement its own section.