The recent advancements in graph neural networks (GNNs) have led to state-of-theart performances in various applications, including chemo-informatics, questionanswering systems, and recommender systems. However, scaling up these methods to huge graphs such as social network graphs and web graphs still remains a challenge. In particular, the existing methods for accelerating GNNs are either not theoretically guaranteed in terms of approximation error, or they require at least a linear time computation cost. In this study, we reveal the query complexity of the uniform node sampling scheme for GraphSAGE, the graph attention networks (GAT), and the graph convolutional networks (GCN). The key advantage of our analysis is that the complexity is completely independent of the numbers of the nodes, edges, and neighbors of the input and depends only on the error tolerance and confidence probability while providing theoretical guarantee for the approximation error. To the best of our knowledge, this is the first work to give a theoretical guarantee of approximation for GNNs within a constant time complexity. Through experiments using synthetic and real-world datasets, we demonstrate the speed and precision of the node sampling scheme and validate our theoretical results.
In this paper, the authors provide a theoretical framework for characterizing the approximation guarantees provided by node sampling to estimate embeddings in various GNN architectures. In particular, they prove several PAC learning-style bounds on the embedding and gradient estimation when using node sampling approaches. They also observe that since the number of nodes selected for sampling is not dependent on the size of the graph, this amounts to a constant time operation for determining embedding and gradient estimates.
Importantly, a variety of existing works have already proposed node sampling (“constant-time GNNs”), so the contribution of this work lies in the theoretical bounds. A moderate set of experiments suggests the empirical behavior follows the theoretical expectations.
Overall (and described in the comments below), I believe this paper provides some interesting theoretical results for an approach that is widely-used in practice. However, I believe the title and abstract are misleading; this paper does not suggest some new sampling strategy; it only gives a theoretical approximation error for existing approaches and network architectures.
Comments
While the authors do make this clear later in the paper, the contributions of this work are only the theoretical bounds; the node sampling algorithms have been proposed in existing work. Both the title and abstract are misleading.
Still, the theoretical bounds do provide an interesting understanding of an approach which is commonly used in practice.
It would be helpful to give the intuition behind the assumptions and proofs. Currently, it is difficult for a non-expert to follow the reasoning and practical implications of much of this work.
Currently, it is not clear if these results are “constructive”, in the sense that they do not obviously suggest some new sampling strategy, etc. Of course, it is still interesting to characterize commonly-used practices, but one does wish the theory suggested, say, a better sampling strategy, etc.
Typos, etc.
“r”, the number of sampled nodes, is not defined in the main body of the paper.
The text in Footnote 1 is a little confusing, in that the text refers to approximating the embedding, while the text in the footnote describes a prediction problem. A reader could easily interpret this to mean that the error on the downstream prediction problem will be bounded (which is not what this work shows).
The text mentions that O_G (v,i) refers to the i^th “neighborhood” of v. Presumably, this should be “neighbor”.
“we take” -> “We take”