Message-passing neural networks (MPNNs) have been successfully applied to representation learning on graphs in a variety of real-world applications. However, two fundamental weaknesses of MPNNs' aggregators limit their ability to represent graph-structured data: losing the structural information of nodes in neighborhoods and lacking the ability to capture long-range dependencies in disassortative graphs. Few studies have noticed the weaknesses from different perspectives. From the observations on classical neural network and network geometry, we propose a novel geometric aggregation scheme for graph neural networks to overcome the two weaknesses. The behind basic idea is the aggregation on a graph can benefit from a continuous space underlying the graph. The proposed aggregation scheme is permutation-invariant and consists of three modules, node embedding, structural neighborhood, and bi-level aggregation. We also present an implementation of the scheme in graph convolutional networks, termed Geom-GCN, to perform transductive learning on graphs. Experimental results show the proposed Geom-GCN achieved state-of-the-art performance on a wide range of open datasets of graphs.
GEOM-GCN: GEOMETRIC GRAPH CONVOLUTIONAL NETWORKS 
The paper introduces a novel GCN framework, whose purpose is to overcome weaknesses of existing GCN approaches, namely loss of structural neighbor information and failure to capture important dependencies between distant nodes. The paper uses a mapping from nodes to an embedded space and introduces a second type of a neighborhood: a proximity in the embedded space. In the embedded space, a set of relations of nodes is defined. For each node v, the paper uses a 2-stage convolution scheme: 1) for each neighborhood type, the nodes in the same relation with v are combined; 2) the resulting nodes are again combined into a new feature vector. This approach allows one to overcome the issues described above. The experiments show that in most cases the approach outperforms the existing GCN solutions, sometimes with a large gap.
I have the following concerns about the paper:
-- My main concern is the learning time, which is an issue for a straightforward GCN implementation. There were multiple attempts to decrease it (GraphSAGE, FastGCN, etc.). Therefore, I would like to see running times on the presented graphs as well as on relatively large graphs (see e.g. https://arxiv.org/pdf/1902.07153.pdf for candidates). If some techniques were used to make the implementation faster, I would be good to include them in the paper (or, if they are standard, they should be referenced). At the very least, I believe it should be prioritized as a future direction.
-- It’s unclear why we should use the same latent space and the same τ for both N_g and N_s. I would expect that mapping into different spaces could provide better results: the two neighborhood types seem very different, and I don’t see why the neighbors should be aggregated in the same way. If using different spaces doesn’t provide an improvement, an explanation for this would be very useful.
-- α and β are defined and shown in Table 2, but they are never used (as it stands now, α and β can simply be removed). If the results in Table 3 correlate with them, then this dependence should be highlighted. In such case, it would also be better to move α and β to Table 3.
-- The paper uses 3 different node embedding strategies. These strategies can be combined in q with different weights (which can be learned as hyperparameters). Will it produce the best of 3 (or better) result?
-- “We use an embedding space of dimension 2 for ease of explanation” But what τ is used in the real implementation?
-- There are various GCN implementations; however, the comparison is performed with only 2 of them. I would like to see either comparison with more implementations, or the explanation why the comparison with the given two suffices.
-- Is it possible to make the implementation available?
While there are a lot of possible improvements, I believe that some of them can be addressed in a future research, and the paper’s novel approach is noteworthy in itself. My current verdict is 5/10, and I’ll be happy to improve it if the above issues are fixed.
Presentation issues:
-- The notation used in definition of m_v^l is unclear.
-- Why τ is a part of each node’s structural neighborhood? It’s a global function, isn’t it?
-- Introduction: I believe that the exact problems which GCNs solve (e.g. node classification) should be mentioned.
-- The flow in Section 2.1 is a bit weird. Namely, it says “To overcome the first weakness”, but the first wickness wasn’t stated in the previous paragraph (of course, one can deduce it, and it also was defined long ago, but it’s disturbing for a reader).
-- Figure 1B is confusing: it looks like the nodes from N_g(v) lie in a small region around v.
-- I think that splitting Figure 1C into 2 figures would make it clearer.