Hierarchical Graph Representation Learning with Differentiable Pooling
Recently, graph neural networks (GNNs) have revolutionized the field of graph representation learning through effectively learned node embeddings, and achieved state-of-the-art results in tasks such as node classification and link prediction. However, current GNN methods are inherently flat and do not learn hierarchical representations of graphs-a limitation that is especially problematic for the task of graph classification, where the goal is to predict the label associated with an entire graph. Here we propose DIFFPOOL, a differentiable graph pooling module that can generate hierarchical representations of graphs and can be combined with various graph neural network architectures in an end-to-end fashion. DIFFPOOL learns a differentiable soft cluster assignment for nodes at each layer of a deep GNN, mapping nodes to a set of clusters, which then form the coarsened input for the next GNN layer. Our experimental results show that combining existing GNN methods with DIFFPOOL yields an average improvement of 5-10% accuracy on graph classification benchmarks, compared to all existing pooling approaches, achieving a new state-of-the-art on four out of five benchmark data sets.
the paper introduces a differentiable clustering operation enabling neural methods acting on graphs to process information in a hierarchical fashion in analogy to CNN pooling layers (but preserving graph symmetry properties).  The paper is very well written, the method is simple (which is not a bad thing!) and widely applicable, and the results are favourable. Building hierarchical representations is an important feature of CNNs and extending this to more general graph datatypes is clearly an important contribution. Overall I strongly support publication of this paper.  I only have minor additional thoughts:  1. When the edges have labels (e.g. single/double/triple bonds in the chemistry example used in the motivation of the method), would equation (4) be extended to transform adjacency matrices for each edge type separately by S? In this case the link prediction objective will need altering. Maybe including an experiment with labelled edges would make the presentation of this otherwise simple method more complete?  2. You have to manually set the maximum number of clusters in each layer (please make this clearer near equation 6 - currently the only explicit reference to this fact is at the end of the caption of Fig2). It is nice that the GNN can learn to use less than the maximum number of clusters, but how sensitive are the results to the cluster count hyperparameters? Are the examples in Fig2 compelling because there is obvious clustering in the graphs? What does the clustering figure look like for data where the clusters are not so obvious?