This paper studies learning the representations of whole graphs in both unsupervised and semi-supervised scenarios. Graph-level representations are critical in a variety of real-world applications such as predicting the properties of molecules and community analysis in social networks. Traditional graph kernel based methods are simple, yet effective for obtaining fixed-length representations for graphs but they suffer from poor generalization due to hand-crafted designs. There are also some recent methods based on language models (e.g. graph2vec) but they tend to only consider certain substructures (e.g. subtrees) as graph representatives. Inspired by recent progress of unsupervised representation learning, in this paper we proposed a novel method called InfoGraph for learning graph-level representations. We maximize the mutual information between the graph-level representation and the representations of substructures of different scales (e.g., nodes, edges, triangles). By doing so, the graph-level representations encode aspects of the data that are shared across different scales of substructures. Furthermore, we further propose InfoGraph*, an extension of InfoGraph for semi-supervised scenarios. InfoGraph* maximizes the mutual information between unsupervised graph representations learned by InfoGraph and the representations learned by existing supervised methods. As a result, the supervised encoder learns from unlabeled data while preserving the latent semantic space favored by the current supervised task. Experimental results on the tasks of graph classification and molecular property prediction show that InfoGraph is superior to state-of-the-art baselines and InfoGraph* can achieve performance competitive with state-of-the-art semi-supervised models.
In this paper, the authors propose a graph-level representation, which extends the existing node-level representation learning mechanism. Besides, both unsupervised and semi-supervised learning are leveraged for InfoGraph and InfoGraph*, receptively. The authors naturally apply Deep Graph Infomax, a contrastive representation learning method, for the whole graph level instead of the previous node embedding learning. The experiments on graph classification and molecular property prediction indicate the effectiveness, even compared to the supervised methods.
The learned graph-level representation looks good to me. Instead of some heuristics based graph-level pooling, the proposed method automatically figure out the best way to produce the fixed-length feature vector in a data-driven approach. Such motivation is pretty reasonable and natural for me. Also, the usage of both unsupervised and semi-supervised learning procedure is well-motivated. Overall speaking, the paper is well written. The definitions of problems, the details of methods, and the settings of experiments are clear to me.
I have some questions and suggestions for the authors:
1. The overall writing looks good to me. Besides, It could be much better if the authors could provide more and better illustrations for the method. Both figures 1 and 2 are not that informative to me, to be honest. I know it could hard to visualize the graph-level representation, but it worth it. There are many steps and equations in the paper, and the illustration could play an essential role in putting all the steps together to demonstrate the big picture.
2. The authors carefully discuss the difference between this submission and concurrent work (Information Maximizing Graph Network) or existing work (Deep Graph Infomax), which helps a lot for the reader to understand the literature better. It could be much better if this submission could be more self-contained. For example, for the semi-supervised learning setting, the authors let the reader read some external papers. I am suggesting that a small section in the appendix could make life much more comfortable.