Graph prediction methods that work closely with the structure of the data, e.g., graph generation, commonly ignore the content of its nodes. On the other hand, the solutions that consider the node's information, e.g., classification, ignore the structure of the whole. And some methods exist in between, e.g., link prediction, but predict the structure piece-wise instead of considering the graph as a whole. We hypothesize that by jointly predicting the structure of the graph and its nodes' features, we can improve both tasks. We propose the Graph Learning Network (GLN), a simple yet effective process to learn node embeddings and structure prediction functions. Our model uses graph convolutions to propose expected node features, and predict the best structure based on them. We repeat these steps sequentially to enhance the prediction and the embeddings. In contrast to existing generation methods that rely only on the structure of the data, we use the feature on the nodes to predict better relations, similar to what link prediction methods do. However, we propose an holistic approach to process the whole graph for our predictions. Our experiments show that our method predicts consistent structures across a set of problems, while creating meaningful node embeddings.
This paper proposes a supervised learning method for predicting the connectivity of a graph based on both the features of nodes in the graph as well as the overall graph structure, rather than just the structure of the graph or just the node features. The approach is evaluated on two synthetic datasets, a “community” dataset and a “geometric figures” dataset.
Unfortunately, I do not think this paper as it stands currently is ready for publication at ICLR for the following reasons: (1) the comparison and discussion with prior literature is lacking; (2) the experiments are rather weak; and (3) the significance and novelty of the method itself seems limited.
1. I am very surprised there was no discussion of [1] (or even better, a comparison to), which is another method which uses information about the full graph (via message-passing) to infer the connectivity of the graph in an unsupervised way. The discussion of the literature on graph neural networks in general is a bit weak, missing important references such as [2-4]. Such approaches, especially message-passing style approaches like [4], do exactly what the current paper suggests has not been done: they make predictions based on information in the nodes while considering the structure of the graph as a whole (via message-passing). Although [4] does not explicitly make edge predictions the approach is straightforward to generalize to making edge predictions, see [5].
2. The experiments in the paper only test the proposed method on very toy domains, and thus feel weak. The results in Table 2, for example, suggest that the proposed method has reached ceiling-level performance and thus to really tell the difference between GLN_f, GLN_c, and any other methods, more difficult problems are called for. The geometric figures dataset, in particular, does not seem to me like it would test the claim that the paper would like to make: that it is important to take into account the fully structure of the graph when predicting edges. Indeed, there is a very simple rule that can be applied in the geometric figures case which does not use global graph information (if the two nodes have the same color, connect them, if they are different colors, do not connect them). It is therefore unsurprising that GLN_c actually does slightly better than GLN_f (according to Table 2) on geometric figures.
Additionally, the experiments do not provide much insight into the architecture itself. For example, the present architecture is meant to repeat the embedding and link-prediction steps some number of times, and in the experiments it seems that these steps are repeated four times. But how important is the repetition in practice? It would be nice to see the effect of repetitions on final performance, to demonstrate whether this is in fact an important component of the model or not. Similarly, there are several different loss functions but it is not obvious to what extent these losses contribute to the final performance of the model. It would be nice if there could be some ablation studies that train the model with different combinations of losses to see which are actually important.
3. Finally, I have some concerns about the model itself. If I understand correctly, both f_l and c_l depend on a number of learned parameters which is a function of the number of nodes in the graph. This is unfortunate, as one of the strengths of the graph neural network approach is that GNNs usually have a number of parameters that is independent of the size of the graph, thus allowing GNNs to scale to graphs of arbitrary size. However, that is not the case in this model. Moreover, the architecture of f_l and c_l do not seem particularly novel. f_l just involves passing the node embeddings through a MLP to produce the link predictions. c_l involves something closer to message-passing, though where weighted combinations are learned on a per-node basis (rather than sharing the same function across all nodes). This could be interesting, even though it sacrifices the scale-free nature of GNNs, if it could be shown to actually outperform existing GNN approaches on more realistic datasets. However, given the lack of experiments demonstrating this, it is hard to say how significant the approach is.
[1] Kipf, Fetaya, Wang, Welling & Zemel (2018). Neural Relational Inference for Interacting Systems. ICML 2018.
[2] Gori, M., Monfardini, G., and Scarselli, F. (2005). A new model for learning in graph domains. IJCNN 2005.
[3] Scarselli, F., Gori, M., Tsoi, A. C., Hagenbuchner, M., and Monfardini, G. (2009). The graph
neural network model. IEEE Transactions on Neural Networks, 20(1):61–80.
[4] Gilmer, J., Schoenholz, S. S., Riley, P. F., Vinyals, O., and Dahl, G. E. (2017). Neural message passing for quantum chemistry. arXiv preprint arXiv:1704.01212.
[5] Battaglia, P. W., Hamrick, J. B., Bapst, V., Sanchez-Gonzalez, A., Zambaldi, V., ... Pascanu, R. (2018). Relational inductive biases, deep learning, and graph networks. arXiv preprint arXiv:1806.01261.