Graph convolutional networks have achieved tremendous success in the tasks of graph node classification. These models could learn a better node representation through encoding the graph structure and node features. However, the correlation between the node labels are not considered. In this paper, we propose a novel architecture for graph node classification, named conditional graph neural fields (CGNF). By integrating the conditional random fields (CRF) in the graph convolutional networks, we explicitly model a joint probability of the entire set of node labels, thus taking advantage of neighborhood label information in the node label prediction task. Our model could have both the representation capacity of graph neural networks and the prediction power of CRFs. Experiments on several graph datasets demonstrate effectiveness of CGNF.
This paper proposed a combination of graph neural networks and conditional random field to model the correlation between node labels in the output.  In typical graph neural nets the predictions for nodes are conditionally independent given the node representations.  This paper proposes to use a CRF to compensate for that.  In terms of the approach, the authors used GCNs to produce unary potentials for the CRF, and have the pairwise potentials on each edge to model the correlation of labels of neighboring nodes.  Learning is done by optimizing pseudo-likelihood and the energy loss, while inference is performed through a couple heuristic processes.
Combining neural nets with CRFs is not a new idea, in particular this has been tried before on image and sequence CRFs.  It is therefore not surprising to see an attempt to also try it for graph predictions.  The main argument for using a CRF is its ability to model the correlations of output labels which was typically treated as independent.  However this is not the case for deep neural networks, as it already fuses information from all over the input, and therefore for most prediction problems it is fine to be conditionally independent for the output, as the dependence is already modeled in the representations.  This is true for graph neural networks as well, if we have a deep graph neural net, then the GNN itself will take care of most of the dependencies between nodes and produce node representations that are suitable for conditionally independent output predictions.  Therefore I’m not convinced that CRFs are really necessary for solving the prediction tasks tried in this paper.
The learning and inference algorithms proposed in this paper are also not very convincing.  CRFs has been studied for a long time, and there are many mature algorithms for learning them.  We could do proper maximum conditional likelihood learning, and use belief propagation to estimate the marginals to compute the gradients.  Zheng et al. (2015) did this for convnets, we could also do this for graph CRFs as belief propagation can be easily converted into message passing steps in the graph neural network.  Pseudo-likelihood training makes some sense, but energy loss minimization doesn’t really make sense and has serious known issues.
On the other hand, the proposed inference algorithms does not have good justifications.  Why not use something standard, like belief propagation for inference again?  Our community has studied graphical models a lot in the last decade and we have better algorithms than the ones proposed in this paper.
Lastly, the experiments are done on some standard but small benchmarks, and my personal experience with these datasets are that it is very easy to overfit, and most of the effort will be put in to prevent overfitting.  Therefore more powerful models typically cannot be separated from overly simple models.  I personally don’t care a lot about the results reported on these datasets.  Besides, there are a lot of questions about the proposed model, but all we get from the experiment section are a few numbers on the benchmarks.  I expect studies about this model from more angles.  One more minor thing about the experiment results: the numbers for GraphSAGE are definitely wrong.
Overall I think this paper tackles a potentially interesting problem, but it isn’t yet enough to be published at ICLR due to its problems mentioned above.