Recently popularized graph neural networks achieve the state-of-the-art accuracy on a number of standard benchmark datasets for graph-based semi-supervised learning, improving significantly over existing approaches. These architectures alternate between a propagation layer that aggregates the hidden states of the local neighborhood and a fully-connected layer. Perhaps surprisingly, we show that a linear model, that removes all the intermediate fully-connected layers, is still able to achieve a performance comparable to the state-of-the-art models. This significantly reduces the number of parameters, which is critical for semi-supervised learning where number of labeled examples are small. This in turn allows a room for designing more innovative propagation layers. Based on this insight, we propose a novel graph neural network that removes all the intermediate fullyconnected layers, and replaces the propagation layers with attention mechanisms that respect the structure of the graph. The attention mechanism allows us to learn a dynamic and adaptive local summary of the neighborhood to achieve more accurate predictions. In a number of experiments on benchmark citation networks datasets, we demonstrate that our approach outperforms competing methods. By examining the attention weights among neighbors, we show that our model provides some interesting insights on how neighbors influence each other.
The paper proposes a semi supervised learning algorithm for graph node classification. The Algorithm is inspired from Graph Neural Networks and more precisely graph convolutional NNs recently proposed by ref (Kipf et al  2016)) in the paper.  These NNs alternate 2 types of layers:  non linear projection and diffusion, the latter incorporates the graph relational information by constraining neighbor nodes to have close representations according to some “graph metrics”. The authors propose a model with simplified projection layers and more sophisticated diffusion ones, incorporating a simple attention mechanism. Experiments are performed on citation textual datasets. Comparisons with published results on the same datasets are presented.
The paper is clear and develops interesting ideas relevant to semi-supervised graph node classification. One finding is that simple models perform as well as more complex ones in this setting where labeled data is scarce. Another one is the importance of integrating relational information for classifying nodes when it is available. The attention mechanism itself is extremely simple, and learns one parameter per diffusion layers. One parameter weights correlations between node embeddings in a diffusion layer. I understand that you tried more complex attention mechanisms, but the one finally selected is barely an attention mechanism and rather a simple “importance” weight. This is not a criticism, but this makes the title somewhat misleading. The experiments show that the proposed model is state of the art for graph node classification. The performance is on par with some other recent models according to table 2. The other tests are also interesting, but the comparison could have been extended to other models e.g. GCN.
You advocate the role of the diffusion layers, and in the experiments you stack 3 to 4 such layers. It would be interesting to have indications on the compromise performance/ number of diffusion layers and on the evolution of these performances when adding such layers.
The bibliography on semi-supervised learning in graphs for classification is light and should be enhanced.
Overall this is an interesting paper with nice findings. The originality is however relatively limited in a field where many recent papers have been proposed, and the experiments need to be completed.