Diffusion-Convolutional Neural Networks
We present diffusion-convolutional neural networks (DCNNs), a new model for graph-structured data. Through the introduction of a diffusion-convolution operation, we show how diffusion-based representations can be learned from graphstructured data and used as an effective basis for node classification. DCNNs have several attractive qualities, including a latent representation for graphical data that is invariant under isomorphism, as well as polynomial-time prediction and learning that can be represented as tensor operations and efficiently implemented on a GPU. Through several experiments with real structured datasets, we demonstrate that DCNNs are able to outperform probabilistic relational models and kernel-on-graph methods at relational node classification tasks.
The authors propose a novel neural network architecture for prediction problems of graphs. They interpret this as a generalization of convolution beyond grid structures. The method is shown to outperform several kernel methods. It performs better on vertex classification, but not on graph classification.
The experimental results look promising, but I have several concerns regarding the presentation and the experiments.  Presentation: - How exactly is diffusion defined here? This is such a core concept, why is there not a formal definition or at least a citation to previous work? I spent 10 minutes googling this, and I haven't found a single obvious definition. I'm not an expert on graphs / social network analysis, but this still seems to be an important omission. In any case, I assume there is some standard way to featurize a node's neighbors that is used here. - Authors strongly emphasize how this work generalizes convolutions. I think this is only true in the graph setting, not in the edge or vertex prediction settings (e.g. I don't see where is the spatial invariance in those settings). Since the contribution is better performance on vertex classification, I don't think it's appropriate to emphasize this story.  Experiments: - One of the claims is that this architecture handles vertex, edge, graph prediction in a unified way. Why are there no edge experiments? There should be at least a comment on this. Also, can it be used for structure learning? - On vertex prediction, the authors compare against baselines including kernel methods, logistic regression and a CRF. But logistic regression only receives features derived from the vertex itself, not its neighbors, so it's not surprising they don't do well. There needs to be experiments that trains a shallow model on the same signal (i.e. on features that are comparable to the diffusion-based features used here). Then we need to see that the neural network model is more accurate or faster. The same holds for the CRF; the authors don't specify which features it uses, but even if it uses edge features, we need to compare against an equivalent featurization. The authors mention they use up to 5 hops of diffusion, which sounds to me like they're looking at neighbors 5 hops away. The CRF and LR need to receive the same features. - I'm not an expert on neural networks applied to graphs, but I'm surprised there are not other neural-network based models to compare against. Why is something like this work not applicable and/or compared against: https://cs.stanford.edu/people/jure/pubs/node2vec-kdd16.pdf  In conclusion, I find the architecture to be elegant and the paper very well-written (except for omitting the definition of diffusion), but the experiments are not entirely convincing, and this is the main factor by which we can judge an applied paper.  Right now, I think this paper is somewhat borderline, but I am happy to readjust my opinion if the authors can comment on some of the above.  UPDATE:  After reading the response and looking closer to related literature, I am down-grading my review.  I strongly disagree with the authors' claim that "many of [other] techniques were developed in parallel with our own and did not have readily available open-source implementations". For example, look at the node2vec paper that came out in parallel: - They have substantially more extensive experiments - They compare against previous neural network based methods, LINE and DeepWalk - These methods are from 2014 and 2015, so I see no excuse for not comparing against them. In fact, they are not even cited, which I think is unacceptable!  The quality of this paper is much lower than that of LINE, DeepWalk and node2vec, and so I strongly argue for the rejection of this paper.