Graph-structured data such as social networks, functional brain networks, gene regulatory networks, communications networks have brought the interest in generalizing deep learning techniques to graph domains. In this paper, we are interested to design neural networks for graphs with variable length in order to solve learning problems such as vertex classification, graph classification, graph regression, and graph generative tasks. Most existing works have focused on recurrent neural networks (RNNs) to learn meaningful representations of graphs, and more recently new convolutional neural networks (ConvNets) have been introduced. In this work, we want to compare rigorously these two fundamental families of architectures to solve graph learning tasks. We review existing graph RNN and ConvNet architectures, and propose natural extension of LSTM and ConvNet to graphs with arbitrary size. Then, we design a set of analytically controlled experiments on two basic graph problems, i.e. subgraph matching and graph clustering, to test the different architectures. Numerical results show that the proposed graph ConvNets are 3-17% more accurate and 1.5-4x faster than graph RNNs. Graph ConvNets are also 36% more accurate than variational (non-learning) techniques. Finally, the most effective graph ConvNet architecture uses gated edges and residuality. Residuality plays an essential role to learn multi-layer architectures as they provide a 10% gain of performance.
The paper proposes an adaptation of existing Graph ConvNets and evaluates this formulation on a several existing benchmarks of the graph neural network community. In particular, a tree structured LSTM is taken and modified. The authors describe this as adapting it to general graphs, stacking, followed by adding edge gates and residuality.
My biggest concern is novelty, as the modifications are minor. In particular, the formulation can be seen in a different way. As I see it, instead of adapting Tree LSTMs to arbitary graphs, it can be seen as taking the original formulation by Scarselli and replacing the RNN by a gated version, i.e. adding the known LSTM gates (input, output, forget gate). This is a minor modification. Adding stacking and residuality are now standard operations in deep learning, and edge-gates have also already been introduced in the literature, as described in the paper.
A second concern is the presentation of the paper, which can be confusing at some points. A major example is the mathematical description of the methods. When reading the description as given, one should actually infer that Graph ConvNets and Graph RNNs are the same thing, which can be seen by the fact that equations (1) and (6) are equivalent.
Another example, after (2), the important point to raise is the difference to classical (sequential) RNNs, namely the fact that the dependence graph of the model is not a DAG anymore, which introduces cyclic dependencies. 
Generally, a clear introduction of the problem is also missing. What are the inputs, what are the outputs, what kind of problems should be solved? The update equations for the hidden states are given for all models, but how is the output calculated given the hidden states from variable numbers of nodes of an irregular graph?
The model has been evaluated on standard datasets with a performance, which seems to be on par, or a slight edge, which could probably be due to the newly introduced residuality.
A couple of details :
- the length of a graph is not defined. The size of the set of nodes might be meant.
- at the beginning of section 2.1 I do not understand the reference to word prediction and natural language processing. RNNs are not restricted to NLP and I think there is no need to introduce an application at this point.
- It is unclear what does the following sentence means: "ConvNets are more pruned to deep networks than RNNs"?
- What are "heterogeneous graph domains"?