We consider the fundamental problem of semi-supervised node classification in attributed graphs with a focus on few-shot learning. Here, we propose Structural Regularized Graph Convolutional Networks (SRGCN), novel neural network architectures extending the well-known GCN structures by stacking transposed graph convolutional layers for reconstruction of input features. We add a reconstruction error term in the loss function as a regularizer. Unlike standard regularization such as L 1 or L 2 , which controls the model complexity by including a penalty term depends solely on parameters, our regularization function is parameterized by a trainable neural network whose structure depends on the topology of the underlying graph. The new approach effectively addresses the shortcomings of previous graph convolution-based techniques for learning classifiers in the few-shot regime and significantly improves generalization performance over original GCNs when the number of labeled samples is insufficient. Experimental studies on three challenging benchmarks demonstrate that the proposed approach has matched state-of-the-art results and can improve classification accuracies by a notable margin when there are very few examples from each class.
This paper proposes to regularize the training of graph convolutional neural networks by adding a reconstruction loss to the supervised loss. Results are reported on citation benchmarks and compared for increasing number of labeled data.
The presentation of the paper could be significantly improved. Details of the proposed model are missing and the effects of the proposed regularization w.r.t. other regularizations are not analyzed.
My main concerns are related to the model design, the novelty of the approach (adding a reconstruction loss) and its experimental evaluation.
Details / references of the transposed convolution operation are missing (see e.g. https://ieeexplore.ieee.org/document/7742951). It is not clear what the role of the transposed convolution is in that case. It seems that the encoder does not change the nodes nor the edges of the graph, only the features, and the filters of the transposed convolution are learnt. If the operation is analogous to the transposed convolution on images, then given that the number of nodes in the graph does not change in the encoder layers (no graph coarsening operations are applied), then learning an additional convolution should be analogous (see e.g. https://arxiv.org/pdf/1603.07285.pdf Figure 4.3.). Could the authors comment on that?
Details on the pooling operation performed after the transposed convolution are missing (see e.g. https://arxiv.org/pdf/1805.00165.pdf, https://arxiv.org/pdf/1606.09375.pdf). Does the pooling operation coarsen the graph? if so, how is it then upsampled to match the input graph?
Figure X in section 2.1. does no exist.
Supervised loss in section 2.2.1 seems to disregard the sum over the nodes which have labels.
\hat A is not defined when it is introduced (in section 2.2.2), it appears later in section 2.3.
Section 2.2.2 suggests that additional regularization (such as L2) is still required (note that the introduction outlines the proposed loss as a combination of reconstruction loss and supervised loss). An ablation study using either one of both regularizers should be performed to better understand their impact. Note that hyper-parameters chosen give higher weight to L2 regularizer.
Section 3 introduces a bunch of definitions to presumably compare GCN against SRGCN, but those measures of influence are not reported for any model.
Experimental validation raises some concerns. It is not clear whether standard splits for the reported datasets are used. It is not clear whether hyper-parameter tuning has been performed for baselines. Authors state "the parameters of GCN and GAT and SRGCN are the same following (Kipf et al; Velickovic et al.)". Note that SRGCN probably has additional parameters, due to the decoder stacked on top of the GCN. Reporting the number of parameters that each model has would provide more insights. Results are not reported following standards of running the models N times and providing mean and std. Moreover, there are no results using the full training set.