Protein Interface Prediction using Graph Convolutional Networks
We consider the prediction of interfaces between proteins, a challenging problem with important applications in drug discovery and design, and examine the performance of existing and newly proposed spatial graph convolution operators for this task. By performing convolution over a local neighborhood of a node of interest, we are able to stack multiple layers of convolution and learn effective latent representations that integrate information across the graph that represent the three dimensional structure of a protein of interest. An architecture that combines the learned features across pairs of proteins is then used to classify pairs of amino acid residues as part of an interface or not. In our experiments, several graph convolution operators yielded accuracy that is better than the state-of-the-art SVM method in this task.
The authors present a novel method for performing convolutions over graphs, which they apply to predict protein interfaces, showing clear improvements over existing methods. The methodological contribution is strong, and the paper mostly clearly written. However, the authors should evaluate compute costs and more alternative methods for performing graph convolutions.
Major comments
=============
The authors highlight that their method differs from related methods by being designed for pairs (or collections) of graphs (l104). However, the two protein graphs a convolved separately with shared weights, and the resulting node features merged afterward. Any other method can be used to convolve proteins before the resulting features are merged. The authors should clarify or not highlight this difference between their and existing methods.
The authors should compare their method to a second graph convolutional network apart from DCNN, e.g. Schlichtkrull et al or Duvenaud et al.
The authors should clarify if they used the features described in section 3.1.1 as input to all methods. For a fair comparison, all methods should be trained with the same input features.
The authors should compare the memory usage and runtime of their method to other convolutional methods. Does the method scale to large proteins (e.g. >800 residues each) with over 800^2 possible residue-residue contacts? The authors should also briefly describe if computations can be parallelized on GPUs and their method be implemented as a user friendly ‘graph convolutional layer’.
The authors should describe more formally (using equations) how the resulting feature vectors are merged (section 2.3). The authors should also clarify how they are dealing with variable-length proteins which result in a variable number of feature vectors. Are the merged feature vectors processed independently by a fully connected layer with shared weights? Or are feature vectors concatenated and flattened, such that the fully connected layer can model interactions between feature vectors as suggested by figure 2? The authors should also clarify if the same output layer is applied independently to feature vectors or jointly.
Section 3.2: The authors should describe more clearly which hyper-parameters were optimized, both for GCN, PAIRpred, and DCNN. For a fair comparison, the most important hyper-parameters of all methods must be optimized.
l221-225: The authors used the AUC for evaluation. Since labels are highly unbalanced, the authors should also compare and present the area under precision-recall curve. The authors should also describe if performance metrics were computed per protein complex as suggested by figure 3, or across complexes.
Minor comments
=============
l28-32: This section should be toned down since the authors use some of the mentioned ‘hand-crafted’ features as input to their own model.
l90: The authors should clarify what they mean by ‘populations’.
l92-99: The authors should mention that a model without downsampling results in higher memory and compute costs. 
l95: typo ‘classiy’
Table 1: The authors should clarify that ‘positive examples’ and ‘negative examples’ are residues that are or are not in contact, respectively.
l155-162: The authors should mention the average protein length, which is important to assess compute costs (see comment above).
l180: The authors should clarify if the ‘Gaussian function’ corresponds to the PDF or CDF of a normal distribution.
Table 2: The authors should more clearly describe in the main text how the ‘No Convolutional’ model works. Does it consist of 1-4 fully connected layers that are applied to each node independently, and are the resulting feature vectors merged in the same way as in GCN? If so, which activation function was used and how many hidden units? Is it equivalent to GCN with a receptive field of 0?
Since the difference between the mean and median AUC is not clear by looking at figure 3, the authors should plot the mean and median as vertical lines. Since the figure is not very informative, I suggest to move it to the appendix and to show instead more protein complexes as in figure 4.
l192: Did the authors both downsample negative pairs (caption table 1) and give 10x higher weight to positive pairs? If so, it should be pointed out in the text that two techniques were used to account for class-imbalance.
l229: What are ‘trials’? Did the authors use different train/test split, or did they train models multiple times to account for the randomness during training?