While there has been an explosion in the number of experimentally determined, atomically detailed structures of proteins, how to represent these structures in a machine learning context remains an open research question. In this work we demonstrate that representations learned from raw atomic coordinates can outperform hand-engineered structural features while displaying a much higher degree of transferrability. To do so, we focus on a central problem in biology: predicting how proteins interact with one another-that is, which surfaces of one protein bind to which surfaces of another protein. We present Siamese Atomic Surfacelet Network (SASNet), the first end-to-end learning method for protein interface prediction. Despite using only spatial coordinates and identities of atoms as inputs, SASNet outperforms state-of-the-art methods that rely on hand-engineered, high-level features. These results are particularly striking because we train the method entirely on a significantly biased data set that does not account for the fact that proteins deform when binding to one another. Demonstrating the first successful application of transfer learning to atomic-level data, our network maintains high performance, without retraining, when tested on real cases in which proteins do deform.
For the task of predicting interaction contact among atoms of protein complex consisting of two interacting proteins, the authors propose to train a Siamese convolutional neural network, noted as SASNet, and to use the contact map of two binding proteins’ native structure.
The authors claim that the proposed method outperforms methods that use hand crafted features; also the authors claim that the proposed method has better transferability.
My overall concern is that the experiment result doesn’t really fully support the claim in the two aspects: 1) the SASNet takes the enriched dataset as input to the neural net but it also uses the complex (validation set) to train the optimal parameters, so strictly it doesn’t really fit in the “transfer” learning scenario. Also, the compared methods don’t really use the validation set from the complex data for training at all. Thus the experiment comparison is not really fair. 2) The experiment results include standard errors for different replicates where such replicates correspond to different training random seeds (or different samples from the enriched set?), however, it doesn’t include any significance of the sampling. Specifically, the testing dataset is fixed. A more rigorous setting is to, for N runs, each run splitting the validation and testing set differently.
Since this paper is an application paper, rather than a theoretical paper that bears theoretical findings, I would expect much more thorough experimental setup and analysis. Currently it is still missing discussion such as, when SASNet would perform better and when it would perform worse, what it is that the state of the art features can’t capture while SASNet can. Moreover, it is the prediction performance that matters to such task, but the authors remove the non-structure features from the compared methods. Results and discussion about how the previous methods with full features perform compared to SASNet, and also how we can include those features into SASNet should complete the paper.
Overall the paper is well written, and I do think the paper could be much stronger the issues above are addressed.
Some minor issues:
1)	on page 4, Section 3, the first paragraph, shouldn’t “C_p^{val} of 55” be “C_p^{test} of 55”?
2)	It is not clear what the “replicates” refer to in the experiments.
3)	Some discussion on why the “SASNet ensemble” would yield better performance would be good; could it be overfitting?