Convolutional Networks on Graphs for Learning Molecular Fingerprints
We introduce a convolutional neural network that operates directly on graphs. These networks allow end-to-end learning of prediction pipelines whose inputs are graphs of arbitrary size and shape. The architecture we present generalizes standard molecular feature extraction methods based on circular fingerprints. We show that these data-driven features are more interpretable, and have better predictive performance on a variety of tasks.
This paper describes a method for learning vector representations of small molecules, or 'fingerprints,' using deep, convolutional neural networks.This is high-quality work, and makes a significant contribution to the important problem of learning useful representations for chemoinformatics. It is clearly written and a pleasure to read.In terms of originality, the neural network architecture presented here is pretty similar to the work of Lusci et al 2013. The three differences in network topology are: 1) This paper proposes the use of a different set of NN weights at each layer of the network, while Lusci ties the weights in each layer together, making the network recursive. 2) This method avoids cycles by limiting the depth of the network, compared to the approach in Lusci et al where the recursive structure rooted at each atom is forced to be a tree (cycles are broken).3) Here, there are intermediate connections from each layer of the recursive network to the output.But these differences, along with other technical choices (e.g. the use of the softmax activation in the output layer), can be important. The technical challenges of using methods like this can be prohibitive to people working in chemoinformatics, so relating this algorithm to the familiar technique of circular fingerprinting is valuable. There are plenty of original perspectives, ideas, and experiments in this work to warrant publication.Specific comments:In Figure 2, there should be a comma in line 8 between ra and r1. Also, atoms have a variable number of neighbors, and the text should mention how this is handled.The circular fingerprints used in the paper have 2048 binary features, which have the advantage of taking up much less memory than 2048 floats. While this doesn't detract from the importance of the supervised learning results presented here, it could be a limitation for doing things like similarity searches over millions or billions of chemicals.The size of the training and test sets were not clear. Were there 1000 training examples from each data set? 1000 minibatches of size 100?The hyperparameters chosen by spearmint should be given, preferably in a table along with the bounds of the search space.One major difficulty with architectures like this is that the computational benefits of doing SGD with minibatches are often lost because the NN architecture for each molecule is different. How was minibatch training done here?