Graph neural networks have recently achieved great successes in predicting quantum mechanical properties of molecules. These models represent a molecule as a graph using only the distance between atoms (nodes). They do not, however, consider the spatial direction from one atom to another, despite directional information playing a central role in empirical potentials for molecules, e.g. in angular potentials. To alleviate this limitation we propose directional message passing, in which we embed the messages passed between atoms instead of the atoms themselves. Each message is associated with a direction in coordinate space. These directional message embeddings are rotationally equivariant since the associated directions rotate with the molecule. We propose a message passing scheme analogous to belief propagation, which uses the directional information by transforming messages based on the angle between them. Additionally, we use spherical Bessel functions and spherical harmonics to construct theoretically well-founded, orthogonal representations that achieve better performance than the currently prevalent Gaussian radial basis representations while using fewer than 1 /4 of the parameters. We leverage these innovations to construct the directional message passing neural network (DimeNet). DimeNet outperforms previous GNNs on average by 76 % on MD17 and by 31 % on QM9.
This is a sophisticated paper on predicting molecular properties at atom as well as molecular levels using physics inspired, extended GNN architectures. Two key extensions are provided above and beyond previous GNN models that operated on graphs derived from pairwise distances between atoms. First, the encoding of atom distances for use in neural messages is no longer done in terms of Gaussian radial basis function representations but in terms of spherical Bessel functions. The provide an orthogonal decomposition at resolutions controlled by associated frequencies. The orthogonality is though lost due to the use of an envelop function to ensure differentiability at cutoff distance defining the graph (essential for molecular simulations) but this appears not to affect performance. The second and the primary contribution of the paper, beyond the architecture itself, is the use of directional embeddings of messages where angles are transformed into cosine basis for neural mappings. In other words, the message sent from atom i to j aggregates messages from i's other neighbors in a manner that takes into account the angle formed by i, j, and k's respective atom positions. Since local directions are equivariant with an overall molecular rotation, the message passing architecture in this new representation remains invariant to rotations/translations. The added directional information, embedded in local basis representation, clearly makes the network more powerful (able to distinguish higher order structures). 
- the authors suggest that the radial information can be transformed simply by element-wise multiplication while angular information requires more complex transformations in message calculations. Is there a physical insight to this or is this simply an empirical finding?
- there are many layers of transformations introduced from the atom embeddings before reaching the property of interest. Are so many layers really necessary? 
- it seems models for QM9 data were trained separately for each different physical target. Is this really necessary? Given the many layers of transformations until the properties are predicted, couldn't the message passing component be largely shared? 
- what exactly is the training data for the molecular simulation tests? The description in the paper is insufficient. A separate model is trained for each molecule, presumably based on samples resulting from physical simulations (?). What is provided to the models based on each "sample"?
- the ablation studies are helpful to assess the impact of the main differences (directionality, bessel vs Gaussian, node embeddings vs message) though I would wish to see what the degradation effect is on QM9 if one used a shared message passing architecture (just sharing the messages, resulting embeddings could be transformed separately for different predictions). 
There's a recent workshop paper also making use of directional information (local coordinate transformations along a protein backbone chain) in message passing/transformer architectures: Ingraham et al., Generative models for graph-based protein design, ICLR workshop 2019