Though machine learning approaches have shown great success in estimating properties of small molecules, the inverse problem of generating molecules with desired properties remains challenging. This difficulty is in part because the set of molecules which have a given property is structurally very diverse. Treating this inverse problem as a conditional distribution estimation task, we draw upon work in learning disentangled representations to learn a conditional distribution over molecules given a desired property, where the molecular structure is encoded in a continuous latent random variable. By including property information as an input factor independent from the structure representation, one can perform conditional molecule generation via a "style transfer" process, in which we explicitly set the property to a desired value at generation time. In contrast to existing approaches, we disentangle the latent factors from the property factors using a regularization term which constrains the generated molecules to have the property provided to the generation network, no matter how the latent factor changes.
# Summary
The paper considers the problem of generating molecules with desired properties using a variant of supervised variational auto-encoders. The key novelty is the idea to learn a representation with disentangled molecular properties which can be modified during generation of novel molecules.
To account for the diversity of molecules in a particular target class the conditional probability of a molecule x given a target property y is modelled with a latent variable z, i.e., p(x | y) = \int p(x | z, y) p(z) dz.
The "style transfer" is achieved by taking an initial molecule x and computing the posterior of novel molecule x' with modified property y' via p(x' | y', x) = \int p(x' | y', z) p(z | x) dz.
The latent representation is learned using a supervised auto-encoder (Kingma et al., 2014), formally specified in Eq. (2). The optimization problem is then further extended by imposing a soft constraint that the molecules from a particular class exhibit a certain property (Eq. 3). The reason for this constraint is to enforce that the conditional probability p(x | y, z) takes into account the information in label y. This part might not be sufficiently well-motivated and would benefit from strengthening the argument for the property predictor and soft constraint (would an illustration be possible here?). In particular, why should it not be possible to encode (and keep fixed during training) the information present in y into the sufficient statistics of p(x | y, z)?
The soft constraint involves an oracle function (see Eq. 3), which evaluates the designed molecules. Section 3.2 deals with the approximation of the oracle via a property predictor function to side-step the fact that the oracle is not necessarily differentiable or CPU-bound. Section 3.3 and 3.4 deal with joint optimization of the supervised variational auto-encoder and gradient estimates.
The approach is evaluated on two datasets, relative to state-of-the-art baselines based on variational auto-encoders and deep learning. The main goals of the experiments are to establish that the approach can learn useful disentangled conditional distributions over molecules and to assess the extent of improvement in the data generation process as a result of using the soft constraint term. The focus of the experiment is on generating molecules with certain range of logP values. The data generating process is simulated/evaluated while controlling for the logP value range and molecular structure.
The experiments provide an objective assessment of the merits of the approach and indicate clear advantages over prior work. 
# Recommendation
The paper is very well written, easy to follow, and properly structured. The work is novel and the idea to disentangle molecular properties from the target property is quite interesting. The experiments are detailed and compare to baselines based on variational auto-encoders and deep learning. The part that could be improved is the choice of the target property. In particular, it would be great to evaluate the approach on the problem of finding molecules binding to a certain protein site. Such a problem is likely to exhibit scaffold hops and activity cliffs which make drug design problems very difficult (e.g., see [6-8]).
# Related work (additional references)
- While mapping of the discrete space of molecules to a continuous latent space (in active learning approaches combined with VAE) allows for gradient computation and the use of active learning/optimization techniques, the very difficult problem of finding latent space pre-images persists (in general, the problem should be in the NP class). The latter refers to finding a molecule that maps to a fixed point in the latent space. There have, however, been some efficient approximations in special cases [1-3].
- In [4] and [5], an approach for generation of molecules with desired properties has been pursued with posterior sampler based on a conditional exponential family model. The representation is based on a tuple kernel mapping mapping (x, y) to some reproducing kernel Hilbert space (without disentanglement). The approach provides a consistent data generation process and a mean to deal with the fact that designs are not independent and identically distributed (e.g., Eq. 2 assumes that examples are IID).
# Black-box target property
- The logP property might not be a good proxy for the effectiveness in generating molecules with desired binding activities (to some protein site) because (to the best of my knowledge) it does not exhibit the "activity cliff property" characteristic to drug design. This refers to the property that a small change in molecular structure results in a completely different activity level (e.g., see [6-8] and [4]). In this sense, the "style transfer" might be beneficial for finding molecules binding to a certain protein site that are structurally similar to some available molecules with bad activity levels (for which a synthesis path might be known or easy to derive).
- An actual black-box property (realized via a docking program) that is expensive to evaluate served as a target property in [5]. That docking program (supplied with suitable binding/docking constraints) can provide a nice proxy for the actual generation of molecules.
# References
[1] G. H. Bakir, J. Weston, and B. Schoelkopf. Learning to find pre-images, NIPS 2004.
[2] C. Cortes, M. Mohri, and J. Weston. A general regression technique for learning transductions, ICML 2005.
[3] S. Giguere, A. Rolland, F. Laviolette, and M. Marchand. On the string kernel pre-image problem with applications in drug discovery, ICML 2015.
[4] D. Oglic, R. Garnett, and T. Gaertner. Active search in intensionally specified structured spaces, AAAI 2017.
[5] D. Oglic, S. Oatley, S. Macdonald, T. Mcinally, R. Garnett, J. Hirst, and T. Gaertner. Active search for computer-aided drug design, Molecular Informatics 2018.
[6] G. Schneider and U. Fechner. Computer-based de novo design of drug-like molecules, Nature Reviews Drug Discovery, 2005.
[7] P. Schneider and G. Schneider. De novo design at the edge of chaos. Journal of Medicinal Chemistry, 2016.
[8] J. Scannell, A. Blanckley, H. Boldon, and B. Warrington. Diagnosing the decline in pharmaceutical R&D efficiency. Nature Reviews Drug Discovery, 2012.