While generative neural networks can learn to transform a specific input dataset into a specific target dataset, they require having just such a paired set of input/output datasets. For instance, to fool the discriminator, a generative adversarial network (GAN) exclusively trained to transform images of black-haired men to blond-haired men would need to change gender-related characteristics as well as hair color when given images of black-haired women as input. This is problematic, as often it is possible to obtain a pair of (source, target) distributions but then have a second source distribution where the target distribution is unknown. The computational challenge is that generative models are good at generation within the manifold of the data that they are trained on. However, generating new samples outside of the manifold or extrapolating "out-of-sample" is a much harder problem that has been less well studied. To address this, we introduce a technique called neuron editing that learns how neurons encode an edit for a particular transformation in a latent space. We use an autoencoder to decompose the variation within the dataset into activations of different neurons and generate transformed data by defining an editing transformation on those neurons. By performing the transformation in a latent trained space, we encode fairly complex and non-linear transformations to the data with much simpler distribution shifts to the neuron's activations. Our technique has the advantage of being generally applicable to a wide variety of data domains, modalities, and applications. We first demonstrate it on image transformations and then move to our two main applications in biology: removal of batch artifacts representing unwanted noise and modeling the effect of drug treatments to predict synergy between drugs.
Overview: This paper presents a new generative modeling approach to transform between data distributions via a technique the authors dub “neuron editing”. Their approach for “neuron editing” can be used to learn how neurons in a DNN encode particular transformations in the latent space, with the hope of using it to be able to generate data that are out-of-sample and/or do not lie on the same manifold as the data used to train the generative model. Experiments were performed on the CelebA dataset. The authors further demonstrate the usefulness of their proposed idea by using it to remove noise/batch effects from data and also predicting synergy between drugs by using their idea to model the effect of drug treatments.
While the idea behind this paper is interesting and their method does seem to provide improvements over other generative models, the paper is a bit difficult to follow and the applications to biology are not explained well enough to understand the implications of their results. The fact that the authors did not compare to StyleGANs also seems a bit suspicious because these models are sota for the types of problems that this paper addresses. Additionally, the authors should consider rewriting the motivation for this problem to be more general. This is a convincing new method to transform distributions in any setting, but the introduction would lead one to believe its applicability is mostly to computational biology. 
Detailed comments:
- The writing and motivation needs to be reworked in order to ensure that the introduction, results, and abstract match in their tone and content. The abstract uses a face recognition/generation application to motivate the work, the introduction focuses solely on computational biology, the results switch between biology and face recognition.
- Section 2: Three desirable properties for a transformation are listed, but what makes these properties desirable?
Equation (1): Not obvious why the piecewise function definition is necessary, since the second equation seems to hold for j = 0 and j = 99 as well. This should either be corrected or simplified.
- “Mode collapse refers to the discriminator being unable to detect differences in variability between real and fake examples.” Not true. Mode collapse is when a large region of the model’s input space maps onto the small region around a single (often bad) sample.
- Section 3: It would be helpful to see gradual interpolation from the original latent representation of a sample along the direction the neuron edit will be performed, and continued beyond the proposed final latent activations, to demonstrate how this process actually affects the model’s output.
- Doesn’t really seem like the authors tried very hard with the other methods (especially the vanilla GAN) and the omission of StyleGAN, the current state-of-the-art in this kind of transformation learning, is conspicuously omitted. It is highly likely that a well-trained StyleGAN would do better than the other GAN/AE techniques compared against.
- A PCA-based transformation is applied directly to the image data, but it would be interesting (and perhaps more informative) to perform an alignment of the PCs of activations in the latent space.
- Section 4: Again, the motivation distinctly focuses on computational biology when one could easily imagine this approach being applicable to a variety of problems.
- Comparisons with StyleGANs would’ve been appreciated, especially given the fact that they’re now considered state-of-the-art when it comes to modifying the latent space and creating out of sample images as proposed in this work. 
- The idea is interesting, though one wonders if there isn’t any other work where the neurons have been “edited” to accommodate different transformations, given that the idea is itself rather intuitive. A more thorough literature review in that regard would be helpful.
- The CelebA experiments help lay establish an intuitive understanding of the proposed technique and were helpful. However, the ideas are a little disconnected with the biological applications of the technique. Better motivation/bridging of the two sets of experiments would be nice.
- A more detailed explanation and analysis of the combinatorial drug application would’ve been helpful to understand the results.