Existing methods of image to image translation require multiple steps in the training or modification process, and suffer from either an inability to generalize, or long training times. These methods also focus on binary trait modification, ignoring continuous traits. To address these problems, we propose ModifAE: a novel standalone neural network, trained exclusively on an autoencoding task, that implicitly learns to make continuous trait image modifications. As a standalone image modification network, ModifAE requires fewer parameters and less time to train than existing models. We empirically show that ModifAE produces significantly more convincing and more consistent continuous face trait modifications than the previous state-of-the-art model.
Overview and contributions: The authors propose the ModifAE model that is based on an autoencoder neural network for continuous trait image modifications. ModifAE requires fewer parameters and less time to train than existing generative models. The authors also present experiments to show that ModifAE produces more convincing and more consistent continuous face trait modifications than the current baselines.
Strengths:
1. Nice presentation of the model.
2. Good experiments to justify improved running time and fewer number of parameters.
Weaknesses:
1. I am not completely convinced by the results in Figure 4. It doesn't seem like the model is able to pick up on subtle facial expressions and generate them in a flexible manner. In fact, the images look very very similar regardless of the value of the traits. Furthermore, the authors claim that "In general, as she becomes more emotional, her smile increases, and as she is made more attractive, her smile increases as well, as smiling subjects are judged as more attractive". I believe attractiveness and emotions are much more diverse and idiosyncratic than just the size of her smile...
2. From Figure 5 it seems like ModifAE generates images that are lower in quality as compared to StarGAN. Can the authors comment on this point? How can ModifAE be improved to generate higher-quality images?
Questions to authors:
1. Weakness points 1 and 2.
2. This did not affect my rating, but I am slightly concerned by the labelings as seen in Figure 1. Is it reasonable to infer traits like "trustworthy", "attractive", "aggressive", "responsible" from images? Are these traits really what we should be classifying people's faces as, and are there any possible undesirable/sensitive biases from the dataset that our models could learn? I would like to hear the author's opinions on the ethical implications of these datasets and models. 
Presentation improvements, typos, edits, style, missing references:
None