Learning Disentangled Representations in Deep Generative Models
Deep generative models provide a powerful and flexible means to learn complex distributions over data by incorporating neural networks into latent-variable models. Variational approaches to training such models introduce a probabilistic encoder that casts data, typically unsupervised, into an entangled and unstructured representation space. While unsupervised learning is often desirable, sometimes even necessary, when we lack prior knowledge about what to represent, being able to incorporate domain knowledge in characterising certain aspects of variation in the data can often help learn better disentangled representations. Here, we introduce a new formulation of semi-supervised learning in variational autoencoders that allows precisely this. It permits flexible specification of probabilistic encoders as directed graphical models via a stochastic computation graph, containing both continuous and discrete latent variables, with conditional distributions parametrised by neural networks. We demonstrate how the provision of structure, along with a few labelled examples indicating plausible values for some components of the latent space, can help quickly learn disentangled representations. We then evaluate its ability to do so, both qualitatively by exploring its generative capacity, and quantitatively by using the disentangled representation to perform classification, on a variety of models and datasets.
This paper investigates deep generative models with multiple stochastic nodes and gives them meaning by semi-supervision. From a methodological point of view, there is nothing fundamentally novel (it is very similar to the semi-supervised work of Kingma et al; although this work has sometimes more than two latent nodes, it is not a complex extension). There is a fairly classical auxiliary variable trick used to make sure the inference network for y is trained over all data points (by supposing y is in fact is a latent variable with an observation 	ilde y; the observation is y if y is observed, or uninformative for unobserved y). Alternatively, one can separate the inference used to learn the generative model (which throws out inference over y if it is observed), from an inference used to *exercise* the model (approximate the complex p(y|x) in the model by a simpler q(y|x) - effectively inferring the target p(y|x) for the data where only x is collected). Results are strong, although on simple datasets. Overall this is a well written, interesting paper, but lacking in terms of methodological advances. Minor: - I feel the title is a bit too general for the content of the paper. I personally don*t agree with the strong contrast made between deep generative models and graphical models (deep generative models are graphical models, but they are more typically learned and un-interpretable than classical graphical models; and having multiple stochastic variables is not exclusive to graphical models, see DRAW, Deep Kalman Filter, Recurrent VAE, etc.). The word *structure* is a bit problematic; here, the paper seems more concerned with disentangling and semanticizing the latent representation of a generative model by supervision. It is debatable whether the models themselves have structure.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Learning Disentangled Representations in Deep Generative Models
Deep generative models provide a powerful and flexible means to learn complex distributions over data by incorporating neural networks into latent-variable models. Variational approaches to training such models introduce a probabilistic encoder that casts data, typically unsupervised, into an entangled and unstructured representation space. While unsupervised learning is often desirable, sometimes even necessary, when we lack prior knowledge about what to represent, being able to incorporate domain knowledge in characterising certain aspects of variation in the data can often help learn better disentangled representations. Here, we introduce a new formulation of semi-supervised learning in variational autoencoders that allows precisely this. It permits flexible specification of probabilistic encoders as directed graphical models via a stochastic computation graph, containing both continuous and discrete latent variables, with conditional distributions parametrised by neural networks. We demonstrate how the provision of structure, along with a few labelled examples indicating plausible values for some components of the latent space, can help quickly learn disentangled representations. We then evaluate its ability to do so, both qualitatively by exploring its generative capacity, and quantitatively by using the disentangled representation to perform classification, on a variety of models and datasets.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



This paper investigates deep generative models with multiple stochastic nodes and gives them meaning by semi-supervision. From a methodological point of view, there is nothing fundamentally novel (it is very similar to the semi-supervised work of Kingma et al; although this work has sometimes more than two latent nodes, it is not a complex extension). There is a fairly classical auxiliary variable trick used to make sure the inference network for y is trained over all data points (by supposing y is in fact is a latent variable with an observation 	ilde y; the observation is y if y is observed, or uninformative for unobserved y). Alternatively, one can separate the inference used to learn the generative model (which throws out inference over y if it is observed), from an inference used to *exercise* the model (approximate the complex p(y|x) in the model by a simpler q(y|x) - effectively inferring the target p(y|x) for the data where only x is collected). Results are strong, although on simple datasets. Overall this is a well written, interesting paper, but lacking in terms of methodological advances. Minor: - I feel the title is a bit too general for the content of the paper. I personally don*t agree with the strong contrast made between deep generative models and graphical models (deep generative models are graphical models, but they are more typically learned and un-interpretable than classical graphical models; and having multiple stochastic variables is not exclusive to graphical models, see DRAW, Deep Kalman Filter, Recurrent VAE, etc.). The word *structure* is a bit problematic; here, the paper seems more concerned with disentangling and semanticizing the latent representation of a generative model by supervision. It is debatable whether the models themselves have structure.
assistant
This paper investigates deep generative models with multiple stochastic nodes and gives them meaning by semi-supervision. From a methodological point of view, there is nothing fundamentally novel (it is very similar to the semi-supervised work of Kingma et al; although this work has sometimes more than two latent nodes, it is not a complex extension). There is a fairly classical auxiliary variable trick used to make sure the inference network for y is trained over all data points (by supposing y is in fact is a latent variable with an observation 	ilde y; the observation is y if y is observed, or uninformative for unobserved y). Alternatively, one can separate the inference used to learn the generative model (which throws out inference over y if it is observed), from an inference used to *exercise* the model (approximate the complex p(y|x) in the model by a simpler q(y|x) - effectively inferring the target p(y|x) for the data where only x is collected). Results are strong, although on simple datasets. Overall this is a well written, interesting paper, but lacking in terms of methodological advances. Minor: - I feel the title is a bit too general for the content of the paper. I personally don*t agree with the strong contrast made between deep generative models and graphical models (deep generative models are graphical models, but they are more typically learned and un-interpretable than classical graphical models; and having multiple stochastic variables is not exclusive to graphical models, see DRAW, Deep Kalman Filter, Recurrent VAE, etc.). The word *structure* is a bit problematic; here, the paper seems more concerned with disentangling and semanticizing the latent representation of a generative model by supervision. It is debatable whether the models themselves have structure.