Variational Inference of Disentangled Latent Concepts from Unlabeled Observations
Disentangled representations, where the higher level data generative factors are reflected in disjoint latent dimensions, offer several benefits such as ease of deriving invariant representations, transferability to other tasks, interpretability, etc. We consider the problem of unsupervised learning of disentangled representations from large pool of unlabeled observations, and propose a variational inference based approach to infer disentangled latent factors. We introduce a regularizer on the expectation of the approximate posterior over observed data that encourages the disentanglement. We also propose a new disentanglement metric which is better aligned with the qualitative disentanglement observed in the decoder*s output. We empirically observe significant improvement over existing methods in terms of both disentanglement and data likelihood (reconstruction quality).
 This paper describes DIP-VAE, an improvement on the beta-VAE framework for learning a disentangled representation of the data generative factors in the visual domain. The authors propose to augment the standard VAE ELBO objective with an extra term that minimises the covariance between the latents. Unlike the original beta-VAE objective which implicitly minimises such covariance individually for each observation x, the DIP-VAE objective does so while marginalising over x. This difference removes the tradeoff between reconstruction quality and disentangling reported for beta-VAE, since DIP-VAE maintains sensitivity of q(z|x) to each observation x, and hence achieves disentangling while preserving the sharpness of reconstructions. Pros: - the paper is well written - it makes a contribution to an important line or research (unsupervised disentangled representation learning) - the covariance minimisation proposed in the paper looks like an easy to implement yet impactful change to the VAE objective to encourage disentanglement while preserving reconstruction quality - it directly compares the performance of DIP-VAE to that of beta-VAE showing significant improvements in terms of disentangling metric and reconstruction error Cons: - I am yet to be fully convinced how well the approach works. Table 1 and Figure 1 look good, but other figures are either tangental to the main point of the paper, or impossible to read due to the small scale. For example, the qualitative evaluation of the latent traversals is almost impossible due to the tiny scale of Table 5 (shouldn*t this be a Figure rather than a Table?) - The authors concentrate a lot on the CelebA dataset, however I believe the comparison with beta-VAE would be a lot clearer on the dSprites dataset (https://github.com/deepmind/dsprites-dataset) (the authors call it 2D shapes). I would like to see latent traversals of the best DIP-VAE vs beta-VAE to demonstrate good disentangling and the improvements in reconstruction quality. This (and larger Table 5) would be better use of space compared to Table 2 and Figure 2 for example, which I feel are somewhat tangental to the main message of the paper and are better suited for the appendix. - I wonder how the authors calculated the disentanglement metric on CelebA, given that the ground truth attributes in the dataset are often rather qualitative (e.g. attractiveness), noisy (many can be considered an inaccurate description of the image), and often do not align with the data generative factors discoverable through unsupervised modeling of the data distribution - Table 3 - the legends for the axes are too small and are impossible to read. Also it would be helpful to normalise the scales of the heat plots in the second row. - Table 3 - looking at the correlations with the ground truth factors, it seems like beta-VAE did not actually disentangle the latents. Would be nice to see the corresponding latent traversal plots to ensure that the baseline is actually trained well. I am willing to increase my score for the paper if the authors can address my points. In particular I would like to see a clear comparison in terms of latent traversals on dSprites between beta-VAE and DIP-VAE models presented in Table 3. I would also like to see where these particular models lie in Figure 1. --------------------------- ---- UPDATE ---------- --------------------------- I have increased my score after reading the revised version of the manuscript.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Variational Inference of Disentangled Latent Concepts from Unlabeled Observations
Disentangled representations, where the higher level data generative factors are reflected in disjoint latent dimensions, offer several benefits such as ease of deriving invariant representations, transferability to other tasks, interpretability, etc. We consider the problem of unsupervised learning of disentangled representations from large pool of unlabeled observations, and propose a variational inference based approach to infer disentangled latent factors. We introduce a regularizer on the expectation of the approximate posterior over observed data that encourages the disentanglement. We also propose a new disentanglement metric which is better aligned with the qualitative disentanglement observed in the decoder*s output. We empirically observe significant improvement over existing methods in terms of both disentanglement and data likelihood (reconstruction quality).
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



 This paper describes DIP-VAE, an improvement on the beta-VAE framework for learning a disentangled representation of the data generative factors in the visual domain. The authors propose to augment the standard VAE ELBO objective with an extra term that minimises the covariance between the latents. Unlike the original beta-VAE objective which implicitly minimises such covariance individually for each observation x, the DIP-VAE objective does so while marginalising over x. This difference removes the tradeoff between reconstruction quality and disentangling reported for beta-VAE, since DIP-VAE maintains sensitivity of q(z|x) to each observation x, and hence achieves disentangling while preserving the sharpness of reconstructions. Pros: - the paper is well written - it makes a contribution to an important line or research (unsupervised disentangled representation learning) - the covariance minimisation proposed in the paper looks like an easy to implement yet impactful change to the VAE objective to encourage disentanglement while preserving reconstruction quality - it directly compares the performance of DIP-VAE to that of beta-VAE showing significant improvements in terms of disentangling metric and reconstruction error Cons: - I am yet to be fully convinced how well the approach works. Table 1 and Figure 1 look good, but other figures are either tangental to the main point of the paper, or impossible to read due to the small scale. For example, the qualitative evaluation of the latent traversals is almost impossible due to the tiny scale of Table 5 (shouldn*t this be a Figure rather than a Table?) - The authors concentrate a lot on the CelebA dataset, however I believe the comparison with beta-VAE would be a lot clearer on the dSprites dataset (https://github.com/deepmind/dsprites-dataset) (the authors call it 2D shapes). I would like to see latent traversals of the best DIP-VAE vs beta-VAE to demonstrate good disentangling and the improvements in reconstruction quality. This (and larger Table 5) would be better use of space compared to Table 2 and Figure 2 for example, which I feel are somewhat tangental to the main message of the paper and are better suited for the appendix. - I wonder how the authors calculated the disentanglement metric on CelebA, given that the ground truth attributes in the dataset are often rather qualitative (e.g. attractiveness), noisy (many can be considered an inaccurate description of the image), and often do not align with the data generative factors discoverable through unsupervised modeling of the data distribution - Table 3 - the legends for the axes are too small and are impossible to read. Also it would be helpful to normalise the scales of the heat plots in the second row. - Table 3 - looking at the correlations with the ground truth factors, it seems like beta-VAE did not actually disentangle the latents. Would be nice to see the corresponding latent traversal plots to ensure that the baseline is actually trained well. I am willing to increase my score for the paper if the authors can address my points. In particular I would like to see a clear comparison in terms of latent traversals on dSprites between beta-VAE and DIP-VAE models presented in Table 3. I would also like to see where these particular models lie in Figure 1. --------------------------- ---- UPDATE ---------- --------------------------- I have increased my score after reading the revised version of the manuscript.
assistant
 This paper describes DIP-VAE, an improvement on the beta-VAE framework for learning a disentangled representation of the data generative factors in the visual domain. The authors propose to augment the standard VAE ELBO objective with an extra term that minimises the covariance between the latents. Unlike the original beta-VAE objective which implicitly minimises such covariance individually for each observation x, the DIP-VAE objective does so while marginalising over x. This difference removes the tradeoff between reconstruction quality and disentangling reported for beta-VAE, since DIP-VAE maintains sensitivity of q(z|x) to each observation x, and hence achieves disentangling while preserving the sharpness of reconstructions. Pros: - the paper is well written - it makes a contribution to an important line or research (unsupervised disentangled representation learning) - the covariance minimisation proposed in the paper looks like an easy to implement yet impactful change to the VAE objective to encourage disentanglement while preserving reconstruction quality - it directly compares the performance of DIP-VAE to that of beta-VAE showing significant improvements in terms of disentangling metric and reconstruction error Cons: - I am yet to be fully convinced how well the approach works. Table 1 and Figure 1 look good, but other figures are either tangental to the main point of the paper, or impossible to read due to the small scale. For example, the qualitative evaluation of the latent traversals is almost impossible due to the tiny scale of Table 5 (shouldn*t this be a Figure rather than a Table?) - The authors concentrate a lot on the CelebA dataset, however I believe the comparison with beta-VAE would be a lot clearer on the dSprites dataset (https://github.com/deepmind/dsprites-dataset) (the authors call it 2D shapes). I would like to see latent traversals of the best DIP-VAE vs beta-VAE to demonstrate good disentangling and the improvements in reconstruction quality. This (and larger Table 5) would be better use of space compared to Table 2 and Figure 2 for example, which I feel are somewhat tangental to the main message of the paper and are better suited for the appendix. - I wonder how the authors calculated the disentanglement metric on CelebA, given that the ground truth attributes in the dataset are often rather qualitative (e.g. attractiveness), noisy (many can be considered an inaccurate description of the image), and often do not align with the data generative factors discoverable through unsupervised modeling of the data distribution - Table 3 - the legends for the axes are too small and are impossible to read. Also it would be helpful to normalise the scales of the heat plots in the second row. - Table 3 - looking at the correlations with the ground truth factors, it seems like beta-VAE did not actually disentangle the latents. Would be nice to see the corresponding latent traversal plots to ensure that the baseline is actually trained well. I am willing to increase my score for the paper if the authors can address my points. In particular I would like to see a clear comparison in terms of latent traversals on dSprites between beta-VAE and DIP-VAE models presented in Table 3. I would also like to see where these particular models lie in Figure 1. --------------------------- ---- UPDATE ---------- --------------------------- I have increased my score after reading the revised version of the manuscript.