Towards Principled Objectives for Contrastive Disentanglement
Unsupervised learning is an important tool that has received a significant amount of attention for decades. Its goal is `unsupervised recovery,* i.e., extracting salient factors/properties from unlabeled data. Because of the challenges in defining salient properties, recently, `contrastive disentanglement* has gained popularity to discover the additional variations that are enhanced in one dataset relative to another. %In fact, contrastive disentanglement and unsupervised recovery are often combined in that we seek additional variations that exhibit salient factors/properties. Existing formulations have devised a variety of losses for this task. However, all present day methods exhibit two major shortcomings: (1) encodings for data that does not exhibit salient factors is not pushed to carry no signal; and (2) introduced losses are often hard to estimate and require additional trainable parameters. We present a new formulation for contrastive disentanglement which avoids both shortcomings by carefully formulating a probabilistic model and by using non-parametric yet easily computable metrics. We show on four challenging datasets that the proposed approach is able to better disentangle salient factors.
This paper is concerned with contrastive disentanglement. The considered problem is interesting and important in the community, and the proposed method seems to be of practical use, according to the empirical results. My concern is that the contribution of the paper on the theoretical or methodological side seems a bit weak. The proposed method relies on the VAE framework for contrastive disentanglement. The authors proposed two modifications: one is to explicit enforce the posterior of the content representation to be a Delta distribution, and the other is to make sure that the background representation has the same distribution across the background and the target. In the original cVAE framework for contrastive disentanglement, the two constraints were considered in a slightly more implicit way. It is not surprising to see that the proposed method seems to outperform cVAE only slightly. To see how much we can really gain by further explicitly enforcing the two constraints, the authors may do some further studies with varied sample sizes. BTW, in the result for CelebA in Table 1, cVAE seems to outperform the proposed method, denoted by *Ours*, according to Acc of z_x, but it was indicated in the text (as well as in by the bold font) that *Ours* performed the best--is there a typo? I acknowledge I read the authors* response and other reviews and would like to keep my original rating.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Towards Principled Objectives for Contrastive Disentanglement
Unsupervised learning is an important tool that has received a significant amount of attention for decades. Its goal is `unsupervised recovery,* i.e., extracting salient factors/properties from unlabeled data. Because of the challenges in defining salient properties, recently, `contrastive disentanglement* has gained popularity to discover the additional variations that are enhanced in one dataset relative to another. %In fact, contrastive disentanglement and unsupervised recovery are often combined in that we seek additional variations that exhibit salient factors/properties. Existing formulations have devised a variety of losses for this task. However, all present day methods exhibit two major shortcomings: (1) encodings for data that does not exhibit salient factors is not pushed to carry no signal; and (2) introduced losses are often hard to estimate and require additional trainable parameters. We present a new formulation for contrastive disentanglement which avoids both shortcomings by carefully formulating a probabilistic model and by using non-parametric yet easily computable metrics. We show on four challenging datasets that the proposed approach is able to better disentangle salient factors.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



This paper is concerned with contrastive disentanglement. The considered problem is interesting and important in the community, and the proposed method seems to be of practical use, according to the empirical results. My concern is that the contribution of the paper on the theoretical or methodological side seems a bit weak. The proposed method relies on the VAE framework for contrastive disentanglement. The authors proposed two modifications: one is to explicit enforce the posterior of the content representation to be a Delta distribution, and the other is to make sure that the background representation has the same distribution across the background and the target. In the original cVAE framework for contrastive disentanglement, the two constraints were considered in a slightly more implicit way. It is not surprising to see that the proposed method seems to outperform cVAE only slightly. To see how much we can really gain by further explicitly enforcing the two constraints, the authors may do some further studies with varied sample sizes. BTW, in the result for CelebA in Table 1, cVAE seems to outperform the proposed method, denoted by *Ours*, according to Acc of z_x, but it was indicated in the text (as well as in by the bold font) that *Ours* performed the best--is there a typo? I acknowledge I read the authors* response and other reviews and would like to keep my original rating.
assistant
This paper is concerned with contrastive disentanglement. The considered problem is interesting and important in the community, and the proposed method seems to be of practical use, according to the empirical results. My concern is that the contribution of the paper on the theoretical or methodological side seems a bit weak. The proposed method relies on the VAE framework for contrastive disentanglement. The authors proposed two modifications: one is to explicit enforce the posterior of the content representation to be a Delta distribution, and the other is to make sure that the background representation has the same distribution across the background and the target. In the original cVAE framework for contrastive disentanglement, the two constraints were considered in a slightly more implicit way. It is not surprising to see that the proposed method seems to outperform cVAE only slightly. To see how much we can really gain by further explicitly enforcing the two constraints, the authors may do some further studies with varied sample sizes. BTW, in the result for CelebA in Table 1, cVAE seems to outperform the proposed method, denoted by *Ours*, according to Acc of z_x, but it was indicated in the text (as well as in by the bold font) that *Ours* performed the best--is there a typo? I acknowledge I read the authors* response and other reviews and would like to keep my original rating.