Learning disentangling representations of the independent factors of variations that explain the data in an unsupervised setting is still a major challenge. In the following paper we address the task of disentanglement and introduce a new state-ofthe-art approach called Non-synergistic variational Autoencoder (Non-Syn VAE). Our model draws inspiration from population coding, where the notion of synergy arises when we describe the encoded information by neurons in the form of responses from the stimuli. If those responses convey more information together than separate as independent sources of encoding information,they are acting synergetically. By penalizing the synergistic mutual information within the latents we encourage information independence and by doing that disentangle the latent factors. Notably, our approach could be added to the VAE framework easily, where the new ELBO function is still a lower bound on the log likelihood p x . In addition, we qualitatively compare our model with Factor VAE and show that this one implicitly minimises the synergy of the latents.
This paper proposes a new approach to enforcing disentanglement in VAEs using a term that penalizes the synergistic mutual information between the latent variables, encouraging representations where any given piece of information about a datapoint can be garnered from a single latent.  In other words, representations where there is no information conveyed by combinations of latents that is not conveyed by considering each latent in isolation.  As the resultant target is intractable to evaluate, a number of approximations are employed for practical training.
The high-level idea is quite interesting, but the paper itself is quite a long way of convincing me that this is actually a good approach.  Moreover, the paper is a long way of the level of completeness, rigor, clarity, and polish that is required to seriously consider it for publication.  In short, the work is still at a relatively early stage and a lot more would need to be done for it to attain various minimum standards for acceptance.  A non-exhaustive list of specific examples of its shortfalls are given below.
1. The paper is over a page and a half under length, despite wasting large amounts of space (e.g. figures 3 and 4 should be two lines on the same plot)
2. The experimental evaluation is woefully inadequate.  The only quantitative assessment is to compare to a single different approach on a single toy dataset, and even then the metric being used is the one the new method uses to train for making it somewhat meaningless.
3. The introduction is completely generic and says nothing about the method itself, just providing a (not especially compelling) motivation for disentanglement in general.  In fact, the motivation of the introduction is somewhat at odds with the work -- correctly talking about the need for hierarchical representations which the approach actually actively discourages.
4. There are insufficient details on the algorithm itself in terms of the approximations that are made to estimate the synergistic mutual information.  These are mostly glossed over with only a very short explanation in the paragraph after equation 15.  Yes there are algorithm blocks, but these are pretty incomprehensible and lack accompanying text.  In particular, I cannot understand what A_w is supposed to be.  This is very important as I suspect the behavior of the approximation is very different to the true target.  Similarly, it would be good to provide more insight into the desired target (i.e. Eq 15).  For example, I suspect that it will encourage a mismatch between the aggregate posterior and prior by encouraging higher entropy on the former, in turn causing samples from the generative model to provide a poor match to the data.
5. The repeated claims of the approach and results being "state-of-the-art" are cringe-worthy bordering on amusing.  Writing like this serves no purpose even when it justified, and it certainly is not here.
6. There are a lot of typos throughout and the production values are rather poor.  For example, the algorithm blocks which are extremely messy to the point where they are difficult to follow, citep/citet mistakes occur almost every other citation, there is a sign error in Equation 16.
This is a piece of work in an exciting research area that,  with substantial extra work, could potentially result in a decent paper due to fact that the core idea is simple and original.  However, it is a long way short of this in its current state.  Along with addressing the specific issues above and improving the clarity of the work more generally, one thing in particular that would need to address in a resubmission is a more careful motivation for the method (ideally in the form of a proper introduction).  
Though I appreciate this is a somewhat subjective opinion, for me, penalizing the synergistic information is probably actually a bad thing to do when taking a more long-term view on disentanglement.  Forcing simplistic representations where no information is conveyed through the composition of latents beyond that they provide in isolation is all well and good for highly artificial and simplistic datasets like dsprites, but is clearly not a generalizable approach for larger datasets where no such simplistic representation exists.  As you say in the first line of your own introduction, hierarchy and composition are key parts of learning effective and interpretable representations and this is exactly what you are discouraging.  A lot of the issue here is one of the disentanglement literature at large rather than this paper (though I do find it to be a particularly egregious offender) and it is fine to have different opinions.  However, it is necessary to at least make a sensible case for why your approach is actually useful.  
Namely, is there actually any real applications where such a simplistic disentanglement is actually useful?  Is there are anyway the current works helps in the longer vision of achieving interpretable representations?  When and why is the synergistic information a better regularizer than, for example, the total correlation?  The experiments you have do not make any inroads to answering these questions and there are no written arguments of note to address them.  I am not trying to argue here that there isn't a good case to be made for the suggested approach in the context of these questions (though I am suspicious), just that if the work is going to have any lasting impact on the community then it needs to at least consider them.