This paper is concerned with the robustness of VAEs to adversarial attacks. We highlight that conventional VAEs are brittle under attack but that methods recently introduced for disentanglement such as β-TCVAE (  (2018)). This motivated us to develop Seatbelt-VAE, a new hierarchical disentangled VAE that is designed to be significantly more robust to adversarial attacks than existing approaches, while retaining high quality reconstructions.
This paper examines adversarial attacks to a VAE. It is known that by small norm perturbations on the conditioning input x of a VAE can dramatically change the generated output. This paper
empirically illustrates that alternative objectives can improve robustness, in the sense of previously proposed adversarial attacks such as (Tabacof et al. (2016); Gondim-Ribeiro et al. (2018); Kos et al.(2018)). This paper is concerned with the stability of reconstructions and their quality, and proposes Seatbelt-VAE to remedy some of the shortcomings in the original VAE objective. 		
The key idea of the Seatbelt-VAE is introducing a conditionally Gaussian chains (of length L) in the encoder and decoder distributions of a VAE. This is a plausible and sensible idea. Then the authors evaluate the robustness of reconstructions under various output attacks.
The methodological part of the paper is quite well written and easy to follow, despite the fact that it is somewhat overloaded with too many abbreviations. The experimental section is harder to read as the motivations and its organization is not clearly stated. Overall, this section feels as if it is too hastily written, many results put into appendix without much discussion. The organization can be much more improved.
The disentanglement achieved by this novel representation is characterized only anecdotally and by contrasting the resulting objectives to a beta-VAE. It would have been much more informative to illustrate and discuss further the representations learned by such a conditionally Gaussian architecture. Figure 6 and 7 partially try to achieve this by showing the interplay of depth L and the inverse-dispersion parameter beta but I found it hard to interpret this results, for which an entire page is devoted.
In the experiments, the ELBO is reported for various methods. I would argue that the ELBO is not a very representative proxy for robustness. For example VAE ELBO and beta-VAE ELBO are both lower bounds of the true marginal likelihood and it is possible that beta-VAE is much lower while attaining a higher robustness in the sense of being resilient to suitably defined attacks.
The authors claim that there are no clear classification tasks for the datasets -- but this is not accurate as both celeb-a has clear classification tasks in the form of predicting attributes. It would have been really quite informative if adversarial accuracy on downstream tasks would have been reported. Relying on qualitative results in Figure 1 is only providing partial evidence about the approach.
Robustness to independent noise, as the authors have, is a good experiment to have -- however typical adversarial examples may be quite structured and such a randomized strategy may not give an accurate indication about the nature of the representation.
Overall, the paper is quite promising but I feel that one more iteration maybe needed.