As deep learning-based classifiers are increasingly adopted in real-world applications, the importance of understanding how a particular label is chosen grows. Single decision trees are an example of a simple, interpretable classifier, but are unsuitable for use with complex, high-dimensional data. On the other hand, the variational autoencoder (VAE) is designed to learn a factored, low-dimensional representation of data, but typically encodes high-likelihood data in an intrinsically non-separable way. We introduce the differentiable decision tree (DDT) as a modular component of deep networks and a simple, differentiable loss function that allows for end-to-end optimization of a deep network to compress high-dimensional data for classification by a single decision tree. We also explore the power of labeled data in a supervised VAE (SVAE) with a Gaussian mixture prior, which leverages label information to produce a high-quality generative model with improved bounds on log-likelihood. We combine the SVAE with the DDT to get our classifier+VAE (C+VAE), which is competitive in both classification error and log-likelihood, despite optimizing both simultaneously and using a very simple encoder/decoder architecture.
Summary
This paper proposes a hybrid model (C+VAE)---a variational autoencoder (VAE) composed with a differentiable decision tree (DDT)---and an accompanying training scheme.  Firstly, the prior is specified as a mixture distribution with one component per class (SVAE).  During training, the ELBO’s KL term uses the component that corresponds to the known label.  Secondly, the DDT’s leaves are parametrized with the encoder distribution q(z|x), and thus gradient information flows back through the DDT into the posterior approximations in order to make them more discriminative.  Lastly, the VAE and DDT are trained together by alternating optimization of each component (plus a ridge penalty on the decoder means).  Experiments are performed on MNIST, demonstrating tree classification performance, (supervised) neg. log likelihood performance, and latent space interpretability via the DDT.  
Evaluation
Pros:  Giving the VAE discriminative capabilities is an interesting line of research, and this paper provides another take on tree-based VAEs, which are challenging to define given the discrete nature of the former and continuous nature of the latter.  Thus, I applaud the authors for combining the two in a way that admits efficient training.  Moreover, I like the qualitative experiment (Figure 2) in which the tree is used to vary a latent dimension to change the digit’s class.  I can see this being used for dataset augmentation or adversarial example generation, for instance.
Cons:  An indefensible flaw in the work is that the model is evaluated on only MNIST.  As there is no strong theory in the paper, this limited experimental evaluation is reason enough for rejection.  Yet, moreover, the negative log likelihood comparison (Table 2) is not an informative comparison, as it speaks only to the power of adding supervision.  Lastly, I do not think the interpretability provided by the decision tree is as great as the authors seem to claim.  Decision trees provide rich and interpretable structure only when each input feature has clear semantics.  However, in this case, the latent space is being used as input to the tree.  As the decision tree, then, is merely learning hard, class-based partitioning rules for the latent space, I do not see how the tree is representing anything especially revealing.  Taking Figure 2 as an example (which I do like the end result of), I could generate similar results with a black-box classifier by using gradients to perturb the latent ‘4’ mean into a latent ‘7’ mean (a la DeepDream).  I could then identify the influential dimension(s) by taking the largest absolute values in the gradient vector.  Maybe there is another use case in which a decision tree is superior; I’m just saying Section 4.3 doesn’t convince me to the extent that was promised earlier in the paper (and by the title).
Comment:  It's easier to make a latent variable model interpretable when the latent variables are given clear semantics in the model definition, in my opinion.  Otherwise, the semantics of the latent space become too entangled.  Could you, somehow, force the tree to encode an identifiable attribute at each node, which would then force that attribute to be encoded in a certain dimension of latent space?