Hierarchical Bayesian methods have the potential to unify many related tasks (e.g. k-shot classification, conditional, and unconditional generation) by framing each as inference within a single generative model. We show that existing approaches for learning such models can fail on expressive generative networks such as PixelCNNs, by describing the global distribution with little reliance on latent variables. To address this, we develop a modification of the Variational Autoencoder in which encoded observations are decoded to new elements from the same class; the result, which we call a Variational Homoencoder (VHE), may be understood as training a hierarchical latent variable model which better utilises latent variables in these cases. Using this framework enables us to train a hierarchical PixelCNN for the Omniglot dataset, outperforming all existing models on test set likelihood. With a single model we achieve both strong one-shot generation and near human-level classification, competitive with state-of-the-art discriminative classifiers. The VHE objective extends naturally to richer dataset structures such as factorial or hierarchical categories, as we illustrate by training models to separate character content from simple variations in drawing style, and to generalise the style of an alphabet to new characters.
- Good work on developing VAEs for few-shot learning.
- Most of the results are qualitative and I reckon the paper was written in haste.
- The rest of the comments are below:
- 3.1: I got a bit confused over what X actually is:
 -- "We would like to learn a generative model for **sets X** of the form".
 --"... to refer to the **class X_i** ...".
 -- "we can lower bound the log-likelihood of each **dataset X** ..."
- 3.2: "In general, if we wish to learn a model for X in which each latent variable ci affects some arbitrary subset Xi of the data (**where the Xi may overlap**), ...": Which is just like learning a Z for a labeled X but learning it in an unsupervised manner, i.e. the normal VAE, isn't it? If not, could you please elaborate on what is different (in the case of 3.2 only, I mean)? i.e. Could you please elaborate on what's different (in terms of learning) between 3.2 and a normal latent Z that is definitely allowed to affect different classes of the data without knowing the classes?
- Figure 1 is helpful to clarify the main idea of a VHE.
- "In a VHE, this recognition network takes only small subsets of a class as input, which additionally ...": And that also clearly leads to loss of information that could have been used in learning. So there is a possibility for potential regularization but there is definitely a big loss in estimation power. This is obviously possible with any regularization technique, but I think it is more of an issue here since parts of the data are not even used in learning.
- "Table 4.1 compares these log likelihoods, with VHE achieving state-of-the-art. To": Where is Table 4.1??
- This is a minor point and did not have any impact on the evaluation but VAE --> VHE, reparameterization trick --> resampling trick. Maybe providing rather original headings is better? It's a style issue that is up to tastes anyway so, again, it is minor.
- "However, sharing latent variables across an entire class reduces the encoding cost per element is significantly": typo.
- "Figure ?? illustrates".