Pose estimation is the task of finding the orientation of an object within an image with respect to a fixed frame of reference. Current classification and regression approaches to the task require large quantities of labelled data for their purposes. The amount of labelled data for pose estimation is relatively limited. With this in mind, we propose the use of Conditional Variational Autoencoders (CVAEs)  with circular latent representations to estimate the corresponding 2D rotations of an object. The method is capable of training with datasets that have an arbitrary amount of labelled images providing relatively similar performance for cases in which 10-20% of the labels for images is missing.
This paper tackles the task of rotation estimation in a setting where both labelled and unlabelled examples are available for training. It proposes to learn a generative model of images (a VAE), where the ‘code’ is factored into a latent vector z and the object rotation r.  As in training a VAE, an image encoder that predicts the distribution over (z, r) and the generator are jointly trained, but with additional supervision on the distribution over r for the labelled examples.
I think the overall idea of learning a disentangled generative model in a semi-supervised setting is simple and elegant, and could in principle help leverage unlabelled data. However,  I do have some concerns regarding the specific contributions of this work, and several reservations about the experiments reported, and would overall argue for rejection.
Concerns:
1) The central empirical result stated is that using this approach allows one to reduce amount of labelled data by 10-20 %. First, even if valid, this is not a very convincing reduction in the amount of supervision. However, I feel this claim is not well-established by the experiments:
1a)  The paper should report a baseline with only using the loss in eqn 3 and only training the encoder (using various fractions of training data) to predict the rotation i.e. purely discriminative training without training a generative model. The current plots of performance vs fraction of labelled data don't mean much until compared to a similar plot for this baseline. The current results don't really highlight the importance of training the generative model or using the unlabelled data.
1b) I think there are some inconsistencies in performances reported in Fig 2. I assume the test set is same despite different training data, because the paper states "All of the trained models are evaluated with respect to the complete test set". In this regard, I am puzzled why using 100% labelled data with 16 renders is significantly better than using 50% labelled data with 32 renders -- these should imply similar number of labelled examples, and more unlabelled ones in the former.
2) While the discussion points to this, the paper would really benefit from having results in a real setting, in particular as pose estimation is a field with a lot of prior methods that have been shown to work in these settings. The current results are all in a setup with synthetic, unoccluded data, without background variation, equidistant camera uniformly sampled along a circle. The central idea of using a generative model would be much more difficult to operationalize in a realistic setting where these simplifying assumptions are not made, and I'd only be convinced about the applicability of the approach by results in that setting. As a possible setup, one case use many imagenet images in conjunction with labelled examples in PASCAL3D+ to try this approach.
3) The overall approach maybe novel in context of pose estimation, but this idea of learning a disentangled generative model is not, and there are several papers which do so with varying amount of supervision e.g. see [1] below for similar ideas, and pointers. While some details here may vary, in context of these prior works, I'd view this paper as mostly applying well-established ideas to a new task.
--
In addition to the above, I have a question regarding the training/testing data:
Q: The dataset description only states data was randomly divided - was this random division at an image level, or model level i.e. could different renderings of the same model be in train and test set?
--
[1] Learning Disentangled Representations with Semi-Supervised Deep Generative Models, NIPS 2017. Siddharth et. al.