We propose a probabilistic generative model for unsupervised learning of structured, interpretable, object-based representations of visual scenes. We use amortized variational inference to train the generative model end-to-end. The learned representations of object location and appearance are fully disentangled, and objects are represented independently of each other in the latent space. Unlike previous approaches that disentangle location and appearance, ours generalizes seamlessly to scenes with many more objects than encountered in the training regime. We evaluate the proposed model on multi-MNIST and multi-dSprites data sets.
This paper introduces a compositional generative model of images, where the image is described by a variable number of latent variables. Moreover, the latent variables are disentangled, in the sense that they represent different parts of the scene, and where appearance and location are described separately. While the generative model is very similar to AIR [1], the central claim of the paper is that the proposed inference scheme can generalize to a much higher number of objects than seen during training, which is demonstrated empirically, and with which AIR struggles. Better generalization is achieved by removing the recurrent core of AIR and replacing it with a fully-convolutional inference network, which predicts appearance vectors at every location in the feature map. The appearance vectors are then stochastically sampled without replacement according to a location distribution. Unlike AIR, this approach does not model the object scale.
I recommend REJECTing this paper. While the improved generalization performance is a useful property, it has been achieved previously and in a very similar fashion by SPAIR [2], which follows a similar model design. SPAIR still uses an RNN, while the proposed approach does not, but this is only a small simplification and does not warrant publication at a top tier conference. There are no other contributions in this paper. Additionally, on the one hand, the experimental evaluation is insufficient: the proposed approach is compared only against a fully-convolutional VAE, while very similar models like AIR [1], SPAIR [2], SuPAIR [3] are not considered. On the other hand, the experimental section focuses on the disentanglement of representations, which is a) evident from the model construction and b) achieved in all previous models. 
The paper is clearly written, and the presented generative model is interesting. Having said that, the problem that this paper addresses is mostly solved in [2] and [4]; also both [2] and [4] scale the general approach introduced in [1] to much higher number of objects (in the hundreds) and more general settings (real images, atari games).
[1] Eslami et. al., “Attend, Infer, Repeat:...”, NIPS 2016.
[2] Crawford and Pineau, “Spatially Invariant Unsupervised Object Detection with Convolutional Neural Networks”, AAAI 2019.
[3] Stelzner et. al., “Faster Attend-Infer-Repeat with Tractable Probabilistic Models”, ICML 2019.
[4] Jiang et. al., “Scalable Object-Oriented Sequential Generative Models”, arXiv 2019.