  We present an unsupervised learning framework for decomposing images into
layers of automatically discovered object models. Contrary to recent approaches
that model image layers with autoencoder networks, we represent them as
explicit transformations of a small set of prototypical images. Our model has
three main components: (i) a set of object prototypes in the form of learnable
images with a transparency channel, which we refer to as sprites; (ii)
differentiable parametric functions predicting occlusions and transformation
parameters necessary to instantiate the sprites in a given image; (iii) a
layered image formation model with occlusion for compositing these instances
into complete images including background. By jointly learning the sprites and
occlusion/transformation predictors to reconstruct images, our approach not
only yields accurate layered image decompositions, but also identifies object
categories and instance parameters. We first validate our approach by providing
results on par with the state of the art on standard multi-object synthetic
benchmarks (Tetrominoes, Multi-dSprites, CLEVR6). We then demonstrate the
applicability of our model to real images in tasks that include clustering
(SVHN, GTSRB), cosegmentation (Weizmann Horse) and object discovery from
unfiltered social network images. To the best of our knowledge, our approach is
the first layered image decomposition algorithm that learns an explicit and
shared concept of object type, and is robust enough to be applied to real
images.

Computer Vision and Pattern Recognition