  Learning to insert an object instance into an image in a semantically
coherent manner is a challenging and interesting problem. Solving it requires
(a) determining a location to place an object in the scene and (b) determining
its appearance at the location. Such an object insertion model can potentially
facilitate numerous image editing and scene parsing applications. In this
paper, we propose an end-to-end trainable neural network for the task of
inserting an object instance mask of a specified class into the semantic label
map of an image. Our network consists of two generative modules where one
determines where the inserted object mask should be (i.e., location and scale)
and the other determines what the object mask shape (and pose) should look
like. The two modules are connected together via a spatial transformation
network and jointly trained. We devise a learning procedure that leverage both
supervised and unsupervised data and show our model can insert an object at
diverse locations with various appearances. We conduct extensive experimental
validations with comparisons to strong baselines to verify the effectiveness of
the proposed network.

Computer Vision and Pattern Recognition