Tagger: Deep Unsupervised Perceptual Grouping
We present a framework for efficient perceptual inference that explicitly reasons about the segmentation of its inputs and features. Rather than being trained for any specific segmentation, our framework learns the grouping process in an unsupervised manner or alongside any supervised task. We enable a neural network to group the representations of different objects in an iterative manner through a differentiable mechanism. We achieve very fast convergence by allowing the system to amortize the joint iterative inference of the groupings and their representations. In contrast to many other recently proposed methods for addressing multi-object scenes, our system does not assume the inputs to be images and can therefore directly handle other modalities. We evaluate our method on multi-digit classification of very cluttered images that require texture segmentation. Remarkably our method achieves improved classification performance over convolutional networks despite being fully connected, by making use of the grouping mechanism. Furthermore, we observe that our system greatly improves upon the semi-supervised result of a baseline Ladder network on our dataset. These results are evidence that grouping is a powerful tool that can help to improve sample efficiency.
The paper proposes a framework of unsupervised learning based on iterative denoising. The framework is designed for a scenario where multiple labels are in the same input, such as image segmentation. The basic idea of the proposed approach is analogous to an EM-like approach to learn a mixture model, where grouping assignments (mixture coefficients) and components are inferred over alternating iterations. The proposed framework takes a corrupted input, and applies as amortized inference a denoising model to estimate the groups and their assignments. The obtained estimations are further fed into the next iteration. The learning objective is the negative log-likelihood of the approximation of the true posterior of denoting, which is modeled here by the mixture of groups. Using artificially generated Shapes and Textured MNIST datasets, the paper evaluates the clustering and semi-supervised classification performance. The empirical results indicate the proposed approach achieves significantly better result in both supervised and semi-supervised scenario, especially when multiple objects are present in the image.
The proposed approach looks unique and interesting. The framework combines the generative model with fast amortized inference to derive efficient iterative approach. The denoising objective naturally fits in the problem setting to make learning feasible both under semi-supervised and unsupervised scenario. This technical proposal looks novel and I believe highly contributing.  The empirical results look also promising. Although the datasets are artificially generated and do not resemble a real-world situation as suggested in Fig 1, the significant performance improvement (Table 1) clearly indicates that the grouping is really helping in identifying multiple target objects in the same input. The experimental design is reasonable considering the purpose of the study.  My biggest complaint is the lack of details in the paper. Sec 2 and 3 do not reveal enough detail of how the denoising objective and the parametric mapping is designed, and I had hard time understanding the model. Especially, m_BU in Fig 2 is never explained in the paper, and readers would have no clue what the mapping model takes as input and output. Also, although the objective is modeled after denoising, the proposed model is quite different from a standard denoising auto-encoder and the paper should not simply state ``we apply denoising auto encoders [20]’’ (Sec 2) since the model takes latent variables m and z from the previous iteration.  I am curious why the paper states *perceptual* groups emerge out of the proposed model. The model seems to simply consider latent source signals in terms of denoising objective, and nothing constrains the groups. I can hardly imagine how the proposed model can lead to the segmentation example in Fig 1, or how the proposed model solves the ``dynamic, ambiguous and task-dependent’’ (Sec 1) grouping. To me, the proposed approach looks like a new, general generative framework based on denoising objective, and not tied to *perceptual* grouping as suggested in Sec 1.  One related question is the robustness to initialization. The paper does not describe how the latent variables are initialized in the experiments nor how one should initialize m and z in general. Perhaps random initialization can work fine in the synthesized datasets, but I do not expect the same in much more complicated input like the segmentation example in Fig 1. Also, it is not clear if the proposed approach always converges.  In overall, I would still give a positive rating. The paper is making a good contribution in proposing a novel clustering framework based on denoising formulation.