We propose the set autoencoder, a model for unsupervised representation learning for sets of elements. It is closely related to sequence-to-sequence models, which learn fixed-sized latent representations for sequences, and have been applied to a number of challenging supervised sequence tasks such as machine translation, as well as unsupervised representation learning for sequences. In contrast to sequences, sets are permutation invariant. The proposed set autoencoder considers this fact, both with respect to the input as well as the output of the model. On the input side, we adapt a recently-introduced recurrent neural architecture using a content-based attention mechanism. On the output side, we use a stable marriage algorithm to align predictions to labels in the learning phase. We train the model on synthetic data sets of point clouds and show that the learned representations change smoothly with translations in the inputs, preserve distances in the inputs, and that the set size is represented directly. We apply the model to supervised tasks on the point clouds using the fixed-size latent representation. For a number of difficult classification problems, the results are better than those of a model that does not consider the permutation invariance. Especially for small training sets, the set-aware model benefits from unsupervised pretraining.
Summary
This paper proposes an autoencoder for sets. An input set is encoded into a
fixed-length representation using an attention mechanism (previously proposed by
[1]). The decoder generates the output sequentially and the generated sequence
is matched to the best-matching ordering of the target output set.
Experiments are done on synthetic datasets to demonstrate properties of the
learned representation.
Pros
- Experiments show that the autoencoder helps improve classification accuracy
  for small training set sizes on the shape classification task.
- The analysis of how the decoder generates data is insightful.
Cons
- The experiments are on toy datasets only. Given the availability of point
  cloud data sets, for example, KITTI which has a widely used benchmark for
point cloud based object detection, it would make the paper stronger if this
model was benchmarked against published baselines.
- The autoencoder does not seem to help much on the regression tasks where even
  for the smaller training set size setting, directly using the encoder to solve
the task often works best. Even finetuning is unable to recover from the
pretrained weights. Therefore, it seems that the decoder (which is the novel
aspect of this work) is perhaps not working well, or is not well suited to the
regression tasks being considered.
- The classification task, for which the learned representations work well
  empirically, seems to be geared towards representing object shape. It doesn't
really require remembering each point. On the other hand, the regression tasks
that could require remembering the points don't seem to be benefit much from the
autoencoder pretraining. This suggests that while the model is able to represent
overall shape, it has a hard time remembering individual elements of the set.
This seems like a drawback, since a general "set auto-encoder" should be able
to perform a wide variety of tasks on the input set which could require remembering
the set's elements.
Quality
This paper describes the proposed model quite well and provides encouraging
preliminary results.
Clarity
The paper is easy to understand.
Originality
The novelty in the model is using a matching algorithm to find the best ordering
of the target output set to match with the sequentially generated decoder
output. However, the paper makes a choice of one ranking based matching scheme
and does not compare to other alternatives.
Significance
This paper proposes a way of learning representations of sets which will be of
broad interest across the machine learning community. These models are likely to
become more relevant with increasing prevelance of point cloud data.
References
[1] Oriol Vinyals, Samy Bengio, and Manjunath Kudlur. Order matters: Sequence to
sequence for sets. arXiv preprint arXiv:1511.06391.