Published as a conference paper at ICLR 2018 LEARNING LATENT REPRESENTATIONS IN NEURAL NETWORKS FOR CLUSTERING THROUGH PSEUDO SUPERVISION AND GRAPH-BASED ACTIVITY REGULARIZATION
In this paper, we propose a novel unsupervised clustering approach exploiting the hidden information that is indirectly introduced through a pseudo classification objective. Specifically, we randomly assign a pseudo parent-class label to each observation which is then modified by applying the domain specific transformation associated with the assigned label. Generated pseudo observation-label pairs are subsequently used to train a neural network with Auto-clustering Output Layer (ACOL) that introduces multiple softmax nodes for each pseudo parent-class. Due to the unsupervised objective based on Graph-based Activity Regularization (GAR) terms, softmax duplicates of each parent-class are specialized as the hidden information captured through the help of domain specific transformations is propagated during training. Ultimately we obtain a k-means friendly latent representation. Furthermore, we demonstrate how the chosen transformation type impacts performance and helps propagate the latent information that is useful in revealing unknown clusters. Our results show state-of-the-art performance for unsupervised clustering tasks on MNIST, SVHN and USPS datasets, with the highest accuracies reported to date in the literature.
This paper presents a method for clustering based on latent representations learned from the classification of transformed data after pseudo-labellisation corresponding to applied transformation. Pipeline: -Data are augmented with domain-specific transformations. For instance, in the case of MNIST, rotations with different degrees are applied. All data are then labelled as "original" or "transformed by ...(specific transformation)". -Classification task is performed with a neural network on augmented dataset according to the pseudo-labels. -In parallel of the classification, the neural network also learns the latent representation in an unsupervised fashion. -k-means clustering is performed on the representation space observed in the hidden layer preceding the augmented softmax layer. 
Detailed Comments:
(*) Pros
-The method outperforms the state-of-art regarding unsupervised methods for handwritten digits clustering on MNIST.
-Use of ACOL and GAR is interesting, also the idea to make "labeled" data from unlabelled ones by using data augmentation.
(*) Cons
-minor: in the title, I find the expression "unsupervised clustering" uselessly redundant since clustering is by definition unsupervised.
-Choice of datasets: we already obtained very good accuracy for the classification or clustering of handwritten digits. This is not a very challenging task.
And just because something works on MNIST, does not mean it works in general. 
What are the performances on more challenging datasets like colored images (CIFAR-10, labelMe, ImageNet, etc.)?
-This is not clear what is novel here since ACOL and GAR already exist. The novelty seems to be in the adaptation to GAR from the semi-supervised to the unsupervised setting with labels indicating if data have been transformed or not.
My main problem  was about the lack of novelty. The authors clarified this point, and it turned out that ACOL and GAR have never published elsewhere except in ArXiv.  The other issue concerned the validation of the approach on databases other than MNIST. The author also addressed this point, and I changed my scores accordingly.