In this paper, we propose a framework that leverages semi-supervised models to improve unsupervised clustering performance. To leverage semi-supervised models, we first need to automatically generate labels, called pseudo-labels. We find that prior approaches for generating pseudo-labels hurt clustering performance because of their low accuracy. Instead, we use an ensemble of deep networks to construct a similarity graph, from which we extract high accuracy pseudo-labels. The approach of finding high quality pseudo-labels using ensembles and training the semi-supervised model is iterated, yielding continued improvement. We show that our approach outperforms state of the art clustering results for multiple image and text datasets. For example, we achieve 54.6% accuracy for CIFAR-10 and 43.9% for 20news, outperforming state of the art by 8-12% in absolute terms.
This paper proposes a method for unsupervised clustering. Similarly to others unsupervised learning (UL) papers like "Deep Clustering for Unsupervised Learning of Visual Features" by Caron et al., they propose an algorithm alternating between a labelling phase and a training phase. Though, it has interesting differences. For example, unlike the Caron et al. paper, not all the samples get assigned a labels but only the most confident ones. These samples are determined by the pruning of a graph whose edges are determined by the votes of an ensemble of clustering models. Then, these pseudo labels are used within a supervised loss which act as a regularizer for the retraining of the clustering models.
Novelties /contributions/good points:
* Votes from the clustering models to create a graph
* Using a graph to identify the most important samples for pseudo labelling
* Modification of the ladder network to be used as clustering algorithm
* Good amount of experiments and good results
Weaknesses:
* The whole experiment leading to Table 1 in page 2 is unclear for me. I have trouble understanding the experiment settings. Could you please rephrase it. About initial/ final clustering for example and the rest as well. The whole thing puzzles me whereas the experiments section at the end is much more clear.
* Lack of motivation about why using the Ladder method rather than another one. Other recent methods have better results in semi-supervised learning.
* Algorithm 1 seems quite ad-hoc. Do more principled algos exist to solve this problem ? You could write about it and at least explain why it would not be feasible here. The sentence "The intuition is that most of the neighbours of that node will also be connected with each other" is unmotivated: no empirical proof for this ?
* Related work section is too light. It is an important section and should really not be hidden or neglected.
* In the experiments, you could add the "Deep Clustering for Unsupervised Learning of Visual Features"  as baseline as well even if they use it for unsupervised learning as they do clustering as well.
* In the experiments, you use the features extracted from ResNet-50 but what about finetuning this network rather than adding something on top or even better starting from scratch. Because here CIFAR-10 benefits greatly from the ImageNet features. I know that you should reproduce the settings from other papers but it might be good to go a bit beyond. Especially, if the settings of previous papers are a bit faulty. 
* Regarding, the impact of number of models in section D of the appendix, there is no saturation at 10 models. So how many models are necessary for saturation of the performance ?
* Minor point: several times, you write "psuedo".
Conclusion: the algorithm is novel and represents a nice contribution. Though, there are a lot of weaknesses that could be solved. So, I am putting "Weak accept" for the moment but it could change towards a negative rating depending on the rebuttal.