On a Theory of Nonparametric Pairwise Similarity for Clustering: Connecting Clustering to Classification
Pairwise clustering methods partition the data space into clusters by the pairwise similarity between data points. The success of pairwise clustering largely depends on the pairwise similarity function defined over the data points, where kernel similarity is broadly used. In this paper, we present a novel pairwise clustering framework by bridging the gap between clustering and multi-class classification. This pairwise clustering framework learns an unsupervised nonparametric classifier from each data partition, and search for the optimal partition of the data by minimizing the generalization error of the learned classifiers associated with the data partitions. We consider two nonparametric classifiers in this framework, i.e. the nearest neighbor classifier and the plug-in classifier. Modeling the underlying data distribution by nonparametric kernel density estimation, the generalization error bounds for both unsupervised nonparametric classifiers are the sum of nonparametric pairwise similarity terms between the data points for the purpose of clustering. Under uniform distribution, the nonparametric similarity terms induced by both unsupervised classifiers exhibit a well known form of kernel similarity. We also prove that the generalization error bound for the unsupervised plugin classifier is asymptotically equal to the weighted volume of cluster boundary [1] for Low Density Separation, a widely used criteria for semi-supervised learning and clustering. Based on the derived nonparametric pairwise similarity using the plug-in classifier, we propose a new nonparametric exemplar-based clustering method with enhanced discriminative capability, whose superiority is evidenced by the experimental results.
This paper presents clustering framework using pairwise similarity function model.Different data partitions are used to construct different nonparametric classifiers.The optimal clustering is obtained by minimizing the generalization error of the learned classifiers related to data partitions.The authors consider two nonparametric classifiers in their framework:the plug-in and the nearest-neighbor classifier. The authors prove also a result interesting in itself, namely that the generalizationerror bound for the unsupervised plug-in clustering is asymptotically  equal to theweighted volume of cluster boundary for Low Density Separation. This paper tights together two important machine learning subjects: multi-class classification problem and clustering. Furthermore, it has strong theoretical background.The paper is however very poorely written.The introduction is well-organized however in the next sections the authorsgo into several technical details and do not summarize their results in the compact understandable form. Two technical lemmas given in Section 2 only deepenthe confusion. Thus, it is extremely difficult to measure the real impact of this paper.Section 2 contains too many technical details that are given before giving the reader any sort of intuition. Besides the presented lemma statements are completely unclear. For instance:Lemma 1: - is it possible to say how n_{0} depends on \sigma_{0} and VC characteristics of K ? - the statements seems not to be put in the grammatical form, the authors should review itLemma 2: - again the authors introduce many parameters that mutually depend on each other,   it is almost impossible to guess from all the asusmptions required by the lemma what it    says in factLemma 3: - similar problems to those mentioned aboveLemma statements are unreadable and some of them are put at the very beginning of the paper.The authors should spend much more time to work on the presentation of their results.Maybe some summary of the obtained theoretical results should be conducted before giving exact statements.The authors should also give some intuition regarding all lemmas presented in the main body of the paper.The authors present how their techniques may be applied to the exemplar-based clustering but the comparisonwith existing state-of-the-art methods is missing. The experimental section is very tiny and does not shed any light of advantages of the technique developed by the authors over other clustering algorithms.The difficulty of the clustering heavily depends on the considered setting: whether data is truly high-dimensional,or maybe lies on the small-dimensional manifolds, whether data points are sparse or dense, whether the groundtruth clusters are convex or there are noassumptions about their shape, what is the objective function the authors aim to minimize or the groundtruthclustering they want to approximate well. Without detailed analysis of the performance of the algorithm in the specificsetting defined by these parameters it is almost impossible to say whether the presented method can be applied in practice and solves a nontrivial problem.What is the computational complexity of the algorithms that use presented technique ? This issue is not addressed at all.Is it possible to extend the analysis for other classifiers (only nearest-neighbor and plug-in classifiers were analyzed) ? Is it possible to release some source code files (with the implementation of the presented method) that were used by the authors to test their approach ? (The last remark is not crucial though.)