On Integrated Clustering and Outlier Detection
We model the joint clustering and outlier detection problem using an extension of the facility location formulation. The advantages of combining clustering and outlier selection include: (i) the resulting clusters tend to be compact and semantically coherent (ii) the clusters are more robust against data perturbations and (iii) the outliers are contextualised by the clusters and more interpretable. We provide a practical subgradient-based algorithm for the problem and also study the theoretical properties of algorithm in terms of approximation and convergence. Extensive evaluation on synthetic and real data sets attest to both the quality and scalability of our proposed method.
The paper describes an approach to clustering which deals automatically with the problem of outlier detection. The whole idea is based on the so-called Facility Location with Outliers (FLO) problem, introduced and studied in theoretical computer science in 2001 (see ref. [8] in the paper). The novelty of the proposed approach lies mainly in the study ot the Lagrangian relaxation of the original (integer) linear programming problem. In particular, the authors show that the Lagrangian relaxation is equivalent to solving the linear relaxation of FLO, and also analyze the convergence properties of the subgradient method used, which makes the algorithm highly scalable. Some experiments on both synthetic and real-world data sets (MINST) are presented.The paper is well written and organized, it’s easy to follow and understand, despite many technicalities, and its motivations and goals are quite clear.Also, I like the idea of introducing, within the machine learning community, this apparently novel formulation of the clustering problem which, to my knowledge, was originally confined only to the theoretical computer science domain. In regards to the statement made by the authors that “clustering and outlier detection are often studied as separate problems,” however, I’d like to draw the authors’ attention to novel clustering formulations which, abandoning the idea of partitioning the input data set, focus instead on the notion of a cluster itself (see, e.g., M. Pelillo, “What is a cluster?” NIPS’09 Workshop on Clustering, and references therein). In fact, in these approaches, which are non-partitional by definition, one can see that the problem of clustering and outlier detetion are simply two faces of the same coin.One problem with the proposed formulation, though, is that while it doesn’t need as input the number of clusters, which is of course good, it neverthless needs to know the number of outliers, which is even more problematic than knowing the number of clusters. Also, it needs to know the “cost” of creating a new cluster, and here, again, it’s not clear how to define them in practical applications (the choice made in the experiments reported here look quite heuristic).My final comment concerns the experimental validation. Indeed, the results presented here do not show a clear and substantial improvement over the other methods used. This is particularly clear in the MNIST experimemts (Table 1) which show that APOC and LR are basically equivalent, and k-means-- is better than the proposed LR algorithm.Minor comments:- Definition 2 is indeed a Proposition