Incremental Clustering: The Case for Extra Clusters
The explosion in the amount of data available for analysis often necessitates a transition from batch to incremental clustering methods, which process one element at a time and typically store only a small subset of the data. In this paper, we initiate the formal analysis of incremental clustering methods focusing on the types of cluster structure that they are able to detect. We find that the incremental setting is strictly weaker than the batch model, proving that a fundamental class of cluster structures that can readily be detected in the batch setting is impossible to identify using any incremental method. Furthermore, we show how the limitations of incremental clustering can be overcome by allowing additional clusters.
This paper presents a number of theoretical results characterizing the limitations/capabilities of incremental clustering. Below I summarize some of the key results, skipping some of less significance (from my point of view). One of the key results is that there exists no deterministic memory-bounded incremental method that can guarantee to detect nice clustering (i.e., a clustering of data in which every element is closer to points in its own cluster than to points in other clusters) from data when a unique nice clustering exists. The main idea of the proof  is to construct a data set with a unique nice clustering and demonstrat that any deterministic incremental algorithm that's memory bounded will necessarily fail with some ordering of the data. The paper also shows that perfect clustering, a stronger notion of clusterability, can be detected by a single-link based incremental algorithm, but not by the popular incremental k-means. By allowing extra clusters, the paper shows that an incremental algorithm can find a refinement of the target nice clustering if allowed 2^{k-1} centers and also shows that it is not possible to use fewer cluster centers.Given that the definition of nice clustering requires a somewhat global view of the clustering structure, it is somewhat intuitive that memory bounded incremental algorithms will fail. Non-the-less, the lower bound results are quite interesting and original (to the best of my knowledge). I followed most of the proofs provided in the paper as well as the appendix and they appear to be sound and some of them non-trivial.  I think they present solid contribution to the theoretical understanding of incremental clustering.  The presentation of the paper has some room for improvement. In particular, the paper itself lacks balance in its space allocation. The first result took 6 pages and was given the (almost) full proof, whereas the later results are provided without any proof or even sketch of proof.  A reader will have to read the supplementary materials to get through the rest of the paper. Further more, the supplementary materials were not separated from the main paper. Rather, an extended version was presented as the supplementary. This makes it difficult to find the proof for a particular theorem because the numbers are not consistent.  I appreciate the attempt to provide the high level intuition for the proof of the first result, but I find the explanation difficult to understand without the detailed proof. I think this explanation needs some work to make it really useful (perhaps with some figures?).  Another issue with the presentation is that it seems to be trying to fit too much into the paper and some of the results seem dis-connected from the rest of the paper. For example, the convex-nice seems to be mainly introduced to help sequential k-means? All previous sections focus on deterministic algorithms, whereas section 5 jumps to a random algorithm, which again seems only loosely connected.Another thing on my wish list for this paper is some concrete practical understanding of these results. For example, the lower bound result on using extra clusters is quite discouraging, requiring at least 2^{k-1} clusters in order to find a proper refinement of the target nice-clustering. In practice, will this be what we typically observe? I understand that this is a theoretical paper, none-the-less, it would have been nice to see some (perhaps) synthetic experiments to demonstrate what to expect in reality when the proposed algorithm is put to use.