Clustering with Same-Cluster Queries
We propose a framework for Semi-Supervised Active Clustering framework (SSAC), where the learner is allowed to interact with a domain expert, asking whether two given instances belong to the same cluster or not. We study the query and computational complexity of clustering in this framework. We consider a setting where the expert conforms to a center-based clustering with a notion of margin. We show that there is a trade off between computational complexity and query complexity; We prove that for the case of k-means clustering (i.e., when the expert conforms to a solution of k-means), having access to relatively few such queries allows efficient solutions to otherwise NP hard problems. In particular, we provide a probabilistic polynomial-time (BPP) algorithm for clustering in this setting that asks O k 2 log k + k log n) same-cluster queries and runs with time complexity O kn log n) (where k is the number of clusters and n is the number of instances). The algorithm succeeds with high probability for data satisfying margin conditions under which, without queries, we show that the problem is NP hard. We also prove a lower bound on the number of queries needed to have a computationally efficient clustering algorithm in this setting.
The paper looks at center-based clustering in Euclidean space. While it is known that the general k-means (or k-median) problem is hard in this setting, it is also known that a variety of "instance niceness" lead to poly-time clustering algorithms. The paper adds to this literature by examining the case where the instance is also "nice" *and* the algorithm has oracle access to a domain expert that is able to say whether two datapoints reside in the same cluster or not. The authors introduce an alternative notion of niceness, which they call "\gamma-margin" (Def. 1) which basically states for the every cluster center c_i, any point x in cluster i and any point y not-in cluster i we have d(y,c_i)>\gamma d(x,c_i). Then, they provide the following two results for such nice inputs: 1. (upper bound) give an efficient algorithm making O(k^2log(k) + k log(n)) oracle queries and clusters the instance perfectly. 2. (lower bound) prove that no poly-time algorithm with no access to an oracle can cluster a \sqrt{3.4}-margin instance. As a result, no poly-time algorithm can cluster a \sqrt{3.4}-margin instance with log(n) oracle calls (as one can naively traverse all possible the answers to these queries in poly-time).
This is a great paper that takes a refreshing and a novel view on an interesting line of works, and while the results are not necessarily hard technically, they are still of major significance to the field. I therefore recommend acceptance and even further - a presentation, as it can stir up an interesting discussion and lead to many follow up works. I know that I for one had many interesting follow-up questions: can such an oracle be used with other stability assumptions? what if the domain expert thinks of a clustering which is not center-based but can be well point-wise approximated by a center-based clustering? can the oracle be used to prune the tree used in [KSS04]'s O(n * exp(k)) approximation of the k-means problem?  My one major complaint is regarding the lower bound presented in the paper - which is confusing, even when one delves in the cumbersome notation. I believe that there are a few technical errors there and presentation errors that subtract from the quality of the paper and the fact that it is extremely well-written otherwise. I expect the authors' rebuttal to include answers to the following questions, as I am afraid they might lead to substantial changes to the proof of the lower bound: (1) Z_l is ill-defined (you cannot define Z_l using S_{l+1} (lines 261-262) (2) Figure 1 uses "d-\epsilon" rather than "d-\alpha" (unless I didn't understand the role of \alpha) (3) It is my understanding the L_1 is the cost of a type B cluster *per R_i* row (line 131 of appendix) and L_1-\alpha per *R_i* row with type A clustering.  And so, I believe L should include a term of L_1*l. (4) As someone who is not familiar with [Vat09], I do not see the "point" of the construction. Namely -- what should "yes" instances be mapped to? Is it to m type-A rows (indicating which sets are selected) and (l-m)-type B rows (the l-m sets not selected)? Or is it a certain selection of the B_{i,j} sets that captures which sets are taken and which aren't? (5) The construction itself is somewhat unclear. In particular, what is w? Is it \eps^{-0.5} (with \eps taken as a parameter) or some large poly(m,l) as stated in the supplementary material? I am unclear as to what is the value of k = O(n^\eps) in this construction... In other words, try to rewrite the construction s.t. it would be clear what the algorithm takes as input, and what is its output. (6) I'd appreciate it if you specifically mention the caveats in the lower bound (actually, this goes to Appendix B as well, as some constructions use Steiner points etc.). In particular, this construction places multiple points at the same location, resulting in an instance with infinite (or exponentially high if you allow miniature points perturbations), but it leaves the question of hard instances with bounded aspect ratio open...  Small comments (not to mention - nitpicking comments): * Lines 152-153 should be mentioned explicitly in the intro, or even written in an "Assumption" bulletin. * Algorithm 1 also takes as parameter \beta (or sets \beta as some sufficiently large constant, say 20.) In which case the Algorithm 1's input should be "a parameter" and not "parameters". * Can the (\gamma-1)^{-4} terms be improved to (\gamma-1)^{-2} if you use Chernoff bounds rather than Hoeffding? (i.e. multiplicative bounds on the expected value?) * Line 213 "of the oracle" => "of an oracle" * Line 217 "Section 3" (capital letter) * Line 220 "without query" => "without queries" * Suggestions: use the canonical \textsc{} to denote the X3C problem. * Section 4.1.2 - use \mathcal{X} like used at the rest of the paper. * Thm 14: as k<|X| we have that this is \log(n) effectively. Also, I am not a fan of using big-O notation in a lower bound theorem (but the theorem statement is clear).