Multiclass Total Variation Clustering
Ideas from the image processing literature have recently motivated a new set of clustering algorithms that rely on the concept of total variation. While these algorithms perform well for bi-partitioning tasks, their recursive extensions yield unimpressive results for multiclass clustering tasks. This paper presents a general framework for multiclass total variation clustering that does not rely on recursion. The results greatly outperform previous total variation algorithms and compare well with state-of-the-art NMF approaches.
The authors study a variant of ratio cut with R 
      clusters where the balancing function is biased towards partitions 
      where each cluster has the same size. The main contribution of the 
      paper is a continuous formulation and an algorithm to optimize the 
      criterion directly, whereas previous algorithms are mostly limited to 
      recursive splitting. The direct solution of multi-cut problems 
      instead of using recursive splitting is an important problem given the 
      new developments in finding balanced graph cuts [3,4,5,11,12,18]. 
      The authors first describe the discrete problem (P) and then 
      derive a relaxation of the problem (P-rlx). Opposite to previous 
      work [11,12,17,18] for the two cluster case, they cannot show that 
      this relaxation is exact, that is the discrete and continuous problems 
      are equivalent. While they argue that the relaxation is very good, 
      this is unfortunately not necessarily true. If there exists a 
      minimizer f=ones_A of the *single* ratio problem which is far better 
      than all other ones (in the extreme case A and A^c are disconnected), 
      then the solution of (P-rlx) is just f_r=c_r 1_A or f_r=c_r 1_A^c and 
      the c_r are chosen such that they sum up to one for every vertex 
      (note that rescaling of f_r does not change the objective). Thus 
      the condition that F generates a partition is *not* enforced by the 
      simplex constraint. In the extreme case where there exists a set A 
      which is disconnected, then the optimal value of the relaxation (P-rlx) 
      is zero whereas the optimal value of the discrete problem (P) clearly 
      need not be zero - thus the relaxation can be arbitrarily bad. A 
      more careful discussion is needed rather than just arguing that the 
      total variation enforces indicator functions in Section 3.1. - this is 
      well-known already in image processing and can be made precise using 
      the relation between submodular set functions and their Lovasz 
      extensions and was the basis of the previous work [3,4,5,11,12,18] to 
      prove that the relaxation is exact. The new claim that the simplex 
      constraint enforces that the sets form a partition is not shown and it 
      is impossible to show as the above counterexample suggests. The 
      algorithm in 4.2. looks similar to previous work and the integration of 
      the simplex constraint is rather standard. I cannot see how the estimate 
      in Theorem 3 is related to descent of the total energy - the authors argue 
      that the ratio B_r^{k+1}/B_r^k tends to one and thus the total energy 
      decreases. However, this is not proven anywhere. It is not even clear 
      if the algorithm does converge. The experimental results are good 
      even though [19] seems to perform better. However, the actual point of 
      the paper that MTV leads to better normalized cuts is not documented, 
      as just clustering errors are reported - at least for the previous 
      methods using recursive splitting [3,11] and MTV one could have shown 
      also the normalized cut values. Pro: - direct tackling 
      multi-normalized-cut is an important problem - new algorithm - 
      good experimental results Con: - relaxation is not tight as 
      argued by the authors - it can be arbitrarily bad - no convergence 
      proof for the algorithm or even descent is guaranteed 
      ------------------------------------------------------------------------------ 
      Added after author response: Tightness: The suggested 
      relaxation is not tight as the example in the review shows - and this 
      problem will remain even if the graph is connected and there is one 
      dominating cut. In the case where the graph is disconnected the 
      suggested relaxation has value zero whereas the original discrete 
      problem need not. Also the spectral relaxation would yield a better 
      relaxation as it sums up the first k eigenvalues (where k is the number of 
      clusters) which would be non-zero (say k=10 and the graph has two 
      connected components). This argument holds even for criterion (P) as 
      one can easily lower- and upper bound it in terms of (1). The 
      argument that the algorithm anyway just finds a solution close to an 
      initial starting point is very weak. If this is the case then the 
      suggested algorithm is very bad. Moreover, then the problem is 
      transferred to finding a good initial solution which is as difficult 
      as the original problem and also not discussed in the paper. 
      Monotonicity and Convergence: If these results are 
      available then it is unclear why they were not put into the 
      paper/supplementary material. Anyway even if the algorithm has better 
      properties as reported in the paper, the problems with the continuous 
      relaxation itself remain. Purity: To report purity is 
      definitely fine. However, a comparison to recursive splitting techniques 
      [3,11] or standard spectral clustering (with k-means in the embedding 
      space) could have been done both with respect to criterion (1) and the 
      suggested criterion (P), in order to judge if the suggested algorithm 
      really produces better cuts - as this is the only criterion one has in 
      practice to judge a given partition. It would even be interesting to 
      see the performance of the NMF methods there.