Robust Bayesian Max-Margin Clustering
We present max-margin Bayesian clustering (BMC), a general and robust framework that incorporates the max-margin criterion into Bayesian clustering models, as well as two concrete models of BMC to demonstrate its flexibility and effectiveness in dealing with different clustering tasks. The Dirichlet process max-margin Gaussian mixture is a nonparametric Bayesian clustering model that relaxes the underlying Gaussian assumption of Dirichlet process Gaussian mixtures by incorporating max-margin posterior constraints, and is able to infer the number of clusters from data. We further extend the ideas to present max-margin clustering topic model, which can learn the latent topic representation of each document while at the same time cluster documents in the max-margin fashion. Extensive experiments are performed on a number of real datasets, and the results indicate superior clustering performance of our methods compared to related baselines.
The paper introduces max-margin Bayesian clustering (BMC) that extends Bayesian clustering techniques to include the max-margin criterion. This includes, for example, the Dirichlet process max-margin Gaussian mixture that relaxes the underlying Gaussian assumption of Dirichlet process Gaussian mixtures by incorporating max-margin posterior constraints, and is able to infer the number of clusters from data. The resulting techniques (DPMMGM and a further one classed MMCTM) are compared to a variety of other techniques in several numerical experiments.The paper combines two clustering approaches: Deterministic and Bayesian clustering. Deterministic approaches can easily handle constraints, but lack the ability to infer the number of clusters. Bayesian techniques can infer the number of clusters, but not handle constraints such as the max-margin. Quality:The resulting BMC algorithm is claimed to be the first extension of regularized Bayes inference to the unsupervised clustering task. Furthermore, the DPMMGM algorithm uses the max-margin constraints to relax the Gaussian assumptions of the standard DPGMM, which is a strong new capability.Clarity:(pg 4, 184) “The max-margin constraints take effects in the model via \tilde\phi_i’s in (8).” Perhaps this claim was better illustrated in earlier work, but why this statement is true is not made sufficiently clear in this paper. The authors need to provide better support for these claims.(pg 4, 208) “Note that DPMMGM does not need the complicated class balance constraints [6] because the Gausses in the pseudo likelihood would balance the clusters to some extent.” Not sure that “Gausses” is an appropriate wording. This statement ending of “to some extent” is not sufficiently precise for this type of publication.(pg 6, 298) controll is misspelled(pg6, 284) Fig 2 shows that the two algorithms generate different results, but the discussion provided here does not give sufficient indication of why the result of the DPMMGM is better. DPGMM is claimed to be more fragmented, but why is that significant here, and, if anything, the DPGMM appears to be a better fit to the data, so what has been lost in switching to the DPMMGM?(pg6, 295) “thus driving the data points to collapse as well.” Pls clarify this statement. Please also expand this discussion to connect the intuition to the figure 3 (a)-(j). There are a lot of figures, a lot of trends, and each figure is difficult to interpret with the lines and clusters. The conclusion “the results indeed follow our intuition,” is far less obvious than the authors claim.Originality:While it would appear that the ideas are original, much of section 3 on robust Bayesian max-margin clustering follows [17] very closely. In particular the key steps between equations (6)-(7)-(8) reuse the techniques introduced in [17]. This suggests the possibility that the core ideas are in the prior work, and the authors are just re-applying them here to a new model formulation, which calls into question the novelty and importance of this work. Without further clarification, this influences the impact score. Significance:The extensive numerical results show that DPMMGM outperforms DPGMM, and similarly with MMCTM vs. SVM and S3VM, which appear to be significant results. However, while the paper contains a lot of material on model development, there appear to be very few theoretical statements on the performance or convergence of the algorithms.