The Infinite Mixture of Infinite Gaussian Mixtures
Dirichlet process mixture of Gaussians (DPMG) has been used in the literature for clustering and density estimation problems. However, many real-world data exhibit cluster distributions that cannot be captured by a single Gaussian. Modeling such data sets by DPMG creates several extraneous clusters even when clusters are relatively well-defined. Herein, we present the infinite mixture of infinite Gaussian mixtures (I 2 GMM) for more flexible modeling of data sets with skewed and multi-modal cluster distributions. Instead of using a single Gaussian for each cluster as in the standard DPMG model, the generative model of I 2 GMM uses a single DPMG for each cluster. The individual DPMGs are linked together through centering of their base distributions at the atoms of a higher level DP prior. Inference is performed by a collapsed Gibbs sampler that also enables partial parallelization. Experimental results on several artificial and real-world data sets suggest the proposed I 2 GMM model can predict clusters more accurately than existing variational Bayes and Gibbs sampler versions of DPMG.
The authors present a novel non-parametric Bayesian model for unsupervised clustering. The model uses a two level hierarchy of Dirichlet process priors to handle clusters which may be multi-modal, skewed and/or heavy tailed. The authors present a collapsed Gibbs sampler for inference which exploits the conjugacy of the model.The authors do an excellent job of motivating the model by explaining the deficiencies of the standard infinite mixture of Gaussians. The review of previous works is good and the model is clearly presented and explained. The model is novel, though conceptually quite similar to the hierarchical Dirichlet process. The authors have chosen an excellent selection of synthetic and real world datasets to benchmark their method. The performance of the method seems to be quite good, in particular the FlowCAP results seem competitive with methods which were tailored to the problem domain.The authors only compare the model against the infinite Gaussians mixture model, with inference done using Gibbs sampling or VB. The paper could be improved if the authors compared against a richer set of models. In particular the authors may want to consider infinite mixtures of student-t's and skew normals to better clarify the advantages their model provides versus distributions which are more flexible than a single Gaussian emission. In addition other probabilistic models for clustering such as the one presented by Iwata et al. [1] are closer to the state of the art for clustering datasets with complex emission distributions.Another minor concern is that authors fix the concentration parameter for the Dirichlet process priors (line 308). I would expect that the model would be somewhat sensitive to the values of these parameters, particularly the relative values at the two levels. It would be helpful if the authors could explore performance on the synthetic dataset across a range of values to determine robustness to these parameters. In addition the method in [2] could readily be applied to place priors on these values instead of fixing them.Another very minor improvement would be for the authors to add details of the distributions and parameterisations they are using in the supplemental material. It would also be good to include explicit equations for the parameter updates described in lines 234-236.[1] Tomoharu Iwata, David Duvenaud, and Zoubin Ghahramani. Warped mixtures for nonparametric cluster shapes. In 29th Conference on Uncertainty in Artificial Intelligence, Bellevue, Washington, July 2013.[2] West, Mike. Hyperparameter estimation in Dirichlet process mixture models. Duke University, 1992.