Tree-Guided MCMC Inference for Normalized Random Measure Mixture Models
Normalized random measures (NRMs) provide a broad class of discrete random measures that are often used as priors for Bayesian nonparametric models. Dirichlet process is a well-known example of NRMs. Most of posterior inference methods for NRM mixture models rely on MCMC methods since they are easy to implement and their convergence is well studied. However, MCMC often suffers from slow convergence when the acceptance rate is low. Tree-based inference is an alternative deterministic posterior inference method, where Bayesian hierarchical clustering (BHC) or incremental Bayesian hierarchical clustering (IBHC) have been developed for DP or NRM mixture (NRMM) models, respectively. Although IBHC is a promising method for posterior inference for NRMM models due to its efficiency and applicability to online inference, its convergence is not guaranteed since it uses heuristics that simply selects the best solution after multiple trials are made. In this paper, we present a hybrid inference algorithm for NRMM models, which combines the merits of both MCMC and IBHC. Trees built by IBHC outlines partitions of data, which guides Metropolis-Hastings procedure to employ appropriate proposals. Inheriting the nature of MCMC, our tree-guided MCMC (tgMCMC) is guaranteed to converge, and enjoys the fast convergence thanks to the effective proposals guided by trees. Experiments on both synthetic and realworld datasets demonstrate the benefit of our method.
The paper proposes an efficient MCMC inference for normalized random measure mixture models. Borrowing the idea of hierarchical Bayesian clustering,the inference algorithm uses tree structures to represent cluster and develops stochastic tree operations to generate proposals; it inherits the benefit of incremental Bayesian clustering algorithm (ICBH)---the operation over trees is efficient and the tree-guided proposals are empirically effective; it overcomes the deficiency of IBHC by developing the inference rigorously in MCMC framework: the transition probability is clearly defined and the convergence is theoretically guaranteed. The paper is well written and the work is solid. The only issue to me is that on real dataset, the proposed algorithm shows slower mixing rate than the competing methods, which is a little disappointing; maybe the authors should test more real datasets for confirmation.