Beta-Negative Binomial Process and Exchangeable Random Partitions for Mixed-Membership Modeling
The beta-negative binomial process (BNBP), an integer-valued stochastic process, is employed to partition a count vector into a latent random count matrix. As the marginal probability distribution of the BNBP that governs the exchangeable random partitions of grouped data has not yet been developed, current inference for the BNBP has to truncate the number of atoms of the beta process. This paper introduces an exchangeable partition probability function to explicitly describe how the BNBP clusters the data points of each group into a random number of exchangeable partitions, which are shared across all the groups. A fully collapsed Gibbs sampler is developed for the BNBP, leading to a novel nonparametric Bayesian topic model that is distinct from existing ones, with simple implementation, fast convergence, good mixing, and state-of-the-art predictive performance.
This paper presents a new Bayesian nonparametric model for mixed-membership modeling of grouped data. The model is based on a Beta-Negative Binomial Process (BNBP). Whereas this stochastic process was exploited by previous work, as mentioned in the paper, there are several aspects that set this work apart. First, this work obtains an EPPF (exchangeable partition probability function), and derives the prediction rules thereon. The relations between Bayesian nonparametrics and EPPF has been studied for quite a while in statistics, following the seminal work by J. Pitman. Yet, this concept has not been extensively leveraged in the practice of BNP research. The connection between BNBP and EPPF made in this paper, to me, represents an important step along the direction of unifying the EPPF analysis and BNP research, which would potentially lead to more work along this line.Second, the idea of randomizing the group sizes (which are treated as given constants in most other work) is interesting. It turns out that this idea makes it possible to derive some marginal distributions.I think this paper has important theoretical contributions. However, the practical utility of this work remains in question. The inference algorithm is obviously more complex than HDP (in both theory and algorithms). The improvement demonstrated in the results is not sufficient to convince me that this increased complexity is worthy in practice. Also, the paper did not provide an intuitive explanation (or justification) as to why the proposed framework would actually work better than HDP in practice.