Mixed counting models that use the negative binomial distribution as the prior can well model over-dispersed and hierarchically dependent random variables; thus they have attracted much attention in mining dispersed document topics. However, the existing parameter inference method like Monte Carlo sampling is quite time-consuming. In this paper, we propose two efficient neural mixed counting models, i.e., the Negative Binomial-Neural Topic Model (NB-NTM) and the Gamma Negative Binomial-Neural Topic Model (GNB-NTM) for dispersed topic discovery. Neural variational inference algorithms are developed to infer model parameters by using the reparameterization of Gamma distribution and the Gaussian approximation of Poisson distribution. Experiments on real-world datasets indicate that our models outperform state-of-the-art baseline models in terms of perplexity and topic coherence. The results also validate that both NB-NTM and GNB-NTM can produce explainable intermediate variables by generating dispersed proportions of document topics. Mixture modeling is an essential topic in statistics and machine learning areas, owing to generating the random probability measure of data samples belonging to multiple clusters. In unsupervised learning tasks such as topic discovery, mixture modeling has gained increasing attention from researchers (Wang et al., 2011; Zhou and Carin, 2012, 2015; Zhou, 2018; Zhao et al., 2019). Specifically, mixture modeling over document words devotes to assign these words to different topics via random probability measures. Hierarchical Dirichlet  The first two authors contributed equally to this work which was finished when Jiemin Wu was an undergraduate student of his final year. Process (HDP) (Teh et al., 2004) is one of the representative methods in mixture modeling, which can characterize the two-level dependency of random probability measures. Although we can use Monte Carlo sampling or variational inference to estimate the parameters in HDP, it requires the help of indirect construction of random variables such as the Chinese Restaurant Franchise (Teh et al., 2004) or the Stick-Breaking construction (Wang et al., 2011) due to the lack of conjugation between the two-tier Dirichlet processes. This makes the inference of HDP mostly complicated (Zhou et al., 2016). The mixed counting models represented by the Negative Binomial (NB) process (Titsias, 2007) and the Gamma Negative Binomial (GNB) process (Zhou and Carin, 2012) have solved this problem to a certain extent, in which, the normalized GNB process has been proven to be equivalent to HDP (Zhou and Carin, 2012). Because both NB and GNB processes satisfy the properties of completely random measures (Charles Kingman, 1967), the generative process of random probability measures among various mixed components is independent and becomes straightforward. Moreover, they naturally introduce non-negative constraints and have been proven as able to model over-dispersed data. In the case of mining latent topics of documents, the over-dispersed property indicates that the variance is larger than the mean for document-topic distributions. When compared to the NB process, the GNB process has an extra feature of describing more flexible stochastic phenomena with hierarchical dependencies. Despite the above advantages, with the increase of data size and observable information, the aforementioned parameter inference method like Monte Carlo sampling or variational inference has gradually become an important factor limiting the usage scenarios of mixed counting models (Miao et al., 2016). The reason is that Monte Carlo sampling has a high computational cost, and variational inference becomes intractable when applied to models with complex variable dependencies (Acharya et al., 2015). Neural variational inference (NVI) is a flexible and fast parameter inference framework based on neural networks (Mnih and Gregor, 2014). It can be regarded as a generalization of variational auto-encoder applicable to natural language processing tasks. Based on NVI, several neural topic models had been proposed and achieved encouraging performance in document modeling (Miao et al., 2016; Srivastava and Sutton, 2017; Miao et al., 2017). These models used the neural network to learn the distribution relationship between input documents and latent topics due to its excellent function fitting ability and scalability. Particularly, the neural network parameters can be trained by back-propagation through the reparameterization of a continuous distribution (Naesseth et al., 2017) or using variance reduction techniques for a discrete distribution (Mnih and Gregor, 2014). However, the hidden variables in the above neural topic models lack good interpretability, and it is also impossible to model over-dispersed and hierarchically dependent document sets for these methods. In this paper, we propose two novel neural mixed counting models dubbed the Negative Binomial-Neural Topic Model (NB-NTM) and the Gamma Negative Binomial-Neural Topic Model (GNB-NTM) based on NB and GNB processes, respectively. The general motivation is to combine the advantages of NVI and mixed counting models. On the one hand, NVI-based models are fast and easy to estimate but hard to interpret. On the other hand, document modeling via mixed counting models is easy to interpret but difficult to infer. In our NB-NTM and GNB-NTM, we develop NVI algorithms to infer parameters by using the reparameterization of Gamma distribution and the Gaussian approximation of Poisson distribution. Extensive experiments on real-world datasets validate the effectiveness of our proposed models in perplexity, topic coherence, and dispersed topic learning. Furthermore, the proposed models can describe the hierarchical dependence of random probability measures and introduce non-negative constraints, which renders the intermediate variables generated by our methods to have good interpretability. The remainder of this article is organized as follows. In Section 2, we summarize the related studies on topic discovery. In Section 3, we introduce the definitions and properties of background methods. The proposed models are described in Section 4, the experimental evaluations are shown in Section 5, and we draw the conclusions in Section 6. Topic discovery aims to use the statistical information of word occurrences to obtain the abstract semantic structure embedded in a document set. From Bayesian methods represented by latent semantic analysis (LSA) (Deerwester et al., 1990), probabilistic latent semantic analysis (PLSA) (Hof-mann, 1999), latent Dirichlet allocation (LDA) (Blei et al., 2003), and Hierarchical Dirichlet Process (HDP) (Teh et al., 2004), topic discovery had been widely researched in natural language processing and applied to many scenarios. For instance, the above models were extended to capture topic relevance (Blei and Lafferty, 2005) and topic evolution over time (Wang and McCallum, 2006; Blei and Lafferty, 2006). Algorithms for short text (Yan et al., 2013), tagged data (Ramage et al., 2009), and stream data (Yao et al., 2009) were also proposed. Considering the importance of prior distributions in LDA-based models, some research efforts tried to use beta and Gaussian distributions instead of the Dirichlet distribution as the prior of probabilistic graphical models (Thibaux and Jordan, 2007; Das et al., 2015). Although the Bayesian method is a natural way to represent the latent structure of a document set in topic discovery, as the structure of such a model becomes deeper and more complex, pure Bayesian inference becomes intractable due to the high dimensional integrals required (Miao et al., 2016). To address this issue, Cheng and Liu (2014) proposed a parallel Monte Carlo sampling method for HDP based on multi-threading. Unfortunately, it needs to traverse every word of all topics (i.e., threads) in the whole corpus when updating the topic-word distribution, rendering a large time cost for thread communication. With the development of deep learning, especially the introduction of NVI, there is a new direction to discover topics based on neural networks. For example, Miao et al. (2016) assumed that word distributions in each document could be represented by hidden variables sampled from multiple Gaussian distributions, and they used the variational lower bound as the objective function of their model named NVDM. Srivastava and Sutton (2017) employed the logical Gaussian distribution to approximate the Dirichlet distribution, which improved the variational auto-encoder and LDA simultaneously. Miao et al. (2017) proposed a method named GSM to model the document-topic distribution explicitly. In their study, the topic-word distribution was introduced into the decoder. Besides the above NVI-based methods, Nalisnick and Smyth (2017) developed a stick-breaking variational auto-encoder for image generation. Nan et al. (2019) proposed a model named W-LDA in the Wasserstein auto-encoder framework. They employed the Maximum Mean Discrepancy (MMD) in W-LDA to match the proposed distribution and the prior distribution. However, the accuracy of MMD relied heavily on the number of samples for each distribution, and the kernel function in MMD had a significant in-fluence on the performance. By leveraging word embeddings, Gupta et al. (2019) proposed a neural autoregressive topic model dubbed iDocNADE to enrich the context of short text. Experiments indicate that iDocNADE outperformed state-of-the-art generative topic models. The recent relevant work to ours is the method proposed in (Zhao et al., 2019), which regarded the NB distribution as the prior in modeling the over-dispersed discrete data. However, the parameters of this method were still derived from the latent variables that obey the Gaussian distribution. Thus, these latent variables do not satisfy the non-negative constraint and lack good interpretability. Furthermore, the above method did not model topics explicitly, making it hard to generate document-topic and topic-word distributions. Let X  NBP( G 0 , p ) denote a NB process defined on the product space R +   , where G 0 is a finite continuous basic measure on a completely separable measure space  , and p is a scale parameter. For each Borel set A   , we use X ( A ) to denote a count random variable describing the number of observations that reside within A . Then, X ( A ) obeys the NB distribution NB( G 0 ( A ) , p ) . Given the k th component  k and its weight r k on  , if G 0 is expressed as G 0 = (cid:80)  k =1 r k   k , where  is the Dirac delta function, then X  NBP( G 0 , p ) can be expressed by X = (cid:80)  k =1 n k   k , where n k  NB( r k , p ) . The NB distribution m  NB( r, p ) has a probability density function f M ( m ) = ( r + m ) m !( r ) (1  p ) r p m , where (  ) denotes the gamma function. For the above probability density function, the mean and the variance are  = r/ (1  p ) and  2 = rp/ (1  p ) 2 =  + r  1  2 , respectively. Because the mean is smaller than the variance, i.e., the variance-to-mean ratio is greater than 1, NB distributions have shown great advantages in overdispersed data modeling (Zhou and Carin, 2012). Moreover, since the NB distribution m  NB( r, p ) can be extended to a Gamma distribution and a Poisson distribution, i.e., m  Poisson(  ) and   Gamma( r, p/ (1  p ) , the NB process mentioned earlier can be extended to a Gamma-Poisson process (Zhou and Carin, 2015) as follows: X  PP () , and   GaP ( G 0 , (1  p ) /p ) , where PP(  ) and GaP(  ) denote the Poisson process and the Gamma process, respectively. The random probability measure corresponding to each mixed component in the NB process can be directly sampled from the NB distribution without resorting to the Chinese Restaurant Franchise, the Stick-Breaking, or other construction methods, because each random measure is independent of the others, i.e., the NB process is completely random. In the NB process, each Poisson process shares the same Gamma process prior with a fixed mean. Based on the NB process, the GNB process assigns another Gamma process as a prior to its mean, making it easier to model over-dispersed data (Zhou et al., 2016). Particularly, the generative process of random variables for the GNB process is as follows: G  GaP ( G 0 ,  ) ,  j  GaP ( G, (1  p j ) /p j ) , and X j  PP ( j ) , where j is the subset index,  is the scale parameter of the first-level Gamma process, and the basic measure G 0 in the NB process is replaced by another random measure G . It has been shown that HDP is a normalized form of the GNB process in (Zhou and Carin, 2012). However, unlike HDP, the GNB process explicitly introduces the parameter p j to control the dispersion degree of instantaneous measurement, making the latter model more flexible. NVI is often used as an efficient parameter inference framework for complex and deep-seated structural models. Inspired by the variational auto-encoder, NVI assumes that the observed data d is subject to a certain probability distribution determined by a hidden variable h . In contrast to variational auto-encoders on handling the case of continuous latent variables (Kingma and Welling, 2014), NVI can deal with both discrete and continuous latent variables. Specifically, a neural network is used to infer the proposed distribution q ( h | d ) . As stated in (Miao et al., 2017), Monte Carlo estimates of the gradient must be employed for models with discrete latent variables. In the case of q ( h | d ) being continuous, the hidden variable h is firstly obtained by sampling from q ( h | d ) through the corresponding reparameterization approach. Then, the likelihood p ( d | h ) is used to reproduce the observed data from hidden variables, and the objective is to minimize the Kullback-Leibler (KL) divergence of the proposed distribution and the actual posterior distribution. Finally, the variational lower bound is obtained by L = E q ( h | d ) log p ( d | h )  DKL [ q ( h | d ) (cid:107) p ( h )] , where the first term is the expectation of the log-likelihood, and the second one is the KL divergence between the inferred distribution and a predefined prior. To sum up, NVI first uses a neural network to infer the proposed distribution q ( h | d ) , and then maximizes the variational lower bound by backpropagation to fit the actual posterior distribution p ( h | d ) . Such a framework learns the distribution of input data well, enabling it to combine with the traditional probability graphical models (e.g., LDA) and infer model parameters quickly (Srivas-tava and Sutton, 2017). However, how to effectively integrate the distributed dependencies in mixed counting models into the framework of variational inference is still quite a challenging problem. In this section, we respectively detail our NB-NTM and GNB-NTM for dispersed topic discovery. With a NB process prior, we propose the NB-NTM to model the counting of document words. Furthermore, a novel NVI framework is developed for parameter inference. Let D = { d 1 , ..., d | D | } be the input with | D | documents and each document d  RV be a bag-of-words representation, where V is the vocabulary size. Since it is impossible to draw all the countably infinite atoms of a Gamma process, we first employ the finite truncation strategy, in which, a number of topics K (i.e., the truncated level) is set manually (Nalisnick and Smyth, 2017; Zhou, 2018). Note that although K is fixed, if K is set to be large enough, not necessarily all topics would be used and hence a truncated model still preserves its nonparametric ability; whereas if K is set to be small, asymmetric priors on the topic weights are also maintained (Zhou, 2018). Then we can express the generative process of NB-NTM for document d as follows: r = f 1 ( d ) , p = f 2 ( d ) , (1)   Gamma ( r , p / ( 1  p )) , (2) n  Poisson (  ) , (3) where f 1 (  ) and f 2 (  ) are two multilayer percep-trons (MLPs) applying to generate the variational parameters r and p . Specifically, r is the component weight of G , i.e., the topic measure at the corpus level, and G = (cid:80) Kk =1 r k   k . represents the weights of topics at the document level, which can be used to estimate the topic measure on d by  = (cid:80) K k =1  k   k . In the above,  k denotes the k th component of  . Finally, n is the component weight of  that represents a Poisson process at the word level, and  = (cid:80) Kk =1 n k   k . The framework of NB-NTM is shown in Figure 1, and the parameter inference process is described as follows. For the logarithmic likelihood of each document d , we can derive the variational lower bound by L =  DKL ( q (  | d ) || p (  )) + E q (  | d ) (cid:104)(cid:80) N d i =1 log p (  i |  ) (cid:105) . In the above, q (  | d ) is the encoder's inference of posterior probability, i.e., Gamma( r , p ) ,  i  RV is the one-hot representation of the word at the i th position, N d is the number of words in document d , and p (  ) is the Gamma prior for  , i.e., Gamma(  , c ) . The KL divergence between q (  | d ) and p (  ) , i.e., Gamma( r , p ) and Gamma(  , c ) , is calculated by following (Mathiassen et al., 2002): DKL ( q (  | d ) || p (  )) = (cid:80) Kk =1 [( r k  1)( r k )  log p k  r k  log ( r k )  (   1)(( r k )+log p k )+ log (  )+  log c + r k p k c ] , where (  ) is the Digamma function. The conditional probability over each word p (  i |  ) is modeled by softmax function, as follows: p (  i |  ) = exp {  ( n TR  i + b i ) } (cid:80) Vj =1 exp {  ( n TR  j + b j ) } , where R and b denote the weight matrix and the bias term, respectively. We present the parameter inference process of NB-NTM in Algorithm 1, in which, the variational lower bound L is used to calculate gradients and model parameters are updated by Adam (Kingma and Ba, 2015). Based on the NB-NTM, we further propose the GNB-NTM by assigning another Gamma process as a prior to the NB process. As shown in Figure 2, the generative process of GNB-NTM for document d is given below:  = f 1 ( d ) ,  = f 2 ( d ) , (4) r  Gamma (  ,  ) , (5)   Gamma ( r , p/ (1  p )) , (6) p = f 3 ( d ) , n  Poisson (  ) . (7) In the above,  and  are the parameters of the first-level Gamma process, and p is the scale parameter of the second-level Gamma process. The differences between GNB-NTM and NB-NTM are three-fold. Firstly, another Gamma process G 0 is introduced over the existing Gamma process G as a prior of its shape parameter, so as to characterize the multi-level dependencies of random variables. In particular, G 0 = (cid:80) Kk =1  k   k . Secondly, a scale parameter p is introduced for each document to describe the dispersion degree of all words in the document. Thirdly, the GNB-NTM employs n + r as the input of the decoder by following the production rule of the observed variable in (Zhou and Carin, 2012). Using n + r as the input also helps to incorporate the global topic information into the decoder's inference of posterior probability q ( r | d ) . Thus, the conditional probability over each word p (  i | r ) is modeled as follows: p (  i | r ) = exp {  ( ( n + r ) TR  i + b i ) } (cid:80) Vj =1 exp {  ( ( n + r ) TR  j + b j ) } . Similar to NB-NTM, the variational lower bound is derived by: L = E q ( r | d ) (cid:104)(cid:80) N d i =1 log p (  i | r ) (cid:105)  DKL ( q ( r | d ) || p ( r )) , where p ( r ) is the Gamma prior for r , i.e., Gamma(  , c ) . The parameter inference for GNB-NTM is presented in Algorithm 2. We use the variational lower bound to calculate gradients and apply Adam to update parameters of GNB-NTM, which are the same as NB-NTM. up-Algorithm 2: Parameter Inference for GNB-NTM date model parameters through back-propagation. Here, we describe the reparameterization approach for smoothing gradients. For the Gamma distribution x  Gamma( ,  ) with  > 1 , the reparameterization can be obtained by the reject-sampling method (Naesseth et al., 2017), i.e., x = 1  (cid:0)   13 (cid:1) (cid:16) 1 +   9   3 (cid:17) 3 , (cid:15)  N (0 , 1) . Besides, the shape augmentation method (Naesseth et al., 2017) is applied to convert   1 to  > 1 to increase the accept rate of each rejection sampler. For the Poisson distribution which is discrete, we use the Gaussian distribution as an approximation (Rezende et al., 2014; Kingma and Welling, 2014). Based on the central limit theorem, N (  = ,  2 =  ) can approximate Poisson(  ) . Thus, we sample from the Poisson distribution directly to avoid the issue of discretization and use the Gaussian distribution as an approximation when calculating the Poisson distribution's gradient. Particularly, the reparameterization of a Gaussian distribution x  N ( ,  2 ) is x =  + (cid:15)  , (cid:15)  N (0 , 1) . We employ the following three datasets to evaluate the effectiveness of our models: Reuters 1 , 20News, and MXM song lyrics (Miao et al., 2017). The Reuters dataset contains 7,758 training documents and 3,005 testing documents. The 20News corpus consists of 18,773 news articles under 20 categories. These news articles are divided into 11,268 training documents and 7,505 testing documents. The 20 categories include sports, electronics, automotive, and so forth, and the number of documents under each category is almost the same. MXM is the offi-cial lyrics collection of the Million Song Dataset, which contains 210,519 training documents and 27,143 testing documents, respectively. By following (Miao et al., 2017), we use the originally provided vocabulary with 5,000 words for MXM, while for Reuters and 20News, we use stemming, stop words filtering, and the 2,000 most frequently occurred words as vocabularies. The statistics of these datasets are presented in Table 1. The following models are adopted as baselines: HDP (Teh et al., 2004), NVDM (Miao et al., 2016), NVLDA and ProdLDA (Srivastava and Sutton, 2017), GSM (Miao et al., 2017), and iDocNADE (Gupta et al., 2019). Among these baselines, HDP is a classical mixture modeling method followed 1 https://www.nltk.org/book/ch02.html the equivalence with the normalized GNB process (Zhou and Carin, 2012). In HDP, the model parameters are estimated by Monte Carlo sampling. NVDM, NVLDA, ProdLDA, and GSM are all neural topic models based on NVI. Considering that word embeddings have shown to capture both the semantic and syntactic relatedness in words and demonstrated impressive performance in natural language processing tasks, we also present the result of a neural autoregressive topic model that leverages word embeddings (i.e., iDocNADE). Particularly, the publicly available codes of HDP 2 , NVDM 3 , NVLDA and ProdLDA 4 , and iDocNADE 5 are directly used. As an extended model of NVDM, the baseline of GSM is implemented by us based on the code of NVDM. To ensure fair comparisons on various NVI-based methods, unless explicitly specified, we set the number of topics to 50, the hidden dimension of MLP to 256, and use one sample for NVI by following (Miao et al., 2017). For the batch size, the learning rate, and other model parameters, grid search is carried out on the training set to determine their optimal values and achieve the held-out performance. To evaluate the quality of topics generated by different models, we use perplexity and topic coherence as evaluation criteria. The perplexity of each model on a testing set (cid:101) D is: perplexity ( (cid:101) D ) = exp (cid:16)  1 | (cid:101) D | (cid:80) (cid:101) d 1 N (cid:101) d log p ( (cid:101) d ) (cid:17) , where log p ( (cid:101) d ) represents the log-likelihood of the model on document (cid:101) d , and N (cid:101) d is the number of words in (cid:101) d . The lower the perplexity is, the more likely for a model to generate (cid:101) D . Therefore, if a model obtains a lower perplexity than others in the testing set, it can be considered as the better one. For all NVI-based topic models, the variational lower bound, which is proven to be the upper bound of perplexity (Mnih and Gregor, 2014), is used to calculate the perplexity by following (Miao et al., 2016, 2017). When calculating the topic coherence, we use the normalised pointwise mutual information (NPMI) which measures the relationship between word w i and other T  1 top words (Lau et al., 2014) as follows: NPMI ( w i ) = (cid:80) T  1 j =1 [log P ( w i ,w j ) P ( w i ) P ( w j ) /  log P ( w i , w j )] . The higher the value of topic coherence, the more explainable the topic is. 2 https://github.com/soberqian/ TopicModel4J 3 https://github.com/ysmiao/nvdm 4 https://github.com/akashgit/ autoencoding_vi_for_topic_models 5 https://github.com/pgcool/iDocNADEe 5.3 Performance Comparison Table 2 shows the perplexity and topic coherence of different models on the test datasets. We can observe that NB-NTM outperforms most baselines, and GNB-NTM performs the best in all cases. The results validate that the NB distribution can model over-dispersed documents well. Furthermore, the latent semantics of these corpora may be hierarchically dependent. In other words, the topics at the corpus level and those of each document are not independent but correlated with one another. In terms of the model efficiency, neural topic models can be trained much faster than HDP on a large corpus by GPU acceleration. Take the large-scaled MXM dataset as an example, the training time of both NB-NTM and GNB-NTM is around one hour using a GeForce GTX 960 GPU, while HDP needs more than three hours to converge using an AMD R5 3600 CPU. Under the same environment, the training time of all NVI-based topic models is close. In general, NVLDA, prodLDA, and NVDM run slightly faster than NB-NTM because the Gaussian reparameterization approach is simpler than the Gamma one. GSM and GNB-NTM are slightly slower than others because the former introduces more parameters to model the topic-word distribution, while the latter introduces more sampling operations. As an illustration, we also qualitatively evaluate the semantic information learned by different models on the 20News training set. The baselines of HDP, NVLDA, and prodLDA, which achieve competitive topic coherence scores, are selected for comparison. Table 3 presents 5 of the most representative topics with the corresponding top 10 words, from which we can observe that although all these models can identify the chosen topics reasonably, our NB-NTM and GNB-NTM perform better than the other baselines in most cases. In this part, we test the impact of the number of topics on the performance of our models. Figure 3 shows the convergence process of NB-NTM and GNB-NTM on the 20News training set with K = 20, 50, 100, 200 in terms of the perplexity. We can observe that as K increases, the perplexity values of both models decrease under each epoch. This is because the NVI framework is essentially an encoder-decoder, and the increase of the topic number enables the models to encode and reconstruct documents better. We also notice that with the continuous growth of K , the improvement of perplexity is getting lower. Table 4 presents the results of our models on the 20News testing set under the above conditions, in which a similar trend can be observed as aforementioned. topics are dispersed, and thus, the intermediate variables can be more explainable. To validate the effectiveness of our models on learning dispersed topics, we first count the total number of words under each manually labeled category (i.e., topic) as the topic-word number distribution shown in Figure 4 (a). Then we run our NB-NTM and GNB-NTM on the entire 20News testing set to get the corresponding values of r . After normalization, the proportion of different topics obtained by NB-NTM and GNB-NTM at the corpus level is presented in Figure 4 (b) and Figure 4 (c), respectively. For the convenience of the result presentation, we set the number of topics to 20 for both models. Note that the 20 topics do not need to correspond to the 20 categories, because we here focus on testing whether the topic proportions generated by our two models are in accordance with their model structures/characteristics. From these results, we can observe that the proportion of topics obtained by NB-NTM is close to the topic-word number distribution. On the other hand, GNB-NTM obtains more dispersed proportions of topics than NB-NTM. These results suggest that GNB-NTM tends to allocate less but more important topics to the corpus, i.e., the topics generated by GNB-NTM are more discriminative. Since the document-topic distribution is not directly modeled and the Gaussian distribution samples are not non-negative, the previous neural methods except GSM cannot obtain explainable intermediate variables. For the baseline of GSM, Miao et al. (2017) had demonstrated that the topics with higher probabilities were evenly distributed on the same 20News dataset, which indicates that our models outperform GSM on learning dispersed document topics. We also study the dispersion of intermediate variables (i.e., topics) at the document level. By randomly select a document as an example, we get the normalized document topic weight  from NB-NTM and GNB-NTM to explore whether the topic distributions of the document generated by our models are reasonable. As shown in Figure 5, the document is about a standard computer, and the most related topics with large topic distributions are all related to computers, which validates the practical meaning of intermediate variables of both NB-NTM and GNB-NTM at the document level. From the keywords in the most related topics, we further observe that GNB-NTM can identify more computer-related words than NB-NTM. When compared to the whole semantic space as shown in Figure 4, both NB-NTM and GNB-NTM generate more dispersed proportions of topics at the document level. This phenomenon is consistent with the over-dispersed feature (i.e., the variance is larger than the mean) of documents. In this paper, we present two neural mixed counting models named NB-NTM and GNB-NTM. Different from the current time consuming Bayesian methods, our models apply to large-scale datasets through the efficient back-propagation algorithm and GPU acceleration. When compared to the existing neural topic models, both NB-NTM and GNB-NTM can well model the random variables with I have a Standard Computer 486DX2/66mhz EISA Tower with 16MB RAM, a Quantum 240MB Hard Drive, 1.2 and 1.44 MB floppies and a Colorado 250MB tape drive. over-dispersed and hierarchically dependent characteristics. Extensive experiments on real-world datasets validate the effectiveness of our models in terms of perplexity, topic coherence, and producing explainable intermediate variables by generating dispersed proportions of document topics. The results also indicate that NB distribution families can characterize text data aptly, which is essentially due to their conformity with the over-dispersed and sparse properties of natural language. We are grateful to the reviewers for their constructive comments and suggestions on this study. This work has been supported by the National Natural Science Foundation of China (61972426), Guang-dong Basic and Applied Basic Research Foundation (2020A1515010536), HKIBS Research Seed Fund 2019/20 (190-009), the Research Seed Fund (102367), and LEO Dr David P. Chan Institute of Data Science of Lingnan University, Hong Kong. This work has also been supported by a grant from the Research Grants Council of the Hong Kong Special Administrative Region, China (UGC/FDS16/E01/19), Hong Kong Research Grants Council through a General Research Fund (project no. PolyU 1121417), and by the Hong Kong Polytechnic University through a start-up fund (project no. 980V).
abstain abstain objective abstain result abstain abstain abstain abstain objective abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain objective abstain abstain abstain objective objective objective method objective abstain objective other other other other other other other other other other other other other other other other other other other other other other other other method other other other other other other other other other other other other other other other other other other other other other other other other abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain other abstain method abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain method abstain abstain method abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other abstain abstain method abstain method method abstain method other abstain abstain abstain method other method abstain abstain abstain abstain abstain abstain abstain abstain method abstain method abstain abstain result abstain abstain result abstain abstain abstain abstain abstain abstain abstain result abstain result abstain abstain abstain abstain abstain abstain abstain result abstain abstain method method abstain abstain method abstain other other other