Geometric Dirichlet Means algorithm for topic inference
We propose a geometric algorithm for topic learning and inference that is built on the convex geometry of topics arising from the Latent Dirichlet Allocation (LDA) model and its nonparametric extensions. To this end we study the optimization of a geometric loss function, which is a surrogate to the LDA's likelihood. Our method involves a fast optimization based weighted clustering procedure augmented with geometric corrections, which overcomes the computational and statistical inefficiencies encountered by other techniques based on Gibbs sampling and variational inference, while achieving the accuracy comparable to that of a Gibbs sampler. The topic estimates produced by our method are shown to be statistically consistent under some conditions. The algorithm is evaluated with extensive experiments on simulated and real data.
The paper proposes a geometric view to topic modeling and develops an inference algorithm based on it. This is an interesting point of view and an inference method. The authors provide theoretical guarantees of their method proving a theorem about convergence of their topic estimates to the true ones.  Topic modeling is considered as a geometric problem, where there is a convex hull of topics and documents, represented as vectors of words probabilities, are points inside this polytope. A topics proportions vector for a document can be then represented as barycentric coordinates of this document w.r.t. this polytope.  The learning procedure consists on two steps: finding centroids of K-means clusters, where K is the number of topics, first, and then extending these centroids to the vertices of the polytope along rays that connect centroids and the barycenter of the polytope. Inference of topic proportions for documents is based on projection of documents points, represented as vectors of normalised words frequencies, to this found polytope.    The algorithm is shown to be fast and accurate in terms of hold-out perplexity and minimum-matching Euclidean distance between true and estimated topics. Moreover, it seems to be possible to extend this approach for more complex topic models. The authors present the extension for the case where the number of topics is unknown and discuss possibility of extensions to correlated and dynamic topic models.  
The paper reads well, despite the fact that a lot of information is presented in a dense form. The authors refer everywhere to related work.  Although the paper is presented well, there are ways to improve it. There are two most crucial aspects which are currently not very well covered in the paper, everything else is not critical, they are some suggestions for improvement.  These two crucial moments are descriptions of tuning of extension parameters in the proposed tGDM algorithm and the first step in the proposed extension of the algorithm called as the nGDM algorithm. tGDM algorithm: The authors claim that extension of the centroids of the k-means clusters towards vertices of the polytope by the fixed length (as it is in the basic proposed algorithm called as the GDM algorithm) is not always a good choice. They mention that "careful tuning of the extension parameters based on optimizing the geometric objective over a small range of m_k helps to improve the performance considerably". And they call this tGDM algorithm. It is not clear for me how exactly this tuning happens. It would be good to add some more details about it. nGDM algorithm: The authors present an extension of their approach to the case when the number of topics is unknown. They say that the first step of their algorithm "involves solving a penalized and weighted k-means clustering". It is unclear how it can be done. If it refers to some known method it should be cited, otherwise it would be also better to add some more details about this step.  Other suggestions:  1. As the material is dense it would be good to summarise the whole algorithm with specific implementations of the steps    2. It looks strange that in sections 3.1 and 3.2 thetas are treated as parameters and then in section 3.3 they transform to be random variables. It is better, at least, to mention about this transformation  3. To continue the previous remark the likelihood in section 3.1 is for the PLSA model, rather than for the LDA one (because of treatment thetas as parameters rather than random variables). It would be better to state this and to clarify about LDA likelihood as the claim about connection between the geometric objective function and LDA likelihood appears several times in the paper 4. The claim "V is much higher and the cluster centroids are often on the boundary of the vocabulary simplex, therefore we have to threshold the betas at 0" (lines 146-148) requires some explanations 5. It would be better to add some citation to the claim in the sentence in lines 67-68 6. Regarding experiments. In the paper it is claimed that the results of the variational EM algorithm are always worse than the others but according to the results in figure 2 in the supplementary materials (where there are results for individual runs of the algorithms rather than averaged ones presented in the paper) for only 3 out of 20 different runs of the variational EM algorithm its results are worse than the results of Gibbs sampling and these results of both algorithms coincide in other 17 runs. It looks not so bad. Maybe averaging procedure of the results of the individual runs of the algorithms is not appropriate. At least it is worth to mention about this behaviour. It is mentioned in the paper that variational EM is unstable. Even if it regards to this behaviour it is better to explain it in more details 7. Information about Gibbs sampling and the variational EM algorithm settings in the experiments on simulated data would be good 8. In Figure 2 there are results referred as 0.1(log(N_m)/N_m)^0.5. There is no information about this in the text and it is unclear what it is about 9. The claim that tuning extension parameters could be computationally expensive on big datasets (lines 248-249) looks strange as there was no possibility to check it 10. It is not very clear why there are no results for Gibbs sampling and the variational EM algorithm in figures 2d and 3d 11. Regarding results presented in the supplementary materials. It is not clear for me what is weighted GDM. As I understand GDM presented in the paper is already weighted 12. It is not clear what the reference "projection with VEM" in figure 3 in the supplementary materials refers to 13. It is not clear why there is no presented topics inferred by the variational EM algorithm in the supplementary materials  Minor things: 1. Please check style requirements, e.g. a paper title should not be capitalised 2. Acronyms GDM, VEM and MLE are undefined 3. It would be better to define the notation "Dir_V" 4. It would be better to distinguish scalars and vectors in notations 5. There is "for m = 1, ..., M" in line 58 but there is not "for k = 1, ..., K" in line 57 6. "z", "d" and "n_m" are never defined 7. B := ... in line 77 rather than B = ... 8. "lets" in line 80 seems to be "let" 9. Missing articles before "document" in lines 83 and 84 10. Please enumerate all equations 11. In equation between 107 and 108 V is used as a set, while it is introduced as a scalar 12. Missing article before "topic polytope" in line 133 13. "Generative" rather than "generating" in line 142 14. It seems that there is only one tetrahedron in the simple simulation experiment (lines 140-146), then "tetrahedra" in line 145 should be "tetrahedron" 15. It is better to add subcaptures a, b, c and d in figures 2 - 4 as they are referred. 16. Missing closing bracket in line 243 17. It is better to add citation to the used NIPS corpora 18. Please checked references: there are lower cases for proper names, e.g. "Hierarchical dirichlet processes"; there are also caps for articles, e.g. in "Proceedings of The 31st International ..."; papers from NIPS are referred as "NIPS, 2012" in line 283 and "Advances in neural information processing systems, 2006a" in line 286; authors' given names appear both in full forms and as initials; a place of a conference is stated only for one reference in line 307