Multilingual Anchoring: Interactive Topic Modeling and Alignment Across Languages
Multilingual topic models can reveal patterns in cross-lingual document collections. However, existing models lack speed and interactivity, which prevents adoption in everyday corpora exploration or quick moving situations (e.g., natural disasters, political instability). First, we propose a multilingual anchoring algorithm that builds an anchor-based topic model for documents in different languages. Then, we incorporate interactivity to develop MTAnchor (Multilingual Topic Anchors), a system that allows users to refine the topic model. We test our algorithms on labeled English, Chinese, and Sinhalese documents. Within minutes, our methods can produce interpretable topics that are useful for specific classification tasks.
The paper proposes an anchor-based approach for learning a topic model in multilingual settings. The model first is built automatically and then can be refined by interaction with a user. The paper addresses a problem of interest of the previous NIPS submissions.  Quality: The paper is technically sound. I like that for bilingual topic modeling it is only require to have a dictionary between two languages but documents should not be aligned for training a topic model. The interactive part could be interesting for applications but is not quite impressive from the methodological point of view since building a model interacting with a user is based on existing methods where the proposed approach for multilingual anchoring is used only for initial topic learning.   The main missing piece in the paper from the technical point of view is absence of description of main optimisation algorithm. How exactly the authors solve their equation (5) in an efficient way.   Could the authors please explain how do they choose the number of anchoring words if they allow multiword anchoring? Or are multi-words only used in the interactive settings?  It also would be beneficial to see more discussion about conceptual comparison of generative and anchor-based approaches for topic modeling.   Other suggestions/questions: 1. It would be an interesting comparison if MCTA results are provided both for limited and unlimited time scenarios. Now it is unclear whether MCTA gives poor results because it has not converged or because it generally learns poorer topics. 2. Follow-up regarding comparison with MCTA, the authors discuss briefly computational time of the methods without specifying what programming languages and hardware are used for implementations 3. Does the method have to be bilingual? It seems that it can be extended for more languages. If it is so, it could be mentioned in the paper, if the method is limited to be bilingual, it should be clearly stated. 4. I am not sure about user study experiment setup because the users were asked to essentially optimise classification performance of the system rather than interpretability of the topics directly. Interpretability is something that they probably based their changes on but this was not their primal goal. And it could be a case that a user made a change to improve interpretability of topics but this harmed classification performance since classification performance is not a direct measure of interpretability of topics and may not be related to it. As follow-up, the arguments in lines 255-257 about why one should care about interpretable features it is again talked about as classification is the end goal of topic modeling which is not correct since then one should most probably use some deep learning approaches rather than topic modeling 5. Figure 3b. What is the baseline here? Is it MCTA? Or is it multilingual anchoring?  Clarity: The paper is mostly well written. I like a lot of examples used, this is really engage reading. Below are some suggestions for improvement: 1. Figure 2 is unreadable. The plots should be enlarged 2. Figure 3b. It is unclear why the plot type has changed from the one used in Figure 3a. There are lines in Figure 3a, but scatter points in Figure 3b. 3. At the beginning of Section 4.1 it is better to remind a reader that “multilingual anchoring” is the proposed method without human interaction and “MTAnchor” is the proposed method with human interaction 4. Table 1 is missing an actual caption what this table contains. The current caption discusses the results 5. Lines 234-236 specifying the evaluation procedure should be before user study experiment since it uses the same procedure 6. It is better to specify that NPMI is 7. Section 4.1 misses description of results from Table 1. For example, the authors could move the current caption of Table 1 into the main body of the paper. 8. Line 245: “a user can reach at least a 0.40 increase” – since the median results among users are the same as multilingual anchoring should it be “a user can reach UP TO a 0.40 increase”? 9. Agruments in line 255-257  10. Section 6 on Related work should probably be moved before Section 3 as in Section 3 the authors refer to JointLDA and in Sections 4 and 5 the authors compare their method with MCTA, both of which are introduced in Section 6. 11. Last sentence of the paper does not seem to be of any use – a very vague statement.  Minor things: 1. S_k in equation (3) is not defined 2. The sentence “A computationally attractive …” in line 43 appears quite rapidly after general words about topic modeling. Maybe some more introductory words could help to direct the subject towards an anchor word approach 3. Typo: inperfection -> imperfection (line 113) 4. The footnote 3 is better to be stated in the main body of the paper 5. Typo: line 161 “not enough translations may be unavailable”. Should it be “available”? 6. Typo: line 192 “anchors are defined anchors so that they take full control …”. Should it be “users define anchors …”? 7. Typo: line 264 missing articles before “experiments” and “multilingual anchoring algorithm” 8. Typo: line 266 grammatically incorrect sentence. Should it be “approach to modeling cross-lingual topics THAT is both fast and accurate”? 9. Typo: line 267 “between topics if not training time is not long enough” – should first “not” be removed?  Originality: The paper addresses an interesting problem in machine learning. The multilingual anchoring algorithm appears to be fairly novel while interactive MTAnchor seems to be a good contribution only as an application rather than a novel method.  The related work is cited well for generative multilingual models and anchor-based methods, while alternative approaches for multilingual data processing are not discussed such as, e.g., Xiao and Guo, 2013 “A Novel Two-Step Method for Cross Language Representation Learning”  Significance: The paper extends the anchor-based approach for topic modeling for multilingual data and could be a nice contribution in that area. The empirical results on comparison to one generative topic model show superiority of the proposed method, although as mentioned above it is unclear whether it is a matter of restricted time for training or general inability of generative topic model. In any way, the proposed method would have an advantage of faster convergence.   As mentioned above, the interactive method proposed in the paper does not seem to provide a significant contribution.  UPDATE AFTER RESPONSE: I would like to thank the authors for their response and addressing many of the concerns raised by the reviewers. Based on the response I would recommend the authors to focus more in general on a topic modeling (exploratory) point of view and less on classification and use classification accuracy also as a way to show how good is a found topic model, as an indicator of interpretability of topics. Also I would strongly encourage the authors to provide the results of MCTA after its convergence (maybe together with unconverged results) and either claim that the proposed algorithm both is more efficient and finds better topics than MCTA or that it is only more efficient if the results are competitive. The current version with only unconverged MCTA results raises questions and it is not very convincing.