Though word embeddings and topics are complementary representations, several past works have only used pretrained word embeddings in (neural) topic modeling to address data sparsity in short-text or small collection of documents. This work presents a novel neural topic modeling framework using multi-view embedding spaces: (1) pretrained topic-embeddings, and (2) pretrained word-embeddings (context-insensitive from Glove and context-sensitive from BERT models) jointly from one or many sources to improve topic quality and better deal with polysemy. In doing so, we first build respective pools of pretrained topic (i.e., TopicPool ) and word embeddings (i.e., WordPool ). We then identify one or more relevant source domain(s) and transfer knowledge to guide meaningful learning in the sparse target domain. Within neural topic modeling, we quantify the quality of topics and document representations via generalization (perplexity), interpretability (topic coherence) and information retrieval (IR) using short-text, long-text, small and large document collections from news and medical domains. Introducing the multi-source multi-view embedding spaces, we have shown state-of-the-art neural topic modeling using 6 source (high-resource) and 5 target (low-resource) corpora. Probabilistic topic models, such as LDA (Blei et al., 2003), Replicated Softmax (RSM) (Salakhutdi-nov and Hinton, 2009) and Document Neural Autoregressive Distribution Estimator (DocNADE) (Larochelle and Lauly, 2012) are often used to extract topics from text collections and learn latent document representations to perform natural language processing tasks, such as information retrieval (IR). Though they have been shown to be powerful in modeling large text corpora, the topic * : equal contribution Topic Topic Words Topic Label Z 1 ( S 1 ) profit, growth, stocks, apple , fall, Trading consumer, buy, billion, shares Z 2 ( S 2 ) smartphone, ipad, apple , app, Product Line iphone, devices, phone, tablet Z 3 ( S 3 ) microsoft, mac, linux, ibm, ios, Operating System apple , xp, windows, software Z 4 ( T ) apple , talk, computers, shares, ? modeling (TM) still remains challenging especially in the sparse-data setting, especially for the cases where word co-occurrence data is insufficient, e.g., on short-text or a corpus of few documents. It leads to a poor quality of topics and representations. To address data sparsity issues, several works (Das et al., 2015; Nguyen et al., 2015; Gupta et al., 2019a, 2020) have introduced external knowledge in traditional topic models, e.g., incorporating word embeddings obtained from Glove (Pen-nington et al., 2014) or word2vec (Mikolov et al., 2013a). However, no prior work in topic modeling has employed multi-view embedding spaces: (1) pretrained topics , i.e., topical embeddings obtained from large document collections, and (2) pretrained contextualized word embeddings from large-scale language models like BERT (Devlin et al., 2019). Though topics and word embeddings are complementary in how they represent the meaning, they are distinctive in how they learn from word occurrences observed in text corpora. A topic model (Blei et al., 2003) is a statistical tool to infers topic distributions across a collection of documents and assigns a topic to each word occurrence, where the assignment is equally dependent on all other words appearing in the same document. Therefore, a topic has a global view representing semantic structures hidden in document collection. On other hand, word embeddings have primarily local view in the sense that they are learned based on local collocation pattern in a text corpus, where the representation of each word often depends on a local context window (Mikolov et al., 2013b) or is a function of its sentence(s) (Peters et al., 2018). Consequently, they are not aware of the thematic structures underlying the document collection. Additionally, recent studies (Peters et al., 2018; Devlin et al., 2019; Liu et al., 2019) have shown a reasonable success in several NLP applications by employing pretrained contextualized word embeddings, where the representation of a word is different in different contexts (i.e., context-sensitive). In context of this work, the representations due to global and local (context-sensitive or context-insensitive) views together are referred as multi-view embeddings. For example in Table 1, consider four topics ( Z 1 Z 4 ) of different domains where the topics ( Z 1 Z 3 ) are respectively obtained from three different high-resource source ( S 1 S 3 ) domains whereas Z 4 from a low-resource target domain T (especially in the data-sparsity settings). Observe that the topics about Trading ( Z 1 ), Product Line ( Z 2 ) and Operating System ( Z 3 ) are coherent and and represent meaningful semantics at document-level via lists of topic words. However in sparse-data settings, the topic Z 4 discovered is incoherent (noisy) and it is difficult to infer meaningful document semantics. Unlike the topics, word embeddings (context-insensitive) encode syntactic and semantic relatedness in fine-granularity and therefore, do not capture thematic structures. For instance, the top-5 nearest neighbors (NN) of apple (below) in word embedding (Mikolov et al., 2013b) space suggest that it refers to a fruit and do not express any topical information (e.g., Trading , Product Line or Health ) in the corpora. Similarly given the NN of the word fall , it is difficult to infer its association with document-level semantics, e.g., Trading as expressed by Z 1 in topic-embedding space. fall NN ==  falling, falls, drop, tumble, rise, plummet Therefore, topic and word embedding spaces encode complementary semantics. Different to context-insensitive word embeddings, the word apple is referring to an organization and contextualized by different topical semantics respectively in the three sources S 1 -S 3 . Thus, it arises the need for context-sensitive embeddings in topic modeling. beddings : To alleviate the data sparsity issues, it is the first work in unsupervised neural topic modeling (NTM) within transfer learning paradigm that employs multi-view embedding spaces via: (a) Global-view Transfer ( GVT ): Pretrained topic embeddings instead of using word embeddings exclusively, and (b) Multi-view Transfer ( MVT ): Pretrained topic and word embeddings (context-insensitive from Glove (Pennington et al., 2014) and context-sensitive from large-scale language models such as BERT (Devlin et al., 2019) jointly to address data sparsity and polysemy issues. Contribution (2) Multi-source Multi-view Neural Topic Modeling : A single source of prior knowledge is often insufficient due to incomplete and non-overlapping domain information required by a target domain. Therefore, there is a need to leverage multiple sources of prior knowledge, dealing with domain-shifts (Cao et al., 2010) among the target and sources. In doing so, we first learn word and topic representations on multiple source domains to build WordPool and TopicPool , respectively and then perform multi-view and multi-source transfer learning in neural topic modeling by jointly using the complementary representations. We evaluate the effectiveness of multi-source neural topic modeling in multi-view embedding spaces using 7 (5 low-resource and 2 high-resource) target and 5 (high-resource) source corpora from news and medical domains, consisting of short-text, long-text, small and large document collections. We have shown state-of-the-art results with significant gains quantified by generalization (perplexity), interpretability (topic coherence) and text retrieval. The code is available at https://github.com/YatinChaudhary/ Multi-view-Multi-source-Topic-Modeling . Consider a sparse target domain T and a set of source domains S , we first prepare two knowledge bases (KBs) of representations (or embeddings) from document collections of each of the |S| sources: (1) WordPool : a KB of pretrained word embeddings matrices { E 1 , ..., E |S| } , where E k  RE  K , and (2) TopicPool : a KB of pretrained latent topic embeddings { Z 1 , ..., Z |S| } , where Z k  RH  K encodes a distribution over a vocabulary of K words. Here, k  [1 , ..., |S| ] in superscript indicates knowledge of k th source, and E and H are word embedding and latent topic dimensions, respectively. While topic modeling on T , we introduce the two types of knowledge transfers from one or many sources: Local ( LVT ) and Global ( GVT ) View Transfer using the two KBs of pretrained word (i.e., WordPool ) and topic (i.e., TopicPool ) embeddings, respectively. Specially, we employ a neural autoregressive topic model, i.e., DocNADE as backbone in building the pools and realizing the multi-source multi-view framework. Table 2 describes the notations used. Notice that the superscript used in notations indicates a source. DocNADE (Larochelle and Lauly, 2012) is an unsupervised neural-network based generative topic model that is inspired by the benefits of NADE (Larochelle and Murray, 2011) and RSM (Salakhut-dinov and Hinton, 2009) architectures. Specifically, DocNADE factorizes the joint probability distribution of words in a document as a product of conditional distributions and efficiently models each conditional via a feed-forward neural network (ff-net), following reconstruction mechanism. DocNADE Formulation : For a document v = ( v 1 , ..., v D ) of size D , each word index v i takes value in { 1 , ..., K } of vocabulary size K . DocNADE learns topics in a language modeling fashion (Bengio et al., 2003) and decomposes the joint distribution p ( v ) = (cid:81) Di =1 p ( v i | v <i ) such that each autoregressive conditional p ( v i | v <i ) is modeled by a ff-net using preceding words v <i in the sequence: h i ( v <i ) = g ( c + (cid:88) q<i W : ,v q ) and g = { sigmoid , tanh } p ( v i = w | v <i ) = exp( b w + U w, : h i ( v <i )) (cid:80) w (cid:48) exp( b w (cid:48) + U w (cid:48) , : h i ( v <i )) for each word i  { 1 , ..., D } where v <i is the subvector consisting of all v q such that q < i i.e., v <i  { v 1 , ..., v i  1 } , g (  ) is a non-linear activation function, W  RH  K and U  RK  H are weight matrices, c  RH and b  RK are bias parameter vectors. H is the number of hidden units (the number of topics to be discovered). Figure 1 (left) (except WordPool ) describes the DocNADE architecture for the i th autoregressive step, where the parameter W is shared in the feed-forward networks and h i encodes latent document-topic proportion. The value of each unit j in the hidden vector signifies contribution of the j th topic in the proportion. Importantly, the topic-word matrix W has a property that the column vector W : ,v i corresponds to embedding of the word v i , whereas the row vector W j, : encodes latent features for the j th topic (i.e., topic-word distribution). We leverage this property to introduce external knowledge via word and topic embeddings. Algorithm 1 (for DocNADE, set both LVT and GVT to False ) demonstrates the computation of log p ( v ) and loss (i.e., negative log-likelihood) L ( v ) that is minimized using stochastic gradient descent. Moreover, computing each h i is efficient (linear complexity) due to NADE architecture that leverages the pre-activation a i  1 of ( i  1) th step in computing a i for the i th step (line #6). See Larochelle and Lauly (2012) for further details. Why DocNADE backbone : It has shown outperforming traditional models such as LDA and RSM. Additionally, Gupta et al. (2019a,b) have extended DocNADE on short texts by introducing context-insensitive word embeddings; however, based on a single-source transfer. Thus, we adopt DocNADE. We describe our transfer learning framework in topic modeling that jointly exploits the complementary prior knowledge accumulated in ( WordPool , TopicPool ), obtained from large document collections (DCs) from several sources. In doing so, we first apply the DocNADE to generate a topic-word matrix for each of the DCs, where its column-vector and row-vector generate E k and Z k , respectively for the k th source. See appendix for the mechanics of extracting word and topic embeddings from the topic-word matrix of a source. LVT+MST Formulation for Multi-source Word Embedding Transfer : As illustrated in Figure 1 (left) and Algorithm 1 (with LVT being True , line #7), we perform transfer learning on a target T using the WordPool of pretrained word embeddings { E 1 , ..., E |S| } from several sources S (i.e., multi-source) under the two schemes: scheme (i) : Using a domain-relevance factor  for every source in the WordPool such that the hidden vector h i encodes document-topic distribution, augmented with prior knowledge in form of pretrained word embeddings from several sources: h i ( v <i ) = g ( c + (cid:88) q<i W : ,v q + (cid:88) q<i |S| (cid:88) k =1  k E k : ,v q ) Here, k refers to the k th source and  k is a weight for E k that controls the amount of knowledge transferred in T , based on cross-domain overlap. scheme (ii) : Using a projection matrix P  RH  P with P = E  |S| in order to align word-embedding spaces of the target and all source domains for all D words in the document v such that: For q  { i, ...D } : e q = concat ( E 1: ,v q , ..., E k : ,v q ) h i ( v <i ) = g ( c + (cid:88) q<i W : ,v q + (cid:88) q<i P  e q ) Unlike scheme (i), the second schema allows us to automatically determine shifts in the target and source domains, identify and transfer relevant prior knowledge from many sources without configuring  for every source. To better guide TM, we also introduce pre-trained contextualized word embedding from BERT, concatenating with e q . GVT+MST Formulation for Multi-source Topic Embedding Transfer : Next, we perform knowledge transfer exclusively using the TopicPool of pretrained topic embeddings (e.g., Z k ) from one or several sources, S . In doing so, we add a regularization term to the loss function L ( v ) and require DocNADE to minimize the overall loss in a way that the (latent) topic features in W simultaneously inherit relevant topical features from each of the source domains S , and thus, it generates meaningful representations for the target T in order to address data-sparsity. The overall loss L ( v ) due to GVT+MST configuration in DocNADE is: L ( v ) =  log p ( v ) + |S| (cid:88) k =1  k H (cid:88) j =1 || A kj, : W  Z kj, : || 22 Target Domain Corpora Source Domain Corpora ID Data Train Val Test K L C ID Data Train Val Test K L C T 1 20NSshort 1.3k 0.1k 0.5k 1.4k 13.5 20 S 1 20NS 7.9k 1.6k 5.2k 2k 107.5 20 T 2 20NSsmall 0.4k 0.2k 0.2k 2k 187.5 20 S 2 R21578 7.3k 0.5k 3.0k 2k 128 90 T 3 TMNtitle 22.8k 2.0k 7.8k 2k 4.9 7 S 3 TMN 22.8k 2.0k 7.8k 2k 19 7 T 4 R21578title 7.3k 0.5k 3.0k 2k 7.3 90 S 4 AGNews 118k 2.0k 7.6k 5k 38 4 T 5 Ohsumedtitle 8.3k 2.1k 12.7k 2k 11.9 23 S 5 PubMed 15.0k 2.5k 2.5k 3k 254.8 T 6 Ohsumed 8.3k 2.1k 12.7k 3k 159.1 23 Table 3: Data statistics: Short/long texts and/or small/large corpora in target and source domains. T 1 T 2 T 3 T 4 T 5 T 6 S 1 I I R D D D S 2 D D D I D D S 3 R R I D D D S 4 R R R D D D S 5 D D D D -Table 4: Domain overlap in source-target corpora. Here, A k  RH  H aligns latent topics in the target T and k th source, and  k governs the degree of imitation of topic features Z k by W in T . Consequently, the generative process of learning meaningful topics in W of the target domain T is guided by relevant topic features { Z } |S| 1  TopicPool . Algorithm 1 (line #11) describes the computation of the loss, when GVT = True and LVT = False . Moreover, Figure 1 (right) illustrates the need for topic alignments between target and source(s). Here, j indicates the topic (i.e., row) index in a topic matrix, e.g., Z k . Observe that the first topic (gray curve), i.e., Z 1 j =1  Z 1 of the first source aligns with the first row-vector (i.e., topic) of W (of target). However, the other two topics Z 1 j =2 , Z 1 j =3  Z 1 need alignment with the target. MVT+MST Formulation for Multi-source Word and Topic Embeddings Transfer : When LVT and GVT are True (Algorithm 1) for many sources, the two complementary representations are jointly used in transfer learning using WordPool and TopicPool , and therefore, the name multi-view and multi-source transfers. Computational complexity of NTM : For DocNADE, the complexity of computing all hidden layers h i ( v <i ) is in O ( DH ) and all p ( v | v <i ) in O ( KDH ) . Thus, the overall complexity of DocNADE is in O ( DH + KDH ) . Within the proposed transfer learning framework, the complexity of computing all hidden layers ( LVT+MST in scheme (i)) and topic-embedding transfer term ( GVT+MST ) is in O ( DH + |S| DH ) and O ( |S| KH ) , respectively. Since |S| <<H , thus the overall complexity of DocNADE with MVT+MST is in O ( DH + KDH + KH ) . Datasets : Table 3 describes the datasets used in high-resource source and low-and high-resource target domains for our experi-Baselines ments. The target domain T consists of four short-text corpora ( 20NSshort , TMNtitle , R21578title and Ohsumedtitle ), one small corpus ( 20NSsmall ) and two large corpora ( TMN and Ohsumed ). However in source S , we use five large corpora ( 20NS , R21578 , TMN , AGnews and PubMed ) in different label spaces (i.e, domains). Here, the corpora ( T 5 , T 6 and S 5 ) belong to medical and others to news . Additionally, Table 4 suggests domain overlap (label match) in the target and source corpora, where we define 3 types of overlap: I (identical) if all labels match, R (related) if some labels match, and D (distant) if a very few or no labels match. Note, our approaches are completely unsupervised and do not use the data labels ( appendix ). Reproducibility : We follow the experimental setup similar to DocNADE (Larochelle and Lauly, 2012) and DocNADEe (Gupta et al., 2019a), where the number of topics ( H ) is set to 200 . While DocNADEe requires the dimension (i.e., E ) of word embeddings be the same as the latent topic (i.e., H ), we follow scheme (ii) (Algorithm 1) to introduce KBs from Model Scores on Target Corpus ( in sparse-data and sufficient-data settings ) Source or Transfer 20NSshort TMNtitle R21578title 20NSsmall TMN Corpus Type PPL COH IR PPL COH IR PPL COH IR PPL COH IR PPL COHB a s e li n e s Baseline TMNVDM 1047 .736 .076 973 .740 .190 372 .735 .271 957 .515 .090 833 .673 without WordProdLDA 923 .689 .062 1527 .744 .170 480 .742 .200 1181 .394 .062 1519 .577 Embeddings DocNADE 646 .667 .290 706 .709 .521 192 .713 .657 594 .462 .270 584 .636 P r o p o s e d 20NS LVT 630 .673 .298 705 .709 .523 194 .708 .656 594 .455 .288 582 .649 GVT 646 .690 .303 718 .720 .527 184 .698 .660 594 .500 .310 590 .652 MVT 638 .690 .314 714 .718 .528 188 .715 .655 600 .499 .311 588 .650 TMNLVT 649 .668 .296 655 .731 .548 187 .703 .659 593 .460 .273 -GVT 661 .692 .294 689 .728 .555 191 .709 .660 596 .521 .276 -MVT 658 .687 .297 663 .747 .553 195 .720 .660 599 .507 .292 -R21578 LVT 656 .667 .292 704 .715 .522 186 .715 .676 593 .458 .267 581 .636 GVT 654 .672 .293 716 .719 .526 194 .706 .672 595 .485 .279 591 .646 MVT 650 .670 .296 716 .720 .528 194 .724 .676 599 .490 .280 589 .650 AGnews LVT 650 .677 .297 682 .723 .533 185 .710 .659 592 .458 .260 564 .668 GVT 667 .695 .300 728 .735 .534 190 .717 .663 598 .563 .282 601 .684 MVT 659 .696 .290 718 .740 .533 189 .727 .659 599 .566 .279 592 .686 MSTLVT 640 .678 .308 663 .732 .547 182 .739 .673 594 .542 .277 568 .674 GVT 658 .705 .305 704 .746 .550 192 .727 .673 599 .585 .326 602 .680 MVT 656 .740 .314 680 .752 .569 188 .745 .685 600 .637 .285 600 .690 Gain%(vs DocNADE)  2.48  10.9  8.28  7.22  6.06  9.21  5.20  4.49  4.26  0.34  37.9  20.7  3.42  8.50 Table 6: State-of-the-art comparisons with TMs: Perplexity (PPL), topic coherence (COH) and precision@recall (IR) at retrieval fraction 0 . pre-trained word embeddings from Glove, FastText ( E =300) (Bojanowski et al., 2017) and BERT-base ( E =768) models. See appendix for the experimental setup, hyperparameters and optimal values of  k  [0 . 1 , 0 . 5 , 1 . 0] and  k  [0 . 1 , 0 . 01 , 0 . 001] . Baselines (Related Works) : (1) Topic Models without Transfer Learning that learn topics in isolation using the given target corpus only. We employ LDA-based variant, i.e., ProdLDA (Srivastava and Sutton, 2017) and neural network-based variants, i.e., DocNADE (autoregressive) and NVDM (non-autoregressive) (Miao et al., 2016). (2) Topic Models with Transfer Learning that leverages pre-trained context-insensitive word embeddings (Pennington et al., 2014). We consider topic models based on both LDA, i.e., Gauss-LDA (Das et al., 2015) and glove-GMM (Nguyen et al., 2015), and neural networks, i.e., DocNADEe (Gupta et al., 2019a). They do not leverage pretrained topic-embeddings (i.e., GVT ), contextualized word-embedding and MST-MVT techniques. (3) Unsupervised Document Representation to quantify the quality of document representations. We use 3 strategies: doc2vec (Le and Mikolov, 2014), EmbSum-Glove and EmbSum-BERT (rep-resent a document by summing the pre-trained embeddings of it's words from Glove and BERT). transfer learning capabilities of the proposed framework, where we build (train) a TM using all source corpora and evaluate on the target corpus T , and (5) Data-augmentation that first augments the target corpus with all the source corpora and then builds a TM to evaluate transfer learning on T . Table 5 summarizes the comparison of this work with the aforementioned baselines. Tables 6 and 7 employ baseline TMs without and with transfer learning, respectively. To evaluate generative performance of DocNADE-based NTM, we compute average held-out perplexity per word: PPL = exp (cid:0)  1 N (cid:80) Nt =1 1 | v t | log p ( v t ) (cid:1) , where N and | v t | are the number of documents and words in a document v t , respectively. Tables 6 and 7 quantitatively show PPL scores on the five target corpora using one or four sources. In Table 6 using TMN (as a single source) for LVT, GVT and MVT transfer types on the target TMNtitle , we see improved (reduced) PPL scores: ( 655 vs 706 ), ( 689 vs 706 ) and ( 663 vs 706 ) respectively in comparison to DocNADE. We also observe gains due to MST+LVT, MST+GVT and MST+MVT configurations on TMNtitle . Similarly in MST+LVT for R21578title , we observe a gain of 5.2% (182 vs 192), suggesting that multi-source transfer learning using pretrained KBs from Model Scores on Target Corpus ( in sparse-data and sufficient-data settings ) Source or Transfer 20NSshort TMNtitle R21578title 20NSsmall TMN Corpus Type PPL COH IR PPL COH IR PPL COH IR PPL COH IR PPL COHB a s e li n e s doc2vec -.090 -.190 -.518 -.200 -EmbSum-Glove -.236 -.513 -.587 -.214 -EmbSum-BERT -.261 -.499 -.594 -.262 -Baseline TM Gauss-LDA -.080 -.408 -.367 -.090 -with Word-glove-DMM -.512 .183 -.633 .445 -.364 .273 -.578 .090 -.705 word and topic embeddings (jointly) helps improving TM, and it also verifies domain relatedness (e.g., in TMNTMNtitle and AGnews -TMN ). Similarly, Table 7 reports gains in PPL (e.g., on TMNtitle , R21578title , etc.) compared to the baseline DocNADEe. PPL scores due to BERT can be not computed since its embeddings are aware of both preceding and following contexts. In Table 8, we show PPL scores on 2 medical target corpora: Ohsumtitle and Ohsumed using 2 sources: AGnews ( news ) and PubMed ( medical ) to perform cross-domain and in-domain transfers. We see that using PubMed for LVT on both the targets improves generalization. Overall, we report a gain of 17.3% ( 1268 vs 1534 ) on Ohsumedtitle and 8.55% ( 1497 vs 1637 ) on Ohsumed datasets, compared to DocNADEe. While PPL is used for model selection, Chang et al. (2009) showed in some cases humans preferred TMs (based on the semantic quality of topics) with higher (worse) perplexities. Therefore, we also estimate the quality of topics. We follow Rder et al. (2015) and Gupta et al. (2019a) to compute COH of the top 10 words in each topic. Essentially, the higher scores imply the coherent topics. Tables 6 and 7 (under COH column) demonstrate that our approaches (GVT, MVT and MST) show noticeable gains and thus improve topic quality. For instance in Table 6, when AGnews is used as a single source for 20NSsmall datatset, we observe a gain in COH due to GVT (.563 vs .462) and MVT (.566 vs .462). Additionally, noticeable gains are reported due to MST+LVT (.542 vs .462), MST+GVT (.585 vs .462) and MST+MVT (.637 vs .462), compared to DocNADE. Importantly, we find a trend MVT > GVT > LVT in COH scores for both the single-source and multi-source transfers. Similarly, Table 7 show noticeable gains (e.g., 40.7%, 10.4%, 7.08%, etc.) in COH due to MST+MVT+Glove +FastText+BERT setting. Moreover, Table 8 shows gains in COH due to GVT on Ohsumedtitle and Ohsumed , using pretrained knowledge from PubMed . Overall, the GVT, MVT and MST boost COH for all the five target corpora compared to the baseline TMs (i.e., DocNADE and DocNADEe). The improvements suggest that the approaches scale across domains. We further evaluate the quality of document representations and perform an IR task using the label information only to compute precision. We follow the experimental setup similar to Gupta et al. (2019a). See the details in appendix . Tables 6 and 7 report precision scores at retrieval fraction 0 . 02 where the configuration MST+MVT outperforms both the DocNADE and DocNADEe for all 4 targets. We observe large gains in precision: (a) Table 6: 20.7% (.326 vs .270) on 20NSsmall , 9.21% (.569 vs .521) on 0.001 0.002 0.005 0.01 0.02 0.05 0.1 0 . TMNtitle , etc., (b) Table 7: 11.9% (.604 vs .540) on TMNtitle and 9.5% (.322 vs .294) on 20NSshort , etc., (c) Table 8: 14.4% (.183 vs .160) on Ohsumedtitle . Additionally, Figures 2a, 2b, 2c and 2d illustrate precision-recall curves on 20NSshort , 20NSsmall , TMNtitle and R21578title respectively, where MST+MVT and MST+GVT consistently outperform the baselines at all fractions. Figures 2a, 2b, 2c and 2d show precision in the zero-shot (source-only training) and data-augmentation (source+target training) configurations. Observe that the latter helps in learning meaningful representations and performs better than zero-shot; however, it is outperformed by MST+MVT, suggesting that a naive (data space) augmentation does not add sufficient prior or relevant information to the sparse target. Thus, we find that it is beneficial to augment training data in feature space (e.g., LVT, GVT and MVT) especially for unsupervised topic models using WordPool and TopicPool . Moreover in the few-shot setting, we first split the training data of TMNtitle into several sets: 20%, 40%, 60%, 80% of the training set and then retrain DocNADE, DocNADEe and Doc-NADE+MST+MVT on each as a sparse target. We demonstrate transfer learning in such sparse-data settings using the KBs: WordPool and TopicPool jointly. Figure 2e plots precision at retrieval fraction 0 . 02 and validates that the proposed modeling consistently outperforms both the baselines: DocNADE and DocNADEe. Beyond IR, we further investigate computing topic coherence (COH) for the zero-shot and data-augmentation baselines, where the COH scores in Figure 2f suggest that MST+MVT outperforms DocNADEe, zero-shot and data-augmentation. For topic level inspection, we first extract topics using the rows of W of source and target corpora. Table 9 shows the topics (top-5 words) from source and target domains. Observe that the target topics become more coherent after transfer learning (i.e., +GVT) from one or more sources. The blue color signifies that a target topic has imitated certain topic words from the source. We also show a topic (the last) improved due to multi-source transfer. Observe that the NNs in the target become more meaningful by gaining knowledge mainly from 20NS source. We have presented a state-of-the-art neural topic modeling framework using multi-view embedding spaces: pretrained topic-embeddings and word-embeddings (context-sensitive and context-insensitive) from one or many sources to improve quality of topics and document representations. This research was supported by the Federal Ministry for Economic Affairs and Energy (Bundeswirtschaftsministerium: bmwi.de), grant 01MD19003E (PLASS: Platform for Analytical Supply Chain Mangement Services, plass.io) at Siemens AG (TechnologyMachine Intelligence), Munich Germany.
abstain objective objective method method result abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain objective method result other other abstain method method abstain abstain abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain method other abstain other method method method method other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain method abstain method method method method other abstain abstain abstain method abstain method abstain abstain method abstain abstain abstain abstain abstain method abstain abstain abstain other abstain abstain abstain abstain abstain method abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain method method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result other