Processing open-domain Chinese texts has been a critical bottleneck in computational linguistics for decades, partially because text segmentation and word discovery often entangle with each other in this challenging scenario. No existing methods yet can achieve effective text segmentation and word discovery simultaneously in open domain. This study fills in this gap by proposing a novel method called TopWORDS-Seg based on Bayesian inference, which enjoys robust performance and transparent interpretation when no training corpus and domain vocabulary are available. Advantages of TopWORDS-Seg are demonstrated by a series of experimental studies. Due to absence of word boundaries in Chinese, Chinese natural language processing (CNLP) faces a few unique challenges, including text segmentation and word discovery. When processing open-domain Chinese corpus containing many unregistered words and named entities, these challenges become more critical as they often entangle with each other: we usually cannot segment Chinese texts correctly without knowing the underlying vocabulary; on the other hand, it is often difficult to precisely discover unregistered words and named entities from open-domain corpus without guidance on text segmentation. Most methods for CNLP in the literature assume that the underlying vocabulary is known and focus on improving performance of text segmentation in closed test. The first category of methods along this research line are simple methods based on Word Matching (Chen and Liu, 1992; Geutner, 1996; Chen, 2003; Shu et al., 2017), which segment a Chinese sentence by matching sub-strings in the sentence to a pre-given vocabulary in a forward or * Corresponding author. reserve order. The second category of methods utilize manually segmented corpus or large-scale pretraining corpus to train statistical models such as Maximum Entropy (Berger et al., 1996; McCallum et al.; Low et al., 2005), HMM (Sproat et al., 1994; Zhang et al., 2003) and CRF (Lafferty et al., 2001; Xue, 2003; Peng et al., 2004; Luo et al., 2019), or deep learning models including CNN (Wang and Xu), LSTM (Chen et al., 2015), Bi-LSTM (Ma et al., 2018) and BERT (Yang, 2019), or hybrid models like Bi-LSTM-CRF (Huang et al., 2015) and LSTM-CNNs-CRF (Ma and Hovy, 2016), to achieve text segmentation directly or indirectly. Methods of this category have led to popular toolkits for processing Chinese texts, including Jieba (Sun, 2012), StanfordNLP (Manning et al., 2014), THULAC (Sun et al., 2016), PKUSEG (Luo et al., 2019), and LTP (Che et al., 2021). A popular strategy adopted by some of these toolkits is to segment the target texts into sequences of basic words first, and capture unregistered words and named entities, which are often word compounds consisting of basic words, later via chunking and syntactic analysis. Although such a strategy can equip these toolkits with some ability on word discovery, it is apparently sub-optimal, because we may mis-segment basic words at the first place without realizing the existence of potential technical words, making it impossible to discover technical word compounds correctly in post analysis such as chunking and syntactic analysis. On the other hand, unsupervised methods are also developed to achieve text segmentation when no pre-given vocabulary and manually segmented training corpus are available. Some methods of this research line segment texts based on local statistics of the target texts, including Description Length Gain (Kit and Wilks, 1999), Mutual Information (Chang and Lin, 2003), Accessor Variety (Feng et al., 2004), Evaluation-Selection-Adjustment Process (Wang et al., 2011), and Normalized Variation 158 of Branching Entropy (Magistry and Sagot, 2012). The others, however, rely on generative statistical models whose parameters can be estimated from the target texts only, including Hierarchical Dirichlet Process (Goldwater et al., 2009), Nested Pitman-Yor Process (Mochihashi et al., 2009), Bayesian HMM (Chen et al., 2014), TopWORDS (Deng et al., 2016) and GTS (Yuan et al., 2020). In general, methods based on word matching and unsupervised learning cannot produce high-quality text segmentation (Zhao and Kit, 2011), although some unsupervised methods are successful on word discovery (Deng et al., 2016). Methods based on supervised learning can achieve excellent performance in closed test (Emerson, 2005), but often suffer from dramatic performance degradation when applied to open-domain Chinese corpus containing many unregistered words and named entities (Liu and Zhang, 2012; Wang et al., 2019). Methods based on deep learning are usually more robust under the pre-training and fine-tuning framework, but still suffer from unstable performance and often fail to correctly segment technical words, which play a key role in deciphering the meaning of domain-specific texts, when applied to open-domain texts (Zhao et al., 2018; Fu et al., 2020). There are also some efforts in the literature to integrate supervised and unsupervised methods for improved performance (Zhao and Kit, 2007, 2008, 2011; Wang et al., 2019; Yang et al., 2019). But, these methods either heavily depend on manually labelled corpus for model training, or suffer from unbalanced emphasis on text segmentation and word discovery, resulting in limited improvement for CNLP in open domain. These facts make processing open-domain Chinese texts a critical bottleneck in computational linguistics even for today. Many factors contribute to the stagnation on development of efficient tools for processing open-domain Chinese texts. From the methodology point of view, we do not have a proper learning framework yet to connect the text segmentation problem to the word discovery problem and deal with them at the same time effectively. From the practical point of view, the lack of proper evaluation criterion in open domain places a critical barrier for fair comparison of different methods and discourages researchers from looking for potential solutions. This study tries to provide solutions to these critical issues. First, we propose a novel Bayesian framework to integrate TopWORDS, an effective word discoverer (Deng et al., 2016), and PKUSEG, a strong text segmenter, leading to a more efficient text segmenter called TopWORDS-Seg, which can achieve effective text segmentation and word discovery simultaneously in open domain. Next, we design a cocktail strategy for method evaluation and comparison by measuring the overall performance of a target method on both text segmentation in benchmark corpus and technical word discovery and segmentation in open-domain corpus. Experimental studies demonstrate that the proposed TopWORDS-Seg outperforms existing methods with a significant margin for CNLP in open domain. Proposed by Deng et al. (2016), TopWORDS is a general approach for offline natural language processing based on unsupervised statistical learning. Assuming that sentences are generated by randomly sampling and concatenating words from an underlying word dictionary (i.e., unigram language model), TopWORDS starts with an over-complete initial word dictionary D containing all plausible word candidates in the target texts, and gradually simplifies the model by removing non-significant word candidates from D based on statistical model selection principles, with the unknown word usage frequencies estimated by EM algorithm (Dempster et al., 1977). TopWORDS is closely related to methods widely used in neural machine translation for constructing sub-word dictionary, and can be viewed as an advanced version of WordPiece (Schuster and Naka-jima, 2012), Byte Pair Encoding (Sennrich et al., 2016) and Unigram Language Model (Kudo, 2018). In practice, TopWORDS is particularly effective on discovering words, technical terms and phrases from open-domain Chinese texts, but tends to segment texts with coarser granularity at phrase instead of word level. In this section, we upgrade TopWORDS from a weak text segmenter with strong ability on word discovery to a more powerful tool enjoying balanced ability on both dimensions via Bayesian inference. Following the setting in Deng et al. (2016), let T = { T 1 ,    , T n } be a collection of unsegmented Chinese text sequences to process, A = { a 1 , a 2 ,    , a M } be the set of Chinese characters involved in T , and DT be the underlying vocabulary behind T unknown to the investigator. We aim to discover DT from T , and predict the invisible word boundary profile B j = ( b j 1 ,    , b jL j ) for each piece of unsegmented Chinese text T j = a j 1 a j 2    a jL j e , where b jl = 1 if there is a word boundary behind the l -th position of T j and 0 otherwise, and e is a special end mark indicating the end of text sequence. To learn DT , we starts with an over-complete initial word dictionary D = { w 1 , w 2 , . . . , w N , e } covering all plausible word candidates in T (i.e., all sub-strings in T whose length   L and frequency   F ) and the end mark e . For simplicity, we always assume that DT  D and all characters in A are covered by D . Under the unigram language model, we have the following likelihood function for a piece of unsegmented text T j given B j and D : P ( T j | D ,  , B j ) = (cid:89) w D (  w ) n w ( B j ) , (1) where  = {  w } w D with  w being the usage frequency of word w in T , and n w ( B j ) counts the number of occurrences of word w in the segmented version of T j based on B j . Let B = { B 1 ,    , B n } being the word boundary profiles of the n text sequences in T . We have P ( T | D ,  , B ) = n (cid:89) j =1 P ( T j | D ,  , B j ) = (cid:89) w D (  w ) n w ( B ) , (2) where n w ( B ) = n (cid:88) j =1 n w ( B j ) . In this study, we propose to specify a joint prior distribution  (  , B ) for (  , B ) to integrate prior preference on word usage and text segmentation into the learning procedure. According to the Bayes Theorem, we have the following posterior distribution of (  , B ) given T and D : P (  , B | T , D )   (  , B )  P ( T | D ,  , B ) , which leads to the following marginal and conditional posterior distributions: P (  | T , D ) = (cid:90) P (  , B | T , D ) d B , P ( B | T , D ,  )  P (  , B | T , D ) . There are various ways to specify the prior distributions  (  , B ) . In this study, we choose to use the independent conjugate prior below for conceptual and computational convenience:  (  , B ) =  (  )   ( B ) , where  (  ) = Dirichlet (  |  ) ,  ( B ) = n (cid:89) j =1  ( B j ) = n (cid:89) j =1 L j (cid:89) l =1  ( b jl ) ,  ( b jl ) = Binary ( b jl |  jl ) , with  = {  w } w D and  = {  jl } being the hyper-parameters controlling the strength of prior information. In this study, we choose to specify  w = 1 ,  w  D , (4) leading to a flat prior distribution for  , but adopt a non-flat prior distribution for  by smoothing the word boundary profiles B  = { B  j } 1  j  n predicted by a pre-given text segmenter S  :  jl = (cid:40) (1   )  b  jl +   , l < L j , 1 , l = L j , (5) where b  jl is the location-specific binary segmentation indicator predicted by S  ,   (0 , 1) is the smoothing parameter, and  > 0 highlights the probability to place a word boundary at each location by a pseudo segmenter that places boundaries randomly in the text sequence. Here, we set  = 0 . 5 by default, and leave  as a hyper-parameter that can be tuned to fit different application scenarios, leading to the following joint prior distribution:   (  , B )  n (cid:89) j =1 L j (cid:89) l =1 (  jl ) b jl (1   jl ) 1  b jl . Given the prior distribution   (  , B ) specified previously, the posterior distribution becomes: P (  , B | T , D )    (  , B )  P ( T | D ,  , B )  n (cid:89) j =1 (cid:34)   ( B j )  (cid:89) w D (  w ) n w ( B j ) (cid:35) , (7) where   ( B j ) = L j (cid:89) l =1 (  jl ) b jl (1   jl ) 1  b jl is a deterministic function of  , as  jl 's degenerate to constants for fixed  based on (5). Under such a Bayesian model, the problem of word discovery can be naturally converted into a statistical model selection problem, as only word candidates whose usage frequency  w is significantly larger than 0 could be meaningful words. We estimate  by the posterior mode   as defined in (3), which can by obtained via the EM algorithm (Dempster et al., 1977) with B as the missing data. Details of the EM algorithm are described in Appendix A. Once the EM algorithm gets converged, we can evaluate the statistical significance of a word candidate w by the likelihood-ratio statistics between the full model and a reduced model with w removed:  w = log (cid:32) P ( T | D ,   ) P ( T | D ,   [ w =0] ) (cid:33) , (8) where   [ w =0] is the modification of   by setting   w = 0 with other elements unchanged. Apparently, a larger  w suggests that word candidate w is more important for fitting the observed texts, and thus is more likely to be a meaningful word. Because  2  w   2 asymptotically under the null hypothesis that the reduced model with w removed is the true model, we can filter out word candidates whose  w <   , where threshold   is the (1  0 . 05 N ) -quantile of the  2 distribution, following the Bonferroni correction principle for multiple hypothesis testing. As demonstrated by Deng et al. (2016), such a model selection strategy can effectively filter out most meaningless word candidates and results in a concise final dictionary containing meaningful words and phrases only. Considering that  w =  n (cid:88) j =1 log (1  r wj ) , where r wj = P  (cid:16) w  B j | T j , D ,   (cid:17) (9) = (cid:88) B j B j I ( w  B j )  P  ( B j | T j , D ,   ) , with notation  w  B j  meaning that word candidate w appears in the segmented version of T j based on B j , we can get  w by calculating r wj for each T j . Alternatively, we can also calculate the posterior probability of existing a word boundary at position ( j, l ) as  jl = (cid:88) B B j b jl  P  ( B j | T j , D ,   ) , (11) and segment T j based on  B j = I (  j   S ) , (12) where  j = (  j 1 ,    ,  jL j ) and  S is a pre-given threshold with 0.5 as the default value. Here, we choose to use the second segmentation strategy, because it leads to more robust results in practice. Integrating the dictionary initialization stage via sub-string enumeration, the prior construction stage guided by a pre-given segmenter S  (i.e., PKUSEG by default), the word discovery stage empowered by EM algorithm and likelihood-ratio tests, and the text segmentation stage based on conditional probability inference, into a united framework, we come up with the TopWORDS-Seg algorithm as demonstrated in Figure 1. Computation issues involved in the algorithm are detailed in Appendix B. A collection of hyper-parameters, including  L ,  F ,  ,  and  S , are associated with the TopWORDS-Seg algorithm, and need be specified to initiate the algorithm. We recommend to set  L = 15 ,  F = 2 and  =  S = 0 . 5 by default. The specification of hyper-parameter  is a bit complicated. To capture unregistered words from open-domain texts more efficiently, we would like to 161 Target text  Segmented text  Final dictionary   Prior distribution () Initial dictionary      Posterior distribution  , ,   , (|,,) Word boundary profiles    Sub-string enumeration Bayesian framework Pregiven segmenter   Prior specification Model selection with   Text segmentation Parameter estimation   Prior for text segmentation    (,) Dictionary Initialization Stage Prior Specification Stage Text Segmentation Stage Word Discovery Stage   Unigram language model Prior for word discovery    (,) Estimated parameters  Figure 1: Flow chart of the TopWORDS-Seg choose a larger  to encourage word discovery. To segment regular texts more precisely, however, we would like to choose a smaller  instead to better utilize the prior information. To get rid of the dilemma, we allow to specify  with different values in different tasks, i.e., using a large  (referred to as  d ) in the word discovery stage and a small  (referred to as  s ) in the text segmentation stage. Based on a wide range of experimental studies, we suggest to set  d = 0 . 5 and  s = 0 . 001 by default. Composed of over 10 billion Chinese character tokens from 3.6 million webpages, Chinese Wikipedia ( https://dumps.wikimedia.org/ ) is one of the largest open-source Chinese corpus. Containing rich contents of various domains and millions of technical terms highlighted by hyperlinks, the Chinese Wikipedia is an ideal corpus for studying CNLP in open domain. Considering that it's computationally expensive to processing all webpages in Chinese Wikipedia, we randomly picked up 1,500 webpages involving 8 million Chinese character tokens (referred to as Chinese Wiki-Rand, or TW-R ) as the representative samples of the general texts in Chinese Wikipedia. Moreover, we selected two collections of special webpages from Chinese Wikipedia with label    " (referred to as Chinese Wiki-Film, or TW-F ) or    " (referred to as Chinese Wiki-Physics, or TW-P ), involving  5 million Chinese character tokens for each, as the representatives of the domain-specific texts in Chinese Wikipedia. Figure 2 (a) and (b) demonstrates a typical Wikipedia web page and histograms for term length and appearance frequency of technical terms involved in TW-R . In this section, we apply TopWORDS-Seg to process these Wikipedia corpora separately, and compare its performance to 6 existing methods, including Jieba (Sun, 2012), StanfordNLP (Man-ning et al., 2014), THULAC (Sun et al., 2016), PKUSEG (Luo et al., 2019), LTP (Che et al., 2021), and TopWORDS (Deng et al., 2016) itself, from various aspects. Due to the lack of gold standard, it is not straightforward to evaluate and compare the performance of different methods on open-domain corpus like Chinese Wikipedia. Here, we propose a cocktail strategy for method evaluation by measuring the overall performance of each method on both open-domain corpuora and benchmark corpus. Let V t be the collection of frequent technical terms in a particular Wikipedia corpus (terms with hyperlinks appear at least 2 times), with n w be the number of occurrences for each w  V t . Suppose V is the discovered vocabulary reported by a particular method M , and m w is the number of successful catches of w by M . Taking advantage of the self-labelled technical terms with hyperlinks in Wikipedia webpages, it is straightforward to measure discovery recall R d and segmentation recall R s for technical terms in V t as below: R d = | V t  V | | V t | and R s = (cid:80) w  V t m w (cid:80) w  V t n w . M to deal with technical terms in open-domain texts. Because it is difficult to directly evaluate the perform of a method M on segmenting non-technical contents of the Wikipedia corpus, we retreat to indirect evaluation by evaluating its performance on segmenting the PKU corpus TP , a benchmark corpus with gold standard released by SIGHAN 2005 Bake-Off (Emerson, 2005), instead. Let F s be the F 1 score of method M on text segmentation for the PKU corpus. Score F s reflects M 's ability to process general Chinese texts without technical contents. Apparently, R d , R s and F s measure the strength of a method comprehensively from various aspects, with both word discovery and text segmentation considered for technical as well as non-technical texts. Such a cocktail strategy provide us a principle to evaluate and compare the overall performance of different CNLP methods in open domains. If a method enjoys high R d , R s and F s values across different corpora stably, we would feel comfortable to claim it as a robust tools for CNLP in open domains. Figure 2 (c) summarizes the performance of TopWORDS-Seg (with the default setting) and the 6 competing methods on the Wikipedia and PKU corpora in terms of R d , R s and F s , with the size of discovered vocabulary | V | reported as well. Comparing these results, we find that TopWORDS-Seg enjoys robust performance on segmenting classic benchmark corpus ( F s = 82 . 2% for TP ), open-domain corpus ( R s = 76 . 5% for TW-R ) and domain-specific corpus ( R s = 76 . 8% and 70 . 8% for TW-F and TW-P respectively), and high effi-ciency on discovering technical terms ( R d > 82% for all three Wikipedia corpora). The other methods, however, all suffer from either missing too many technical terms in the Wikipedia corpora ( R d ranging from 45% to 77% as in supervised methods), or segmenting the PKU corpus poorly ( F s = 50 . 4% as in TopWORDS). Considering that TopWORDS-Seg reports a vocabulary that is 16K smaller than TopWORDS, it actually outperforms TopWORDS significantly in all dimensions. Moreover, considering that both TopWORDS and TopWORDS-Seg tend to segment Chinese texts at coarser granularity with technical terms and phrases preserved as composite words instead of cutting them into smaller language units, the text segmentation standard adopted by the PKU corpus, which tends to segment Chinese texts at finer granularity, may over-punish them. To ease the impact on performance evaluation due to segmentation granularity, we choose to mask part of the PKU corpus TP where method M is not consistent with the standard segmentation only on granularity (with the concrete criteria detailed in Appendix C), and measure the F 1 score of method M on the masked version of TP only, leading to a masked version of F s referred to as F m . The proportion of masked corpus (i.e., mask rate ) is also calculated for each method and reported in Figure 2 (c). TopwORDS-Seg achieves an improved F m = 93 . 7% with a mask rate of 16.6%, suggesting that TopwORDS-Seg actually segments the PKU corpus very well. Meanwhile, a much higher mask rate of 50.4% is obtained for TopWORDS, which is consistent to our impression that TopWORDS tends to preserve too many sub-phrases in text segmentation. In addition, because some methods based on supervised learning, e.g., Jieba, THULAC and PKUSEG, can receive external vocabulary for processing open-domain corpus, there exists an alternative strategy to integrate TopWORDS with thses methods by simply forwarding the vocabulary discovered by TopWORDS to them. We refer to approaches based on this strategy as TopWORDS-Jieba/THULAC/PKUSEG, and report their performance on both Chinese Wikipedia corpus and PKU corpus in Figure 2 (c) as well. Unfortunately, although this family of approaches achieve a higher R d in general, they tend to report an over-large vocabulary and segment texts with coarser granularity like TopWORDS does. These results indicate that simply concatenating TopWORDS to other methods does not necessarily lead to an improved approach, and thus imply that the proposed strategy based on Bayesian inference is not trivial. The heatmaps in Figure 2 (d) demonstrate the similarity on text segmentation of different methods on four different target corpora, where the similarity between any two methods M i and M j is measured by  ij = (cid:80) T T D sum ( B ( i ) T  B ( j ) T ) (cid:80) T T D sum ( B ( i ) T  B ( j ) T ) , with B ( i ) T denoting the predicted word boundary vector of text sequence T by method M i . From the figure, we can see clearly that text segmentation 163 (a) A typical web page in Chinese Wikipedia. (b) Key characteristics of technical terms involved in Chines Wikipedia. (c) Results on PKU, Chinese Wiki-Rand, Chinese Wiki-Film and Chinese Wiki-Physics datasets of different methods. (d) Similarity on text segmentation of different methods on four different target corpora. (e) Segmentation results on a typical sentence reported by TopWORDS-Seg is very similar to the results reported by supervised methods, but is significantly different from the result reported by TopWORDS for all four corpora. Such results confirm the strength of TopWORDS-Seg on text segmentation in addition to word discovery, and provide strong evidences to support TopWORDS-Seg as a powerful tool for processing open-domain Chinese texts. Figure 2 (e) shows an illustrative example of text segmentation of PKUSEG, TopWORDS and TopWORDS-Seg for a piece of target text, respectively. Apparently, PKUSEG segments the target text almost perfectly except for chopping the technical term allotropes (     ) into three substrings by mistake, due to the lack of ability to recognize unregistered words. TopWORDS, however, successfully recognizes and segments the technical term allotropes correctly, but segments the other part of the target text with coarser granularity leaving phrases like physical properties (     ) and extremely different (     ) as unsegmented language units. TopWORDS-Seg, as expected, segments the target text perfectly, with 164 the technical term allotropes correctly recognized and the rest part segmented with proper granularity. Written by Goodfellow et al. (2016), the book Deep Learning has become a classic tutorial for deep learning. In 2017, its Chinese version was published in China (see Figure 3 (a) for the book's cover), which is composed of more than 400,000 Chinese character tokens (referred to as TD ). Covering rich technical contents in the domain of machine learning, including over 800 technical terms as listed in the Index Table at the end of the book, such a book is an ideal target for testing the performance of the proposed TopWORDS-Seg in real application. Feeding full text of the book to TopWORDS-Seg and competing methods respectively, we obtained results as summarized in Figure 3. Figure 3 (b) shows that TopWORDS-Seg discovers 84.1% technical terms listed in the Index Table of the book with a vocabulary of 10.7K discovered words. TopWORDS achieves a slightly higher R d = 85 . 0% at the price of a larger vocabulary with 12.8K discovered words. Other methods based on supervised learning result in much lower R d with the vocabulary size varying between 6.8K to 12.2K. Figure 3 (d) shows the most frequent words discovered by TopWORDS-Seg. Figure 3 (e) displays part of the technical terms captured by TopWORDS-Seg but missed by all supervised methods, which are all meaningful technical terms like unsupervised learning (      ) and stochastic gradient decent (       ). Figure 3 (f) summarizes typical pseudo words and phrases reported by TopWORDS but eliminated by TopWORDS-Seg, which are all common collocations widely used but usually not treated as words in Chinese, e.g., in the model (    ) and it is because of (    ). These results suggest that TopWORDS-Seg is indeed more effective than competing methods on word discovery. In terms of text segmentation, the heatmap in Figure 3 (c) visualizes the similarity between TopWORDS-Seg and other approaches on this corpus in a similar fashion as in Figure 2 (d). Again, the performance of TopWORDS-Seg is very similar to the supervised methods, and demonstrates significant difference from TopWORDS, suggesting that TopWORDS-Seg is a robust tool with balanced ability on processing open-domain Chinese texts. In this paper, we proposed TopWORDS-Seg, a powerful tool for processing open-domain Chi-165 nese texts based on Bayesian inference with balanced ability on text segmentation and word discovery. A series of experimental studies confirm that TopWORDS-Seg can discover unregistered technical terms in open-domain texts effectively, and achieve high-quality text segmentation on both benchmark and open-domain corpora. Taking advantage of the Bayesian framework, TopWORDS-Seg is ready to process large scale open-domain Chinese texts without extra training corpus or pregiven domain vocabulary, leading to an ideal solution to a critical bottleneck existing in computational linguistics for decades. Moreover, combing the strong points of PKUSEG and TopWORDS via Bayesian inference, TopWORDS-Seg enjoys transparent reasoning process, and is fully interpretable to most people. In practical applications, such a property is very attractive to many researchers and practicers. Meanwhile, TopWORDS-Seg also suffers from a few obvious limitations. For example, although the current learning framework is effective to discover frequent words, it tends to miss many rare words that appear only a few times in the texts. For another instance, because PKUSEG is more reliable on segmenting general texts, but less reliable on segmenting technical texts, in the ideal case we should adopt prior information provided by PKUSEG adaptively when processing texts of different types. Unfortunately, TopWORDS-Seg does not take such a natural idea into consideration yet, and simply use the PKUSEG prior at the same intensity everywhere. These deficiencies partially explain why TopWORDS-Seg still misses about 15% technical terms in both experimental studies reported in this paper. More research efforts are needed to fill in these gaps in future. This research is partially supported by the National Scientific and Technological Innovation 2030 Major Project (No: 2020AAA0106501), the Guo Qiang Institute of Tsinghua University, the Beijing Natural Science Foundation (Z190021), and the Scientific-Technological Innovation Plan Program of Universities guided by the Ministry of Education of China. Changzai Pan is supported by China Scholarship Council.
abstain abstain objective abstain abstain abstain abstain abstain abstain other abstain abstain objective abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain method abstain abstain other abstain abstain abstain abstain abstain other abstain method method abstain abstain abstain method abstain method other abstain abstain abstain other abstain abstain abstain method abstain method abstain method other abstain abstain abstain method abstain abstain method method method abstain abstain abstain other abstain abstain abstain abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain other result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain objective abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain other other