How much data do they use to train the embeddings?
11,529,432 segmented words and 20,402 characters
Recent studies have consistently given positive hints that morphology is helpful in enriching word embeddings. In this paper, we argue that Chinese word embeddings can be substantially enriched by the morphological information hidden in characters which is reflected not only in strokes order sequentially, but also in character glyphs spatially. Then, we propose a novel Dual-channel Word Embedding (DWE) model to realize the joint learning of sequential and spatial information of characters. Through the evaluation on both word similarity and word analogy tasks, our model shows its rationality and superiority in modelling the morphology of Chinese.
Word embeddings are fixed-length vector representations for words BIBREF0 , BIBREF1 . In recent years, the morphology of words is drawing more and more attention BIBREF2 , especially for Chinese whose writing system is based on logograms. UTF8gbsn With the gradual exploration of the semantic features of Chinese, scholars have found that not only words and characters are important semantic carriers, but also stroke feature of Chinese characters is crucial for inferring semantics BIBREF3 . Actually, a Chinese word usually consists of several characters, and each character can be further decomposed into a stroke sequence which is certain and changeless, and this kind of stroke sequence is very similar to the construction of English words. In Chinese, a particular sequence of strokes can reflect the inherent semantics. As shown in the upper half of Figure FIGREF3 , the Chinese character “驾" (drive) can be decomposed into a sequence of eight strokes, where the last three strokes together correspond to a root character “马" (horse) similar to the root “clar" of English word “declare" and “clarify". Moreover, Chinese is a language originated from Oracle Bone Inscriptions (a kind of hieroglyphics). Its character glyphs have a spatial structure similar to graphs which can convey abundant semantics BIBREF4 . Additionally, the critical reason why Chinese characters are so rich in morphological information is that they are composed of basic strokes in a 2-D spatial order. However, different spatial configurations of strokes may lead to different semantics. As shown in the lower half of Figure 1, three Chinese characters “入" (enter), “八" (eight) and “人" (man) share exactly a common stroke sequence, but they have completely different semantics because of their different spatial configurations. In addition, some biological investigations have confirmed that there are actually two processing channels for Chinese language. Specifically, Chinese readers not only activate the left brain which is a dominant hemisphere in processing alphabetic languages BIBREF5 , BIBREF6 , BIBREF7 , but also activate the areas of the right brain that are responsible for image processing and spatial information at the same time BIBREF8 . Therefore, we argue that the morphological information of characters in Chinese consists of two parts, i.e., the sequential information hidden in root-like strokes order, and the spatial information hidden in graph-like character glyphs. Along this line, we propose a novel Dual-channel Word Embedding (DWE) model for Chinese to realize the joint learning of sequential and spatial information in characters. Finally, we evaluate DWE on two representative tasks, where the experimental results exactly validate the superiority of DWE in capturing the morphological information of Chinese.