Investigation of Sentiment Controllable Chatbot

Conventional seq2seq chatbot models attempt only to find sentences with the highest probabilities conditioned on the input sequences, without considering the sentiment of the output sentences. In this paper, we investigate four models to scale or adjust the sentiment of the chatbot response: a persona-based model, reinforcement learning, a plug and play model, and CycleGAN, all based on the seq2seq model. We also develop machine-evaluated metrics to estimate whether the responses are reasonable given the input. These metrics, together with human evaluation, are used to analyze the performance of the four models in terms of different aspects; reinforcement learning and CycleGAN are shown to be very attractive.

I. INTRODUCTION
In contrast to goal-oriented dialogue systems [1], [2], chatbot chats with human users on any subject domain of daily life [3], [4]. The conventional chatbot is based on the seq2seq model [5], generating meaningful responses given user input. It is usually emotionless, which is a major limitation of modern chatbots as emotion plays a critical role in human social interaction, especially in chatting [6]. Hence we seek to train the chatbot to generate responses with scalable sentiment by setting the chat mode. For example, for the input "How was your day today?", the chatbot may respond, "It is wonderful today" or "It is terrible today" depending on the sentiment set, in addition to simply generating a reasonable response. This mode can either be set by the developer or the user, or determined dynamically based on the dialogue context. The techniques mentioned here may be extended to conversational style adjustment, so the machine may imitate the conversational style of someone the user is familiar with, to make the chatbot more friendly or more personal [7], [8].
Substantial effort has been focused on the conversational fluency and content quality of generated responses, for example, by enriching the content diversity [9]- [11], considering additional information [12], and addressing unknown words [13], [14]. Responses have also been generated with controllable factors. The sentiment of a given sentence can be modified using non-parallel data [15]. A chatbot can change the style of responses by optimizing a given sentiment-related function [16]. However, little work has been reported on scaling the sentiment of a chatbot; it remains difficult to evaluate a chatbot with adjustable sentiment properly [17], [18].
This work was financially supported by the Ministry of Science and Technology of Taiwan.
In this paper, we investigate four approaches to scale the sentiment of chatbot responses and use a set of evaluation metrics and human evaluation with which we analyze the approaches. This journal paper is an extension of a conference paper [19], but with additional results on two more corpora.

II. RELATED WORK
The approaches presented in Sections III-B, III-C, III-D and III-E are related to Sections II-A1, II-A2, II-B1 and II-B2, respectively.

A. Controllable Sentence Generation
Sentence generators based on deep learning show promising text generation capabilities, but cannot easily control the generated text. Hence, a series of research aims at controlling the generated sentences, for example, the writing styles or the topics of the generated sentences.
1) Controlled by Input Factors: Sentence generation models can take some factors as input to influence the generation of its outputs. Conditioned recurrent neural networks (CRNN) are used to control the linguistic style of generated text [20]. Affect-LM customizes the degree of emotional content in generated sentences through an additional design parameter [21]. The conditional transformer language model (CTRL) [22] trains to condition on control codes that govern style, content, and task-specific behavior.
This category of approach has been used in dialogue generation. The persona model [12] encodes personas in distributed embeddings that capture individual characteristics such as background information and speaking style, and the embeddings influence the output of the decoder. Instead of encoding personas, in Section III-B, the persona-based model is used to control sentiment. A conversational model is proposed to generate informative responses with controlled sentence function, for example, interrogative, imperative, declarative, etc [23]. This paper is closely related to the Emotional Chatting Machine (ECM) [24]. ECM is a neural conversational model that can generate corresponding responses based on given emotional categories. The basic idea of ECM is similar to the persona-based model, but with more sophisticated network architectures, including internal and external memories. However, it needs the dialogues involving emotional responses to train the model, which is not always available.
2) Controlled by External Function: In this approach, the sentence generation model is explicitly taught to generate sentences with certain aspects [25]- [28]. A hand-crafted or machine-learned function guide the sentence generator to output sentences considered having the desired aspect (like sentiment or topic) based on the function. This approach has been used to make the dialogue generation model generate emotional responses, but hand-crafted functions are used in the previous work [25]. It has been shown that the attribute models learned from data can successfully guide the sentence generator [26]. Because here we focus on dialogue generation, in Section III-C, besides considering the attribute, or sentiment, of the responses, we further guide the model to generate the responses coherent with the input sentences by coherence models.

B. Text Style Transfer
The text style transfer model transfers the input sentence from one style into another. The text style transfer approaches that do not utilize parallel data are used in Sections III-D and III-E. Below are two main categories of approaches to achieve text style transfer without parallel data. All the related work mentioned below only focuses on text style transfer, not dialogue generation as in Section III.
1) Manipulating Latent Space: The latent representations of auto-encoders can be manipulated to induce a change in the output space to achieve text style transfer [16], [29]. The approach used in Section III-D belongs to this category. One way to manipulate latent space to achieve text style transfer is feature disentangle. By separating the content from the style in the latent space, we can modify the style without changing the content [15], [30]- [32].
2) Direct Modification: Instead of manipulating the latent space, this category of approach directly finds a model that can transform the text from one style to another. The approach used in Section III-E belongs to this category. A simple approach is to delete phrases associated with the sentence's style and retrieve new phrases to replace them [33]. The idea similar to CycleGAN [34] or StarGAN [35], which has widely used in image style transfer, has also been used. This category of approaches uses a discriminator to control the style of the generated content [36], [37], and reconstruction loss to maintain the content [38].

III. SENTIMENT CONTROLLABLE CHATBOT
In Section III-A we briefly review the conventional seq2seq chatbot. The four approaches used here are presented in Sections III-B to III-E. All use the seq2seq chatbot as the basic model. The persona-based approach (Section III-B) and reinforcement learning (Section III-C) modify the training algorithm of the conventional seq2seq chatbot. Plug and play (Section III-D) and CycleGAN (Section III-E) modify the response of an off-the-shelf seq2seq chatbot. Below we assume that the chatbot response is to be positive conditioned on the input, although it is simple to generalize the approaches to scalable sentiment.

A. Seq2seq Model
Here we use the attention-based seq2seq model [39] shown in Figure 1 to train a simple chatbot using a corpus of dialogue pairs. In all discussions here, x is the input sentence to the seq2seq chatbot, and y is the output of the seq2seq model.ŷ is the reference response in the training corpus. In the training phase, we input the sentence x (a sequence of one-hot vectors) to the encoder, and the seq2seq model learns to maximize the probability of generating the sentenceŷ given x.  The persona-based model was originally proposed to generate sentences that mimic the responses of specific speakers [12]. It is very similar to the seq2seq model, except that extra information is added to the input of the decoder at each time step. In the original work [12], this extra information is the trained speaker embedding. Here we replace the speaker embedding with a sentiment score (a scalar between 0 and 1) from a sentiment classifier, as shown in Figure 2. This sentiment classifier [40] is trained on a corpus of sentences with labeled sentiments to determine whether a sentence is positive or not. The input of the classifier is a sentence z, and the output is a score SC(z) between 0 and 1 indicating how positive the input is. The input of the decoder at every time step is then the concatenation of the word embedding and a sentiment score. During training the sentiment score of the reference sentence SC(ŷ) is used, and the decoder learns to generate the reference sentence. For testing given the same input, we scale the sentiment of the output by entering the desired sentiment score.

C. Reinforcement Learning
Here we use exactly the seq2seq chatbot shown in Figure 1; the only modification is a set of reward functions designed to scale the response sentiment using reinforcement learning. The components of the reward functions are developed as follows.
1) Semantic Coherence 1: In addition to being a good sentence, the response y should be semantically relevant to the input x. Hence we pre-train a different seq2seq model on a large dialogue corpus to estimate this semantic coherence with a probability P coh (y|x). The first reward is therefore where x and y denote the input and response of the baseline seq2seq chatbot (not the pre-trained seq2seq model), and N y is the length of y for normalization. 2) Semantic Coherence 2: The semantic coherence mentioned above can be estimated in a completely different way. We use the same dialogue corpus to train a RNN discriminator, in which two RNN encoders are used to represent the input x and its corresponding response y as two embeddings; these two embeddings are concatenated and followed by a fully connected layer to produce a score D RNN (x, y) between 0 and 1 which indicates whether x and y are good dialogue pairs. This score is therefore the second reward: 3) Sentiment Score: The third reward is based on the sentiment classifier mentioned in Section III-B: where y is the seq2seq chatbot response. The total reward is then the linear interpolation of the three rewards mentioned above: where α and β are hyper-parameters ranging from 0 to 1 and α + β < 1. We employ the reinforcement learning algorithm with policy gradient [41]. As shown in Figure 3, to generate dialogue responses here, we borrow the concept of plug and play as used in generating images [42]. Here we pre-train a variational recurrent autoencoder (VRAE) [43] in addition to using the same dialogue corpus. The VRAE encoder on the left transforms a sentence y into a fixed-length latent vector h 0 , while the VRAE decoder on the middle right generates a sentence y based on a vector h . The VRAE encoder and decoder are also jointly learned from the dialogue corpus for the chatbot.

D. Plug and Play Model
The following steps take place on-line, when the user enters a sentence. Given an input x, the seq2seq baseline first generates a response y which is then encoded into a latent code h 0 by the VRAE encoder. Then the latent code h 0 is modified into h , based on the following equation: (5) where SC denotes the sentiment classifier and γ and δ are the weights of the loss function term and the regularization term. The first term on the right-hand side of Eq. (5) indicates that we seek a code h such that when decoded into a sentence Decoder(h) using the VRAE decoder, the resulting sentiment score SC(Decoder(h)) is maximized. The second term of Eq. (5) prevents the code h from drifting too far from h 0 . To solve Eq. (5), we calculate the gradient of the sentiment score with respect to the latent code h and apply gradient ascent to the latent code iteratively, until the sentiment score output reaches a pre-defined value. Because Eq. (5) is solved on-line after the user enters an input sentence, this approach is more time consuming. Since the argmax layer between the decoder and sentiment classifier in SC(Decoder(h)) is nondifferentiable, we use soft argmax [44] to approximate argmax so that the gradient back-propagates throughout the whole network, from the sentiment classifier to the decoder. Here we adopt the very powerful cycle generative adversarial network (CycleGAN), which proved successful in image style transformation even without paired data [34]. As illustrated in Figure 4, we show a way to use CycleGAN to transform the sentiment of sentences from negative to positive. The model is trained on two sets of sentences in a corpus with labeled sentiments: a positive sentiment set P and a negative sentiment set N . The sentences in the two sets are unpaired; that is, for a given sentence in P, it is not known which is the corresponding sentence in N . We train two seq2seq translators: G to transform a negative sentence y n to positive and F for positive y p to negative. We also train discriminators D P and D N . They take a sequence of word embeddings as input and learn to distinguish whether the sequence is from word embeddings of a real sentence, or it was generated by G or F . With the continuous word embeddings as the translator output, the gradient can be back-propagated from the discriminator to the translator. Note that F and G transform sequences of word embeddings to sequences of word embeddings. We pretrain the word embedding model with Word2Vec [45]; here it is fixed during CycleGAN training. To transform the output sequence of word embeddings into a sentence, we simply select those words whose embeddings have the highest cosinesimilarity to each given word embedding in the sequence.

E. CycleGAN
The concept of W-GAN [46] is used to train D P and D N . The loss function of the discriminator D P is where y n is a negative sentence sampled from N , and G(y n ) is the output of translator G taking y n as the input. D P learns to minimize Eq. (6), that is, to give low scores to the translated output (the first term on the right) and high scores to real positive sentences y p (the second term). The loss function of the discriminators D N is parallel to Eq. (6): As in improved W-GAN, gradient penalty is applied here. The loss functions for training translators G and F are − D P (G(y n )).
(9) The first terms on the right-hand side of Eqs. (8) and (9) are the same. Given a positive sentence y p , after being transformed into a negative sentence by F and then transformed back to positive by G, it should be very close to the original sentence y p ; likewise for the second terms. The last terms of Eqs. (8) and (9) are different: F learns to generate output F (y p ) that is considered by D N to be a real negative sentence, whereas G learns to generate output G(y n ) that is considered by D P to be a real positive sentence. In this way translators F and G learn to transform the sentences from one sentiment (positive or negative) to the other. Notice that the discriminators D P and D N are jointly trained with the translators F and G. During testing, for any chatbot output y, we simply use G to transform it into a positive sentence G(y).

IV. EXPERIMENTAL SETUP
We trained and tested all our models, including the seq2seq and the four proposed models, on the following three corpora. The first two are in Chinese, whereas the third is in English. Using the training set, we trained five models, including the seq2seq baseline and the four proposed models; we evaluated these models using the testing set. All the evaluation metrics obtained are the average over the testing data. More dataset details are provided in Appendix A.
1) We used the Chinese Emotional Conversation Generation (CECG) task [47], originally offered by the NII Testbeds and Community for Information Access Research (NT-CIR) Project for the Short Text Conversation Task (STC) competition. CECG contains around 1.7M dialogue pairs; each sentence is labeled by one of the following six kinds of sentiments: like, sad, disgusted, angry, happy, and other. We reclassified five of these six sentiment categories into positive and negative categories: like and happy as positive sentiments, and sad, disgusted, and angry as negative sentiments. Both corpora were split into training and testing sets (the latter included 1k dialogue pairs). This corpus was used in both the sentiment classifier and the other models. 2) We collected data from the PTT Boy-Girl board containing the titles and all the article replies from page 1 to page 4000, and we used the articles as context and the replies as responses. Replies include "like" and "boo", which roughly correspond to positive and negative sentiment. However, as this dataset contains no true dialogue data, and the sentiment is not always precise, we use this dataset only for demonstration and not the main experiment. As with the previous dataset, this corpus was also used in both the sentiment classifier and other models.

3) For English, the Twitter chatting corpus is available on
Marsan-Ma's GitHub repository [48] using TensorFlow. This corpus, which contains 3.7M dialogue pairs, is split into training and testing sets, the latter of which includes 28k dialogue pairs. The sentiment classifier used in this work was trained from the Twitter Sentiment Analysis Corpus [49], which consists of 15M data with labeled sentiment (0 or 1). This corpus was also split into a training and testing set. The trained sentiment classifier achieved an 87% accuracy on the validation set.

A. Evaluation Metrics
Evaluation is always difficult in language generation; this is even more so for chatbots. Here we propose two metrics: semantic coherence 1 and 2 (COH1, COH2) for chatbots, which are scores reflecting the degree to which the output sentence y is a proper response to the input sentence x. These are in fact the semantic coherence 1 and 2 mentioned in Section III-C (Reinforcement Learning) designed for the reward function. However, the seq2seq model and the RNN discriminator used to obtain these two scores are re-trained here and are thus different models.
The third metric is the sentiment classifier score (SCL) used to measure how positive the output sentence is. This is the sentiment classifier score SC(y) used in the persona-based model mentioned in Section III-B. Likewise, the sentiment classifier used here is re-trained and is thus different.
The fourth metric is the language model score (LM) which measures whether the output sentence y is a good sentence in terms of a language model [50]. The language model used here is composed of a two-layer GRU [51] model: which is the language model probability P (y) for a sentence y, normalized with the sentence length N y . Eq. (10) is also known as the negative log perplexity (PPL). Note that SCL and LM, the third and fourth metrics, consider only the output sentence y -not the input x. COH1 and COH2, the first and second metrics, however, consider the output y given the input x. In all the following tables, larger evaluation metrics represent better performances.

B. Metric Models
The coherence score (1 and 2) in Chinese uses the PTT Gossiping board dialogue corpus collected by Justin Yang [52], which contains about 400,000 dialogue pairs. Here we split the data into training and testing data sets; the latter contains 1,000 pairs, whereas the other owns the rest.
For the LM score, in Chinese, we crawled the replies of articles from the top 50 most popular PTT boards, for a total of 25 million tokens of data. In English, the corpus of the One Billion Word Benchmark [53] was used.
The coherence score for English and sentiment score for both Chinese and English share the same data with the experiments mentioned in Section IV. Although the metrics are trained with the same corpus, as mentioned above, the models of the metrics are different because of re-training, which guarantees a certain extent of fairness.

VI. INDIVIDUAL MODELS
Due to the space limitation, we only show the results of the individual models on Chinese CECG dataset. The details of the hyper-parameter setup in the following experiments are shown in Appendix B.

A. Sentiment Classifier
To find a proper sentiment classifier, we evaluated six different ways of segmenting Chinese words. Including word-based and character-based methods, we also evaluated different neural network architectures: a CNN with max pooling (CNN for short), a GRU with the last hidden state output (GRU-last), and a GRU averaging all hidden state outputs for the whole sequence (GRU-avg). The total number of characters used in character-based segmentation was 7,297, and the number of words in word-based segmentation was 50,000. We trained these six models with a batch size of 32 and 50,000 epochs. We evaluated the performance using the accuracy and area under the Receiver Operating Characteristics curve (AUC). For the accuracy score, the output scores of the models greater than 0.5 were taken as predictions of positive sentiment; otherwise, they were negative. The predictions were then compared to the real answers to calculate the accuracy rate. The best results of all architectures shown in Table I, GRU-last with word segmentation yielded the best performance on both accuracy and AUC scores. GRU-last is then applied in the following experiments as the sentiment classifier.

B. Persona-Based Model
With a fixed sentiment score input of 1.0, we first compared the performance between the 64-, 128-, and 256-neuron seq2seq models of the persona-based model during the inference step. As shown in Table II  Further, we applied other sentiment scores as input to generate sentences with different degrees of sentiment. Table III shows the results of 0.0, 0.5, and 1.0 as inputs respectively on the 256-neuron model during inference. As expected, the lower we set the input, the lower the resultant sentiment scores: this suggests the model is already transforming sentences to different sentiments.

C. Reinforcement Learning
In this experiment, different coefficient reward combinations were adopted to test the performance of the model. Below we report the results after 2500 training epochs.
In the first three sets of experiments, we fixed R 2 as 0.0 and adjusted the proportion of R 1 and R 3 to determine how the four metrics were affected. The results in Table V show that the COH1 score increases as R 1 rises from 0.0 to 0.8, and the SCL score falls as R 3 decreases from 1.0 to 0.2, which is not surprising as rewards R 1 , R 2 , and R 3 in Eqs. (1), (2), and (3) were in parallel with the COH1, COH2, and SCL score respectively. Interestingly, COH2 degrades compared to the pretrained MLE baseline, showing that the RL model is highly

D. Plug and Play
VAE is used for sentence generation in the plug and play model, and KL cost annealing and vocabulary truncation are applied to improve VAE performance. In this experiment, we first compared the performance between models with and without KL cost annealing. These two models truncate vocabulary randomly at a probability of 0.3. In Table VI    In Table VIII, we see that the 0.3 output successfully reconstructs the original input. The 0.0 output also seems reasonable, although it fails to reconstruct the original sentence: it replaces "禮貌" ("courtesy") with "動作" ("action"), probably because it depends more on the decoder language model. With 0.7 truncation, the model outputs an incorrect sentence, replacing "提 高" ("improve") with "以" ("by"); this could be because the many unknown words hinder the model from learning correct sentence grammar. The model with KL cost annealing and with a vocabulary truncation of 0.3 outperforms all others, so we will use this setting in the following experiments.  Tables IX apply sentiment gradient ascent and descent respectively. Both positive and negative sentiment transfers generate corresponding sentiment outputs. However, most sentiment transfers replace words with strong sentiment bias. For example, in Table IX, "哈哈" ("haha") is added after a positive transfer (sentiment gradient ascent), and "淘汰" ("eliminate") is added after a negative transfer (sentiment gradient descent); although each of these words is a strong indicator of sentiment style, it leads to incorrect grammar.

E. CycleGAN
When training the CycleGAN model, heterogeneous input styles are preferred; thus the data used here is re-marked and selected by the sentiment classifier, leaving only those sentences with strong style bias. In the CycleGAN model training phase, training alternates between the generator and discriminator. Once one of the agents (generator or discriminator) is trained, the other's parameters are fixed and used for inference. If one is better than the other, a training imbalance occurs, and the process is no longer "adversarial". Since the discriminator is easier to train in most cases, the discriminator is stronger than the generator generally, leading to an awkward generator. We evaluate two kinds of generators: generators G and F, created after the generator and the discriminator are trained alternatively using different training epochs. Furthermore, we also tried to add Identity Loss (short for ID loss), which was used to help to keep the content after transferring, in total loss during training the generator. The result is illustrated in Tables X and XI: the training epoch ratios of the generator and discriminator are 1:1, 1:3, and 3:1 after 10000 iterations and 100000 iterations respectively.
The 1:1 result is most stable. For the 1:3 model, although it achieves good performance at 10000 iterations, the model's generator and discriminator both collapse at 100000 iterations; this may be because the generator is unable to deceive the robust discriminator after several iterations of training; repeated training in this case leads to collapse. Although the 3:1 model is relatively stable, its generator is not as good as that of the 1:1 model. Moreover, the experiment also shows if there is an improvement when the ID loss is added: the result shows no significant difference from when only generator loss is used as the total loss.
The 1:1 model with 100000 training iterations is the best model; there will be a comparison between this model and other trained models later. Tables XII contain examples of  sentences transformed by generators G and F. In the left table  of Table XII, as the un-transformed output is already a positive response, generator G has little effect on the result; generator F, however, produces an identifiable result with its clearly negative sentence. In the right table of Table XII, the original output is a negative response; generator G substitutes "討厭" ("hate") for "喜歡" ("like"), making a more positive sentence. Generator F makes little modification to the sentence.

A. Chinese CECG Dataset
Table XIII evaluates the four models using four metrics. The results better than the baselines are in blue, and the ranking of each method is also in the table. The personabased model (using sentiment score equal to 1.0 as input during inference) yields the highest SCL. The RL model yields the best performance on COH1 and LM. The plug and play and CycleGAN models transform the original output sentences directly, resulting in similar SCL scores, but worse than persona-based model and RL model. CycleGAN has better LM score than plug and play, which shows that CycleGAN does better with respect to grammar.

B. Chinese PTT Dataset
In this section, we switched to the PTT dataset to demonstrate the performance of the above four models. To train a sentiment classifier, we also tried CNN, GRU-last, and GRUavg on different word segmentation methods at first. However, as the dirty PTT dataset resulted in poor sentiment classifier performance, we improved the sentiment classifier using the following three steps. First, we compared the performance between CNN, GRU-last, and GRU-avg, and chose GRU-last because of its out-performance (see Table XIV). Second, we picked the more credible data (higher absolute scores) by GRU-last, shrinking the data from 1,402,303 to 1,149,000. Finally, we trained a new sentiment classifier on the remaining data using GRU-last-2. This final sentiment classifier was then used in the four models. GRU-last-2 here indicates the same architecture as GRU-last but not the same model.
The evaluation results on the PTT dataset are shown in Table XV. SCL scores are all high in these four models. It shows that all the four approaches can successfully generate positive responses, but persona-based and RL models are better than plug and play and Cycle GAN. Although the SCL score of persona-based model is the highest, it obtained the worst COH1 and COH2, which shows that its generated responses are not coherent with respect to the inputs. Similar to that on the CECG dataset, the RL model yields the best results in terms of COH1 and LM.

C. English Corpus
The results are listed in Table XVI. For the persona-based model, its SCL score is the highest; however, its COH1, COH2, and LM are the lowest. The RL model performed better than all other models in three out of the four metrics: COH1, COH2, and LM, but not the SCL score. The SCL scores of plug and play model and CylcleGAN are worse than personabased and RL models.

D. Examples
Tables XVII list outputs of the four models on Chinese CECG dataset. The persona-based and RL models usually output reasonable and positive sentences. From the data we also observed both the persona-based and RL models made complicated changes to output sentences; this was rarely seen in other models. The plug and play model tends to generate collapsed sentences. The CycleGAN model typically adjusts sentences by replacing negative words with positive ones; this is not always a significant transformation.

VIII. HUMAN EVALUATION
We performed a subjective human evaluation against sentences generated by five models each on three corpora (CECG, PTT, and English), for a total of 15 results with 30 subjects, all of whom were graduate students. They were asked to answer three questions about the output sentences: (1) Coherence: Is the output sentence a good response to the input? (2) Sentiment: Is the output sentence positive? (3) Grammar: Is the output sentence grammatically correct? They were asked to give scores ranging from 0 to 5, based on a few reference examples with given scores 1, 3, 5 to scale the scores. The average results (normalized to the range from 0 to 1) of three models are listed in Tables XVIII, XIX, and XX. The results better than the baselines are in blue, and the ranking of each method is also in the tables.
For coherence, all the approaches are worse than the baselines, except RL models on Chinese PTT dataset. Except for the second rank on the English corpus, the RL models worked the best among the four approaches in terms of coherence.
For sentiment, all the models can successfully modify the sentiment of responses, except plug and play models, which performed the worst on every corpus. For grammar, the RL models yielded the best performance on each corpus.

IX. DISCUSSION
The persona-based model and RL model can both generate the responses largely different from the original seq2seq model. For the persona-based model, its sentiment score was the highest on all the datasets in terms of both machine and human evaluation (the only exception is the human evaluation on Chinese CECG). However, its coherence and grammar were both worse than the RL model in terms of both machine and human evaluation. This shows that although personabased model can successfully generate very positive responses, the coherence and the grammar of the responses are poor. It tries to output sentences that carry the correct sentiment, but not necessarily relevant given the input. The RL model is generally the best model of the three corpora from all aspects. This is because the reward R 1 and R 2 in Eqs. (1) and (2) were in parallel with coherence, and R 1 in Eq. (1) also takes into account word ordering which leads to correct grammar. Its sentiment score was also high (although not as high as the persona-based model) because its reward R 3 is also in parallel with the sentiment, which yielded positive output. One may argue that the RL model overfits the machine-evaluation metrics since it learns to optimize those metrics. This issue is addressed in Section VIII through human evaluation. Humans also consider that the RL model has reasonable responses with correct grammar.
The plug and play model and CycleGAN only transform the output responses of an off-the-shelf seq2seq model. The plug and play model attempted to modify the latent code of the sentences. As the sentiment classifier primarily considered sentiment without really encoding sentence grammar, when maximizing the sentiment classifier's output, it sometimes transforms the original output of the seq2seq model into collapsed sentences. For CycleGAN, since the two translators directly outputted word embeddings carrying both sentiment and semantics, the models found mappings between words like "bad" to "good", "sorry" to "thank", "can't" to "can". However, this entailed only changes or deletions of specific words and not complex modifications of whole sentences. Since it only changes a few words of the original responses, its responses are not far from the original ones. This explains why the grammar of plug and play is worse than CycleGAN in terms of both machine and human evaluation in all cases. When it comes to sentiment and coherence, humans always    consider CycleGAN is better than plug and play, whereas their performances on COH1, COH2, and SCL are comparable. Because it is difficult for humans to read the sentences with poor grammar, humans usually consider that the sentiments of the collapsed sentences are incorrect, and they are less coherent with the inputs.

X. CONCLUSION
In this paper, we attempted to adjust the sentiment of the chatbot response given the input. We investigated four different models for the tasks; All of the models are based on the conventional seq2seq model. The performances of the four models in terms of machine-evaluated metrics and human evaluation are reported. The persona-based model and RL models, which alter the original seq2seq model parameters, yield good results. The persona-based model is good at exporting sentences of high sentiment score that might be suitable for cases when a chatbot only needs to reply with simple sentences that carry strong sentiment. The RL

APPENDIX A DATASET STATISTICS
Below we present the details of the datasets used in different tasks. In the sentiment classifiers (Section A), there are three corpora: Chinese, PTT, and English. As described in Section VII-B, the corpus used in the PTT task was refined by a special filtering processing which reduced the size from 1,402K to 1,149K. In the experiments (Section B), each of the three corpora are shared by four tasks -the persona-based model, the reinforcement learning model, the plug and play model, and the CycleGAN model. In the metrics (Sections C and D), there are for metrics for the Chinese, PTT, and English tasks. The Chinese and PTT tasks share the same metrics trained by the corpora described in Section C. The WMT11 corpus used for the LM in Section D used the data preprocessing described in [53]. For word-based segmentation, low-frequency words were eliminated to reduce the word dimensions, which led to the vocabulary size shown in the table, which was smaller than the actual size.


Annotation Interoperability for the Post-ISOCat Era

With this paper, we provide an overview over ISOCat successor solutions and annotation standardization efforts since 2010, and we describe the low-cost harmonization of post-ISOCat vocabularies by means of modular, linked ontologies: The CLARIN Concept Registry, LexInfo, Universal Parts of Speech, Universal Dependencies and UniMorph are linked with the Ontologies of Linguistic Annotation and through it with ISOCat, the GOLD ontology, the Typological Database Systems ontology and a large number of annotation schemes.

Background
The divergence and heterogeneity of linguistic annotations even for comparable language resources for the same language variety has long been recognized as a key problem in the advancement of human language technology. The creation of language resources and tools is a laboursome and cost-intense process, but if these cannot be easily combined with each other, be it to feed the output of one annotator (say, a POS tagger) as input to another (say, a parser), or to increase the amount of available training data by combining multiple corpora of the same kind (say, morphology, part of speech annotations, dependency syntax or phrase structure syntax), potential synergies between those annotation efforts cannot be exploited and progress in the field is thus unnecessarily delayed. Very often, this is the situation for low-resource languages, where no standard annotation for a particular phenomenon has been established yet, but different researchers develop multiple schemas independently from each other. With ISOCat, 1 the ISO TC37 Data Category Registry (DCR), a possible solution to this problem has been proposed in the 2000s, a central repository of linguistic terminology managed in the context of a community process with a relatively low entry barrier, where individual researchers or institutions would just register their resourceor application-specific terminology. ISOCat focused on the elementary level of linguistic terms, and deliberately excluded the relations between these terms, but eventually, it was imagined that these relations could be added in a separate relation registry (Schuurman and Windhouwer, 2011), and the process to develop these relations and terms would converge towards a consistent terminological inventory whose content would be the basis for a subsequent standardization process within ISO TC 37. In the standardization approach, terminology harmonization is achieved by aggregation and consolidation across all possible user communities within a centralized, monolithic repository. An alternative approach emerging simultaneously to ISOCat was the idea of distributed terminology harmonization by creating links among independent, selfcontained domain terminologies and between them and one 1 http://www.ISOCat.org/ or multiple 'upper models', especially by means of ontologies and Semantic Web technologies: 2 Different user communities formalize and provide their respective terminology in a stand-alone, self-contained ontology, and these ontologies are subsequently linked with each other by means of designated relations (e.g., rdfs:subClassOf, skos:broader, owl:equivalentClass, or owl:sameAs) between identical or near-identical concepts. The general idea is probably best described in the title of a seminal paper by Dimitriadis et al. (2009): "How to integrate databases without starting a typology war". The fundamental insight is that so far, any approach to develop or to enforce standard terminology in linguistics was rejected by the majority of the scientific community, and given the degree of specialization in various branches of linguistics, this is unlikely to change. The General Ontology of Linguistic Description (Farrar and Langendoen, 2010, GOLD), currently hosted at Lin-guistList, is a similar effort to formalize reference terminology as an ontology for the field of language documentation and descriptive linguistics. The authors of GOLD see its role as providing a lingua franca as a basis for annotation projects to map their data categories to in order to foster conceptual interoperability. In natural language processing and corpus linguistics, a similar concept was implemented with the Ontologies of Linguistic Annotation (Chiarcos and Sukhareva, 2015, OLiA). Historically, OLiA originated as an OWL formalization of the EAGLES recommendations (Leech and Wilson, 1996), extended by a linking with GOLD and the morphosyntactic and syntactic profiles of ISOCat as well as definitions and introduced by various annotation schemes it was applied to. OLiA owes its continued relevance to its application beyond its original use case: It has been conceived as a meta-vocabulary for tagset documentation and cross-resource corpus querying, but with the emergence of the Linguistic Linked Open Data cloud since 2010 (Chiarcos et al., 2012a), 3 it evolved to become the primary vocab-ulary to formalize linguistic annotations for Semantic Web applications and language resources in the web of data. A fundamental insight of the 2000s was that the development and usability of widely used, shared annotation terminology for linguistic annotations must be based on web technologies, and in particular, resolvable URIs. The technical standard in this regard still remains to be ISOCat, which provides persistent URIs, even though these are redirected to a static dump now rather than the underlying live system. Since Chiarcos et al. (2012b), linked data has been recognized as a key element to facilitate language resource interoperability, and this trend intensified with the more recent trend to shift from XML tecnologies of JSON(-LD) in the language resource community. Accordingly, RDF dumps of resources developed on the basis of proprietary formalisms have been made available at an increasing rate. Along with the improvement in structural (format and access) interoperability, also the content of all major terminology resources became increasingly harmonized. OLiA already provideded an indirect linking between ISOCat and GOLD, but a direct bridge between both resources was established and GOLD and ISOCat began to converge when the 2010 edition of GOLD was mirrored within ISOCat (Kemps-Snijders, 2010). However, this process, as well as the addition of a large number of tagsets and domain vocabularies, contributed to the emergence of terminological (near-)doublets. Without relational data structures to express identity or near-identity, or an effective community process to eliminate such doublets, the ISOCat repository came to house an increasing number of duplicate and near-duplicate records where even elementary concepts existed multiple times, distinguished by their respective owners and the wording of their definitions, but not by their label (e.g., 'part of speech' as DC-1345, DC-3747 and DC-5294;'verb' as DC-1424 and DC-4949;'dative case' as DC-1265 and DC-3148;'past' as DC-1347 and DC-4966;'masculine gender' as DC-1883 and DC-3312). Among the linguistic terminology resources around 2010, ISOCat excelled as being the most widely used, most fundamental, and richest. Unfortunately, its popularity and continued growth ultimately led to its abandonment, as the unrestricted addition of new records, together with very weak facilities for expressing the relations between entries, produced a largely unstructured collection of redundant entries and near-synonyms. 4 In the original conception of ISOCat, a community process was envisioned to consolidate duplicate entries and to formulate consensus definitions. For the specific case of ISOCat, however, this process, however, never produced any concrete results as it remained hard to motivate researchers to engage in abstract work such as consolidating terms and definitions at this scale. External extensions of ISOCat, however, involved the development of full-fledged domain ontologies that were grounded in ISOCat profiles. This includes an ontology for lexical data structures (LexInfo, see below), an 4 ISOCat provided an inventory of hierarchical relations (dcif:isA), but these were optional and not enforced. Out of 682 data categories in the morphosyntactic profile, only one third (274) used hierarchical relations, whereas the majority was provided as an unstructured list. ontology for linguistic annotations (Chiarcos, 2010) and an ontology of language resource metadata (Zinn et al., 2012). Neither of these ontologies, however, were adopted by ISO-Cat administrators nor considered as a possible input to the ISOCat community process or its future development. To some extent, this was due to the technological choices made in ISOCat architecture which was designed on the basis of an application-specific, XML-based format, well en par with the state of the art in early 2000's language resource technology, but orthogonal to RDF technology. But despite its influence in numerous branches of research, ISOCat failed in general to deliver on its promises and was eventually discontinued in 2014 (Schuurman et al., 2015).

Linguistic Annotation Terminology since 2010
ISOCat content remains available as a static resource only http://ISOCat.tbxinfo.net/, and two direct successor systems are being developed -along with other more recent efforts to harmonize linguistic data categories. Whereas ISOCat was relatively widely used, these efforts target more specific communities and use cases. These include terminologies that directly build on ISOCat, esp., the CLARIN Concept Registry (Sect. 2.1.), developed as a central component of the CLARIN infrastructure, and LexInfo (Sect. 2.2.), a vocabulary for the terminology of lexicalconceptual resources. We distinguish these terminology repositories from other recent standardization efforts (Sect. 2.3.) developed independently from ISOCat by communities that aim to create cross-linguistically compatible annotations such as syntax, morphosyntax and inflectional morphology.

CLARIN Concept Registry and DatCatWeb
As a replacement for ISOCat, Schuurman et al. (2015) introduced the CLARIN Concept Registry (CCR), 5 which is based on OpenSKOS (Brugman and Lindeman, 2012). They aimed to avoid the issues with ISOCat by allowing only CLARIN National Content coordinators to update the registry, and by requiring a "good definition" of a concept that is unique, meaningful, reusable and concise. However, progress on this effort is slow, and even at the time of writing this, basic concepts such as 'part-of-speech' have not reached the 'approved' status. Simultaneously, ISO TC37 has been developing DatCatInfo as an ISOCat successor registry initially populated with ISOCat concepts (Warburton and Wright, 2020). 6 DatCat-Info is developed in close connection with the TermBase eXchange format (TBX). As of early 2019, 2,977 data categories (approximately half the DCs from ISOCat) have been migrated to DatCatInfo, and are currently undergoing continued revision in order to eliminate duplicates and establish a coherent view on the terminology. The future division of labour between the CCR and Dat-CatInfo is not clear, although they clearly diverge and we may anticipate a specialization of the CCR for applications in language technology and a specialization of DatCatInfo for lexical and terminological resources. In any case, both systems will provide resolvable URIs (at the time of writing, only CCR does), and for domain-specific vocabularies, it will be possible to link them to each of them. In fact, the capability to facilitate linking with multiple external reference models has been the motivation for modular architectures such as OLiA (see below).

LexInfo
LexInfo 7 is the representative vocabulary for linguistic data categories for lexical-conceptual resources in the context of Linguistic Linked Open Data, especially because of its intrinsic ties with the popular OntoLex-Lemon vocabulary (Cimiano et al., 2016). Originally, it was designed as an ontology for "associat[ing] linguistic information with respect to any level of linguistic description and expressivity to elements in an ontology" (Cimiano et al., 2011). In this function, it predates Ontolex-Lemon, but with LexInfo v.2.0, it was re-designed to serve as a terminology backend of OntoLex-Lemon with the goal of making Ontolex-lemon itself agnostic of any linguistic category system. The Lex-Info ontology developed out of an RDF edition of the Lexical Markup Framework (Francopoulo et al., 2006, LMF), i.e., a major source of ISOCat concepts, so that LexInfo is largely compatible with ISOCat. LexInfo provides an axiomatized set of linguistic categories, covering areas such as part of speech, tense, number, animacy, degree, mood, register, etc. These categories are largely derived from ISO-Cat, but LexInfo provides a stronger axiomatization and a coherent global organization.
LexInfo v.2.0 is the reference vocabulary for linguistic categories and features in lexical-conceptual resources in the web of data. Since December 2019, version 3.0 is in preparation, 8 with the goal to increase its cross-linguistic applicability and its compability with OntoLex-Lemon, a novel aspect here is that this development is conducted in the style of an open source project in order to facilitate the participation of the wider community.

Other Recent Standardization Efforts
The Universal Dependencies (Nivre et al., 2016, UD) 9 aim to provide cross-linguistically applicable annotations for dependency syntax, parts of speech and morphosyntactic features. UD differs from earlier standardization efforts in that it is coupled with the creation and the release of open source corpora with the corresponding annotations. The Universal Dependencies have been embraced enthusiastically by the NLP and language resource communities due to coincidence with an increased interest in the syntactic annotation of low-resource languages, and so far, more thann 100 treebanks in over 70 languages are being provided. Inspired by the wide success of UD, similar efforts have been undertaken for other areas of application, e.g., with UniMorph (Sylak-Glassman et al., ) 10 for morphology.
With ISOCat development stalled, and new standardization initiatives emerging, we see a great risk in increasing fractionalization in language resource development. With this paper, we provide a linking for the existing terminology repositories for linguistic data categories. We do not address language resource metadata, for which we refer the interested reader to a parallel effort, the development of METASHARE-OWL (McCrae et al., 2015), initiated at the 1st Summer Datathon on LLOD (SD-LLOD 2015), and currently continued in the context of the European Language Grid (ELG) and the Linked Data for Language Technology (LD4LT) Community Group of the W3C. 11

Approach
With the goal of linking multiple, and increasingly fractionalizing vocabularies for linguistic data categories, we focus both on vocabularies grounded in ISOCat, i.e., LexInfo, CCR and OLiA, as well as on novel vocabularies for morphosyntactic and syntactic annotation developed independently, i.e., the Universal Dependencies vocabularies (Universal Parts of Speech, UD v.1, UD v.2) and UniMorph. Instead of mapping or linking each of them with every other, we use the OLiA Reference Model as an intermediate layer and thereby link them with older vocabularies that OLiA is grounded in, i.e., GOLD and ISOCat. We further describe how this linking can be used to derive mapping tables that can be used to facilitate interoperability in NLP applications and transformation tasks.

Ontologies of Linguistic Annotation
The Ontologies of Linguistic Annotation (Chiarcos, 2008;Chiarcos, 2010, OLiA) have been designed as a mediator between various terminology repositories on the one hand and linguistically annotated resources (more precisely, their annotation schemes), on the other hand (Schmidt et al., 2006). OLiA applies linked data principles to leverage several distributed terminology repositories: It provides the formalization of the mapping from annotations via the OLiA Reference Model to several existing terminology repositories ('External Reference Models') by means of a modular architecture of interdependent OWL2/DL ontologies: Annotation models and (external) reference models each constitute self-contained, standalone ontologies, whereas the linking between them is a physically separated ontology that imports the respective models and asserts rdfs:subClassOf (rdfs:subPropertyOf) relations about their concepts (and properties). The OLiA ontologies are available from http://purl. org/olia under a Creative Commons Attribution license (CC-BY), and they are developed as an open source project using GitHub. 12 Four different types of ontologies are distinguished ( Fig. 1): (1) The OLiA Reference Model is an OWL ontology that specifies the common terminology that different annotation schemes can refer to.
(2) Multiple OLiA Annotation Models formalize annotation schemes and tagsets. Fig. 1 illustrates this with an annotation model developed as part of the Korean NLP2RDF stack (Hahm et al., 2012).
(3) For every The OLiA ontologies cover different grammatical phenomena, including inflectional morphology, word classes, phrase and edge labels of different syntax annotations, as well as extensions for coreference, discourse relations, discourse structure and information structure (Chiarcos, 2014). Annotations for lexical semantics are only covered to the extent that they are found in syntactic and morphosyntactic annotation schemes.

Universal Part of Speech Tags (UP)
In order to facilitate annotation projection on the basis of parallel text, Petrov et al. (2012) introduced a highly reductionist, but 'universal' (i.e., cross-linguistically applicable, not universal in the sense of linguistic theory) representation of morphosyntactic annotation, based on a mapping of various annotation schemes to a minimal set of about a dozen part-of-speech tags. We integrate his information in OLiA by directly interpreting UP concepts as OLiA concepts and converting the mapping files from their original TSV format to OLiA annotation and linking models, using the following interpretation: The generated UP models are integrated into the stable re-lease of OLiA in a separate directory http://purl. org/olia/up. On the one hand, this reflects their original source, but more importantly, the number of links that these linking models provide is much shallower than that of regular OLiA annotation models which aim to capture the full semantics of every tag. We thus distinguish regular linking models from UP linking models. Also, note that we did not include the UP linking models for annotation schemes previously covered by OLiA. Note that UP tags are provided without definitions, there is thus no way to evaluate in a language-independent manner whether this mapping is correct. However, OLiA design encapsulates annotation models, so that OLiA users are free to replace an existing linking model with their own, e.g., by copying the linking model provided and adjusting it according to their specifications.

UD v.1
The Universal Dependencies comprise 'universal' part of speech annotations (originating in UP), morphosyntactic features (optional, originating from Zeman 2008, and, indirectly, from EAGLES) and syntactic dependencies (originating from the Stanford dependencies, De Marneffe and Manning 2008). UD data and documentation are available from https://universaldependencies.org/ and have been subject to a long (and on-going) process of refinement that led to two iterations of the UD vocabulary (UD v.1 and UD v.2). In relation to OLiA, which aims to cover all types of linguistic annotation, we see the UD community as working on a much more restricted problem, and on a limited (albeit impressive) scale of linguistic phenomena. We thus integrate UD specifications in the same way as an annotation model. The UD ontology is, however, not compiled into a single file, but we provide three annotation models (one for parts of speech, morphosyntactic features, and dependency labels each), and their respective linking to OLiA. In these models, every 'universal' category (tag, feature value, dependency) is represented by a class, with its original documentation as an rdfs:comment, every language-specific category (tag, feature value, dependency) is represented as an instance of the corresponding class, again with its original documentation. Using an ontology browser such as Protégé 13 , it is thus possible to quickly navigate from language-specific definitions to universal definitions, or to explore the range (and definitions) of language-specific instances of a universal category. All elements in the RDF graphs are identified by URIs that resolve to the respective page in the documentation, we thus complement the human-readable website with a conceptual layer that captures both its textual and structural content. Fig. 2 shows an example of the UD v.1 ontology for parts of speech.
Our UD ontologies are automatically derived from the documentation which resides in GitHub and is generated from Markdown using Jekyll. 14 We adopt the original URI Figure 2: UD v.1 sub-ontology for parts of speech as visualized by Protégé schema of the UD documentation, such that our URIs resolve to the original web page. For UD v.1, this was initially implemented using an experimental workflow that uses Jekyll templates to generate an RDF representation of the UD guidelines as part of the HTML rendering process: 15 Aside from mapping user-provided variables to specific HTML elements, we embedded RDF triples using the RDF in attributes specification (Herman et al., 2015, RDFa) to be able to read off an RDF representation directly from the website. As this RDF representation was generated from the original source, it was automatically synchronized with every modification of the website, and included all descriptive and all formal elements of the original website. As UD v.1 development is stalled, the UD v.1 specifications are now provided in a static fashion, we provide another RDF converter that now processes the generated HTML content, instead of the underlying Markdown. For these ontologies, linking models have been created manually, where UD concepts were defined as subclasses of OLiA concepts. We only link universal categories for parts of speech, morphosyntactic features and dependency labels, language-specific extensions are not linked on an individual level, resp., only captured via their anchoring in universal categories. The linking of parts of speech is largely equivalent with UP linking, the linking of dependency labels was based on the existing linking for Stanford dependencies. Neither of these required the introduction of novel OLiA concepts. As for morphosyntactic features, the situation is similar, as OLiA is partially based on the specifications of EAGLES which model. For a related piece of work, we would like to refer to Passos (2018) who provides a manually constructed and richer ontology for UD -which is, however, no longer synchronized with the ongoing improvement of UD documentation. 15 This prototype is available via http://fginter.
had a strong influence on the Interset inventory (Zeman, 2008) that represents the basis for UD feature annotations. The UD v.1 ontologies are included with OLiA 16 and, together with the build scripts, accessible from the OLiA GitHub repository.

UD v.2 and UniMorph
The revised UD v.2 vocabulary aims for establishing a greater level of coherence among the different layers of UD annotation and between different languages. It has been converted and linked analoguously to UD v.1 documentation, it should be noted, however, that the UD v.2 documentation is far less homogeneous than UD v.1 documentation, and that much language-specific information still refers to UD v.1. We provide the UD v.2 vocabularies and their linking with OLiA, 17 and the build scripts via the OLiA GitHub repository. The Universal Morphology (UniMorph) project is a recent community effort aiming to complement the Universal Dependencies and their focus on syntax with coverage of inflectional morphology. UniMorph provides inflection tables for 110 languages using a TSV format, with lemma, form, and morphological features. Compatibility with UD is a requirement of the UniMorph community that has been partially achieved only (McCarthy et al., 2018), but by reference to a common reference vocabulary the relation between both vocabularies can be expressed easily. To this end, we created e a machine-readable representation of the UniMorph vocabulary in OWL and its linking with OLiA, closely following the approach taken for UD, 18 with build scripts included in the OLiA GitHub repository.

LexInfo
We focus on LexInfo v. 2.0, as this is closely coupled with the highly popular OntoLex-Lemon vocabulary and the reference vocabulary for lexical data categories in the Linguistic Linked Open Data cloud community. Unlike UP, UD or UniMorph, LexInfo already comes as an ontology. We thus focus on its linking rather than its modelling choices. LexInfo is complementary to OLiA in the sense that OLiA takes a focus on annotations and the processing of natural language, whereas LexInfo provides an inventory of formal data categories for linguistic features of lexical entries and related information in dictionaries, wordnets and multilingual ontologies. So far, both ontologies have not been put into relation, although OLiA has been applied for encoding features of lexical resources, as well (Eckle-Kohler et al., 2015). Creating an interlinking between LexInfo and OLiA, and, via OLiA, with Uni-Morph and UD thus comes with the prospect of enormous synergies between lexical resources and natural language processing as well as for the enrichment of lexical resources and morphological resources, cf. Declerck and Racioppa (2019). Similar to OLiA, LexInfo is partially based on ISO-Cat, but it differs from the OLiA Reference Model in that it provides individuals rather than classes. Accordingly, it is not possible to establish formal equivalence relations (owl:equivalentClass), to assert identity (owl:sameAs) or to use the conventional OLiA linking properties (rdfs:subClassOf). Instead, LexInfo terms can only be defined as instances of OLiA concepts, so that Lex-Info is linked with OLiA in the style of an OLiA annotation model, 19 i.e., a specialization for the domain of lexical information.
We provide a manual linking for LexInfo 2.0 with OLiA. 20 The linking is facilitated by the fact that LexInfo is based on ISOCat. No extensions of OLiA were necessary to represent LexInfo concepts. It is to be noted, however, that terminology relevant to the internal structure of lexical resources rather than their grammatical characteristics (another concern of the LMF model that which LexInfo developed from) have not been added to the OLiA Reference Model, but only information that could be potentially found in linguistic annotation.

CLARIN Concept Registry
Within the CLARIN infrastructure, the CLARIN Concept Registry (CCR) serves to provide semantically interoperable annotation terminology for various services and resources. It provides a collection of concepts, identifiable by persistent identifiers, that are relevant for the domain of language resources. At its core, the CLARIN CCR provides a revised subsection of ISOCat concept in SKOS, although under different URIs (handles). 21 The actual RDF data is not publicly available, but can be recovered from the HTML rendering of the CCR browser.
We provide a script that retrieves a partial RDF/Turtle representation for the morphosyntactic and syntactic facets of the CCR. In total, the morphosyntactic and syntactic facets of the CCR provide 484 terms, out of which 4 have been approved, 3 have been expired, and 477 (98.6%) remain at candidate status. CCR terms are generally more (or, at least as) abstract than OLiA Reference Model terms. By analogy with the existing ISOCat linking (Chiarcos, 2010), we integrate CCR as an external reference model, i.e., to provide a linking that defines OLiA concepts as subclasses of (or equivalent classes with) CCR concepts. For technical reason, however, this is not possible with the native CCR data model: 19 In this context, and given that other types of linguistic terminology resources have been linked in this way with the OLiA Reference Model (Dimitrova et al., 2016), it would be more adequate to use the term "domain model" rather than "annotation model": LexInfo is not concerned with annotations, but addresses domainspecific information for the domains of lexical and terminological resources (dictionaries, glossaries, word nets, terminologies, multilingual ontologies), where this information is actual content rather than an annotation attached to a content element. However, this comes with other connotations, so that we stay with the conventional term. 20 http://purl.org/olia/external/lexinfo 21 Via its public interface, the CLARIN CCR does not provide direct links to ISOCat concepts, but encodes them as comments in skos:changeNote literals. By means of regular expressions, the partial SKOS data, together with these ISOCat links have been retrieved.
The RDF data drawn from the CCR portal natively represents terms as instances of skos:Concepts. In this form, CCR concepts are OWL instances (not classes) and can thus not serve as superclasses of OLiA concepts. Aside from parsing RDF triples out of the HTML, we thus perform OWL conversion as a post-processing step by replacing every skos:Concept with rdfs:Class, skos:broader with rdfs:subClassOf, etc., and add an OWL header. The original URIs are preserved and resolve to HTML pages in the CCR portal. After this conversion, OLiA concepts are manually be defined as subconcepts of the CCR ontology using rdfs:subClassOf in the linking model. The original extracted CCR data, its ontological formalization and its linking is provided in the external branch of OLiA, 22 the crawler and conversion scripts are accessible via the OLiA GitHub repository.

Interoperability in Practice: Mapping Annotations
So far, we described the conversion and the linking of five post-ISOCat sources of linguistic reference terminology with OLiA, and thereby, with each other, with older resources such as ISOCat, GOLD, and the Typological Database System (TDS) ontology and a large number of annotation schemes. Each of these vocabularies is associated with a significant number of language resources that adhere to its specifications, and as a result, it is now possible to map annotations, resp., linguistic features of lexical resources from one vocabulary onto another. It is to be noted, however, that such a mapping is not necessarily lossless: For a particular annotation scheme, linking with OLiA provides cross-linguistically applicable, intensionally defined concepts to formalize the meaning of tags. But OLiA does not necessarily capture resource-, tagset-or language-specific constraints that may apply to a particular tagset. This includes lexeme-specific tags, such as TO in the Penn Treebank tagset (for to in all its uses). In the linking, this is modelled as a subclass of olia:Preposition or (owl:unionOf) olia:Unique ('particle'), etc. While the possible functions of the elements tagged with TO are correctly captured in this way, we lose the information that the tag requires the presence of a particular word. Another aspect where we encounter possible information loss between annotation models and OLiA are implicit constraints imposed by the underlying data structure of tagsets. Usually, this is a list or a tree, but in either case, the categories they posit are extensionally disjoint (in order to enable unambiguous tagging). In an ontology, they can overlap. If the underlying categories overlap, tagsets usually define what tag to be used. An attributive possessive pronoun like her in her garden is both a determiner (syntactically) and a pronoun (morphologically and semantically).
If a tagset has a special tag for attributive possessive pronouns, it can be linked to the intersection of both classes (multiple inheritance), but if a tagset has only tags for pronoun and determiner (e.g., UD PRON and DET), a choice has to be made by the annotators or the tagset designers. In UD, for example, attributive possessive pronouns should be tagged DET -even for languages without grammaticalized determiners. By linking DET with olia:Determiner, an attributive possessive pronoun in English would be correctly represented as determiner, but the annotation does not provide the information that it would be a pronoun, as well. When using OLiA to map from an annotation model that treats attributive possessive pronouns as pronouns to UD, we would not be able to predict the DET tag. When using OLiA for an annotation model that treats attributive possessive pronouns as a distinct class, OLiA would preserve the information that her is in both categories, but it would not be able to disambiguate the choice between DET or PRON in UD as a target tagset, because it is unaware of tagsetspecific disambiguation rules. As far as cross-linguistically applicable categories and features are concerned, we assume that an OLiA encoding for a particular tagset is lossless with respect to intensional semantics (if a particular category does not exist, it can be created), but that it can lose information about extensional restrictions. This is adequately expressed in the requirement to use of rdfs:subClassOf for the linking of annotation model concepts and the OLiA Reference Model. But this also means that the mapping from OLiA to an annotation model may be noisy, because extensional restrictions and disambiguation rules are not available. The situation is different between OLiA and external reference models, e.g., concepts of CCR, GOLD, TDS or ISO-Cat, as we expect these to exist on the same or a higher level of abstraction as OLiA concepts. With the current linking, we do thus provide a lossless mapping from Lex-Info, UP, UD, UniMorph and other annotation models via OLiA to CCR, GOLD and ISOCat. The mapping from CCR, GOLD, ISOCat or OLiA to UD and other annotation models, however, be lossy. The overall relations of the vocabularies described here here are summarized in Fig. 3. For the reasons detailed above, all of them were integrated into the overall architecture of OLiA by means of rdfs:subClassOf (or, rdf:type, i.e., instance) relations. For UP, UD, and UniMorph, which are resource-or application-specific vocabularies, this is semantically adequate. LexInfo does provide a similar degree of abstraction as the OLiA Reference Model, but here, this modelling is required for technical reasons, i.e., that Lex-Info individuals cannot be formalized as equivalent or superclasses of the OWL classes provided by the OLiA Reference Model. The situation is different for the CCR, which can be linked as an external reference model (that provides superclasses for OLiA concepts) after a conversion from SKOS to OWL. As for using rdfs:subClassOf relations to retrieve upper classes ('upward search'), our modelling assumes that this is lossless (if the linking is correct), as for using rdfs:subClassOf relations retrieve subclasses ('downward search'), our modelling assumes that this can be lossy (even if the linking is correct, we may miss extensional constraints). With these vocabularies being linked, it is now possible to retrieve, say, the ISOCat concept for a given UniMorph annotation, say, the feature DAT (dative case) by upward search: In the UniMorph ontology, "DAT" is the value of the UniMorph label for unimorph:DAT, an instance of uni- This example shows an upward search, but analogously, search can be performed from ISOCat down to a specific annotation model, or from one annotation to another. At the moment, OLiA provides linkings with 94 annotation models for morphosyntax and/or syntax, applicable to more than 100 languages, the terminology repositories described here cover hundreds of language resources, as well, by linking them via the OLiA Reference Model, it is now possible to map each of these vocabularies to every other. As the example shows, knowledge graphs, semantic technologies and ontologies are expressive and powerful devices, but they also come with considerable technological overhead for a relatively simple practical problem, the mapping of tags from one representation to another. More than the formal treatment of interoperability, this is of interest to the language resource community. Using SPARQL SELECT, mapping tables can be easily generated with the queries above, as every variable binding of the SELECT statement will result in a row in a TSV, XML or JSON table, with one column per return variable. It is to be emphasized, however, that such mappings are not necessarily lossless or that they result in unambiguous results.
A suitable heuristic to eliminate redundant or too generic results is to restrict the results to elements on the shortest paths between two vocabulary elements in a SPARQL property path. In SPARQL, path length calculation can be implemented with aggregates: SELECT ?x ?z (COUNT(?y) AS ?length) WHERE { ?x rdfs:subClassOf * ?y. ?y rdfs:subClassOf+ ?z } Using a subsequent filtering step, mappings between two vocabularies can be restricted to pairs with minimal distance for every element of the source vocabulary.

Discussion and Outlook
In this paper, we describe the formalization of several linguistic annotation vocabularies by means of ontologies, and their respective linking with each other and existing terminology repositories by using the OLiA Reference Model as an intermediate representation. We provide the corresponding ontologies, resp. with their linking models, together with the Ontologies of Linguistic Annotations under a CC-BY license. Grounding annotation schemes in formal ontologies, and interlinking them with each other establishes a high degree of interoperability at a comparably low cost. In particular, this approach differs from full-fledged standardization in that it does not require revisions of the actual annotation, but can be solely performed at the level of the vocabularies themselves. From linked and formalized vocabularies, mapping tables that can be generated from the linking by means of SPARQL SELECT statements. As these involve queries over several RDF graphs, these are comparably complex. They do not, however, have to be developed by the end user. Instead, they can be statically compiled from a pair of vocabularies, and published along with software or data they are to be applied to. In general, the generated mapping rules will be approximative in the sense that they can be reductionistic (similar to the mappings provided by UP, the target annotations can be more coarse-grained) or underspecified (if the target vocabulary provides a higher degree of granularity than the source vocabulary, all alternative tags will be listed).
While it is not possible to guarantee 1:1 correspondences in mapping tables generated from such data (they might not even exist), this is nevertheless an efficient approach as it allows to retrieve approximative mapping rules for the transformation of annotations in accordance with dozens or even hundreds of language resources. With the linking of UP, UD, UniMorph, LexInfo, and the CCR, this functionality is now available for the most influential vocabularies for linguistic annotation terminology in the post-ISOCat era. This paper is a resource description, in the sense that mappings between various vocabularies are being provided. At the same time, it employs linkings with the OLiA Reference Model as a basis for establishing a shared semantic space between them. While this seems to bring OLiA into a similar position as ISOCat and GOLD previously had (and for which they failed), we would like to emphasize that the point we are trying to make is something different: Using HTTP(S)-resolvable URIs for identifying concepts, semantically typed relations that hold between them (i.e., RDF properties), and the technical means to access remote data sets (i.e., RDF federation), it is possible to harmonize existing, distributed vocabularies. This is an insight that was already underling the development of the GOLD ontology. But in addition to that, we do not rely on centralized, monolithic repositories for data categories, application-specific interfaces and protocols, and formal means for concept registration and standardization by means of a designated commission or an editorial board as adopted by standardizationbased approaches in the early 2000s. Instead, OLiA (like UD and, since January 2020, LexInfo) adopts a lean, software-inspired development workflow with a low entry barrier to its contributors: The vocabulary is maintained as an open source project and available via an open platform (GitHub). Privileged users (administrators) exist and they serve a similar role as the editorial board of ISOCat and GOLD, but anyone can fork a copy, modify it according to his needs -and request to merge his changes or additions back into the main branch. 24 Our main contribution is thus to demonstrate the application of open source development principles and linked data technology to adress annotation interoperability challenges in a distributed setting: Concepts and definitions of different providers are defined in self-contained formal models (annotation models, terminology repositories) and can subsequently refer to vocabularies or reference concepts developed by a broader, and open community with a low entry barrier.


s2fieldsofstudy:
Computer Science
Physics

text:
No scalar hair behaviors of static massive scalar fields with nodes

We study no scalar field hair behavior for spherically symmetric objects in the scalar-Gauss–Bonnet gravity. In this work, we focus on static massive scalar fields with nodes. We analytically obtain a bound on the coupling parameter. Below the bound, the static massive scalar field with nodes cannot exist outside the object. In particular, our conclusion is independent of surface boundary conditions.

The mostly studied scalar configurations are the cases with no nodes. Theoretically, scalar configurations can possess nodes. It is well known that, in the general case, scalar configurations with nodes are usually unstable [46][47][48][49]. So it is natural to conjecture that scalar configurations with nodes may finally evolve into the more stable nodeless solutions. However, it is still meaningful to study the solution with nodes. Firstly, the solution with nodes may be sufficiently stable, which means that the perturbation growth time is extremely large [50]. In this case, the unstable node solution stays for a long time and can be observed from physical aspects. And secondly, in some gravity models, it is very surprising that the solution with nodes seems to be the endpoint of the tachyonic instability [51]. a e-mail: yanpengphy@163.com (corresponding author) In particular, for massless scalar fields non-minimally coupled to the Gauss-Bonnet invariant in the background of black holes, an interesting relation n = √η n+1 − √η n = √ 3 2 π for n → ∞ was obtained through WKB approach in [52] and this relation is also precisely supported by numerical data in [27], where n is the number of nodes. For massive scalar fields in black hole spacetimes, with both analytical and numerical methods, we demonstrated that this relation still holds in the large node number limit [53]. On the other side, some quantum-gravity theories suggest that quantum effects may prevent the formation of classical black hole absorbing horizons and the horizonless compact star may serve as an alternative [54][55][56][57][58][59][60][61]. So it is also interesting to search for properties independent of surface boundary conditions. In this work, we plan to study the existence (or nonexistence) of scalar fields with nodes outside general spherically symmetric objects.
In the following, we shall consider a gravity system with a scalar field coupled to the Gauss-Bonnet invariant in the background of general spherically symmetric objects. With analytical methods, we obtain a bound on the scalar-Gauss-Bonnet coupling parameter, below which there is no hair behavior for static massive scalar fields with nodes. We summarize main results in the last section.

Bounds on the scalar-Gauss-Bonnet coupling parameter
We consider a gravity with a scalar field coupled to the Gauss-Bonnet invariant R 2 G B . The general Lagrange density can be written as [26][27][28][29][30][31] Here m is the scalar field mass and the Gauss-Bonnet invariant is defined as In the probe limit, there is R 2 G B = 48M 2 r 6 . The function f ( ) describes the nontrivial coupling between scalar fields and the Gauss-Bonnet invariant. In the linearized regime, one can generally put the coupling function in the quadratic form f ( ) = η 2 with η as the model parameter [28][29][30].
The exterior metric solution of spherically symmetric objects is [29] In the probe limit, the metric is given by the function g(r ) = 1 − 2M r , where M is the mass of the objects. We define r s ≥ 2M as the object surface radii. When r s = 2M, the metric is a black hole.
The scalar field differential equation is We choose to study a static massive scalar field in the form Here l is the spherical harmonic index and l(l + 1) is the characteristic eigenvalue of the angular scalar eigenfunction S lm (θ ) [27,62,63]. For simplicity, we label R lm (r ) as ψ(r ).
One can obtain the scalar field equation where g = 1 − 2M r and R 2 G B = 48M 2 r 6 . In this work, we focus on scalar fields with nodes. That is to say, there is at least a point r 0 satisfying ψ(r 0 ) = 0. The bound-state massive static scalar fields satisfy asymptotically decaying behaviors ψ(r → ∞) ∼ 1 r e −mr . In the range (r 0 , ∞), the scalar field boundary conditions are According to boundary conditions (6), the scalar field ψ(r ) must possess one extremum point r = r peak between the vanishing point r = r 0 and the infinity. It may be a positive maximum extremum point satisfying ψ(r peak ) > 0, ψ (r peak ) = 0, ψ (r peak ) ≤ 0, otherwise it will be a negative minimum extremum point with ψ(r peak ) < 0, ψ (r peak ) = 0, ψ (r peak ) ≥ 0.
Multiplying both sides of (5) with ψ, we obtain the new equation At the extremum point, relations (9) and (10) yield the inequality The extremum point is outside the gravitational radius r peak > r s ≥ 2M, which yields With relations (11) and (12), we obtain the inequality From relation (13), one deduces that The inequality (14) is equal to The extremum point is outside the object surface r peak > r s ≥ 2M. It yields the relation For scalar fields with nodes, we obtain a lower bound on the coupling parameter. Below this bound, the static scalar field with nodes cannot exist outside the object surface. It implies that static massive scalar fields with nodes usually cannot exist outside objects of large mass. Here we obtain a no hair behavior for scalar fields with nodes in the region In particular, for η = 0, (17) always holds. It means minimally coupled massive scalar fields always cannot exist outside general spherically symmetric objects, such as black holes and horizonless stars.

Conclusions
We studied no scalar field hair behavior for spherically symmetric objects in the scalar-Gauss-Bonnet gravity. We considered a static massive scalar field with nodes. Through ana-lytical methods, we obtained a bound on the scalar-Gauss-Bonnet coupling parameter as η ≤ 4 3 m 2 M 4 , where η is the coupling parameter, m is the scalar field mass and M is the mass of objects. Below the lower bound, static massive scalar fields with nodes cannot exist outside the objects. In particular, our analysis doesn't depend on the surface condition. So this no hair behavior for scalar fields with nodes is a very general property. Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecomm ons.org/licenses/by/4.0/. Funded by SCOAP 3 .