We hope our empirical study may potentially allow others to design better attention mechanisms given their particular applications.

for SP, removing PE from our proposed attention variant dramatically degrades the performance (from 24.28 to 30.92).


The standard cross-entropy loss function is implemented, and the thresholds of all the classes are set as 0.5.

(4) Compared with OntoNotes and BBN datasets, the FIGER show a relatively less improvement when applying these policies.


We found no evidence that attention patterns that are mappable onto core frame-semantic relations actually improve BERT's performance.

we pose the following research questions: 1.


large number of such relations could be investigated, we chose to examine semantic role relations defined in frame semantics, since they can be viewed as being at the intersection of syntax and semantics.

two heads (middle) demonstrate their ability to capture semantic relations.


In this experiment, our goal was to understand whether different syntactic and semantic relations are captured by self-attention patterns.

effects of this operation vary across tasks, and for QNLI and MNLI, it produces a performance drop of up to -0.2%.


During training, it takes one sentence as well as the cross-sentence (document-level) information as input and predicts the ground-truth translation sentence in other languages.

our approach contains two key components: mining implicitly aligned sentence pairs and aligning topic distributions.


While the "no supervision at all" premise behind fully unsupervised CLWE methods is indeed seductive, our study strongly suggests that future research efforts should revisit the main motivation behind these methods and focus on designing even more robust solutions, given their current inability to support a wide spectrum of language pairs.

7 Experiments with other monolingual vectors such as the   vocabularies to the 200K most frequent words.


The work of Goran Glavaš is supported by the Baden-Württemberg Stiftung (AGREE grant of the Eliteprogramm).

the first attempts at fully unsupervised CLWE induction failed exactly for these use cases, as shown by .


In addition, we investigate two different ways to capture relation and attribute features.

we construct count-based Nhot vectors X r and X a for these two aspects of features, respectively, where the (i, j) entry is the count of the j-th relation (attribute) for the corresponding entity e i .


Feeding the next target token assumes that we know it in advance and thus calls for separate translation and alignment models.

we train the layer average baseline with a batch size of 7168 tokens on 64 Volta GPUs for 30k updates and apply a learning rate of 1e-3, β 1 = 0.9, β 2 = 0.98.


Crowdsourced using Concept-Net (Speer and Havasi, 2012), these questions mostly probe knowledge related to factual and physical commonsense (e.g., "Where would I not want a fox?").

8 In order to ensure even higher quality, we validate the dev and test data a second time with five workers.


The second column of Table 4 shows that our NMN outperforms the baseline significantly (+10 points in EM score) on the adversarial evaluation, suggesting that our NMN is indeed learning stronger compositional reasoning skills compared to the BiDAF baseline.

these results demonstrate the contribution of each module toward achieving a self-assembling modular network with the strong overall performance.


We further train both models on the adversarial training set, and the results are shown in the last two columns of Table 4.

for multi-hop QA, directly matching the semantics of the question and context leads to the entity that bridges the two supporting facts (e.g., "Shirley Temple"), or the entities that need to be compared against each other (e.g., nationalities of Scott and Ed).


CompTreeNN Premise and hypothesis are processed as a single aligned tree, following the structure of the composition tree in Figure 3.

these methods can provide powerful insights, but the issue of fairness looms large.


In computer vision, it is common to adversarially train on artificially noisy examples to create a more robust model (Goodfellow et al., 2015;Szegedy et al., 2014).

let Dom be a map on N T that assigns a set to each node, called the domain of the node.


These methods have the advantage that the complexity of individual examples can be precisely characterized without reference to the models being evaluated.

the essence of natural logic reasoning is recursive composition up a tree structure where the premise and hypothesis are composed jointly, so this bottleneck proves extremely problematic.


This information (especially π ik ) is not present in any existing KB.

this information (especially π ik ) is not present in any existing KB.


Both of XPAD's new biases encourage XPAD to make state-change predictions that result in more enables edges, either by counting them (g edge ) or summing their likelihoods (g kb ), resulting in some overlap in their overall effect (Table 6.2).

the g edge and g kb scores together help XPAD discover meaningful action dependency links.


Our work is also related to the domain of question answering and reasoning in knowledge graphs (Das et al., 2018;Xiong et al., 2018;Hamilton et al., 2018;Xiong et al., 2017;Welbl et al., 2018;Kartsaklis et al., 2018), where either the model is provided with a knowledge graph to perform inference over or where the model must infer a knowledge graph from the text itself.

our benchmark suite-termed CLUTRR (Compositional Language Understanding and Text-based Relational Reasoning)-contains a large set of semi-synthetic stories involving hypothetical families.


In particular, we collected paraphrases for stories containing k = 1, 2, 3 supporting facts and then replaced the entities from these collected stories with placeholders in order to re-use them to generate longer semi-synthetic stories.

we highlight some key diagnostic capabilities available via different variations of CLUTRR below.


To get a sense of the data quality and difficulty involved in CLUTRR, we asked human annotators to solve the task for random examples of length k = 2, 3, ..., 6.

r contains the logical rules that govern all the generated stories in CLUTrr, while G contains the grounded facts that underlie a specific story.


The accuracy of machine learned speech recognizers (Hinton et al., 2012) and speech synthesizers (van den Oord et al., 2016) are good enough to be deployed in real-world products and this progress has been driven by publicly available labeled datasets.

in this case, users are asked to pretend they have a personal assistant who can help them take care of various tasks in real time.


ASSISTANT: I'm sorry, but it looks like the 4:10 and the 6:10 pm showings are sold out.

it is doubtful that new data-driven systems built with this type of corpus would show much improvement since they would be biased by the existing system and likely mimic its limitations (Williams and Young, 2007).


human conversation has a different distribution of understanding errors and exhibits turn-taking idiosyncrasies which may not be well suited for interaction with a dialog system (Williams and Young, 2007;Serban et al., 2017).


One set of instructions is for the "agents" and the other is for the "customers".

for example, "Please do this" cannot be interpreted without a broader context.


On the other hand, Chit-chat style dialogues without goals have been popular since ELIZA and have been investigated with neural techniques (Weizenbaum, 1966;Li et al., 2016Li et al., , 2017.

for transferring money, you will also need: last 4 digits of account to move from, last 4 digits of account to move to, and the sum of money to be transferred.


To our knowledge this is a novel collection strategy as we explicitly guide/prod the participants in a dialogue to engage in conversations with specific biases such as intent change, slot change, multi-intent, multiple slot values, slot overfilling and slot deletion.

chit-chat style dialogues without goals have been popular since ELIZA and have been investigated with neural techniques (Weizenbaum, 1966;Li et al., 2016Li et al., , 2017.


We refer to the new utterance as the complete version of the original utterance.

or sequence-based metrics (i.e., BLEU and EM).


DSTC 2 is a research challenge focused on improving the state-of-the-art in tracking the state of spoken dialogue systems.

each word can be transformed into vector by embedding lookup, and we sum up vectors in each tuple to be the input sequence of the context-aware memory.


• LSTM (Tang et al., 2016a) uses the last hidden state vector of LSTM to predict sentiment polarity.

for convenience, we denote the output of the l-th layer for node i as h l i , where and h L i is the final state of node i.


Trees Aiming to address the limitations of existing approaches (as discussed in previous sections), we leverage a graph convolutional network over dependency trees of sentences.

• Extensive experiment results verify the importance of leveraging syntactical information and long-range word dependencies, and demonstrate the effectiveness of our model in capturing and exploiting them in aspectbased sentiment classification.


In this evaluation, we report precision, recall, and F1 scores.

here f * (•) is a ReLU-activated neural perceptron described above.


unfortunately, these methods highly rely on prior knowledge (e.g., manually-designed rules) or external linguistic resources (e.g., dependency parsers), which are inflexible and prone to bringing in knowledge errors.


• AD-SAL: it advances the AD-AL by conducting selective adversarial learning.

our model can still outperform them by a large margin, which demonstrates the effectiveness of the proposed methods.


Our model can automatically model complicated relations among aspect and opinion words via the DMI as transferable knowledge.

we propose an empirically alternating strategy to train the L M +ρL O and L D iteratively, which separates the whole word representation learning into a discriminative stage and a domain-invariant stage.


Aspect level sentiment classification is a finegrained sentiment analysis task.

• We extend CAN to multi-task settings by introducing ACD as an auxiliary task, and applying CAN on both ALSC and ACD tasks.


The details will be introduced in Section 3.

aspect level sentiment classification (aLSC) is a fine-grained sentiment analysis task, which aims at detecting the sentiment towards a particular aspect in a sentence.


We apply a dropout of p = 0.7 after the embedding and LSTM layers.

85.23% and 83.73% of the multiaspect sentences are non-overlapping in Rest14 and Rest15, respectively.


In this paper, we introduce orthogonal regularization to constrain the attention weights of multiple non-overlapping aspects, as well as sparse regularization on each single aspect.

given the sentence S with K aspects, A s = {A s 1 , ..., A s K }, for each aspect A s k , its attention weights are calculated by: where u s k is the embedding of the aspect A s k , e L ∈ R L is a vector of 1s, u s k ⊗ e L is the op-eration repeatedly concatenating u s k for L times.


As expected, our weakly supervised approach does not outperform the fully supervised (*-Gold) models.

our student model is an embedding-based neural network: a segment is first embedded (h i = EMB(s i ) ∈ R d ) and then classified to the K aspects (p i = CLF(h i )) (see Section 2.1).


User-generated reviews can be decomposed into fine-grained segments (e.g., sentences, clauses), each evaluating a different aspect of the principal entity (e.g., price, quality, appearance).

the student is able to generalize better than the teacher and predict aspects even in segments that do not contain any seed words.


Our goal is to predictŷ u,c ∈ [0, 1], which measures how likely user u will engage in conversation c. Here to estimateŷ u,c , two types of information are encoded: replying history of users and interaction structure of conversations.

we also observe that both baseline models work poorly.


Other examples, an analysis of the US Presidential Election in 2016 (Allcott and Gentzkow, 2017) revealed that fake news was widely shared during the three months prior to the election with 30 million total Facebook shares of 115 known pro-Trump fake stories and 7.6 million of 41 known pro-Clinton fake stories.

one is that the number of training examples in RumourEval (including 5,568 tweets) is relatively limited as compared with PHEME (including 105,354 tweets), which is not enough to train deep neural networks.


A manual examination of examples in the mixed categories found two main types of cases.

such manual approaches often require an exhaustive list of linguistic cues, which costs a significant amount of human effort and its generalizability may be limited by the small sample size.


Table 2 shows there are fewer tweets targeting disability in Arabic compared to English and French and no tweets insulting people based on their sexual orientation which may be due to the fact that the labels of gender, gender identity, and sexual orientation use almost the same wording.

for deep learning based models, we run bidirectional LSTM (biLSTM) models with one hidden layer on each of the classification tasks.


One option would be to manually map those labels onto one another.

category III is for predicting veracity; they encourage retrieving evidence documents as part of their task description, but do not distribute them.


In this paper, we advance this line of research.

13 for the Stance dataset, N2V user representations are more informative.


Press media, one form of mass media, manifests itself in large, diachronic collections of newspaper articles; such corpora provide a promising avenue for studying public opinion and testing theories, provided scholars can be confident that the measures they obtain over time are substantively invariant (Davidov et al., 2014).

the core intuition behind consistency regularization is that ensembled predictions are more likely to be correct than single predictions (Laine and Aila, 2017;tarvainen and Valpola, 2017).


We show representations for both source domain samples (green) and target domain samples (blue).

the student network is updated via backpropagation, then the teacher network is updated with an exponential average of the student network's parameters (Tarvainen and Valpola, 2017).


We further use several layers of transformer encoders (Vaswani et al., 2017) to learn the correlation between different feature fields.

the matrix product between query qW Q i and key HW K i after softmax normalization is an attention weight that indicates important words among the projected value vectors HW V i .


In recent years, neural network based prediction methods have shown great success on this Twitter user geolocation prediction task (Rahimi et al., 2017;Miura et al., 2017).

with the feature fusion layer, our model can accommodate various feature combinations and achieves state-of-the-art results over three commonly used benchmarks under different feature settings.


We only get metadata for about 53% and 67% users in Twitter-US and Twitter-World respectively.

such private information is only accessible to internet service providers.


In order not to over-burden the workers, we filtered out conversations consisting of more than 20 comments.

(2017) is different from the other datasets as it investigates the behavior of hate-related users on Twitter, instead of evaluating hate-related tweets.


we examine the model using the simple leaveone-out method similar to Ribeiro et al.


2) we cannot guarantee that the resulting text is still a natural input.

as we expect, tokens strongly associated with aggression, such as " ", "dead" and many other curse words appear frequently as the most influential tokens.


Since the majority of tweets usually has rationale rank 0, the averaged rationale rank is small and the difference between each model is "diluted" by the prevailing zeros.

surprisingly, words such as "a" and "on" are frequently considered the most influential by the model as well.


Because of the lower quality, we omit these language-pairs from our experiments below.

the English-tagalog and English-Indonesian tasks yielded 99% precision, the English-Romanian and English-Greek tasks 87%, and the English-Russian task 85%.


Further, compared with MTL2 which uses a "parallel" architecture to make predictions for two tasks, our Hierarchical-PSV performs better than MTL2.

(1): where the multiplication operation expands the receptive field of a GCN layer, and adding an identity matrix elevates the importance of t i itself.


(2017) project the utterances and intent labels to a same semantic space and then compute the similarities between utterances and intent labels (Chen et al., 2016;Kumar et al., 2017).

for example, the word "book" has different meanings in the two utterances: "Book a restaurant in Michigan for 4 people" and "Give 4 out of 6 points to this book".


To improve business effectiveness and user satisfaction, accurately identifying the intents behind user ut-terances is indispensable.

to implement zero-shot intent classification more easily and intelligently, recent works rely more on the word embeddings of intent labels, which can be easily pretrained on text corpus.


13, the overall loss function of the proposed dimensional attention capsule network is: where β is a trade-off parameter.

it is extremely challenging not only because user queries are sometimes short and expressed diversely, but also because it may continuously encounter new or unacquainted intents popped up quickly from various domains.


For this purpose, we investigate domain adaptation for person-job fit, focusing on the text match between job postings and candidate resumes.

now, we study how such a mechanism work and why it is useful to improve the performance.


A simple transfer strategy is to share the whole matrix in both target and source domains.

for our deep match network, we also prepare three simple single-domain variants, include (1) Tgt-Only trains the model with only target domain data, (2) Src-Only trains the model with only source domain data, and (3) Mixed trains the model with a simple mixture of source and target domain data.


Due to space limit, we only select six sentences for each document.

such a method will make it less flexible to capture domain-specific match information, since the local match information is likely to be varied across domains.


For example,  designs a character-level CNN which alleviates the sparsity by mining different levels of information within the texts.

as shown in Table 3, we compare our HGaT with four variant models.


A possible explanation is that the Paragraph Vector model assigns a distinct vector to each of the news articles, thus inconsistency between news in different news browsing sequences is easily created by the sequence-level fine-tuning.

bERT (Devlin et al., 2018) proposes a powerful pre-trained sentence encoder which is trained on unsupervised datasets based on the masked language model and next sentence prediction.


Finally, we evaluate our LS system in an endto-end manner and compare its performance to that of the current state-of-the-art systems, including those reported in Paetzold and Specia (2016a) and the DRESS-LS system of Zhang and Lapata (2017).

we then compare the results to the SOTA simplification systems DRESS-LS by Zhang and Lapata (2017), which contains a specialised LS model, and the P&S simplification system (Paetzold and Specia, 2016a).


We also show that combining out-of-domain, out-of-genre Medical Literature data with out-of-domain EHRs can provide significant improvement over using just out-ofdomain EHRs at the section and sentence level, depending on training data size.

we also found that Past Medical History was often incorrectly labeled as Medications.


We follow this approach using our RNN as well as the BERT model where we first tune BERT to the MedLit data and then continue to tune that model on the EHR data.

it is common that the exact format of sections in a particular EHR does not adhere to known patterns.


1, the word "NFL" is more informative than "Today" for representing this news.

the attention weights of the search query and webpage title views are computed similarly.


Second, webpage titles are more important than search queries.

we apply a CNN network to learn contextual representations of words within news titles by capturing their contexts.


1 both users may have similar interests on Star Wars, and both books have related topics.

in addition, different words and sentences in the same review may also have different importance.


This is probably because our approach can mine the user-user and item-item relatedness by modeling the first-order and second-order interactions between users and items.

by optimizing the loss function, all parameters can be tuned via backward propagation.


For example, in the sentence "This story book is very interesting" , the word "interesting" is more important than the word "story" in representing this book.

first, we want to validate the effectiveness of the attention network in the review content-view.


In addition, we propose to apply a three-level attention network to select important words, sentences and reviews to learn informative user and item representations.

selecting important words via a word-level attention network can learn more informative sentence representations.


(2018) replace words with semantically and syntactically similar adversarial examples.

moreover, DISP works better on the random attack because the embeddings of the original tokens tend to have noticeably greater Euclidean distances to randomlypicked tokens than the distances to other tokens.


According to (Hirschauer, 2010), coverage and divergence should be considered for the acceptance decision of a paper.

furthermore, some researchers casted the problem as a time series task, and focused on analyzing temporal features or patterns in the process of citation growth (Davletov et al., 2014;Xiao et al., 2016;Yuan et al., 2018).


They typically perform on the nodes by updating the representations during training.

we connect distinct nodes based on simple heuristic rules and generate different edge representations for the connected nodes.


In this setting, entities mapped to the same KB ID are considered as mentions of an entity concept and pairs of mentions correspond to the pair's multiple instances.

we name our model EoG, an abbreviation of Edge-oriented Graph.


We briefly describe the model settings in this section.

document-level RE is not common in the general domain, as the entity types of interest can often be found in the same sentence (Banko et al., 2007).


With the released data, we randomly sample 5,000 sentence pairs from it as the parallel corpus with limited data volume.

then the contentonly representation is coupled with the representation of a style that differs from the input to produce a style-transferred sentence.


We analyse the sentences generated by S2S model and by the CPLS model, and the statistics show that the average length of the text generated by S2S model is shorter, which may lead to the bias of the style classifier.

bLEU calculates the N-gram overlap between the generated sentence and the references, thus can be used to measure the preservation of text content.


Compared with the Chinese literacy style datasets, the formality datasets are less challenging as discussed before.

this result may be explained by the fact that the edit distance between formal and informal texts are smaller than between ancient poems and modern Chinese texts.


The reason we choose the Chinese ballad is that the content domain of the two styles should be close.

it is more challenging for model to preserve the content meaning when transferring between ancient poems and modern Chinese text.


3% of questions are unlikely to have an answer anywhere (e.g., 'what guides Santa home after he has delivered presents?').

this makes privacy a well-motivated application domain for NLP researchers, where advances in enabling users to quickly identify the privacy issues most salient to them can potentially have large real-world impact.


Experiments conducted a large-scale real-world product description dataset show that our model achieves the state-of-the-art performance in terms of both traditional generation metrics as well as human evaluations.

non-entity words are labeled as "normal word".


In this section, we formally define the problem of Fine-Grained Entity Typing in KB.

yao Ming is associated with a typepath: BasketballP layer ⊂ Athlete ⊂ P erson ⊂ Agent ⊂ T hing Many researches have been carried out in this field.


Other more recent approaches include two-level reinforcement learning models (Takanobu et al., 2019), two layers of attention-based capsule network models , and self-attention with transformers (Verga et al., 2018).

through ablation analysis, we observe that the memory buffer and the KG features contribute significantly to this performance gain.


DR: How often do you have pain in your arms?

(Miwa and Sasaki, 2014;Katiyar and Cardie, 2016;Zhang et al., 2017;Zheng et al., 2017;Verga et al., 2018;Takanobu et al., 2019) also seek to jointly learn the entities and relations among them together.


Columns 1, 2, 3: lower is better / 4, 5, 6: higher is better.

though SCLS ( Step 3) is a convex problem, Figure 4 shows that AP+ADMM and AP+ActiveSet improve Specificity and Sparsity over AP+ExpGrad, making the learned topics even more comparable to Gibbs.


As a result, the basic Latent Dirichlet Allocation (Blei et al., 2003) is still prevalent for practitioners despite various recent advances (Srivastava and Sutton, 2017;Xun et al., 2017;Xu et al., 2018).

due to the efficiency of anchor-based inference, the moment construction often becomes the most expensive step for large corpora, but it is trivially parallelizable as the last averaging step is the only computation that couples individual documents.


Alternatively, one can leverage third-order moments to provide sufficient statistics for identifiable topic inference (Anandkumar et al., 2012a,b).

overall, correlated topic modeling via tensor decomposition is not as flexible as using JSMF even if we factor out the trivial difference in time and space complexities.


Provided with the set of anchor words S and the convex coefficientsB ki = {p Z 1 |X 1 (k|i)}, one can easily recover B by applying Bayes' rule: , where c i := p X 1 (i) is the unigram probability of the word i, which is equal to j C ij .

our RAW method takes less than 10 minutes to find 50 topics on Songs.


Meanwhile, one of the distractors has a lower score even though it shares the lexical item "Walgreens" with the context of the edit.

our architecture tackles both Comment Ranking and Edit Anchoring tasks by encoding specific edit actions such as additions and deletions, while also accounting for document context.


It is important to recall that the main goal of our work is to develop fast and efficient on-device neural text classification approach, which can achieve near state-of-the-art performance while satisfying the on-device small size and memory resource constrains.

we train the network with cross entropy loss and apply softmax over the output layer to obtain predicted probabilities y C k for each class C during inference.


Also, characterlevel models are prone to errors due to long-range dependency (Sennrich, 2016).

mRR and PmRR are highly dependent on the length distribution of test data.


Currently, neural machine translation systems widely use subword segmentation as de facto.

common practice is to uniformly sample from all possible prefixes within minimal length constraint in characters (or words) real distribution of the prefix length may differ to the uniform distribution.


After extracting candidates, reranking algorithms (e.g., LambdaMART (Burges, 2010)) with additional features are used to align final candidates.

we use the same model size for the character baseline and our variants for the fairness since our goal is proposing a new method and comparing between baseline and ours rather than achieving the best performance with the restriction of having a similar number of parameters.


We search for the sentence with the same word w pi from the current document, and feed the found sentence into the same Bi-LSTM model.

then, we provide some benchmark models on this dataset to boost the research of dialogue symptom diagnosis.


To make it more clear, we show the inference results for each symptom with and without graph in Figure 5.

(Yaghoobzadeh and Schütze, 2016) used knowledge base and aggregated corpus-level contextual information to learn an entity's classes.


True, False, and Uncertain are the inference results for whether the symptom exists in the patient.

these observations have verified the effectiveness of modeling the associations between symptoms via graphs for symptom inference.


Most models use encoder-decoder architecture.


For sentence fusion (Section 5.1), the input consists of two sentences, which sometimes need to be swapped.

the performance of the tagger is impaired significantly when leaving out the SWAP tag due to the model's inability to reconstruct 10.5% of the training set.


Different from traditional NQG task, the goal of CQG is to enhance the interactiveness and persistence of chit-chatting.

the research of open-domain conversational question generation (CQG) Hu et al., 2018) is still in its infancy.


To speed up convergence, we initialize our model through pre-training the reading network and the generation network.

the salient spans and the news title are then fed to the generation network to synthesize a comment.


(2018) do not publish their code for metric calculation, we employ a popular NLG evaluation project available at https://github.com/ Maluuba/nlg-eval, and modify the scripts with the scores provided in the data according to the formulas in (Qin et al., 2018) to calculate all the metrics.

a news article and a comment is not a pair of parallel text.


However, since the encoder in the topic modeling component is expected to be as a discriminator and guarantee that texts generated by the sequence modeling decoder have specific topic information, the encoder cannot be with the discrete representations (e.g., one-hot representation) of texts as input.

we compare several baselines including models based on language model (i.e., LSTM LM, LSTM+LDA, Topic-RNN, TDLM) and models based on VAE (i.e., LSTM VAE, VAE+HF, TGVAE, DCNNVAE, DVAE).


this work has been supported by National Key R&D Program of China (No.


DVAE (Xiao et al., 2018) uses a Dirichlet latent variable to improve VAE.

no extra training supervised by labeled data is needed for this discriminator.


Inferring the objects from the vision side helps learn the object relationships, and inferring from the language side helps learn the crossmodality alignments.

the learning rate is set to 1e−4 instead of 5e − 5.


One important difference between the standard use of CRFs for sequence labeling and our task is that our "labels" do not correspond to a fixed set of classes that can be predicted for any input, but are as specific to the particular input example as the sequences to be labeled themselves.

there could be more than one candidate region with high IoU with the gold region, and they should all be considered as correct grounding for the phrase.


For each dialogue, we invite at least four different workers to annotate.

after that, we concatenate the embeddings of the starting word (x * start ) and the ending word (x * end ) of each span, as well as its weighted embedding (x i ) and the length feature (φ(i)) to form its final representation e: On top of the extracted mention representation, we then compute the contextual score as follows: where [ ] represents the concatenation, e p and e n are the mention representation of the targeting pronoun and current candidate mention, and ⊙ indicates the element-wise multiplication.


In this work, we focus on jointly leveraging the contextual and visual information to resolve pronouns.

extensive experiments demonstrate the effectiveness of the proposed model.


We extract key frames within the annotated segment of each step via comparing similarity of different frames followed by manual filtering of redundant frames.

we annotate the theme for each makeup video.


Based on the list, we search their official channels on the YouTube and crawl makeup instructional videos together with available meta data such as video id, duration, title, tags and English subtitles generated by YouTube automatically.

due to the data limitation, fine-grained semantic comprehension which requires to capture semantic details of multimodal contents has not been well investigated.


Without bells and whistles, DEBUG surpasses the performance of the state-of-the-art models over various benchmarks and metrics at the highest speed.

we set the ground truth confidence of each frame based on its "centerness" in the segment.


as we have seen in the Section 2, human curated NER datasets are by no means perfect.


These numbers become 7000 and 4000 when k = 5, and 9000 and 7000 when k = 10.

in the forth sentence, looking at its paragraph, our annotators figure out that this is a table about ships and vessels loading items at different locations.


We first randomly partition the training data into k folds: We then train k NER models separately based on these k folds.

we use f (D, w) to describe the training process of an NER model using the training set D weighted by w. This training process will return an NER model M = f (D, w).


With more iterations, the confidence of being correct lowers like a binomial distribution, which is the reason that we chose an exponential decaying weight function in Equation 4.

in this paper, we conduct empirical studies to understand these mistakes, correct the mistakes in the test set to form a cleaner benchmark, and develop a novel framework to handle the mistakes in the training set.


We conduct extensive experiments on both the original CoNLL03 NER dataset and our corrected dataset.

we first estimate the precision of the detected mistakes of a single iteration.


Recent advances in deep learning have yielded stateof-the-art performance on many sequence labeling tasks, including NER (Collobert et al., 2011;Ma and Hovy, 2016;Peters et al., 2018).

we begin by training a NER model Θ using the above model's outputs as training data.


We present a bootstrapping recipe for improving low-resource NER.

we first compare the NER performance on the same number of annotated tokens.


Besides, the code for computing Grad-CAM-Text was adapted from keras-vis 6 , whereas we used scikit-learn (Pedregosa et al., 2011) for decision tree construction.

for the n-gram random baseline (and other n-gram based explanation methods in this paper), n is one of the CNN filter sizes [2,3,4].


We proposed three human tasks to evaluate local explanation methods for text classification.

a confident but incorrect answer results in a large negative score.


performance gain of the four proposed models, the retrieval precisions of GMSH, BMSH, GMSH-S and BMSH-S using 32-bit hashing codes on the three datasets are plotted together in Figure 4.

for BMSH, the difference between the best and worst precisions on the three datasets are 0.0123, 0.0052 and 0.0134, respectively, which are small comparing to the gains that BMSH has achieved.


Generative hashing is often used to generate hashing codes in an unsupervised way.

it can be obviously seen that GMSH-S and BMSH-S outperform GMSH and BMSH by a substantial margin, respectively.


To increase the modeling ability of (1), we may resort to more complex likelihood p θ (D|z), such as using deep neural networks to relate the latent z to the observation x i , instead of the simple softmax function in (2).

hashing is promising for large-scale information retrieval tasks thanks to the efficiency of distance evaluation between binary codes.


Furthermore, it is observed that the latent embeddings learned by BMSH-S can be clustered almost perfectly.

the averaged value over all testing documents is then reported.


The samples from the Bernoulli mixture can be generated by first choosing a component c ∈ {1, 2, • • • , K} from Cat(π) and then drawing a sample from the chosen distribution Bernoulli(γ c ).

a mapping from the latent representation z to the cor-responding label y is learned for each document.


In (Chaidaroon and Fang, 2017), variational deep semantic hashing (VDSH) is proposed to solve the semantic hashing problem by using the variational autoencoder (VAE) (Kingma and Welling, 2013).

data dependent hashing seeks to learn a hash function from the given training data in a supervised or an unsupervised way.


In this paper, we improve the top-1 MIPS performance by graph-based index.

for example, on fastTextfr, to reach the recall at 95%, ip-NSW requires about 0.3% computations while IPDG only needs 0.07% computations.


We found that the edge selection method is vital for the trade-off of effectiveness and efficiency in searching.

for un-normalized vectors, although cosine similarity is still widely applied, the final matching scores of word embeddings are usually weighted (Acree et al., 2016;Srinivas et al., 2010) by ranking-based coefficients (e.g., the side information), which transforms the problem back to search via inner product (see Eq.


To address this limitation, one solution is to incorporate relational dependencies of different words into their embeddings.

following (Keller and Lapata, 2003) and (de Cruys, 2014), we select three dependency relations (nsubj, dobj, and amod) as follows: • nsubj: The preference of subject for a given verb.


It verifies that the multiplicative composition used in our approach is able to introduce new information to the base codebook.

given the over-parameterization of deep neural nets, the effective compression of them has been receiving increasing attention from the research community.


(2017) introduce the Word Embedding Association Test (WEAT), which provides results analogous to earlier psychological work by Greenwald et al.

previous mitigation attempts rely on the operationalisation of gender bias as a projection over a linear subspace.


coreference resolution (Rudinger et al., 2018;Zhao et al., 2018).

bias is removed only insofar as the operationalisation allows.


Hybrid Ori results in smallest difference for cosine similarity and MRR gap between two gender forms.

we translate a query word and report the precision at k (i.e., fraction of correct translations that are ranked not larger than k) by retrieving its k nearest neighbours in the target language.


For gender in semantics, we follow the literature and address only binary gender.

given an English word E i (a noun, an adjective or a verb), an English occupation word E o , and the corresponding Spanish/French translation S i of E i , we adopt the analogy test "E i :E o = S i :?"


Figure 1 shows that the inanimate nouns lie near the origin point on the semantic gender direction, while the masculine and feminine forms of the occupation words are on the opposite sides for both directions.

extensive effort has been put toward analyzing and mitigating gender bias in word embeddings (Bolukbasi et al., 2016;Zhao et al., 2018b;Dev and Phillips, 2019;Ethayarajh et al., 2019).


Experiments show that BILEXNET substantially outperforms translation baselines and approaches the performance of a fully supervised English semantic relation classifier (Section 7).

in BILEXNET, words of the Equivalence class can occur in a parallel sentence pair where they are aligned to each other.


We use OntoNotes Release 5.0 1 , which contains a number of annotations including word senses for Chinese.

on average across the test sets, their approach did not outperform SVM with word embedding features.


We then split 80% of the numbers into a training set and 20% into a test set.

we investigate this using a numerical extrapolation setting: we train models on a specific integer range and test them on values greater than the largest training number and smaller than the smallest training number.


A: 53 Q: How long was the shortest touchdown pass?

we can understand the source of numeracy by isolating and probing these embeddings.


Furthermore, subtracting a baseline (Weaver and Tao, 2001) on R(ã, z) is also applied to reduce variance.

recently, Natural Language Interfaces to Databases (NLIDB) has received considerable attention, as they allow users to query databases by directly using natural language.


Instead of generating the restated query, we recombine the predicted precedent answerw x and the predicted follow-up answerw y to produce the restated answerw.

in the implementation of the REiNFORCE algorithm, we set M to be 20.


It reflects the approximate upper bound of AnsAcc, as the correctness of SQL-related words is a prerequisite of correct execution in most cases; BLEU, referring to the cumulative 4-gram BLEU score, evaluates how similar the predicted queries are to the golden ones (Papineni et al., 2002).

let f j = softmax(A :,j ), where f j ∈ R n denotes the attention weights on x according to y j .


As no intermediate annotation is involved, we design re-wards to jointly train the two phases by applying reinforcement learning (RL) (Sutton and Barto, 1998).

contextual information is essential for more accurate and robust semantic parsing, namely contextdependent semantic parsing.


As the interaction proceeds, the user question becomes more complicated as it requires longer SQL query to answer.

on SparC, we report two metrics: question match accuracy which is the score average over all questions and interaction match accuracy which is average over all interactions.


Furthermore, we also investigate the effect of copying segment.

in the context-dependent scenario, the contextual history is crucial to understand the follow-up questions from users, and the system often needs to reproduce partial sequences generated in previous turns.


With the help of the proposed pruning method, our model can effectively alleviate the imbalanced distribution of arguments and non-arguments, achieving faster convergence during training.

roth and Lapata (2016) introduced syntactic paths to guide neural architectures for dependency SrL.


During training and inference, the syntactic rule takes effect by excluding all candidate arguments whose predicate-argument relative position in parse tree is not in the list of top-k frequent tuples.

to investigate the most contribution of syntax to multilingual SRL, we perform experiments using the gold syntactic parse also officially provided by the CoNLL-2009 shared task instead of the predicted one.


Wikipedia examples are longer paragraphs of 66 words on average.

we had initial success sharing the parameters of the two towers which allows training much deeper models without increasing the parameter count.


(2) Are signals from relevance and semantic matching complementary?

additionally, the use of CNN layers allows us to explicitly control the window size for phrase modeling, which has been shown to be critical for relevance matching (Dai et al., 2018;Rao et al., 2019).


Second, we employ the normalized weights to sum the extracted representations as the final syntactic representation for word w i , denoted as rep syn i .

this kind of sharing strategy somewhat weakens the representation framework maintains distinct model parameters for each task, due to the neutralization of knowledge introduced by the auxiliary task.


7 Note that these sentential and phrasal paraphrases are obtained by automatic methods.

they aim to estimate features in a single sentence, which has little interaction with semantic equivalence assessment in a sentence pair.


To further improve time efficiency, we optimize objectives of the variational E-step and M-step simultaneously instead of alternatively.

, K} is the vector for the encoding of the k-th column name in the table.


A key part of the agent is a world model: it takes a percept (either an initial question or subsequent feedback from the user) and transitions to a new state.

under the MISP framework, we design an interactive semantic parsing system ( Figure 2), named MISP-SQL, for the task of text-to-SQL translation.


we think that the low performance on big AMR graphs is mainly attributed to two reasons: • Big AMR graphs are usually mapped into long sentences while seq2seq model tends to stop early for long inputs.


We employ a multi-layer graph attention network to propagate sentiment features from important syntax neighbourhood words to the aspect target.

the basic idea is that at layer 0 the hidden state for an aspect target node h t 0 is only dependent on the target's local features and at each layer l information related with the target from l-hop neighbourhood is added into the hidden state by the LStM unit.


BERT-AVG uses the average of the sentence representations to train a linear classifier.

sentiment features can be propagated recursively from the leaf nodes to the root node.


The comparison here suggests that LSTM and self-attention neural networks are able to capture better implicit structures than handcrafted features.

one idea to resolve this issue is to design an alternative mechanism to capture such useful structural information that resides in the input space.


First of all, we compare our model EI with the work proposed by Zhang et al.

in general, our model Ei outperforms all the baselines.


In this section, we investigate the effects of different routing iteration numbers.

for aspect term food, the sentimental polarity is posi-tive, but for aspect term price, the polarity is negative while for aspect term drinks, the polarity is neutral.


The main challenge in aspect-level sentiment classification is that one sentence expresses multiple sentiment polarities, resulting in overlapped feature representation.

we propose a novel capsule network and iterative EM routing method with interactive attention (IACapsNet) to solve this problem.


For instance, all baselines miss proper noun "wendys" in the first example.

the other type of error is the output that contains the original sentiment, which can be attributed to the failure of the separation of content and sentiment.


Trait goes beyond these models by incorporating sentiments and attributes in a flexible way, which eliminates the model's dependency on specific attribute types.

for each user, we use 80% of reviews for training and 20% for testing.


Throughout, * , †, and ‡ indicate significance at 0.05, 0.01, and 0.001, respectively.

it is not recognized by existing approaches.


We construct 4 strategies relying on smooth transitions from a low state λ of each task weight varying with the number of epochs.

since the presence of a polarity implies the presence of at least one entity, we expect that a joint prediction will perform better than an entitiy-based predictor only.


The entire network can be trained end-to-end to learn parameters α and β in addition to the sentence embedding and classifier, or the network can be optimized to learn only the weights α and β and weights of the classifier.

work such as ELMo (Peters et al., 2018) and BERT (Devlin et al., 2018) propose deeply connected layers to learn sentence embeddings by exploiting bi-directional contexts.


all the aspect terms of an input sentence in one forward pass.

comparing with DIFD-AT+MMD and DIFD-AT+cORAL, DIFD is more robust considering that DIFD outperforms the two methods in most experimental settings.


All models are optimized using Adam optimizer (Kingma and Ba, 2014) with gradient clipping equals to 5 (Pascanu et al., 2012).

numerous existing models (Tang et al., 2016b;Tay et al., 2017;Fan et al., 2018;Xue and Li, 2018) typically utilize an aspect-independent encoder to generate the sentence representation, and then apply the attention mechanism (Luong et al., 2015) or gating mechanism to conduct feature selection and extraction, while feature selection and extraction may base on noised representations.


Based on the deep transition Gated Recurrent Unit (GRU) Pascanu et al., 2014;Miceli Barone et al., 2017;Meng and Zhang, 2019), an aspect-guided GRU encoder is thus proposed, which utilizes the given aspect to guide the sentence encoding procedure at the very beginning stage.

the results in 6 show that all modules have an overall positive impact on the sentiment classification.


In this solution, two major challenges exist which are illustrated as follows.

during clause and word selection, a sentiment rating predictor is employed to provide reward signals to guide the above clause and word selection.


However, this soft-attention mechanism has the limitation that the softmax function always assigns small but non-zero probabilities to noisy clauses, which will weaken the attention given to the few truly significant clauses for a particular aspect.

besides, λ 1 , λ 2 are weight parameters.


On the other hand, when model training is finished, i.e., both high-level and low-level policy finish all their selections, the goal of sentiment rating predictor is to perform DASC.


This amounted to 103 rebuttal-speech pairs, since not all 55 GP-claims were mentioned in two speeches.

we thank the anonymous reviewers for their valuable comments, and Hayah Eichler for creating the initial GPR-KB.


Next, we establish baseline results for determining whether a GP-claim is mentioned in a speech, and compare them to results obtained for iDebate claims.

these works present a neural-based generative approach, and experiment with user-written posts.


Here bert is trained only on explicitly-mentioned claims, with respect to (ostensibly) semantically  similar sentences.

to compare the bert baseline to others, the precision-recall curves for both prior 9 and w2v were computed over speeches from bert-test.


A related task, that of generating a response which need not be a rebuttal or even argumentative, has been the subject of much research, especially in the context of dialog systems, chat bots, and question answering.

given a training set, the a-priori probability that a GP-claim will be mentioned in a speech can be computed.


(2016), and low-rank adaptation methods (Jaech and Ostendorf, 2018;Kim et al., 2019), but these did not improve the model performance.

directly incorporating attributes into the weight matrix may cause harm in the performance of the model.


Ex.1 shows an anecdotal example illustrating this behavior that the emotion cause clause c −1 adjoins the emotion word happiness.

the regularization term is formally expressed as: where m is a hyper-parameter for margin.


Words or phrases that discourage critical thought and meaningful discussion about a given topic.

even taking into account that γ is a pessimistic measure (Mathet et al., 2015), these values are low.


Labeling the object of the propaganda campaign as either something the target audience fears, hates, finds undesirable or otherwise loves or praises (Miller, 1939).

in our precision and recall versions, we give partial credit to imperfect matches at the character level, as in PD.


Popular CNN architectures use convolution with a fixed window size over the words in a sentence.

for example in the sentence, "it is also stupider", we see "is" having the near majority share, even though it tells nothing about the sentiment of the sentence.


Very deep CNN architectures were proposed based on character level features (Zhang et al., 2015) and word level features (Conneau et al., 2016) which significantly improved the performance in text classification.

we find max pooling to be very arbitrary in selection of crucial features and hence contributing minimal to the overall task.


The outputs from the first subnetwork and second subnetwork are concatenated and connected to the output layer through a dense connection.

this does not relate the same way of selecting most relevant features from convoluted features in texts.


Some recent studies in NLP have investigated the effect of interactions on the overall persuasive power of posts in social media (Tan et al., 2016;Hidey and McKeown, 2018).

recent studies in computational argumentation have mainly focused on the tasks of identifying the structure of the arguments such as argument structure parsing (Peldszus and Stede, 2015;Park and Cardie, 2014), and argument component classification (Habernal and Gurevych, 2017;Mochales and Moens, 2011).


We then follow the same procedure above, for fine-tuning.

incorporating the flat representation of the larger context along with the claim representation consistently achieves significantly better (p < 0.001) performance than the claim representation alone.


In this section, we conduct experiments to validate our model which we denote as CDT on benchmark datasets.

to verify this assumption, we trace from the input embeddings to the final embedding.


The state-of-the-art methods for representation learning have integrated dependency trees with neural networks.

the BiLStM and the GCN can be interpreted as message passing networks.


These observations motivate us to develop a neural model which can operate on the dependency tree of a sentence, with the aim to make accurate sentiment predictions with respect to specific aspects.

(Mou et al., 2015) exploit the short paths of dependency trees to learn representations of sentences using convolutional neural networks, while preserving dependency information.


• Target-agree (TA): similar to ST, but uses a forward NMT model with right-to-left decoder .

since the decision boundary does not have analytical form due to nonlinearity, computing the geometric distance is intractable.


In real experiments, losses are averaged over all training data.

we design a more implicit loss to help the student refrain from the incoherent translation results by acting towards the teacher in the hidden-state level: where , and φ is a penalty function.


We record the number of ROOT rule probabilities instead of predicting them  since there are only a small number of such rules.

in this work, we propose a novel universal grammar induction approach that represents language identities with continuous vectors and employs a neural network to predict grammar parameters based on the representation.


These assignments are non-arbitrary; indeed, Corbett (1991, Ch.

although this humorous take on German grammatical gender is clearly a caricature, the quote highlights the fact that the relationship between the grammatical gender of nouns and their lexical semantics is often quite opaque.


Although there are many theories about the assignment of inanimate nouns to grammatical genders, to the best of our knowledge, the linguistics literature lacks any large-scale, quantitative investigation of arbitrariness of noun-gender assignments.

the portion of the lexicon where this relationship is clear usually consists of animate nouns; nouns referring to people morphologically reflect the sociocultural notion of "natural genders."


Our approach begins by first retrieving the set of sentences X such that each sentence contains at least one referring expression that refers to an entity of the type we are doing perturbation on (person, in our case).

whether the PSA assumption holds in individual sentences will depend on the sentential context; however, the corpus-level trends that we measure in the model scores/labels are still indicative of systemic biases in the model.


It is well-established that gender bias exists in language -for example, we see evidence of this given the prevalence of sexism in abusive language datasets (Waseem and Hovy, 2016;Jha and Mamidi, 2017).

the clusters we mentioned so far all lean heavily toward one gender association or the other, but some clusters are interesting precisely because they do not lean heavily -this allows us to see where semantic groupings do not align exactly with gender association.


The label for a bag of sentences is calculated by averaging the k highest probabilistic scores.

first, we compare the positive hate crime labels predicted for Patch with the fBI's city-level hate crime reports.


The optimizer was Adam (Kingma and Ba, 2014).

we are more interested in recognizing the sentiment polarity of an event that pertains to commonsense knowledge (e.g., getting money and catching cold).


We can see that MOGANED achieves 1.6% and 1.7% improvement on precision and F 1 -measure, respectively, compared with the best baselines.

(2016)) only use the given sentences, which suffer from the low efficiency problem in capturing long-range dependency; dependency tree based methods utilize the syntactic relations (i.e., arcs) in the dependency tree of a given sentence to more effectively capture the interrelation between each candidate trigger word and its related entities or other triggers.


As Figure 1 shows, we divide the concepts into two types: the superordinate concepts representing more abstractive concepts, and the finegrained argument roles.

aCE 2005 (LDC2006T06) is the most widelyused dataset in event extraction.


Unlike most work on event extraction, we consider the realistic setting where gold entity labels are not available.

each token d i is predicted as an event trigger by assigning it a label t i .


This shows that representing the likely upcoming sentence helps the model form discourse expectations, which the classifier can then use to predict the coherence relation between the actually observed arguments.

(2018)) or a causal graph for biomedical concepts and events.


We can obtain the information of sentences boundaries in most cases, however, sometimes we cannot obtain the information of paragraph boundaries.

by replacing the similarity score, sim(,) with the dissimilarity score, 1−sim(, ), we can obtain the optimal tree in terms of the split score.


The evaluation results on RST-DT and PCC showed the effectiveness of our proposal; the dynamic programming-based approach with the span merging score, exploiting three granularity levels in a document, achieved .811 and .784 span F 1 scores on RST-DT and PCC, respectively.

we cannot compare the score with our score because the test set differs from ours.


Also concurrent, SpanBERT (Joshi et al., 2019), another self-supervised method, pretrains span representations achieving state of the art results (Avg.

yet, both variants perform much worse with longer context windows (512 tokens) in spite of being trained on 512-size contexts.


For each label, we further rank all neurons based on their sensitivity, and obtain an importance ranking for the label.

although it remains unclear what information the neurons exactly encode, we speculate that there are at least two kinds of information, based on the observed patterns: • Coarse-grain types of the current word.


In this work, we use the PLSR approach as a baseline for our model.

gathering and collating human-elicited property knowledge for concepts is very labour intensive, limiting both the number of words for which a rich feature set can be gathered, as well as the completeness of the feature listings for each word.


it consists of 1,981 scenarios and 4,110 multiplechoice questions in the geography domain at high school level, where diagrams (e.g., maps, charts) have been manually annotated with natural language descriptions to benefit NLP research.


In this paper, we introduce GeoSQA-an SQA dataset in the geography domain consisting of 1,981 scenarios and 4,110 multiple-choice questions at high school level.

we test the effectiveness of a variety of methods for question answering, textual entailment, and reading comprehension on GeoSQA.


2 Moreover, we found that the drop in accuracy in ToM is mostly caused  by memory questions.

while methods such as Recurrent Entity Networks have shown promise for keeping track of the state-of-the-world in our experiments, this is still in scenarios where the complexity of natural language is relatively simple.


Almost all previous state-of-the-art QA and RC models find answers by matching pas- DrQA (Chen et al., 2017) 37.7 44.5 41.9 48.7 32.3 38.3 29.8 -R 3 (Wang et al., 2018a) 35.3 41.7 49.0 55.3 47.3 53.7 29.1 37.5 OpenQA (Lin et al., 2018) 42.2 49.3 58.8 64.5 48.7 56.3 28.7 36.6 TraCRNet (Dehghani et al., 2019) 43.2 54.0 52.9 65.1 ----HAS-QA (Pang et al., 2019) 43.2 48.9 62.7 68.7 63.6 68.9 --BERT (Large) (Nogueira et al.,   sages with questions, aka inter-sentence matching (Wang and Jiang, 2017;Wang et al., 2016;Seo et al., 2017;Song et al., 2017).

we can see that this method brings us 4.7% EM and 4.1% F 1 improvements.


Does explicit inter-sentence matching matter?

during training, passages corresponding to the same question are taken as independent training instances.


Hereafter, we use sliding window method.

we do not know whether it is still required for BERT.


• We propose a Chinese span-extraction reading comprehension dataset which contains near 20,000 human-annotated questions, to add linguistic diversity in reading comprehension field.

use paraphrase or syntax transformation to add difficulties for answering.


In Figure 2a, multigranular interaction (2:1) between the bi-gram "United States" and the uni-gram "USA" allows the matching.

in InsuranceQA, the variance 5894  of word signals are low.


Then, we retrieve the neighbor triple of them, and reserve ones that contain lemmas of any token of the question.

the test set is not public, which needs to submit the model to the organization 1 to get the results.


We align question tokens with column names and cell text using the Levenshtein edit distance between n-grams in the question and the table text, similar to previous work (Shaw et al., 2019).

the model is doing the right thing but missing one of the values.


For nodes with multiple features, such as column and cell nodes, we reduce the set of feature embeddings to a single vector using the mean.

creating labeled data for this task can be expensive and time-consuming.


On In-suranceQA, this strategy alone improves over previously reported results by a minimum of 1.6 points in P@1.

we used Max-Pooling and Max-Min-Pooling, the latter being obtained by concatenating the outputs of Max and Min Pooling.


Obtaining such questions is hard for two reasons: (1) teaching crowdworkers about coreference is challenging, with even experts disagreeing on its nuances (Pradhan et al., 2007;Versley, 2008; Re-Byzantines were avid players of tavli (Byzantine Greek: τάβλη), a game known in English as backgammon, which is still popular in former Byzantine realms, and still known by the name tavli in Greece.

a: Barack Obama; Obama; Senator Obama Beyond training workers with the detailed instructions shown above, we ensured that the questions are of high quality by selecting a good pool of 21 workers using a two-stage selection process, allowing only those workers who clearly understood the requirements of the task to produce the final set of questions.


The scores ρ are used for computing the probability P (d | P, Q) as well as for pruning.

the Discrete Reasoning Over Passages (DROP) (Dua et al., 2019) dataset demonstrates that as long as there is quantitative reasoning involved, there are plenty of relatively straightforward questions that current extractive QA systems find difficult to answer.


We could also have crawled data from websites such as Yahoo Answers instead.

we have experimented with adding more Transformer layers on top of BERT but the performance did not improve.


For example, "start" and "started" have LCS "start".

the trained seq2seq model is prone to generating these "safe" questions, similar to the undiversified response generation in seq2seq-based dialogue model .


We present example story completions in Table 2 and full sampled stories in our appendix and Table 4.

we introduce a two-stage training pipeline ( Figure  1) to improve model performance both in terms of perplexity and CSR on story generation.


DUC-05 contains 1,600 summaries (50 questions x 32 systems); in DUC-06, 1,750 summaries are included (50 questions x 35 Figure 2: Illustration of different flavors of the investigated neural QE methods.

the three fine-tuned BERt versions clearly outperform all other methods.


7 The BERT multi-task versions perform better with highly correlated qualities like Q4 and Q5 (as illustrated in Figures 2 to 4 in the supplementary material).

the latter seems to be aligned with the definitions of Q3 (Referential Clarity), Q4 (Focus) and Q5 (Structure & Coherence).


That implies taking an action based on the current observation, where the action is picking a wordỹ t from the vocabulary V. r(ỹ, y) is a reward function with r(ỹ, y) = 1 if y = y.

experiments on the benchmark datasets show that (1) imitation learning is constantly better than reinforcement learning; and (2) the pointer-generator models with imitation learning outperform the state-of-theart methods with a large margin.


To rectify this, prior work has proposed losses which encourage overall coherency or other desired behavior (Li et al., 2016;Zhang and Lapata, 2017;.

similarly for LIM-ERICK, we consider only those samples to be acceptable which have line endings of the rhyming form AABBA.


In the first example, the baseline fails to recognize Len-shaped as an adjective, while the unifiedmodel succeeds by utilizing lexical features which are the input of question type prediction layer.

our work solves this problem as Section 3.3 shows.


(2018) incorporated a question word generation mode to generate question word at each decoding step, which utilized the answer information by employing the encoder hidden states at the answer start position.

baseline: How many provisions made provisions for concentrations?


However, RL models have poor sample efficiency and lead to very slow convergence rate.

an interesting and efficient way to express the relation between China and Germany.


In this paper, we introduce it as a DSR for deep RL.

rL methods usually start from a pretrained policy, which is established by optimizing XENT at each word generation step.


• We also introduce a self-attention based database schema encoder that enables our model to generalize to unseen databases.

for the classification, we applied attention-based bi-directional LSTM following Zhou et al.


The LM updated with textbook data (BERT+Textbook), improves performance on the domains included in additional pre-training (Phy and Gov).

• RQ3 can be answered as the updated BERT LM model does not appear to generalize well on unseen domains, as the evidence suggests that LM becomes more domain-specific.


Automatic grading is the task of evaluating the correctness of a student answer for a specific question by comparing it to a reference answer.

we collect textbooks corresponding to the domains and chunk them into paragraphs and feed each paragraph as a document for pretraining.


These all illustrate the diversity of linguistic and semantic challenges in WIQA.

each edge is labeled with a polarity, + or -, indicating whether the influence is positive (causes/increases) or negative (prevents/ reduces).


For ESIM, we take the dimension of hidden states of BiLSTMs to be 500.

the relations in our knowledge graph come from two sources: the Metathesaurus and the Semantic Network of UMLS.


With the emergence of knowledge graphs in different domains, the proposed approach can be tried out in other domains as well for future exploration.

we experiment with fusing embeddings obtained from knowledge graph with the state-of-theart approaches for NLI task, which mainly rely on contextual word embeddings.


Caching reduces the baseline's decoding speed from 210 seconds to 128.5; CMLMs do not use cached decoding.

in terms of speed, each mask-predict iteration is virtually equivalent to a refinement iteration.


Various non-autoregressive translation models, including our own CMLM, make the strong assumption that the individual token predictions are conditionally independent of each other.

our decoder is bi-directional, in the sense that it can use both left and right contexts to predict each token.


In this work, we propose a new method for modeling the copying mechanism for APE.

it shows the heatmap of the Enc-Dec-Attention, which averages over 8 different heads.


Census stands for the census data dataset and human stands for the human judgments dataset.

experiments suggest that our approach is effective in detecting human stereotypes, and is tied robustly to graph structure.


What's more, our results indicate that gender bias learned from large-scale texts is different from that within our minds, introducing a new perspective for lexical-level stereotype-related research.

in this method, we consider only local consistency (without the second term of Equation 1).


For each sentence in the sarcasm corpus S , a candidate negative situation phrase is extracted.

the notion behind this metric is that sarcasm typically requires more context than its literal version, requiring to have more words present at the target side.


In typical RL settings, the learner is typically initialized to a random policy distribution.

sG RL:, same as sG NORMAL but also applies reinforcement learning, 3.


After the candidates for a positive phrase are obtained, their Part of Speech tags are extracted with the help of a POS tagger.

such operations would require additional resources such as sentiment dictionary, sense disambiguation tools, whereas the neural classification based filtering can only work with binary sentiment labeled data.


As shown in examples in Figure 3, we compare our phrase segmentationbased padding (2-2-3 schema) to two less common schemas (i.e.

during training, the padded lines are used instead of the original poem lines.


For a given change B, the input to condition inference is the set of pairs u, k , where u k is a phone trigram in some underlying form and the label k can be positive ( ), negative (⊥), or unknown (?).

by contrasting examples in the first column, we infer that the insertion happens when the suffix /z/ occurs after a strident (like /s/ in /mIs/); otherwise, /z/ and /d/ are devoiced whenever they occur after a voiceless obstruent (like /p/ in /zIp/).


baselines on constructed Chinese character variation graph to get graph based character embeddings.

the formulations of the gate function are listed as follows:   , and , the output can be: where ω is the scale parameter, and s l is a weight parameter for the combination of each layer, which can be learned through the training process.


Given the context of mention m, we form its representation from involved contextualized word vectors with a mention-aware attention mechanism where C is the number of contextual words, and a c i is defined as a c i = Softmax(e c i ), where where ⊕ represents concatenation, and v c ∈ R da and W c ∈ R da×(2dr+1) are trainable parameters.

our Model* is a variant where we replace ELMo embeddings with Bert (large cased model) contextualized word representations.


Although they occur in the same sentence, our model is able to focus on different context words for different mentions with the mention-aware attention mechanism.

we calculate the attention for context words in a mention-aware manner, allowing the model to focus on different parts of the sentence for different mentions.


(2018c) introduced a new dataset called Multi-Axis Temporal RElations for Start-points (MATRES).


Our model is trained with Adam (Kingma and Ba, 2014).

the underlying assumptions behind most NER systems are that an entity should contain a contiguous sequence of words and should not overlap with each other.


The underlying assumptions behind most NER systems are that an entity should contain a contiguous sequence of words and should not overlap with each other.

• Empirical results show that our system achieves a significant improvement compared with previous methods, even in the absence of external features that previous methods used.


Since their method directly predicts a relation label for each surface pattern, it is more robust to the sparsity of surface patterns among a specific higher arity entity tuple.

player (Andre Johnson, Texans, 2009 season).


4 Here, all entities in each candidate tuple (e 1 , ..., e n ) are mentioned in the same text section T in a given set of documents.

these methods train the neural networks in a supervised manner using distant supervision (Mintz et al., 2009) and, therefore, may suffer from the lack of sufficient positive labels when a well-populated knowledge base is not available.


After that, u is used to compute the probabilities of it being a valid name for each type: where s is the sigmoid function and k th dimension of O G u indicates the probability of u being a valid mention of type y k .

by capturing mention regularity entailing gazetteers, the region-based models can be enhanced with more accurate name knowledge, and thereby the need of fully-annotated training data can be reduced.


To this end, this paper proposes Gazetteer-Enhanced Attentive Neural Networks (GEANN), whose architecture is shown in Figure 1.

for example, a region encoder should know "George Washington" is a valid PER name because "George" is a common first name and "Washington" is a common last name.


Thus, we used a random sample of 105 articles (for each year 5).

another goal is to train a neural network with a larger and better balanced training set to use the model to study a larger corpus.


From a humanities perspective, a VA constitutes an interesting phenomenon of enculturation (Holmqvist and Płuciennik, 2010) that deserves to be studied more in-depth, based on larger corpora.

attributing a particular property to a person by naming another person, who is typically wellknown for the respective property, is called a Vossian antonomasia (Va).


The best approach, using a BLSTM, reached a precision of 86.9% and a recall of 85.3%.

we can only determine precision and recall of our new approaches based on the baseline data set described in Section 3.


Then, the model calculates the j-th attention vectorŝ j using the context vector o j : where W e is a weight matrix and tanh is the hyperbolic tangent function.

this section describes the NER model using a character-level NLM (Akbik et al., 2018), which is our baseline comparison with our approach.


For r ∈ {r 1 , ..., r N }, o r is calculated by averaging,  Table 2: Accuracies (%) on few-shot DA.

they are from the same domain, yet in a real-world scenario, we might train models on one domain and perform few-shot learning on a different one.


These results contain two types of errors: idiosyncratic casing in the gold data and failures of the truecaser.

prior solutions have included models trained on lowercase text, or models that automatically recover capitalization from lowercase text, known as truecasing.


(2003) proposed a statistical, wordlevel, language-modeling based method for truecasing, and experimented on several downstream tasks, including NER.

experiment 5 shows that if the training data is also truecased, then the performance is good, especially in situations where the test data is known to contain no case information.


Fact is a proposition describing objective facts that can be verified using objective evidence and therefore captures the evidential facts in persuasions: Empire Theatres in Canada has a "Reel Babies" showing for certain movies.

revealing the role of EUs no significance is observed in a positive vs. negative case in Table 2.


Restaurant ACC F1 pos F1 neg ACC F1 pos F1 neg ACC F1 pos F1 neg NBSVM-uni (Wang and Manning, 2012)    From the figure, we can see that the test accuracy drops from around 0.8 to 0.5 when the noise rate increases from 0 to 0.5, but our NETAB outperforms CNN.

following (Kim, 2014), We also randomly select 10% of the test data for validation to check the model during training.


(2019) proposed a joint model that determines stance and sentiment simultaneously.

finally, we show that the proposed AT-JSS-Lex model achieves remarkable improvements in performance over strong baselines and prior works on the SemEval-2016 stance detection dataset.


Based on massive amounts of data, recent pretrained contextual representation models have made significant strides in advancing a number of different English NLP tasks.

ruder and Plank (2018) propose a novel multitask tri-training method that reduces the time and space complexity of classic tri-training for sentiment analysis.


(2017), researchers have developed a variety of methods for learning the structured/unstructured parts (i.e., style/content) of latent representations.

we use 512 hidden states and 128-dimensional word vectors.


This shows how, in contrast to the state of the art (Kozlowski et al., 2018;Garg et al., 2018;Lauretig, 2019;Zhao et al., 2018), informative priors allow easy integration with other, more complex, probabilistic models.

lack of interpretability and the unsupervised nature of word embeddings have limited their use within computational social science and digital humanities.


This conclusion is based on the observation that a diagnostic classifier trained over the supposedly debiased data representations could still predict gender, age and race above chance level in their experimental setup.

the mean of sentences extracted from the model trained with adversarial training is closest to the neutral value 5, but independent ttests show that the differences between all classes are insignificant (p > 0.05).


We also construct an artificial dataset from PAN16 TWIT where the main task labels are preserved but the demographic label is randomly shuffled (PAN16 RAND), allowing us to run experiments with no PREVALENT or SAMPLE-SPECIFIC gender correlations, only ACCIDENTAL.

the results show that humans had difficulties determining the gender of the author.


• A novel deep learning framework augmented with socio-linguistic features to detect sarcasm targets in sarcastic texts.

to the best of our knowledge, only Joshi et al.


For instance, in aspect based sentiment analysis, which deals with the identification of sentiment expressed toward different aspects or dimensions of the entities present in the text, it is very important to identify the sarcasm targets and sentiments toward them in the texts.

(2017) present a compilation of past works including the datasets, approaches, issues and trends in automatic sarcasm detection.


(2011) and the WNBA and NBA basketball commentaries of Aull and Brown (2013), but we emphasize that FOOTBALL is the first large-scale sports commentary corpus annotated for race.

such prior scholarship forms conclusions from small datasets 1 and subjective manual coding of race-specific language.


It works both at a character-and a word-level, thereby effectively handling incomplete or missing words.

the predicted sequence alters accordingly to ἀρτεμιδώρ, thereby illustrating the importance of context in the prediction process.


To simplify comparisons, all AG accentuation was discarded, as inputting accents was timeconsuming for the human evaluations described in the following paragraph.

we provide a set of the Top 20 predictions decoded using beam search.


to show the effectiveness of the lexical centrality of our tensor embedding method, we conduct an experiment on SemEval 2017 task 6B (Potash et al., 2017) consisting of tweeted responses to specific thematic prompts generated as part of a tV show.


In this way, we can rank the degree of humor effectively via lexical centrality (Radev et al., 2015), namely, regarding the distance to the lexical center as an indicator of the degree of humor.

their system uses an n-gram language model trained on a 6.2GB subset of the News Commentary Corpus and the News Crawl Corpus.


Adam (Kingma and Ba, 2014) is used for model optimization.

first, the interactions between words in news title are important for understanding the news.


Massive news articles are generated everyday and it is impossible for users to read all news to find their interested content (Phelan et al., 2011).

in this paper, we propose a neural news recommendation approach with multi-head selfattention (NRMS).


with the rapid growth of social media applications such as Facebook and Twitter, a significantly increasing number of individuals are using these social media public platforms to release humorous texts.


We can find that sentences with high probability are not always highly rated by the human evaluations.

human evaluation results show that the proposed method can generate significantly more natural anagrams than baseline methods.


It is also impractical if applied to the anagram generation task.

the generated anagrams contain natural sentences.


Table 4 shows the results of trying to generate fa-mous anagrams.

many anagram generation software and web services exist including the Internet Anagram Server 2 and the Anagram Artist 3 .


In this setting, we want to incorporate the consistency between the claim (C) and perspective (P ) representations.

for example, given the claim "Make all museums free of charge" is opposed by the perspective "State funding should be used elsewhere".


However, our model is not able to capture that the negation phrase 'do not require' opposes the claim.

there is an abundance of contentious claims on the Web including controversial statements from politicians, biased news reports, rumors, etc.


Our consistency-aware model BERT CONS outperforms all the other baselines.

our model is not able to capture that the negation phrase 'do not require' opposes the claim.


These text classification systems were chosen because both are commonly used in industry.

the classifiers are calibrated on the development set to a precision close to 0.90 and maximum recall.


For a patient's length of ICU stay of T hours, we have time series observations, x t at each time step t (1 hour interval) measured by instruments along with doctor's note n i recorded at irregular time stamps.

pre-trained word and sentence embeddings have also shown good results for sentence similarity tasks (Chen Figure 1: Doctor notes compliments measured physiological signals for better ICU management.


Davis and Goadrich (2006) suggest AUCPR for imbalanced class problems.

we dropped all clinical notes which doesn't have any chart time associated and also dropped all the patients without any notes.


Analysis : In Table 3, we analyze our models on Politics, the largest dataset.

the Instant Messaging systems (e.g., Slack) often require users to manually organize messages in threads.


For Bulgarian, the best results are obtained by Pilot 0, followed by Pilot 1.

in the last row e, we report the percentage of non-tying comparisons where Pilot 2 was judged better than Pilot 0, that is, P2 better ignoring ties than P0 (%) equals #(P2 better than P0) #(P2 better than P0) + #(P2 worse than P0) × 100% Figures 3 and 4 provide a graphical representation of Tables 10 and 11, respectively.


The statistics regarding the selected entries are shown in Table 6.

1 BabelNet has been applied in many natural language processing tasks, such as multilingual lexicon extraction, crosslingual word-sense disambiguation, annotation, and information extraction, all with good performance (Elbedweihy et al., 2013;Jadidinejad, 2013;Navigli et al., 2013;Ehrmann et al., 2014;Moro et al., 2014).


1 BabelNet has been applied in many natural language processing tasks, such as multilingual lexicon extraction, crosslingual word-sense disambiguation, annotation, and information extraction, all with good performance (Elbedweihy et al., 2013;Jadidinejad, 2013;Navigli et al., 2013;Ehrmann et al., 2014;Moro et al., 2014).

babelNet is both a multilingual encyclopedic dictionary, with lexicographic and encyclopedic coverage of terms, as well as a semantic network comprising 14 million entries which connects concepts and named entities in a very large network of semantic relations.


In (Saif et al., 2014) word contexts are adopted to generate sentiment orientation for words.

here, we compare the contribution of the DPLs with a well-known lexicon, i.e.


Notice that the Best-System 11 here reported is measured over the full test set.

the approach is based on the possibility to acquire comparable representations (e.g.


We compare the DPLs with another Italian polarity lexicon, called SENTIX in (Basile and Nissim, 2013).

sA deals with the problem of deciding whether a portion of a text, e.g.


In our case, a topic corresponds to a layer of the multiplex network and the nodes represent the users (Boccaletti et al., 2014).

there are communities that are only present in some topics, such as communities 3, 7, 8, 11, 12 or 19.


In (Hatzivassiloglou and McKeown, 1997) the authors model the corpus as a graph of adjectives joined by conjunctions.

to capture in-domain word polarities smaller domain focused dataset might work better (García-Pablos et al., 2015).


For this purpose, we employ leave-one-out within the languages, for which the value of the target feature is recorded in WALS.

in training, the dependent features are not used.


We first show the estimation accuracy for each feature both for the trained classifier and the majority baseline in Figure 3.

we (i) regard one such language as a test instance and the remaining languages as training instances, (ii) represent both the training and test instances with the features other than the target feature, (iii) see whether the value of the target feature in the test instance is correctly estimated or not, (iv) iterate this process for all those languages to calculate the estimation accuracy.


The goal of this paper is firstly to present a thorough analysis of the adequacy of currently used taggers for historical Dutch and secondly to explore methods for generating higher accuracy tags.

using a dictionary derived from a parallel corpuswhich finds a translation for the words in the corpus, rather than merely respelling them -results in an accuracy of 0.91 for within domain text, but does not generalise well to different domains (see Table 3).


the corpus was built in the framework of an interdisciplinary study jointly carried out by computational linguistics and experimental pedagogists and aimed at tracking the development of written language competence over the years and students' background information.


the average occurrences of these errors made by students born in Italy and abroad, we can claim that students born abroad make more errors than their mates in both the first and second year (see Table 8).

interestingly, the statistical distribution of some typologies of errors is correlated with the student background information we collected.


This model was trained with all possible features available (including those derived from manual annotations) from each four turn snippet taken into account.

an example of a snippet provided for annotation is shown in Table 1.


In all experiments, radial basis function kernels has been used.

the reported research is partly funded by the EU FP7 Metalogue project, under grant agreement number: 611073.


The Italian LUNA Human-Human Corpus ) is a collection 572 dialogues in the hardware/software help desk domain.

due to the greater drop in cross-domain evaluation of the 'legacy' models and better in-domain performance of the mapped ISO annotation, we conclude that the transfer of legacy annotation to the ISO standard is beneficial.


We observe that half of the chains have only two mentions, and that roughly 5.7% of the chains gather 10 mentions or more.

coreference resolution on biomedical texts took its place as an independent task in the BioNLP field; see for instance the Protein/Gene coreference task at BioNLP 2011 (Nguyen et al., 2011).


words (Word2vec rare) the baseline is significantly outperformed for one parameter set, while the performance for the other parameter sets confirms this tendency.

as this study also indicates, this varying constitution of test sets may lead to very different results testing a model on them.


note that this evaluation considers solely named entities (whereas our adapted system produces mixed chains, our gold reference contains only named entities).


Clark and Manning (2015), Durrett and Klein (2014) or Björkelund and Kuhn (2014).

ziering (2011) improved the scores of SU-CRE by integrating linguistic features.


In (Dakwale et al., 2012) scheme there is no provision to mark a relation between mentions of a chain, while MUC scheme 3 [1] has provision to mark limited relation types between mentions.

when a pronoun refers to time or time referring/representing an event or a clause in given discourse, for that we decided to mark "Anaphora-T" relation (T stands for Temporal) between those two mentions.


Most of the variables useful for open-domain pronoun resolution are irrelevant here, particularly gender, person, and animacy, since the entities and events mentioned are invariably referred to as neuter, 3 rd person, and inanimate in English (it, its, them, etc.).

finally, domain general systems are able to make assumptions that do not hold in this domain.


This violates the system's general assumption that a full antecedent will precede the anaphor, and by several paragraphs.

for example, a phosphorylated ASPP2 protein matches the ASPP2 and the phosphorylated protein but not the activated ASPP2.


Moreover, although RoR-siRNA alone was able to increase the p53 level, it did not cause p53 phosphorylation or acetylation.

linking full mentions as referring to the same real-world entity constrains later sieves and aids in assembly.


Error categories should be adapted to be more user-oriented than research-oriented: the number of categories should be limited, in order to focus on central error types only; the phrasing of categories should be changed in order to be usable by trainees with a less extensive knowledge of syntactic concepts and jargon; remediation possibilities should be systematically included and diversified in order to provide a clearer illustration of error types.


in the case of 45 students, more than one essay is available which could provide interesting material for learner profiling.


SW1203 contains in total 144 essays, where 35 students have written all the 3 essays.

annotating learner data manually is an extremely time-consuming and costly enterprise.


Alderson (2007) agrees with Little to say that "the methodologies being used [to compile these descriptions] are unclear or suspect".

as was the case for the coverage approach of Laufer and Ravenhorst-Kalovski (2010), this list only describes a global learning goal and does not contain any indications for appropriateness at different levels of learner proficiency.


In addition, many existing libraries define a complex class hierarchy, making it difficult for some users to use or adapt the modules.

most of these libraries are not designed specifically for NLP tasks.


This could be attributed to the fact that the rule to transform the FLELex distributions into a single level (i.e.

as for the lexical words, the model was less accurate, having correctly classified between 81.1% and 91.3% of them.


the A1 level, which was none different in the 51-text sample (Figure 3).

most studies on word substitution have relied upon the combined use of a synonym database and a strategy to rank substitution candidates by difficulty.


We can thus safely state that a graded lexical resource such as FLELex could be used to correctly distinguish between the complexity of two synonyms and hence to automatically substitute complex words with their easier synonyms observed in the resource.

we would like to thank the study participants for the time they spent giving us an essential feedback on their lexical competence, which enabled us to gain some valuable insights.


All learners signed an agreement of collecting their data to be used for research purpose in the lab.

in many cases, the human learners were skipping the conditional phrase of the utterance of the robot and gave a response by mimicking the part of the sentence that gives a short answer.


It is relatively complex compared to traditional evaluation schemes, by involving an interplay between the developement of systems to be evaluated and of oracles, and can be expected to need a few rounds of evaluation to reach maturity.

this new evaluation protocol for adaptive systems is not only expected to drive progress for such systems, but also to pave the way for a specialisation of actors along the value chain of their technological development.


In order to limit biases due to oracles with poor performance, each oracle should be itself evaluated and given more or less weight in the abovementioned average depending on its performance.

this can also be seen as a richness and an opportunity to be representative of real usage.


Conversely, if R Comp is low, little is likely to be gained by taking the union of system outputs.

in ablation, it is important to see the differences in system behaviours.


Slightly harder is the case of cheating precision.

1 This is a harmonic weighted mean of precision and recall.


When evaluating using these metrics, focus is given to true positives, false positives and false negatives.

this makes the evaluation sets fairly skewed.


We split the source corpora at the token level, assigning the same number of tokens to each KSC.

within each KSC set, the number of words in the corpora varies between 111k and 114k.


All three platforms were developed and have grown around the data that, sometimes by little more than coincidence, happened to be around at the time of development at the respective sites.

participants were also asked about their knowledge and experience in different areas directly relevant to working with oral corpora.


The resulting picture is very diverse: interplatform comparisons (e.g.

however, a noticeable proportion of users revealed that they were either not familiar with the respective functions of the platforms or that they found them difficult to use.


There are 3 labels (begin, inside, outside).

lastly, these embeddings outperforms CSlM for all tasks.


These embeddings achieve respectively 71.4% and 70.7% of overall accuracy.

in that work, we have shown that the combination with auto-encoders yields significant improvement on the ASR error detection task.


The entire Odin framework is available as part of the Processors NLP library.

from the languages that support syntax, Stanford's Tregex matches patterns over constituency trees (Levy and Andrew, 2006).


On the other hand, the surface-based grammar misses the ubiquitination event involving "TRAF6" (and the negative regulation of this ubiquitination), because the last two tokens of the sentence are not explicitly handled by the rules.

for brevity, we omit examples with explicit taxonomies in this section.


After a manual study of 500 "borderline" documents, a quality threshold of 91% has been experimentally set.

in order to do so, we used a slightly modified version of the reliability computation algorithm presented in section 8.


The metadata record comprises the corpus name, year, title and authors.

the number of algorithms is 21 instead of 25.


The motivation for developing the lexicon is threefold: firstly, the lexicon is a Dir-w-e-c CBoW PMI Figure 4: Learning curves for the TOEFL, BLESS, and SimLex-999 tests for the three different models (RI-dir-w-e-c, CBoW and PMI) using Wikipedia as data.

an artifact of only using addition for accumulating the distributional vectors in RI is that all vectors will tend towards the dominant eigenvector of the distributional space (this is the case also for standard DSMs built without RI).


In this paper, we argue that the KTimeML has some limitations, and propose a revised version of the KTimeML.

the domains of the Wikipedia documents are personage, music, university, and history.


println (t); } The class "TuplesDb" has also public methods to express many kinds of queries.

the hypernymy relations extraction consist of two phases.


They were asked to determine the relevance on a three-point scale: definitely relevant, 8 http://qt21.metashare.ilsp.gr/repository/browse/qtlpenglish-greek-corpus-for-the-medicaldomain/665f3832a93211e3b7d800155dbc020119068d540 possibly relevant and not relevant.

the average precision was also higher than the average recall for both tools.


There are also connections with research on health and medicine (cancer development, brain development, lymphocyte development, vaccine development).

the #Likelihood method (329 occurrences) measures the associative strength of terms thus enhancing the exclusive links with other terms, while the #Cosine similarity method (934 occurrences) analyses the similarity on the basis of the occurrences they have in common.


Moreover, "The Commission shall make available the information communicated by Member States to the European Parliament and the Council and shall ensure that it is also available to consumers and suppliers who request it."

as a next step, we will concentrate on the most significant extension of our linguistic coverage, and start with the inclusion of the surface syntactic representations of Duty elements by means of subcategorised finite and infinite clauses as noted in section 4.2.


For instance, in "The consumer shall return to the supplier any sums and/or property he has received from the supplier without any undue delay and no later than within 30 calendar days", the consumer is clearly bound to an action, and the duty is therefore taken into account.

in addition, "sums" has also been erroneously annotated as DutyCounterPart.


In this way we try to identify relations within Hohfeldian Duty constructs which involve any defined term or term definition elements.

these are not suitable to the Semantic Web, which requires models that are computationally tractable; (Francesconi, 2015) provides just such a rich and tractable model.


Gamification allows us to obtain resources without paying users by carefully designing a game suitable for information extraction.

(2014) constructed video games with the purpose of validating and extending knowledge bases.


We extracted English spelling errors using a word-typing game as in Rodrigues and Ritting (2012).

in the former approach, it is not clear which word corresponds to the spelling error, and the latter approach requires an annotation cost for the crowdsourcing.