This section describes how we categorize comparison questions.

when the same procedure is done on the training set and the model is re-trained, the accuracy increases to 60.10 F1 on the adversarial distractors.


1 Document Retrieval Knowledge bases (KBs) are considered as an essential resource for answering factoid questions.

after the propagation, the SGREaDER finally outputs a vectorized representation for each entity, encoding the knowledge indicated by its linked neighbors.


After choosing the best hyperparameters, we resort back to train on the original set.

standard Seq2Seq models ignore the patterns and may give ill-formed results.


We thank Wesam Al Nabki, Eduardo Fidalgo, Enrique Alegre, and Ivan de Paz for sharing their dataset of Onion sites with us.

onion named entities is comparable and relatively low.


6 To summarize, results suggest both that (1) legal and illegal texts are different in terms of their named entities and their coverage in Wikipedia, as well as that (2) standard databases and standard NLP tools for named entity recognition (and potentially other text understanding tasks), require considerable adaptation to be fully functional on text related to illegal activity.

the SVM model achieves an accuracy of 85.3% in the full setting.


It spawned quite a few set expansion techniques, such as Bayesian Sets (Ghahramani and Heller, 2006), SEAL (Wang and Cohen, 2007), SEISA (He and Xin, 2011) and others (Sarmento et al., 2007;Wang and Cohen, 2008;Wang et al., 2015).

we find that our method outperforms existing methods across all 4 datasets 12 .


In accordance to our assumption, the confidence score s e represents the degree of remoteness between the candidate concept e and the concept cluster H to which it belongs.

searching: For each concept c ij in a potential cluster H i , we search its directly-connected concepts in knowledge base KB, e.g., (c ij , r, e) with e ∈ E and r ∈ R, and use the distance between e and c ij to determine whether to merge it into H i .


We opted to use a balanced training set to make sure that our data and consequently our models are not biased towards a particular class.

7 Table 2 (upper part) shows the results for binary violation.


As we see, the introduction of our method to the standard backbone model (Trans.-Base) leads to substantial improvements across the validation and test sets.

language models are still important in reducing the size of the candidate set, regularizing word embeddings and generating fluent sentences.


While our approach OR-NMT can outperform the baseline system RNNsearch and the competitive comparison systems by directly incorporate the sentencelevel oracle and noise perturbations for relieving the overcorrection problem.


Our sentence encoder g averages 300 dimensional subword unit 3 embeddings to create a sentence representation.

the sentences were selected nearly randomly with the only constraints that they be between 3 and 25 tokens long and also that the outputs for SIMILE and BLEU were not identical.


Again, using a beam size of 8, we computed the cost for all generated translations and plotted their histogram in Figure 1.

training with BLEU has some limitations: it doesn't assign partial credit, it has a limited range of output values, and it can penalize semantically correct hypotheses if they differ lexically from the reference.


specifically, for each ED instance x = (t, c), we generate a positive lexical-enhanced training instance (t, c, t) with label (y, 1), and n negative instances 4 (t, c, w) with label (y, 0), where w is a word randomly sampled from context c. For each ED instance x = (t, c) in the train dataset D, the event classifier loss is: and the lexical binary classifier loss is: the loss function of lexical-enhanced ∆-learning is: By adding the auxiliary lexical classification task, this learning algorithm will ensure the learned representation be both event-related and lexicalrelevant.


Then the PCNNs model (Zeng et al., 2015) designed the multi-instance learning paradigm for RE.

nevertheless, semantics of the sentence could become entirely different as the segmentation changes.


(2) We also model multiple senses of polysemous words with the help of external linguistic knowledge, so as to alleviate polysemy ambiguity.

the word representation tends to favor the commonly used senses in the corpora, which will disturb the model when the correct sense of the current word is not the common one.


In the next step for the syntactic structure, we employ the dependency tree for the input sentence to generate the adjacent/weight matrix A syn = (a syn ij ) i,j=1..n , where a syn ij is set to 1 if x i is connected to x j in the tree, and 0 otherwise.

the formal definition of the EFP task is as follows.


Finally, the weight matrix A for GCNs would be the linear combination of the sematic structure A sem and the syntactic structure A syn with the trade-off λ: Given the weight matrix A, the GCNs (Kipf and Welling, 2016) are applied to augment the representations of the words in the input sentence with the contextual representations for EFP.

for instance, the word "left" in the sentence "She left yesterday."


would express an event that certainly happened (i.e., corresponding to a score of +3 in the benchmark datasets) while the event mention associated with "leave" in the sentence "She forgot to leave yesterday."

despite the rich context information over the whole sentence, the features in h k are not directly designed to focus on the import context words for factuality prediction.


One key component of temporal understanding is time expressions (timexes) that help anchor events to the time axis, but few recent systems effectively use the knowledge derivable from time expressions in their models.

for GloVe, the performance of the basic temporal model is similar to that when the time expression 6 See the appendix for more samples from the distant data.


The input to the system consists of two sentences, s 1 = {x 1 1 , x 1 2 , ...x 1 n } and s 2 = {x 2 1 , x 2 2 , ...x 2 m } containing e 1 and e 2 respectively.

these embeddings do not encode the full richness of temporal information needed for this task.


Our proposed timex model achieves an accuracy of 97.3%.

we did not find the results to be statistically significant according to a bootstrap resampling test (GloVe p-value = 0.349, ELMo p-value = 0.267).


We set the dimension of memory representations to k = 20 and compare our model and the baselines on the Original and Noisy tasks.

our work, on the other hand, considers a challenging scenario where the context is order of magnitude larger than the memory.


Dataset TriviaQA (Joshi et al., 2017) is a realistic text-based question answering dataset which includes 950K question-answer pairs from 662K documents collected from Wikipedia and the web.

this is a seemingly ill-posed problem since for most of the time, the scheduling should be performed without knowing which question will arrive next.


We see that EMR models which consider the relative importance between the memory entries (EMR-biGRU and EMR-Transformer) outperform both the rulebased baselines and EMR-Independent.

it outputs an action based on π(i|S (t) ), which denotes the selection of i th memory entry to delete.


06 Evaluate the models with the testing set using w as the sample weights.

we find that the graph-based features (Leakage and Advanced) make the problem feasible on a wide range of datasets.


In the second example, the model seems to focus more on the word 'largest' than the word 'fifth-' in the question.

where • indicates inner product operation and We use term-frequency-based encoding to obtain the sparse embedding s k i:j for each phrase.


As for the parser, we compare with state-of-the-art parsers in terms of training and testing speed.

the generated parse tree can be malformed, which mismatches the sentence with incorrect number of leaves.


Unlike in our task, the tasks that are investigated in those papers do not require the mapping of natural language to a meaning representation over a space of compositional logical forms.

the extracted features indicate that the logical form size is larger than n, for any n ≥ 2.


Combined Effect The impact of our features and application logic filtering is much more substantial for GMDP models, both on average and for the four domains where GMDP outperforms ADAGRAD (four rightmost domain columns of the table).

our training algo- Figure 1: Two examples from our dataset.


We measure model performance by calculating the accuracy of generated logical forms, and compare performance of our approach(HSP) with various competitive baselines.

these approaches miss the opportunity to utilize question decomposition information for complex question semantic parsing.


BERT LARGE This is the large BERT model released by the authors, which we used as a baseline.

since MT-DNN uses BERT LARGE to initialize its shared layers, the gain is mainly attributed to the use of MTL in refining the shared layers.


Dependency parsing allows us to design our extraction method such that each S1 and S2 is interpretable as a full sentence in isolation, and the appropriate conceptual relation holds between the pair.

(2018), who trained a sentence encoding model on five language-related tasks.


(2017)) use to compare models focus on sentiment, entailment, and similarity.

models like BERT (Devlin et al., 2018) are trained to predict random missing words in very large corpora (see Table 1 for the data requirements of the models we compare).


Figure 5 reflects classification performance for the model trained on the full dataset, that we later show generalization results for.

inspecting the matrices shows very similar confusability, suggesting that training on unbalanced data does not greatly decrease sensitivity to non-frequency predictors.


This indicates information tends to implicitly propagate through the interaction.

to make the tasks feasible, SCONE (Long et al., 2016) and SequentialQA (Iyyer et al., 2017) exclude many questions with rich semantic and contextual types.


2010, annotators decide relative orders instead of a complete order over all systems.

sMATCH only checks edge labels, thus two edges with the same label but conveying different meanings can be considered as equivalent by sMATCH.


On the other hand, we observe the reconstruction model could potentially go beyond surface token-wise match between NL utterances and MRs, promoting more complex (longer) candidate MRs that more adequately encode the semantics of the input utterance 7 (e.g., in Example 3, z 5 receives a much higher reconstruction score, while the difference between the discriminative matching scores is small).

this research was supported by NSF Award No.


Comparing these labels with the ground-truth test labels yields an overall GLUE score of 87.1-well above BERT's 80.5 and Big-Bird's 82.9-and yields single-task scores that are substantially better than both on six of nine tasks.

our annotators, however, are only given 20 examples and a short set of instructions for training, while BERT and BigBird are fine-tuned on the 364k-example QQP training set.


(2018), we allow some source names to be changed or swapped in an as-graph constant after their original assignments.

in some graphbanks (DM, PAS, PSD), we also have an artificial word at the end of each sentence, that is used to connect the graphs.


This section presents details of the heuristics discussed in Section 3 of the main paper, concerning grouping (edge attachment) and source heuristics.

control verbs are identified automatically, and their sources are assigned annotations which enforce the appropriate argument sharing.


Also, our best single model (16) achieves more than 9 Smatch points on top of BO (0).

for all the remaining metrics, our parser consistently achieves the second best results.


We complement the state with an attention over the input sentence (Bahdanau et al., 2014).

(3) Incorporating contextualized vectors (with BERT) and compare their effectiveness with detailed ablation experiments.


We build upon a transition-based parser (Ballesteros and Al-Onaizan, 2017) that uses Stack-LSTMs .

we use general attention (Luong et al., 2015).


Context features are generated by averaging the representations of the utterances (as per Section 4) present in the context.

4 We call this the speaker-independent setup.


We extend the definition of an utterance 2 to include consecutive multi-sentence dialogues of the same speaker to prioritize completeness of information.

some of the context videos contain multi-party dialogue between speakers participating in the scene.


lowed to extract each of them is described below: Text Features: We represent the textual utterances in the dataset using BERT (Devlin et al., 2018), which provides a sentence representation u t ∈ R dt for every utterance u.

the penalty term C is kept as a hyper-parameter which we tune based on each experiment (we choose between 1, 10, 30, 500, and 1000).


We split the data in this way in order to encourage our models to learn more domain independent features, that are applicable across the diverse set of controversial topics.

with this structure, we can identify indirect support/oppose relationships even between nodes without parent-child relationships if they (a) Number of controversial topics with the given predefined categories.


These learning objectives together allow the model to learn representations that can be easily fine-tuned to achieve state-of-the-art performance for a wide range of natural language processing tasks.

to existing datasets, ours consists of argument trees where each root node represents the argument thesis (main claim) and every other node represents a claim that either supports or opposes its parent.


Sentiment grammars provides a principled way for explicitly modeling sentiment composition, and through parameterizing the emission rules with neural encoders, it can take the advantage of deep learning.

table 3 shows that our methods beat the baseline on every task.


while there has been significant progress in sentiment analysis for English, not much effort has been invested in analyzing Japanese due to its sparse nature and the dependency on large datasets required by deep learning.


We learn such reason embedding by comparing the reasons using utterance-level (dis)agreement supervision, so that reasons supporting agreed (disagreed) stances would have similar (different) representations.

our work is mostly related to the task of detecting agreement and disagreement in online discussions.


We empirically verified this by transferring learned parameters across domains.

we simply treat our multi-label classification task as a set of binary classification tasks, where MLP classifiers share parameters except for those of an output layer over motive categories.


category c. We use the following class weight (Morik et al., 1999).

table 2 shows the distributions of human motive labels.


The comprehensive results show that by incorporating refined context-aware embeddings of targets and aspects into the neural models can substantially improve the performance of aspect detection.

for example, in sentence "location1 is your best bet for secure although expensive and location2 is too far.


Each transcript has been independently annotated by at least two annotators 4 .

relations can be predicted between arguments (Cabrio and Villata, 2013) or between argument components (Stab and Gurevych, 2017).


Instead of making a prediction based on the comparison result of a single alignment process, a stacked model with multiple alignment layers maintains its intermediate states and gradually refines its predictions.

the results of the second baseline show that vanilla residual connections without direct access to the original pointwise features are not enough to model the relations in many text matching tasks.


For a sequence of length l, We denote the input and output of the n-th block as x (n)  l ), respectively.

the alignment layer and the prediction layer can be simplified on some of the datasets.


It comprises of a convolutional layer, a fully connected projection layer and an inner product layer for the final predictions.

concatenation of these feature maps represents an input triple.


For the low-node, the number of 2-hop neighbors is 439 and 3-hop neighbors is 1543.

dISTMULT  learns embeddings using a bilinear diagonal model which is a special case of the bilinear objective used in NTN and TransE.


Several recent works suggest that convolutional neural network (CNN) based models generate richer and more expressive feature embeddings and hence also perform well on relation prediction.

additionally, W 1 denotes the linear transformation matrix.


Finally SemAligner (Maharjan et al., 2016), like this work, chunks input sentences into phrases before alignment.

i had a sleep paralysis dream that i was abducted by aliens.


I had a sleep paralysis dream that I was abducted by aliens.

evaluating only on the 406 sentence pairs that contain at least one possible requires systems not only to perform well on longer alignments, but also to avoid sacrificing performance on short alignments.


The results of all methods for infrequent entities are worse than the results on all entities; however, we observe the same trends among the different methods.

the above models need grounding of logical rules.


We reform the normal knowledge graph representation into a Markov chain with relations and entity-pairs as its states.

our method can incorporate millions of automatically discovered entailment rules.


This feature is narrative in the sense that it takes into account whether an entity is central to the story in the text.

our model involves several novel discourse and narrative features, as well as a small number of feature modifications.


Alternatively, future models might answer correctly only by picking up on spurious patterns, in which case a new development of the benchmark -using these models as adversaries -would place us in the same position as we are right now.

for instance, while ESIM+ELMo obtained 59% accuracy on SWAG, it obtains only 33.3% accuracy on HellaSwag.


Humans can read a narrative like this, shown in Figure 1, and connect it to a rich model of the world: the dog is currently dry and not soapy, and it actively doesn't want to be bathed.

we will use these activity labels as additional structure to test generalization ability.


We observe that due to weak signal strength and enlarged search space from multiple domains, WEAK-COMBINED model performs poorly across domains.

each expert e k provides a transformed training dataset to the student , where (p k θ ) i is the expert's probability distribution on the entire program space w.r.t input utterance x i .


Since distributional similarity does not capture parent-child relations, the relationships are not registered as parent-child but as co-hyponym relationships.

this Poincaré distance enables us to capture the hierarchy and similarity between words simultaneously.


In our experiments, we use the output of three different systems.

to embeddings in the Euclidean space where the cosine similarity u |u||v| is commonly applied as a similarity measure, Poincaré embeddings use a hyperbolic space, specifically the Poincaré ball model (Stillwell, 1996).


We evaluate GLEN on datasets from Upadhyay et al.

distributional vectors (i.e., embeddings) cannot be directly used to reliably detect LE.


1 (top), our method outperforms all previous unsupervised results sharply.

on the former task, we use the original set of 60 questions (PDP-60) as the main benchmark.


Namely, our method achieves an accuracy of 60.3%, nearly 3% of accuracy above the previous best result.

the concept of self-attention is further expanded within BERt by the idea of so called multi-head outputs that are incorporated in each layer.


Generally, this is prohibitively expensive and therefore not viable.

as of today, state-of-the-art accuracy on the WSC-273 for single model performance is around 57%,  and (Trinh and Le, 2018).


MAS score of different words are indicated with colors, where the gradient from blue to red represents the score transition from low to high.

all these approaches require the answer to explicitly exist in the text.


Since the introduction of BERT, the NLP community continues to be impressed by the amount of ideas produced on top of this powerful language representation model.

we achieve 68.3% accuracy, better than the more recently reported results from (Liu et al., 2017) (66.7%), who makes use of three KBs and a supervised deep network.


Secondly, the results of BERT WIKI seem to indicate that this dataset alone does not help BERT.

there have been several attempts at solving WSC273.


CFNet is the model with both the Coreference alignment and the conversation Flow modeling.

questions can be raised by either part in real scenario.


Unsupervised text style transfer aims to learn two conditional distributions p(x 1→2 |x 1 ) and p(x 2→1 |x 2 ) which alter the style of a sentence and preserve the style-independent content.

the uniform policy assures that all operators are trained on sufficient and diversified data.


In this section we compare several automatic evaluation metrics by checking their correlation with the scores assigned by humans to table-to-text models.

here, we describe two simple models of lexical entailment, inspired by work on the Recognizing Textual Entailment Challenge (Dagan et al., 2006).


For TRIVIAQA we have for all questions, contexts from three different sources -Wikipedia (TQA-W), Bing web snippets (TQA-U), and Google web snippets (TQA-G).

questions mostly require soft matching of the language in the question to a local context in the text.


F (•) is a standard nonlinear transformation function (i.e., F (x) = σ(W x + b), where σ indicates non-linearity function), and is shared between context and question.

this may be due to the requirement of reducing the batch size significantly.


we can conclude that explanations help elucidate the longer and more complicated compositional questions.


In this work, we address the interpretability of ML based question answering (QA) models on a combination of knowledge bases (KB) and text documents.

for every z , we obtain the corresponding original representation f and give this reduced input to the TextKBQA model.


an explanation method is assumed to be successful if it helps annotators identify the better model.


Attention weights at hop 3 performs best among the attention-based methods, but worse than LIME and IP.

they do not analyze the importance of the facts used by these QA systems.


lingual word embeddings are modular by language: vectors in one language are consistently closer to each other than vectors in another language (Figure 1).

examples of seed terms include "earthquake" and "flood".


These existing metrics are limited by either requiring external resources or considering only part of the embedding structure (e.g., intra-lingual but not cross-lingual neighbors).

we focus on MUSE  since it is unstable, especially on distant language pairs (Artetxe et al., 2018a;Søgaard et al., 2018;Hoshen and wolf, 2018) and therefore requires an effective metric for model selection.


Our earlier experiments all use k = 3 since it gives the highest Pearson's and Spearman's correlation on the tuning dataset ( Figure 6).

our work develops an intrinsic metric which is highly correlated with multiple downstream tasks but does not require external resources, and considers both intraand cross-lingual neighbors.


We also compute HPE-skew, the (positive) skewness of the empirical distribution of HPE on the corpus tokens.

note that to ensure a false discovery rate of at most = .05, all reported -values have to be corrected using Benjamini and Hochberg (1995) Highlighted on the right are deu and fra, for which we have many Bibles, and eng, which has often been prioritized even over these two in research.


Our language difficulty estimates were largely stable across datasets and language model architectures, but they were not significantly predicted by linguistic factors.

we compute HPE-mean from dependency parses of the Europarl data, generated using UDPipe 1.2.0 (Straka et al., 2016) and freely-available tokenization, tagging, parsing models trained on the Universal Dependencies 2.0 treebanks (Straka and Straková, 2017).


Our hypothesis that scrambling makes language harder to model seems confirmed at first: while the non-parametric (and thus more weakly powered) Spearman's = .196/.092 ( = .394/.691), Pearson's = .486/.522 ( = .032/.015).

the RNN is then trained over the sequence of units, which looks like this: "the |ex|os|kel|eton |is |gener|ally |blue".


This provides clear evidence of M-BERT's multilingual representation ability, mapping structures onto new vocabularies based on a shared representation induced solely from monolingual language model training data.

dE i ), we compute the vector pointing from one to the other and average it over all pairs: where M is the number of pairs.


There is a big gap between this model's zeroshot performance and M-BERT's, showing the pretraining is helping learn a useful cross-lingual representation for grammar.

cross-script transfer is less accurate for other pairs, such as English and Japanese, indicating that M-BERT's multilingual representation is not able to generalize equally well in all cases.


In addition, the presence of verbs in past participle (VBN) is the most distinctive part-of-speech pattern of complaints.

words regularly describing negative sentiment or emotions are not one of the most distinctive features for complaints.


Besides, the full version of our model gets the best BLEU-3, BLEU-4 and METEOR values by introducing the linguistic-based question evaluation model during adversarial training.

our model is composed by a generator and a discriminator.


The model uses a neural-network that takes sentences as input and generate a 4800 dimension embedding for each sentence.

social media conversations are in the format of a tree where a node can have many children.


To evaluate our models, we use a humanlabeled Twitter dataset that contains stance labels and rumor labels for around two thousand rumour threads related to five different events.

additional operation like 'Sum' or 'MaxPooling' is not needed.


Aside from the difficulties in obtaining training data, a major obstacle to the application of end-to-end models to multi-document summarization is the sheer size and number of source documents which can be very large.

most previous multi-document summarization methods are extractive operating over graph-based representations of sentences or passages.


Similarly to Algorithm 1, Algorithm 2 requires (i) the input text, (ii) the taxonomy T , (iii) the set of tuples C, (iv) the threshold θ f and (v) the level of generalization d. In lines 6 − 25, a term c i is candidate for generalization when its frequency f i is below the specified threshold θ f (line 7).

nevertheless, semantic-based methods are not able to achieve comparable performance to deep learning approaches (Gupta and Gupta, 2019) and for this reason, a framework utilizing semantic-based data generalization for the enhancement of sequenceto-sequence (seq2seq) deep learning abstractive summarization is presented in this work.


The intuition behind this transformation is the fact that machine learning systems tend to require a sufficient number of training samples prior to producing accurate predictions.

the underlined words are those that have been generalized and vice versa.


1 The score distribution on which evaluation metrics are tested (blue zone) differs from the one in which they now operate (red zone).

(ii) We observe low and even some negative correlations in this range.


LexRank is a graph-based extractive summarizer often used as a baseline.

(2013) used accuracy and found ROUGE-4 to perform well.


We refer to this dataset as (W) as it covers the whole scoring range.

the sampling still covers a large diversity of high-scoring summary and reveal general properties of evaluation metrics.


Unfortunately, existing metrics disagree wildly in this range.

we cannot clearly decide which one is the best.


JS-2 and R-2 have the strongest correlation, while R-L is less correlated with the others.

they performed analysis on the best scoring summaries from 6 systems which remain average compared to nowadays standard.


To manage this issue we apply kernel smoothing over the language model to adapt it from the full vocab V down to the candidate word list C. Our smoothing process focuses on the output embeddings from the pretrained language model.

in abstractive summarization, we use K = 6 in the candidate list and use the fixed embeddings at the bottom layer of ELMo language model for similarity.


In this paper, we examine these questions, focusing in particular on the relationship between irregularity and frequency.

similarly, Georgian verbs often have different roots depending on their tense, aspect, or mood marking.


It is often observed that there are differences in the prevalence of irregularity across languages (Stolz et al., 2012).

this result is consistent with the findings of Cotterell et al.


As discussed above, we hold out whole lexemes, including all of their inflected forms during training.

note that our definition of ι conditions on L −the language without the target lexeme-rather than on L −w -the language without the target word.


However, there is wide variability in the average irregularity score between languages.

we follow the hyperparameter used by wu and Cotterell (2019).


By measuring which words or lexemes are most predictable from one another, a general picture of morphological relatedness within a language can be built in a bottom-up way.

the correlation between irregularity and frequency is considerably more robust when considered at the lexeme level.


6 Here we present the first study to probe the relationship between irregularity and frequency at scale.

to many models of inflectional morphology, we collapse syncretic forms of a word into a single paradigm slot, thereby assuming that every every surface form w in a paradigm is distinct.


Simply put, language does not occur in a vacuum.

to prior work in machine learning, where mappings between vision and language have been examined (Kiros et al., 2014;Vinyals et al., 2015;Xu et al., 2015), our goal in integrating visual and linguistic data is not to accomplish a task such as image search/captioning that inherently requires a mapping between these modalities.


Significant correlations (greater than zero) across subjects were found between the dissimilarity structures of the neural estimates for All Verbs and the VERB model in the LPG (r = 0.04, p < 0.01), LIFG (r = 0.04, p < 0.01), but not the LMTP (Table 1).

fMRI Data Acquisition fMRI images were acquired with a Siemens MAGNETOM Trio 3T System with a 32-channel head matrix coil.


In the evolution of the free-order language with markers, accuracy was relatively stable in both speaking and listening (99.82% and 100%, respectively, for the last-generation agent, averaging across 25 runs).

a θ 1 , after training, is in its turn fixed and used to train the next agent a θ 2 , etc.


This might be explained by a universal preference for efficient and non-redundant grammatical coding (Fedzechkina et al., 2016a;Qian and Jaeger, 2012;Zipf, 1949).

among the fixed-order languages, both backward and noniconic prefer redundant coding.


NomBank (Meyers et al., 2004) extended the argument structure for instances of common nouns.

1 Benchmark results using recent state-of-the-art approaches demonstrate that good accuracy is possible, but complexity and run time are open challenges.


It is used for many downstream language processing tasks, e.g., coreference resolution, question answering, summarization, entity linking, relation extraction and knowledge base population.

the ACE corpora (Mitchell et al., 2004;Walker et al., 2005) consist of data of various types annotated for entities, relations and events.


Therefore, words in a nested mention should not be regarded as the anchor word of outer mentions, and therefore a word can only be assigned as the anchor word of the innermost mention containing it.

figure 5 shows an error case stemming from postpositive attributive.


Bag Loss selects anchor words according to their associations with the bag type.

besides, Muis and Lu (2017) developed a gap-based tagging schema to capture nested structures.


Experiments show that ARNs can effectively detect nested entity mentions and achieve the state-of-the-art performance on all above three datasets.

bag Loss guarantees that at least one anchor word (the one with highest P (c i |x i ), and its corresponding w i will be 1.0) will be selected for each bag.


For textual networks, when the embedding objective is network topology reconstruction, a natural choice is to treat the text as context (Tu et al., 2017).

for (i), if the sequence embeddings used do not explicitly address the syntactic structure of the text, a relatively dense attention score matrix can be expected.


Table 5 shows the results of our model compared to baselines in different evaluation settings: TDM extraction (Table 5a), TDM extraction excluding papers with "Unknown" annotation (Table 5b), and TDMS extraction excluding papers with "Unknown" annotation (Table 5c).

then, for each paper, we extract bag-of-word features with tf-idf weights from the DoctAEt representation described in Section 4.


Finally we link these candidates to the valid TDM triples in a taxonomy 12 based on Jaccard similarity.

our model is formally a structured conditional random field.


On average, each product contains about 10 attributes.

the Christmas test set contains 1, 121 types of attributes, 708 of which are new attributes.


Since keyphrases can provide a high-level topic description of a document, they are beneficial for a wide range of natural language processing tasks such as information extraction (Wan and Xiao, 2008), text summarization  and question answering (Tang et al., 2017).

though existing keyphrase generation methods have achieved remarkable performance on this task, they generate many overlapping phrases (including sub-phrases or super-phrases) of keyphrases.


(2010) conducted very first study utilizing user comments from their social media post.

(3) We validate our proposed framework on three popular benchmark datasets, namely Twitter (Sarker et al., 2016), CADEC (Karimi et al., 2015) and MEDLINE (Gurulingappa et al., 2012a) for pharmacovigilance mining, having different annotation schemes.


One line of prior works designed rules (Mukherjee and Garain, 2008;Hosseini et al., 2014) or templates Zhou et al., 2015;Mitra and Baral, 2016) to map problems to expressions, where rules or templates are collected from training data.

the key to solving the problem is to reveal the underlying mathematical relations (such as addition and subtraction) among quantities, and then generate equations to find solutions.


This reveals that one main challenge for automatically solving arithmetic word problems is to recognize the irrelevant information that should be labeled with "0".

the N node is used to tag a word token if and only if the token represents a quantity.


"Impv-BR" and "Impv-Seq2Seq" denote the improvement of our model compared to BR-LR and Seq2Seq, respectively.


Given a text sequence x containing m words, the MLC task aims to assign a subset y containing n labels in the total label set Y to x.

the performance of classifier is sensitive to the pre-defined label order.


With the ever-increasing accuracy in speech recognition and complexity in user-generated utterances, it becomes a critical issue for mobile phones or smart speaker devices to understand the natural language in order to give informative responses.

as a text classification task, the decent performance on utterance-level intent detection usually relies on hidden representations that are learned in the intermediate layers via multiple non-linear transformations.


From the right part of Figure  6, we found that with the word operetta for the theatre organ being recognized in the lower capsule, the slots I-object name and B-object type contribute more to the correct intent capsule SearchCreativeWork, when comparing with other routing alternatives to other intent capsules.

we propose a re-routing schema for Slot-Caps where the dynamic routing-by-agreement is realized by the following equation that replaces the Line 7 in Algorithm 1: whereûẑ is the intent activation vector with the largest norm.


They both use three linearchain CRF layers for the three different prediction tasks: CRF-RA is for task t a ; CRF-RO is for task t o ; CRF-M is for task t m .

we mine aspect and opinion term extraction rules that are mainly based on the dependency relations between words, since their effectiveness has been validated by existing rule based approaches (Zhuang et al., 2006;Qiu et al., 2011).


This paper uses statistics on development set due to its compact size.

we also compared our methods with the top systems in TAC-KBP 2017 Evaluation.


(2017), the probability P (y j |x i ; θ) of classifying instance x i into the wrong class y j can be directly regarded as the mislabeling risk of that instance.

the srcb system in English used additional CRF based models to deal with multi-word triggers in English, which is not considered in our model and leads to a significant higher recall than other competitors.


We consider event extraction as a two-stage task, which includes trigger extraction and argument extraction, and propose a Pre-trained Language Model based Event Extractor (PLMEE).

the input of the trigger extractor follows the BERt, i.e.


Our base model (RnnOIE, § 2) performs better than non-neural systems, confirming the advantage of supervised training under the sequence labeling setting.

"Wrong argument" is where extracted arguments do not coincide with ground truth, which is mainly caused by merging multiple arguments in ground truth into one.


This paper focuses on Relation Extraction (RE), which is the task of entity mention detection and classifying the relations between each pair of those mentions.

only relations between exactly two arguments and where the two pairs (span i, span j) and (span j, span i) are considered different.


span i is defined by all the tokens from START(i) to END(i) inclusive, for 1 ≤ i ≤ N .

with existing token-level models: our model is able to use span-specific features, allows for overlapping entity mentions and does not use sequential decoding.


We propose two completely neural network architectures for nested nested named entity recognition which do not explicitly build or model any structure and infer the relationships between nested NEs implicitly: • In the first model, we concatenate the nested entity multiple labels into one multilabel, which is then predicted with a standard LSTM-CRF (Lample et al., 2016) model.

sequence-to-sequence (seq2seq): The encoder is a bi-directional LsTM and the decoder is a LsTM.


PFAs have some conditions on their structure.

building off of our previous work, we have considered the problem of incrementally computing the infix probabilities of each prefix of a given string.


These problems find use in many natural language processing tasks such word prediction and text error correction.

finally, each state must be accessible and co-accessible.


The simplicity of these methods allow them to apply to many NLP sequence tasks and various neural model architectures.

whether the word is the Right/Left/None-most dependent of its governor.


Although the main reason for this is because of the performance of the K&K parser, the important point is that our proposed approximation method enables us to use the K&K parser for the nonlocal dependency identification task.

although post-processing approach can use any parser in pre-processing, our approach outperforms the post-processing approach, even if the pre-processing parser is assumed to always generate gold PTB trees.


We observed in preliminary experiments, as shown in Table 1, that casting the problem as 3task learning led to worse results.

these methods are complex and intended for statistical parsers.


By contrast, in soft attention mechanism, a probability distribution which reflects the importance of tokens is calculated over each token of the input sequence.

furthermore, we introduce a novel trick to reduce the high variance in training brought by REINfORCE method.


In the prism module, the hidden size of BiLSTM is the same as in CNN-BiLSTM-CRF architecture.


More precisely, define the set Z D to contain all the unique label type sequences appearing as a subsequence of some sequence y (m) ∈ D. Then, if we're willing to tolerate some errors in reconstructing y, we can use a dynamic program to minimize the number of mislabelings in our now "prediction"ŷ, plus the number of distinct segments used in formingŷ multiplied by a constant c, as follows: where J(0) = 0 is the base case and |z| is the length of sequence z.

in Figure 3 we plot both the F 1 score and the average number of distinct segments used in predicting eachŷ against the c parameter from the dynamic program above, for the CoNLL 2003 NER development data in both the standard and zeroshot settings.


In Figure 3 we plot both the F 1 score and the average number of distinct segments used in predicting eachŷ against the c parameter from the dynamic program above, for the CoNLL 2003 NER development data in both the standard and zeroshot settings.

we consider three zero-shot transfer scenarios: training with Universal POS Tags on the Penn Treebank and then predicting the standard, fine-grained POS tags, training on the CoNLL 2003 NER data and predicting on the fine-grained OntoNotes NER data (Hovy et al., 2006) using the setup of Strubell et al.


not the case of the Reddit conversation data used for pretraining, and these domain candidates may be better suited to empathetic responding than generic conversation utterances.

as shown in Figure 3, the distribution of emotion label prompts is close to evenly distributed, with a few that are selected slightly more/less often.


Instead of using unstructured text knowledge, CCM (Zhou et al., 2018) relies on structured knowledge to generate rich-information response.

these results demonstrate that our method is consistently and significantly better than the other state-of-the-art approaches.


Finally, compared to message-length-restricted Twitter conversations (Ritter et al., 2010), Reddit conversations tend to be more natural.

first, in-domain and Reddit data can be fused into a single training set: besides expensive retraining for each task, the disbalance between in-domain and Reddit data sizes effectively erases the target task signal.


We stop training early when perplexity on the held-out validation set had increased monotonically for two epochs.

they do not need to be placed on the ground or on top of another block, but can be put anywhere as long as one of their sides touches another block.


The improved performance shows the effectiveness of position embedding for the images in a multimodal dialogue setting.

as opposed to a static image in VQa, MMD deals with dynamic images making the task even more challenging.


SDEN: The model described in Bapna et al.

popular contextual SLU models Bapna et al., 2017) exploit the dialogue history with the memory network (Weston et al., 2014), which covers all three main stages of memory process: encoding (write), storage (save) and retrieval (read) (Baddeley, 1976).


5, where we test each set D p i ∈ D test by selecting one dialogue and training with all the others.


While in document tasks sentences are the default semantic modeling unit at the coarse-grained level, utterances might be a more suitable counterpart in spoken dialogues, as dialogues often consist of incomplete sentences.

then the segment indices of turns were labeled for turn-level context collection.


The hybrid information can benefit the intent detection task.

the ID subnet can takes a new r slot and exports a new r inte .


Assuming that we have two spaces, the semantics space X and the natural language space Y, given n data pairs {(x i , y i )} n i=1 , the goal of NLG is to generate corresponding utterances based on given semantics.

table 1: the NLU performance reported on micro-F1 and the NLG performance reported on BLEU, ROUGE-1, ROUGE-2, and ROUGE-L of models (%).


Table 2 shows the experimental results of the slot-independent SUMBT model on MultiWOZ corpus.

the slot-independent SUMBT which learned the shared information from all across domains and slots significantly outperformed those baselines, resulting in 91.0% joint accuracy.


Identifying the unknown (novel) user intents that have never appeared in the training set is a challenging task in the dialogue system.

second, we introduce margin loss on BiLsTM to learn discriminative deep features, which is suitable for the detection task.


The reason is that softmax loss only focuses on whether the sample is correctly classi- fied, and does not require intra-class compactness and inter-class separation.

to begin with, we use BiLStM (Mesnil et al., 2015) to train the intent classifier and use it as feature extractor.


u t is the t-th utterance, h ct encodes context information up to time t. For the inter-utterance representation, we follow the way proposed by Park et al.

in practice, we usually need to understand the meaning of utterances and capture their semantic dependency, not just word-level alignments (Luo et al., 2018).


CSRR employs hierarchical latent variables based on VAEs to represent the meaning of utterances and meanwhile learns the relationship between query and response.

second, because of the existence of discourse-level and pair-level latent variables, responses are more coherent.


What distinguishes us from previous work is that we stochastically sample global structures and do it in a differentiable fashion.

see the hyperparameters in Appendix A.2.


The resulting dependencies represent two kinds of relation: operand to argument and operand to closing parenthesis (Figure 2).

intuitively, using GCNs yields a form of intra-attention.


Another option is to use the differentiable surrogate for both passes (Forward relaxed).

mC sampling may have a better exploration of the search space but it is slower to converge.


Finally, the attention vector is derived by combining the context vector with the current target hidden state ℎ .

finally, conclusions and future directions are drawn in Section 5.


(2019) proposed an aggressive inference network training.

for text modeling, researchers have shown that SGD performs better than other adaptive optimization methods such as Adam (Wilson et al., 2017;Keskar and Socher, 2017).


Partial-input baselines are valuable sanity checks for datasets, but as we illustrate, their implications should be understood carefully.

for example, partial-input baselines are used to identify the "hard" examples in SNLI (Gururangan et al., 2018), verify that SQuAD is well constructed (Kaushik and Lipton, 2018), and that SWAG is challenging (Zellers et al., 2018).


Existing augmentation methods generate new training samples by replacing one word in the original sentences with another word (Wang et al., 2018;Kobayashi, 2018;Xie et al., 2017;Fadaee et al., 2017).

we leave the exploration of leveraging monolingual data to future work.


The aggregation layer takes in the domain specific and shared features as inputs ( Figure  2), aggregates them and concatenates these aggregated vectors to form a combined representation.

others have proposed a domain discriminator that maximizes the domain classification loss between source and target domains (Cohen et al., 2018;Shah et al., 2018).


It turns out that in all but one case, the more words, the better the performance.

in this paper, we proposed a data-driven approach for extracting class descriptions for jointly training text with their class descriptors, based on pure statistical tests.


In future work, we will further explore an efficient strategy that can jointly train all modules of the deep model with minimal increase in training complexity.

we use beam search with a beam size of 5 and length penalty 0.6 for both tasks.


The overall framework is illustrated in Figure 2.

their methods are specially designed for the recurrent architecture which has been significantly outperformed by the state-ofthe-art transformer model.


We propose two variants of MHA, namely a black-box MHA (b-MHA) and a white-box MHA (w-MHA).

but the fluency of examples generated by genetic attack is still not satisfactory and it is inefficient as the gradient is discarded.


Prediction: Contradiction Genetic: empties trash cans are sitting on a beach.

in contrast to previous language generation models using M-H, b-MHA's stationary distribution is equipped with a language model term and an adversarial attacking term.


The novel idea behind our hybrid generative-discriminative method is to explicitly capture the fine-grained information inferred from raw texts, utilizing the information to help interpret the predicted classification results and improve the overall performance.

5 Experiments and Analysis As the goal of this study is to propose an explanation framework, in order to test the effectiveness of proposed GEF, we use the same experimental settings on the base model and on the base model+GEF.


The hyperparameter settings for both datasets are listed in Table 3.

there is a significant shortcoming of this generating process: it fails to build strong connections between the generative explanations and the predicted overall results.


It is also effective under other attack classes, and can mitigate the adversarial effect in wordpiece models by 21%, character-only models by 19%, and in word, and word+char models by over 4.5% .

these gains do not consistently translate in all other settings, as lower WER is necessary but not sufficient.


To be noted, for all these normalization methods, we can take various loss functions.

a further step of normalization is needed to form the final score of an output structure.


Such an expert may also anticipate important cues from the client.

γ = 1 is the best for both the model C C and F C , while γ = 0 is the best for C T and γ = 3 for F T .


text-only or response selection settings (Zhu et al., 2018;Mazaré et al., 2018;Dinan et al., 2018), These approaches consider the knowledge to be grounded in text, whereas in VGDS, the knowledge is grounded in videos (with multimodal sources of information).

all of these methods are restricted to answering specific queries, and do not maintain a dialogue context, unlike what we aim to achieve in VGDS.


We then augment the data by automatically extracting keywords of each utterance.

as noted above, there is typically no data available for direct learning of such a strategy.


Turkers can  also choose "no preference" if the conversations are equally good or bad.

there is a strong demand to enable the integration of goals and strategy into opendomain dialogue systems, and it imposes challenges to both: first, how to define the goal for an open-domain chat system; and second, how to encode dialogue strategy into the response production process.


The conversations cover a broad range of topics such as work, family, personal interest, etc; and the discussion topics change frequently during the course of the conversations.

we ask a human turker to converse with the Kernel agent and a randomly selected comparison agent in the same test case.


For the full annotation scheme, please refer to https://gitlab.com/ucdavisnlp/ persuasionforgood.

once the scheme was finalized, each coder separately coded the rest of the conversations.


For example, the sentence "Even five cents can help save children's life."

the brief description of user persona lacks quantitative analysis of users' sociodemographic backgrounds and psychological characteristics, and therefore is not sufficient for interaction effect analysis between personalities and dialogue policy preference.


It is interesting to note that for Emotion appeal, the average negative sentiment score is 0.22, higher than the average positive sentiment score, 0.10.

we identified 10 persuasion strategy categories that can be divided into two types, 1) persuasive appeal and 2) persuasive inquiry.


In this scenario, the entropy of individual utterances is computed based on the clustered data.

some additional techniques did not yield meaningful improvement and were excluded from further evaluation.


For these it seems straightforward to maximize the conditional probability P (T |S), S and T denoting a specific source and target utterance.

widely used as a simple engagement indicator (Serban et al., 2017b;Tandon et al., 2017;Baheti et al., 2018).


2012learns multi-prototype embeddings by clustering the context window features of a word.

the second column indicates the inter annotator agreement -the first number is the fraction of questions for which at least 2 annotators agreed and the second indicates the fraction on which all three agreed.


For the baseline methods, they all use the same pre-trained word embeddings as our model and their hyper-parameters are tuned to the best on the validation set.

2, the sememes of a word (or an MWE) can finely depict the semantics of the word (MWE).


(3) MUL still performs the worst, which is consistent with the results of the last experiment.

a constituent's meaning may vary with the other constituent, and accordingly, the sememes of a constituent should have different weights when the constituent is combined with different constituents (we show an example in later case study).


The benefits of including word and bigram frequency also point to possible further improvements using n-grams, tf-idf, or other task-agnostic linguistic features.

for example, if assigning scores between zero and Money is the Root of All Evil.


This, combined with the study's unusual evaluation metrics, precludes a meaningful comparison with Shahaf et al.

the ability to assess the degree of humour embodied in an utterance may be necessary for the agent to make a contextually appropriate, humanlike response -for example, a groan for a terrible joke, a chuckle for a middling one, or uproarious laughter for a clever one.


The first task is to probe for S-classes.

we lowercase the dataset to match the vocabularies of GLOVE(6B), FASTTEXT(wiki) and our embeddings.


but there is a surprising gap regarding collocation modeling.


However, they are compositionally less rigid than some other types of multiword expressions such as, e.g., idioms (as, e.g., [to] kick the bucket) or multiword lexical units (as, e.g., President of the United States or chief inspector).

the second variant, referred to as rvAE, starts from the same high-dimensional relation vector, but then uses a conditional autoencoder to obtain a lower-dimensional and potentially higher-quality 300d vector 6 .


All these verb variants (perfective, imperfective, reflexive) are presented as different entries in RuWordNet.

but two words toplenyi and mindalnyi cannot be considered as sense-related words.


), we need to delete the character in red in order to guarantee the correctness of the sentence.

to reduce the searching space while ensuring that the target characters are not excluded, we build a confusionset matrix M ∈ R n * w , where w is the size of the vocabulary, n corresponds to the number of characters in X, in which each element is 0 or 1.


Note that during training the ground truth c t j−1 is fed into the network to predict c t j , while at test time the most probableĉ t j−1 is used.

utility of M by comparing the experimental results of Ours − and Ours + , we can observe that the latter achieves better performance, which validates the effectiveness of utilizing Confusionsets that can help improve the probability of generating correct target characters.


Modified Initialization As pointed out in Lample et al.

created data will be referred to asŜ m A )B .


Raganato and Tiedemann (2018) use the self-attention weights of the Transformer's encoder to induce a tree structure for each sentence and compute the unlabeled attachment score of these trees.

translation pairs were batched together by approximate sequence length.


In previous approaches, to include words in the target sentence for which the translation is unknown the token unk is often used which can be handled by later steps.

it is not possible to always copy when the translation is unknown.


We assume ε cannot be applied with the sequence pair (s, t) if s = x, then we have the transition function δ as follows, where s • x represents concatenating a sequence s and a word x.

to provide this information explicitly, we embed the number of previous delay tokens to a vector and add this to the sum of the word embedding and position embedding as the input of the decoder.


Details of other hyperparameters can be found in Appendix A.1.

multilingual NmT has led to impressive gains in translation accuracy of low-resource languages (LRL) (Zoph et al., 2016;Firat et al., 2016;Gu et al., 2018;Neubig and Hu, 2018;Nguyen and Chiang, 2018).


Using the adversarially learned representation, de-identification models reach an F 1 score of 97.4%, which is close to the non-private system (97.67%).

the representation allows training a de-identification model while being robust to adversaries trying to re-identify protected information or building a lookup table of representations.


A number of approaches have been proposed for nested NER.

following the unfold ops, a positional encoding P of dimension e (we use a learned encoding) is added, giving tensor I s : I s is then passed through the Embed Update layer.


When pretraining using a LM objective, models are typically trained until model fit stops improving (i.e., perplexity on validation data).

other work has suggest that random initializations have a significant effect on the failure cases of BERT, pointing to the brittleness of fine-tuning (Phang et al., 2018).


We next describe an alternative approach.

learning (DRel) A seminal DRel model, from which we start our survey, is Structural Correspondence learning (SCl) (Blitzer et al., 2006(Blitzer et al., , 2007 that introduced the idea of pivotbased DRel.


(Pentina et al., 2015;Oh et al., 2015;Gong et al., 2016;Zhang et al., 2017)).

in this paper we propose a Task Refinement Learning (TRL) approach, in order to solve these problems.


After the model converges we compute for each of the 500 pivots its 10 nearest neighbor and compute the percentage of these neighbors that belong to the same class, PosPiv or NegPiv, as the pivot.

while in CL the prediction task is fixed but the trained algorithm is exposed to increasingly more complex training examples in subsequent stages, in TRL the algorithm is trained to solve increasingly more complex tasks in subsequent stages, but the training data is kept fixed across the stages.


t ∈ R Dv , and a salience s There are two components to the model: the memory unit, which stores and tracks the states of the entities in the text; and the recurrent unit, which controls the memory via a set of gates.

the reader also learns from the GAP supervision that pronouns are important, and therefore stores the pronoun his even though it is also a possessive determiner.


We only provide a higher-level description of the method; for a complete derivation and the theory justifying the algorithm we refer the reader to the works by Hsu et al.

to Equation 2, which uses the longest context x of length n to compute the conditional probability, the interpolated model leverages the ability of the PNFA to model substring expectations of all lengths up to n. This is similar to classic interpolation of language models (Rosenfeld, 1994;Chen, 2009).


It can be challenging to train multi-task neural networks that outperform or even match their single-task counterparts.

the difference with Single→Multi was not statistically significant, suggesting there is little value in multiple rounds of distillation.


For content and fluency, we compute the average score across all 5 raters for each item, and average those scores to get a final rating for each model, such that higher content and fluency scores are better.

we note that while these measures are common for machine translation, they are not well-suited to this task, since they are based on ngram overlap which is not a constraint within the model; we include them for comparative purposes.


The types of semantic errors the models make are more relevant than how well they conform to test references.

these models suffer from two critical bottlenecks: (1) a data bottleneck, i.e.


We also use METEOR as a metric, whose results are more closed to a normal distribution (Dobre, 2015).

training with multi-task learning seems to hurt the overall performances a little.


With the development of deep learning, data-driven methods using neural networks are proposed to produce comments with high quality and flexibility.

apart from the text generation models, there are three crucial modules in our approach: the internal chess engine, the move encoder, and the multichoices encoder.


The dataset is available for download at: https://rloganiv.github.io/linked-wikitext-2.

this work was supported in part by Allen Institute of Artificial Intelligence (AI2), and in part by NSF award #IIS-1817183.


(6)), our model failed to learn a meaningful output and only learned to generate a single word for any combination of input sentence and style.

by contrast, in the ablation study, we trained our model with only one of them.


they are all the positive reviews for a specific product), and we refer this shared characteristic as the style of these sentences.

it is not easy to disentangle the stylistic property from the semantics of a sentence.


Variational auto-encoders (VAEs) are widely used in natural language generation due to the regularization of the latent space.

word embeddings are 300-dimensional and initialized randomly.


Table 3: Performance of paraphrase generation.

the above approaches cannot be applied to syntactic modeling in VAE's continuous latent space, and thus, we do not enjoy the two benefits of VAE, namely, sampling and manipulation, towards the syntax of a sentence.


Contextualized representation approaches allow to compute the embedding of a given token depending on the context it appears in, as opposed to the fixed, context-free vectors provided by GloVe, therefore allowing to capture more information for OOV tokens.

ours: what type of disease did chopin have ?


Clustering: K-Means clustering is performed on the sentence embeddings with compositional ngram features (Pagliardini et al., 2018) on each step of the recipe in U. Aligning with our intu-ition, when k is 3, it is observed that these clusters roughly indicate categories of desserts, drinks and main course foods (pizza, quesadilla etc,).

(2018) attained the highest scores on human readability in this task by attending to both global and local contexts.


In SSiD model, for each step in the recipe, we identify which phase it is in using the clustering model and use the phase sequence to decode state transitions from the FSM.

ssiL model seems to generate recipe that is conditioned on the structure of that particular example.


For lexical constraints, we identified words with a PMI score above the threshold θ.

to improve the conservative rewriting of the paraphrase generation model, we proposed the identification of words to be paraphrased and the addition of negative lexical constraints on beam search.


They used words in reference sentences as positive or negative constraints and succeeded in generating diverse paraphrases via machine translation.

of a detailed analysis of the model outputs, our PMI method always improves the Base method in terms of Add and Del in both model architectures.


For example, for Informal→Formal style transfer, GEC performs well because most of its edits improve the sentences' grammaticality and make the sentences become more formal; in contrast, for Formal→Informal style transfer, GEC improves sentences' grammaticality but affects their styles, making them become less informal.

(2019) and use the GEC system to edit their results.


We also conduct experiments on the WMT17 Automatic Post-Editing (APE) task.

for sentence compression, BLEU decreases from 60.38 to 58.77 after GEC post editing.


Sensitivity to such paraphrasing needs to be improved for better reliability of QA models on unseen test questions.

to previous work, we introduce a more guided approach to generate diverse paraphrases, by using a paraphrase suggestion together with a source question to generate a paraphrased question.


There are strong theoretical arguments as to why a better fusion of information retrieval and machine comprehension should be benefi-cial.

analogous to prior work, we use the test splits of CuratedTREC, WikiMovies, and WebQuestions, as well as the development split for SQuaD, though only for the final evaluation.


We conduct a series of experiments to evaluate our RankQA system.

number of occurrences both  Let x i ∈ R d be the d-dimensional feature vector for the answer candidate c i , i = 1, .


the original performance increase can only be achieved when all features are included.


This approach, named "RankQA", overcomes the limitations of a two-stage process in the status quo whereby both stages operate largely in isolation and where information from the two is never properly fused.

(2018) utilize neural networks in order to filter noisy text passages, while Kratzwald and Feuerriegel (2018) forward a query-specific number of text passages.


An additional penalty is used to regularize the parameters and prevent the network from overfitting.

we account for the different characteristics of the datasets and thus employ task-specific models that are trained separately on every dataset.


Being able to infer the semantics from under-specified language is what sets QA apart from traditional IR.

a random sentence (pseudo-query) and its context (pseudo evidence text) are derived from the text snippet: "...Zebras have four gaits: walk, trot, canter and gallop.


, S N , this approach independently computes answer i and y none i from each paragraph S i , where answer i and y none i denote the answer candidate from ith paragraph and the score indicating ith paragraph does not contain the answer.

q Which pizza chain has locations in more cities, Round  Table Pizza has locations in how many cities?


Abduction is a long studied task in AI, where normally, both the observation (hypothesis) and the domain knowledge (known fact) is represented in a formal language from which a logical solver abduces possible explanations (missing knowledge).

table 3: Compares (a) the number of correct facts that appears across any four passages (b) the number of correct facts that appears in the passage of the correct hypothesis (c) the accuracy for tF-IDF, BERt model trained on StS-B dataset and BERt model trained on OpenBook dataset.


Please note that because different relations may correspond to the different number of samples in the test set, the micro average accuracy may be affected by the distribution of unseen relations in the test set.

the relation detection step selects the correct relation (marked with bold font) from a set of candidate relations.


the SimpleQuestion dataset, 99% of the relations in the test set also exist in the training data, which means their embeddings could be learned well during training.


Each sample in SQ includes a human annotated question and the corresponding knowledge triple.

we re-organize the SimpleQuestion dataset as SimpleQuestion-Balance to evaluate the performance for seen and unseen relations, separately.


Some attempts have been made to build machine translation systems for language pairs without direct parallel data, where they relying on one or more other languages as the pivot (Firat et al., 2016;Ha et al., 2016;Chen et al., 2017).

less study has been done in relation detection.


We re-organize the SimpleQuestion dataset as SimpleQuestion-Balance to reveal and evaluate the problem, and propose an adapter which significantly improves the results.

for those relations which are never seen in the training data (called unseen relations), their embeddings have never been trained since initialization.


We do not assume a global knowledge base.

question answering (qA) has been a popular topic in natural language processing.


In the literature, such multi-hop reading comprehension tasks are frequently solved via end-to-end differentiable (deep learning) models (Sukhbaatar et al., 2015;Peng et al., 2015;Seo et al., 2016b;Raison et al., 2018;Henaff et al., 2016;Kumar et al., 2016;Graves et al., 2016;Dhingra et al., 2018).

for simplicity, we assume that there is only one correct answer per example, but an adaptation to multiple correct answers would be straight-forward, e.g.


In contrast, rule-based models are easily interpretable, naturally produce explanations for their decisions, and can generalise from smaller quantities of data.

we conjecture that this might be due to NLPROLOG's reliance on explicit reasoning, which could make it less susceptible to spurious correlations between the query and supporting text.


Each natural language statement is first translated into a triple, where the first and third element denote the entities involved in the sentence, and the second element denotes the textual surface pattern connecting the entities.

in such models, explaining the reasoning steps leading to an answer and interpreting the model parameters to extrapolate new knowledge is a very challenging task (Doshi-Velez and Kim, 2017;Lipton, 2018;Guidotti et al., 2019).


For example, as illustrated in Figure 2, the word "does" in question span attends to the words in all the quantity spans.

the SEQ2SEQ model 1 https://github.com/lijierui/ group-attention takes the text of the whole problem as the input and corresponding equation as the output.


Question-Related Attention: The question can also derive distinguishing information such as whether the answer value is positive.


Ideally, we want models to be able to reason that "What color is the rose?

in this paper, we propose evaluation for QA systems that measures the extent to which model predictions are consistent.


The results of this approach are promising: consistency evaluation reveals gaps in models, and augmenting training data produces models that are more consistent even in human generated implications.

if (q, a) → (q , a ), workers should be able to answer q correctly even in the absence of the image or paragraph.


these questions usually have complicated linguistic phenomena, such as co-reference, ellipsis and so on.


It should be pointed out that the new parameters introduced in the proposed approach are jointly updated with NMT model parameters in an endto-end manner.

rNN Intuitively, we can treat G = {g 1 , .


Topics related to business, social issues, and tribal and electoral politics were more likely to see code switching into Naijá.

in modeling whether a comment will switch, we show that topic-driven variation, tribal affiliation, emotional valence, and audience design all play complementary roles in behavior.


Second, we develop a new classifier for distinguishing Naijá and English, identifying over 24K cases of code switching.

the effect sizes of each hypotheses variables differed substantially, pointing to the complexity of code switching behavior.


Our method outperforms the baseline on top-N sizes smaller than or equal to 300.

the top-N selection part of the search has been accelerated using hashing methods to avoid a full sort (Shi et al., 2018;Pagh and Rodler, 2004).


To see this, note that no swap can ever decrease the value of one of the D[n].

for example, Volta V100 contain 64 cores per SM, while Geforce GTX 1080s contain 128 cores per SM.


Unlike in-Algorithm 2 Sparse matrix multiplication using the CSR format.

furthermore, because for each k, we compare C[k] with every element of D, it must be the case that after looping over all n (line 4), we have C[k] ≤ D[n] for all n. when the algorithm finishes, D contains the top-N values.


2009 and section 4.2), suggesting that our BipSkip method performs in a manner comparable to the SCA method for automatic cognate detection (List, 2014), which is based on pairwise sequence comparison methods using improved sound class models and alignment techniques.

markov Chain monte Carlo (mCmC) methods are used to estimate the posterior probability of Ψ.


In the future, we would like to more faithfully capture the semantics of documents by explicitly modeling entities and their linking.

we obtain alignment weights via multi-attention: where β align km = P (a k = m|h y k , [h x 0 : h xm ]), i.e., the probabilistic distribution over alignments from sentences to the kth prediction in the decoder, where a k = m denotes the kth prediction aligned to the mth sentence.


Aspect-based summarization has previously been considered in the customer feedback domain (Hu and Liu, 2004;Zhuang et al., 2006;Titov and Mc-Donald, 2008;Lu et al., 2009;Zhu et al., 2009), where a typical system discovers a set of relevant aspects (product properties), and extracts sentiment and information along those aspects.

[...] the seven-times wimbledon champion, who has not played since the semifinals [...] sport seven-times wimbledon champion could make a return to the court at the end of april in the german open .


The token replacement method, on the other hand, replaces the identity terms with a token that is surrounded with neutral words in the embedding space, so it results in greater improvement on the synthetic dataset.

(2018) highlight the bias in toxic comment classifier models originating from the dataset.


From a fairness angle, our technique shares similarities with adversarial training (Zhang et al., 2018a;Madras et al., 2018) in asking the model to optimize for an additional objective that transitively unbiases the classifier.

the lack of actionability angle is often overlooked.


In this section, we present our Concept Interaction Graph (CIG) to represent a document as an undirected weighted graph, which decomposes a document into subsets of sentences, each subset focusing on a different concept.

the key difference is that our method XI compares a pair of articles over a CIG in per-vertex decomposed fashion.


Let c A (v) and c B (v) denote the context vectors obtained for S A (v) and S B (v), respectively.

furthermore, in GCN hidden representations of a size between 16 and 128 yield good performance.


• Matching by a large-scale pre-training language model: BERT (Devlin et al., 2018).

our model significantly outperforms BERT.


By using overlapping community detection, each keyword may appear in multiple concepts.

the task of classifying whether two news articles are reporting the same breaking news event or belong to the same story is critical to news apps and search engines for clustering, redundancy removal and topic summarization.


To 1 https://www.tira.io/task/authorship-verification/ get to the bottom of this rather baffling outcome, we carried out a systematic analysis of the entire evaluation chain, its problem definition, its corpora, its evaluation procedure, and of course our model, in search of any sources of bias that may have artificially inflated the performance of our approach.

1 reality caught up with us when we applied our verifier to other authorship verification problems with little success.


Even experienced investors may be confused, however, if the candidates are 6.9 and 7.

spithourakis and Riedel (2018) used language models to predict numerals in clinical and scientific datasets.


R@K leads to excessive penalization when documents have more than K gold labels.

6 5 See Appendix C for a more detailed discussion on the evaluation measures.


in such cases, control may not be a desired property of interactive systems.


from a training batch (we use dynamic size training batches), more data points are actually created for the word embedding task because of large number of (context, pivot) pairs.


We select the single top document retrieved against a sentence t i because a sentence is a weak representation of information need.

hence, the size of the vocabulary, |RC ∪ T C|, is much larger compared to TC and it enables the model to access RC.


SST in (Buch et al., 2017) effectively generates proposals in a single pass.

the Ours (Video Only) model considers video content without transcripts in the procedure extraction module.


The input embedding of each time step is the concatenation of: 1.

the labels of 210 testing videos are unpublished, we can only adopt the training and validation dataset for our experiment.


The use of POS is also found to be generally helpful.

because these datasets rely on videos uploaded on YouTube, it has been observed they can be potentially biased towards unusual situations (Kay et al., 2017).


In order to alleviate this shortcoming, we propose to use multitask learning (MTL) and exploit transcribed speech within the end-to-end visuallygrounded setting, and thus combine some features of both the pipeline and end-to-end approaches.

within Natural Language Processing (NLP), Luong et al.


Additionally, we see that the inclusion of a text-specific speech encoder S2T degrades performance: compare for exam-ple row 6 to 7, and row 11 to 12.

the linear mappings I2S and I2t project 4096 dimensions down to 1024.


Each question-answer pair in the VisDial dataset is accompanied by a list of 100 answer candidates, and the goal is to find the correct answer among all the candidates.

where The function SumPooling(x, k) in (6) means using a onedimensional non-overlapped window with the size k to perform sum pooling over x.


4): Finally, as in the vanilla Transformer decoder, the third sub-layer is a feed-forward network that processes the representation for the next n+1 layer: The three sources of information (image, object labels, and web entity labels) are treated symmetrically in the above equations.

among the 3 sub-layers, we modify the encoderattention sub-layer by introducing a mechanism that combines information coming from different encoders.


These input representations are constructed by summing up a trainable segment embedding with the subtoken embeddings and the type embedding (Fig.

the labels-only baseline and the proposal of Lu et al.


See Section 5 for a description of how we split the data into training, validation, and test sets.

across all workers, the maximum, median, and minimum numbers of tasks finished by a worker are 1,419, 3, and 1, respectively.


In this work, we propose a goal-driven collaborative task that combines language, perception, and action.

the proposed CoDraw game is related to several well-known language and vision tasks that study grounded language understanding (Karpathy and Fei-Fei, 2015;Donahue et al., 2015;de Vries et al., 2016).


Popular tasks include image captioning, visual question answering (VQA), and visual question generation.

a 2-layer MLP with sigmoid function is employed.


We evaluate the performance of VIFIDEL against human judgments on two popular IDG datasets ( §4).

we now expand the basic version of VIFIDEL to use (one or more) human references when avail-able.


We see that, as the number of objects increase from one to two, the VIFIDEL score also increases, given that these objects are also mentioned in the system caption.

in fact, they often focus on a subset of the image content.


Our proposal is similar to CIDEr in that we capture consensus information given a set of human references.

in this paper, we explore image information in the form of explicit object detections, both using gold or predicted object instances for a given image.


(2018) used hierarchical attention strategies to learn multimodal representations, Wang et al.

we hypothesize such performance drop occurs because the asynchronous (and much longer) data streams introduce more difficulty in recognizing important features and computing the appropriate attention.


We choose the self-attention transformer (Vaswani et al., 2017).

the lengths of each modality vary significantly, where audio and vision sequences may contain up to > 1, 000 time steps.


Here, {v p } P p=1 are extracted from an intermediate layer of a CNN, C and p are the number of channels and the position index of v p .

to better understand CMAS's ability in detecting abnormalities, we report its precision and average False Positive Rate (FPR) for abnormality term detection in table 3 and table 4.


After examining an area, Planner (PL) determines: 1) whether to terminate the generation process; 2) which writer should generate the next sentence.

table 3 shows that CMAS-RL obtains the highest precision and the lowest average FPR on both datasets, indicating the advantage of CMAS-RL for detecting abnormalities.


The image encoder is pretrained on ImageNet (Deng et al., 2009)).

we propose a single layer LSTM with soft-attention (Xu et al., 2015) as GSE.


when comparing among machine-edited stories (row y and |), among pre-and post-edited stories (row z and }), or among any combinations of them (row~, and ), all metrics result in weak correlations with human judgments.


The goal is to detect a generic <subject, predicate, object> triplet present in an image.

it also aims to find the subset of relational words R ✓ .


On the other hand, while image captions are easier to obtain, they are often not ⇤ Equal Contribution completely descriptive of an image (Krishna et al., 2017).

note that our model is at a significant disadvantage when compared to supervised approaches.


For some methods, we show that there is a sharp increase in the size of the compact representation for a small increase in the size of the grammar, which gives us a strong indication of how difficult it would be to apply the method to much larger grammars.

1 Assume we have a list of subautomata A1 ..... Am that is ordered from lower-level to higher-level automata; i.e., if an automaton Ap occurs as the label of a transition of automaton Aq, then p < q; Am must be the start symbol S. This order is a natural result of the way that subautomata are constructed during our depth-first traversal of the grammar, which is actually postorder in the sense that a subautomaton is output after all subautomata occurring at its transitions have been output.


We also need parameters P(q0, ql]w, v) for the probability of choosing a head transition (qo, ql, w,v,O,O) given this pair of headwords.

(We have also tried using automatic word-clustering techniques to merge states further, but for the limited domain corpora we have used so far, the results are inconclusive.)


The comments of the anonymous FSMNLP and CL reviewers were extremely useful.

the advantage of this approach is that the subset construction algorithm can remain simple because the input automaton is c-free.


We shall revisit these approaches in Section 6.4.

the rule compiler then compiles all the rules into another machine.


Since an ntuple contains n-1 lexical elements (the first element is the surface representation), the lexicon component consists of n -1 sublexica.

a flexible formalism that leaves the grammar writer with ample room to choose the appropriate linguistic theory for an application.


A Bradford Book • Language, Speech, and Communication series 524 pp., 111 illus.

this book, written by leading experts in the field, brings together linguistic, psychological, and computational perspectives on some of the fundamental issues.


This phenomenon is modeled in IGEN by having a special word #<w:IT> whose meaning can range over the various items constituting the weather; this word is constrained to appear only as the subject of an appropriate verb.

this system, however, starts with a list of information to be expressed and merely arranges it into a coherent pattern; it is thus not a planner in the sense used here (as Hovy makes clear).


That's not surprising; after all, the options that should be used have all been removed from IGEN's vocabulary.

this contrasts with the English construction, in which it is the crucial element and can be used with a range of verbs.


The goal of convincing the user to take his umbrella, though, produces the output: (25) I1 pleut.

this strategic decision (that WATER can be omitted) is encoded permanently into the tactical component; there is no way for the strategic component to control this decision.


As a first step in the syntactic definition of a language, the set of categories or types is stated.

structural ambiguity in G can be appreciated, for instance, in relation to the granularity of graphical objects, as the same drawing will have different syntactic analysis depending on whether it is interpreted as a whole or as an aggregation of parts.


The syntactic and semantic structures of the three languages permit expression and interpretation of information in each of the modalities, and the ability to systematically find correlated expressions in different modalities with the same semantic values.

the definition of this geometrical interpreter will allow us to perform inferences about the geometry of the drawing in a very effective fashion.


To conclude this section, we believe the formalization of the syntax and semantics of graphical representations in a form compatible with the syntax and semantics of natural language, as in the scheme in Figure 3, may be a point of departure for investigating how the graphical or visual context helps to resolve natural language ambiguities at different levels of representation and processing.

if the expression is supported by an overt gesture indicating the city of Paris, for instance, it would be deictic and its interpretation would have to be worked out with the same machinery; although in this latter situation the translation relation between graphical and linguistic referents could be asserted directly in the translation model as the gesture would render unnecessary the constraint satisfaction part of the deictic inference.


The cursor itself is modeled as a graphical object of type dot within the graphical domain.

the natural language expression of Figure 2 can be constructed in a compositional fashion.


In this second view, the context of interpretation of a natural language term is defined as a set of expressions of a graphical language with well-defined syntax and semantics.

when one is instructed to interpret a multimodal message, like Figures 1 and 2, not all information in the scheme of Figure 3 is available.


~+ and ,klinks(u,v) correspond to the two peaks of the distribution cooc (u,v), which is illustrated in Figure 2.

a third advantage is that statistical models can provide more accurate information about the relative importance of different translations.


I shall describe empty sequences as consisting of a special NULL word, so that each word sequence will contain exactly one word and can be treated as a scalar.

's (1993b) Models 4 and 5, would be less beneficial with noisier training bitexts or for language pairs with less similar word order.


Many researchers have proposed greedy algorithms for estimating nonprobabilistic word-to-word translation models, also known as translation lexicons (e.g., Catizone, Russell, and Warwick 1989;Gale and Church 1991;Fung 1995;Kumano and Hirakawa 1994;Melamed 1995;Wu and Xia 1994).

so, the probability of a correct match for a rare word is proportional to the fraction of words around it that can be linked with higher confidence.


This kind of literature has a pedagogical role to play in theoretical linguistics and card-carrying syntacticians should read the book.

the general argument developed in the second chapter is somewhat awkward: the authors want to show that the three main economy principles proposed by Chomsky (Procrastinate, the Smallest Derivation Principle, and "Have an Effect on Output") are unnecessary.


does MOOSE ever produce an infelicitous one?

it is unclear whether the computational flow of information embodied in the architecture in Stede's Figure 8.1 can remain unchanged when tackling less closely related languages.


To say the least, this is not a book for the mathematically faint of heart.

m6nnich demonstrates how to use a kind of "macro grammar" to define tree languages in mSO for the non-context-free phenomena that one can find in natural languages by replacing the normal substitution process that rewriting systems assume in their notion of derivation by a higher-order variant in which operations such as composition can be explicitly represented as objects in the macro language.


The third is a collection of evidence from Dutch and Swiss German (Huybregts 1976(Huybregts , 1984Shieber 1985) for the non-context-freeness of at least some natural languages.

this mode of characterization, which is subsumed by an area of research called descriptive complexity theory, is at once more declarative than an automaton or rewriting system, more flexible in terms of the primitive relations or concepts that it can provide resort to, and less wedded to the tacit, not at all unproblematic, assumption that the right way to view any language, including natural language, is as a set of strings.


The second is Rogers's landmark thesis (Rogers 1994), which demonstrated that significant portions of Government-and-Binding Theory (GB) can be very transparently expressed in MSO.

while MSO may be a very elegant choice, it does not quite appear to be a sufficient choice.


Only recently, however, does it seem that a sufficient quantity of this information is electronically available to produce a widespread need for automatic summarization.

after this is a section on evaluation, which contains papers that cover well the issues of this important area, with perhaps a slight let-down at the end.


This conviction that the wider SFG framework has major advantages, so significant that they outweigh the identified problems, should be respected as deeply challenging for HPSG theory.

the terminology used within SFG theory sometimes appears to be unnecessarily daunting.


After identifying some fundamental problems in SFG theory, the author explains that these problems have been fully solved in HPSG.

teich's book will be of interest to three groups of readers.


We therefore devised an extraction system for the full word frequency range.

any low-frequency word is singled out as a significant item whenever it occurs at least once in the window.


Of the following two sentences, repeated from (10), the second is more acceptable:  For helpful remarks I thank Gabriel Bes, anonymous reviewers, and various audiences, and especially for recent comment, Ewan Klein and Mark Steedman.

this has semantics, after some simplification, as in (44), which is equivalent to (39).


The preposition by projects an agentive adverbial phrase; is is a functor over (post)nominal modifiers (the man outside, John is outside, etc.)

~xVy(love y x) b. Vy3x(love y x) while the dominant reading of (7a) is (8a), that of (7b) is (8b), i.e., the preference is for the first quantifier to have wider scope.


As shown, all frequent DA types saw improvement, but the highest wins were observed for typically short DAs, such as ANSWERS and BACKCHANNELS.

(1997) was to simultaneously segment and tag DAs, whereas the other results rely on a prior manual segmentation.


A DA represents the meaning of an utterance at the level of illocutionary force (Austin 1962).

here we investigate the interaction of prosodic models with the dialogue grammar and the word-based DA models discussed above.


For example, by definition DECLARATIVE-QUESTIONS are not marked by syntax (e.g., by subject-auxiliary inversion) and are thus confusable with STATEMENTS and OPINIONS.

our labeling effort also showed that content-and task-related distinctions will always play an important role in effective DA labeling.


This increases the effectiveness of full-text search and helps to provide users with better feedback.

the search strategy we use is a variant of the stack algorithm for sequential decoding (Anderson and Mohan 1984).


However, if exclusion was exploited, both encoder and decoder will recognize that escape from order 1 to order 0 is inevitable because the order 1 model adds no characters that were not already predicted by the order 2 model.

first, some errors are obviously caused by deficiencies in the training data, such as improperly segmented common words and names.


Analogous failures occur with Chinese text.


The search strategy we use is a variant of the stack algorithm for sequential decoding (Anderson and Mohan 1984).

the third and fourth characters taken together mean 'middle school,' which does not occur in the meaning of the sentence.


Once the discourse-marker-based algorithm has hypothesized all relations it could, a word-co-occurrence-based algorithm examines every sentence/paragraph boundary for which a marker-based rhetorical relation has not been hypothesized and uses cohesion to produce such a hypothesis.

the distribution of relations inferred by the program is somewhat similar, with the most frequently used relations being JOINt~ ELABORAtION, and CONtRASt as well.


The preference for discourse trees that are skewed to the right is also problematic when handling texts that start by providing some background information or by motivating the reader before presenting the main idea.

the first set, {SAtELLItE_NUCLEUS, AFtER, CLAUSE, CONCESSION, 1, -1}, specifies that the marker signals a rhetorical relation of CONCESSION that holds between two clause-like units.


This is consistent with the information presented in Table 6, which specifies that a rhetorical relation of EXAMPLE holds between a satellite, the sentence that contains the marker, and a nucleus, the sentence that went before.

in naturally occurring conversations, so marks the terminal point of a main discourse unit and a potential transition in a participant's turn, whereas and coordinates idea units and continues a speaker's action (Schiffrin 1987).


Finally, my rhetorical parser uses only surface-form methods for determining the markers and textual units and uses clause-like units as the minimal units of the discourse trees.

according to the results derived from the corpus analysis, the use of the discourse marker although at the beginning of a sentence or paragraph correlates with a rhetorical relation of ELaBORaTION that holds between a satellite, the sentence or paragraph that contains the marker, and a nucleus, the sentence or paragraph that precedes it.


Unfortunately, cohesion is not enough for determining this relation.

just consider, for instance, the highly ambiguous connective and.


The analysts were supposed to use only the set of relations proposed by RST and the relation TEXTUAL to link the subtrees subsuming the title and body of a text.

the discourse structures built for sentences that have clause-like units as leaves are correct in most of these cases.


• The feature Distance to salient unit is given by the maximum of the values of the database field Distance to salient unit of the instances that were consistent with the discourse usage being considered.

my rhetorical parser uses a mathematical model in which this ambiguity is acknowledged and appropriately treated.


The first factor concerns our representation choice.

hence, to derive the rhetorical structure of text (4) it is sufficient to hypothesize with respect to the occurrence of the connective the exclusively disjunctive hypothesis rhet_rel(CONTRAST, 1, 3) (9 rhet_rel(CONTRAST, 1, 4) (9 rhet_rel(CONTRAST, 2, 3) (grhet_rel (coNTRAST,2,4), because this hypothesis subsumes all the other possible rhetorical relations that may be signaled by the connective.


We would like to thank the anonymous CL reviewers for their valuable and insightful comments.

it is clear that satisfactory accuracy can be achieved with only 10 training texts.


In general, these measures are not computationally expensive.

they can be easily incorporated into a real-time application.


The same procedure was followed based on the VR, CWF-30, and CWF-50 approaches.

it is set to "skip" or surrender attempts to parse clauses after reaching a time-out threshold.


The major components and assumptions of the approach to contextual interpretation here are as follows: .

any alternative robust parsing system could be plugged in instead.


All three supervised learning methods highly prioritized the perfect indicator.

table 1 sun~narizes the three aspectual distinctions, which compose five aspectual categories.


This representation enables complex interactions between indicator values.

in addition to this, surprisingly, we discovered a decision tree-like rule encoded by these weights, as described below in Section 5.1.4.


As shown above, learning methods successfully produced models that were specialized for the classification task.

the guidelines described above in Section 5.1 were used in order to test for fundamental aspectual class.


(10) I stared at it for 10 minutes (culminated process).

a system translating from a language without explicit perfective markers, such as English, to one with explicit perfective markers must first detect the aspectual category of an input phrase in order to determine the form of the output (Stys 1991;Dorr 1992).


The remaining five indicators were discovered during the course of this research.

we can focus our efforts on the remaining clauses.


A comparison between human markers for this test was performed over a different corpus, and is reported below in Section 5.2.1.

the same set of 14 indicators were combined in different ways, successfully improving classification performance for both stativity and completedness, and revealing linguistic insights for each.


The TD says that "two markables should be recorded as coreferential if the text asserts them to be coreferential at ANY TIME" (Hirschman and Chinchor 1997, page 11).

finding all the NPs that might participate in coreference becomes even harder if the annotation scheme is extended to cover event coreference (noted in the "wish list" in Section 1.4 of the TD) since it is often extremely difficult to determine which events can serve as antecedents (Hirschman and Chinchor 1997): Be careful not to get the gel in your eyes.