The input of the task is a sequence of L positive integers x 0:T −1 and the output is simply x x 0 .

the output is chosen from the input sequence, with index determined by x 0 .


At the same time, the accuracy is on par with that of LSTM.

the combination of (R, N ) = (7, 4) even achieves slightly better accuracy than LSTM while having a 1.5x speedup.


The performance of LSTM-Jump is superior to LSTM in terms of both accuracy and efficiency under all settings in our experiments.

the fastest LSTM-Jump configuration achieves a remarkable 6x speedup over LSTM, while also having respectively 1.4% and 4.4% higher accuracy in Children's Book Test -Named Entity and Children's Book Test -Common Noun.


In this paper, we focus on learning how to skim text for fast reading.

we propose a "jumping" model that after reading every few tokens, it decides how many tokens should be skipped by sampling from a softmax.


Given two feature extractors f 1 , f 2 ∈ F, feature addition (denoted by +) produces a feature extractor f 1 + f 2 that adds up the images of f 1 and f 2 .

for any example x ∈ X, we have For example, the feature extractor w + w -1 will map the word alone to a vector that is one for the basis elements w=alone and w -1 =went.


Multiplication distributes over addition on both sides.

for any x, y, z ∈ K, we The additive identity is an annihilating element with respect to multiplication.


That is, for any x, y, z ∈ K, we The additive identity is an annihilating element with respect to multiplication.

for any x ∈ K, we have x ⊗ 0 = 0 = 0 ⊗ x.


Next, recall from §4 that each variable x i can take values from a finite set A i .

if a base feature extractor f i corresponds to the variable x i , then, we define x i 's domain to be the set each variable can either be the bias feature extractor or the feature extractor associated with it.


Second, as we will see, feature extractors farther from the root will appear in inner terms in the factorized form.

frequent or more expensive feature extractors should be incentivized to appear higher in the tree.


The message from B to D has to marginalize out the variable x 3 .

we have The summation is over the domain of x 3 , namely {1, w -2 }.


Our word-level decoder (see Figure 4) differs from the standard sequential decoder described in § 2 in that it takes the chunk representations (c) k as input: In a standard sequential decoder, the hidden state iterates over the length of a target sentence and then generates an end-of-sentence token.

its hidden layers are required to memorize the long-term dependencies and orders in the target language.


This means that the last state and the last output of a chunk are not being fed into the word-level decoder at the next time step (see the black part in Figure 4).

18is always initialized before generating the first word in a chunk.


4   chunking errors affect the translation quality, we removed extremely long chunks from the training data.

among the 2 million preprocessed translation pairs, we excluded sentence pairs that matched any of following conditions: (1) The length of the source sentence or target sentence is larger than 64 (3% of whole data); 2The maximum length of a chunk in the target sentence is larger than 8 (14% of whole data); and 3The maximum number of chunks in the target sentence is larger than 20 (3% of whole data).


Hierarchical RNNs are used for various NLP tasks such as machine translation (Luong and Manning, 2016), document modeling Lin et al., 2015), dialog generation (Serban et al., 2017), image captioning (Krause et al., 2016), and video captioning (Yu et al., 2016).

and Luong and Manning (2016) use hierarchical encoder-decoder models, but not for the purpose of learning syntactic structures of target sentences.


This is part of the for-mulation of the conditional GRU and is described further in §2.2.

the encoder is a bi-directional RNN with GRU and the decoder is an RNN with a conditional GRU.


We use publicly available pre-trained CNNs for image feature extraction.

we extract spatial image features for all images in our dataset using the 50-layer Residual network (ResNet-50) of .


The word-sampling method can also be applied to zero-resource NMT with a small source-pivot corpus.

the size of the source-pivot corpus is orders of magnitude smaller than that of the pivot-target corpus.


Although the bottom-up tree encoder can take advantage of syntactic structure, the learned representation of a node is based on its subtree only; it contains no information from higher up in the tree.

the representation of leaf nodes is still the sequential one.


In our case, the prior knowledge is the source syntactic information.

we build our model on top of the word coverage model proposed by Tu et al.


We start with a very high threshold enabling precision of 100% and lower the threshold until recall of 100% is reached.

we sort the test examples by score in descending order and then go down the list of scores in order to complete the entire precision-recall curve.


The setting considered in the current paper is much more challenging since it does not use bilingual dictionaries or word alignments from bitexts.

the setting in the current paper is the one mentioned as future work on page 663 of (Bergsma and Kondrak, 2007): "we plan to investigate approaches that do not re-   quire the bilingual dictionaries or bitexts to generate training data."


We first apply a narrow convolution between C and a filter F ∈ R d×n of width n to obtain a feature map f ∈ R k−n+1 .

the computation of the j-th element of f is defined as where A, B = Tr(AB T ) is the Frobenius inner product and b is a bias.


To get a clearer picture, we measured perplexity for only specific subsets of words in our test data: specifically, given target word w i , we measure perplexity of word w i+1 .

we analyze the perplexities when the inflected words of interest are in the most recent history, exploiting the recency bias of our LSTM-LM.


Note that If we choose W = U Σ, we exactly obtain w 1 , w 2 = s(w 1 , w 2 ), since W W = XX in this case.

the cosine similarity of the embeddings w 1 , w 2 coincides with the intuitive similarity s(w 1 , w 2 ).


Note that the gradient ∇F (X i ) can be naturally considered as a matrix in R n×m .

point X i + ∇F (X i ) leaves the manifold M d , because its rank is generally greater than d. why Riemannian optimization methods The standard Riemannian gradient method first projects the gradient step onto the tangent space at the current point X i and then retracts it back to the manifold: where R is the retraction operator, and p T M is the projection onto the tangent space.


In this section, we present our framework Sememe-Encoded WRL (SE-WRL) that considers sememe information for word sense disambiguation and representation learning.

we learn our models on a large-scale text corpus with the semantic regularization of the sememe annotation in HowNet and obtain sememe, sense and word embeddings for evaluation tasks.


SAC utilizes the attention scheme to automatically select appropriate senses for context words according to the target word.

sAC conducts word sense disambiguation for context words to learn better representations of target words.


In this paper, we propose a novel method to model sememe information for learning better word representations.

we utilize sememe information to represent various senses of each word and propose Sememe Attention to select appropriate senses in contexts automatically.


Before delving into the details of our model, we first describe a dataset we constructed to examine the ability of our model to capture the compositional characteristics of characters.

the dataset must satisfy two desiderata: (1) it must be necessary to fully utilize each character in the input in order to achieve high accuracy, and (2) there must be enough regularity and com-2060 10 0 10 1 10 2 10 3 10 4 10 6 10 0 10 1 10 2 10 3 10 4 10 5 ⎯⎯ Chinese ⎯⎯"Japanese ⎯⎯ Korean Rank < 20% Freq.


Late Fusion Instead of learning a joint representation like early fusion, late fusion averages the model predictions.

it takes the output of the softmax layers from both models and averages the probabilities to create a final distribution used to make the prediction.


Emphasis of the VISUAL Model In order to delve deeper into what the VISUAL model has learned, we measure a modified version of the occlusion sensitivity proposed by Zeiler and Fergus (2014) by masking the original character image in four ways, and examine the importance of each part of the character to the model's calculated representations.

we leave only the upper half, bottom half, left half, or right half of the image, and mask the remainder with white pixels since Chinese characters are usually formed by combining two radicals vertically or horizontally.


(2015) introduced a heterogeneous dataset, Chinese Net-Bank, by pretraining word embeddings.

they learn an LSTM RNN model based on NetBank first, then initialize a new model with the pretrained embeddings obtained from NetBank, and then train it on CPB.


Therefore, we also need to deal with domain adaptation problem when using heterogeneous data.

the proposed approach should be robust to harmful features learned on incompatible datasets.


We wish to learn this embedding function such that sentences with high semantic similarity have high cosine similarity in the embedding space.

we focus on the setting of Wieting et al.


For simplicity we use the mini-batch for this set, but it could be a different set.

we choose t 1 for a given s 1 , s 2 as follows: where S b ⊆ S is the current mini-batch.


That is, we choose t 1 for a given s 1 , s 2 as follows: where S b ⊆ S is the current mini-batch.

we want to choose a negative example t i that is similar to s i according to the current model.


However, since we use a bi-LSTM, the model essentially has two RNNs, and accordingly we have two context vectors, and associated attentions.

context f (i, t) = lstm in(t i−1 ) for the forward RNN and context b (i, t) = lstm in(t i+1 ) for the backward RNN.


In formal semantics, modification is modeled as function application.

let M H be a class label consisting of a head H, which we assume to be a common noun, preceded by a modifier M .


They are denoted by the noun (Heim and Kratzer, 1998): The interpretation of a modifier M is a function that maps between sets of entities.

modifiers select a subset 3 of the input set: This formalization leaves open how one decides whether or not "e satisfies M ".


Second, the modifier is a function that can be applied in a truth-theoretic setting.

applying "1950s" to the set of "musicians" returns exactly the set of "1950s musicians".


Instead, we follow Paşca (2015) and take advantage of facts involving distributionally similar modifiers.

rather than looking only at tuples in D in which the object matches M , we consider all tuples, but discount the weight proportionally to the similarity between M and the object of the tuple.


As a baseline compositional model, we augment the Hearst baseline via set intersection.


Parsing for deep dependency representations can be viewed as the search for Maximum Subgraphs for a certain graph class G (Kuhlmann and Jonsson, 2015), a generalization of the MST perspective for tree parsing.

we have the following optimization problem: Given an arc-weighted graph The choice of G determines the computational complexity of dependency parsing.


Though theoretically a 1EC graph is not necessarily P2, the empirical evaluation demonstrates the high overlap of them on linguistic annotations.

almost all 1EC deep dependency graphs are P2.


During the search for maximal 1EC/P2 graphs, we can freely delete bad edges whose scores are negative.

we figure out some edges, in each construction step, which can be created without violating either 1EC or P2 restriction.


The graphical illustration of our algorithm uses undirected graphs 1 .

we use e (i,j) to include the discussion about both a (i,j) and a (j,i) .


In this paper, we explore the strength of the graphbased approach.

we enhance the Maximum Subgraph model with new parsing algorithms for 1EC/P2 graphs.


In this work, we focus on the analysis and evaluation of θ, because θ is a model of the quality indicators of a summary, and thus crucial in order to understand the properties of "good" summaries.

we compare θ functions of different summarizers by measuring the correlation of their θ functions with human judgments.


General purpose optimization techniques are especially appealing, because they offer a decoupling of θ and O and allow investigating complex θ functions without making any assumption on their mathematical properties.

this supports future work on identifying an "optimal" θ as a model of relevant quality aspects of a summary.


We implement a scheduling mechanism that cycles through all languages during training, one language at a time.

model parameters are updated after one batch from one language before moving to the next one.


Then, we perform a simple concatenation of N context vectors: Unlike the single-source setting, the training data consists of N -way parallel sentencesemantics pairs.

each training instance consists of N semantically equivalent sentences and their corresponding logical form.


The approach to RTE taken in this paper follows the Lexical Overlap Hypothesis (LOH), which states that the higher the number of lexical matches between T and H, the more likely the T-H pair is entailing rather than non-entailing (Zeller, 2016).

h is more likely to be entailed by T if most of its lexical content also occurs in T. While this hypothesis is a simplification of the TE problem, it has been shown to perform reasonably well for some datasets .


For ZvdP2016, most errors can be attributed to oversplitting.

25 out of its 37 (67.5%) misclassifications compared to FF2010 can be attributed to this problem.


At each time step, the action sLSTM predicts the next action based on the (current) hidden states of the buffer, stack and action sLSTM.

where W a is the vector of the action a.


2 Second, we let the next word prediction of the translation decoder as a generator of RNNG.

the generator of RNNG will output a word, when asked by the shift action, according to the conditional distribution defined by the translation decoder in Eq.


A major challenge in training the proposed hybrid model is that there is not a parallel corpus augmented with gold-standard target-side parse, and vice versa.

we must either parse the target-side sentences of an existing parallel corpus or translate sentences with existing goldstandard parses.


On the other hand, what we  134,453 2,656 2,999 (33,867,27,347,313 2,169 2,999 (33,820,30,684,492 2,818 2,998 (32,442,27,979,82) Jp-En 100,000 1, 790 1,812 (23,509, 28,591, 80)  propose here is a special case of distant supervision in that the external parser provides noisy annotations to otherwise an unlabeled training set.

we use SyntaxNet, released by Andor et al.


We observe a similar pattern as that observed in the goodness of fit setting, with a shift in the best performing distribution from Bernoulli (which simply models if a feature exists or not) toward something more Gaussian (Poisson or Normal) as we move along from message-level to county-level analysis and from unigrams to lexica.

note that at higher levels of analysis (at user and county levels) as the distribution of features becomes closer to Normal, modeling features as Bernoulli is clearly sub-optimal where as at the message level modeling unigrams as a Bernoulli is superior.


We also note that similar to arc-eager transitions, arc-swift transitions must also satisfy certain pre-conditions.

an RArc[k] transition requires that the top k − 1 elements in the stack are already attached; LArc[k] additionally requires that the k-th element is unattached, resulting in no more than one feasible LArc candidate for any parser state.


Since the parse tree is defined by a sequence of transition actions a, we write p(x, y) as p(x, a).

3 The joint distribution p(x, a) is factorized into a sequence of transition probabilities and terminal probabilities (when actions are GEN), which are parametrized by a transitional state embedding u: where I is an indicator function and u t represents the state embedding at time step t. the conditional probability of the next action is: where a t represents the action embedding at time step t, A the action space and b a the bias.


To satisfy the independence assumptions imposed by the generative model, u t uses only a restricted set of features defined over the output buffer and the stack -we consider p(a) as a context insensitive prior distribution.

we use the following features: 1) the stack embedding d t which encodes the stack of the decoder and is obtained with a stack-LSTM (Dyer et al., 2015(Dyer et al., , 2016; 2) the output buffer embedding o t ; we use a standard LSTM to compose the output buffer and o t is represented as the most recent state of the LSTM; and 3) the parent non-terminal embedding n t which is accessible in the generative model because the RNNG employs a depth-first generation order.


(2016), and the LSTM language modeling generative parser (LM) of Choe and Charniak (2016).

we present and use a beam-based search procedure with an augmented state space that can search directly in the generative models, allowing us to explore A → A for these generative parsers A independent of any base parsers.


Levy and Goldberg (2014) showed that the function achieves its global maximal value when for each word-pair (x, y) the inner product of the embedding vectors x • y is equal to pmi(x, y).

they showed that S(m) ≤ S(pmi) for every matrix m. We next show that the value of the function S(m) at its maximum point, the PMI matrix, has a concrete interpretation, namely it is exactly the Jensen-Shannon Mutual Information (JSMI) between words and their contexts.


Hence, as in the second term, to compute the third term of S(m) (8) for the case of m = pmi, we can sum only over the positive pairs (x, y) that actually appear in the corpus.

2 for the special case m = pmi, it is feasible to compute the exact score (8) and not just its approximation (9) that is based on negative sampling.


We also consider ROUGE (Lin, 2004), a recall oriented metric for summarization evaluation.

rOUGE-L, a variation based on longest common subsequence, is computed by treating claim as reference and each candidate sentence as sample summary.


Results in Table 3 show that using composite features with argument type information (Comp(type, Sen) + Comp(type, Simi)) can improve the ranking performance.

the best performance is achieved by adding composite features to sentence features, similarity features, and ngram features.


The intuition is that, if the context is less relevant, we should mainly focus on the query itself, but if the context is relevant, we should focus more evenly across context and the query.

our explicitly weighting approach could be viewed as heuristic attention.


We propose to build discourse segmenters without making any data assumptions.

rather than segmenting sentences, our systems segment documents directly.


An approach that is perhaps better aligned with the across-topics setting is to notice that in detailing one's arguments, one tends to utilize a specially structured discourse, and that discourse role could provide a clue to the argumentative function of a sentence, without reliance on what the sentence is actually saying (beyond discourse connectives that are used to help identify the discourse role).

argumentative essays often have a fairly standard structure, where a general claim (or stance, or thesis) on the issue is introduced in the beginning of the essay, followed by a sequence of main points, each elaborated using supporting statements, and finally followed by a conclusion that often re-states the thesis and provides a high-level summary of the argument.


Our results do not generally answer the question of what view of argumentation quality is preferable, but they clarify where theory can learn from practice and vice versa.

practical approaches indicate what to focus on to simplify theory, whereas theory seems beneficial to guide quality assessment in practice.


This way, we can correlate each dimension with all reason labels in Table 2 including Conv.

we compute Kendall's τ based on all argument pairs given for each label.


(2017a) on CrowdFlower in order to evaluate whether lay annotators suffice for a theory-based quality assessment.

we asked the crowd to rate the same 304 arguments as the experts for all 15 given quality dimensions with scores from 1 to 3 (or choose "cannot judge").


RST is a descriptive framework that has been widely used in the analysis of discourse organization of written texts (Taboada and Mann, 2006b), and has been applied to various natural language processing tasks, including language generation, text summarization, and machine translation (Taboada and Mann, 2006a).

the availability of RST annotations on a selection of 385 Wall Street Journal articles from the Penn Treebank 1 ) has facilitated RST-based discourse analysis of written texts, since it provides a standard benchmark for comparing the performance of different techniques for document-level discourse parsing (Joty et al., 2013;.


We introduce a simple and effective method to learn discourse-specific word embeddings (DSWE) for implicit discourse relation recognition.

dSWE is learned by performing connective classification on massive explicit discourse data, and capable of capturing discourse relationships between words.


We adapt the neural network model described in Figure 1 as the classifier for implicit discourse relation recognition (CDRR).

we concatenate some surface features with the last hidden layer as the input of the softmax layer to predict discourse relations.


Finally, we conduct experiments to investigate the impact of connectives used in training DSWE on our results.

we use the explicit discourse instances with the top 10, 20, 30, 60 most frequent or all connectives to learn DSWE, accounting for 78.9%, 91.9%, 95.8%, 99.4% or 100% of total instances, respectively.


However, system summaries cannot achieve ROUGE=1 since summarization systems cannot reproduce reference summaries in most cases.

the maximum ROUGE score that can be achieved by compressive summarization is unclear.


As Table 3 shows, our model trained on atomic data outperforms the baseline in all but one project with an average gain of 5 BLEU points.

we observe bigger gains for java projects such as CoreNLP and guava.


Despite the small data increase, we also trained our model on full datasets as a way to confirm the generative power of our model.

we wanted to test the model is able leverage on atomic data to also capture and compress multifile changes.


We propose novel semantic features for English event detection derived from automatic translations into thirteen languages.

we explore the use of Cangjie 1 radicals in Chinese and Japanese.


S contains 5 ⇥ 10 10 tokens (280 GB of plain text) extracted from Web pages as well as around 80,000 sentences from various domain-targeted sources.

for each test set, we use the corresponding training questions Q tr to retrieve domain-relevant sentences from S. for each multiplechoice question (q, A) 2 Q tr and each choice a 2 A, we use all non-stopword stemmed tokens in q and a as an ElasticSearch 1 query against S. We take the top 200 hits, run Open IE v4, 2 and aggregate the resulting tuples over all a 2 A and over all questions in Q tr to create the tuple KB (T ).


At the cost of restricting the permissible type sets slightly, we can speed up the greedy procedure further.

we can require that the predicted type set T be connected in some constraint graph over the types-either the co-occurrence graph, the complete graph, or the graph underlying the type system.


Table 1 describes these rules (see column "Our mapping"), which were hand-written by consulting the annotation guidelines of each of the corpora.

in converting FactBank we take only the author's perspective annotations as these comply with the annotations of the other corpora, and for MEANTIME we use their proposed abstraction into factual, uncertain and possible (in the future).


This analysis reveals that all corpora are significantly skewed towards the factual end of the scale, where the majority of the annotation mass is located.

we find that MEANTIME is especially biased, assigning the factual value (+3) to 90% of its event annotations.


Our hope was that this would improve performance by focusing on the more semantically-significant portions of the predicate-argument structure.

we extracted the following features from the predicted AMR structures: immediate parents, grandparents and siblings of the target node, lemma and POS tag of the target and preceding token in the sentence, and a Boolean feature based on the AMR polarity role (indicating semantic negation).


As a result, gradients near these points are non-informative and scheduled sampling has difficulty assigning credit for errors.

the gradient does not provide information useful in distinguishing between local errors without future consequences and cascading errors which are more serious.


While scheduled sampling  is an effective way to rectify exposure bias, it cannot differentiate between cascading errors, which can lead to a sequence of bad decisions, and local errors, which have more benign effects.

scheduled sampling focuses on learning optimal behavior in the current step given the fixed decoding decision of the previous step.


We propose a deep network, which not only achieves competitive accuracy for text classification, but also exhibits compositional behavior.

while creating hierarchical representations of a piece of text, such as a sentence, the lower layers of the network distribute their layer-specific attention weights to individual words.


We propose an Attention Gated Transformation (AGT) network, where each layer's feature generation is gated by a layer-specific attention mechanism (Bahdanau et al., 2014).

through distributing its attention to the original given text, each layer of the networks tends to in-crementally retrieve new words and phrases from the original text.


Our AGT method adds the right two columns of Figure 1.

1) the transform gate T l now is not a function of y l−1 , but a function of the selection vector s + l , which is determined by the attention distributed to the given input x by the l th layer (will be discussed next), and 2) the function f takes as input the concatenation of y l−1 and s + l to create feature representation y l .


Having built one s + for each layer from the given text x, the activation of the transform gate for layer l (l > 0) (i.e., Equation 3) is calculated: To generate feature representation y l , the function f takes as input the concatenation of y l−1 and s + l .

equation 2 becomes: where [...;...] denotes concatenation.


Note that, Equation 8 indicates that the representation y l depends on both s + l and y l−1 .

although Equation 7 states that the gate activation at layer l is computed by s + l , the gate activation is also affected by y l−1 , which embeds the information from the layers below l. Intuitively, the AGT networks are encouraged to consider new words/phrases from the input text at higher layers.


The figure indicates that AGT generated very spiky attention distribution.

most of the attention weights are either 1 or 0.


Empirically, we observed that, if with only s + 0 in the AGT network, namely removing s + i for i > 0, the test-set accuracy dropped from 50.5% to 48.5%.

transforming a linear combination of the bag-of-words features was insufficient for obtaining sufficient accuracy for the classification task.


Since the same training data is used for NMT, SMT and neural system combination, we further design a smart strategy to simulate the real training data for neural system combination.

we make the following contributions in this paper: • We propose a neural system combination method, which is adapted from multi-source NMT model and can accommodate both source inputs and different system translations.


We note that at the same hash size, features with a larger space perform worse.

with a 2 12 hash size, CHAR4 outperforms 5-and 6-grams.


We filter out any sentence segment whose corrected version differs from it by more than one token due to segmentation issue.

we only focus on the cases in which the error can be corrected by replacing one single token.


We propose another hit rate measure which takes the segment length (len) into account.

we regard one test instance as correct if the answer is ranked within the top max(1, len * r% ) candidates.


Our computational approach to compositor attribution operates on the sources of evidence that have been considered by bibliographers.

we focus on jointly modeling patterns of orthographic variation and spacing preferences across pages of a document, treating compositor assignments as latent variables in a generative model.


Here, f (m, d) is a feature function defined on modern word m paired with diplomatic word d, while w c is a weight vector corresponding to compositor c. To capture word-specific preferences we add an indicator feature for each pair of modern word m and diplomatic spelling d. We refer to these as WORD features below.

to capture general orthographic preferences we introduce an additional set of features based on the edit operations involved in the computation of Levenshtein distance between m and d. each operation is added as a separate feature, both with and without local context (previous or next character of the modern word).


Manual analysis of spacing has revealed differences across pages.

the choice of spaced or non-spaced punctuation marks is hypothesized by biobliographers to be indicative of compositor preference and specific typecase.


This is possibly a result of the reduced reliability of full word forms due to mistakes in OCR.

interesting is the result that spacing, rarely a factor considered in NLP models, improves the accuracy significantly for our system when compared with EDIT features alone.


Recently, a few caption datasets in languages other than English have been constructed (Miyazaki and Shimizu, 2016;Grubinger et al., 2006;Elliott et al., 2016).

the study of Miyazaki and Shimizu (2016) is closest to the present study.


This method consists of a convolutional neural network (CNN) and long short-term memory (LSTM)3.

cNN first extracts features from a given image, and then, LSTM generates a caption from the extracted features.


The aim of this experiment is to show the necessity of a Japanese caption dataset.

we evaluate quantitatively and qualitatively how fluent Japanese captions can be generated by using a neural network-based caption generation model trained on STAIR Captions.


They obtained 221 statements from CHANNEL 4 2 and POLITIFACT.COM 3 , a Pulitzer Prize-winning website.

politiFact covers a widerange of political topics, and they provide detailed judgments with fine-grained labels.


job, home state, and credit history are also provided.

the credit history includes the historical counts of inaccurate statements for each speaker.


As a soft alternative to this, we can directly include the topic distributions estimated by HDP for each document (Figure 1(c)).

for each update, we use the topic distribution to compute a weighted sum over the word-topic representations (r 2 ): where T is the total number of topics, d i the document containing w i , and ppτ k |d i q the probability assigned to topic τ k by HDP in document d i .


In this paper, we focus on Arabic word embeddings.

we provide a thorough evaluation of the quality of four Arabic word embeddings that have been generated by previous work (Zahran et al., 2015;Al-Rfou et al., 2013).


Note that we consider a question to be answered wrongly if at least one of the words in the question are not present in the word embeddings.

we take into consideration the coverage of the embeddings as well (Gao et al., 2014).


A graph is converted to tree structures before constructing a RNN for each tree.

we construct a tree T = (V T , E T ) of depth d rooted at v t using a breadth-first search algorithm from G. V T and E T are the sets of vertices and edges in the tree.


Interestingly, LR and LP are effective methods for Gender and Age compared to TADW and TriDNR.

tADW performs poorly on Gender, and triDNR marginally outperforms GLStMN on the dense social network, Usertype.


This is because the existing methods can not assign multiple labels to each instance, or are difficult to generate virtual users and access their information (e.g., age and gender).

the following steps have been performed.


The results of the random and average ones over STS are similar, because C = 2 on STS.

our model performs better than baselines in aggregating noisy labels, especially when the noise scale becomes large.


In our approach, we use an attentive Seq2Seq rerank model to optimize the joint results.

for a question, we first use an IR model to retrieve a set of QA pairs and use them as candidate answers, and then rerank the candidate answers using an attentive Seq2Seq model: if the top candidate has a score higher than a certain threshold, it will be taken as the answer; otherwise the answer will be offered by a generation based model (see Fig.


Our rerank model uses the same attentive Seq2Seq model to score candidate answers with regarding to an input question.

we choose mean probability, denoted as s Mean-Prob in Eqn.


In our experiments, the batch approach helps to save 41% of the processing time when comparing with the asynchronous way.

more than 75% of questions take less than 150ms with rerank and less than 200ms with generation.


It is worth noting that pretraining on SQuAD-T (Table 2) yields 0.5 point lower MAP than pretraining on 50% of SQuAD.

roughly speaking, span-level supervision data is worth more than twice the size of sentence-level supervision data for the purpose of pretraining.


In this paper, we propose to alternatively use Silver Standard, i.e., a high-quality automatic annotated data, to train the crowd workers.

we introduce a self-training strategy for crowd-sourcing, where the workers are first trained with simpler examples (which we assume to be less noisy) and then gradually presented with more difficult ones.


The main idea is (i) to automatically select a subset of less-noisy examples applying an automatic classifier, (ii) training the annotators with such subset, and (iii) iterating this process after retraining the classifiers using the annotated data.

the educated crowd workers can provide higher quality annotations, which can be used by the system in the next iteration to improve the quality of its classification.


That is, the educated crowd workers can provide higher quality annotations, which can be used by the system in the next iteration to improve the quality of its classification.

this cycle gradually improves both system and human annotators.


To select CS I , we train a classifier C on DS, and then used it to label CS examples.

we used MultiR framework (Hoffmann et al., 2011) to train C, as it is a widely used framework for RE.


In the future, it would be interesting to study if our method generalizes to other difficult or even simpler tasks.

our approach opens up many research directions on how to best train workers or best select data for them, similarly to what active learning methods have been doing for training machines.


The generative story of our model is as follows: (1) generate the dialogue act of the current dialogue turn conditioned on the previous dialogue act and the previous utterance P Θ Θ Θ (z t |z t−1 , y t−1 ); and (2) generate the current utterance conditioned on the previous utterance and the current dialogue act P Θ Θ Θ (y t |z t , y t−1 ).

p Θ Θ Θ (z 1:T , y 1:T ) is decomposed as: Furthermore, each utterance is generated by a sequential process whereby each token y t,n is conditioned on all the previously generated tokens y t,<n , as well as the external conditioning context consisting of the dialogue act z t and the previous turn's utterance y t−1 , i.e., (2) Importantly, the decomposition of the joint distribution in Equation 1 allows dynamic programming for exact decoding ( §2.2).


For prediction, we choose the sequence of dialogue acts with the highest posterior probability: Since the joint probability is decomposed further according to Equation 1, we can make use of dynamic programming to find the highest probability sequence of dialogue acts.

the model endows each latent variable z t with a unary potential P Θ Θ Θ (y t |z t , y t−1 ) and binary potential P Θ Θ Θ (z t |z t−1 , y t−1 ) functions.


Intuitively, p(t | w) indicates how much the word w focuses on topic t. 1 At the next step of TPR, the word scores (2) are accumulated into keyphrase scores.

for each topic t, a candidate keyphrase is ranked by the sum of the word scores By combining the topic specific keyphrase scores R t (phrase) with the probability p(t | d) derived from the LDA we can compute the final keyphrase scores across all K topics: Single Topical PageRank (STPR) was recently proposed by Sterckx et al.


Depending on the downstream applications, it is often useful to be able to control the balance between these two competing principles.

sometimes keyphrases with high topic specificity (i.e., phrases that are representative exclusively for certain topics) are more appropriate, while other times keyphrases with high corpus specificity (i.e., phrases that are representative of the corpus as a whole) are more appropriate.


To complement the quantitative results in Table 3, Table 4 presents a concrete example, showing that varying α can lead to qualitative changes in the top ranked keyphrases.

when α = 0 the corpus specificity of the keyphrases SR extracts is high.


After that, in the entity disambiguation phase, we utilize the selected mentions as a bridge between uninformative entities and other mentions.

we comparing features of a non-seed mention to those of seed mentions of its entity candidates to determine which entity it should be linked to.


Due to the lack of annotated data, we approach this problem by performing clustering, which works in an unsupervised fashion.

we represent features (described later in this section) of each mention as a vector and measure the distance between two mentions using cosine distance.


Following the proven effectiveness of Support Vector Machine (SVM) by numerous experiments on text classification tasks, we adopt the use of linear SVM for NLI.

we use linear SVM implemented by scikit-learn package (Pedregosa et al., 2011) to perform the multi-class classification.


It seems when sentences are fed in reverse order, the model can more easily detect the unexpected word order associated with the reparandum to detect disfluencies.

that the disfluency is observed "after" the fluent repair in a backward language model is helpful for recognizing disfluencies.


In this section, we extend this technique further, and con-sider training HolE solely in the frequency domain.

real-valued embeddings e k , w r ∈ R n in the original "time" domain are abolished, and instead we train their DFT counterparts ε k = F(e k ) ∈ C n and ω k = F(w r ) ∈ C n in the frequency domain.


All of these can be performed efficiently (in linear time) in the frequency domain.

the following relation holds for the dot product between any "time" vectors x, y ∈ R n .


Let us discuss the stochastic gradient descent (SGD) update with respect to these frequency vectors.

we are interested in whether conjugate symmetry of vectors is preserved by the update.


For NMT sentences selection, our hypothesis is that the NMT system itself can be used to score each sentence in the training data.

an NMT system embeds the source sentence into a vector representation 1 and we can use these vectors to measure a sentence pair's similarity to the in-domain corpus.


• 3) The sentence pairs (f, e) in the outof-domain corpus F out are classified into two sets: the sentences close to in-domain sentences, and those that are distant.

we firstly calculate the vector centers of in-domain C F in and out-of-domain C Fout corpora, respectively.


Corresponding distance difference δ e is, δ f , δ e and δ f e = δ f + δ e can be used to select sentences.

the sentence pairs (f, e) with δ f (or δ e , δ f e ) less than a threshold are the new selected in-domain corpus.


The threshold for the sentence selection method was selected on development data.

we selected the top ranked 10%, 20%,...,90% out-of-domain data to be added into the in-domain data, and the best performing models on development data were used in the evaluation on test data.


Most of the methods (without further training) converged after similar batches training.

in researched the highest BLEU performance on dev faster than other methods (without further training), then decreased and finally converged.


(2016a) by back-translating monolingual data and adding it to the parallel training data.

we back-translate sentences from the target side of WMT'15 that are not included in our low-resource baseline with two settings: keeping a one-to-one ratio of back-translated versus original data (1 : 1) following the authors' suggestion, or using three times more back-translated data (3 : 1).


We show that this approach leads to generating more rare words during translation and, consequently, to higher translation quality.

we report substantial improvements in simulated lowresource EnglishÑGerman and GermanÑEnglish settings, outperforming another recently proposed data augmentation technique.


In this paper, we propose a new type of decoder for NMT, which splits the decode state into two parts and updates them in two different timescales.

we first predict a chunk timescale state for phrasal modeling, on top of which multiple word timescale states are generated.


We find that our proposed model improves the dl4mt baseline on both the translation adequacy and fluency aspects.

the over translation error rate drops by 6%, which confirms the assumption in the introduction that splitting the fast and slow varying components in different time-scales could help alleviate the over translation errors.


Attention helps when separately encoding the plot and ending, but not when encoding the entire story.

we suspect this is because when we use ENCSTORY, the higher BiLSTM processes the sequence the first four sentence representations are in a different space from the ending due to the use of attention.


This paper describes an approach to determine whether people participate in the events they tweet about.

we determine whether people are participants in events with respect to the tweet timestamp.


Finally, Context features mostly indicate the presence of pronouns in the tweet.

we include the position of any pronoun with respect to the event, and a specialization of this feature for pronouns I, me, and we.


The current paper presents the first study where user cognitive structure is used to build a predictive model of income.

we first develop a classifier using a weakly supervised learning framework to automatically time-tag tweets as past, present, or future.


Finally, we tested whether individual differences in past, present, and future orientation are related to income.

we frame the income prediction task as regression using linear as well as non-linear learning algorithms where temporal orientation served as predictive features.


For classification, a linear Support Vector Machine (lSVM) 1 is used.

we trained three binary classifiers (one per class) 2 using one-vs.-rest, and label a tweet with the class that assigned the highest score.


Unlike previous approaches which mainly rely on hand-crafted rules and feature engineering, we automatically extract features for tweets to build our tweet-level Temporal Orientation Classifier.

we use CNNs to automatically extract tweet vectors as the features for classification.


Results show that our proposed framework performs consistently better than its counterpart.

results show that with 30K training examples, better results can be obtained by our approach than relying on 120K training items for the state-of-theart supervised machine learning approach (Base-line2).


The three agglutinative languages Basque, Hungarian, and Korean suffer the most with masked words.

the suffixes are the most informative for parsing in these three languages, since they cause the most loss while masked, and the least loss while unmasked.


In NLP, jackknifing has commonly been used to describe a procedure by which the training input is adjusted to correspond more closely to the expected test input, and it is in this latter sense that we use the term here.

in parsing research, the n-fold jackknifing proceeds as follows.


Thus, to adapt via linear jackknifing, we must i) approximate the expected input data tagging accuracy, while at the same time ii) accounting for the fact that the accuracy associated with any p depends on treebank size as well.

given two treebanks D 1 < D 2 , we would typically need p 1 > p 2 with the goal of matching the same test-set accuracy.


We introduce Probabilistic FastText, a new model for word embeddings that can capture multiple word senses, sub-word structure, and uncertainty information.

we represent each word with a Gaussian mixture density, where the mean of a mixture component is given by the sum of n-grams.


PFT can model rare words, uncertainty information, hierarchical representations, and multiple word senses.

we represent each word with a Gaussian or a Gaussian mixture density, which we name PFT-G and PFT-GM respectively.


Our multimodal word representation can also disentangle meanings, and is able to separate different senses in foreign polysemies.

our models attain state-of-the-art performance on SCWS, a benchmark to measure the ability to separate different word meanings, achieving 1.0% improvement over a recent density embedding model W2GM (Athiwaratkun and Wilson, 2017).


We represent each word with a Gaussian mixture with K Gaussian components.

a word w is associated with a density function are the mean vectors and {Σ w,i } are the covariance matrices, and {p w,i } K k=1 are the component probabilities which sum to 1.


For word w, we estimate the mean vector µ w with the average over n-gram vectors and its dictionary-level vector.

where z g is a vector associated with an n-gram g, v w is the dictionary representation of word w, and N G w is a set of n-grams of word w. Examples of 3,4-grams for a word "beautiful", including the beginning-of-word character ' ' and end-of-word character ' ', are: • 3-grams: be, bea, eau, aut, uti, tif, ful, ul • 4-grams: bea, beau .., iful ,ful This structure is similar to that of FASTTEXT (Bojanowski et al., 2016); however, we note that FASTTEXT uses single-prototype deterministic embeddings as well as a training approach that maximizes the negative log-likelihood, whereas we use a multi-prototype probabilistic embedding and for training we maximize the similarity between the words' probability densities, as described in Sections 3.2 and 3.3 Figure 1a depicts the subword structure for the mean vector.


This loss function together with the Gaussian mixture model with K > 1 has the ability to extract multiple senses of words.

for a word with multiple meanings, we can observe each mode to represent a distinct meaning.


This technique has an effect of reducing the importance of words such as 'the', 'a', 'to' which can be predominant in a text corpus but are not as meaningful as other less frequent words such as 'city', 'capital', 'animal', etc.

word w has probability P (w) = 1 − t/f (w) where f (w) is the frequency of word w in the corpus and t is the frequency threshold.


The hyperpameter α affects the scale of the loss function; therefore, we adjust the learning rate appropriately for each α.

the learning rates used are γ = {10 −4 , 10 −5 , 10 −6 } for the respective α values.


Thus a natural extension is to use an arbitrary linear transformation which will be learned from the data, and hence guaranteed to do at least as well as any of the above ideas.

we find the transform that can best recover existing word vectors v w -which are presumed to be of high qualityfrom their additive context embeddings v additive w .


The GloVe model is closely related to the notion of pointwise mutual information (PMI), which is defined for two words i and j as PMI(i, j) = log P (i,j) P (i)P (j) , where P (i, j) is the probability of seeing the words i and j if we randomly pick a word position from the corpus and a second word position within distance W from the first position.

the PMI between i and j is usually estimated as follows: where it is straightforward to see that after the reparameterization given by (1) In this paper, we will use the following variant of the formulation in (1): (2) Despite its similarity, this formulation differs from the GloVe model in a number of important ways.


First, we use smoothed frequency counts instead of the observed frequency counts x ij .

the PMI between words i and j is given as: where the probabilities are estimated as follows: where α ≥ 0 is a parameter controlling the amount of smoothing and n is the size of the vocabulary.


For efficiency reasons, in practice, we only consider a small subset of all context words j for which x ij = 0, which is similar in spirit to the use of negative sampling in Skip-gram (Mikolov et al., 2013b).

the set J i contains each j such that x ij > 0 as well as M uniformly 1 sampled context words j for which x ij = 0, where we choose M = 2 • |{j : Second, following (Jameel and Schockaert, 2016), the weighting function f (x ij ) has been replaced by 1 j is the residual variance of the regression problem for context word j, estimated follows: Since we need the word vectors to estimate this residual variance, we reestimate σ 2 j after every five iterations of the SGD optimization.


We have empirically found that omitting this bias term does not affect the performance of the model, while it allows us to have a more direct connection between the vector w i and the corresponding PMI scores.

let us define PMI W as follows: Clearly, when the word vectors are trained according to (2), it holds that PMI we can think of the word vector w i as a low-dimensional encoding of the vector (PMI S (i, 1), ..., PMI S (i, n)), with n the number of words in the vocabulary.


This view allows us to assign a natural interpretation to some word vector operations.

the vector difference w i − w k is commonly used as a model for the relationship between words i and k. For a given context word j, we have The latter is an estimation of log P (i,j) P (i)P (j) − log P (k,j) P (k)P (j) = log P (j|i) P (j|k) .


In particular, the vector difference w i − w k is commonly used as a model for the relationship between words i and k. For a given context word j, we have The latter is an estimation of log P (i,j) P (i)P (j) − log P (k,j) P (k)P (j) = log P (j|i) P (j|k) .

the vector translation w i − w k encodes for each context word j the (log) ratio of the probability of seeing j in the context of i and in the context of k, which is in line with the original motivation underlying the GloVe model (Pennington et al., 2014).


We define: where if p < q < r and r−p ≤ W , and weight(p, q, r) = 0 otherwise.

y ijk reflects the (weighted) number of times word j appears between words i and k in a sentence in which i and k occur sufficiently close to each other, in that order.


The measures SI 3 and SI 4 are closely related to SI 1 and SI 2 respectively 2 .

the following identities are easy to show: Using smoothed versions of the counts y ijk , we can use the following probability estimates for SI 1 (i, j, k)-SI 4 (i, j, k): where y ij * = k y ijk , and similar for the other counts.


in the context of hypernym detection (Baroni et al., 2012).

finally, Avg averages the vector representations of the words occurring in sentences that Diff, contain i and k. let r avg ik be obtained by averaging the word vectors of the context words appearing between i and k for each sentence containing i and k (in that order), and then averaging the vectors obtained from each of these sentences.


To analyze the benefit of our proposed word embedding variant, Table 2 shows the results that were obtained when we use standard word embedding models.

we show results for the standard GloVe model, SkipGram and the Continuous Bag of Words (CBOW) model.


Similarly, in Table 3, we show what happens if the relation vectors s ik , s ki , t ik and t ki are omitted.

for the results in Table 3, we only use context words that appear between the two target words.


We evaluate this ability using a dataset that was produced in the aftermath of SemEval 2012 Task 2.

we have used the "Phase2AnswerScaled" data from the platinum rankings dataset, which is available from the SemEval 2012 Task 2 website 5 .


Finally, we consider the problem of relation extraction from a text corpus.

we consider the task proposed in (Riedel et al., 2010), which is to extract (subject,predicate,object) triples from the New York Times (NYT) corpus.


The key idea of ER is to directly learn a specialization function in a supervised setting, using lexical constraints as training instances.

our model, implemented as a deep feedforward neural architecture, learns a (non-linear) function which "translates" word vectors from the distributional space into the specialized space.


Furthermore, we show that, by coupling the ER model with shared multilingual embedding spaces (Mikolov et al., 2013a;Smith et al., 2017), we can also specialize distributional spaces for languages unseen in the training data in a zero-shot language transfer setup.

we show that an explicit retrofitting model trained with external constraints from one language can be successfully used to specialize the distributional space of another language.


To map the latent representations from different languages to a shared-latent space, we propose the weightsharing constraint to the two AEs.

we share the weights of the last few layers of two encoders that are responsible for extracting highlevel representations of input sentences.


For the encoder and decoder, we follow the newly emerged Transformer (Vaswani et al., 2017).

the encoder is composed of a stack of four identical layers 2 .


Weight sharing Based on the shared-latent space assumption, we apply the weight sharing constraint to relate the two AEs.

we share the weights of the last few layers of the Enc s and Enc t , which are responsible for extracting high-level representations of the input sentences.


To this end, we shuffle the input sentences randomly.

we apply a random permutation ε to the input sentence, verifying the condition: where n is the length of the input sentence, steps is the global steps the model has been updated, k and s are the tunable parameters which can be set by users beforehand.


In the first stage, we train the proposed model with denoising auto-encoding, backtranslation and the local GANs, until no improvement is achieved on the development set.

we perform one batch of denoising autoencoding for the source and target languages, one batch of back-translation for the two languages, and another batch of local GAN for the two languages.


Word-by-word translation (WBW) The first baseline we consider is a system that performs word-by-word translations using the inferred bilingual dictionary.

it translates a sentence word-by-word, replacing each word with its nearest neighbor in the other language.


The GANs also significantly improve the translation performance of our system.

the global GANs achieve improvement up to +0.78 BLEU points on English-to-French translation and the local GANs also obtain improvement up to +0.57 BLEU points on English-to-French translation.


As the final vocabulary V contains all individual characters in the corpus, character-based segmentation is also included in the set of segmentation candidates S(X).

subword segmentation with the unigram language model can be seen as a probabilsitic mixture of characters, subwords and word segmentations.


We use the Adam optimizer (Kingma and Ba, 2014) with β 1 = 0.9, β 2 = 0.999, = 10 −6 and vary the learning rate according to this schedule: (1) Here, t is the current step, n is the number of concurrent model replicas used in training, p is the number of warmup steps, s is the start step of the exponential decay, and e is the end step of the decay.

we first increase the learning rate linearly during the number of warmup steps, keep it a constant until the decay start step s, then exponentially decay until the decay end step e, and keep it at 5 • 10 −5 after the decay ends.


Next, we explore how the features extracted by an encoder can be further enhanced by incorporating additional information.

we investigate the combination of transformer layers with RNMT+ layers in the same encoder block to build even richer feature representations.


The distant supervision sources provide partial supervision for ultra-fine types; KBs often provide more general types, while head words usually provide only ultra-fine types, without their generalizations.

the absence of a type at a different level of abstraction does not imply a negative signal; e.g.


Instead of updating all labels for each example, we divide labels into three bins (general, fine, and ultra-fine), and update labels only in bin containing at least one positive label.

the training objective is to minimize J where t is the target vector at each granularity: Where 1 category (t) is an indicator function that checks if t contains a type in the category, and   J category is the category-specific logistic regression objective: 5 Evaluation Experiment Setup The crowdsourced dataset (Section 2.1) was randomly split into train, development, and test sets, each with about 2,000 examples.


We observe that every supervision source improves performance in its own right.

the naturally-occurring head-word supervision seems to be the prime source of improvement, increasing performance by about 10% across all metrics.


This paper, by contrast, investigates the potential of using very simple constraints to improve the KG embedding task.

we examine two types of constraints: (i) non-negativity constraints on entity representations and (ii) approximate entailment constraints over relation representations.


We choose ComplEx (Trouillon et al., 2016) as our basic embedding model, since it is simple and efficient, achieving state-of-the-art predictive performance.

suppose we are given a KG containing a set of triples O = {(e i , r k , e j )}, with each triple composed of two entities e i , e j ∈ E and their relation r k ∈ R. Here E is the set of entities and R the set of relations.


If For all the methods, we create 100 mini-batches on each dataset, and conduct a grid search to find hyperparameters that maximize MRR on the validation set, with at most 1000 iterations over the training set.

we tune the embedding size d ∈ {100, 150, 200}, the L 2 regularization coefficient η ∈ {0.001, 0.003, 0.01, 0.03, 0.1}, the ratio of negative over positive training examples α ∈ {2, 10}, and the initial learning rate γ ∈ {0.01, 0.05, 0.1, 0.5, 1.0}.


Then we investigate the semantic purity of these dimensions.

we collect the representations of all the entities on DB100K (real components only).


For our geometrical analysis, we first define a term 'alignment to mean' (ATM) of a vector v belonging to a set of vectors V, as the cosine similarity 1 between v and the mean of all vectors in V. We also define 'conicity' of a set V as the mean ATM of all vectors in V.  By this definition, a high value of Conicity(V) would imply that the vectors in V lie in a narrow cone centered at origin.

the vectors in the set V are highly aligned with each other.


Hyperparameters: We experiment with multiple values of hyperparameters to understand their effect on the geometry of KG embeddings.

we vary the dimension of the generated vectors between {50, 100, 200} and the number of negative samples used during training between {1, 50, 100}.


Parameters h j and H j are the hidden vectors at the j th step of the wordlevel and sentence-level decoder respectively defined as: where The non-linear transformation, φ (we choose tanh), is used to connect the word-level encodings to the sentence decoder and the sentence-level encodings to the word decoder.

the word-level decoder updates its state by considering a sum of sentence encodings, weighted by the attentions from the previous state and mutatis mutandis for the sentence-level decoder.


they show that methods that account for saliency using both the following considerations perform better than methods that consider either one of them alone, and SWAP-NET is based on the same principles.


Then, we extend the seq2seq model to jointly learn template saliency measurement (Rerank) and final summary generation (Rewrite).

a Recurrent Neural Network (RNN) encoder is applied to convert the input sentence and each candidate template into hidden states.


Then, we jointly conduct reranking and rewriting through a shared encoder.

both the sentence x and the soft template r are converted into hidden states with a RNN encoder.


Considering that the matching networks based on hidden states have demonstrated the strong ability to measure the relevance of two pieces of texts (e.g., ), we propose to jointly conduct reranking and rewriting through a shared encoding step.

we employ a bidirectional Recurrent Neural Network (BiRNN) encoder  to read x and r. Take the sentence x as an example.


We utilize the hidden states of x and r to predict the saliency s of the template.

we regard the output of the BiRNN as the representation of the sentence or template: Next, we use Bilinear network to predict the saliency of the template for the input sentence.


Therefore, we leverage the strong rewriting ability of the seq2seq model to generate more faithful and informative summaries.

since the input of our system consists of both the sentence and soft template, we use the concatenation function 3 to combine the hidden states of the sentence and template: The combined hidden states are fed into the prevailing attentional RNN decoder  to generate the decoding hidden state at the position t: where y t−1 is the previous output summary word.


To make our model more general, we retain the default settings of OpenNMT to build the network architecture.

the dimensions of word embeddings and RNN are both 500, and the encoder and decoder structures are two-layer bidirectional Long Short Term Memory Networks (LSTMs).


In practice, a system that can provide various candidate summaries is probably more welcome.

two candidate templates with large text dissimilarity are manually fed into the Rewriting module.


After splitting, the text is amenable for further fine-tuned simplification operations.

we show that neural Machine Translation can be effectively used in this situation.


We test other variants of SEMoses, where phrase-based MT is used instead of NMT.

we incorporate semantic information in a different manner by implementing two additional models: (1) SETrain1-Moses, where a new training corpus is obtained by applying the splitting rules to the target side of the Table 5: System outputs for one of the test sentences with the corresponding human evaluation scores (averaged over the 3 annotators).


5 we include terms from the following sources: • All terms in the NRC Emotion Lexicon (Mohammad and Turney, 2013).


Medical abstracts often mention the same information in multiple places.

interventions and outcomes are typically described at the beginning of an abstract when introducing the purpose of the underlying study, and then again when discussing methods and results.


Beyond identifying the spans of text containing information pertinent to each of the PIO elements, we consider the task of predicting which of the detailed labels occur in each span, and where they are located.

we begin with the starting spans and predict a single label from the corresponding PIO hierarchy for each token, evaluating against the test set of 200 documents.


We are concerned here with the construction of datasets and evaluation of systems within natural language processing (NLP).

humans providing responses that are used to derive graded values on natural language contexts, or in the ordering of systems corresponding to their perceived performance on some task.


The underlying idea for the variance updates is that these updates always decrease the size of the variances σ 2 , which means uncertainty of the instances (S i , S j ) always decreases as we observe more pairwise comparisons.

trueSkill becomes more confident in the current estimate of µ i and µ j .


As iterations go on, the instances along the diagonal have higher match quality, indicating that competitive matches are more likely to be selected for a next iteration.

match-quality helps to choose informative pairs to compare at each iteration, which reduces the number of less informative annotations (e.g., a pairwise comparison between the highest and lowest instances).


We draw data from Reddit subcommunity /r/ChangeMyView (henceforth CMV), which focuses on facilitating open discussions on a wide range of disputed issues.

cMV is structured as discussion threads, where the original post (OP) starts with a viewpoint on a controversial topic, followed with detailed reasons, then other users reply with counter-arguments.


Therefore, multiple passes of retrieval will be conducted if more than one query is created.

we first collect topic signature words of the post.


Figure 1 sketches how we operationalize the hypothesis.

we simultaneously model the topic similarity and stance dissimilarity of a candidate counterargument to the argument.


To capture this intuition, we rely on expert knowledge from argumentation theory (see Section 2).

we follow the notion that a counterargument attacks either the conclusion of an argument, the argument's premises, or both.


We, therefore, posit that combining graph alignment task with deep representation learning across multi-relational graphs has potential to induce a synergistic effect on both tasks.

we identify that a key component of graph alignment process-entity linkage-also plays a vital role in learning across graphs.


For a given attribute a represented as key-value pair, we use paragraph2vec (Le and Mikolov, 2014) type of embedding network to learn attribute embedding.

we represent attribute embedding vector as: where a ∈ R y , a key ∈ R u and a val ∈ R v .


As there can be large number of neighbors, we collect the neighborhood set for each entity as a pre-processing step using a random walk method.

given a node e, we run k rounds of random-walks of length l following (Hamilton et al., 2017) and create set N (e) by adding all unique nodes visited across these walks.


This is conventional loss function used to learn knowledge graph embeddings.

given a p-th triplet (e s , r, e o ) p from training set D, we sample C negative samples by replacing either head or tail entity and define a contrastive max margin function as shown in (Socher et al., 2013): where, γ is margin, e s c represent corrupted entity and g (e s c , r p , e o p ) represent corrupted triplet score.


We compare the performance of our method against state-of-the-art representation learning baselines that use neural embedding techniques to learn entity and relation representation.

we consider compositional methods of RESCAL (Nickel et al., 2011) as basic matrix factorization method, DISTMULT (Yang et al., 2015) as simple multiplicative model good for capturing symmetric relationships, and Complex (Trouillon et al., 2016), an upgrade over DISTMULT that can capture asymmetric relationships using complex valued embeddings.


They are full probabilistic models of the joint set of variables, rather than embedding-based approximations of only low-order joint and conditional probabilities.

any set of our parameters can answer any arbitrary probabilistic question (possibly requiring intractable computation), rather than being fixed to modeling only certain subsets of the joint.


Graph structures are ubiquitous in representations of natural language.

many wholesentence semantic frameworks employ directed acyclic graphs as the underlying formalism, while most tree-based syntactic representations can also be seen as graphs.


In this work we propose a model for graph-tosequence (henceforth, g2s) learning that leverages recent advances in neural encoder-decoder architectures.

we employ an encoder based on Gated Graph Neural Networks (Li et al., 2016, GGNNs), which can incorporate the full graph structure without loss of information.


Our approach outperforms strong s2s baselines in both tasks without relying on standard RNN encoders, in contrast with previous work.

for NMT we show that we avoid the need for RNNs by adding sequential edges between contiguous words in the dependency tree.


Our approach significantly outperforms the s2s baselines both with individual models and ensembles, while using a comparable number of parameters.

we obtain these results without relying on scoping heuristics.


These results are not strictly comparable because they used different training set versions and/or employ additional unlabelled corpora; nonetheless some insights can be made.

our g2s ensemble performs better than many previous models that combine a smaller training set with a large unlabelled corpus.


Interestingly, we found different trends when analysing the CHRF++ numbers.

this metric favours the PB-SMT models for both language pairs, while also showing improved performance for s2s in En-Cs.


Our approach addresses shortcomings from previous work, including loss of information from lineari-sation and parameter explosion.

we showed how graph transformations can solve issues with graph-based networks without changing the underlying architecture.


In this paper, we investigate the role of context in an LSTM LM, through ablation studies.

we analyze the increase in perplexity when prior context words are shuffled, replaced, or dropped.


Given a pretrained language model, we perturb the prior context in various ways at test time, to study how much the perturbed information affects model performance.

we alter the context length to study how many tokens are used, permute tokens to see if LSTMs care about word order in both local and global contexts, and drop and replace target words to test the copying abilities of LSTMs with and without an external copy mechanism, such as the neural cache (Grave et al., 2017b).


We determine the effective context size by varying the number of tokens fed to the model.

at test time, we feed the model the most recent n tokens: (1) where n > 0 and all tokens farther away from the target w t are dropped.


Figure 1a shows that the difference in dev loss, between truncated-and infinite-context variants of the test setting, gradually diminishes as we increase n from 5 tokens to 1000 tokens.

we only see a 1% increase in perplexity as we move beyond a context of 150 tokens on PTB and 250 tokens on Wiki.


We investigate whether the effective context size varies across different types of words, by categorizing them based on either frequency or parts-ofspeech.

we vary the number of context tokens in the same way as the previous section, and aggregate loss over words within each class separately.


To determine the range in which word order affects model performance, we permute substrings in the context to observe their effect on dev loss compared to the unperturbed baseline.

we perturb the context as follows, where ⇢ 2 {shu✏e, reverse} and (s 1 , s 2 ] denotes the range of the substring to be permuted.


To study the effect of content and function words on model perplexity, we drop them from different regions of the context and compare the resulting change in loss.

we perturb the context as follows, where f pos (y, span) is a function that drops all words with POS tag y in a given span.


of target word occurrences in context, we experiment with dropping all the distant context versus dropping only occurrences of the target word from the context.

we compare removing all tokens after the 50 most recent tokens, (Equation 1 with n = 50), versus removing only the target word, in context of size n = 300: where f word (w, span) drops words equal to w in a given span.


This perturbation is similar to Equation 4, except instead of dropping the token, we replace it with a different one.

we experiment with replacing the target with <unk>, to see if having the generic word is better than not having any word.


Following the plan in Table 2, each constraint type can be converted in an equivalent form E q [φ(X, Y )] = b, compatible with PR.

each of these constraint types in our vocabulary can be expressed as equations about expectation values of joint indicator functions of label assignments to instances and their attributes.


As shown in Figure 1, the main idea is to model the hidden states of all words simultaneously at each recurrent step, rather than one word at a time.

we view the whole sentence as a single state, which consists of sub-states for individual words and an overall sentence-level state.


At each recurrent step, information exchange is conducted between consecutive words in the sentence, and between the sentence-level state and each word.

each word receives information from its predecessor and successor simultaneously.


Compared to our work, their motivations and network structures are highly different.


We take an LSTM structure similar to the baseline BiLSTM for modelling state transition, using a recurrent cell c t i for each w i and a cell c t g for g. As shown in Figure 1, the value of each h t i is computed based on the values of i+1 and g t−1 , together with their corresponding cell values: where ξ t i is the concatenation of hidden vectors of a context window, and gates that control information flow from ξ t i and x i to c t i .

i t i controls information from the input x i ; l t i , r t i , f t i and s t i control information from the left context cell c t−1 i−1 , the right context cell c t−1 i+1 , c t−1 i and the sentence context cell c t−1 g , respectively.


We study the influence of attention on both S-LSTM and BiLSTM for classification.

additive attention (Bahdanau Dataset Training Development Test #sent #words #sent #words #sent #words Movie review (Pang and Lee, 2008)  Here W α , u and b α are model parameters.


Instead, we propose slanted triangular learning rates (STLR), which first linearly increases the learning rate and then linearly decays it according to the following update schedule, which can be seen in Figure 2: where T is the number of training iterations 4 , cut f rac is the fraction of iterations we increase 3 An unrelated method of the same name exists for deep Boltzmann machines (Salakhutdinov and Hinton, 2009).

4 the number of epochs times the number of updates per epoch.


The average query from any of the human-generated datasets joins more than one table.

task-oriented datasets require joins and nesting.


Another advantage is that the column-cell relation and question-cell connection can be naturally integrated in the model, which will be described below.

our encoder takes a question as the input.


Another important linguistic issue for agglutinative languages is the complex interaction between morphology and syntax, which is usually achieved via derivational morphemes.

unlike inflectional morphemes that only give information on word-level semantics, derivational morphemes provide more clues on sentence-level semantics.


In order to exploit nonlinear connections, we learn the parameters φ of f via a simple linear layer followed by sigmoid activation.

we train a new model that learns how to best combine the predictions from subword models.


The gap between the two does not increase with the distance, suggesting that the performance gap is not related to long range dependencies.

both characters and the oracle handle long range dependencies equally well.


In contrast, the alignment model Q ψ (a|c, R, w), as will be explained below, is approximating the intractable posterior P θ,φ (a|c, R, w) within that probabilistic model.

we assume the following model for generating the AMR graph: 4 θ, φ and ψ denote all parameters of the models.


Instead, in perturband-max an approximate schema is used where noise is assumed factorizable.

first noisy scores are computed asφ( 1) and an approximate sample is obtained by a = argmax a n i=1φ (g i , h a i ), Such sampling procedure is still intractable in our case and also non-differentiable.


One remaining question is how to use the soft inputâ = S t (Φ ψ , Σ) in the concept and relation identification models in equation 5.

we need to define how we compute P θ (c|S t (Φ ψ , Σ), w) and P φ (R|S t (Φ ψ , Σ), w, c).


Our model substantially outperforms all the previous models on both datasets.

it achieves 74.4% Smatch score on LDC2016E25 (R2), which is an improvement of 3.4% over character seq2seq model relying on silver data (van Noord and Bos, 2017).


Now we would like to confirm that joint training of alignments with both concepts and relations is beneficial.

we would like to see if alignments need to be induced in such a way as to benefit the relation identification task.


For AMR parsing, another way to avoid using pre-trained aligners is to use seq2seq models (Konstas et al., 2017;van Noord and Bos, 2017).

van Noord and Bos (2017) used character level seq2seq model and achieved the previous state-of-the-art result.


To simplify the semantic interpretation process, we add extra label information to enrich the nonterminals in CFG rules.

we consider the count of external nodes of a corresponding HRG rule.


Boxer constructs DRSs based on a pipeline of tools involving POS-tagging, named entity recognition, and parsing.

it relies on the syntactic analysis of the C&C parser (Clark and Curran, 2007), a general-purpose parser using the framework of Combinatory Categorial Grammar (CCG; Steedman 2001).


Our second decoder attempts to take this into account by distinguishing conditions which are local and correspond to words in a sentence from items which are more global and express semantic content (see Figure 3(b)).

we model sentence specific conditions using a copying mechanism, and all other conditions G which do not correspond to sentential tokens (e.g., thematic roles, rhetorical relations) with an insertion mechanism.


We thus use F 1 instead of exact match accuracy.

we report D-match 4 a metric designed to evaluate scoped meaning representations and released as part of the distribution of the Parallel Meaning Bank corpus (Abzianidze et al., 2017).


To account for the distinct nature of various NLP tasks that may require different semantic features, we compare SWEM-based models with existing recurrent and convolutional networks in a pointby-point manner.

we consider 17 datasets, including three distinct NLP tasks: document classification (Yahoo news, Yelp reviews, etc.


First, we explore a series of pooling operations, rather than only average-pooling.

a hierarchical pooling operation is introduced to incorporate spatial information, which demonstrates superior results on sentiment analysis, relative to average pooling.


We define d as the dimension of the final sequence representation.

d represents the dimension of hidden units or the number of filters in LSTM or CNN, respectively.


Surprisingly, on topic prediction tasks, our SWEM model exhibits stronger performances, relative to both LSTM and CNN compositional architectures, this by leveraging both the average and max-pooling features from word embeddings.

our SWEM-concat model even outperforms a 29-layer deep CNN model (Conneau et al., 2016), when predicting topics.


Our goals are similar to those of PPDB, which has likewise been generated for many languages (Ganitkevitch and Callison-Burch, 2014) since it only needs parallel text.

we follow the approach of , who used NMT to translate the non-English side of parallel text to get English-English paraphrase pairs.


By using the mini-batch to select negative examples, we may be limiting the learning procedure.

if all potential negative examples in the mini-batch are highly dissimilar from s, the loss will be too easy to minimize.


Compared to prior work on phrasal embeddings (Wieting et al., 2015;Pavlick et al., 2015), our work generalizes the phrases by introducing (typed) variables.

we replace words that correspond to entity mentions or pronouns with typed variables such as PersonX or PersonY, as shown in examples in Table 1.


Shown in Table 4, our results indicate a gender bias in the behavior ascribed to characters, consistent with psychology and gender studies literature (Collins, 2011).

events with female semantic agents are intended to be helpful to other people (intents involving FRIEND, FAMILY, and AFFILIATION), particularly relating to eating and making food for themselves and others (INGEST, BODY).


Our approach decodes nuanced implications into more explicit statements, helping to identify and explain gender bias that is prevalent in modern literature and media.

our results indicate that modern movies have the bias to portray female characters as having pro-social attitudes, whereas male characters are portrayed as being competitive or pro-achievement.


Their model is also designed to choose a case label for a pair of a predicate and its argument candidate.

their model can assign the same case label to multiple arguments by itself, while our model does not.


we model document level distributional patterns of coreferent event mentions that may be specific to a genre in ILP.

news article often begins with a summary of the overall story and then introduces the main events and their closely associated events.


Therefore, We add the following objective function (equation 3) to the basic objective function and add the new constraint 4 in order to encourage coreferent event mentions to occur in topic transition sentences.

let ω represents the set of sentences in a document and Ω denotes the set of sentence pairs i.e.


Therefore, we further model the distributional characteristics of global event chains and encourage the main event chains to have a large number of coreferential mentions and a long stretch (the number of sentences that are present in between the first and last event mention of a chain), to avoid creating partial chains.

we add the following objective function (equation 5) and the new constraints (equation 6 and 7): First, we define an indicator variable σ ij by equation 6 3 , corresponding to each event mention pair, that takes value 1 if (1) the event mentions at index i and j are coreferent; (2) the event mention at index i doesn't corefer to any of the mentions preceding it; and (3) mention at index j doesn't corefer to any event mention following it.


We model this through a simple objective function and constraints (equation 10).

for the event pairs that belong to the first α (or the last β) sentences in a document, we add the negative (positive) sum of their indicator variables (x) in objective function Θ D .


We identify probable subevents by using surface syntactic cues corresponding to identifying a sequence of events in a sentence (Badgett and Huang, 2016).

a sequence of two or more verb event mentions in a conjunction structure are extracted as subevents.


But as the goal of this study is to leverage discourse level topic structure in a document for improving event coreference resolution performance, we only evaluate the ILP system using regular documents (news articles) in the KBP corpora.

we train our event extraction system and local coreference resolution classifier on 310 documents from the KBP 2015 corpus that consists of both discussion forum documents and news articles, tune the hyper-parameters corresponding to ILP using 50 news articles 6 from the KBP 2015 corpus and evaluate our system on 4 The ECB+ (Cybulska and Vossen, 2014) corpus is another commonly used dataset for evaluating event coreference resolution performance.


But we determined that this corpus is not appropriate for evaluating our ILP model that explicitly focuses on using discourse level topic structures for event coreference resolution.

the ECB+ corpus was created to facilitate both cross-document and indocument event coreference resolution research.


When the classification capacity of discriminator declines, the accuracy of being predicted as negative sample on N D gradually drops; thus,p increases.


As we can see, our proposed models perform much better than NovelTagging model in Entity-PairOverlap class and SingleEntityOverlap classes.

our models achieve much higher performance on all metrics.


Detecting events in an automatic way is challenging, not only because an event can be expressed in different words, but also because a word may express a variety of events in different contexts.

the frequent utilization of common words, ambiguous words and pronouns in event mentions makes them harder to detect: 1) Generalitytaken home <Transport> Ambiguity 1campaign in Iraq <Attack> Ambiguity 2political campaign <Elect> Coreference -Either its bad or good <Marry> A promising solution to this challenge is through semantic understanding.


By contrast, other work shows a nearly constant gap.

sELF yields a minimum gap in each task, which changes negligibly from 3.5% to 3.4%.


(2017), we augment the data by flipping all pairs, except for relations involving document creation time (DCT).

if a pair (e i , e j ) exists, we add (e j , e i ) to the dataset with the opposite label (e.g.


To fully understand an article, we must be able to organize the processed information across sentences and paragraphs.

to interpret temporal relations between entities in a sentence, sometimes we also look at relations with other entities elsewhere in the text.


The pairwise model classifies individual entity pairs, where entities are events and time expressions (timexes).

for each pair, we only use the local context, and the relation of one pair does not affect the classification results for other pairs.


Occurrences of events, referring to changes and actions, show regularities.

certain events often co-occur and in a particular temporal order.


Then by leveraging the double temporality characteristic of narrative paragraphs, we distill general temporal event knowledge.

we extract event pairs as well as longer event sequences consisting of strongly associated events that often appear in a particular textual order in narrative paragraphs, by calculating Causal Potential (Beamer and Girju, 2009;Hu et al., 2013) between events.


Specifically, we extract event pairs as well as longer event sequences consisting of strongly associated events that often appear in a particular textual order in narrative paragraphs, by calculating Causal Potential (Beamer and Girju, 2009;Hu et al., 2013) between events.

we obtained 19k event pairs and 25k event sequences with three to five events from the 287k narrative paragraphs we identified across three genres, news articles, novel books and blogs.


Guided by the prior narratology studies (Greimas, 1971;Mani, 2012) and our observations, we use context-free grammar production rules to identify sentences that describe an event in an actantial syntax structure.

we use three sets of grammar rules to specify the overall syntactic structure of a sentence.


Negative instances are paragraphs that are not likely to be narratives and do not present a plot or protagonist character, but are similar to seed narratives in others aspects.

similar to seed narratives, we require a non-narrative paragraph to contain at least four sentences with no more than 20% of sentences being interrogative, exclamatory or dialogue; but in contrast to seed narratives, a non-narrative paragraph should contain 30% of or fewer sentences that have the actantial sentence structure, where the longest character entity chain should not span over 20% of sentences.


In addition, since it is infeasible to apply the trained classifier to all the paragraphs in a large text corpus, such as the Gigaword corpus (Graff and Cieri, 2003), we identify candidate narrative paragraphs and only apply the statistical classifier to these candidate paragraphs.

we require a candidate paragraph to satisfy all the constraints used for identifying seed narrative paragraphs but contain only 30% 7 or more sentences with an actantial structure and have the longest character entity chain spanning over 20% 8 of or more sentences.


We choose Maximum Entropy (Berger et al., 1996) as the classifier.

we use the MaxEnt model implementation in the LIBLIN-EAR library 9 (Fan et al., 2008) with default parameter settings.


Features for Identifying Plot Events: Realizing that grammar production rules are effective in identifying sentences that contain a plot event, we encode all the production rules as features in the statistical classifier.

for each narrative paragraph, we use the frequency of all syntactic production rules as features.


In addition, to capture potential event sequence overlaps between new narratives and the already learned narratives, we build a verb bigram language model using verb sequences extracted from the learned narrative paragraphs and calculate the perplexity score (as a feature) of the verb sequence in a candidate narrative paragraph.

we calculate the perplexity score of an event sequence that is normalized by the number of events, P P (e 1 , ..., e N ) = N N i=1 1 P (e i |e i−1 ) , where N is the total number of events in a sequence and e i is a event word.


First, by leveraging the double temporality characteristic of narratives, we only consider event pairs and longer event chains with 3-5 events that have occurred as a segment in at least one event sequence extracted from a narrative paragraph.

we extract the event sequence (the plot) from a narrative paragraph by finding the main event in each sentence and chaining the main events 11 according to their textual order.


We adopt the existing metric, Causal Potential (CP), which has been applied to acquire causally related events (Beamer and Girju, 2009) and exactly measures the two aspects.

the CP score of an event pair is calculated using the following equation: where, the first part refers to the Pointwise Mutual Information (PMI) between two events and the second part measures the relative ordering or two events.


While each candidate pair of events should have appeared consecutively as a segment in at least one narrative paragraph, when calculating the CP score, we consider event co-occurrences even when two events are not consecutive in a narrative paragraph but have one or two other events in between.

the same as in (Hu and Walker, 2017), we calculate separate CP scores based on event co-occurrences with zero (consecutive), one or two events in between, and use the weighted average CP score for ranking an event pair, formally, CP (e i , e j ) = 3 .


We do this by linking the boundary sentences across posts and by linking sentences in the same post chronologically.

we connect the first sentence of post p j to the last sentence of post p i if p j replies to p i , and sentence s t+1 is linked to s t if both s t and s t+1 are in the same post.


Second, instead of requiring supervised signals at each time step, deep reinforcement learning model optimizes its policy based on an overall reward signal.

it learns to directly optimize the overall evaluation metrics, which is more effective than models that learn with loss functions that heuristically define the goodness of a particular single decision.


Pronoun Resolution Given an anaphoric zero pronoun zp, a set of candidate antecedents are required to be selected from its associated text.

we adopt the heuristic model utilized in recent Chinese anaphoric zero pronoun resolution work (Chen and Ng, 2016;Yin et al., 2017a,b) for this purpose.


For each time step, our reinforcement learning agent predicts the zero pronoun-candidate antecedent pair by using 1) the zero pronoun; 2) information of current candidate antecedent and 3) antecedent information generated by antecedents predicted in previous states.

our reinforcement learning agent is designed as a policy network ⇡ ✓ (s, a) = p(a|s; ✓), where s represents the state; a indicates the action and ✓ represents the parameters of the model.


The encoder learns to encode the zero pronoun by its associated text into its vector-space semantics.

two standard recurrent neural networks are employed to encode the preceding text and the following text of a zero pronoun, separately.


In this way, we generate the vector for zp, the v zp , and representation vectors of all its candidate antecedents, which are denoted as Moreover, we employ pooling operations to encode antecedent information by using the antecedents that are predicted in previous states.

we generate two vectors by applying the max-pooling and average-pooling, respectively.


Moreover, on purpose of better illustrating the effectiveness of the proposed reinforcement learning model, we run a set of experiments with different settings.

we compare the model with and without the proposed reinforcement learning process using different pre-training iterations.


(2017b) employed recurrent neural network to encode zero pronoun and antecedents.

a twolayer antecedent encoder was employed to generate the hierarchical representation of antecedents.


Since the first feature of him is a d selectee, we can merge these two words to obtain the following VP category, where ✏ is the empty string (The reason for the commas separating the left and right dependent string components from the head string component is to allow for subsequent head movement of the latter (see Stabler (2001) The strings of the two merged elements have been here been concatenated, but this will not always be the case.

if the selected item has additional features behind its selectee, then it will need to check these in subsequent derivational steps via applications of Move.


The significant β ′ coefficient of C count * C power means that the β of C count is dependent on C power .

the strength of alignment varies significantly depending on different power levels (i.e., high vs. low) of the prime speaker (reflected by the different slopes in Figure 1).


These effects are an important confound to take into account when examining higher-level features, such as social power.

the effect of social power cannot be reliably detected by linear models once introducing utterance length.


The latter should be a relatively stable effect that is independent on the lowlevel linguistic features.

our findings suggest that the probability change of LIWC categories is more likely to be a case of automatic alignment, rather than an intentional accommodation, because it is better explained by lowerlevel linguistic features (utterance length).


However, multiple arguments are typically made in a persuasive essay.

the arguments of an essay are typically structured as an argument tree, where the major claim, which is situated at the root of the tree, is supported by one or more claims (the children of the root node), each of which is in turn supported by one or more premises.


Our goal in this paper is to annotate and make publicly available a corpus of persuasive student essays that addresses the aforementioned weaknesses via designing appropriate annotation schemes and scoring rubrics.

not only do we score the persuasiveness of each ar-gument in each essay (rather than simply the persuasiveness of the overall argument), but we also identify a set of attributes that can explain an argument's persuasiveness and annotate each argument with the values of these attributes.


Additionally, Specificity produced unexpectedly low agreement in claims and major claims.

quantifying how well a claim summarizes its argument turned out to be a complicated and subjective task.


Next, we conduct an oracle experiment in an attempt to understand how well these attributes, when used together, can explain the persuasiveness of an argument.

we train three linear SVM regressors (using the SVM light software (Joachims, 1999) with default learning parameters except for C (the regularization parameter), which is tuned on development data using grid search) to score an argument's persuasiveness using the gold attributes as features.


So a lack of well-established primary education goes hand in hand with a high rate of illiteracy, and this interplay compromises a country's future development.

if countries, especially developing ones, are determined to take off, one of the key points governments should set on agenda is to educate more qualified future citizens through elementary education.


Its Relevance score is 6: its relationship with its parent is clear because the two components exhibit coreference.

p2 contains a reference to primary/elementary education and shows how this affects a country's inability to transition from developing to developed.


Similar trends are found when conducting such experiments to Text Simplification (TS) ( §4).

we show that (1) the distribution of valid simplifications for a given sentence is longtailed; (2) common measures for TS dramatically under-estimate performance; (3) additional references alleviate this under-prediction.


However, as this is highly computationally demanding, we estimate it using sampling.

for every M = 1, ..., 20 and x i , we sample Y i 1000 times (with replacement), and estimate P (y ∈ Y i ) as the covered probability mass P D i {y : y ∈ Y i }.


Intuitively, this is because estimating the mean of g(z) incurs an equally large variance as estimating µ.

g(z) is only useful if we have additional information about g beyond the samples Algorithm 1 shows the estimator.


Figure 6 also shows the conjectured confidence intervals if we were able to eliminate noise in human judgments (noiseless humans) or have a automatic metric that correlated perfectly with average human judgment (perfect metric).

we use the mean of all (2-3) humans on each z for the perfect g(z) and use the mean of all humans on each z for the "noiseless" Y (z).


There has been much work on improving the quality of automatic metrics.

there is interest in learning models (Lowe et al., 2017a;Dusek et al., 2017) that are able to optimize for improved correlations with human judgment.