We use a CNN architecture with a single convolutional and pooling layer, as shown in Figure 1.

we firstly represent a given social media message of length l words (padded where necessary) using a sentence matrix S ∈ R d×l : where each column of S is the d-dimensional vector (i.e.


We then apply a convolution operation using a filter w ∈ R d×h to a window of h words.

the filter w is convolved over the sequence of words in the sentence matrix S to create a feature matrix C. Each feature c i in C is extracted from a window of words x i:i+h−1 , as follow: where f is an activation function, such as sigmoid or tanh, and b ∈ R is a bias.


Table 2 shows statistics for the three datasets.

twADR-S is the dataset provided by Limsopatham and Collier (2015a), which contains 201 twitter phrases and their corresponding SNOMED-Ct 5 concept.


When comparing among the existing baseline approaches, we observe that DNorm and P-MT are the most effective baselines.

dNorm outperforms the other baselines for the TwAdR-S (accuracy 0.2983) and AskAPatient (accuracy 0.7339) datasets, while P-MT with GNews embeddings is the most effective baseline for the TwAdR-L dataset (accuracy 0.3371).


In addition, the choice of embeddings has a marked impact on the achieved performance.

the GNews embeddings benefit both CNN and RNN more than the BMC embeddings, which is in line with the previous finding that GNews is more useful than BMC for the EmbSim and P-MT baselines.


We have motivated the importance of semantics when normalising medical concepts in social media messages.

as social media messages are typically ambiguous, we argue that effective concept normalisation should deal with them at the semantic level.


In this paper, we propose a model of deep fusion LSTMs (DF-LSTMs) to model the strong interaction of text pair in a recursive matching way.

dF-LSTMs consist of two interdependent LSTMs, each of which models a sequence under the influence of another.


To better understand the behavior of our model, we perform error analysis on Out-of-Vocabulary words (OOV).

we partition each data set into four subsets -in-vocabulary words (IV), out-of-training-vocabulary words (OOTV), out-of-embedding-vocabulary words (OOEV) and out-of-both-vocabulary words (OOBV).


tures that are extracted by exploiting context sentence windows and writing topic to improve relation prediction.

we derive features using discourse relations between argument components and windows of their surrounding sentences.


The authors first use topic prompts in development data of unannotated persuasive essays to semi-automatically collect argument and domain seed words.

they used 10 argument seed words: agree, disagree, reason, support, advantage, disadvantage, think, conclusion, result, opinion.


Therefore we prioritize the context window of source component when it overlaps with the target context window.

we keep overlapping context sentences in the source window, and remove them from the target window.


Finally, the FULL model includes all features in BASELINE and COMBINED models.

the FULL model is the COMBINED model plus word pairs and production rules.


in TA, the topic is chosen by the topic with the maximum median U .


Median encodes how "confusing" a topic is.

2 topic k * is the topic that its documents confuse the classifier most.


The goal of this model is to better handle control predicates and other phenomena that can be observed from the dependency structure of a sentence.

we aim to model the semantic relationships between a predicate and its arguments by analyzing the dependency path between the predicate word and each argument head word.


In the context of semantic role labeling, we define each path as a sequence from a predicate to its potential argument.

1 we define the first item x 1 to correspond to the part-of-speech tag of the predicate word w i , followed by its actual word form, and the relation to the next word w i+1 .


For both steps, we train several LSTM-based neural network models as described in Section 2.

we train separate networks for nominal and verbal predicates and for identification and classification.


We also discuss the trade-off between prediction accuracy and instancy.

we investigate how the prediction performance improves as we wait for more user dialogues to be collected.


The proposed method consistently outperforms the baseline throughout the number of sessions in the observation period.

the proposed method predicts well the dropout of users whose number of sessions is around five compared to the baseline.


Selection To arrive at the desired extension of a full referring expression-an individual object, in our case-, one additional element is needed, and this is contributed by the determiner.

for uniquely referring expressions ("the red cross"), what is required is to pick the most likely candidate from the distribution: the prediction of an expression such as "the brown shirt guy on right" is computed by first getting the responses of the classifiers corresponding to the words, individually for each object.


Positive Instances Getting positive instances from the corpus is straightforward: We pair each word in a referring expression with the representation of the region it refers to.

if the word "left" occurs 20,000 times in expressions in the training corpus, we have 20,000 positive instances for training its classifier.


We found no significant gains over the vanilla ENT baseline with both strategies.

down-sampling resulted in gains of only 0.002 and 0.001 F-score and up-sampling resulted in a drop of 0.011 and 0.013 F-score on clinicaldev and newswire-dev, respectively.


Most relevant to this work is the comparison between Open-IE and Semantic Role Labeling (Carreras and Màrquez, 2005).

they suggest that SRL's longer arguments introduce noise which hurts performance for downstream tasks.


Our system improves recall Overall, our system significantly outperforms both baselines by more than 8% gain in F1 score.

the numbers show clearly that we improve recall in the frequent categories of prepositional and adjectival modifiers.


These topic models were originally applied to monolingual data; however, various recent studies have proposed the use of probabilistic topic models in multilingual set-  Most multilingual topic models, including bilingual LDA (BiLDA) (Mimno et al., 2009;Ni et al., 2009), model a document-aligned comparable corpus, such as a collection of Wikipedia articles, where aligned documents are topically similar but are not direct translations 2 .

these models assume that the documents in each tuple share the same topic distribution and that each cross-lingual topic has a language-specific word distribution.


This study proposes a bilingual segmented topic model (BiSTM) that extends BiLDA to capture segment-level alignments through a hierarchical structure.

biSTM considers each document as a set of segments and models a document as a document-segment-word structure.


As can be seen in Figure 3, BiSTM introduces a segmentlevel layer between the document-and word-level layers in both languages.

persegment topic distributions for each language, ν e and ν f , are introduced between per-document topic distributions θ and topic assignments for words, z e and z f .


We assume that each segment is a sequence of topically-related passages.


(ii) We empirically verify the effectiveness of CPRA on a real-world, large-scale KB.

we evaluate CPRA on benchmark data created from Freebase.


The KB is then encoded as a graph G, with entities represented as nodes, and triple (h, r, t) a directed edge from node h to node t. We formally define KB completion as a binary classification problem.

given a particular relation r, for any entity pair (h, t) such that (h, r, t) / ∈ O, we would like to judge whether h and t should be linked by r, by exploiting the graph structure of G. Let R ⊆ R denote a set of relations to be predicted.


Our intuition is that relations sharing more common paths (features) are probably more similar in classification, and hence should be coupled.

we start with |R| clusters and each cluster contains a single relation r ∈ R. Here |•| is the cardinality of a set.


We notice that in most cases FB15K encodes a relation and its reverse relation at the same time.

once a new fact is observed, FB15K creates two triples for it, e.g., (x, film/edited-by, y) and (y, editor/film, x).


Actually, to enhance graph connectivity, PRA-style methods usually automatically add an inverse version for each relation in a KB (Lao and Cohen, 2010;Lao et al., 2011).

for each observed triple (h, r, t), another triple (t, r −1 , h) is constructed and added to the KB.


6 For this reason, we remove reverse relations in FB15K.

we regard r 2 to be a reverse relation of r 1 if the triple (t, r 2 , h) holds whenever (h, r 1 , t) is observed, and we randomly discard one of the two relations.


A conventional, count-based n-gram language model is not well-suited due to the issue of data sparsity.

the number of rows in the table storing n-gram statistics will explode as the number of possible sentence combinations grows exponentially with respect to both the vocabulary size, each sentence's length and the number of context sentences.


(8) and conditioning w t on both the preceding and following words, we get a context-dependent questionanswering model.

the proposed larger-context language model can be used for context-dependent question-answering, however, with computational overhead.


nouns, adjec-  tives and verbs are the ones which become more predictable by a language model given more context.


Differences in the category rankings by frequency and by coverage reveal that practices in some categories are less tightly clustered than others.

data Retention is the second rarest practice category but ranks fourth by segment coverage.


By contrast, in the CL learning regime, the incremental loss is computed as − log p (y t ), where p (y t ) is normalized only over the values of y t that are possible continuations once the grammar-induced constraints are taken into account, ignoring whatever weights the RNN predictor may (wrongly) believe should be put on impossible continuations.

the DSP-CL model incorporates the prior knowledge about well-formed derivation sequences that we have thanks to the grammar.


Paraphrase-based approaches (Fader et al., 2013;Berant and Liang, 2014) generate variants of the input question using a simple hand-written grammar and then rank these using a paraphrase model.

in their setting, the logical form assigned to the input question is that of the generated sentence which is most similar to the input question.


Formally, the pairwise similarity is defined as: where the subscripts i p and i t denotes position and timestamp for sentence i, respectively.

we treat sentences written consecutively within one minute as local descriptions and only calculate literal cosine similarity for them.


To utilize the dataset more sufficiently and draw more reliable conclusions, we perform crossvalidation during evaluation.

we randomly divide the dataset into three parts with equal sizes, i.e.


We also conduct manual pyramid evaluation in this study.

we use the modified pyramid scores as described in (Passonneau et al., 2005) to manually evaluate the summaries generated by different methods.


The input of the neural rule selection model is a vector representation fort, another vector representation forẽ, and a set of ξ vector representations for both source-side and target-side context words of r: In our model, C (r) is calculated differently depending on the number of nonterminals included in the rule.

equation 7 defines C out (r, n) to be context words (n-grams) around r and C in (r, n, X k ) to be boundary words (n-grams) covered by nonterminal X k in r. 1 The context words used for a translation rule r with K nonterminals are shown as below.


Despite the fact that the CSRS model can share information among instances using distributed word representations, it still poses an extremely sparse learning problem.

the numbers of unique subtreest and stringsẽ are extremely large,  and many may only appear a few times in the corpus.


Our preliminary results show that reranking candidates can be constructed by modifying unconfident actions in the baseline parser output, and letting the baseline parser re-decode the sentence from the modified action.

revising two incorrect actions of the baseline parser yields oracle with 97.79% UAS, which increases to 99.74% by revising five actions.


During the process, if we correct every encountered incorrectly determined action and let the baseline parser re-decode the sentence from the point, we need to revise 2,052 actions, averaging 1.2 actions per sentence.

the baseline parser can parse the 1,695 sentences correctly with 2,052 action being revised.


After learning BiDRL, we represent each document in the training and test dataset by the concatenation of its vector representation in both the source and the target languages.

for each training document d ∈ S, we represent it as where d is its corresponding translation and v * is the learned document representation in BiDRL.


14shows, for each node in the feature extractor of grSemi-CRFs, its gradient consists of two parts: (1) the gradients back propagated from high layer nodes (i.e., longer segments); and (2) the supervising signals from Semi-CRFs.

the supervision in the objective function is added to each node in grSemi-CRFs.


2 Given a set of tuples, our goal is to develop a parametric model that can provide a confidence score for new, unseen tuples.

we want to design and train models that define a function score(t 1 , R, t 2 ) that provides a quality score for an arbitrary tuple t 1 , R, t 2 .


For the DNN LSTM model, we first create a single vector for the two terms using an LSTM.

we concatenate t 1 , a delimiter token, and t 2 to create a single word sequence.


The task is to separate the true and false tuples in our test set.

the labels are 1 for true tuples and 0 for false tuples.


A training tuple τ = t 1 , R , t 2 is considered "similar" to τ if R = R , one of the terms matches exactly, and the other term has the same head word.


• Model C: further collapse all logical forms whose top-level arguments have the same denotation.

we map utterances  Figure 2: Derivations generated for the last utterance in Figure 1.


Our strategy was to automatically generate sequences of world states and ask Amazon Mechanical Turk (AMT) workers to describe the successive transformations.

we started with a random world state w 0 .


We construct a log-linear model for Model B by constructing features φ(z B ) (omitting the dependence on x i , c i for convenience) based on the Model A features φ(z A ).

φ(z B ) is the component-wise maximum of φ(z A ) over all z A that project down to z B ; φ(z C ) is defined similarly: Concretely, Model B's features include indicator features over LF conditions in Table 3 con-joined with every n-gram of the entire utterance, as there is no alignment.


However, for morphological features, results in Table 10 indicate that context-based word representation works slightly better than character-based representation.

character-based embedding (AE) does outperform context-based embedding (CW, SG) on decoding verbal morphological features, even though AE does not access any context information.


Specifically, character-based embedding (AE) does outperform context-based embedding (CW, SG) on decoding verbal morphological features, even though AE does not access any context information.

word form could be an explicit and discriminative cue for the model to decode the morphological feature of a word.


Low-dimensional embeddings will not be naturally aligned due to the non-unique nature of the SVD and the stochastic nature of SGNS.

both these methods may result in arbitrary orthogonal transformations, which do not affect pairwise cosine-similarities within-years but will preclude comparison of the same word across time.


6 Measuring semantic displacement After aligning the embeddings for individual timeperiods, we can use the aligned word vectors to compute the semantic displacement that a word has undergone during a certain time-period.

we can directly compute the cosinedistance between a word's representation for different time-periods, i.e.


We now show how diachronic embeddings can be used in a large-scale cross-linguistic analysis to reveal statistical laws that relate frequency and polysemy to semantic change.

we analyze how a word's rate of semantic change, depends on its frequency, f (t) (w i ) and a measure of its polysemy, d (t) (w i ) (defined in Section 4.4).


Our goal is to complement semantic role representations with additional spatial knowledge.

our goal is to infer temporally-anchored spatial knowledge between x and y, where semantic roles ARG i (x verb , x) and ARGM-LOC(y verb , y) exists in the same sentence.


An unabridged natural answer would be "not before or during drove, but certainly after drove for a few minutes until he was done picking up the package."

it is intuitive to consider polarity (whether x is or is not located at y) and temporal anchors (for how long?).


Annotators understood that Hsia was in Milan temporarily: for a few minutes before stopped, during the full duration of stopped and for a few hours after stopped.

hsia was elsewhere, then went to Milan and left after visiting with hsiao for a few hours.


x verb (y verb ) denote the verbs to which x (y) attach, and x roles (y roles ) denote the semantic roles of x verb (y verb ).

they arrived to Madison Square Garden shortly before honored and left shortly after, but stayed in New York City for some hours.


It is worth noting that mins, days and inf obtain relatively high F-measures for before: 0.80, 0.67 and 0.50 respectively.

we can distinguish whether an entity is somewhere only for a few minutes or days (but not longer) before an event, or at all times before an event.


Word sense disambiguation (WSD), as well as more general problems involving word senses, have been of great interest to the NLP community for many years (for a detailed overview, see Agirre and Edmonds (2007) and Navigli (2009)).

there has recently been a lot of work on unsupervised techniques for these problems.


Unlike older topic modelling methods such as LDA (Blei et al., 2003), HDP is nonparametric, meaning the number of topics used by the model is automatically learnt, and does not need to be set as a hyperparameter.

the model automatically learns the "right" number of topics for each lemma.


Rather, it is suggested that it is more important for topic modelling to use high-performance learning algorithms so that topics are learnt in correct proportions, in which case "junk" topics can easily be ignored (Buntine and Mishra, 2014).

it is likely that the previously-found performance advantage of HDP over LDA was actually due to properties of their respective Gibbs sampling algorithms.


The most immediate extension of our work would be to apply our sense learning method to a broader range of data.

we intend to expand LEXSEMTM by applying HCA-WSI across the vocabularies of languages other than English, and also to multiword lemmas.


A variety of factors converges in making proper nouns easy for subjects in the LAMBADA task.

when the context clearly demands a referential expression, the constraint that the blank be filled by a single word excludes other possibilities such as noun phrases with articles, and there are reasons to suspect that co-reference is easier than other discourse phenomena in our task (see below).


In comparative terms (and focusing on perplexity and rank, given the uniformly low accuracy results) we observe a stronger performance of the traditional N-Gram models over the neuralnetwork-based ones, possibly pointing to the difficulty of tuning the latter properly.

the best relative performance on LAMBADA is achieved by N-Gram w/cache, which takes passage statistics into account.


2015, uses an attention mechanism to better focus on the relevant part of the document for a given property.

attentive Reader first generates a representation u of the property using the final state of an LSTM while a second LSTM is used to read the document and generate a representation z t for each word.


(2013), our preliminary experiments with the treatment of rare words suggest that morphological features are useful for all SPMRL languages except French.

for Basque, Hungarian and Korean, improvements are significantly large.


However, because the connections for any specific parsing decision are fixed given the derivation, we can still extract examples for training off-line by unrolling the network structure from gold derivations.

we can utilize our approach with the same simple stochastic optimization techniques used in prior works.


We only retain discrete features over the labels in the incrementally constructed tree (Figure 4).

for any token of interest, we feed the hidden layer of the tagger network evaluated for that token as input to the parser.


The important implication at training time is that, unlike a recurrent network, the parser decisions are conditionally independent given a fixed history.

if we unroll the network structure ahead of time given the gold derivation, we do not need to perform inference when training with respect to these examples.


In addition to our analysis on user-environment emotional contrast and demographics, we discovered which users are more "opinionated" relative to their environment on Twitter.

users in which demographic group amplify less neutral but more subjective tweets e.g., positive, negative.


These results indicate that most user traits can be quite accurately predicted using solely the emotional tone and emotional contrast features of the users.

given the emotions expressed by a user, and contrasting these with the emotions expressed by user environment, one can accurately infer many interesting properties of the user without using any additional information.


The DBSCAN algorithm provides us with tunable parameters that can isolate better clusters.

the parameter controls the maximum distance between any two points in the same neighborhood.


Namely, is relation R that compares convingcingness of two arguments transitive, antisymmetric, and total?

does is exhibit properties such that if A≥B and B≥C, then A≥C (total ordering)?


We assume that π is a K-vector (where we take the limit as K → ∞) where π k describes the population proportion of documents that contain latent feature k. We suppose that π is generated by the stick-breaking construction (Doshi-Velez et al., 2009).

we suppose that η k ∼ Beta(α, 1) for all K. We label π 1 = η 1 and for each remaining topic, we assume that π k = k z=1 η z .


Response to Treatment Vector We assume that a K−vector of parameters β describes the relationship between the treatment vector and response.

we use a standard parameterization and suppose that τ We approximate the posterior distribution with a variational approximation, building on the algorithm introduced in (Doshi-Velez et al., 2009).


In this modified update step, we remove the component of the update that incorporates information about the outcome.

for individual i in the test set for category k we have the following update step For each text in the test set we repeat this update several times until ν test has converged.


We use softmax selection, which is widely applied in practice, to trade-off exploration vs. exploitation.

for each experiencereplay, we first generate 200 episodes of data (about 3K tuples in "Saving John" and 16K tuples in "Machine of Death") using the softmax selection rule in (2), where we set α = 0.2 for the first game and α = 1.0 for the second game.


The other contributions of this work includes a simple yet efficient algorithm which starts with identifying transferable knowledge from across multiple source domains useful for learning the target domain task.

it identifies relevant class-labels from the source domains such that the instances in those classes can induce classseparability in the target domain.


For this paper, we adopt a simpler method-MAP estimation of all latent vectors.

we seek the w, m that jointly maximize (6).


This finds the global minimum (convex objective) and is extremely fast even when we have over a hundred million real variables.

we update where Σ def = Σ −1 is the inverse covariance matrix and W k def = {i : k ∈ M i }.


We wish to optimize the model parameters λ, Σ i , Σ i by empirical Bayes.

we do not have a prior on these parameters, but simply do maximum likelihood estimation.


This predicts the embedding x via the vector-offset formula , where a, b, c ∈ C ∩ L are three other words sharing i's coarse part of speech such that M i can be expressed as M a + (M b − M c ).

9 the baseline chooses a, b, c uniformly at random from all possibilities.


Most of the relationseven the most difficult ones from our clustering experiment -are classified with very high Fscore.

with a simple linear transformation of the embedding dimensions, we are able to achieve near-perfect results.


For instance, the random pairs (have, works), (turn, took), and (works, started) were incorrectly classified as VERB 3 , VERB Past and VERB 3Past , respectively.

the model captures syntax, but lacks the ability to capture lexical paradigms, and tends to overgenerate.


This is mainly due to the fact that major components in the existing translation systems, such as language models and phrase tables, are a count-based estimator of probabilities.

a probability of a subsequence of symbols, or pairs of symbols, is estimated by counting the number of its occurrences in a training corpus.


(2013) proposed a method to improve character-level translation with phrasebased translation systems, however, with only a limited success.

(3) Vanishing Gradient to neural machine translation, a major reason behind the wide adoption of word-level modelling is due to the difficulty in modelling long-term dependencies with recurrent neural networks (Bengio et al., 1994;Hochreiter, 1998).


The activation of the faster layer is computed by whereȟ 1 t −1 andĥ 2 t −1 are the gated activations of the faster and slower layers respectively.

these gated activations are computed by the faster layer's activation is based on the adaptive combination of the faster and slower layers' activations from the previous time step.


layer has finished processing the current chunk and is about to reset itself (g 1 t ≈ 1).

the slower layer does not receive any input from the faster layer, until the faster layer has quickly processed the current chunk, thereby running at a slower rate than the faster layer does.


We used slightly different subsets of the full feature set for different languages.

we left out surface form features and/or bilingual features in some settings because they decreased performance, presumably due to over-fitting.


The input vectors are then passed to the LSTM recurrent layer, which computes a compositional representation − → h t at every time step t by performing nonlinear transformations of the current input x t and the output of the previous time step − → h t−1 .

the recurrent layer in a LSTM RNN is constituted with hidden units called memory blocks.


In this paper, we are concerned with automatically identifying the type of a situation entity (SE), which we assume to be expressed by a clause.

we present a system for automatically labeling clauses using the inventory of STATE: The colonel owns the farm.


The exception is removing local ordering probabilities, which yields a higher d. These observations support the hypothesis that all of the components of the objective function contribute to segmenting the album into subsequences of photos depicting the same event.

we note the degradation when removing the global ordering probabilities, indicating that approaches which model only local event transitions such as hidden Markov models would not be suitable for this task.


Similarly, we have seen increased adaptation of deep neural networks for natural language processing.

sequence-to-sequence training using recurrent neural networks has been successfully applied to machine translation (Cho et al., 2014;Bahdanau et al., 2015;Sutskever et al., 2014;Kalchbrenner and Blunsom, 2013).


We then examined the results and selected promising participants (comprising a "white list") for future task requests.

only the participants on the white list could see the next task.


As you can see, it outperformed the other two models for every metric.

the CIDEr-D score was about 4% higher than that for the monolingual baseline.


(2014) use similar language-based features as ours.

in (Fu et al., 2014), linguistic regularities between pretrained word vectors  are modeled as projection mappings.


We formulate the distribution p(z|x) through a model which leverages rich multi-modal features.

let c n be the set of child nodes of category x n in a taxonomy encoded by z.


To handle this, we restrict the candidate value of z n in eq 3, ensuring that the new z n is always a tree.

given a tree T , we define a structure operation as the procedure of detaching one node x n in T from its parent and appending it to another node x m which is not a descendant of x n .


Moreover, we leverage the distributed representations of images and words to construct compact and effective features.

each image i is represented as an embedding vector v i ∈ R a extracted by deep convolutional neural networks.


the hypernym-hyponym relationship between words can be approximated by a linear projection operator between word vectors Fu et al., 2014), we design a similar strategy to (Fu et al., 2014) between images and words so that the parent can be "predicted" given the image embedding of its child category and the projection matrix.

let (x n , x n ) be a parent-child pair in the training data, we learn a projection matrix Φ which minimizes the distance between Φv i n (i.e.


In addition to the embedding-based features, we further leverage lexical features based on the surface forms of child/parent category names.

we employ the Capitalization, Ends with, Contains, Suffix match, LCS and Length different features, which are commonly used in previous works in taxonomy induction (Yang and Callan, 2009;Bansal et al., 2014).


whether our model can learn statistical patterns from one hierarchy and transfer the knowledge to build a taxonomy for another collection of out-of-domain labels.

we select two trees as the training set to learn w. In the test phase, the model is required to build the full taxonomy from scratch for the third tree.


We use Ancestor F 1 as our evaluation metric (Kozareva and Hovy, 2010;Navigli et al., 2011;Bansal et al., 2014).

we measure F 1 = 2P R/(P + R) values of predicted "is-a" relations where the precision (P) and recall (R) are: We compare our method to two previously state-of-the-art models by Fu et al.


To investigate how the image representations affect the final performance, we compare the ancestor-F1 score when different pre-trained CNNs are used for visual feature extraction.

we employ both the CNN-128 model (128 dimensional feature with 15.6% top-5 error on ILSVRC12) and the VGG-16 model (4096 dimensional feature with 7.5% top-5 error) by Simonyan and Zisserman (2014), but only observe a slight improvement of 0.01 on the ancestor-F1 score for the later one.


3 Modeling Physical Causality for Action Verbs Verb semantics have been studied extensively in linguistics (Pustejovsky, 1991;Levin, 1993;Baker et al., 1998;Kingsbury and Palmer, 2002).

for action verbs (such as run, throw, cook), Hovav and Levin  propose that they can be divided into two types: manner verbs that "specify as part of their meaning a manner of carrying out an action" (e.g., nibble, rub, scribble, sweep, flutter, laugh, run, swim), and result verbs that "specify the coming about of a result state" (e.g., clean, cover, empty, fill, chop, cut, melt, open, enter).


We apply logistic regression to train a separate classifier for each attribute.

for the features of a verb, we use the Distributional Memory (typeDM) (Baroni and Lenci, 2010) vector.


To support the claim that learning ROUGE scores for individual sentences is easier than solving the whole summarization task, it is sufficient to choose a basic learner with simple features and little in-domain training data (models are trained on one DUC dataset and evaluated on another).

we employ a support vector regression (SVR).


While we justified our choice of ROUGE-1 recall and ROUGE-2 recall as optimization objectives by their strong correlation with human evaluation methods, ROUGE-N has also various drawbacks.

it does not take into account the overall discourse coherence of a system summary (see the supplemental material for examples of summaries generated by our framework).


We prove the following formula: Proof: Let g ∈ S * be a n-gram in the reference summary, and k ∈ [1, m] the number of sentences in which it appears.

in order to prove the formula (20), we have to find an expression for the (k) that gives to g the correct contribution to the formula: First, we observe that g does not appear in the terms that contain the intersection of more than k sentences.


Even if any of the existing rules do not apply to a part of the sequence of the given POS tags, we stick to Principle (P3) as much as possible.

we determine partial phrase structures according to the given POS sequence to which the existing rule set applies.


Note that our model is trained on the user guessesê, not the reference translations e * .

we seek parameters θ that would explain why all users made their guesses.


Our main goal is to investigate the similarities and differences between these two phenomena, and contrast them with native language.

we are interested in the reasons for the differences between translations and originals, on one hand, and native and non-native language, on the other.


We propose a unified computational umbrella for exploring two related areas of research on bilingualism: translation studies and second language acquisition.

we put forward three main hypotheses: (1) The three language varieties have unique characteristics that make them easily distinguishable.


We use supervised and unsupervised classification (Section 4) to show that the three language varieties are easily distinguishable.

we show that native and advanced non-native productions can be accurately separated.


Bootstrapping is a statistical technique involving random re-sampling (with replacement) from the original sample; it is often used to assign a measure of accuracy (e.g., a confidence interval) to an estimate.

let C N , C NN and C T denote native, non-native and translated sub-corpora of equal size (780K tokens).


Limited by the amount of text available for each individual language, we set out to test this hypothesis by inspection of two language families, Germanic and Romance.

the Germanic family consists of NN texts delivered by speakers from Austria, Germany, Netherlands and Sweden; and the Romance family includes NN speakers from Portugal, Italy, Spain, France and Romania.


Autoencoders are trained using a local unsupervised criterion (Vincent et al., 2010;Hinton and Salakhutdinov, 2006;Vincent et al., 2008).

the basic autoencoder in Figure 1(a) locally optimizes the hidden representation h of its input x such that h can be used to accurately reconstruct x, wherex is the reconstruction of x, the learning parameters W ∈ R d ×d and W ∈ R d×d are weight matrices, b h ∈ R d and bx ∈ R d are bias vectors for the hidden and output layers respectively, and g is a nonlinear function such as tanh(.).


When context information is not readily available, we use topic models to determine such context for individual inputs (Blei et al., 2003;Stevens et al., 2012).

we use Non-Negative Matrix Factorization (NMF) (Lin, 2007): Given a training set with n instances, i.e., X ∈ R v×n , where v is the size of a global vocabulary and the scalar k is the number of topics in the dataset, we learn the topic matrix D ∈ R v×k and context matrix C ∈ R k×n using the following sparse coding algorithm: where each column in C is a sparse representation of an input over all topics and will be used as global context information in our model.


(2012) presented an RNN model that uses document-level context information to construct more accurate word representations.

given a sequence of words, the approach uses other words in the document as external (global) knowledge to predict the next word in the sequence.


Despite the simplicity of the features, we can see that the aspects can be inferred from the articles.

the model trained on the BoW features achieves an RMSE that is very close to that of the BaselineM, whereas the   In this paper, we proposed an annotated corpus for controlling the editorial quality of online news through 14 aspects related to editors perceived quality of news articles.


In addition to the Full model, we created an ensemble system that combines the probabilities described above using a weighting schema, which we call "Full weighted model".

we add an exponent coefficient to the probabilities, thus allowing to control the contribution of each model.


In contrast, we are concerned with a very large dataset of templates extracted from a single corpus, where any two templates are potential paraphrases.

paraphrasal templates can be extracted from sentences which are not in fact paraphrases; for example, the sentences "The population of Missouri includes more than 1 million African Americans" and "Roughly 185,000 Japanese Americans reside in Hawaii" can produce the templated paraphrases "The population of [american state] includes more than [number] [ethnic group]" and "Roughly [number] [ethnic group] reside in [american state]".


Using this metric, we apply single-linkage agglomerative clustering, with the stopping criteria defined as a threshold τ for the maximum sum of squared errors (SSE) within any cluster.

the algorithm stops linking if the cluster C that would be created by the next link satisfies: Where µ C is the centroid of C and d is the Euclidean distance.


However, keep in mind that it is not a comparable figure across the methods, since different corpora are used.

it is expected to be significantly higher for parallel corpora, where the entire corpus consists of potential paraphrases (and that fact is reflected in Table 4, where some methods that use parallel corpora have a PPS that is an order of magnitude higher than other methods).


This problem is easily solved by using transition-based decoders, which excel in this kind of setting due to their incremental approach and their low theoretical complexity.

we use a transition-based decoder that extends the Swap transition system of Nivre (2009) in order to introduce sentence boundaries during the parsing process.


Since each sentence constitutes its own subtree under the root node, a regular sentencebased oracle can be used to derive the oracle transition sequence for complete documents.

we apply the sentence-based oracle to each sentence, and then join the sequences with a SB transition in between.


Moreover, they show that updates covering a greater subsequence can also be valid and define the max-violation update.

max-violation updates extend the beam beyond early updates and apply updates where the maximum difference in scores as defined by Equation (1) between the correct solution and the best prediction (i.e., the maximal violation) is used for update.


The average length of seen transition sequences never even goes above 50%.

more than half of the training data is never used.


As the inference is intractable, we propose to sample the full search space to improve the efficiency.

our approach has the following advantages: 1.


Instead of using all sentences in the monolingual corpora, we constructed smaller monolingual corpora with zero OOV ratio: 2.56M Chinese sentences with 47.51M words and 2.56M English English sentences with 37.47M words.

the monolingual corpora we used in the experiments are much smaller than those used by MOSES.


In the English-to-Chinese direction, we obtain similar findings.


This function results from the composition of simple differentiable functions or layers.

f composes an input mapping from discrete word indexes to continuous vectors, a succession of linear operations followed by hyperbolic tangent non-linearities, plus one final linear operation, followed by a softmax normalization.


Even when larger units are explicitly represented in DSMs (McCarthy et al., 2003;Reddy et al., 2011;Mikolov et al., 2013c;Ferret, 2014), it is not clear whether the quality of these representations is comparable to the representations of single words.

when building vectors for larger units, their generally lower frequencies in corpora (Kim and Baldwin, 2006) may combine with morphosyntactic phenomena to increase sparsity even further, often requiring non-trivial preprocessing (lemmatization and word reordering) to conflate variants.


Indeed, various annotation scales have been proposed as means to collect human judgments about compositionality.


PPMI For each target word or compound, we extract from the corpus its neighboring nouns and verbs in a symmetric sliding window of w words to the left/right 5 , using a linear decay weighting scheme with respect to its distance d to the target (Levy et al., 2015).

each cooccurrence count of target-context pairs is incremented by w + 1 − d instead of 1.


The impact of WINDOWSIZE has a similar trend to the one found for the Reddy++ and Reddy datasets (Figure 2(c)).

the larger window was preferred by most models, but the average difference between the best and the worst size for each DSM is only 0.01.


Furthermore, we introduce anaphora constraints (Section 2.2) via a new set of variables that capture the process of rewriting pronouns to make them  explicit mentions.

x REF ij = 1 if we should rewrite the jth pronoun in the ith unit with its antecedent.


Denote by T syn(kl) a nontrivial (supports some compression) subtree of T syn that is completely contained in an EDU (i, j).

we build the following combined compression tree, which we refer to as the augmentation of T rst with T syn(kl) : we maintain the existing tree structure except for the EDU (i, j), which is broken into three parts: the outer two depend on each other (is a claims adjuster and .


(2013), since their parser is not publicly available.

article on Speak-Up, program begun by Westchester County Office for the aging to bring together elderly and college students.


Let s i and s j be two tokenised and truecased sentences in languages l 1 and l 2 , respectively, S i the set of tokens in s i , S j the set of tokens in s j , T ij the set of expanded translations into l 2 for all tokens in S i , and T ji the set of expanded translations into l 1 for all tokens in S j .

the StACC similarity score is then computed as in Equation 1: (1) the score is defined as the average of the Jaccard similarity coefficients obtained between sentence token sets and expanded lexical translations in both directions.


A likely explanation for these results is the nature of the features that compose the LEX-ACC model.

the features related to alignment obliqueness and number of initial/final aligned words might be detrimental in the case of Basque, which exhibits free word order.


We model these properties using a hidden vector which is the consistent unification of the features of the middle word and the context.

there must exist a hidden vector which entails both of these vectors, and is consistent with prior constraints on vectors.


Description We present an end-to-end generation system that performs content selection and surface realization jointly.

our approach uses an attention-based neural network to model the conditional distribution of a NL summary n given a code snippet c. we use an LSTM model that is guided by attention on the source code snippet to generate a summary one word at a time, as shown in Figure 2.


For efficiently maintainable online content, we need automatic synthesis of ASL from a sparse script representing the lexical items and basic elements of the sentence.

we are studying how to model and generate ASL animations that include syntactic facial expressions, conveying grammatical information during entire phrases and therefore constrained by the timing of the manual signs in a phrase (Baker-Shenk, 1983).


Our recent work  has established a set of demographic and technology experience questions which can be used to screen for the most critical participants in a user study of ASL signers to evaluate animation.

we screened for participants that identified themselves as "deaf/Deaf" or "hard-of-hearing," who had grown up using ASL at home or had attended an ASL-based school as a young child, such as a residential or daytime school.


To facilitate the creation of ASL content that can easily be updated or maintained, we have investigated technologies for automating the synthesis of ASL animations from a sparse representation of the message.

this paper has focused on the synthesis of syntactic ASL facial expressions, which are essential to sentence meaning, using a data-driven methodology in which recordings of human ASL signers are used as a basis for generating face and head movements for animation.


Importantly, a parser that employs our undirected inference algorithm can generate all possible trees, projective and non-projective.

the undirected MST problem ( § 2) has a randomized algorithm which is O(m) at expectation and with a very high probability ((Karger et al., 1995)), as well as an O(m • α(m, n)) worst-case deterministic algorithm (Pettie and Ramachandran, 2002), where α(m, n) is a certain natural inverse of Ackermann's function (Hazewinkel, 2001).


At the last part of this section we therefore present a simple, O(n) time, local enhancement procedure, that improves the score of the directed tree generated from the output of the undirected MST parser with respect to the edge scores of a standard directed MST parser.

our procedure improves the output of the undirected MST parser with respect to a directed model without having to compute the MST of the latter, which would take O(n 2 ) time.


We conclude this section with a final remark stating that the output class of our inference algorithm is non-projective.

it can generate all possible parse trees, projective and non-projective.


one of the schemes employed by existing directed MST parsers) and then transform these directed weights into undirected ones.

given two vertices u and v with directed edges (u, v) and (v, u), weighted with , u) respectively, the goal is to compute the weight s u (û , v) of the undirected edge (û , v) connecting them in the undirected graph.


While the directed MST parser (D-MST) is the best performing model across almost all test sets and evaluation measures, it outperforms our best model, U-MST-uf-lep, by a very small margin.

for D-UAS, D-MST outperforms U-MST-uf-lep by up to 1% in 14 out of 20 setups (in 6 setups the difference is up to 0.5%).


We link microblog posts using reposting and replying rela-tions to build conversation trees.

the root of a conversation tree refers to the original post and its edges represent the reposting/replying relations.


So, the topics of leaders on tree t are directly sampled from the topic mixture θ t .

to model the internal topic correlations within the subgraph of conversation tree consisting of a leader and all its followers, we capture parentchild topic transitions π k ∼ Dir(γ), which is a distribution over K topics, and use π k,j to denote the probability of a follower assigned topic j when the topic of its parent is k. if message m is sampled as a follower and the topic assignment to its parent message is z t,p(m) , where p(m) indexes the parent of m, then z t,m (i.e., the topic of m) is generated from topic transition distribution π z t,p(m) .


We first define the notations of all variables needed by the formulation of Gibbs sampling, which are described in Table 1.

the various C variables refer to counts excluding the message m on conversation tree t. For each message m on a tree t, we sample the leader switcher y t,m and topic assignment z t,m according to the following conditional probability distribution: where g(s, k, t, m) takes different forms depending on the value of s: For each word n in m on t, the sampling formula of its background switcher is given as the following: where To evaluate our LeadLDA model, we conducted experiments on real-world microblog dataset collected from Sina Weibo that has the same 140character limitation and shares the similar market penetration as Twitter (Rapoza, 2011).


However, due to the severe sparsity of short text posts, we modify the calculation of commonly-used topic coherence measure based on word co-occurrences in messages tagged with the same hashtag, named as hashtag-document, assuming that those messages discuss related topics 8 .

we calculate the coherence score of a topic given the top N words ranked by likelihood as below: where w k i represents the i-th word in topic k ranked by p(w|k), D(w k i , w k j ) refers to the count of hashtag-documents where word w k i and w k j cooccur, and D(w k i ) denotes the number of hashtagdocuments that contain word w k i .


It happens mainly because the performance of SF is very sensitive to the wrongly mapped frames.

if a frame is mismapped, then all sentences under it would be mislabeled as events.


Black and Wilcox (2011) note that writers take informed and careful decisions when attributing names to their characters.

while care is taken to have names that are easily identifiable and phonologically attractive, or that are important for personal reasons, these are not the only considerations: names are chosen so that they match the personality, the past, and the cultural background of a character.


The phonological features are important separation criteria as evidenced by the drop in performance when they are excluded from the experimental setup (Table 5).

using all features except phonological features is equivalent to using phonological features alone (about F = 79% in both cases) and slightly worse that using all name-intrinsic features (about F = 80%).


The position of each phoneme also seems to play an crucial role Phonemes Class /p/, /b/ (bilabial plosive) P /l/ (alveolar lateral) Table 7: Consonants behavior in the classification task.

we note that starting with a vowel or a consonant are among the most discriminating features.


In sentences which contain the AN naturally, there is a clear bias toward judgements of "entailment."

in contexts when an AN appears, it is often the case that this A is superfluous: the information carried by the A is sufficiently entailed by the context that removing it does not remove information.


inferring entailment relations based on the entailments generated by INS and DEL editsdoes not enforce that all of the ANs fit into one of the five entailment relations defined by natural logic.

we observe many instances (∼5% of p/h pairs) in which INS is determined to generate a contradiction, while DEL is said to generate an entailment.


We test the performance of those same implementations on our Add-One task.

we test the following models: a basic Sum-of-words model (Sum), which represents both p and h as the sum of their word embeddings, an RNN model, and an LSTM model.


Second, essays are longer and more formally written than the text typically used in previous stance classification research (e.g., debate posts).

a student essay writer typically expresses her stance on the essay's topic in a thesis sentence/clause, while a debate post's author may never even explicitly express her stance.


To understand how strong n-grams are when evaluated in our leave-one-prompt-out crossvalidation setting, we employ them as features in our second baseline.

we train a multiclass classifier on our data set using a feature set composed solely of unigram, bigram, and trigram features, each of which indicates the number of times the corresponding n-gram is present in the associated essay.


Stancetaking text indicating neutrality tends to be phrased somewhat differently than stancetaking text indicating any other class.

neutral text often makes claims that are about the prompt part's subject, but which are tangential to the proposition expressed in the prompt part.


When we believe there is strong evidence that an instance should belong to one of the Strongly classes, we turn on the corresponding (Dis)Agree Strongly feature.

if we find a relevant stancetaking path that appears to agree with the prompt part (as described in Section 4.1.2), but do not find any such path that appears to disagree with it, we turn on the Agree Strongly feature.


When training our approach, we perform exhaustive feature selection to determine which sub-set of the four sets of features (i.e., n-gram, duplicated Faulkner, path-based, and knowledge-based features) should be used.

we select the feature groups and learner jointly by performing cross validation on the training folds, choosing the combination yielding the highest average of micro and macro F-scores in each fold experiment.


Since all the systems we examined fared poorly on identifying Somewhat classes, one may wonder how these systems would perform if we considered a simplified version of the task where we merged each Somewhat class with the corresponding Strongly class.

since Faulkner's approach was originally not designed to distinguish between Strongly and Somewhat classes, it may seem fairer to compare our approach against Duplicated Faulkner on the four-class essay stance classification task, where stance can take one of four values: Agree (created by merging Agree 10 All significance tests are approximate randomization tests with p < 0.01.


To encourage diversity among committee members, we train them with different datasets and features.

each base segmenter adopts one of three types of feature templates (shown in Table 1), and are trained on randomly sampling training sets.


Based on the comparative study of the two learning frameworks and through error analysis of the output of the state-of-the-art systems, we identify key strengths and weaknesses of each of these approaches and demonstrate their complementarity.

the machine translation method learns from parallel data without requiring further linguistic input and is better at correcting complex mistakes.


• As a result, we build several systems that combine the strengths of both frameworks and demonstrate substantial progress on the task.

the best system outperforms the previous best result by 7.4 F score points.


In this work, we focus on reducing such dependencies and propose a domain-invariant framework for the disease name recognition task.

we propose various end-to-end recurrent neural network (RNN) models for the tasks of disease name recognition and their classification into four pre-defined categories.


Our motivation can be summarized in the following question: can we identify disease name and categorize them without relying on feature en-gineering, domain-knowledge or task specific resources?

we can say this work is motivated towards mitigating the two issues: first, feature engineering relying on linguistic and domain-specific knowledge; and second, bring flexibility in capturing influential words affecting model decisions irrespective of their occurrence anywhere within the sentence.


In this case RNN models are giving better results than the NN model for both the tasks.

performance of Bi-LSTM models are best than others in both the tasks.


We measured performance in terms of accuracy across all possible topic pairings.

we paired each of the 4 topics in the Guardian corpus with each of the 3 remaining topics: train on Politics, test on Society; train on Politics, test on UK; train on Politics, test on World; etc.


In the field of linguistics, each language is assumed to have a basic word order that is fundamental to its sentence structure and in most cases there is a generally accepted theory on the word order for each structure.

even if there are several possible word orders for essentially same sentences consisting of the same elements, only one of them is regarded as the canonical word order and the others are considered to be generated by scrambling it.


Thus, while these analyses are reliable for the example they focus on, they cannot be easily generalized to other examples 1 .

another manual analysis or measurement is required to consider the canonical word order of another example.


With regard to Claim D, Matsuoka (2003) asserted that the canonical order varies depending on the semantic role of the dative argument.

the DAT-ACC order is more preferred when the semantic role of dative argument is animate Possessor than when the semantic role is inanimate Goal.


Therefore, in this work, we decided to extract examples only from reliable parts of dependency parses.

we used a corpus consisting of more than 10 billion unique sentences extracted from the Web, selected parse trees that have no syntactic ambiguity, and then extracted examples only from the selected parse trees.


We can see that there are differences in the   proportions of the ACC-DAT order.

when the dative argument's category is PLACE-FUNCTION such as shita (bottom) and yoko (side) or ANIMAL-PART such as te (hand) and mi (body), the ACC-DAT order is more preferred than otherwise.


First, we collected examples that satisfied the following two conditions: the accusative argument belongs to ARTIFACT-OTHER category, and the dative argument belongs to either PLACE-INSTITUTION or PERSON category.

we call the former Type-A 11 , and the latter Type-B hereafter, and consider that the semantic role of the dative argument is inanimate Goal in most cases 11 the categories of the accusative and dative arguments of a Type-A example are ARTIFACT-OTHER and PLACE-INSTITUTION, respectively.


On the other hand, graphs (a), (b), and (c) have quite different tendencies.

all search methods compete with each other for CW embeddings, while they could not perform well for GV embeddings.


Learning distributed representations for relation instances is a central technique in downstream applications as a number of recent studies demonstrated the usefulness of distributed representations for words (Mikolov et al., 2013;Pennington et al., 2014) and sentences (Sutskever et al., 2014;Kiros et al., 2015).

semantic modeling of relations and their textual realizations (relational patterns hereafter) is extremely important because a rela- increase the risk of: cause: Corpus (ukWaC) ... evidence that passive smoking increases the risk of lung cancer by 10%.


In instancebased judgment, an annotator judges a pair of relational patterns whose variable slots are filled with the same entity pair.

he or she does not make a judgment for a pair of relational patterns with variables, "X prevent Y" and "X reduce the risk of Y", but two instantiated statements "Cephalexin prevent the bacteria" and "Cephalexin reduce the risk of the bacteria" ("X = Cephalexin, Y = the bacteria").


Ratings of the Gold examples were used merely for quality assessment of the workers.

we discarded the Figure 2: Number of judgments for each similarity rating.


To examine the usefulness of the dataset and distributed representations for a different application, we address the task of relation classification on the SemEval 2010 Task 8 dataset (Hendrickx et al., 2010).

we explore whether high-quality distributed representations of relational patterns are effective to identify a relation type of an entity pair.


Therefore, the use of this function encourages the coverage of more different words in the summary.

the diversity of the summary is enhanced.


For topic modeling, we adopted the most popular LDA to discover topics in the two datasets, respectively.

we used the LDA module implemented in the MALLET toolkit 1 .


We have two goals in the evaluation: comparison of different summarization methods for topic labeling, and comparison of different kinds of labels (summaries, words, and phrases).

we compare our proposed summarization method (denoted as Our Method) with the following typical summarization methods and all of them extract summaries from the same candidate sentence set for each topic: MEAD: It uses a heuristic way to obtain each sentence's score by summing the scores based on different features : centroidbased weight, position and similarity with first sentence.


When no prefix or suffix exists, the corresponding embedding is set to zero.

we place an extra forward LSTM layer on top of the Bidirectional LSTM layer and learn segment embeddings using LSTM-Minus based on this forward LSTM.


Then we use the annotated gold answers of these questions and Wikipedia to create the training data.

we treat the sentences that contain correct/incorrect answers as positive/negative examples for the refinement model.


Since this dataset contains only questionanswer pairs and annotated topic entities, instead of relying on gold relations we rely on surrogate gold relations which produce answers that have the highest overlap with gold answers.

for a given question, we first locate the topic entity e in the Freebase graph, then select 1-hop and 2-hop relations connected to the topic entity as relation candidates.


We perform inference on Freebase using joint EL and RE, and then inference on Wikipedia to validate the results.

we treat the top two predictions of the joint inference model as the candidate subject and relation pairs, and extract the corresponding answers from each pair, take the union, and filter the answer set using Wikipedia.


While the eventual aim is a completely automatic system, our methodology overcomes several major challenges in the manual semantic annotation of clinical images for cognitive assessment, even with manual transcriptions.

our methodology is fully objective, sensitive to differences between groups, and generalizable to new stimuli which is especially important if longitudinal analysis is to avoid the socalled 'practice effect' by using multiple stimuli.


With the coupling vectors generated, we have the following equation for the decoding process with attention mechanism: Inspired by the observation that a human couplet generator might recompose the clause for several times, we propose a polishing schema for the couplet generation.

after a singlepass generation, the couplet generator itself shall be aware of the generated clause as a draft, so that polishing each and every character of the clause becomes possible.


In the ILLG setting, the two parties do not initially speak a common language, but nonetheless need to collaboratively accomplish a goal.

we created a game called SHRD-LURN, 1 in homage to the seminal work of Winograd (1972).


Our features are n-grams (including skip-grams) conjoined with tree-grams on the logical form side.

on the utterance side (e.g., 'stack red on orange'), we use unigrams ('stack', * , * ), bigrams ('red', 'on', * ), trigrams ('red', 'on', 'orange'), and skip-trigrams ('stack', * , 'on').


We generate logical forms from smallest to largest using beam search.


5 However, for semantic parsing, we would not be in this regime even if we have a large amount of training data.

we are nowhere near that regime in SHRDLURN, and most of our utterances / logical forms are seen only once, and the importance of modeling pragmatics remains.


Most players use utterances that actually do not match our logical language in Table 1, even the successful players.


Rather than assuming any particular parametric relationship between the predictor and target variables, kernel regression assumes only that the value of the target vari-able is a smooth function of the value of the predictors.

given a new point in predictor space, the value of the target at that point can reasonably be estimated by the value of the targets at points that are nearby in the predictor space.


They note that this metric induces a kernel function whose parameters are set entirely from the data.

mLKR can learn a weight matrix W for a mahalanobis metric that optimizes the leave-one out mean squared error of kernel regression (mSE), defined as: whereŷ i is estimated usingŷ j for all i = j, as in Eq.


Recently, deep learning techniques showed good performance in capturing the indicative information in such paths.

several papers show improved performance using recurrent neural networks (RNN) that process a dependency path edge-by-edge.


First, we focused on improving path representation using LSTM, resulting in a path-based model that performs significantly better than prior path-based methods, and matches the performance of the previously superior distributional methods.

we demonstrated that the increase in recall is a result of generalizing semantically-similar paths, in contrast to prior methods, which either make no generalizations or over-generalize paths.


We propose a general framework capable of enhancing various types of neural networks (e.g., CNNs and RNNs) with declarative first-order logic rules.

we develop an iterative distillation method that transfers the structured information of logic rules into the weights of neural networks.


Methodologically, our approach can be seen as a combination of the knowledge distillation (Hinton et al., 2015;Bucilu et al., 2006) and the posterior regularization (PR) method (Ganchev et al., 2010).

at each iteration we adapt the posterior constraint principle from PR to construct a rule-regularized teacher, and train the student network of interest to imitate the predictions of the teacher network.


We encode the FOL rules using soft logic (Bach et al., 2015) for flexible encoding and stable optimization.

soft logic allows continuous truth values from the interval [0, 1] instead of {0, 1}, and the Boolean logic operators are reformulated as: Here & and ∧ are two different approximations to logical conjunction (Foulds et al., 2015): & is useful as a selection operator (e.g., A&B = B when A = 1, and A&B = 0 when A = 0), while ∧ is an averaging operator.


For the first property, we apply a commonly-used strategy that imposes the rule constraints on q through an expectation operator.

for each rule (indexed by l) and each of its groundings (indexed by g) with confidence λ l .


Our empirical results show that both models substantially improve over the base network that is trained with only datalabel instances.

in general q performs better than p. q is more suitable when the logic rules introduce additional dependencies (e.g., span-Algorithm 1 Harnessing NN with Rules , Parameters: π -imitation parameter C -regularization strength 1: initialize neural network parameter θ 2: repeat 3: Sample a minibatch (X, Y ) ⊂ D 4: Construct teacher network q with Eq.


As training goes on, we gradually bias towards emulating the teacher predictions to effectively distill the structured knowledge.

we define π (t) = min{π 0 , 1 − α t } at iteration t ≥ 0, where α ≤ 1 specifies the speed of decay and π 0 < 1 is a lower bound.


We further leverage the list structures within and across sentences of the same documents.

named entities at corresponding positions in a list are likely to be in the same categories.


For the base neural network we use the "nonstatic" version in (Kim, 2014) with the exact same configurations.

word vectors are initialized using word2vec (Mikolov et al., 2013) and fine-tuned throughout training, and the neural parameters are trained using SGD with the Adadelta update rule (Zeiler, 2012).


We see that: 1) Although all methods lead to different degrees of improvement, our framework outperforms all other competitors with a large margin.

2) compared to the pipelined method in Row 6 which is in analogous to the structure compilation work (Liang et al., 2008), our iterative distillation (section 3.2) provides better performance.


The subsampling is conducted on the sentence level.

for instance, in "5%" we first selected 5% training sentences uniformly at random, then trained the models on these sentences as well as their phrases.


We have developed a framework which combines deep neural networks with first-order logic rules to allow integrating human knowledge and intentions into the neural models.

we proposed an iterative distillation procedure that transfers the structured information of logic rules into the weights of neural networks.


Notwithstanding the apparent formal stability of Icelandic cases, there is evidence that they are experiencing subtle changes in their functions (Barðdal, 2011;Barðdal and Eythórsson, 2003;Eythórsson, 2000).

barðdal argues that an accumulation of small syntatico-semantic shifts has finally resulted in a shift in the Icelandic dative's functions (i.e., 'dative sickness'), possibly triggered by earlier changes in nominatives and accusatives (e.g., 'nominative sickness').


Neural network approaches have taken the field of natural language processing (NLP) by storm.

variants of long short-term memory (LSTM) networks (Hochreiter and Schmidhuber, 1997) have produced impressive results on some of the classic NLP tasks such as part-ofspeech tagging , syntactic parsing (Vinyals et al., 2015) and semantic role labeling (Zhou and Xu, 2015).


Our model achieves state-of-the-art accuracy on all of these tasks, matching or outperforming LSTMs while being significantly faster.

for dependency parsing on the Wall Street Journal we achieve the best-ever published unlabeled attachment score of 94.61%.


(2015) for each training stage of our model.

we use averaged stochastic gradient descent with momentum, and we tune the learning rate, learning rate schedule, momentum, and early stopping time using a separate held-out corpus for each task.


Our model configuration is basically the same as the one originally proposed by Chen and Manning (2014) and then refined by .

we use the arc-standard transition system and extract the same set of features as prior work: words, part of speech tags, and dependency arcs and labels in the surrounding context of the state, as well as k-best tags as proposed by .


A recent result by Liberty (2013) gives a deterministic matrix sketching algorithm that tightly preserves the covariance structure needed for PCA.

given a sketch size l, the algorithm computes Y ∈ R l×d such that This result guarantees that the error decreases in O(1/l); in contrast, other approximation techniques have a significantly worse convergence bound of O(1/ √ l).


We use a bag-of-words vector to represent a sentence.

each sentence is a ddimensional vector x ∈ R d where d is the size of the vocabulary and x i is the count of an n-gram i in the sentence (we use up to n = 3 in experiments); we denote this representation by SENT.


We propose using random hashing to avoid this problem.

we pre-define the hash size H we want, and then on encountering any word w we map w → {1 .


Once we learn a PCA projection matrix Π, we use it in both training and test times to obtain a dense feature vector of a bag-of-words sentence representation.

if x is the original bag-ofwords sentence vector, the new representation is given by where ⊕ is the vector concatenation operation.


Next, we expand the information available about these users by collecting information about their social networks.

for each user mentioned in a tweet by one of the 102,328 users, we collect up to the 200 most recent English tweets for these users from January and February 2015.


An n-dimensional vector then encodes the user's social network as a bag-of-words over this user vocabulary.

a user is represented by a summation of the one-hot encodings of each neighboring user in their social network.


On average, 70.46% of the pairs follow the observation.

the pairs keep the property more often (i.e., 74.51%) when the polarity of T1 is negative.


Parameters α and λ are tuned on a grid of values 4 by cross validation on the source data.

we  Figure 2: Impact of the noise parameter p on the average accuracy for the 12 Amazon adaptation tasks.


More importantly, this approach inevitably leaves out some nonlocal information which could be useful.

though such a model can exploit similarities between words and other embedded categories, and learn interactions among those atomic features, it cannot exploit any other details of the text.


The central idea behind this approach is exploiting the power of recurrent neural networks to let the model decide what apsects of sentence context are important to making parsing decisions, rather than relying on fallible linguistic information (which moreover requires leaving out information which could be useful).

we model an input sentence using Long Short-Term Memory networks (LSTM), which have made a recent resurgence after being initially formulated by Hochreiter and Schmidhuber (1997).


We then ask how the embedding vectors in one time period (graph) can be predicted from the embedding vectors of neighbor words in previous time periods.

we postulate that there is a linear functional relationship that couples a word's today's meaning with its neighbor's meanings in the past.


(2010) and Fleck (2008) are designed to handle phonological variability.

they are designed to cope with words which have multiple transcribed pronunciations ([wan] and [want] for "want"); this kind of alternation can insert or delete whole segments, or change a vowel sound from one perceptual category to another.


We then use a Metropolis-Hastings acceptance test to correct for the discrepancy between our finite-state encoding and the actual model probability caused by repetitions of a lexical item within the same utterance.

for each vowel s ij , we sample a cutoff c ij ∼ U [0, P (s ij |X ij )].


This representation is used to predict whether the word is an event trigger or not.

we first use a Bi-LSTM to encode semantics of each word with its preceding and following information.


As show in (Chen et al., 2015), we take all the words of the whole sentence as the input and each token is transformed by looking up word embeddings.

we use the Skip-Gram model to pre-train the word embeddings to represent each word (Mikolov et al., 2013;Bahdanau et al., 2014).


This approach has been used for event detection in previous studies (Nguyen and Grishman, 2015;Chen et al., 2015).

we use multiple convolutional filters with different widths to produce local context representation.


Our evaluation task was based on the adequacy criterion.

judges were asked to estimate how much of the meaning of the human translation was expressed in the MT output (see Figure 1).


Diff As shown in Table 1, the agreement is consistently lower for annotators using different references.

the same MT outputs systematically receive different scores when differ-ent human translations are used for their evaluation.


This seems to indicate that the effect of reference bias is minimal half-way through the evaluation process.

when the annotators are already acquainted with the task but not yet tired, they are able to better recognize meaning-preserving variation between different translation options.


Although there are many ways to incorporate this, we propose a simple method whereby the entity descriptions are used to initialize the entity representations of the model, which we show to have empirical benefits.

we first decompose the description of a given entity into a sequence of word vectors, and combine them into a single embedding by averaging.


Additional decisions need to be made concerning which parts of the entity description to include.

if an entity description or word definition is longer than several sentences, using the entire description could cause a 'dilution' of the desired embedding, as not all sentences will be equally pertinent.


Our goal is to build a model which can accurately identify paraphrase rules that both 1) simplify the input phrase and 2) preserve its meaning.

we want to avoid a model which favors "simple" words (e.g.


For each of the proposed rules from all four systems, we collect human judgements on Amazon Mechanical Turk, using the same annotation interface as before.

we ask 7 workers to view each pair and indicate which of the two phrases is simpler, or to indicate that there is no difference.


Noun compounds are often idiomatic or noncompositional.

the meaning of the whole does not come directly from the meaning of the parts.


The fact that the Lite Speech2Sign platform is intended primarily for use by sign language experts who may only have modest skills in computer science has dictated several key design decisions.

1) the formalism used is simple and minimal and 2) no software need be installed on the local machine: all processing (compilation, deployment, testing) is performed on a remote server accessed through the web interface.


We demonstrate the value of collecting semantic parse labels for knowledge base question answering.

(1) unlike previous studies on small-scale datasets, we show that learning from labeled semantic parses significantly improves overall performance, resulting in absolute 5 point gain compared to learning from answers, (2) we show that with an appropriate user interface, one can obtain semantic parses with high accuracy and at a cost comparable or lower than obtaining just answers, and (3) we have created and shared the largest semantic-parse labeled dataset to date in order to advance research in question answering.


In order to evaluate the costs and benefits of gathering semantic parse labels, we created the WEBQUESTIONSSP dataset 1 , which contains semantic parses for the questions from WEBQUESTIONS that are answerable using Freebase.

we provide SPARQL queries for 4,737 questions.


In the case that semantic parses are available, we change the score that we use for evaluating the quality of a query graph.

we assign the query graph score to be zero whenever the query graph is not a subgraph consistent with the semantic parse label and to be the F 1 score described above otherwise.


We evaluate the ability of the model to predict attributes that discriminate the intended referent from the context.

we ask the model to return all discriminative attributes for a pair, independently of whether they are positive for the referent or for the context (given images of a cat and a building, both +is_furry and −made_ of_bricks are discriminative of the cat).


We test moreover a model that is trained with supervision to predict attributes and then deterministically computes the discriminative attributes.

we implemented a neural network with one hidden layer, which takes as input a visual instance, and it is trained to predict its gold attribute vector, casting the problem as logistic regression, thus relying on supervision at the attribute level.


Thus, Equation 1 is equal to: 4 Model We follow Esuli and Sebastiani (2006), using positivity and negativity attributes to define lexicons.

each word takes the form w = (n, p), where n denotes negativity and p denotes positivity (n, p ∈ R).


(2013;, and use a collection of Open IE-style assertions (Banko et al., 2007) as our knowledge base.

we used hand-crafted syntactic rules 5 to extract over 63 million unique subject-relation-object triplets from Google's Syntactic N-grams (Goldberg and Orwant, 2013).


Results show that our method is significantly better than previous combination approaches, including sentence-level constrained methods and a recent phrase-level combination method.

it improves the BLEU (Papineni et al., 2002) score by up to +5.5% on EN-ZH and +2.4% on EN-FR over a phrase-based baseline (Koehn et al., 2003) and decreases the TER (Snover et al., 2006) error by up to -4.3%/-2.2%, respectively.


We tune an adaptation trick by considering predicted class scores as augmented features for target data.

we use the source classifiers as a pivot to transfer knowledge from source to target.


It is trained on the (text-level) "instances" of relatively common names (appearing more than 100 times in the corpus) from the 3 NE label types derived based on the Brown clustering.

to build a training set, we pass through the corpus and each time we come across a common name in a particular document, we build a feature vector corresponding to all the contexts in that document, with the label taken from the clustering.


The general tendency among human judges was to assume reviews are real: overall, 87% of the individual answers judged a review to be real.

only a minority of judges suspected fraud, irrespective of whether they had access to the metainformation or not.


To show that our model can better predict topic proportions of documents containing OOV words than other topic models, we conducted an experiment on a classification task.

we infer topics from the training dataset and predicted topic proportions of held-out documents using collapsed Gibbs sampler.


nl-cLDA is noninteractive in the sense that constraints are inferred from the word embeddings instead of being interactively identified by humans.

we apply k-means to word embeddings to cluster words.


This paper presents the question of whether it is possible to significantly reduce the memory needs for the use and training of word embeddings.

we ask "what is the impact of representing each dimension of a dense representation with significantly fewer bits than the standard 64 bits?"


However, when the dimension is 200, this approach works much better, and outperforms the 4-bit truncation.

binarization works better for skipgram than for CBOW with 200 dimensions.


In this paper, we aim to consider the continuous time information along with the textual information for classifying sequences of temporal textual data.

we consider the problem of rumour stance classification in Twitter, where tweets provide temporal information associated with the textual tweet content.


which might lead to violation of the Hawkes Process mutual-excitation property.

we noticed that in some scenarios occurrences of tweets cause decrease in the intensity value rather than spikes.


More strikingly, there are many cases where bilingual parsers outperform monolingual ones, even in this evaluation on purely monolingual datasets.

there are 12 cases where a bilingual parser obtains statistically significant gains in LAS over the monolingual baseline, and 9 cases with significant gains in UAS.


The surface form of a reference indicates to the hearer how activated its corresponding node is expected to be.

surface form is an instruction for how to retrieve suitable referents, guiding the resolution of coreference.


In this work, we show that the fine-grained categorisation in the Accessibility hierarchy can be leveraged to improve the discriminative power of a strong system, compared to coarser-grained typologies from previous work.

this work contributes valuable empirical support for the psycholinguistic theory.


However, we generate up to three pairs per mention since antecedents are latent at the entity level.

for he in the following, we generate pairs 1, 14and (14,14).


On the link-based MUC and B 3 metrics, our recall gains are larger than our precision gains.

specialisation enables coreference indicators to accrue sufficient weight so as to promote new coreference links, a known problem case for modern systems.


Sentence completion questions, such as those previously featured on the Scholastic Aptitude Test (SAT), were designed to assess this type of verbal reasoning ability.

given a sentence containing 1-2 blanks, the test taker was asked to select the correct answer choice(s) from the provided list of options (College Board, 2014).


It postulates that speakers will strive to keep information density approximately constant.

if one interlocutor decreased their rate of information transmission, the other one would increase it in response.


We explore the applicability of machine translation evaluation (MTE) methods to a very different problem: answer ranking in community Question Answering.

we adopt a pairwise neural network (NN) architecture, which incorporates MTE features, as well as rich syntactic and semantic embeddings, and which efficiently models complex non-linear interactions.


In this paper, we tackle the task from a novel perspective: by using ideas from machine translation evaluation (MTE) to decide on the quality of a comment.

we extend our MTE neural network framework from (Guzmán et al., 2015), showing that it is applicable to the cQA task as well.


We have explored the applicability of machine translation evaluation methods to answer ranking in community Question Answering, a seemingly very different task, where the goal is to rank the comments in a question-answer thread according to their appropriateness to the question, placing all good comments above all bad ones.

we have adopted a pairwise neural network architecture, which incorporates MTE features, as well as rich syntactic and semantic embeddings of the input texts that are non-linearly combined in the hidden layer.


Most relevant to this work is the comparison between Open-IE and SRL.

they suggest that SRL's longer arguments introduce noise which hurts performance for downstream tasks.


Of the factors mentioned above, only the first is modelled in this summariser (called FT14).

we use the frequency of a proposition being retained in memory as the only indicator of its summaryworthiness.


We observe that CL-Eigenwords is competitive with BilBOWA and CL-LSI.

cL-Eigenwords performed very well for the most frequent words (ranks 1-1000) in this particular parameter setting.


1 However, the needs of language learners go beyond mere lists of collocations: the cited studies reveal that language learners often build "miscollocations" (as, e.g., *give a suggestion or *have the curiosity) to express the intended meaning.

they fail to observe, in Kilgarriff's terms, the "collocationality" restrictions of L2, which imply that in language production, one of the elements of a collocation (the base) is freely chosen, while the choice of the other (the collocate) depends on the base (Hausmann, 1989;Cowie, 1994).


In the second experiment, the bases were modeled by training their word vectors over a 2014 dump of the English Wikipedia, while for modeling collocates, again, GoogleVecs has been used.

we assumed that Wikipedia is a standard register corpus and thus better for modeling B, while GoogleVecs is more suitable for modeling C. The figures in Section 3.2 below will give us a hint whether this assumption is correct.


3 At this stage of our work, we considered only collocations that involve single word tokens for both the base and the collocate.

we did not take into account, e.g., phrasal verb collocates such as stand up, give up or calm down.


the full pipeline promotes good collocate candidates to the first positions of the ranked result lists and is also best in terms of accuracy.


However, it should be also stressed that our evaluation was very strict: a retrieved collocate candidate was considered as correct only if it formed a collocation with the base, and if it belonged to the target semantic gloss.

the first condition might be too rigorous, given that, in some cases, there is a margin of doubt whether a combination is a free co-occurrence or a collocation; cf., e.g., huge challenge or reflect [a] concern, which were rejected as collocations in our evaluation.


We suggest here a simple way for doing so, based on the aforementioned notion of argument "recycling".

that the predicate of a claim -what it says on the topic at hand -may be applicable to other topics as well.


Each candidate claim is a different message, and the selection stage attempts to identify those which are coherent and relevant, rather than best realized.

while the classical NLG paradigm is to first select the content and then realize it in a natural language, here our building blocks from the onset are natural language elements, and statistical methods are used to determine which content selectionsimplied by combining them -are valid.


This is a list of n-grams which are presumably related to the topic.

given an n-gram, we assume its appearance in Wikipedia articles follows a hyper-geometric distribution, and estimate the distribution's parameters by counting the n-gram's appearance in a large set of Wikipedia articles.


While at the onset we presented claim synthesis as an alternative to argumentation mining for the purpose of generating arguments, it is interesting how the two augment each other.

we have started looking at whether claim synthesis can generate claims which do not appear in our corpus , and whether matching Evidence to claims (Rinott et.


A growing body of evidence shows that human interpretation and production of natural language are inter-related (Clark, 1996;Pickering and Garrod, 2007;Zeevat, 2011;Zeevat, 2015).

evidence shows that during interpretation, listeners simulate how the utterance is produced; and during language production, speakers simulate how the utterance will be perceived.


The idea is that the quality of a topic can be measured by how far topic words are from some random words.

if human evaluators can consistently separate random words from topic words, these topics are good, otherwise, they are not good.


In this paper, we hypothesize that this measure considers just one important aspect in assessing the quality of statistical topics.

we investigate the topic interpretability by examining the "coherence" of a topic generated by topic modeling algorithms, i.e., how close topic words are within a topic.


The intuition behind choosing entropy is that it will measure the unpredictability in the Turker selections.

if the Turkers are confused about which second word to choose, then their answers will be scattered amongst the remaining five words.


Instead, we want to move beyond privacy in our ethical analysis and look at the wider social impact NLP may have.

we want to explore the impact of NLP on social justice, i.e., equal opportunities for individuals and groups (such as minorities) within society to access resources, get their voice heard, and be represented in society.


To address this problem, in this paper we propose an adversarial multi-task framework, in which the shared and private feature spaces are in-herently disjoint by introducing orthogonality constraints.

we design a generic sharedprivate learning framework to model the text sequence.


To facilitate this, we give some explanation for notations used in this paper.

formally, we refer to D k as a dataset with N k samples for task k. where x k i and y k i denote a sentence and corresponding label for task k. The key factor of multi-task learning is the sharing scheme in latent feature space.


Adversarial networks have recently surfaced and are first used for generative model (Goodfellow et al., 2014).

the goal is to learn a generative distribution p G (x) that matches the real data distribution P data (x) GAN learns a generative network G and discriminative model D, in which G generates samples from the generator distribution p G (x).


We notice that there is a potential drawback of the above model.

the task-invariant features can appear both in shared space and private space.


These datasets are collected based on the raw data 1 provided by .

we extract the sentences and corresponding labels from the unprocessed original data 2 .


AM as Sequence Tagging: We frame AM as the following sequence tagging problem.

each input token has an associated label from Y, where Y consists of all four-tuples (b, t, d, s) where b is a BIO encoding indicating whether the current token is non-argumentative (O) or begins (B) or continues (I) a component; t indicates the type of the component (claim C, premise P, or major claim MC for our data).


Then, given different tasks, each task k 'feeds' from one of the hidden layers in the network.

the hidden states encoded in a specific layer are fed into a multiclass classifier f k .


A side-product of our modeling framework is that the induced intermediate representations can contribute to rationalizing neural predictions (Lei et al., 2016).

they can shed light on the kinds of representations (especially predicates) useful for semantic parsing.


SPADES (Bisk et al., 2016) contains 93,319 questions derived from CLUEWEB09 (Gabrilovich et al., 2013) sentences.

the questions were created by randomly removing an entity, thus producing sentence-denotation pairs (Reddy et al., 2014).


Since creating a manual gold standard for these large datasets is time-consuming, we compared the induced representations against the output of a syntactic parser.

we converted the questions to event-argument structures with EASY-CCG (Lewis and Steedman, 2014), a high coverage and high accuracy CCG parser.


This work provides a theoretical justification for the presence of additive compositionality in word vectors learned using the Skip-Gram model.

it shows that additive compositionality holds in an even stricter sense (small distance rather than small angle) under certain assumptions on the process generating the corpus.


The Skip-Gram model learns vector representations of words based on their patterns of co-occurrence in the training corpus as follows: it assigns to each word c in the vocabulary V , a "context" and a "target" vector, respectively u c and v c , which are to be used in order to predict the words that appear around each occurrence of c within a window of ∆ tokens.

the log probability of any target word w to occur at any position within distance ∆ of a context word c is taken to be proportional to the inner product between u c and v w , i.e., letting Further, Skip-Gram assumes that the conditional probability of each possible set of words in a window around a context word c factorizes as the product of the respective conditional probabilities: (2) (Mikolov et al., 2013a) proposed learning the Skip-Gram parameters on a training corpus by using maximum likelihood estimation under (1) and (2).


This work guarantees that word vectors can be recovered by factorizing the so-called PMI matrix, and that algebraic operations on these word vectors can be used to solve analogies, under certain conditions on the process that generated the training corpus.

the word vectors must be known a priori, before their recovery, and to have been generated by randomly scaling uniformly sampled vectors from the unit sphere 1 .


While our primary motivation has been to provide a better theoretical understanding of word compositionality in the popular Skip-Gram model, our connection with the SDR method illuminates a much more general point about the practical applicability of the Skip-Gram model.

it addresses the question of whether, for a given corpus, fitting a Skip-Gram model will give good embeddings.


, c m has the same meaning as the single word c if for every other word w, Although this is an intuitively satisfying definition, we never expect it to hold exactly; instead, we replace exact equality with the minimization of KL-divergence.

we state that the best candidate for having the same meaning as the set of context words C is the word We refer to any vector that minimizes (4) as a paraphrase of the set of words C. There are two natural concerns with (4).


It has been observed that word vectors trained using nonlinear models like Skip-Gram tend to encode semantic relationships between words as linear relationships between the word vectors (Mikolov et al., 2013b;Pennington et al., 2014;Levy and Goldberg, 2014).

analogies of the form "man:woman::king:?"


Whereas the SDR objective measures the distance between the pmfs given by (normalized versions of) G and e WH T , the Skip-Gram objective measures the distance between the pmfs given by (normalized versions of) the rows of G and e WH T .

sDR emphasizes fitting the entire pmfs, while skip-Gram emphasizes fitting conditional distributions.


While we discuss the role played by event arguments in event coreference in the previous subsection, KBP 2016 addresses event argument detection as a separate shared task.

the KBP 2016 Event Nugget Detection and Coreference task focuses solely on trigger detection and event coreference.


It is worth mentioning that the KBP Event Nugget Detection and Coreference task, which started in 2015, aims to address a major weakness of the ACE 2005 event coreference task.

aCE 2005 adopts a strict notion of event identity, with which two event mentions were annotated as coreferent if and only if "they had the same agent(s), patient(s), time, and location" (Song et al., 2015), and their event attributes (polarity, modality, genericity, and tense) were not incompatible.


Our model, which is a structured conditional random field, operates at the document level.

given a test document, we first extract from it (1) all single-word nouns and verbs and (2) all words and phrases that have appeared at least once as a trigger in the training data.


Features representing the NULL candidate antecedent: Besides m j 's word and m j 's lemma, we employ feature conjunctions given their usefulness in entity coreference (Fernandes et al., 2014).

we create a conjunction between m j 's lemma and the number of sentences preceding m j , as well as a conjunction between m j 's lemma and the number of mentions preceding m j in the document.


The top half of the table shows the results on the English evaluation set.

row 1 shows the performance of the best event coreference system participating in KBP 2016 .


Row 2 shows the performance of the independent models, each of which is trained independently of the other models.

each independent model is trained using only the unary factors associated with it.


For example, unk1 indicate the first unknown word, say "apple", in the current sample, but in another sample the unk1 may indicate the unknown word "orange".

the unknown word labels are indicating position features rather than the exact word; • Insert these unknown marks in the vocabulary.


According to the above thought, we propose the discourse mode identification task.

we make the following contributions: • We build a corpus of narrative essays written by Chinese students in native language.


The translation model computes a distribution over the V possible target words y i+1 by transforming the LSTM output h i via a linear layer with weights W o and bias b o : The conditional input c i at time i is computed via a simple dot-product style attention mechanism (Luong et al., 2015a).

we transform the decoder hidden state h i by a linear layer with weights W d and b d to match the size of the embedding of the previous target word g i and then sum the two representations to yield d i .