Although w is technically not updated when y =ŷ, it is still considered a new vector.

t is incremented for every training instance, so t ← t + 1 is inserted after the line 5. c is updated by adding the partial vector ∂ as follows (to be inserted after the line 7): each dimension in c represents the accumulative penalty (or reward) for a particular feature and a label.


The traditional machine learned modeling approaches trained with small and clean general text, such as news articles, perform poorly when applied to tweets, because tweets are structurally very different from general text.


However, since the expanded gazetteers cover significant proportions of the entities in the training data, the weight of gazetteers features are easily inflated and thus the model tends to rely too much on lexical features extracted from the gazetteers fea-tures to assign a tag rather than the contextual features such as n-gram, a phenomenon called "feature under-training".

we often observe noticeable performance degradation at test time when the entity value does not exist in the training set or the entity dictionary.


It differs in the following respects: (a) it does not artificially restrict the depth of the dependencies it considers and can thus be viewed as an infinite order dependency language model; (b) it not only estimates the probability of a string but is also capable of generating dependency trees; (c) finally, contrary to previous dependencybased language models which encode syntactic in-formation as features, our model takes tree structure into account more directly via representing different types of dependency edges explicitly using LSTMs.

there is no need to manually determine which dependency tree features should be used or how large the feature embeddings should be.


While this seems to conflict with the worse result obtained by the bidirectional-RM, it is important to realize that prediction corresponds to the whole sentence probability.

a badly chosen word can have a negative effect on the score of future words.


The resulting model can therefore employ a training objective that includes not only discourse relation classification, but also word prediction.

it outperforms state-ofthe-art alternatives for two tasks: implicit discourse relation classification in the Penn Discourse Treebank, and dialog act classification in the Switchboard corpus.


However, it could still be questioned whether the comparison was sufficiently strict, given that sets of random words which do not overlap in form have a priori lower chance of being semantically related than sets of words that share a phonestheme.

in the first part of our study (Section 4) we present a stricter validation method for candidate phonesthemes that also includes considerations related to morphological diversity, which were ignored in previous work.


Our aim is to conduct a stricter test than previously done in the literature.

rather than comparing candidate phonesthemic clusters to sets of random words, as done by Otis and Sagi (2008) and Abramova et al.


Unlike the earlier models relying on arbitrary IDs, our model is learning to associate words to actual feature-based visual representations.

once the model is trained on IFC, we can test its generalization capabilities to associate known words with new object instances that belong to the right category.


For example, the tweet "give me paypall or i will tell my mum and ddos u" shows a metaphor rather than a DDoS event.

the event mention extraction task involves a classification task that filters out true events from all tweets that contain event keywords.


The main difference between our model and the baseline is in the definition of f s , the former being a deep neural network and the latter being manual features.

equation 1 can be regarded as a softmax layer in our deep neural model, for which all the features and parameters are trained automatically and consistently.


In addition, sequences in MT are much longer, with averaged length of 36.7 being about 5 times longer than that in grapheme-to-phoneme conversion.

we believe that the history is more likely to contain incorrect predictions and the issue of unbalanced outputs may be more serious.


introduce noise into history predictions in training to balance the mistmatch, while we try to make the history predictions in testing as accurate as those in training by using of two directional models.

theirs focuses on this problem from the view of training instead of both modeling and training as ours, but it is possible and promising to apply their approach to optimize our joint model.


In freestyle conversations experimented in this paper, different people would use very different words to express their opinions.

documents in a same topic could be quite different because of the intra-topic variability problem.


However, all those works for document representation paid little attention to the variability of intra-topic documents.

they could hardly solve the intra-topic variability problem in a direct way.


It is very difficult to model the noise signal in a document since it could come from various sources.

in this paper, we focus on the noise created by the variability among intra-topic documents.


Since the goal of this paper is not dimensionality reduction, the dimensionality of the new document representation is the same as the source document representation.

the size of P is d × d. This paper proposes to learn P by minimizing the following intra-topic variability where w ij is the i-th row and j-th column element of a weighting matrix W created in this work.


One major limitation is that it is not designed to learn from dependencies which lie in the surrounding but not quite immediate neighborhood.

the feature vectors have to be explicitly modeled to include the surrounding contextual information.


When instantiated, they are combined with each legal output action to score the action.

actions are atomic in feature instances.


To make quality vectors, we regard that the probability of the target word y j involves the quality information about whether the target word y j in target sentence is properly translated from source sentence.

by decomposing the softmax function 8 of (4),  the quality vector q y j for the target word y j is computed by where • is an element-wise multiplication.


All of quality information about possible K y target words at position j of target sentence is encoded in t j .

by decoding t j , we are able to get quality vector q y j for the target word y j ∈ R Ky at position j of target sentence.


Especially for relation classification, the processing of the relation arguments might be easier with knowledge of the succeeding words.

in bi-directional RNNs, not only a history vector of word w t is regarded but also a future vector.


Therefore in bi-directional RNNs, not only a history vector of word w t is regarded but also a future vector.

this leads to the following conditioned probability for the history h t at time step t ∈ [1, n]: the network can be split into three parts: a forward pass which processes the original sentence word by word (Equation 1); a backward pass which processes the reversed sentence word by word (Equation 2); and a combination of both (Equation 3).


(2014), our CNN does not have an additional fully connected hidden layer.

we increased the number of convolutional filters to 1200 to keep the number of parameters comparable.


However, affective resources with valence-arousal ratings are still very rare, especially for the Chinese language.

this study builds 1) an affective lexicon called Chinese valence-arousal words (CVAW) containing 1,653 words, and 2) an affective corpus called Chinese valencearousal text (CVAT) containing 2,009 sentences extracted from web texts.


We can then write unary Thus we can regard the bilinear form as a function computing a weighted inner product over some real embedding v y U representing state y and some real embedding representing input factor t. The rank of W gives us the intrinsic dimensionality of the embedding.

if we want to induce shared low-dimensional embeddings across different states it seems reasonable to impose a low rank penalty on W .


Similarly, we can also find the top k predictions in O(kN 2 ).

for this application the scoring function of the bilinear CRF will take the form of: The unary potentials measure the compatibility between an image and a semantic argument, the first binary potential measures the compatibility between a locative and a predicate, and the second binary potential measures the compatibility between a predicate and an actor.


Since most of these task extraction methods are based on relating a user's current query to one of her previous tasks, these methods cannot be directly used in finding and extracting sub-tasks.

while task extraction methods abound, very little has been done to explicitly identify the sub-tasks from within complex search tasks.


The independent model is a generalization of IBM Model 1 (Brown et al., 1993) to the lexicon learning problem, and -like IBM Model 1 -its loglikelihood function is concave (see Appendix A).

the EM algorithm will converge to a global optimum of the data loglikelihood under this model.


Fundamentally, a lexicon constrains the space of logical forms searched by a semantic parser; these constraints improve efficiency and can improve accuracy as long as they do not exclude correct logical forms.

a promising approach to building a semantic parser is to automatically learn a lexicon using one of our models, then (perhaps) manually specify new parsing operations to correct any problems with the learned lexicon.


However, only for few terms we observe nested candidates in the most similar words.

we require methods to generate "back-off" candidates.


(2015) found so effective in a different structured prediction architecture.

the k th -level BiLStM ( Figure 6) reads a sequence of input vectors x k) , and produces a sequence of vectors γ After this deep generalization, we define γ i:j to be the concatenation of all γ this novel deep BiLStM architecture has more connections than a pair of deep LStMs, since α while we may informally regard α (k) i as being a deep summary of the prefix x 0:i , it actually depends indirectly on all of x (except when k = 1).


The vocabulary of the words of a language encode information about what correct sequences of characters in a language look like.

we learn a language model over the character sequences in a vocabulary extracted from a large unlabeled corpus.


It has been shown that taking an ensemble of models which were initialized differently and trained independently leads to improved performance (Hansen and Salamon, 1990;Collobert et al., 2011).

for each model type used in this work, we report results obtained using an ensemble of models.


Previous studies on German corpora reveal that almost half of the corpus types are compounds, whereas individual compounds are very infrequent (Baroni et al., 2002).

an automatic compound analysis is indispensable and represents an essential component in many natural language processing (NLP) tasks such as machine translation (MT) or information retrieval (IR).


Relying exclusively on the highest string similarity for the normalization, would lead to candidate lemmas that result from linguistically unmotivated operations, e.g., the German Hühner 'chickens' would be normalized to the most string-similar lemma Hüne 'giant' (ED=2) and not to the correct but less string-similar lemma Huhn 'chicken' (ED=3).

a linguistic restrictor is indispensable for finding the underlying lemma for a given constituent.


1 The same is not true for the targets, since they are unknown beforehand.

when the lexicon is used to map puns to phonemes the vocabulary size is essentially unlimited.


The Hempelmann cost matrix does a poor job of separating likely targets from the rest of the vocabulary.

many times the true target is pruned before the application of the language model.


2 To fit the model, we maximize the log-likelihood of the training data , with respect to the model parameters θ. Optimization is done with gradient-based methods-requiring the computation of log Z(w) and ∇ log Z(w), which is intractable.

3 we turn to sampling (Rubinstein and Kroese, 2011) and stochastic gradient methods.


Exact decoding, argmax s,u p(s, u | w), is intractable.

we use a sampling approximation: argmax s,u (i) p(s, u (i) | w) where u (1) .


All tested algorithms, except for our joint model, directly segment and parse queries and hence require these queries to be POS-tagged.

we POS-tagged the testset queries with the OpenNLP 10 POS tagger which was adapted to queries using the self-training algorithm of Ganchev et al.


Success of BRAVE-S can be attributed to its ability to incorporate both non-compositional and compositional meaning observed in entire sentence and the individual words respectively.

making it different from other models which use only bag-of-words  or bi-grams .


Reuters and TED) produce unfavorable results.

we understand that the choice of label/class-aligned corpora is crucial.


Additionally we often had to provide a reference point for the queries, as users usually searched their current map cut-out and while all abandoned theme parks in the world is a short list, all restaurants for example would be too many to realistically handle.

we fell back to 3 chosen cities in those cases, Heidelberg, Paris and Edinburgh.


At test time, a critical issue is the fact that monolingual SMT does not ensure that the translations are valid MRL formulae.

a k-best list (sorted from most probable to least) is generated which needs to be traversed until a valid formula is found.


The context-to-output model (coDCLM) applies the additional information directly to the worddecoding phase.

equation 2 becomes: We now extend the DCLM by leveraging the information at the inter-document level, taking advantage of the structure of the repository -a tree in reddit.


Let L * ∼ be a set of word index pairs (v, w) for which l * (v) = l * (w) holds.

we want to maximize: our objective is given by: or, equivalently, by: subject to Q being an orthogonal matrix.


Infrequent words can occur in frequent contexts.

the frequency information in a single word embedding is limited.


While the correlation for 400-dimensional embeddings shown in Table 1 is .661, the correlation of 60-dimensional embeddings is only .568.

even though we show that the information in 400-dimensional embeddings that is relevant for sentiment can be condensed into a single dimension, hundreds of dimensions seem to be needed if we use word2vec to collect sentiment information.


Table 1 shows that adjectives predominantly carry speaker views.

a noun ending in -ness (thus originating from an adjective) may be similarly likely to convey a speaker view.


malaria mosquito are arguments of relation 'mosquito causes malaria').

the aim has been to predict verbs for those compounds that match those abstract relations (e.g.


7 Then, a label propagation algorithm (Talukdar et al., 2008) is run on a word-similarity graph generated from the opinion verbs.

labels from the seeds can be expanded to the remaining opinion verbs.


In this paper we set α 1 = 2 and α 2 = 1 + σ(P e (v p |r i )), where P e (v p |r i ) is the initial bias of the verb-to-relation mapping.

in this paper we define P (θ) as: We can see from this that σ(P e (v p |r i )) is a conjugate prior on P (v p |r i ; θ) with σ as the confidence parameter.


5 Although some verbs are ambiguous (e.g., the verb "play" may express several relations: mu-sicianPlaysMusicalInstrument, athletePlaysSport, actorPlaysMovie, etc), knowing the types of the verbs' arguments can help disambiguate the verbs (e.g., the verb "play" that takes a musicalInstrument type as object is more likely to express the musicianPlaysMusicalInstrument relation).

we incorporate argument type checking in our EM process to ensure that it maps verbs to relations whose argument types match: • In the E-Step, we make sure that unlabeled instances are only labeled with relations that have the same argument types as the instance and that share some verbs with the instance.


Currently, there is no benchmark for slot filling.

it is not possible to directly compare results that were submitted to the Shared Task to new results.


The official SF Shared Task evaluations only assess whole systems (with potential subsequent faults in their pipelines (Pink et al., 2014)).

we expect component wise comparisons to be a valuable addition to the Shared Task: With comparisons of single components, teams would be able to improve their modules more specifically.


The most important challenges for relation classification for slot filling are little or noisy (distant supervised) training data, data from different domains and test sentences which have been extracted with a pipeline of different NLP components.

their quality directly depends on the performance of the whole pipeline.


It connects the representations of the three contexts and, thus, can draw conclusions based on cooccurring patterns across contexts.

the result vector after convolution and pooling is fed into a fully connected hidden layer.


Each group has their own datasets and comparisons across groups are difficult.

we have developed a script that creates a clean dataset based on manually annotated system outputs from previous Shared Task evaluations.


In contrast to other relation classification benchmarks, this dataset is not based on a knowledge base (such as Freebase) and unrelated text (such as web documents) but directly on the SF assessments.

it includes exactly the SF relations and addresses the challenges of the end-to-end task: noisy data, possibly incomplete extractions of sentences and data from different domains.


The proportion of non-news more than doubled from 12.5% to 26.6%.

when using 2012/2013 as the development and 2014 as the test set, we are faced with a domain adaptation problem.


However, as we argued in Section 1, an entity surface does not possess meaning; rather, it serves as an anchor to link pieces of information about it.

we hypothesize that the meaning representation v(e; D, q) of an entity e should be dynamically constructed from its surrounding contexts, and the meanings are "accumulated" through the reader reading the document D. We explain the construction of v(e; D, q) in Section 2.1, and propose a max-pooling process for modeling information accumulation in Section 2.2.


The embedding space generated by existing translation-based embeddings do not represent transitive and symmetric relations precisely, because they ignore the role of entities in triples.

we introduce a role-specific projection which maps an entity to distinct vectors according to its role in a triple.


This method learns a vector representation of a knowledge graph, and the plausibility of a certain knowledge within the graph is measured with algebraic operations in the vector space.

new knowledge can be harvested from the space by finding knowledge instances with high plausibility.


That is, transitive relations and symmetric relations lose their transitivity or symmetricity in the vector space generated by the translation-based models.

the models can not complete only new knowledge with such relations, but also new knowledge with a relation affected by the relations.


About 20% of triples in FB15K have a transitive or a symmetric relation.

the ignorance of logical properties of relations becomes a serious problem in knowledge graph completion.


An entity should be a different vector in the embedding space according to its role.

the solution to preserve the logical properties of relations in the embedding space is to distinguish the role of entities while embedding entities and relations.


This can be implemented by projecting a head entity onto an embedding space by a head projection operator and a tail entity by a tail projection operator.

an identical entity is represented as two distinct vectors.


NTN is a highly expressive embedding model which has a bilinear tensor layer instead of a standard linear neural network layer.

it can process various interactions between entity vectors.


The basic idea of this embedding is that entities and relations are represented as vectors, and relations are treated as operators to translate entities into other positions on an embedding space.

they try to find vector representations of h, t, and r so that the vector of t becomes the sum of the vectors of h and r. TransE (Bordes et al., 2013) is the simplest translation-based graph embedding.


For this, it introduces projection vectors for entities and relations, and then constructs the mapping matrices by multiplying these entity and relation projection vectors.

every relation in TransD has multiple entity-specific spaces.


The entities in TransR are mapped into vectors in different relation space according to a relation.

its score function is defined as where M r ∈ R m×n is a mapping matrix for a relation r which is represented as r ∈ R m .


Thus, its score function is defined as where M r ∈ R m×n is a mapping matrix for a relation r which is represented as r ∈ R m .

in the logical property preserving TransR (lppTransR), the mapping matrix of each relation is split into a head mapping matrix M r h ∈ R m×n and a tail mapping matrix M rt ∈ R m×n .


Their complexity mainly depends on the number of relations.

the increased complexities of the logical property preserving embeddings is not significant, when compared with TransE, TransR, and TransD.


Thus, we generated negative triples for FB15K by following the strategy of (Socher et al., 2013).

the classification accuracies on FB15K can not be compared directly with previous studies, and the accuracies on FB15K in this table are those obtained with our dataset.


However, as noted before, more than 17% triples are such ones in FB15K.


At each time step, the decoder LSTM receives c i in addition to the previously-output word.

the hidden state 1 at time i of the decoder is defined as , which is used to predict word w i : where g is an affine transformation.


However, for our task of aligning text and speech where the speech is represented as a sequence of phones or PLP vectors, the source can easily be several times larger than the target.

we expect that a target word will commonly align to a run of several source elements.


segment (omit) (T) In sum , we've conducted an analysis on the Japanese language , using a grammar different from school grammar, called derivational grammar.

(,) we were able to produce a simpler analysis (than the conventional method).


LEX and POS yield high accuracy as some features are overfitting, e.g., in this dataset, most interpreters used "parsing" for "構文解析" while the translator used "syntactic analysis".

they do not reveal much about the characteristics of Interpretese except for frequent use of "and" and CC, which indicates segmentation.


Computers can load much larger resources into memory and weigh quality of different translations in an instant, thus potentially rendering the speaker's message more accurately.

directly learning from corpus of human interpretation may lead to suboptimal results (Shimizu et al., 2014).


As N increases, the BLEU score grows, peaking at 21.61 and 20.34 BLEU points when N =6.

we achieve a final improvement of 1.55 and 1.58 BLEU points on the development and test data, respectively.


Specif-ically, it is difficult to define discrete, fixed-length features on clusters, which can be of variable size (or shape).

global coreference features tend to be either too coarse or too sparse.


As a result, global coreference features tend to be either too coarse or too sparse.

early attempts at defining cluster-level features simply applied the coarse quantifier predicates all, none, most to the mention-level features defined on the mentions (or pairs of mentions) in a cluster (Culotta et al., 2007;Rahman and Ng, 2011).


In particular, we define where W c and b c are parameters of the embedding.

we will refer to the j'th hidden state of the RNN corresponding to X (m) as h (m) j , and we obtain it according to the following formula we will effectively run an RNN over each (sequence of mentions corresponding to a) cluster X (m) in the document, and thereby generate a hidden state h (m) j corresponding to each step of each cluster in the document.


Except for Clark and Manning (2015), all of the state-of-the-art coreference resolvers explicitly model anaphoricity detection (Martschat and Strube, 2015;Wiseman et al., 2015;Peng et al., 2015).

modeling search space pruning as singleton detection can provide additional information for the state-of-the-art coreference resolution systems.


(2015) that also use some gold annotations for their features, we extract all of our surface features from 'auto_conll' files.


However, apart from features like these, a large number of linguistically motivated features have been proposed which either do not have a significant impact or are only applicable to a specific language or domain.

designing general linguistic features which provide information that is not captured by surface features deserve more attention in order to gain higher recall and better generalization.


we use a perceptron in Cort while an anchored SVM would have performed slightly better.


Additional pruning of potential anaphors in the preprocessing step by the singleton model hurts the recall of the latent model.

we add the output of the singleton model as a new feature for both anaphor and antecedent.


We demonstrate that the full feature is an outer product among these parts.

a parameter tensor scores each feature to produce a prediction.


As is common in multi-class classification tasks, each template generates a different feature for each label y.

a feature φ = w g ∧ w c ∧ u ∧ y is the conjunction of the four parts.


Compared to their work, our usage of CP-decomposition is different in the application to feature learning: (1) We focus on dimensionality reduction of existing, well-verified features, while  generates new features (usually different from ours) by combining some "atom" features.

their work may ignore some useful features; it relies on binary features as supplementary but our model needs not.


These methods treat features as atomic elements and ignore the inner structure of features, so they learn separate embedding for each feature without shared parameters.

they still suffer from large parameter spaces when the feature space is very huge.


In SCWE and CWE, representing these ambiguous characters with the same embeddings may undermine word embed-  dings.

sCWE+M achieves a better performance by applying multiple-prototype character embeddings.


The second task, proposed at SemEval-2013(Dzikovska et al., 2013, asks to assign a label (e.g., correct or irrelevant) to a student response that shows how appropriate it is as an answer to the question.

from a machine learning perspective, the first is a regression task and the second is a classification task.


Firstly, during the phrase-pair extraction process that occurs in the training of a typical PBSMT system, all possible alternative phrase-pairs are extracted that are consistent with a set of alignment points.

the source and target sides of these extracted phrase pairs may over-lap.


These experiments showed an increase of 0.07 BLEU points from stack  size 100 to stack size 200, followed by a sequence of scores that did not vary more than 0.01.

we conclude that the gains we obtained through interlocking the phrases, could not have been obtained by simply increasing the amount of searching performed by the baseline system.


However, different evaluators focused on different aspects of the translations, which increased the subjectivity of their judgments.

evaluations suffered from low inter-and intra-annotator agreements (Turian et al., 2003;Snover et al., 2006).


For each sentence we selected the best and worst translations according to a human evaluation score based on the expected wins (Callison-Burch et al., 2012).

we had 60 references with two corresponding translations each, adding up to a total of 120 evaluation tasks.


Unlike the prior work which additionally requires traditional sparse features to achieve state-of-the-art performance, our extention alone delivers the same level of accuracy, while being substantially faster.

the labeling model can be applied either as a refinement (re-labeling) step on top of existing parsers with negligible cost of computation, or as a part of a decoupled procedure to simplify and speed up the dependency parsing decoding.


→ phrase" and " adverb + verb → phrase".

for most opposing polarity phrases, their sentiment cannot be accurately determined based on the POS and sentiment of the constituents alone.


Our main purpose in studying computational approaches to the detection of ancillary information is its practical application to PI.

given a pair of sentences (in general two texts), we define ATFs as ancillary information if their semantics: (i) only appears in one of the two sentences and (ii) does not change the main meaning of the sentence, i.e., either the sentences are paraphrases or, if they are not, such ATFs are not the reason for their different meaning.


This kind of mistakes can be corrected by ATC.

we created the following ensemble model: given a pair to be classified, we apply our heuristic for ATF extraction.


The second advantage is that POS taggers and parsers can analyze error-corrected candidates more properly than they analyze erroneous sentences, which enables more accurate features to be obtained.

the fact that taggers for N-best corrected results work much better than for learner original sentences is promising.


It is worth noting here that each dimension of embedding vectors obtained by conventional methods has no interpretable meaning.

we basically need to retrain D -dimensional embedding vectors even if we already have a well-trained D-dimensional vector.


To reduce the additional computational cost of the retraining, and to improve the 'usability' of embedding vectors, we propose a framework for incrementally determining embeddings one dimension at a time from 1 to D. As a result, our method always offers the relation that 'any D -right-truncated em-bedding vector is the solution for D -dimensional embeddings of our method'.

in actual use, we only need to construct a relatively higherdimensional embedding vector 'just once', i.e., D = 1000, and then truncate it to an appropriate dimension for the application.


This is mainly caused by inconsistent scale problem.


6 and 7 both become convex optimization problems.

the global optimum solution can be obtained when ∂ e iΨ = 0 for all i and ∂ o jΨ = 0 for all j, respectively.


Moreover, 'GloVe (trunc)' and 'GloVe (retrain)' in D = 1000 are equivalent, as are 'SGNS (trunc)' and 'SGNS (retrain)'.

these results 8 Results for SGNS and GloVe are the average performance of ten runs as suggested in (Suzuki and Nagata, 2015) were omitted from the table.


Treatments of this task range from template-matching to narrative-building, and methods including integer linear programming, factorization, and more appear in the literature.

of the variety of approaches, several datasets have emerged.


Additionally, in some datasets one may find significant repetition of common words, a small set of equation templates used again and again, or problem texts which map to highly degraded grammatical structures.

designing general algorithms that can address different problem types while still being robust to the variations across datasets has remained a challenge.


We formally define the lexical overlap of a dataset D as where is the number of problem pairs in D. Given a dataset D and a target subset size k, MAWPS automatically finds a subset D of reduced lexical overlap by solving the following optimization problem: This is the remote clique problem 1 , which is NP hard.

we resort to a greedy strategy which gives an approximation ratio of 2 (Birnbaum and Goldman, 2007).


The user only mentioned the summary "Deepening Crisis for the Boeing 787" to refer to the actual event.

the single-genre event extractor missed the event trigger "landing" and the event arguments "All Nippon Airways" and "western Japan".


However, in the related tweet, "Battling" is a strong indicator of an "Attack" event and the event extractor for tweets is able to catch it.

we are able to extract the overall event tuples {EventPhrase=[Advance,Battling], EventPatic-ipant=[France,Islamist,Mali]}.


For each component of the triples we extracted from PPDB the following labels: equivalence, forwardEntailment, backwardEntailment, independence or noRelation.

the matching of an SVO triple in the claim to one in the headline is represented by a concatenation of three labels, each corresponding to the relation between the subjects, the verbs and the objects (SVO).


Machine translation (Koehn, 2010) lets us discover translation correspondences in bilingual texts, but a word and its translation often do not cover the exact same semantic space: distinct word senses might translate differently (Gale et al., 1992;Diab and Resnik, 2002, among others); semantic relations and associations do not always translate, an important issue when constructing multilingual ontologies (Fellbaum and Vossen, 2012); and words in parallel text might be translated non-literally due to lexical gaps (Santos, 1990;Bentivogli and Pianta, 2000) or decisions of the translator, as becomes clear when comparing multiple translations of the same source text (Bhagat and Hovy, 2013).

correct word translations found in parallel corpora exhibit a variety of relations including equivalence, hypernymy, and meronymy.


On monolingual tasks, despite reaching better performance, supervised systems do not really learn entailment relations for word pairs, but instead learn when a particular word in the pair is a "prototypical hypernym" (Levy et al., 2015).

1 we limit our investigation to unsupervised models.


Bucks et al., 2000, citing Sabat, 1994, assert that these structured tests break down language into artificial components that fail to represent the psychological and sociological context involved in daily conversations.

such tests may be insensitive to early linguistic decline, when anomalies are already detectable by patients' families (Key-DeLyria, 2013).


Automatic computation of such features are still challenging for automatic systems.

several publications integrated heuristics for computing such features.


Patients labeled normal can quickly show sign of decline and MCIs can recover over time.

for our analysis, we focused more on the capacity of our classifier to detect good descriptions rather than to strictly predict the target class.


No current information extractor is perfectly accurate and different models exhibit different strengths and weaknesses.

many stateof-the-art KBs in academia and industry employ multiple complementary information extractors for KBP.


We observe that the PMI method (for both vehicles and events) and the Dictionary Definition method (for events) produced low MRR scores < 0.10.

we decided not to use these candidate generation methods.


The only exception is that it makes an extra comparison between each translation option and previous PR pairs, which ignores all the phrases that overlap with the source side of a PRP.

a lot of translation options are ignored, which makes the search space much smaller than standard decoding.


From a machine learning perspective, encoding the information above consists in manually defining expressive rules and features based on syntactic and semantic patterns.

methods for automatizing feature engineering are remarkably important also in the light of fast prototyping of commercial applications.


In previous work, we have shown the importance of encoding information about the relatedness between Q and AP into their structural representations.

we employ lexical and question class match, described hereafter.


Hence, we require an additional lookup table layer for the word overlap features LT Wo (•) with parameters W o ∈ R do×2 , where d o ∈ N is a hyper-parameter of the model, which indicates the number of dimensions used for encoding the word overlap features.

we augment word embeddings with additional dimensions that encode the fact that a given word in a pair is overlapping or semantically similar and let the network learn its optimal representation.


Indeed, the table shows no improvement of model combinations over CH.

we trained a simplified version of CH, CH smpl , which employs shallow chunk-based representations without the question focus or question class information, i.e., only using the basic relational information represented by the lexical match REL tags.


In this respect, our framework combines CTKs and CNNs by generating a global kernel.

the time complexity during training is basically given by (i) training CNNs, (ii) extracting their embeddings and (iii) use these embeddings during the CTK training.


We tested our models for answer passage reranking in QA on two benchmarks, WikiQA and TREC13.

they are directly comparable with many systems from previous work.


For instance, the GRU encoder obtains only 63% MRR, 5% worse than the RCNN model's MRR performance.

the LSTM and GRU encoder do not benefit clearly from pre-training, as suggested in Table 2.


While the label clusters are neatly separated, the examples of each individual label do not appear to be clustered by language.

rather than partitioning each (label, language) combination, the model is learning unified label representations that are independent of the language.


Both of these additional sets of parameters are regularized towards keeping the initial direct transfer model.

our strongly governed unsupervised tagger can succeed where an unguided unsupervised tagger would typically fail.


A number of recent studies have explored the use of cosine similarity of word vectors as a measure of semantic relations between words.

for example, if two words have high cosine similarity in German (target), the corresponding words in English (source) should also be similar.


5 Note that their model requires at least one prototype for each POS category.

we select 14 prototypes (the most frequent word from each category) for the baseline, while our method only uses ten translation pairs.


Evaluation Unlike other unsupervised methods, all models in our experiments can identify the label for each POS tag because of knowledge from either the source languages or prototypes.

we directly report the token-level POS accuracy for all experiments.


For example, the anchor words of Spanish have an average of 1.5 possible translations to English, while for Indonesian the average ambiguity is 2.7.

the isometric assumption holds better and the EM algorithm finds a better local optimum for Indo-European languages than for non-Indo-European languages.


On the other hand, if the embedding space has larger dimen-  sion, the space will be more complex and mapping embedding spaces will be more difficult given only ten translation pairs.

we observe a performance drop with either smaller or larger dimensions.


In his Bayesian generative model, each feature of a language has a latent variable which determines whether it is derived from an areal cluster or the tree.

his model can be seen as a mixture model.


Since the WALS database was sparse, we selected languages for which at least 30% of the features were present.

we obtained 64 creoles and 541 non-creoles.


We apply additive operations to these factors in log-space in a way similar to the Sparse Additive Generative model (Eisenstein et al., 2011).

of this extension, every feature j of creole i has its own mixing proportion, where m j,k is a factor specific to feature type j and n i,k is the one specific to creole i.


In the meanwhile, it has been argued that creole genesis only occurred in modern and early-modern, exceptional circumstances and cannot be responsible for most historical changes.

identifying the social conditions under which creoles arise (Tria et al., 2015) is another research direction to be explored.


Their POS tagger accuracy fell by 14% to 65% without using gold language labels and normalization.

language identification and normalization are critical for POS tagging , which in turn is critical further down the pipeline for shallow parsing as evident in Table 5.


We hypothesize that the injection of crosslingual information will guide learning towards inducing more informative sense-specific word representations.

using this information at training time would benefit the model even though crosslingual information is not available to the encoder at test time.


Our intuition is that similar sounds appear in similar contexts, even if they are not present in the same language (e.g., /s Q / in Arabic is adapted to /s/ in Swahili).

if our model effectively captures cross-lingual signals, similar sounds should have smaller distances in the vector space, which can improve the shortest path results.


In a DAE, high-dimensional input data is corrupted according to some noise function, and the model is trained to recover the original data from the corrupted version.

of this process, DAEs learn to represent the data in terms of features that explain its important factors of variation (Vincent et al., 2008).


We also experiment with a variant (+AE) in which the encoded (source) representation must predict its own words as target in addition to those of adjacent sentences.

in FastSent+AE, (1) becomes (2) 8 We searched po, px ∈ {0.1, 0.2, 0.3} and observed best results with po = px = 0.1.


In other words, they simplified both tasks by avoiding the challenging task of identifying the locations of argument components.

their approach cannot be applied in a realistic setting where the input is an unannotated essay.


We noticed many of the missed relationships involve ACs that mention some of the same entities.

a coreference resolver could help us build features that describe whether two ACCs are talking about the same entities.


First, existing approaches to classifying argumentativeness usually focus on specific text domains (e.g., education) and registers (e.g., student essays).

many used features capture not only local linguistic properties of a text segment, but also global document properties (e.g., that a segment is part of the introduction).


The reason is that we apply distant supervision to derive a robust resource from the metadata of debate portals only.

we allow for a rich feature space without requiring to use advanced machine learning methods.


(3) We delete some keywords from the non-argumentative instances that are used by the community to organize a discussion (e.g., "this house" or "this debate").

of applying the defined mapping functions, we obtained a large argumentation mining corpus, called Webis-Debate-16.


If for a given debate there are multiple polls then the index is computed taking the mean of poll scores.

the influence index P of speaker u ∈ U D is: where p i is the poll percentage assign to speaker u in poll i in the reference polls.


We suspect that this might be due to the fact that depending on the location of a debate, certain candidates may bring bigger crowds into the debate's venue, therefore emotion cues can be a deliberate biased way of support.

emotion cues happening within the debate venue may not reflect the emotions induced to the audiences that followed the broadcast of the debate.


(3) The performance of mLSTM with bi-LSTM sentence modeling compared with the model with standard LSTM sentence modeling when d is set to 150 shows that using bi-LSTM to process the original sentences helps  (86.0% vs. 85.7% on the test data), but the difference is small and the complexity of bi-LSTM is much higher than LSTM.

when we increased d to 300 we did not experiment with bi-LSTM sentence modeling.


In CNN models, convolutional operators process word vectors sequentially using small windows.

sentences are essentially treated as a bag of n-grams, and the long dependency information spanning sliding windows is lost.


Despite the fact that CNN is an order-sensitive model, traditional convolution operators extract local features from each possible window of words through filters with predefined sizes.

sentences are effectively processed like a bag of n-grams, and long-distance dependencies can be only captured if we have long enough filters.


Documents in IMDB consist of several sentences and thus very long: the average length is 241 tokens per document and the maximum length is 2526 words (Dai and Le, 2015).

there is no result reported using CNN-based models due to prohibited computation time, and most previous works are unordered models including variations of bag-of-words.


(i) MVCNN requires that input word embeddings have the same dimensionality.

to incorporate a second set of word vectors trained on a corpus (or using a model) of interest, one needs to either find embeddings that happen to have a set number of dimensions or to estimate embeddings from scratch.


We can use multiple filter sizes with different heights, and for each filter size we can have multiple filters.

the model comprises k weight vectors w 1 , w 2 , ...w k , each of which is associated with an instantiation of a specific filter size.


Because our goal is to select the network that makes the most accurate predictions, we take the reward to be identically the negative log-probability from the execution phase, i.e.

the update to the layout-scoring model at each timestep is simply the gradient of the log-probability of the chosen layout, scaled by the accuracy of that layout's predictions.


But sufficient labeled data is critical for the generalization of an NER system, especially for neural networkbased models.

transfer learning for NER is a practically important problem.


Furthermore, it will be beneficial if we process the sequence in not only a forward but also a backward way.

bidirectional LSTM (Bi-LSTM) was employed in many previous works (Chiu and Nichols, 2016;Ma and Hovy, 2016;Lample et al., 2016) to capture bidirectional information in a sequence.


CRF decodes hidden vectors H to a label sequenceŷ = (ŷ 1 ,ŷ 2 , ...,ŷ n ).

our goal is to improve label prediction accuracy on the target domain D t by utilizing the knowledge from the source domain D s : training a transferable model p(y|X) requires both H(X) and p(y|H) to be transferable.


The process is a one-to-one mapping for types in the original dataset and we didn't add or drop any type or sentence in the original dataset.

we can directly compare the results of our proposed model with or without this refinement.


Due to lack of labeled data, the scoring mechanism cannot distinguish between noisy and non-noisy extractors.

an extractor is judged by its ability to extract more positive and less negative extractions.


Then a new loop (lines 03-10) of hop-1, hop-2 and hop-3 extensions and cluster reliability tests starts.

the algorithm consists of k it iterations.


A "cloze1" evaluation would aim to predict {(499, 2199)} as the missing 3 Since that expression is the product of both probability distributions and probability densities, our "cross-entropy" metric is actually the sum of both entropy terms and (potentially negative) differential entropy terms.

a value of 0 has no special significance.


(2016) could be seen as the best alternative for translating into MRLs as it works at the character level on the decoder side and it was evaluated in different settings on different languages.

we consider it as a baseline model in our experiments.


The approach in the second scenario proposes a multi-task learning architecture, by which in one task we learn translations and in the other one morphological annotations.

all network modules -especially the last hidden layer just before the predictors-should provide information which is useful enough to make correct predictions in both channels, i.e.


Speech repairs are indicated under "EDITED" nodes in Switchboard parse trees, which include structure under these nodes that is not of interest for simple text clean-up.

some studies report flattened-edit parseval F1 scores ("flat-F1"), which is parseval computed on trees where the structure under edit nodes has been eliminated so that all leaves are immediate children.


There are probably minor differences in handling of word fragments and scoring edit regions.

this table primarily shows that our framework leads to more benefits from sentence-internal prosodic cues than others have obtained.


Not included in this analysis are sentence boundary errors, which also change the "gold" parse.

prosody may be more useful than results here indicate.


Each layer encodes every second output of the previous layer.

the sequence is downsampled by a factor of 4, decreasing the computation load for the attention mechanism and the decoders.


Intuitively, the interactions with the most vulnerable regions of the OH's reasoning are most critical.

in our complete model, the interaction embeddings are weighted by the vulnerability scores computed in the first component.


To better measure these features, we need much richer information than the dataset provides (discussion text and ∆s).

our proposed architecture is not to replace prior work on argumentation features, but rather to complement it at a higher, architectural level that can poten-tially integrate various features.


We ran a paired sample t-test and confirmed that the average attention weight of sentences addressed by successful comments was significantly greater than that of sentences addressed by unsuccessful comments (p < 0.05).

as expected in the case where the attention works as designed, the model more often picks out the sentences that successful challengers address.


We first approximate the novelty of a challenger's information with the topic similarity between the challenger's sentence and the OH's sentence, and then see if there is a correlation between topic similarity and each dimension of interaction embeddings (details are in Appendix D).

we found only a small but significant correlation (Pearson's r = −0.04) between topic similarity with one of the three dimensions.


Both in theoretical and computational linguistics, discourse and information structure of sentences are thus receiving increased interest: attention has shifted from the analysis of isolated sentences to the question how sentences are structured in discourse and how information is packaged in sentences analyzed in context.

a rich landscape of approaches to discourse and information structure has been developed (Kruijff-Korbayová and Steedman, 2003).


It has been pointed out that evaluating the annotation of a theoretical linguistic notion only intrinsically is problematic because there is no nontheoretical grounding involved (Riezler, 2014).

besides a comparison to the gold standard, we also evaluated the resulting annotation in a larger computational task, the automatic meaning assessment of short answers to reading comprehension questions.


Given a variable-length words sequence X = (x 1 , x 2 , ..., x L ) in a paragraph, the word-level Bi-LSTM will process the input sequence by using two separate LSTMs, one process the word sequence from the left to right while the other follows the reversed direction.

at each word position t, we obtain two hidden states − → h t , ← − h t .


We believe that this is due to the fact that different models learn different sentence structures and, hence, combining predictions at the probability level results in incoherent utterances.

instead of combining the models at the log-probability level, we accumulate the top 10 predicted utterances from each model type using beam search and allow the reranker (see Section 4.4) to rank all candidate utterances taking the proportion of slots they successfully realized into consideration.


In this dataset, we observed the slot order to be fixed across all the MRs, both in the training and the test set.

for the majority of the time, the model was training on MRs with slot orders it would never encounter in the test set, which ultimately led to a decreased performance in prediction on the test set.


The quality of the training data inherently imposes an upper bound on the quality of the predictions of our model.

in order to bring our model to produce more sophisticated utterances, we experimented with filtering the training data to contain only the most natural sounding and structurally complex utterances for each MR. For instance, we prefer having an elegant, singlesentence utterance with an apposition as the reference for an MR, rather than an utterance composed of three simple sentences, two of which begin with "it" (see the examples in Table 5).


In this model, we assume that the syllables of the generated words and the notes in the input melody have a one-to-one correspondence.

the position of the incoming note/rest for a word position t (referred to as a target note for t) is uniquely determined by the syllable counts of the previously generated words.


In Section 3, we indicated that the positions of rests and their durations are important factors for modeling boundaries of lyrics.

we collect a sequence of notes and rests around the current word position (i.e., time step t) and encode their information into context melody vector n t (see the bottom of Figure 5).


Two of the three writers had experience with music composition, but none of them had experience with writing lyrics.

9 we obtained 50 (workers) × 4 (melodies) × 5 (lyrics) samples in total.


Importantly, most automatic measures are based on local n-gram patterns, providing only a limited and myopic perspective of overall text quality.

while models trained to directly optimize these measures can yield improvements on the same measures, they may not lead to better quality in terms of overall coherence or discourse structure.


When relative order rewards are computed by the teacher, the correct sentence reward is indexed for each token.

when training with a relative order teacher, words only receive rewards for the sentences they belong to: where |S| is the number of sentences in the generated recipe, and 1 is an indicator variable identifying word y t belonging to sentence s j .


The model quickly learns to generate simple sequences that exploit the teacher for high rewards despite being incoherent recipes (e.g., Figure 4).

it is possible that generated sequences are no longer readable (Pasunuru and Bansal, 2017;Paulus et al., 2018).


The latent assumption of the word generator is that each word is independent and the score is irrelevant to each other.

the scores of a word and its synonyms may be of great difference, which means the word generator learns the word itself rather than the relationship between words.


3 The Simple English Wikipedia is built for "the children and adults who are learning the English language", and the articles are composed with "easy words and short sentences".

simple English Wikipedia is a natural public simplified text corpus.


creases above 2, however, we see a significant decrease in Top 1 accuracy, and a slower decrease in Top 5 accuracy.

in our model, we chose to use a context window of 1.


If an entity has multiple entity types we pick the entity type that is mentioned the most in the first sentence of each Wikipedia article.

the textual contexts will opt for entity types that is more natural to appear in free text and therefore questions.


Since AES models are designed to learn by analyzing human-graded essays, AES models could inherit rating biases present in the scores from human raters, and this may result in systematic errors.

our objective in this paper is to examine the extent to which rater bias affects the effectiveness of stateof-the-art AES models.


If the norm is unknown to us, then bias is hard to identify.

our approach for debiasing the training set starts by finding the norm (in terms of the subjectivity within rater comments) for each score value.


The variable typing classifier and the type disambiguation algorithm determine the types of the variables x and y as "vector".

the variable nodes in figure 1(a) will be substituted with their type, producing the SLT in figure 1(b).


Moreover, DKRL fundamentally uses TransE (Bordes et al., 2013) method for encoding structural information.

we can retrieve relevant entities & relation and obtain the complete the fact using t = h + r. This reduces the complexity of fact retrieval as the number of entities/relations is much less compared to the number of facts, thus making the retrieval process faster.


The world knowledge features are retrieved using the data x, using a separate model where, x w = F (x, Θ (2) ).

our modified objective function can be expressed as where, Θ = {Θ (1) , Θ (2) }.


Similar procedure of clustering followed by the encoding of the cluster entities is done for relations as well.

both the entity and relation space were reduced to contain fewer elements, one each for each cluster.


Despite the success of training multi-lingual NMT systems; there are a couple of challenges to leverage them for zero-resource languages: Lexical-level Sharing Conventionally, a multilingual NMT model has a vocabulary that represents the union of the vocabularies of all source languages.

the multi-lingual words do not practically share the same embedding space since each word has its own representation.


It is worth noting that shaded parts in Figure2 are trainable during NMT training process.

each source word e x is represented as a mixture of universal tokens M of E U .


We concatenate the vocabularies of all source languages in the multilingual setting where special a "language marker " have been appended to each word so that there will be no embedding sharing on the surface form.

we avoid sharing the representation of words that have similar surface forms though with different meaning in various languages.


Unlike user ratings on products, our input lacks explicit feedback from users about negative preferences and nonresponse.

we follow one-class Collaborative Filtering (Hu et al., 2008;Pan et al., 2008), which weights positive instances higher during training and is thus suited to our data.


The predicted score p u,c thereby reflects the tendency for a user u to be involved in conversation c. As pointed out by McAuley and Leskovec (2013), these latent vectors often encode hidden factors that are hard to interpret under a CF framework.

in Section 3.2, we present a novel probabilistic model which can extract interpretable topics and discourse modes as word distributions.


We aim to separate discourse, topic, and background information for conversations.

for each word w c,m,n of message m, a ternary switcher x c,m,n ∈ {DISC, TOPIC, BACK} controls word w c,m,n to fall into one of the three types: discourse, topic, and background.


In addition, the dependency parser was trained on linguistics data to produce complete dependency trees, some parts of which may be redundant and hence confuse the scene graph generation process.

our model abandons the two-stage pipeline, and uses a single, customized dependency parser instead.


When predicting the transition action, the feature function φ(c) that summarizes the current configuration c = (σ, β, T ) is simply the concatenated BiLSTM vectors of the top three elements in the stack and the first element in the buffer: MLP Scoring Function The score of transition action y under the current configuration c is determined by a multi-layer perceptron with one hidden layer: where Hinge Loss Function The training objective is to raise the scores of correct transitions above scores of incorrect ones.

at each step, we use a hinge loss defined as: where Y is the set of possible transitions and Y + is the set of correct transitions at the current step.


(2015), we used F-score similarity instead of the likelihood of the maximum a posteriori CRF solution when ranking the images based on the region descriptions.

the numbers we report in Table 3 are not directly comparable with those reported in Schuster et al.


In these cases, machines make mistakes unless they consider all information together.

the alternative decoys suggested our procedures are better designed to gauge how well a learning algorithm can understand all information equally well.


This disparity exists in the test samples too.

the following rule, computing each answer's likelihood of being correct, if C is never seen in training, # times C as target # times C as target+(# times C as decoys)/K , otherwise, should perform well.


Diachronic distributional models are distributional models where the vector for a word changes over time.

we can calculate the cosine similarity between the vectors for a word at two different time points to measure how much that word has changed over time and we can perform a nearest neighbor analysis to understand in what direc-tion a word is changing.


Large scale change in the meaning of a word is the result of change happening one person at a time.

semantic change must be a gradual process.


Thus, all of these models reflect human judgments comparable to synchronic models.

the predictions of the models correlate with human judgments.


Our model is differentiable with respect to time.

we can get the derivative of use W (w, t) with respect to t to model how word w is changing usage at time t. We l2-normalize use W (w, t) beforehand to reduce frequency effects.


We see that we have a fairly even distribution of points until the 1940s, a large burst of points in the 1950s-1960s, and two points in the 1980s.

there are many time periods before the 1940s (which may be caused by noisiness of the data in the first half of the century), a big transition between time periods in the 1950s-1960s, and a transition between time periods in the 1980s.


Only 16% of the 100 nodes in h 1 are zero for time points between 1900 and 2009.

a vast majority of nodes do not correspond to transitions between time periods.


Our method learns embedding from raw text, where a hypernym embedding should be larger than the embedding of its hyponym in every dimension.

dIVE can be viewed as an unsupervised and reversed form of order embedding.


We target all verbs, not only prototypical verbs of possession (e.g., have, get).

our approach extracts possessions intuitive to humans when there is no specific possession cue (e.g., we extract a control possession from The [computer] y at work was slow, [I] x didn't get anything done).


As a result, mapping by a 3D (n × n × k) tensor is replaced with an array of k "triple inner products" of two input vectors and a weight vector.

we obtain two new NTN models where the number of parameters is reduced from O(n 2 k) to O(nk).


To alleviate this problem, some models such as NTNs (Socher et al., 2013a) have explored 3D tensors to yield more expressive mapping: The output of this mapping is an array of k bilinear products in the form of x T 1 W [i] x 2 .

this is also a transformation from R 2n to R k .


NTN-Diag also has fewer parameters than the smallest (m = 1) NTN-SMD.

we conclude that NTN-Diag is a better alternative of NTN than NTN-SMD is, in terms of both accuracy and computational cost.


The above baseline Multi achieves state-of-theart performance for multi-domain sentiment analysis (Liu et al., 2016), yet the domain indicator d i is used solely to select softmax parameters.

domain knowledge is hidden and under-utilized.


However, these multilabel criteria cannot meet our requirement exactly as none of them considers the ranking among emotions which are considered relevant.

by incorporating PRO loss (Xu et al., 2013), the loss function for the instance x i is defined as follows: (1) where e t refers to the emotion belonging to relevant emotion set R i or the threshold Θ of instance x i while e s refers to the emotion which is less relevant than e t denoted as ≺.


Therefore, by incorporating PRO loss (Xu et al., 2013), the loss function for the instance x i is defined as follows: (1) where e t refers to the emotion belonging to relevant emotion set R i or the threshold Θ of instance x i while e s refers to the emotion which is less relevant than e t denoted as ≺.

(e t , e s ) represents four types of emotion pairs: i.e., (relevant, relevant), (relevant, irrelevant), (relevant, threshold), and (threshold, irrelevant).


The set sizes of the four different types of emotion pairs mentioned above are Note that l t,s is non-convex and difficult to optimize.

a large margin surrogate convex loss (Vapnik and Vapnik, 1998) implemented in hinge form is used instead as follows: where (u) + = max{0, u}.


5:  Suppose we have sufficient training examples such that for each label pair (e t , e s ), there exists x i ∈ Gsatisfying e t ∈ R i ∪ {Θ}, e s ∈≺ (e t ).

the objective in Eq.6 becomes equivalent to max w j min 1≤s<t≤T +1 1 ||wt−ws|| and can be rewritten as min Moreover, to overcome the complexity brought in by the max operator, the objective of the optimization problem can be re-written by approximating the max operator with the sum operator.


For example, based on the gender of the subject Hindi has two forms for word 'goes': 'जाता है | jAtA hai' (male) or 'जाती है | jAtI hai' (female).

if representation of one word (जाता है | jAtA hai) is missing in Hindi embedding we can still find its representation in English through its translation i.e.


The inputs to the first LSTM are not only token embeddings but three additional features: embedding of the predicate, embedding of the context of the predicate and an indicator feature (1 if the current token is in the predicate context, 0 otherwise).

every sentence is processed as many times as there are predicates in it.


If the shared LSTMs are taskinvariant, the discriminator should perform badly.

we update the shared parameters to maximize the discriminator's cross-entropy loss.


However, in the 10-fold CV setting, the size of the tests sets is 3 times smaller than the dev set size ( Table 2, row 3), and, consequently, results in high-variance estimates on the test sets.

we additionally evaluate our models with 4-fold CV.


Reimers and Gurevych (2017) show that MTL is especially sensitive to the selection of HPs.

a firm and solid comparison of the different MTL models requires thorough HP optimization, to properly control the number of parameters and regularization of the models.


A distinguishing property of the hard cases is the distance of the role from the opinion.

future work should advance the model' s ability to capture long-range dependencies.


We show that the best MTL model for ORL is the model which uses shared layers only.

it seems reasonable to consider only a small and uniform SRL label set {A0, A1}.


While ontologies like Open Multilingual WordNet (OMW) (Bond and Foster, 2013) and BabelNet (Navigli and Ponzetto, 2012) contain cross-lingual links, these resources are semiautomatically generated and hence contain noisy edges.

to get reliable and high-quality test beds, we collect evaluation datasets using Crowd-Flower 4 .


The selftraining process may erroneously generate some bad or incorrect questions which can sidetrack the learning process.

we implement a question selection oracle which determines which questions to add among the potentially very large set of questions generated by the QG model in each iteration.


Time Complexity: GO and CiO require updating the model, M 2 and ECiO require performing inference on candiate questions, and CiO -ECiO requires retraining as well as inference.

m 2 and ECiO are computationally most efficient.


However, the drop happens at a much slower rate for our selftrained model.

we can conclude that our approach requires less labeled data as compared to the baselines.


Only the meaning-based framework proposed by  can handle general MWPs via understanding and reasoning.

it is possible to explain how the answer is obtained in a human comprehensible way .


But before we do that, we need to know how difficult the task of solving the given MWPs is.

we first propose a way to measure how easy that a system solves the problem by simply guessing.


Table 2 lists the accuracies of different systems in solving the MWPs 14 Since the dataset is not large enough for splitting a development set, we choose those hyper parameters based on the test set in coarse grain.

the DNN performance we show here might be a bit optimistic.


Recall that OSS is created by selecting some MWPs which both Illinois Math Solver (Roy and Roth, 2016) and our system 16 can correctly solve.

both systems have 100% accuracy in solving the OSS dataset.


The much higher accuracy of our system on the noisy dataset shows that our meaning-based approach understands the meaning of each quantity more.

it is less confused 17 with the irrelevant quantities.


NEWSROOM summaries are written by humans, for common readers, and with the explicit purpose of summarization.

nEWSROOM is a nearly two decade-long snapshot representing how singledocument summarization is used in practice across a variety of sources, writers, and topics.


In contrast, when a predicate directly embedded by forget is untensed, as in (4b) and (5b), our inference is dependent on whether forget is negated.

any model that correctly predicts factuality will need to not only be able to represent the effect of individual words in the outside context on factuality inferences, it will furthermore need to represent their interaction.


An even number (or zero) will yield a factuality inference and an odd number will yield a nonfactuality inference.

as for outside context, any model that correctly predicts factuality will need to integrate interactions between words in the inside context.


It must also be mentioned that the triple "(Abugida, language /language writing system/ languages, Khmer language)" is included in the training data.

we can infer the first triple in Table 4 based on the above triple due to the general logic that "language/human language/writing system" and "/language/language writing system/languages" are a pair of inverse relations.


Therefore, we can infer the first triple in Table 4 based on the above triple due to the general logic that "language/human language/writing system" and "/language/language writing system/languages" are a pair of inverse relations.

we believe it is important to incorporate the logic rules into knowledge embeddings, especially for the entities and relations with limited occurrences.


We call a formula constructed by conjunctions and existential quantifiers a basic formula in event semantics.

a set of basic formulas ϕ in event semantics is defined as: The formula in (2) is an instance of a basic formula, which captures the predicate-argument structure of a sentence.


Recently, an unprecedented amount of false information has been flooding the Internet with aims ranging from affecting individual people's beliefs and decisions (Mihaylov et al., 2015;Mihaylov and Nakov, 2016) to influencing major events such as political elections (Vosoughi et al., 2018).

manual fact checking has emerged with the promise to support accurate and unbiased analysis of rumors spreading in social medias, as well as of claims made by public figures or news media.


The rationale behind this metric is that the binary related/unrelated classification task is expected to be much easier, while also being arguably less relevant to fake news detection than the stance detection task, which aims to further classify relevant instances as agree, disagree, or discuss.

the former task is given less weight and the latter task is given more weight through the weighted accuracy metric.


p(y t+1:T |y t , y 1:t−1 , x) is uniform.

at any given time, there is no information from the future when incorporating the global structure.


In a cross-domain evaluation setting, the test data is drawn from a different distribution as the training data.

less expressive models may be preferred as they may learn more abstract representations that will generalize better to outof-domain data.


Such methods are a way to inject domain-specific knowledge to tailor the learned word representations to a given task.

we can exploit the synonymy/antonymy constraints to learn semantic word representations that are better candidates for ontology matching.


We have shown that with same width of recurrent layer, our architecture with group number K achieves a compact model, which has K times less number of parameters than the standard recurrent network.

with same number of parameters, group recurrent networks can provide more possibility to try more complex model without any additional computation and parameter overhead.


At last, we note that our architecture focuses on improving the efficiency of recurrent layers.

the whole parameter and computational cost reduction depend on the ratio of recurrent layer in the entire network.


Our model is based on RNNSearch model (Bahdanau et al., 2014), but replacing the standard LSTM layer with our group LSTM layer.

we name our model as Group RNNSearch model.


Besides, we compare with several other widely adopted methods, our models also show strong performance.

we can keep the good performance even though we reduce the parameters of the recurrent layers by nearly 50%, which greatly proves the effectiveness of our method.


The best accuracy we achieve on labeling a claim accompanied by the correct evidence is 31.87%, while if we ignore the evidence we achieve 50.91%.

we believe that FEVER is a challenging testbed that will help stimulate progress on claim verification against textual sources.


While this information is usually available in justifications provided by the journalists, they are not in a machine-readable form.

also considering the small number of claims, the task defined by the dataset proposed 1 http://fever.ai 2 https://github.com/awslabs/fever 3 https://github.com/sheffieldnlp/ fever-baselines Claim: The Rodney King riots took place in the most populous county in the USA.


However, textual relations are different from words in that they are sequences composed of words and typed dependency relations.

we use recurrent neural networks (RNNs) for embedding, which respect the compositionality of textual relations and can learn the shared sub-structures of different textual relations (Toutanova et al., 2015).


The objective is to maximize: It also corresponds to local statistics based embedding, i.e., when the embedding model is trained on individual occurrences of relational facts with distant supervision.

we call it Local Relation Embedding (LoRE).


Our results show that textual relation embedding trained on global co-occurrence statistics captures useful relational information that is often complementary to existing methods.

it can greatly improve existing relation extraction models.


First, event knowledge is crucial for implicit argument detection.

we build on models for narrative event prediction (Granroth-Wilding and Clark, 2016;Pichotta and Mooney, 2016a), using them to judge how coherent the narrative would be when we fill in a particular entity as the missing (implicit) argument.


we use gold coreference annotation for creating evaluation data, but automatically generated coreference chains for creating training data.


about these events and their "typical" order.

existing systems almost always attempt to solve the situations shown in Example 1, even when they are actually presented with input as in Example 2.


In practice, e.g., in the TBDense dataset , roughly 30%-40% of the P-Before pairs are T-After.

it is important to be able to capture their temporal order rather than simply taking their physical order if one wants to understand the temporal implication of verbs.


However, the time complexity of their model is O(n 3 ), where n is the number of tokens in the sentence, making inference slow.

we do not adopt their parse tree-based representation of nested entities and propose instead a linear time directed hypergraph-based model similar to that of Lu and Roth (2015).


As described in Section 3.2, we can assign probabilities to the different types of edges in the hypergraph and at the time of decoding we choose for each token the (normal) edge(s) with maximum probability and the hyperarcs with probability above a predefined threshold.

we can extract edges at the time of decoding.


For hyperarcs, we distribute the probability mass uniformly over all the gold labels in the gold label set (e k g = 1 |G| for all k).

for the example described earlier in Section 4.3, both the labels "L PER" and "I PER" receive a probability of 0.5 in the gold label distribution e k t , conditioned on the label "I PER" from the previous time step.


However, gold labels are not available at test time.

we use the predicted labels from the previous time step as an input to the current time step to find the most likely label set.


We follow the same dataset split as Finkel and Manning (2009); Lu and Roth (2015); Muis and Lu (2017).

the first 90% of the sentences were used in training and the remaining 10% were used for evaluation.


Only if the previous sentence in the corpus is accessible -"And if you ride inside that tank, it is like riding in the bowels of a dragon" -can we understand that "they" in S2 refers to the tank and hence is a VEH.

our model can be improved by providing additional context for each sentence rather than making predictions on each sentence in the corpus independently.


Then y is exactly distributed as p θ (y | x).

for each given x, y :t−1 , observe that ∝ p θ (y : we can easily construct the needed distribution (7) by normalizing (12) over all possible values of y t .


In that algorithm (which could be used to find the Viterbi sequence), g denotes the score-so-far of a partial solution y :t , and h denotes the optimal score-togo.

g + h would be the score of the best sequence with prefix y :t .


We consider those in the remainder of this section.

a hidden Markov model (HMM) specifies a normalized joint distribution p θ (x, y) = exp G T over state sequence y and observation sequence x, 3 the posterior p θ (y | x) is proportional to exp G T , as required by equation (1).


Specifically, we take z(x) = log y∈Yp (y | x) ≤ log Z, where Y is the full set of distinct particles y that we ever drew for input x, including samples from the beam search models, while constructing the experimental results graph.

16 the offset KL divergence is a "best effort" lower bound on the true exclusive KL divergence KL(p||p).


Push computations can be composed recursively: combining two consecutive push computations followed by a reduce transition yields a new push operation.

the derivation of a sentence under the transition system can be seen as a composition of push computations.


The marginal probability is independent of arc directionality, as it does not influence future decisions.

unsupervised training based on this model cannot learn to predict arc directions.


In this case, the shift decision decomposes into shift and right-arc transitions, where shift is implicitly deciding that the shifted word will be reduced through a left-arc.

the only real difference between the arc-hybrid and arceager transition systems under dynamic programming is the information conditioned on when arc directionality is predicted.


2010which opted out of splitting contractions and possessives.


For example, besides being a discourse marker, RT can abbreviate the verb retweet; emoticons and hashtags may be used as content words within a sentence; and at-mentions can be normal vocative proper nouns: see Figure 2.

the cri-teria for annotating a token as non-syntactic must be context-dependent.


Moreover, if inference is performed on GPU, the squared term (m × n ≈ m 2 ) is amortised due to parallelism.

while training our model is somewhat slower than monolingual models of word representation, which typically run in O(m), it is not at all impracticably slower.


In contrast, the negative association is hard to define (Levy and Goldberg, 2014b).

it is reasonable to replace the negative entries in the PMI matrix by 0, such that the negative association is treated as "uninformative".


With some further derivations, we can show that the algorithm only requires O(nnz(A) + nk) time to update each column, 5 so the overall complexity is O(nnz(A)k + nk 2 ) time per epoch, which is only proportional to number of nonzero terms in A.

with the same time complexity as GloVe, we can utilize the information from all the zero entries in A instead of only sub-sampling a small set of zero entries.


For the analogy task, only the answer that exactly matches the annotated answer is counted as correct.

the analogy task is more difficult than the similarity task because the evalu-ation metric is stricter and it requires algorithms to differentiate words with similar meaning and find the right answer.


However, we believe that studies of the effect of language on persuasion should take into account other, extra-linguistic, factors that can affect opinion change: in particular, we propose an experimental framework for studying the effect of language on persuasion that aims to control for the prior beliefs of the reader as denoted through their self-identified political and religious ideologies.

we study a more fine-grained prediction task: for an individual voter, predict which side/debater/argument the voter will declare as the winner.


Abusive adjectives often modify an abusive noun as in brainless idiot, smarmy liar or gormless twat.

we mined Twitter for adjectives modifying mentions of our extracted nouns (PAT noun ).


The difference between these datasets is the source from which they originate.

different topics are represented in the different datasets.


The highlighted words are all more likely to occur in the christianity newsgroup, but on their own they are not intuitive to lay people.

workers guessed incorrectly that the predicted label was atheism.


Similar to RNN-RBM (Boulanger-Lewandowski et al., 2012), we constrain RNN hidden units (u (t) ) to convey temporal information, while RSM hidden units (h (t) ) to model conditional distributions.

parameters (b v (t) , b h (t) ) are time-dependent on the sequence history at time t (via a series of conditional RSMs) denoted by Θ (t) ≡ { V (τ ) , u (τ ) |τ < t}, that captures temporal dependencies.


While there are substantial differences between variational approaches and CD, and thus in the manner the likelihood for different models is estimated -both approximations work well for the respective family of models in terms of approximating the true likelihood.

perplexities computed based on these approximated likelihoods are indeed comparable.


Crosslingual Gap (GAP) A low CNPMI score could indicate a topic pair where each language has a monolingually coherent topic but that are not about the same theme (Topic 6 in Figure 1).

we add two features to capture this information using the Bible: mismatch coefficients (MC) and internal comparison coefficients (ICC): where α is a smoothing factor (α = 0.001 in our experiments).


We perform the same process given (X = w 2 , Y = w 1 ) with the exception of swapping the X and Y in the dependency paths to be appended.

we add 2k dependency paths to the set of dependency paths for each word pair.


This representation captures the information of co-occurrence dependency paths of (w , w 2 ) in a generalized fashion.

,h (w 1 ,w 2 ) is used to construct the pseudo-path representation v p−paths(w 1 ,w 2 ) .


Our model bears not only word pair representations but also dependency path representations as context vec-tors.

we intend to apply these representations to various tasks, which path representations contribute to.


Concurrently, LEAR (re-)arranges vector norms so that norm values in the Euclidean space reflect the hierarchical organisation of concepts according to the given LE constraints: put simply, higher-level concepts are assigned larger norms.

lEAR simultaneously captures the hierarchy of concepts (through vector norms) and their similarity (through their cosine distance).


Whereas the similarity can be inferred from the standard cosine distance, the LEAR optimisation embeds lexical entailment as a combination of the symmetric ATTRACT term and the newly defined asymmetric LE j cost function.

the metric used to determine whether two words stand in the LE relation must combine the two cost terms as well.


The goal of AMR is to abstract away from the syntactic realization of the original sentences while maintaining its underlying meaning.

different phrasings of one sentence are expected to provide identical AMR representations.


In this case, UDR would pick the relation (what, hope, pobj).

when there is a co-anchor to a head tree, we add all arcs that involve the head tree to the co-anchor tree.


In the TAG grammar, both "may" and "have" adjoin into "known."

when the head of a modal has another child with adjunction, we add an arc from the child to the modal.


to be a softmax distribution given where ✓ are the parameters of the encoder f .

y is a valid distribution over the supporting set's labels .


Additionally, since the observed entries of Y are generated based on high and low enough performance, it is safe to assume that most observed entries are correct and only a few may be incorrect.

we introduce a sparse matrix E to capture the observed incorrect entries in Y.


One main reason is that deep networks could easily fit any task partitions if we optimize on training loss only, making the learned metrics not generalize, as discussed in Section 6.

this work adopts a pipeline training approach and employing validation sets for task clustering.


The results are reported in the multi-task learning setting instead of the fewshot learning setting focused in this paper.

we put the results to a non-archive version of this paper 6 for further reference.


This makes possible to discover different clusters and meta-words if we change the word representation.

the proposed strategy is highly adaptable to other domains, where the specialization would be achieved by changing the word representation for the problem.


The proposed framework is based on the idea of clustering words using the semantic "distance" in the word embedding space.

the first step of the algorithm consists of clustering the word embedding vectors v i and finding the cluster centers to create the proposed meta-words.


This data set was released in the context of the sexual predator identification task at PAN-CLEF'12 and comprises a large number of chat conversations that include real sexual predators.

the task approached is that of identifying those conversations that potentially include a sexual predator, as in (Villatoro-Tello et al., 2012;Escalante et al., 2013Escalante et al., , 2016.


TCOR (Lavelli et al., 2004) In Term Co-occurrence Representation (TCOR) the semantics of a word is captured by observing its co-occurrences with other words across documents in the corpus.

each word vi is associated to a vector t i = ti,1, .


Without loss of generality, we normalized so that P N i=1d i = 1 and 8i :d i 0.

the objectives of C and D that we are minimizing are: For the feature extractors, the training of domain feature extractors is straightforward, as their sole objective is to help C perform better within their own domain.


The proof of both theorems can be found in the Appendix.

by the non-negativity and joint convexity of the f-divergence (Csiszar and Korner, 1982), we have: Fs is N log N when using NLL loss, and 0 for the L2 loss.


The optimum value above is achieved if and only if P 1 = P 2 = • • • = P N = P for either loss.

the loss of F s can be interpreted as simultaneously minimizing the classification loss  J C as well as the divergence among feature distributions of all domains.


However, it still does not outperform CNN and is much slower.

we conclude that, for text classification tasks, CNN is both effective and efficient in extracting local and higher-level features for making a single categorization.


There are two equivalent definitions of the generalized Jensen-Shannon divergence: the original definition based on Shannon entropy (Lin, 1991), and a reshaped one expressed as the average Kullback-Leibler divergence of each P i to the centroid P (Aslam and Pavlu, 2007).

we adopt the latter one here: Now substituting D ⇤ into J D Fs : by the non-negativity of JSD (Lin, 1991), we have the following corollary: Corollary 1.


For each document example in the clickbait dataset, naturally, we have two views, the headline and the paragraph.

we construct the two classifiers in co-training based on these two views.


Paragraph Classifier The paragraphs usually have much longer sequences than the headlines.

we utilize the CNN-non-static structure in Kim (2014) as the paragraph classifier to capture the paragraph information.


We find that the vectors assigned to Cluster 1 generate words which are nouns, pronouns, indefinite and definite articles, and adjectives, while the vectors assigned to Cluster 0 generate verbs, prepositions, conjunctions, and adverbs.

cluster 1 contains the noun-related words, cluster 0 the verb-like words   (verbs, prepositions and conjunctions are all potentially followed by noun-phrase complements, for example).


As shown in Figure 1, when designing the model, the hidden size in each layer is much smaller than the hidden size of the corresponding layer in the residual-connected model.

while each encoder layer perceives information from its previous layers, each decoder layer z l+1 has two information sources: previous layers z i , i ≤ l, and attention values a i , i ≤ l. in order to allow dense information flow, we redefine the generation of (l+1)-th layer as a nonlinear function over all its previous decoder layers and previous attentions.


In DenseNMT, each layer directly provides information to its later layers.

the structure allows feature reuse, and encourages upper layers to focus on creating new features.


Our method only collects translation pieces to help NMT for translating a new sentence and does not influence the training process of NMT.

our method does not increase the NMT training time.


target-sentence state, deciding which word to output next.

of this ability to capture long-distance dependencies, NMT has achieved great improvements in a number of areas that have bedeviled traditional methods such as phrasebased MT (PBMT; Koehn et al.


One other phenomenon that was poorly handled by PBMT was homographs -words that have the same surface form but multiple senses.

pBMT systems required specific separate modules to incorporate long-term context, performing word-sense (Carpuat and Wu, 2007b;pu et al., 2017) or phrase-sense (Carpuat and Wu, 2007a) disambiguation to improve their handling of these phenomena.


As a result, PBMT systems required specific separate modules to incorporate long-term context, performing word-sense (Carpuat and Wu, 2007b;Pu et al., 2017) or phrase-sense (Carpuat and Wu, 2007a) disambiguation to improve their handling of these phenomena.

we may wonder: do NMT systems suffer from the same problems when translating homographs?


M e = M c ) since the vocabulary space is the same for context network and encoder.

our context network only slightly increases the number of model parameters.


Similarly, we only report results on the RNNSearch and λ is set as 0.7. shall be very time consuming because we need to do much more sampling.

there is a trade-off between the accuracy and computation complexity here.


(2016) to modify an LSTM layer to include a memory with different cells for each previous output representation.

at each time step, the hidden layer can select past information dynamically from the memory.


Both tasks predict one word at a time conditioned over previous words, however, in NMT the previous targetword-inputs are not given, they have to be generated by the decoder.

the output could be conditioned over previous erroneous predictions  Figure 2 shows a comparison of the distribution of attention of the different self-attentive models described in this paper, on Spanish-to-English NMT (the other two language pairs exhibit similar distributions).


For example, it is relatively easy to guess the "comma" by using the history words in language model task, where there are no bilingual information at all.

during the training, the model tries to adjust the parameters for highly frequent words from y i−1 and s i while neglecting the attention model.


Despite being similar, input and output embeddings solve different tasks.

fully tying input and output embedding sub-networks in subwordaware neural language models is worse than letting them be slightly different.


For example, even the ill-rhyming flip-flsp has the same edit distance as the correct reduplicative flip-flop.

we use a distance measure (MIR) defined over the phoneme space (Hixon et al., 2011) which effectively captures the similarity of two phonetic sequences by modeling the affinities between pairs of phonemes using a point access mutation matrix and is superior over metrics like PER (phoneme error rate).


When assigning the label O to non-entity regions, we only follow the detection order rule.

two gold label sequences {O, B-Protein, O, O, O, O} and {O, B-DNA, I-DNA, I-DNA, I-DNA, O} are assigned to the given word sequence "Mouse interleukin-2 receptor alpha gene expression" as shown in Figure 2.


Although it is possible to ignore the latter entity based on 'lex' attribute and its belonging sentence, this rule fails to deal with entity "c-jun gene" in the example of "c-fos and c-jun genes" as the 'lex' of "c-jun gene" is mistaken as "c-jun genes".

in this case, we ignored "c-fos transcript" and instead kept the "c-jun transcripts" as a flat entity.


Intuitively, when a human judges a premise-hypothesis relationship, s/he might consider back-and-forth reading of both sentences before coming to a conclusion.

it is essential to encode the premise-hypothesis dependency relations from both directions to optimize the understanding of their relationship.


For space efficiency, common knowledge graphs such as Freebase (Bollacker et al., 2008), Yago (Suchanek et al., 2007), and NELL (Mitchell et al., 2015) by default only stores beliefs, rather than disbeliefs.

when training the embedding models, there is only the natural presence of the positive examples.


FrameId is crucial to the success of Semantic Role Labeling as FrameId errors account for most wrong predictions in current systems (Hartmann et al., 2017).


This leads to a lexicon that is a magnitude smaller than the FrameNet lexicon.

the initial situation for the German case is more difficult.


The semantic similarity model can identify subtle meaning differences, and correctly classify them as divergent.

the non-parallel detector has a higher false positive rate (22%) than the semantic similarity classifier (14%), while having similar false negative rates (11% v/s 12%).


Complete remission with reversal of pulmonary damage was achieved , as reported by CT scan , pulmonary function tests and functional status .

[...] query , in severe cases an aggressive treatment , combining and glucocorticoids as used in systemic vasculitis , is suggested .


Then the final hidden states of each chunk will be fed into chunk-level RNN with its original sequence order kept.

the chunk-level RNN can deal with pre-encoded chunk data with less sequential steps.


(2017), whereby pre-trained GloVe word vectors (Pennington et al., 2014) were used to initialize the embedding layer, which were not updated during training.

the embedding layer does not depend on the training set, and the effective vocabularies are the same.


Each sentence is a step that may change the state of one or more participants.

a process paragraph with m sentences and n participants will result in an (m + 1) × n grid representation.


The weak performance of EntNet and QRN on ProPara is understandable: both systems were designed with a different environment in mind, namely a large number of examples from a few conceptual domains (e.g., moving objects around a house), covering a limited vocabulary.

they might not scale well when applied to real procedural text, which justifies the importance of having a real challenge dataset like ProPara.


is the information in Y explainable by both L(T ) and C, then our goal is to choose L such that this variance is maximized after C has been fixed.

with this in mind, we formalize the task of deconfounded lexicon induction as finding a lexicon L that maximizes an informativeness coefficient, which measures the explanatory power of the lexicon beyond the information already contained in the confounders C. highly informative lexicons cannot simply collect words that reflect the confounds.


It can also help detect "soft" outages like slowdowns and account blockages, things that direct monitoring cannot always detect.

this paper is not suggesting a replacement, but rather a new source of valuable information.


This is relevant to this paper because we know which days are attacks (in training).

when a tweet is on an attack day, we assign the tweet a label ATTACK, and bias the Labeled LDA learner to assign its words to an attack-related topic.


The increasing popularity of microblogs results in a huge volume of daily-produced user-generated data.

such explosive growth of data far outpaces human beings' reading and understanding capacity.


Attention (Att (LSTM) and Att (BiLSTM)) and MemNN can recognize critical content in conversations, which would indicate keyphrases in target posts.

our keyphrase extraction framework with attention or MemNN encoder has generally better F1 scores than those with other encoders.


The reason might be that it considers each word in conversations independent and equally important.

when using this encoder, non-topic words like "if " and "LOL" may distract the keyphrase tagger in identifying the key information.


For non-keyphrases, MemNN highlights non-topic words, e.g., "be", "to".

features learned for keyphrases and non-keyphrases are different, which can thus benefit keyphrase tagger to correctly distinguish keyphrases from non-keyphrases.


The summary for the social topic elaborates on universal basic income and social security.

for the same article, the proposed approach is capable of generating different summaries tuned to the input topic of interest.