The standard significance testing methods in NLP are therefore rank-or randomization-based nonparametric tests (Yeh, 2000;Riezler and Maxwell, 2005;Berg-Kirkpatrick et al., 2012).

most system comparisons across words, sentences or documents use bootstrap tests (Efron and Tibshirani, 1993) or approximate randomization (Noreen, 1989), while studies that compare performance across data sets use rank-based tests such as Wilcoxon's test.


This is the false belief that performance on the test section of the Wall Street Journal (WSJ) part of the English Penn treebank is representative for performance on other texts in English.

it is the belief that our samples are always representative.


In this paper, we critically examine significance results in NLP by simulations, as well as running a series of experiments comparing state-of-the-art POS taggers, dependency parsers, and NER systems, focusing on the sensitivity of p-values to various factors.

we address three important factors: Sample size.


If researchers needed to test more than 7 approaches to "hit the nail", they would never get to write papers.

with ↵ = 0.05, and set to 0.5, we find that the probability of a research finding being truegiven there is no selection bias and with perfectly valid metrics -is just 50%: if researchers do a perfect experiment and report p < 0.05, the chance of that finding being true is the chance of seeing tail when flipping a coin.


We employ the Polylingual Topic Model (Mimno et al., 2009), which is originally used to model corresponding documents in different languages that are topically comparable, but not parallel translations.

we employ our previous work (Mason and Charniak, 2013) which extends this model to topically similar images and natural language captions.


This correlation has broad implications on the ability to perform inference from native language structure to second language performance and vice versa.

it paves the way for a novel and powerful framework for comparing native languages through second language performance.


Our work incorporates an NLI component, but departs from the performance optimization orientation towards leveraging computational analysis for better understanding of the relations between native language typology and ESL usage.

our choice of NLI features is driven by their relevance to linguistic typology rather than their contribution to classification performance.


Our feature set, summarized in table 2, contains features which are strongly related to many of the structural features in WALS.

we use features derived from labeled dependency parses.


Importantly, as opposed to the WALS based prediction, our ESL based method does not require any typological features for inferring language similarities and constructing the similarity tree.

no typological information is required for the target languages.


We show that learning to rank models based on relational syntactic structures defined between the clues and the answer can improve both modules above.

our approach accesses the DB using a search engine and reranks its output by modeling paraphrasing.


It incorporates additional knowledge sources, provides a solver for the Italian language and improves the clues retrieval model from DB.

it enables partial matching to retrieve clues that do not perfectly overlap with the query.


It was placed 92nd out of more than 600 opponents in the 2013 ACPT.

for QA using syntactic structures, a referring work for our research is the IBM Watson system (Ferrucci et al., 2010a).


Syntactic Tree Kernel (STK), also known as a subset tree kernel (Collins and Duffy, 2002), maps objects in the space of all possible tree fragments constrained by the rule that the sibling nodes from their parents cannot be separated.

substructures are composed by atomic building blocks corresponding to nodes along with all of their direct children.


It generates all possible connected tree fragments, e.g., sibling nodes can also be separated and be part of different tree fragments.

a fragment is any possible tree path, from whose nodes other tree paths can depart.


However, we make a simplifying assumption that the intervals do not overlap and every real number belongs to an interval.

our goal is to induce a segmentation of the line: event mentions corresponding to the same interval are then regarded as paraphrases.


We consider two evaluations: one which measures the model's ability to distinguish the named color from a random alternative-analogous to the evaluation in Yu and Siskind (2013)-and one which measures the absolute difference between predicted and true color values.

in the first evaluation the model is presented with the name of a color and a pair of candidates, one  Table 2: Results for the color selection task.


Then the simple regression model described in the preceding section, extended to include alignment and constraint variables, has the form of the factor graph shown in Figure 4.

the joint distribution p(T, V) is the product of four groups of factors: Alignment factors ψ ta , which use a log-linear model to score neighboring pairs of factors with a feature function ϕ a : Constraint factors ψ tc , which map text features onto constraint values: Prediction factors ψ acv which encourage predicted constraints and path features to agree: A global factor ψ a * (A 1 , A 2 , • • • ) which places an arbitrary combinatorial constraint on the alignment.


As in the case of stocks, most of the prediction errors on this task are a result of misalignment.

many of the dialogues make passing reference to already-visited landmarks, or define destinations in empty regions of the map in terms of multiple landmarks simultaneously.


A deeper analysis (Section 8) shows that common lexical-semantic resources, on which we rely as well, are either too noisy or provide inadequate recall regarding lexical entailment.

we find that predicate inference within propositions often goes beyond inference between the predicates' linguistic meanings.


The following methods use distant supervision from WordNet (as in Berant et al's work, Section 2.2.1) to explicitly train component-level entailment estimators.

we train a logistic regression model for each component as specified in Equa-tions 1 in Section 5.1.


The F 1 difference between these methods is statistically significant with McNemar's test (1947) with p 0.01.

cEc corrected Joint Features 7621 times, while the opposite occurred only 4048 times.


In this work, we produce such connections by finding cross-lingual word pairs using the Wikitionary 1 , which works as free bilingual dictionaries between language pairs.

we first constructed a source language dictionary with all words that appeared in the sentences from the source language domain and translate these words to the target language using the Wikitionary.


In this paper, we explore a third strategy by using machine translation to create synthetic training data from the original sourceside annotations.

we apply this technique to dependency parsing, using a cross-lingually unified treebank for adequate evaluation.


All of them are based on automatic word alignment and subsequent extraction of translation equivalents as common in phrase-based SMT.

word alignment is performed using GIZA++ (Och and Ney, 2003) and IBM model 4 as the final model for creating the Viterbi word alignments for all parallel corpora used in our experiments.


The second component, P π (u | t), conditions on the previous tag type, t, and assigns higher probability to pairs of tags that can be combined.

the probability of transitioning from t to u in the Dirichlet mean distribution is given by 1 We discuss the two mixture components in turn.


We propose an improvement, P G , expressed as a probabilistic grammar: where A, B, C are categories and a is an atomic category (and terminal): We have designed this grammar to capture several important CCG characteristics.

we encode four main ideas, each captured through a different parameter of the grammar and discussed in greater detail below: 1.


Translation models learned over small amounts of parallel data suffer from the problem of low coverage.

they do not include translations for many words and phrases.


(2012) found that only 2% of phrase pairs in German-English, Czech-English, Spanish-English, and French-English phrase tables consist of multi-word source and target phrases and are non-compositional.

for these languages, the vast majority of phrase pairs in a given phrase table could be composed from smaller units.


Therefore, before moving to phrases, we use a bilingual lexicon induction technique to identify translations for unigrams.

because we assume a setting where we have some small amount of parallel data, we follow our prior work on supervised bilingual lexicon induction (Irvine and Callison-Burch, 2013b).


showed that relations between words are reflected to a large extent in the offsets between their vector embeddings (queen − king ≈ woman − man), and thus the vector of the hidden word b * will be similar to the vector b − a + a * , suggesting that the analogy question can be solved by optimizing: where V is the vocabulary excluding the question words b, a and a * , and sim is a similarity measure.

they used the cosine similarity measure, defined as: resulting in: Since cosine is inverse to the angle, high cosine similarity (close to 1) means that the vectors share a very similar direction.


Although originally designed to improve language models, later works have shown that such word embeddings are useful in various other NLP tasks, including measuring semantic similarity with vector arithmetics (Collobert and Weston, 2008;Mikolov et al., 2013).


Furthermore, they use 'discounting' methods to improve the estimates of conditional probabilities when learning data is sparse.

in this work we use the Kneser-Ney n-gram model (Kneser and Ney, 1995).


Suppose the tokenized output sentence H of a grammatical error correction system given the above sentence is: There is no doubt , tracking system has brought many benefits in this information age .

the set of system edits is e = {a doubt → doubt}.


One of the most popular approaches to nonspecific error type correction, incorporated to various extents in many teams' systems, was the Language Model (LM) based approach.

the probability of a learner n-gram is compared with the probability of a candidate corrected ngram, and if the difference is greater than some threshold, an error was perceived to have been detected and a higher scoring replacement n-gram could be suggested.


This was also the practice adopted in HOO 2011, HOO 2012, and CoNLL-2013.

after the teams submitted their system output and the error annotations on the test essays were released, we allowed the teams to propose alternative answers (goldstandard edits), to be submitted within four days after the initial error annotations were released.


As described in Section 2.5, we can evaluate performance by error type in order to identify and remove unnecessary corrections.

we tried to optimise our best hybrid system (#6) by filtering out types with zero precision.


We thus derive the confusion sets using the training data.

for every source word that is tagged as a word form error in the training data, the confusion set includes all labels to which that word is mapped in the training data.


We found that for every error type, the combined model is superior to each of the single classifiers, as it combines the advantages of both of the classifiers so that they complement one another.

while each of the learner and native components have similar precision, since the predictions made differ, the recall of the combined model improves.


the correction phase, multiple sentence candidates are automatically generated (based on the transformation rules) and they are checked against the language model to see which one yields the lowest perplexity.

once an error is found, its correction way tends to be applied provided that the language model offers another solution.


Suffix: Suffix, one of the important factors, helps to capture the grammatical agreements between predicates and arguments within a sentence.

the endings of plural nouns and inflected verb variants are useful for the detection of agreement violations that shown up in word morphologies.


6 Lastly, we present an experimental validation of the online-to-batch application of our algorithm.

we would like to evaluate predictions that use the final weight vector w T,K by comparing the generalization error with the empirical error stated in Theorem 3.


We train a coreference model by learning a pairwise mention scoring function.

given a mention-pair ) is a pairwise feature vector and w 2 is the weight vector.


We ensure a balanced training of the mention head detection model by adding sub-sampled invalid mention head candidates as negative examples.

after mention head candidate generation (described in Sec.


The sequence labeling component builds on the following assumption: Assumption Different mentions have different heads, and heads do not overlap with each other.

for each m i, j , we have a corresponding head h a,b where i ≤ a ≤ b ≤ j.


Adding context parameters never hurts, and in most cases, using priors based on intrinsic, cross-lingual aspects of the CCG formalism to bias those parameters toward connectivity provides further gains.

biasing the model toward trees in which constituent labels are combinable with their adjacent supertags frequently helps the model.


We propose a strategy that uses the left-to-right order of the string to constrain the structure of the derivation forest and experiment with different tactics in dealing with unaligned words on the string side and unaligned edges on the graph side.


In this paper, we present a hybrid approach for performing token and sentence levels Dialect Identification in Arabic.

we try to identify whether each token in a given sentence belongs to Modern Standard Arabic (MSA), Egyptian Dialectal Arabic (EDA) or some other class and whether the whole sentence is mostly EDA or MSA.


We study the impact of source length and verbosity of the tuning dataset on the performance of parameter optimizers such as MERT and PRO for statistical machine translation.

we test whether the verbosity of the resulting translations can be modified by varying the length or the verbosity of the tuning sentences.


These are very strong positive correlations and they show that MERT tends to learn SMT parameters that yield translations preserving the verbosity, e.g., lower verbosity on the tuning dataset will yield test-time translations that are less verbose, while higher verbosity on the tuning dataset will yield test-time translations that are more verbose.

mERT learns to generate a fixed number of words per input word.


It combines the advantages of the cross-lingual annotation projection methods, which project labeled information into the target language domain, and the cross-lingual representation learning methods, which learn latent interlingual features.

we first train a dependency parser on the labeled source language data and use it to infer labels for the unlabeled source language sentences of the parallel resources.


The signature-based 502 interlingual features together with the 13 universal POS tag features can be used as language independent features for all the words in the vocabulary constructed across the source and target language domains.

we can form a word-feature matrix with the constructed vocabulary and the total 515 language independent features.


We have also conducted experiments for the learning scenarios where a small set of labeled training sentences from the target language is available.

we conducted experiments with a few different numbers of additional labeled training sentences from the target language, {500, 1000, 1500}, using three methods, RLAP, Delex and Proj2.


CBOW derives v ctx(w) based on averaging over the context words.

it estimates the probability of each w given its local context.


The basic hypothesis of CA was formulated by Lado (1957), who suggested that "we can predict and describe the patterns that will cause difficulty in learning, and those that will not cause difficulty, by comparing systematically the language and culture to be learned with the native language and culture of the student".

lado postulated that divergences between the native and foreign languages will negatively affect learning and lead to increased error rates in the foreign language.


For each error type, the null hypothesis of the test is that error fraction samples of all the native languages are drawn from the same underlying distribution.

rejection of the null hypothesis implies a significant difference between the relative error frequencies of at least one language pair.


In addition to the theoretical contribution, the outcome of our work has a strong potential to be beneficial in practical setups.

it can be utilized for developing educational curricula that focus on the areas of difficulty that are characteristic of different native languages.


4: Train a skip-gram word embedding model on the resulting list of word and POS sequences from both the source and target language We assume the same POS tagset is used for both the source and target language, 2 and learn word embeddings for each word type in both languages into the same syntactic space of nearby POS contexts.

we develop a predictive model of the tags to the left and right of a word, as illustrated in Figure 1 and outlined in Algorithm 1.


The next step in our research is building a system for retrieval of semantically equivalent questions.

given a corpus and a question, the task is to find all questions that are semantically equivalent to the given one in the corpus.


For discriminative training, we also need incorrect parses to act as the negative examples.

we want negative examples that will allow us to optimize the pruning decisions made by the parser.


The figure shows the expected output for the Turkish word gençleşmelerin 'of rejuvenatings'.

it shows the full labeled morphological segmentation, from which three representations can be directly derived: the unlabeled morphological segmentation, the stem/root 1 and the structured morphological tag containing POS and inflectional features.


(i) The labels are needed for many tasks, for instance in sentiment analysis detecting morphologically encoded negation, as in Turkish, is crucial.

for many applications UMS is insufficient.


Therefore, we optimize the F β objective, where β 2 balances the recallprecision trade-off.

4 we expect structured resources to facilitate high-precision inferences, and are thus more interested in lower values of β 2 , which emphasize precision over recall.


We trained both variants with back-propagation (Rumelhart et al., 1986) and gradient ascent.

we optimized F β using a variant of Jansche's (2005) derivation of F β -optimized logistic regression (see suplementary material 5 for full derivation).


In particular, we optimized F β using a variant of Jansche's (2005) derivation of F β -optimized logistic regression (see suplementary material 5 for full derivation).

this model can theoretically quantify how indicative each edge type is of R. it can differentiate weakly indicative edges (e.g.


Classification We represent each path p as a binary "bag of edges" φ, i.e.

the set of edge types that were applied in p. Given a term-pair (x, y) represented as a path-set paths(x, y), and a whitelist w, the model classifies (x, y) as positive if: : 1.


Figure 4 demonstrates that three ITEMRESP inference algorithms, Gibbs sampling (Gibbs), meanfield variational inference (Var), and the iterated conditional modes algorithm (ICM) (Besag, 1986), are brought into better agreement after optimizing their hyperparameters via grid search.

the algorithms in Figure 4b are in better agreement, particularly near the extremes, than the algorithms in Figure 4a.


However, one could also argue that the original behavior of CSLDA is in some ways desirable.

if two classes of documents are mostly the same both topically and in terms of annotator decisions, perhaps those classes ought to be collapsed.


We describe CSLDA, a generative, data-aware crowdsourcing model that addresses important modeling deficiencies identified in previous work.

cSLDA handles data in which the natural document clusters are at odds with the intended document labels.


The third type we consider comprises all verbs whose perspective is that of the speaker.

these are verbs whose sentiment is primarily that of the speaker of the utterance rather than persons involved in the action to which is referred.


The verb just conveys to make something worse.


Beside obvious errors in automatic frame assignment, we also found that there are problems inherent in the frame design.

the notion of SP-verbs ( §3.3) is not properly reflected.


Our overarching research goal is to learn compositional models of grounded attributes towards describing an object in a scene, rather than just identifying it.

we do not only learn to recognize instances of objects, but also learn attributes constrained to feature spaces that will be compatible with contextual modifiers such as dark/light in terms of color, or small/large in terms of size and object classification.


A crucial task throughout the entire development life cycle is requirements traceability, which is concerned with linking requirements in which one is a refinement of the other.

one is given a set of high-level (coarse-grained) requirements and a set of lowlevel (fine-grained) requirements, and the goal of requirements traceability is to find for each highlevel requirement all the low-level requirements that refine it.


In light of this weakness, we recast requirements traceability as a supervised binary classification task, where we classify each pair of highand low-level requirements as positive (having a link) or negative (not having a link).

we propose a knowledge-rich approach to the task, where we extend a supervised baseline employing only word pairs and LDA-induced topics as features (see Section 4) with two types of humansupplied knowledge.


In the context of requirements traceability, rationales are human-annotated words or phrases in a pair of high-and low-level requirements that motivated a human annotator to establish a link between the two.

rationales contain the information relevant to the establishment of a link.


Hence, nouns/verbs that appear in only the test data in a fold experiment are not included in that experiment's ontology.


Motivated by previous work, we create features based on the topics induced by an LDA model for a requirement.

we first train an LDA model on our data to obtain n topics, where n is to be tuned jointly with C on the development (dev) set.


The reason is simple: as discussed previously, a negative instance in our task stems from the absence of evidence that the two requirements should be linked.

after removing the rationales from a pair of linked requirements, the pseudo-instance created from the remaining text fragments should be labeled as negative.


Given this observation, we create a positive pseudo-instance from each pair of linked requirements by removing any text fragments from the pair that are not part of a rationale.

we use only the rationales to create positive pseudo-instances.


We let the learner decide how confidently it wants to classify these additional training instances based on the dev data.

we tune this confidence parameter µ jointly with the C value to maximize F-score on the dev set.


As mentioned before, we derive additional features for the SVM learner from the verb and noun clusters in the hand-built ontology.

we derive five types of features: Verb pairs.


As another example, the noun pair feature (folder, folder) is suggestive of a positive instance, but the instance may turn out to be negative if one requirement concerns creating folders and the other concerns deleting folders.

we need to develop features that encode the relationship between verbs and nouns.


We select the nouns, noun phrases (NPs) and verbs in the training set to be clustered.

we select a verb/noun/NP if (1) it appears more than once in the training data; (2) it contains at least three characters (thus avoiding verbs such as be); and (3) it appears in the high-level but not the low-level requirements and vice versa.


The "No pseudo" column of Table 4 shows the results when the learner learns from only real training instances (i.e., no pseudo-instances).

rows 1 and 2 show the results of the two unsupervised baselines, tf-idf and LDA, respectively.


Comparing these results with the corresponding "Pseudo pos only" results, we can see that  additionally employing negative pseudo-instances consistently improves performance: F-scores rise by 0.8-4.1% on Pine and 3.0-4.9% on World-VistA.

the improvements in F-score in three of the six cases (Pine/Baseline, World-VistA/manual, WorldVistA/induced) are statistically significant.


In this subsection, we empirically verify the correctness of this claim.

the "Pseudo residual" column of Table 4 shows the results when each of the "No pseudo" systems is additionally trained on the pos-itive pseudo-instances created using Zaidan et al.


Noun phrases containing adjectives are extracted using part-of-speech information and dependency arcs from the gold annotation.

all treebanks are converted to coarse universal part-of-speech tags, using existing conventional mappings from the original tagset to the universal tagset (Petrov et al., 2012).


We ask, then, if a model is sufficient where the AP is not divided into LeftAP and RightAP, but holistically represented by the size of the whole AP.

a simpler Heaviness model does not make a difference between left and right dependent of adjectives: all heavy adjectives are predicted to move post-nominally.


Our representation has three advantages over existing alternatives: First, being based on symmetric word relationships, it is highly suitable for word similarity prediction.

on the SimLex999 word similarity dataset, our model achieves a Spearman's ρ score of 0.517, compared to 0.462 of the state-of-the-art word2vec model.


Interestingly, the impressive results of these models are achieved despite the shallow linguistic information most of them consider, which is limited to the tendency of words to co-occur together in a pre-specified context window.

very little information is encoded about the syntactic and semantic relations between the participating words, and, instead, a bag-of-words approach is taken.


Our SP-based word representation is flexible.

by exploiting the semantics of the pattern based features, our representation can be adapted to fit the specific needs of target NLP applications.


In light of this, we construct our word embeddings to be controllable of antonyms.

our model contains an antonym parameter that can be turned on in order to generate word embeddings that represent antonyms as dissimilar, and turned off to represent them as similar.


Each column vector iñ W ∈ R 2d(c+1)×|W| corresponds to a word.

we assign a logistic regression model for each word, and we can train the embeddings using the one-versus-rest approach to make p(w in i |f ) larger than p(w ′ |f ) for w ′ ̸ = w in i .


The first baseline is RelEmb itself, but without applying the learning method on the unlabeled corpus.

we train the softmax classifier from Section 3.3 on the labeled training data with randomly initialized model parameters.


To avoid overwriting value by multiple satisfied rules, the rules are listed in ascending order of temporal unit.

the rules for seconds or minutes are listed before the rules for hours or days.


Algorithm 1 Online LIFE 1: INPUT: a;b t ;g;b c ;w p ;S s ;t 1 ;t 2 ;t 3 2: for s=1 to S stream do 3: if s=1 then 5: 6: else 8: for each item w i ∈ Dic s−1 do 11: if w i ∈ Dic s then 12: 13: 14: for each w i ∈ Dic s do 16: 17: 18: end for for each w i ∈ Dic s do 20:  33: The LS dictionary of O-LIFE is updated as it reads data.

|Dic prev | ≤ |Dic cur | and Dic prev ⊆ Dic cur , where Dic prev is the previous dictionary and Dic cur is the current dictionary.


From a linguistic perspective, a spine corresponds to the projection of the word in the constituent tree.

the spine of a word consists of the constituents whose head is the word.


This means that left and right arc operations can add spinal edges in a free manner, without explicitly encoding how these edges relate to each other.

we rely on the statistical model to correctly build a spine by adding left and right spinal edges along the transition process in a bottom-up fashion.


It is easy to see that these constraints do not prevent the transition process from ending.

even though the constraints invalidate arc operations, the arc-eager process can always finish by leaving tokens in the buffer without any head assigned, in which case the resulting derivation is a forest of several projective trees.


These results have sparked interest in applying DNNs to natural language processing problems as well.

in machine translation (MT), there has been an active body of work recently in utilizing neural language models (NLMs) to improve translation quality.


Binary discrimination First, we trained L2regularized logistic regression models to discriminate between all pairs of POS tags only using gaze features.

for example we selected all words annotated as NOUN or VERB, and trained a logistic regression model to discriminate between the two in a five-fold cross validation setup.


It mainly relies on the quantitative analysis of the glosses associated to synsets, and on the use of the resulting vector space model representations for semisupervised synset classification.

temporal classifiers are learned over manually labeled synsets (seed list), and new learning synsets are chosen based on their specific semantic relation (e.g.


The s-t mincut algorithm is based on finding minimum cuts in a graph, and uses pairwise relationships among examples in order to learn from both labeled and unlabeled data.

it outputs a classification corresponding to partitioning a graph in a way that minimizes the number of similar pairs of examples that are given different labels.


6 Then, depending on the classification task, a two-class SVM classifier is built from the Weka platform.

7 the SVM membership scores are transformed into probability estimates based on Platt calibration (Niculescu-Mizil and Caruana, 2005), which are directly mapped to edge weights.


Table 5 shows that our configurations (MC1, MC2) perform significantly better than previous approaches.

they achieve highest accuracies for temporal vs. atemporal and past, present, future classifications with improvements of 11.3% and 10.3%, respectively, over the second-best strategy, namely TWnH.


The s-t mincut approach performs consistently better than the propagation strategy.

we show that with 400 labeled examples better results can be obtained than relying on 1264 training items within a propagation paradigm.


Comparative classification results are reported in Table 7 and show small improvements in the mincut strategy, when compared to propagation strategies.

for tweet classification, TWnP shows similar results mainly due to its large coverage of temporal senses (counterbalanced by low precision as confirmed by Table 5).


In order to produce comparative results with the best-performing system at TempEval-3, namely UTTime (Laokulrat et al., 2013) for the above task, we follow the guidelines and use the same datasets provided by the organizers (UzZaman et al., 2013).

we restrict our experiment to a subset of relations, namely BEFORE (past), AF-TER (future), and INCLUDES (present), with all other relations mapped to the NA−RELATION for the following two subtasks: event to document creation time and event to same sentence event.


To solve this classification problem, we adopt a simple supervised learning strategy based on state-of-the-art characteristics, plus features from a time-augmented version of WordNet.

each pair of entities to be classified as BE-FORE, AFTER, INCLUDES or NA-RELATION is encoded with the following features: -String features: the tokens and lemmas of each entity pair; -Grammatical features: the part-of-speech tags TWnP TWnH MC2 Sentence classification (p,r,f1) (69.7,66.1,66.7) (68.2,70.5,69.3) (69.8,67.6,68.6) (73.3,70.1 71.4) Tweet classification (p,r,f1) (51.4,47.1,49.1) (50.4,52.8,51.5) (51.8,48.2,49.8) (52.8,50.6,51.6) Table 7: Results for temporal sentence and tweet classification performed on 10-fold cross validation with SVM with Weka default parameters.


The underlying idea is that instead of using a single view on the data (as done in prior work), multiple views result in better temporal classification accuracy.

both intrinsic and extrinsic experimental results confirm the soundness of the proposed approach and support our initial hypotheses.


J unsup is a quadratic function of {µ k }, and it can be minimized by setting its derivative with respect to {µ k } to zero.

then, we can easily solve {µ k } as µ k is equal to the mean of all the data points assigned to cluster k. the classical k-means algorithm only uses unlabeled data, and solves the clustering problem under the unsupervised learning framework.


By monitoring the filtered MRR after each training epoch, we selected the best model with the highest filtered MRR on the validation set.

for the entity prediction task, we selected δ = 10 and λ = 0.005 for WN11, δ = 5 and λ = 0.01 for FB13, and δ = 5 and λ = 0.005 for NELL186.


Hits@10 scores than TransE on both FB13 and NELL186 datasets.

on NELL186 TransE-NMM gains a significant improvement of 279 − 214 = 65 in the filtered mean rank (which is about 23% relative improvement), while on the FB13 dataset, TransE-NMM improves with 0.267−0.213 = 0.054 in the filtered MRR (which is about 25% relative improvement).


(2014), Table 4 shows that TransE in fact obtains a very competitive accuracy.

compared to the relation path model TransE-COMP (Guu et al., 2015), when model parameters were randomly initialized, TransE obtains 85.2 − 80.3 = 4.9% absolute accuracy improvement on the WN11 dataset while achieving similar score on the FB13 dataset.


Figure  2 shows that TransE-NMM gains better scores with higher τ value.

when τ = 500 TransE-NMM does significantly better than TransE in all entity prediction metrics.


Levy and Goldberg (2014b) proved that when the objective function in Equation (1) is applied to single-word contexts, it is optimized when: where PMI(t, c) = log p(t,c) pα(t)p(c) is the pointwise mutual information between the target word t and the context word c. The analysis presented in Levy and Goldberg (2014b) is valid for every cooccurrence matrix that describes the joint distribution of two random variables.

it can be applied to our case, where the context is not just a single word but an entire sentential context of a target word.


Our model is closely related to language models, as can be seen in section 2.2 and tables 1 and 2.

it has a lot in common with LSTMbased language models, as both train LSTM neural networks with the objective to predict target words based on their (short and long range) context, and both use techniques, such as negative sampling, to address large vocabulary computational challenges during training (Jozefowicz et al., 2016).


3 In addition, we experimented with the following variations: (1) ignoring stopwords (2) performing a weighted average of the words in the context using tf-idf weights (3) considering just the 5-word window around the target word instead of the whole sentence.

in the WSD experiment the context provided for the target words is a full paragraph.


Yet, sentential context representations aim to reflect the information in the sentence only inasmuch as it is relevant to the target slot.

different target positions in the same sentence can yield completely different context representations.


As such, we hypothesize that it could contribute to various NLP systems that model context.

semisupervised systems may benefit from using our model, as it may carry more useful information learned from large corpora, than individual pretrained word embeddings do.


This metric evaluates the event prediction, taking into account prediction of each constituent.

it is defined as average of the accuracy of the predicate, the dependency, the first argument and the second argument predictions.


Most methods to date rely on structural features such as sentence position, number of upper-case words, or title words, and a wide range of measures of sentence centrality as signals for what is important in source documents.

in news articles, such as those used for the DUC2002 single-document summarization and the DUC2004 multi-document summarization tasks, 1 it is quite common that the most important information is repeated most frequently.


We then present the novel preference-based summarization system CPSum in detail.

we explain our training procedure and (contextual) sentence ranking methodology.


Oversampling: With oversampling we aim for hiding the important information in the corpus by increasing the frequency of unimportant information.

we iteratively search for a sentenceŝ witĥ where sim is a similarity measure for two sentences, and addŝ to a random document in topic D. Since we duplicate the sentences we make sure that we do not introduce new, important information to the corpus which is not reflected in the summary.


In order to investigate the effect of CF on language learning, we need to quantify the CF exchanges present in the corpus.

we need to find mechanisms for automatically detecting these.


This should not necessarily be interpreted as omissions receiving a higher proportion of CF over other error types, but rather as a consequence of omission errors being predominant at this stage of development (Saxton et al., 2005).

most instances of CF (30.8%) occur as a response to subject omission errors (SOEs); an example can be found in excerpt (2a), Section 3.2.


In this work, we seek a predictive model of the circumstances in which incidental acquisition is possible.

when can a learner guess the meaning of a novel word?


In the future, we hope to adapt the L1/L2 mix to the individual learner's competence.

we wish to present learners with interesting macaronic text that they are able to read with minimal assistance, but which still challenges them: text within the learner's "zone of proximal development" (Vygotsky, 1978).


The present paper evaluates a larger set of features under a more tightly controlled experimental setup.

in the present paper, our model does not have to predict which context words the learner understands, because there is only one L2 word per trial: any context words are always in L1.


First, the labels for the combined and cloze tasks tend to be more similar to one another, and quite different from the word task labels.

in the majority of cases, subjects completing cloze and combo tasks choose words that fit the context they've observed, while spelling influence in the word task doesn't appear to be quite as strong.


(2015) extended the notion of contrasting sentiments beyond the textual content at hand.

they analyzed previous posts to estimate the author's prior sentiment towards specific targets (i.e., entities).


A primary goal of this work is to explore whether neural models can effectively learn these rich contextualizing features, thus obviating the need to manually craft them.

the model we propose similarly aims to combine lexical clues with extralinguistic information.


We now investigate the user embeddings in more detail.

we are interested in two questions: first, what aspects are being captured in these representations; and second, how they contribute to the improved performance of our model.


To use stack LSTMs, we require vector representations of the elements that are stored in the stacks.

we require vector representations of atoms (words, possibly with part-of-speech tags) and parse fragments.


Our approach is then designed according to the requirements placed by this unrestricted proofreading protocol.

the ability of the system to suggest new translations coherent with the set of potentially disjoint translation segments validated by the user.


Our main contribution, described in Section 3, is a new proof-reading protocol focused on providing a more natural interaction between the user and the system.

we give complete freedom to the user to validate or to correct any part of the translation at any given interaction.


We share with (Sanchis-Trilles et al., 2008) the idea of making a more sophisticated use of the mouse actions performed by the user while interacting with the system, and with (González-Rubio et al., 2013) the common ITP formulation for both phrase-based and hierarchical MT models.

we significantly modify the prefix-based ITP implementation presented in the latter work to support the proposed unrestricted proof-reading protocol.


Then, we search for the optimal translation and alignment on it according to Equation (2).

we use hypergraphs to represent such search space.


Given a node ν and a coverage, we compute its score from its ingoing hyperedges.

q(ν, [m, n]) will be equal to the maximum score of the partial solutions computed from any ingoing hyperedge.


For simplicity, we restrict them to be fully-aligned to at most one user-validated segment 3 .

given a leaf hypernode λ ∈ V: where P MT (λ) is the MT probability (language plus translation model) of λ, and P E (w(λ),f m ) is the error correction probability between the target text covered by the leaf hypernode, w(λ), and the m-th user-validated segment in f .


Each correction corresponds to an edit operation performed by the user.

we used the following measures in our experiments: Word stroke ratio (WSR): Proposed in (Tomás and Casacuberta, 2006) as the quotient between the number of words edited by the user (wordstrokes), and the number of words in the final translation.


A similar trend (but in the opposite direction) can be observed for MAR: as we allowed the user to validate more words, the number of mouse actions increased until stabilization.

our ITP approach allows to reduce user typing effort at the expense of an increase in the number of mouse actions.


• We intrinsically evaluate our proposed models in terms of their posterior distributions on training and test data in Section 3.

we show that differences in uncertainty modelling are not captured by the usual point estimate metrics commonly used for this task.


Traditionally, wikification has been considered a downstream task of NER.

a named entity recognizer is first applied to identify mentions of interest, and then a wikifier is used to ground the extracted mentions to Wikipedia entries.


We can see that even though the disambiguation is not perfect, the FreeBase types still provide valuable information.

although "Albrecht Lehmann" is not an entry in Wikipedia, the wikifier still links "Albrecht" and "Lehmann" to people.


As in Ratinov and Roth (2009), we use the gazetteers as features.

we group them by topic, and use the name of the gazetteer file as the feature.


Our approach is inspired by, and extends, previous works on coreference resolution which show that incorporating external knowledge into a CR system is beneficial.

a variety of approaches (Ponzetto and Strube, 2006;Ng, 2007;Haghighi and Klein, 2009) have been shown to benefit from using external resources such as Wikipedia, WordNet (Miller, 1995), or YAGO (Suchanek et al., 2007).


We observe an improvement for Dcoref++ over the other systems, for all the metrics.

dcoref++ increases by 4 absolute points the CoNLL F1 score.


The analogous task on the level of event mentions is called (e) event linking (or: event coreference resolution) and is the focus of this paper.

the task is 1 Based on an example in (Araki and Mitamura, 2015).


The all singletons/one instance baselines predict every input link to be negative/positive, respectively.

the all-singletons baseline performs well, due to the large fraction of singleton event mentions in the dataset.


To estimate the coherence score, we again use cosine similarity between the vector of entity v e and that of context entities v ce .

to combine the proposed contextual information described above with standard NED features, we employ a method of supervised machine learning to rank the candidate entities given mention m and document d. we use Gradient Boosted Regression trees (GBRt) (Friedman, 2001), a stateof-the-art point-wise learning-to-rank algorithm widely used for various tasks, which has been recently adopted for the sort of tasks for which we employ it here (Meij et al., 2012).


We observed that approximately 48.6% errors were caused by metonymy mentions (Ling et al., 2015) (i.e., mentions with more than one plausible annotation).

our NED method often erred when an incorrect entity was highly popular and exactly matched the mention surface (e.g., "South Africa" referring to the entity South Africa national rugby union team rather than the entity South Africa).


Inspired by the recent works on word embedding (Bengio et al., 2003;Mikolov et al., 2013a;Mikolov et al., 2013b), knowledge embedding (Bordes et al., 2011;Bordes et al., 2013;Socher et al., 2013;Wang et al., 2014b) and joint embedding KBs and texts (Wang et al., 2014a;Zhong and Zhang, 2015), we propose to learn representations for entity disambiguation.

from KBs and texts, we jointly embed entities and words into the same low-dimensional continuous vector space.


Thus we also hope to encode such information into our embedding.

we further incorporate "url-anchor" co-occurrence to the training objective.


learning mappings from substring in one language to another, as opposed to learning single character mappings.

along with learning mappings of the form (k hindi → k bengali ), we also try to learn mappings of the form (kaa hindi → kaa bengali ).


Our pruning experiments also reveal some patterns in the distribution of redundancy in NMT.

we find that higher layers, attention and softmax weights are the most important, while lower layers and the embedding weights hold a lot of redundancy.


In order to prune NMT models aggressively without performance loss, we retrain our pruned networks.

we continue to train the remaining weights, but maintain the sparse structure introduced by pruning.


We evaluate the effectiveness of our pruning approaches on a state-of-the-art NMT model.

1 an attention-based English-German NMT system from (Luong et al., 2015a) is considered.


The favorable performance of the pruned and retrained models raises the question: can we get a shortcut to this performance by starting with sparse models?

rather than train, prune, and retrain, what if we simply prune then train?


In this work, we model how speakers plan the presentation of discourse structure optimally in terms of informativeness.

we propose a model that predicts whether the speaker will use or omit a discourse connective, given the sense of discourse relation s/he wants to convey.


A model that predicts the markedness of discourse relations not only contributes to a better understanding of the human language production mechanism, but is also important in generating natural, humanlike texts and dialogues.

the degree of markedness in discourse relations differs cross-lingually.


In order to explain how human speakers choose the optimal level of markedness in his utterance, we model how speakers rationally balance between ambiguity and redundancy.

we use the Rational Speech Acts (RSA) model (Frank and Goodman, 2012) to predict how speakers reason about the ambiguity of an utterance.


We define context C as the surrounding discourse relations.

the discourse contexts (and their abbreviation in Table 2) are: the full discourse sense annotated in PDTB (S), the 4-way top level sense (TS), the form of discourse presentation (F) such as 'explicit' or 'implicit' 6 , and the pair of sense and form (SF or TSF).


Experiments on large data condition SMT tasks showed that this outperforms state-ofthe-art language-model-based data selection methods significantly.

when the size of the in-domain data is small, semi-supervised CNN classifier can still select in-domain bilingual sentences to train an adapted SMT system.


2013b, we distinguish between high frequency and low frequency words.

we replace content words (nouns, verbs, adjectives, and adverbs), which are often low frequency, with their part-of-speech tags.


We investigate whether our model can benefit from state of the art text comprehension models, for which this task was designed.

we experiment with an LSTM-based (Hochreiter and Schmidhuber, 1997) recurrent neural network language model (RNNLM; Mikolov et al., 2010).


We treat this problem as a combinatory optimization problem and also employ Lagrangian Relaxation to solve the problem.

the parsing phase consists of two steps.


The intuition is that we use one tree to capture long distance information and the other two to capture coordination information.

1 we decompose each given graph y into three tree-like subgraphs g 1 , g 2 and g 3 .


With this property, each individual problem g k ← arg max g k f k (g k ) + u A k g k can be decoded easily, with modifications to the first order weights Algorithm 2: The Joint Decoding Algorithm Initialization: set u (0) , A 1 , A 2 , A 3 to 0, for k = 0 to K do of the edges in the three models.

let w k = u A k , then we can modify the ω The update of w 1 , w 2 , w 3 can be understood in an intuitive way.


Chinese is featured by its semantic-based orthography in the writing system.

chinese characters are composed of radicals and components, which are ideographic or phonetic symbols.


For example, the condition of word order (labeled as VO) has higher effectiveness in the verb 吃 chi 'eat' than in the verb 斷 duan 'snap'.

the senses of each verb can be identified by the most relevant syntactic conditions.


As shown in Figure 5, the group of the material radicals and the group of the movement radicals have different arrangement of the conditions.

the literal sense of a larger semantic group can also be identified by its syntactic distribution.


Several important adjustments should be made to our algorithm to account for peculiarities of this set-up.

(a) the OntoNotes guidelines do not provide annotations for singleton entities and (b) the official shared task score (MELA) relies strongly on B 3 and CEAF metrics.


In this section, we evaluate empirically the performance of the collaborative partitioning approach for a variety of ensembles.

we investigate ensembles of different size and composition with respect to the components' quality and assess different coreference scoring metrics as criteria for partition similarity.


To speed up the system, we use the techniques discussed in Section 3.1 above.

we rely on a very simple unweighted voting scheme: each component contributes equally to the final score.


In our last experiment, we depart from the CoNLL outputs to run the collaborative partitioning algorithm on top of the state-of-the-art coreference resolvers.

we com-bine three very different high-performing systems, berkeleycoref (Durrett et al., 2013), ims-hotcoref (Björkelund and Kuhn, 2014) and lspe (Haponchyk and Moschitti, 2017b;Haponchyk and Moschitti, 2017a).


In this work, we investigate the task of NED in a setting where only local and noisy context is available.

we create a dataset of 3.2M short text fragments extracted from web pages, each containing a mention of a named entity.


Given a question q, we consider each non-stopword token in q as a candidate for being an essential question term.

defining what is essential and what isn't is not an easy task and involves some level of inherent subjectivity.


The Fleiss' kappa statistic (Fleiss, 1971) for this task was κ = 0.58, indicating a level of inter-annotator agreement very close to 'substantial'.

all workers agreed on 36.5% of the terms and at least 4 agreed on 69.9% of the terms.


Here we report a second crowd-sourcing experiment that validates our hypothesis that the question terms marked above as essential are, in fact, essential for understanding and answering the questions.

we ask: Is the question still answerable by a human if a fraction of the essential question terms are eliminated?


First, for our supervised baseline, given (q l , q, a) as before, we ignore q and compute how often is q l annotated as essential in the entire dataset.

the score for q l is the proportion of times it was marked as essential in the annotated dataset.


The update of an input word is propagated to the embeddings of its associated senses, if any.

despite not being included in the input layer, senses still receive the same gradient of the associated input word, through a virtual connection.


In word similarity tasks for each word class, we show that our framework is both effective and efficient.

it improves the Spearman's ρ correlation with human scores on SimLex-999 over the best previously proposed class-specific contexts by 6 (A), 6 (V) and 5 (N) ρ points.


It is furthermore argued that the asymmetric nature of KL-divergence makes it a natural choice for modeling hypernymy.

it is proposed that the word embeddings could be improved by imposing that words that are in a hypernym relation have a low KL-divergence, allowing for a natural way to combine corpus statistics with available taxonomies.


we make the assumption that the higher PPMI(w, w j ) the stronger w is related to w j , but we do not otherwise assume that the numerical value of PPMI(w, w j ) is relevant.


In this way, we can also take into account the distribution of words within each ellipsoid.

for each word w j , we first determine a set of words w whose vector p w is inside these ellipsoids.


In particular, for each word w j , we first determine a set of words w whose vector p w is inside these ellipsoids.

for each word w that occurs at least once in the context of w j , or is among the 10 closest neighbors in the vector space of such a word, we test whether φ j , i.e.


For D-GloVe, we have therefore tuned the parameters using the ranges discussed in (Jameel and Schockaert, 2016).

we have used the parameters that gave the best results on the Google Analogy Test Set (see below).


Rather than treating this problem as a classification task, which would require selecting a threshold in addition to producing a score, we treat it as a ranking problem.

we evaluate to what extent the word pairs that are in a valid hypernym relation are the ones that receive the highest scores.


Levy and Goldberg (2014) demonstrated analytically that word2vec is implicitly factorizing a word-by-context matrix whose cell values are shifted PMI values.

the objective function and the input to word2vec are formally equivalent to traditional DSMs; thus the models should behave alike in the limit.


When it comes to syntagmatic relatedness between words, both models required accurate parameter settings.

the default configuration of both SVD and SGNS aims at optimizing the space in a way that paradigmatically similar words are put together.


Joint Learning: To extend conventional embedding learning, we combine context learning from an original corpus with external knowledge encoded by a regularizer, where the shared vocabulary set forms a bridge connecting the two spaces.

the objective function for CBOW with integrating the regularizer can be formulated as maximizing where not only w i , but also R(w i ) is predicted by the context words w i+j via their embeddings υ i+j .


Next we convert each − → Φ into a 0-1 vector based on the maximum values in − → Φ .

positions with maximum values are set to 1 and the rest are set to 0 (e.g.


Recently, neural network models have been used for AES (Alikaniotis et al., 2016;Dong and Zhang, 2016;Taghipour and Ng, 2016), giving better results compared to statistical models with handcrafted features.

distributed word representations are used for the input, and a neural network model is employed to combine word information, resulting in a single dense vector form of the whole essay.


This is centered around the idea of a propensity score, which Rosenbaum and Rubin (1983) define as the probability of being assigned to a treatment group based on observed characteristics of the subject, P (z i |x i ), typically estimated with a logistic regression model.

what is the "propensity" of a subject to obtain treatment?


Coupling Looking at the two hyperparameters independently does not tell the whole story, due to interactions between the two.

we observe that lower thresholds (lower τ ) work better when using heavier regularization (lower λ), and vice versa.


We choose shallow discourse parsing as the task to show FES-LM's applicability on news.

we evaluate on identifying the correct sense of discourse connectives (both explicit and implicit 10 We also tried Neural-LSTM (Pichotta and Mooney, 2016a) and context2vec (Melamud et al., 2016) model, but we cannot get better results.


We use the same language model training and feature generation techniques as FES-LM.

for F-LM, it is the same model as FC-SemLM defined in Peng and Roth (2016).


Other works aimed at learning a collection of structured events (Chambers, 2013;Cheung et al., 2013;Balasubramanian et al., 2013;Bamman and Smith, 2014;Nguyen et al., 2015;Inoue et al., 2016).

ferraro and Van Durme (2016) presented a unified probabilistic model of syntactic and semantic frames while also demonstrating improved coherence.


In order to guide the described beam decoding with the LM scores we perform a "synchronization".

we continue the beam search based on the character cED scores till the step s1 where all the segmentation hypothesis {(y 1 y 2 .


In order to make the comparison between the token regime and the type regime more fair in terms of the amount of training and testing data, we add one more experiment.

we train the cED model in the type regime using the same number of iterations as in the token regime: 100,000 iterations.


The two tasks took the form of "intruder" tasks, whereby subjects were tasked with identifying an intruder topic word for a given topic, or an intruder topic for a given document.

in the word intrusion task, an intruder word was added to the top-5 topic words, and annotators were asked to identify the intruder word.


In the case of the third and fourth documents, there is actually content further down in the document which is relevant to the topics the human annotators select, but it is not apparent in the document snippet presented to the annotators.

the effect is caused by resource limitations for the annotation task, that our automated method does not suffer from.


We address the problem of cross-language adaptation for question-question similarity reranking in community question answering, with the objective to port a system trained on one input language to another input language given labeled training data for the first language and only unlabeled data for the second language.

we propose to use adversarial training of neural networks to learn high-level features that are discriminative for the main learning task, and at the same time are invariant across the input languages.


Furthermore, we explore the question of whether adversarial training can be used to improve the performance of the network when we have some unlabeled examples in the target language.

we adapt the Domain Adversarial Neural Network model from (Ganin et al., 2016), which was originally used for domain adaptation, to our crosslanguage setting.


We apply this method to an extension of the SemEval-2016 Task 3, subtask B benchmark dataset for question-question similarity reranking (Nakov et al., 2016b).

we hired professional translators to translate the original English questions to Arabic, and we further collected additional unlabeled questions in English, which we also got translated into Arabic.


Note that maximizing L(U, V, w, U l , w l ) with respect to {U l , w l } is equivalent to minimizing the discriminator loss L l (ω) in Equation 6, which aims to improve the discrimination accuracy.

when put together, the updates of the shared parameters {U, V, w} for the two classifiers work adversarially with respect to each other.


When backpropagating to the generator network, they consider a slight variation of the reverse gradients with respect to the discriminator loss.

if ρ is the discriminator probability, instead of reversing the gradients of log(1 − ρ), they use the gradients of log ρ.


We trained Arabic-English cross-language word embeddings from the concatenation of these bitexts using bivec (Luong et al., 2015), a bilingual extension of word2vec, which has achieved excellent results on semantic tasks close to ours (Upadhyay et al., 2016).

we trained 200dimensional vectors using the parameters described in (Upadhyay et al., 2016), with a context window of size 5 and iterating for 5 epochs.


3 These features were proposed in the previous literature for the questionquestion similarity problem, and they are necessary to obtain state-of-the-art results.

we calculated the similarity between the two questions using machine translation evaluation metrics, as suggested in (Guzmán et al., 2016).


In particular, we calculated the similarity between the two questions using machine translation evaluation metrics, as suggested in (Guzmán et al., 2016).

we used BLEU (Papineni et al., 2002); NIST (Doddington, 2002); TER v0.7.25 (Snover et al., 2006); METEOR v1.4 (Lavie and Denkowski, 2009) with paraphrases; Unigram PRECISION; Unigram RECALL.


We further used as features the cosine similarity between question embeddings.

we used (i) 300-dimensional pre-trained Google News embeddings from (Mikolov et al., 2013), (ii) 100dimensional embeddings trained on the entire Qatar Living forum (Mihaylov and Nakov, 2016), and (iii) 25-dimensional Stanford neural parser embeddings (Socher et al., 2013).


1 As we interact with each student through a simple online tutoring system, we would like to track their evolving knowledge state and identify their learning style.

we would like to discover parameters and hyperparameters that can explain the evidence so far and predict how the student will react in future.


Our model assumes that when a student recalls or guesses the translation, he or she is attempting to solve a structured prediction problem of choosing the best translation, based on salient features of the input-output pair.

we characterize the student's knowledge as a vector of feature weights, which is updated as the student interacts with the system.


If the terminal does match, the algorithm increments the rule.

for each possible value of l k+2 , the algorithm constructs a new agenda item containing the same contents as the old agenda item, but with rule position k + 1.


the maximum priority of items in the agenda never increases.


Still, this approach is rather shallow as two words of the same sentence would be associated with almost the same context vector.

the approach does not really transform the embedding of a given word as a function of its context.


However, since there are not enough large disambiguated corpora, we need to approximate the word senses with coarse-grained information, e.g., the category of the context.

we can train a network to decide if two target words are sampled from sentences belonging to the same category.


In this work, we propose to use the aforementioned context/type matching heuristic as a guideline to derive simple neural baseline architectures for the extractive QA task.

we develop a simple neural, bag-of-words (BoW)-and a recurrent neural network (RNN) baseline, namely FastQA.


Furthermore, we explored several supervised domain adaptation techniques.

we demonstrated the usefulness of forgetting cost for factoid questions.


In Proto-Indo-European (PIE) roots, which are mostly structured CVC, stop-V-stop combinations have been found to be statistically underrepresented (Iverson and Salmons, 1992).

pIE seems to obey a cross-linguistic constraint that disfavors two similar consonants in a root.


This paper explores the simplest incarnation of the idea of similarity avoidance; namely, that two adjacent segments are preferably different in some way and that this difference reveals itself globally.

it is not assumed that the constraint is absolute; rather, an algorithm is developed that induces grouping of unknown phoneme symbols so as to maximize potential alternation of clusters in a sequence of symbols, i.e.


The rightmost column shows the total number of identified consonants in the language.

l, n, and r are always in this set, while s is nearly always present.


Social media has provided a rich opinion content that is valuable for diverse decision-making processes (Montoyo et al., 2012;Oliveira 2016), and sentiment analysis is being increasingly used to predict stock market variables (Antweiler and Frank 2014;Yu eta l., 2013;Schumaker et al., 2012).

social media messages are a useful source for supporting stock market decisions (Bollen et al., 2011;Oliveira et al., 2013).


We use long short-term memory (LSTM) based model (Hochreiter and Schmidhuber, 1997), which uses memory and gated mechanism to compute the hidden state.

we use a bidirectional LSTM model (Bi-LSTM) similar to the ones used in (Graves, 2013;Huang et al., 2015).


This reduction enables new ways of framing the learning problem.

it allows us to perform zero-shot learning: define new relations "on the fly", after the model has already been trained.


We used distant supervision to select the specific sentences in which each R(e, a) manifests.

we took the first sentence s in D to contain both e and a.


The outputs of the BiDAF model are the confidence scores of y start and y end , for each potential start and end.

we denote these scores as z start , z end ∈ R N , where N is the number of words in the sentence s. z start i indicates how likely the answer is to start at position i of the sentence (the higher the more likely); similarly, z end i indicates how likely the answer is to end at that index.


It is easy to figure out that not all sibling factors summed by Algorithm 1 are included by Algorithm 3.

if two normal edges that are adjacent to each other (say e 1 and e 2 ) are inserted with an empty edge, e 1 and e 2 are taken as a sibling part by Algorithm 1 but not 3.


To this end, we modify the covertinside span structure as well as its the construction rule.

we explictly utilize the index of the inner child provided by a covert-inside span.


We conduct experiments on both English and Chinese treebanks.


We apply entropy (a measure of uncertainty inherited from information theory) to a Distributional Semantic Model (DSM).

we exploit the idea of semantic generality applied in hypernym detection, to detect metaphoric change as a special process of meaning innovation.


It is followed by RHN k,L which consists of k residualized recurrent layers.

these are Recurrent Highway Network layers (Zilly et al., 2016), which are closely related to GRU networks, with the crucial difference that they increase the depth of the transform between timesteps; this is the recurrence depth L. The output of the final recurrent layer is passed through an attention-like lookback operator Attn which takes a weighted average of the activations across time steps.


Though we assume that the scores of the underlying statistical model can be used to define a probability model, we show that this is not a serious limitation.

as a baseline we use a probabilistic version of the neural mention-ranking model of Wiseman et al.


In this section we introduce neural mention ranking, the framework which underpins current stateof-the-art models (Clark and Manning, 2016a).

we consider a probabilistic version of the method proposed by Wiseman et al.


Let p(m i ∈ E u ) be the probability that mention m i corresponds to entity E u .

we now show that it can be computed recursively based on p(a i = j) as follows: if u < i, we consider all possible m j with which m i can be coreferent, and which can correspond to entity E u .


In this paper we marry these approaches, proposing NN models inspired by ideas from both.

our basic model receives the nonpivot features of an input example, encodes them into a hidden layer and then, instead of decoding the input layer as an autoencoder would do, it aims to decode the pivot features.


In contrast to these works our model can learn useful cross-domain representations for any type of input example and in our cross-domain sentiment classification experiments it learns document level embeddings.


To exploit these observations, in AE-SCL-SR the reconstruction matrix w r is pre-trained with a word embedding model and is kept fixed during the training and prediction phases of the neural network.

the i-th row of w r is set to be the vector representation of the i-th pivot feature as learned by the word embedding model.


As the documents with the highest similarity are ranked 1, the positive difference between the ranks of AE-SCL and those of AE-SCL-SR indicate that AE-SCL's rank is lower.

aE-SCL-SR learns more similar representations for documents with similar pivot features.


Nevertheless, the situation with dependency-based SRL has not changed: even recent state-of-the-art methods for this task heavily rely on syntactic features (Roth and Lapata, 2016;FitzGerald et al., 2015;Lei et al., 2015;Roth and Woodsend, 2014;Swayamdipta et al., 2016).

roth and Lapata (2016) argue that syntactic features are necessary for the dependency-based SrL and show that performance of their model degrades dramatically if syntactic paths between arguments and predicates are not provided as an input.


This contrasts with most other applications of LSTM encoders (for example, in syntactic parsing (Kiperwasser and Goldberg, 2016;Cross and Huang, 2016) or machine translation (Sutskever et al., 2014)), where sentences are typically encoded once and then used to predict the entire structured output (e.g., a syntactic tree or a target sentence).

when identifying arguments of a given predicate, we add a predicatespecific feature to the representation of each word in the sentence by concatenating a binary flag to the word representation of Section 2.1.


To make use of this information, we propose an approach that jointly models multiple morphosyntactic prediction tasks using a multi-task learning scheme.

we adopt parameter sharing in our bi-directional LSTM model in the hope that the shared parameters will store information beneficial to multiple tasks.


In order to do this, we adopt a multi-task learning approach (Collobert et al., 2011;Yang et al., 2016;Bingel and Søgaard, 2017;Martínez Alonso and Plank, 2017).

we use parameter sharing in the hidden layers of our bi-LSTM model so that we can generate a unified model that can carry information beneficial to each task.


For both data sets, we conduct additional experiments to investigate which morphosyntactic category in the tag dictionary embeddings contributes most to the performance.

instead of using all morphosyntactic categories to create the tag dictionary embeddings, we use only one at a time.


Specifically, instead of using all morphosyntactic categories to create the tag dictionary embeddings, we use only one at a time.

we skip the last step of concatenating all the sub-vectors defined for each morphosyntactic category, and use only one of the sub-vectors for the tag dictionary embeddings.


W rh acts like a key phrase detector that learns to capture the pattern of generation tokens or the relationship between multiple tokens.

the new input x t consists of information of the original input token w t , the DA representation d t , and the hidden context h t 1 .


This work proposes a multi-document summarization system that exploits the representational power of deep neural networks and the sentence relation information encoded in graph representations of document clusters.

we apply Graph Convolutional Networks (Kipf and Welling, 2017) on sentence relation graphs.


First, as prior methods in representing document clusters often adhere to the standard of cosine similarity (Erkan and Radev, 2004), our initial baseline approach naturally used this representation.

we add an edge between two sentences if the tf-idf cosine similarity measure between them, using the bag-of-words model, is above a threshold of 0.2.


While G-Flow's ADG provides many improvements from baseline graph representations, it suffers several disadvantages that diminish its ability to aid salience prediction when given to the neural network.

the ADG lacks much diversity in its assigned edge weights.


Each edge weight in the original ADG is then transformed by this sentence personalization score and normalized over the total outgoing scores.

for directed edge (u, v) ∈ E, the weight is The inclusion of the sentence personalization scores allows the PDG to account for macro-level features in each sentence, augmenting information for salience estimation.


This problem differs from traditional entity ranking (Kang et al., 2015) in that the entity rankings are driven by user queries and are optimized to their (ad-hoc) information needs, while entity relatedness ranking also aims to uncover the meanings of the the relatedness from the data.

while conventional entity semantic relatedness learns from data (editors or content providers' perspectives), and entity ranking learns from the user's perspective, the entity relatedness ranking takes the tradeoff between these views.


We then adopt the matching histogram mapping in (Guo et al., 2016) for the soft interaction of the ranking model.

denote the bag of entities representation of e s as C e s , and that of e t as C e t ; we discretize the soft matching (calculated by cosine similarity of the embedding vectors) of each entity pair in (C e s , C e t ) into different bins.


For learning representation from entity temporal signals, the intuition is to model the low-level temporal correlation between two multivariate time series.

we learn to embed these time series of equal size T into an Euclidean space, such that similar pairs are close to each other.


Whereas, temporal models yield substantial improvement overall.

the TS-CNN-Att performs better than the no-attention model in most cases, improves 11% for Pearson@10, and 3% when considering the total rank.


For tweet publication time, we use only time of the day information (e.g., 12:19) and split it into multiple bins.

we interpret every 10 minutes as a bin (144 bins in total).


Our joint model effectively utilizes different sources of information including tweet message, users' social relationships, and metadata fields embedded in tweets and profiles.

we employed a neural network model to generate a dense vector representation for each information field and then used the concatenation of these representations as the feature for classification.


Although we attribute the latter to sparsity, more German data would be needed to evaluate this hypothesis.

this can be achieved by relaxing some of the constraints we impose on the data.


We cast the problem of generating adversarial examples as an optimisation problem.

we propose a continuous inconsistency loss that measures the degree to which a set of sentences causes a model to violate a rule.


examples that cause NLI models to violate available background knowledge -as an optimisation problem.

we look for substitutions sets S = {X 1 → s 1 , .


(2017) and generate the sentences by editing prototypes extracted from a corpus.

for searching substitution sets whose sentences jointly have a high probability and are highly adversarial, as measured the inconsistency loss in Eq.


We investigate the relation between the transposition and deletion effects in word reading, i.e., the finding that readers can successfully read "SLAT" as "SALT", or "WRK" as "WORK", and the neighborhood effect.

we investigate whether lexical orthographic neighborhoods take into account transposition and deletion in determining neighbors.


Taking into consideration both the neighborhood effect and flexible letter encoding, we define the following research question: are the neighborhoods also defined using flexible letter encoding?

if we know that readers activate "SALT" upon reading "SLAT", does this also imply that the lexical neighborhood of "THREE" includes "THERE"?


In a practical sense, the significance is not that important: as all of these values are really small, there seems to be little reason to prefer one of the metrics over the other.

even though the difference between old 20 and the rd20 of a one hot encoded representation is significant, the difference in explained variance is so small to not really matter.


We propose a new neural network framework with global attention to tackle these challenges.

for each token in a query sentence, we propose to retrieve sentences that contain the same token from the document-level and corpuslevel contexts (e.g., document-level and corpuslevel supporting evidence for "Zywiec" in Figure 1).


toronto.edu/leaderboard/ 2 https://github.com/DigitalPhonetics/ reading-comprehension level separately (see Figure 1).

we first apply the compare-aggregate model to each plot sentence P i individually to obtain question and answer-weighted representations T w q i , T w a ij for each sentence.


Wang and Jiang (2016) implemented the aggregation operation as a single-layer CNN following Kim (2014).

they used a 1D convolution with filter sizes {1,3,5}, to capture unigrams, trigrams and 5grams.


The human evaluator also conducted an extensive comparison of the baseline word-level models with the hierarchical CNN models.

she looked at those questions where the performance of both model types differed most (in terms of the number of models out of 11 that solved the question correctly).


We then learn the transformation matrix that aligns German to English.

this approach fine-tunes German embeddings by applying a linear transformation that maps them into the English space.


Even if the language is mostly different, we believe that the parts that are relevant to churn detection stay the same.

if a model trained on tweets gives promising results on chatbot conversations, then it confirms that there is an underlying churn intent pattern that can be generalized across mediums.


As an alternative to learning a single model, we can use the text representation layer of the aforementioned single model in the word expert model.

after training the single model with the whole Wikipedia, we use the learned model of the LSTM as the text representation layer of the word expert models.


Thus, we introduce position embeddings when modelling the sentence and further generate the positionaware representations.

the position information is considered as a kind of features and embedded into position embeddings.


Since the generated data is based on discrete symbols (words), we usually adopt policy gradient method (Yu et al., 2017) to update model parameters of the generator.

given a bunch of translation candidates sampled from the generator, confidence scores calculated by the discriminator are employed as rewards to update the generator.


Our BGAN-NMT also obtains satisfactory translation quality against other existing NMT systems.

our BGAN-NMT outperforms Adversarial-NMT* by 1.14 BLEU points.


We present several methods for determining whether entities are discussed in a text, even though, potentially, they are not explicitly written.

we design a neural model that handles extraction of multiple entities jointly.


We study several deep and non-deep models for this task, that learn to extract latent entities from texts and overcome the fact that these are not mentioned explicitly.

we study a model that combines a neural recurrent network (Bi-GRUs) and multitask learning, showing that joint prediction of correlated entities could refine the performance.


(3) We present several algorithms for this task.

we find that learning multiple latent entities in a multi-task learning setting, while selecting the correct entities to learn together, reaches the best results for LEE.


This is done when the RNN component is sequentially fed by the embedding vectors {v t }, and iteratively compute a hidden state vector {h t } based on the previous hidden state and the current input embedding vector, using some function f .

moreover, the output of this unit {o t } in each timestamp t, is computed based on the current hidden state using a function g. we use a GRU unit as a RNN as presented by Cho et al.


It should be noted that multi-task learning approach is much more effective in the multiple latent entity extraction (Table 4) compared to the single latent entity extraction case (Table 2).

multi-task learning approach along with task-grouping performs much better than the other baselines.


In this paper, we presented a new task of latent entities extraction from text, which gives a new insight over the original named-entity recognition task.

we focus on how to extract an entity when it is not explicitly mentioned in the text, but rather implicitly mentioned in the context.


Document keywords and keyphrases enable faster and more accurate search in large text collections, serve as condensed document summaries, and are used for various other applications, such as categorization of documents.

keyphrase extraction is a crucial component when gleaning real-time insights from large amounts of Web and social media data.


In the document from Figure 1, the extracted keyphrases include {topological shape, topological shapes} and {molecular equivalence number, molecular equivalence numbers, molecular equivalence indices}.

four out of the ten keyphrase "slots" are taken by redundant phrases.


In this paper we take a step back from the differences between MDS and TLS and consider the following question: Can MDS optimization models be expanded to yield scalable, well-performing TLS models that take into account the temporal properties of TLS, while keeping MDS advantages such as modularity and performance guarantees?

we make the following contributions: • We adapt the submodular function model of Lin and Bilmes (2011) to TLS (Section 3).


In MDS, typical constraints are upper bounds in the number of sentences or words, corresponding to cardinality (|S| ≤ m) or knapsack constraints ( s∈S |words(s)| ≤ m) for some upper bound m. When optimizing a submodular monotone function under such constraints, GREEDY has a performance guarantee of ≈ 0.63 and ≈ 0.39 respectively (Calinescu et al., 2011;Lin and Bilmes, 2011).

for cardinality constraints, the output is at least 0.63 as good as the optimal solution in terms of objective function value.


Criticism towards traditional distributional models and the benchmarks used to evaluate them (Batchkarov et al., 2016) are now compelling more researchers to consider evaluation techniques that analyse how well distributional models encode different aspects of grounded meaning (Lucy and Gauthier, 2017;Collell and Moens, 2016;Gladkova and Drozd, 2016).

one aspect of cognitive plausibility that is lacking in dense representations is in their interpretability, something that could be solved using sparsity (Faruqui et al., 2015;Senel et al., 2017).


In this paper, we combine both text and image-based data in conjunction with matrix factorisation strategies to build sparse and multimodal distributional models, with the goal of demonstrating that these models are more interpretable with respect to human semantic knowledge about concepts.

we show that these models attain a structural composition and semantic representation that is closer to the way humans represent concepts, evaluated using human similarity judgements, human semantic feature knowledge, and neuroimaging data.


Word2Vec (Mikolov et al., 2013) uses shallow neural networks with negative sampling techniques, which are trained to predict either the word from the context or the context from the word using a fixed window of words as the context.

we choose the CBOW version (predict the word using the context) of this model which was trained using the gensim package with the minimum word count threshold set to 0 (i.e., a vector representation was created for all words in the corpus).


However, a stronger test of how closely an embedding model relates to human-elicited conceptual knowledge is to investigate whether the embedding dimensions encode interpretable, human-like semantic properties directly.

does an embedding model learn a set of basis vectors for the semantic space that corresponds to verbalisable, human semantic properties like is-round, is-a-fruit, and so on?


It can be easily verified that W −0.25 , as computed over X, would recover back the original embeddings, that is, M (XW −0.25 ) = M (Z).

assuming that the embeddings X capture some second order similarity, it is possible to transform them so that they capture the corresponding first order similarity, and one can easily generalize this to higher order similarities by simply using smaller values of α.


Here we describe a novel framework that uses a simple reference game to study the semantic resources speakers and listeners use to facilitate reference.

placing strong constraints on word choice and modes of interaction allows us to better isolate specific aspects that contribute towards the complexity of natural language semantics.


Language provides a multitude of different resources for its users to cooperatively achieve reference.

language provides truthconditional semantic structures.


Learning attention functions requires large volumes of data, but many NLP tasks simulate human behavior, and in this paper, we show that human attention really does provide a good inductive bias on many attention functions in NLP.

we use estimated human attention derived from eyetracking corpora to regularize attention functions in recurrent neural networks.


Behind our approach lies the simple observation that we can correlate the token-level attention devoted by a recurrent neural network, even if trained on sentence-level signals, with any measure defined at the token level.

we can compare the attention devoted by a recurrent neural network to various measures, including token-level annotation (Rei and Søgaard, 2018) and eye-tracking measures.


We then compute the minimum squared error between the normalized eye-tracking measure and the normalized attention score.

in multi-task learning, we optimize each task for a fixed number of parameter updates (or mini-batches) before switching to the next task (Dong et al., 2015); in our case, we optimize for a target task (for a fixed number of updates), then improve our attention function based on human attention (for a fixed number of updates), then return to optimizing for the target task and continue iterating.


We therefore define an epoch to comprise a fixed number of batches, and sample every batch of training examples either from the target task data or from the eye-tracking corpus, as determined by a coin flip, the bias of which is tuned as a hyperparameter.

we define an epoch to consist of n batches, where n is the number of training sentences in the target task data divided by the batch size.


Overall, this, in combination with our results in Table 3, suggests that the regularization effect from human attention enables our architecture to learn to better attend to the most relevant aspects of sentences for the target tasks.

human attention provides the inductive bias that makes learning possible.


We did not experiment with other representations, however.

we only considered the signal from token-level, normalized mean fixation durations.


Evaluating sentences on their fluency, on the other hand, is a linguistic ability of humans which has been the subject of a decade-long debate in cognitive science.

the question has been raised whether the grammatical knowledge that underlies this ability is probabilistic or categorical in nature (Chomsky, 1957;Manning, 2003;Sprouse, 2007).


Specifically, we test our hypothesis that SLOR should be a suitable metric for evaluation of compression fluency which (i) does not rely on references; (ii) can naturally be applied at the sentence level (in contrast to the system level); and (iii) does not need human fluency annotations of any kind.

the first aspect, i.e., SLOR not needing references, makes it a promising candidate for automatic evaluation.


(2017) aimed at modeling human acceptability judgments automatically, with the goal to gain insight into the nature of human perception of acceptability.

they tried to answer the question: Do humans judge acceptability on a gradient scale?


Comparing the metrics, the two SLOR approaches perform best for SEQ2SEQ and T3.

they outperform the best word-overlap metric baseline by 0.244 and 0.097 Pearson correlation as well as 0.012 and 0.012 MSE, respectively.


Second, we simulate a setting in which we have only access to (i) system outputs which should be evaluated on fluency, (ii) reference compressions, and (iii) large amounts of unlabeled text.

we assume to not have fluency ratings for system outputs, which makes training a regression model impossible.


3 In this example, we experiment with increasing dimensions in the recurrent layer while maintaining the number of trainable parameters, whereas in Section 4.2 we increase sparseness while maintaining dimensions.

each LSTM layer is made sparse in such a way that the hidden dimension 1150 is increased by a factor 1.5 (chosen ad hoc) to 1725, but the embedding dimensions and total number of parameters remain the same (within error margins from rounding to integer dimensions for the dense blocks).


An exponential decay in the number of words that are assigned longer representations is one possible way to implement this.

we propose to have the number of words that receive a trainable parameter at dimension j decrease with a factor α j (α ∈ ]0, 1]).


Then we produce a final ranking by combining these rankings.

we sum the ranking of each sentence according to each policy to get a rank score, and re-rank the sentences according to their rank score.


In the case of providing a text line as input, we are essentially providing more context about the input in comparison to the word level models and the RNN (or LSTM) cells are powerful enough to capture the long-term dependencies.

for Indian languages, this decision is beyond a question of performance.


The above objective can be thus reduced to: where the first term computes the reconstruction likelihood p(x|y); and the second term is the entropy of the approximated posterior q(y|x) for regularization.

we use the semantic parser to compute the approximated posterior q(y|x).


MARMOT approximates the CRF objective using a pruning strategy which enables training higher-order models and handling large tagsets.

the tagger first predicts the POS part of the label and based on that, constrains the set of possible morphological labels.


We are going to focus on those features for which relevant data is available, and there's reason to think they could be captured in lexical (semantic) vectors because they are properties of words (in contrast to the more discourse-oriented features, such as +Top.)

we focus on lexical restriction, number and animacy in the definition of intervention similarity.


The lack of correlation prompts a more detailed analysis of the results.

notice that in the experimental work a binary (not continuous) distinction -animate vs. inanimate, plural vs. singular -was manipulated and correlated to the acceptability and re-action times.


We are, instead, requiring a correlation between similarity and acceptability in the animacy case and similarity and number in the reaction times.

we are imposing a stricter correspondence, which requires the level of similarity to continuously vary with all the experimental results.


To fit the fully unsupervised learning paradigm, we do not introduce any biases into our sampling of words in training our model.

we excluded approaches that overweighted adjectives or speaker identification (e.g "said X on Tuesday") in noising.


In typical RNN encoderdecoder architectures, the final hidden state of the encoder is used as the initial hidden state of the decoder.


However, it is necessary to study the effect of the second parameter as well, in order to quantify the applicability of ngrams for embedding comprehensively.

we want to compare the effects of both parameters; we will be able to give recommendations for both parameters.


Leveraging known Pinyin linguistic characteristics can improve the accuracy of our model and reduce the size of the annotation task.

this is done by grouping the Pinyin components into initial clusters according to the Pinyin pronunciation tables (ISO, 2015) and only annotating the pairs within each cluster along with a single pairwise distance between clusters.


However, we observe that the phonetic similarity of Pinyin is greatly impacted by the pronunciation methods and the place of articulation -this allows us to improve the accuracy and simplify the annotation task.

this is done by grouping Pinyin components into initial clusters and only annotating pairs within each cluster, and representative cluster pairs.


Figures 3 and 4 visualize the computed encodings of initials when setting n=1 and n=2 We see that when n = 2, the locations of initial coordinates align well with Table 2,.

the twelve groups are clustered in a pattern that is defined in Section 2.3.1.


However, while Table 2 indicated the basic clusters for the initials, our learned model goes further than Table 2 by actually quantifying the inter-and intra-cluster similarities.

clusters "c, ch, j, q, x" are tighter than clusters "c, c, h" and "d, t", whereas the clusters "m" and "n, r, l" are well separated from other clusters.


So, if not persuasive arguments, what makes a news editorial effective or ineffective then?

: How can we measure its argumentation quality?


To avoid newspaper-related side effects in the study of argumentation quality, we decided to control this dimension by annotating news editorials from one source only.

we resort to the online portal of the New York Times for two practical reasons: (1) The political typology quiz is tailored to people from the United States.


As a proxy, we therefore decided to model the reader's prior beliefs by identifying the reader's political ideology.

we profile the reader as being liberal or conservative based on the nine groups of the political typology developed by the PEW Research Center.


As mentioned before, we decided to restrict our study to editorials from a single news portal (The New York Times), in order to exclude the portal impact on the quality assessment.

we used the New York Times Annotated Corpus (Sandhaus, 2008), which comprises around 1.8 million news articles written between 1987 and 2007.


Our analysis of the corpus provided first insights: As expected, readers with identical beliefs largely agree on the effect of editorials.

we provide empirical evidence for the hypothesis that editorials rarely change the stance of readers.


The recognition of the utility of corpus-derived distributed representations of words for a broad range of Natural Language Processing (NLP) tasks (Collobert et al., 2011) has led to a resurgence of interest in methods of distributional semantics.

the neural-probabilistic word representations produced by the Skip-gram and Continuous Bag-of-Words (CBOW) architectures (Mikolov et al., 2013a) implemented in the word2vec and fastText software packages have been extensively evaluated in recent years.


Neural-probabilistic methods have shown utility as a means to generate semantic vector representations of words (Bengio et al., 2003).

the Skip-gram and CBOW neural network architectures (Mikolov et al., 2013a) implemented within the word2vec and fastText software packages provide scalable approaches to online training of word vector representations that have been shown to perform well across a number of tasks (Mikolov et al., 2013a;Levy et al., 2015;.


Consequently, we also generated n-gram based variants of each of our models, by adapting the approach described in  to our SGNS configuration.

we decomposed each word into character n-grams, after introducing characters indicating the start (<) and end (>) of a word.


We did not engage in tuning of these hyperparameters to improve performance of our models, but rather were guided by prior research in selecting hyper-parameter settings that are known to perform well with the SGNS baseline model.

we used a subsampling threshold t of 10 −5 .


We report average accuracy, where a result is considered accurate if d is the top-ranked result aside from a, b and c. 7 To evaluate the effects of encoding word order on the relative distance between terms, we used a series of widely used reference sets that mediate comparison between human and machine estimates of pairwise similarity and relatedness between term pairs.

we used Wordsim-353 (Finkelstein et al., 2001), split into subsets emphasizing similarity and relatedness (Agirre et al., 2009); MEN (Bruni et al., 2014) and Simlex-999 (Hill et al., 2015).


These improvements in analogy retrieval were not accompanied by better correlation with human es-   Results of these experiments are shown in Table 2, and suggest an advantage for models encoding position in sequence labeling tasks.

for the sentence chunking shared task (CoNLL00), the best results obtained with an order-aware model (EARPx pos + subwords) exceeds the best baseline result by around 2%, with smaller improvements in performance on the other two tasks, including a .43% improvement in accuracy for part-of-speech tagging (PTB, without subwords) that is comparable to the .37% improvement over a SGNS baseline reported on this dataset by Ling and his colleagues (2015) when using separate (rather than shared) positiondependent context weights.


In summary, we focus on entity disambiguation by leveraging only the local information.

we propose using both representationfocused model and interaction-focused model for semantic matching and view them as complementary to each other.


In next section, we will introduce our proposed aggregated semantic matching for entity disambiguation.

given a tweet t with a sequence of words w 1 , ..., w n , our goal is to identify the possible entity mentions in the tweet t. every word w i in tweet t requires a label to indicate that whether it is an entity mention word or not.