Error Analysis Like POS-taggers, the learned supertagger frequently confuses nouns (N) and their modifiers (N/N), but the most frequent error made by the English (6) experiment was (((S\NP)\(S\NP))/N) instead of (NP nb /N).

these are both determiner types, indicating an interesting problem for the supertagger: it often predicts an object type-raised determiner instead of the vanilla NP/N, but in many contexts, both categories are equally valid.


In this work, as in most type-supervised work, the tag dictionary was automatically extracted from an existing tagged corpus.

a tag dictionary could instead be automatically induced via multi-lingual transfer (Das and Petrov, 2011) or generalized from human-provided information .


In step 1, as the jump factor 1 is dropped, we do not know the orientation between bǎ and tā.

several jump distances are known: from X 1 to bǎ is distance -2 and tā to kǎolv jìnqù is 2.


Accordingly our model supports rules than cannot be represented by a 2-SCFG (e.g., step 3 in Figure 5 requires a 4-SCFG).

the hierarchical phrase-based model allows only 2-SCFG as each production can rewrite as a maximum of two nonterminals.


It makes use of the decoding mechanism of the phrase-based model which jumps over the source words and hence can hold discontinuous phrases naturally.

their method doesn't touch the correlations between phrases and probability modeling which are the key points we focus on.


The jump factor gives big improvements of about 1% BLEU in both language pairs.

when using parallel backoff, the performance improves greatly for Chinese-English but degrades slightly on Arabic-English.


Our approach uses minimal phrases as its basic unit of translation, in order to preserve the manyto-many links found from the word alignments.

we now seek to assess the impact of the choice of these basic units, considering instead a simpler word-based setting which retains only 1to-1 links in a Markov model.


We could easily use larger phrase pairs as the basic unit, such as the phrases used during decoding.

doing this involves a hard segmentation and would exacerbate issues of data sparsity.


There are about 83 million unique phrases up to length three in the English Wikipedia.

we ignore target phrases that appear fewer than three times, reducing this set to 10 million English phrases.


The phrase was no one is composeable from había nadie given the seed model.

the phrase polling stations is composeable from centros electorales using induced translations.


The small model fails to align the adjoined clitic los with its translation them.

our loose definition of compositionality allows the English stop word them to appear anywhere in the target translation.


In the decipherment task, translation models are learned from comparable corpora without any parallel text (Ravi and Knight, 2011;Dou and Knight, 2012;Ravi, 2013).

we begin with a small amount of parallel data and take a very different approach to learning translation models.


In fact, both representations achieve the same accuracy on the SEMEVAL task.

there is a large performance gap in favor of the neural embedding in the open-vocabulary MSR and GOOGLE tasks.


Previous approaches use taskspecific information, by either relying on a (word-pair, connectives) matrix rather than the standard (word, context) matrix (Turney and Littman, 2005;Turney, 2006), or by treating analogy detection as a supervised learning task (Baroni and Lenci, 2009;Jurgens et al., 2012;Turney, 2013).

the vector arithmetic approach followed here is unsupervised, and works on a generic single-word representation.


This choice is inspired by recent work on learning syntactic categories (Yatbaz et al., 2012), which successfully utilized such language models to represent word window contexts of target words.

we note that other richer types of language models, such as class-based (Brown et al., 1992) or hybrid (Tan et al., 2012), can be seamlessly integrated into our scheme.


This would discriminate better between like and surround.

in this case sentences such as "Mary's son likes the school campus" and "John's son loves the school campus" will not provide any evidence to the similarity between like and love, since "Mary's son the school campus" is a different feature than "John's son the school campus".


The bottom-up hypothesis holds that infants converge onto the linguistic units of their language through a statistical analysis over of their input.

the top-down hypothesis emphasizes the role of higher levels of linguistic structure in learning the lower level units.


A possible reason for this is that some error types are more regular than others, and so in order to boost accuracy, simple rules can be written to make sure that, for example, the number of a subject agrees with the number of a verb.

it is a lot harder to write a rule to consistently correct Wci (wrong collocation/idiom) errors.


Finally, the inability of the M 2 Scorer to combine corrections from different annotators (as opposed to selecting only one annotator's corrections for the whole sentence) can also result in underestimations of performance.

it is clear that exploring these combinations during evaluation is a challenging task itself.


Tuning three to five times would require 24 to 40 tuning runs in our setup.

we already have eight parameter vectors obtained from distinct tuning sets and decide to average these parameters.


Our result with additional answers is 38.58%, we remain on third place after CUUI (45.57%) and CAMB (43.55%) which switched places.

we do not consider the evaluation on alternative answers to be meaningful as it is strongly biased.


Preposition (Prep) 5.43% I do not agree *on/with this argument that surveillance technology should not be used to track people .

word form (wform) 4.87% , the application of surveillance technology serves as a warning to the *murders/murderers and they might not commit more murder .


This correctiom improves the local coherency of the sentence.

the resulting construction is not consistent with "to almost luxury", suggesting a more complex correction (changing the word "become" to "are going").


Detection of errors can be achieved statistically and rule-based, the task of the hybrid approach being to resolve any conflicts that arise between the outputs of the two approaches.

even the most advanced systems are only able to distinguish between a limited number of error types and the task of correcting an error is even more difficult.


We adopted the former solution: we used Google 1T n-grams corpus (see section 2.2) from which the selectional restrictions can be learned quite successfully.

dealing with collocations is difficult, as correction does not involve only syntax, but also semantics.


The article by Adam Pauls and Dan Klein (2011) describes an ingenious way to create a data structure that reduces the amount of RAM needed to load the 1T corpus.

the system they propose is written in Java, a language that is objectoriented, and which, for any object, introduces an additional overhead.


The result of the system was not good or as we expected, first because our approach is simple and was motivated to test the use of a syntactic n-grams language model, second because the poor election of candidates to correct the errors.

this task gave us the opportunity to test the behaviour in different conditions and now we have a reference to improve our system.


But this method still needs a candidate generation mechanism for each error category.

the SMT based method (Brockett et al., 2006) formulates the grammar correction problem as a problem of translation of incorrect sentences to correct sentences.


Tuning to BLEU ensures that the parameter weights are set such that the fidelity of translations is high.

ensuring fidelity is not the major challenge in grammar correction since the meaning of most input sentences is clear and most don't have any grammatical errors.


Comparing the results of systems S1, S3 and S5, it is clear that using the SMT method alone gives the highest F-0.5 score.

the recall is higher for systems which use the custom modules for some error categories.


Tuning the SMT system to the F-β metric did not improve performance over the BLEU-based tuning.

we plan to further investigate to understand the reasons for this behaviour.


We use the Enchant Python Library to correct the spelling errors 1 .

using only one best result is not very accurate.


For example, in , the determinater errors are decomposed into: For a task with a few error types such as merely determinative and preposition error in HOO 2012, manually decomposition may be sufficient.

for CoNLL-2014, all 28 error types are required to be corrected and some of these types such as Rloc-(Local redundancy) and Um (Unclear meaning) are quite complex that the manual decomposition is time consuming and requires lots of grammatical knowledges.


In addition, dividing the error detection and correction into two steps alleviates the application of machine learning classifiers.

an approach that considers error types individually may have negative effects:  This approach assumes independence between each error type.


We summarize our conclusions in Section 5.

with phrase-based translation models, factored models make use of additional linguistic clues to guide the system such that it generates translated sentences in which morphological and syntactic constraints are met .


Even with these additions and transformations, there are still too few training data available to train a good translation model.

we think the grammar error correction system should 1) correct most kinds of errors in a unified framework and 2) use as much unlabeled data as possible instead of using large amount of human annotated data.


We select feedback of varying grade by directly inspecting the optimal w * , thus this feedback is idealized.

the experiment also has a realistic background since we show that α-informative feedback corresponds to improvements under standard evaluation metrics such as lowercased and tokenized TER, and that learning from weak and strong feedback leads to convergence in TER on test data.


In coreference resolution, a fair amount of research treats mention detection as a preprocessed step and focuses on developing algorithms for clustering coreferred mentions.

there are significant gaps between the performance on gold mentions and the performance on the real problem, when mentions are predicted from raw text via an imperfect Mention Detection (MD) module.


Standard methods define mentions as boundaries of text, and expect exact boundaries as input in the coreference step.

mentions have an intrinsic structure, in which mention heads carry the crucial information.


16 The mention head candidate generation module has a bigger impact on MDER compared to the joint framework.

they both have the same level of positive effects on PGR for coreference resolution.


Cardie and Pierce (1998) propose to select certain rules based on a given corpus, to identify base noun phrases.

the phrases detected are not necessarily mentions that we need to discover.


CCM is able to learn which POS contexts are likely, and does so via a probabilistic generative model, providing a statistical, data-driven take on substitutability.

since there is nothing intrinsic about the POS pair DET-VERB that indicates a priori that it is a likely constituent context, this fact must be inferred entirely from the data.


(2007) to sample parse trees for sentences in the raw training corpus according to their posterior probabilities.

due to the very large sets of potential supertags used in a parse, computing inside charts is intractable, so we design a Metropolis-Hastings step that allows us to sample efficiently from the correct posterior.


MSA is used in formal settings, edited media, and education.

the spoken, and, currently written in social media and penetrating formal media, are the informal vernaculars.


The misclassified word in the first examples (bED meaning "each other") has a gold class other.

the gold label is incorrect and our system predicted it correctly as lang2 given the context.


In the 10-fold-cross-validation on the sentTrnDB using Comp-Cl alone, we note that performance results slightly decreased (from 87.3% to 87.0%).

given the sparsity of the feature (it occurs in less than 1% of the tokens in the EDA sentences), 0.3% drop in performance is significant.


Training models from scratch for every new domain requires human annotated labeled data which is expensive and time consuming, hence, not pragmatic.

transfer learning techniques allow domains, tasks, and distributions used in training and testing to be different, but related.


If θ 1 is low, C t may get trained on incorrectly predicted pseudo labeled instances; whereas, if θ 1 is high, C t may be deficient of instances to learn a good decision boundary.

θ 2 influences the number of iterations required by the algorithm to reach the  Figure 4: Illustrates how the weight (w t ) for target domain classifiers varies for the most and least similar domains with number of iterations.


SCL based domain adaptation does not yields generous improvements as selecting the pivot features and computing the cooccurrence statistics with noisy short text is arduous and inept.

the proposed algorithm iteratively learns discriminative target specific features from such perplexing data and translates it to an improvement of at least 6.4% and 3.5% over the baseline and the SCL respec- Figure 7: Results comparing the accuracy of the proposed approach with existing techniques for cross domain categorization on the real world dataset.


We observed that MERT learns the verbosity of the tuning dataset very well, but this can be a disadvantage because we do not know the verbosity of unseen test sentences.

pRO is affected by both the verbosity and the source-side length of the tuning dataset.


For each word that appeared in the dependency relation arcs, we can use the number of appearances of its interlingual features as the corresponding feature values.

the sentences in the target language are not fully labeled.


That is, it estimates the probability of each w given its local context.

sKIP-GRAM applies softmax to each context word of a given occurrence of word w. In this case, v ctx(w) corresponds to the representation of one of its context words.


If we treat word representations as fixed, the graph transformer is a simple linear-chain CRF.

if we can treat the word representations as model parameters, the model is equivalent to a neural network with word embeddings as the input layer, as shown in Figure 1.


Another difference is that we tuned the hyperparameters with random search, to enable replication using the same random seed.

the hyperparameters for the state-of-the-art methods are tuned more extensively by experts, making them more difficult to reproduce.


This paper addresses the issue systematically on a large scale.

to previous work in both sociolinguistics and NLP, we consider syntactic variation across groups at the level of treelets, as defined by dependency struc-tures, and make use of a large corpus that includes demographic information on both age and gender.


In principle, we could directly check for significant differences in the demographic groups and use Bonferroni correction to control the family-wise error (i.e., the probability of obtaining a false positive).

given the large number of treelets, the correction for multiple comparisons would underpower our analyses and potentially cause us to miss many significant differences.


This approach restricts the findings to the phenomena defined in the hypothesis, in this case the word list used.

our approach works beyond the lexical level, is data-driven and thus unconstrained by prior hypotheses.


Existing studies on variation have thus mostly focused on lexical and phonological variation.

we study the effect of age and gender on syntactic variation across several languages.


Although this avoids the need for a target language treebank, most approaches have still used large parallel corpora.

parallel data is scarce for low-resource languages, and we report a new method that does not need parallel data.


Supervised approaches to dependency parsing have been very successful for many resource-rich languages, where relatively large treebanks are available (McDonald et al., 2005a).

for many languages, annotated treebanks are not available, and are very costly to create (Böhmová et al., 2001).


Given a source-language parse tree along with word alignments, they generate the targetlanguage parse tree by projection.

their approach relies on many heuristics which would be difficult to adapt to other languages.


In summary, existing work generally starts with a delexicalized parser, and uses parallel data typological information to improve it.

we want to improve the delexicalized parser, but without using parallel data or any explicit linguistic resources.


The experimental results presented in Figure 2 correspond to these findings: we can see improvements in the neural network performance when increasing the word embeddings dimensionality from 50 to 100 and from 100 to 200.

the Ask Ubuntu data set containing approximately 121M tokens is not big enough for an improvement when increasing the dimensionality from 200 to 400.


The performance of an SVM-based approach to this task was shown to depend highly on the size of the training data.

the CNN with in-domain word embeddings provides very high performance even with limited training data.


Because |C| is the number of hidden-to-hidden connection types, we cannot apply caching to reduce this term.

|C| is much smaller than |F| (here |C|=3).


The structure follows the proposed pipeline approach by Klinger and Cimiano (2013).

1 in contrast to their work, we focus on the detection of phrases only, and exploit the detection of relations only during inference, such that the detection of relations has an effect on the detection of phrases, but is not evaluated directly.


We use exact match precision, recall and F 1 -measure for evaluation.

it should be noted that partial matching scores are also commonly applied in fine-grained sentiment analysis due to the fact that boundaries of annotations can differ substantially between annotators.


Our approach relying on instance filtering comes close to results produced by a system trained on manually annotated data for the target language on the task of predicting aspect phrases.

to these results for the aspect phrase recognition, the impact of filtering training instances is negligible for the detection of subjective phrases.


This can as well be observed in the number of predictions the models based on different thresholds generate: While the number of true positive aspects for the coffee machine subdomain is 1100, only 221 are predicted with a threshold of the manual quality assignment of 0.

a treshold of 9 leads to 560 predictions and a threshold of 10 to 1291.


For the fusional languages (English, German and Indonesian) we see modest gains in performance on both root detection and stemming.

for the agglutinative languages (Finnish, Turkish and Zulu) we see absolute gains as high as 50% 10 Thus in our experiments there are no stem alternations.


Finite-state morphological analyzers output a small set of linguistically valid analyses of a type, typically with only limited overgeneration.


14 In fact, this is usually solved by viewing it as a different problem, morphological guessing, where linguistic knowledge similar to the features we have presented is used to try to guess POS and morphological analysis for types with no analysis.

our training procedure learns a probabilistic transducer, which is a soft version of the type of hand-engineered grammar that is used in finite-state analyzers.


One may consider connecting multiple resource graphs at the term nodes.


The generative models (CSLDA and MOMRESP) tend to excel in lowannotation portions of the learning curve, partially because generative models tend to converge quickly and partially because generative models naturally learn from unlabeled documents (i.e., semi-supervision).

mOmRESP tends to quickly reach a performance plateau after which additional annotations do little good.


When data clusters and label classes are misaligned, MOMRESP falters (as in the case of the Cade12 dataset).

cSLDA's flexible mapping from topics to labels is less sensitive: topics can diverge from label classes so long as there exists some linear transformation from the topics to the labels.


(2014) adapted CBOW (Mikolov et al., 2013a) to train word embeddings on different datasets: free text documents from Wikipedia, search click-through data and user query data, showing that combining them gets stronger results than using individual word embeddings in web search ranking and word similarity task.

these two papers either learned word representations on the same corpus (Turian et al., 2010) or enhanced the embedding quality by extending training corpora, not learning algorithms (Luo et al., 2014).


5 The table also shows that WordNet may not be appropriate for our present verb categorization task.

it may be suitable for other subtasks in sentiment analysis, particularly polarity classification.


The relatively strong naming performance of G can be attributed to the fact that many demonstrations had similarities among the objects presented that could be learned from choosing any of the objects.

reference resolution performance for G averaged a .34 F1-score compared with a .70 F1-score for our best performing configuration.


Indeed, when evaluating these VSMs with datasets such as wordsim353 (Finkelstein et al., 2001), where the word pair scores re-flect association rather than similarity (and therefore the (cup,coffee) pair is scored higher than the (car,train) pair), the Spearman correlation between their scores and the human scores often crosses the 0.7 level.

when evaluating with datasets such as SimLex999 (Hill et al., 2014), where the pair scores reflect similarity, the correlation of these models with human judgment is below 0.5 (Section 6).


For example, in word classification tasks, words such as "big" and "small" potentially belong to the same class (size adjectives), and thus representing them as similar is desired.

antonyms are very dissimilar by definition.


The table shows that when the antonym parameter is off, our model generally recognizes antonyms as similar.

when the parameter is on, ranks of antonyms substantially decrease.


The feature vector g n ∈ R 2d×1 is the concatenation of N(n 1 ) and N(n 2 ): Words between the noun pair contribute to classifying the relation, and one of the most common ways to incorporate an arbitrary number of words is treating them as a bag of words.

word order information is lost for bag-of-words features such as averaged word embeddings.


The main advantage of such models is that they allow a large family of rich features that include dependency features, constituent features and conjunctions of the two.

the consequence is that the additional spinal structure greatly increases the number of dependency relations.


In this case, to produce a constituent tree from the spinal forest, we promote the last tree and place the rest of trees as children of its top node.

in this section we describe the performance of the transition-based spinal parser by running it with different sizes of the beam and by comparing it 2 this is not always the case.


However, unlike , the arc-eager parser does not substantially benefit of using the triplets during training.

our best model (obtained with beam=64) provides 92.14 UAS, 90.91 TA and 89.32 TAS in the test set including punctuation and 92.78 UAS, 91.53 6 in absolute terms, our running times are slower than typical shift-reduce parsers.


In that work dependency labels encode the constituent node where the dependency arises as well as the position index of that node in the head spine.

we use constituent triplets as dependency labels.


The Accuracy column shows the cross-validation accuracy, and the Diff column shows the improvement over the majority class baseline.

this is partly due to the increased class imbalance of trolls vs. non-trolls, which can be seen by the decrease in the improvement of our classifier compared to the majority class baseline.


Feature selection techniques for traditional documents have been aplenty and a few seminal survey articles have been written on this topic (Blitzer, 2008).

for short text there is much less work on statistical feature selection but more focus has gone to feature engineering towards word normalization, canonicalization etc.


In supervised learning, we can estimate the accuracy of a model on a subset of the labeled data and choose the model with the highest accuracy.

here we focus on type-supervised learning, which uses constraints over the possible labels for word types for supervision, and labeled data is either not available or very small.


Fully supervised training of NLP models (e.g., part-of-speech taggers, named entity recognizers, relation extractors) works well when plenty of labeled examples are available.

manually labeled corpora are expensive to construct in many languages and domains, whereas an alternative, if weaker, supervision is often readily available.


Using the 300 labeled sentences for semi-supervised training and model selection reduced the error by 44.6% (comparing to the model with best average accuracy using only type-level supervision with average performance of 85.05, the semi-supervised average is 91.8).

using the 300 sentences to select hyperparameters only reduced the error by less than 5% (the average accuracy was 85.75).


Some researchers believe that, in some cases, WSI methods may perform better than WSD systems (Jurgens and Klapaftis, 2013;Wang et al., 2015).

we argue that WSI systems have few advantages compared to WSD methods and according to our results, disambiguation systems consistently outperform induction systems.


We found that only 4% of errors are caused by wrong sentence or word alignment.

69% of erroneous sense-tagged instances are the result of a Chinese word associated with multiple senses of a target English word.


We notice that the DSO corpus generally improves the performance of our system.

since the annotated DSO corpus is copyrighted, we are unable to release a dataset including the DSO corpus.


Since the MUN dataset does not cover all target word types in the all-words shared tasks, the accuracy achieved with MUN alone is lower than the SC and SC+DSO settings.

the evaluation results show that IMS trained on MUN alone often performs better than or is competitive with the WordNet Sense 1 baseline.


assumptions, and makes it capable of modeling complex distributions over sequences, including those with long-term dependencies.

by breaking the model structure down into a series of next-step predictions, the rnnlm does not expose an interpretable representation of global features like topic or of high-level syntactic properties.


When labeled by SU-Time 1 or HeidelTime 2 , the adverb currently is correctly tagged with the PRESENT_REF value.

if we change the sentence to "Apple's iPhone is one of the most popular smartphones at the present day", no temporal mention is found, although one may expect that within this context currently and present day share some equivalent temporal dimension.


Re-sults show that TRMC2 outperforms all other approaches and achieves highest performance in terms of precision, recall, and F1-measure.

more important still is the fact that a simple learning strategy with some temporal lexicon (MC2 or TWnH) leads to improved results, when compared to some solution that does not take advantage of such a resource (UTTime, here).


For example, if we know the place of birth and/or the place of death of a person and/or the location where the person is living, it is likely  that we can predict the person's nationality.

if we know that a person works for an organization and that this person is also the top member of that organization, then it is possible that this person is the CEO of that organization.


Unlike the case of single-word contexts, it is not feasible to explicitly compute here this PMI matrix due to the exponential number of possible sentential contexts.

the objective function that we optimize still aims to best approximate it.


We have investigated the impact of corrective feedback on first language acquisition, in particular on the reduction of subject omission errors in English-a type of error which we found to be the most commonly met with CF in our corpus study.

to previous small-scale studies in psycholinguistics, we have addressed this problem using a comparatively large data-driven setting.


A similar project by Labutov and Lipson (2014) likewise considers the effect of context on guessing the L2 word.

it does not consider the effect of the L2 word's spelling, which we show is also important.


We train a log-linear model to predict the words that our subjects guess on training data, and we will check its success at this on test data.

from an engineering perspective, we do not actually need to predict the user's specific good or bad answers, but only whether they are good or bad.


As the empirical guessability increases, so does the median model probability assigned to the correct answer.

in our applications, we are less interested in only the 1-best prediction; we'd like to know whether users can understand the novel vocabulary, so we'd prefer to allow WordNet synonyms to also be counted as correct.


The URLL distance between this tree and the gold standard in Figure 1 is 0.12.

10 the MDL costs do not allow us to prefer any one of the context models over the others.


When classification techniques are used, we obtain the best F-score of 79.8% with SVM (O).

when sequence labeling techniques are used, the best F-score is 84.2%.


This means that our intuition that sequence labeling will better capture conversational context reflects in the forms of sarcasm for which sequence labeling improves over classification.

examples where our system makes errors can be grouped as: • Topic Drift: Eisterhold et al.


(2016) for modeling sarcasm understandability of readers.

as far as we know, these features are being introduced in NLP tasks like sentiment analysis for the first time.


Recent work in sarcasm detection on social media has tried to incorporate contextual information by exploiting the preceding messages of a user, to e.g., detect contrasts in sentiments expressed towards named entities (Khattri et al., 2015), infer behavioural traits (Rajadesingan et al., 2015) and capture the relationship between authors and the audience (Bamman and Smith, 2015).

all of these approaches require the design and implementation of complex features that explicitly encode the content and (relevant) context of messages to be classified.


To train our neural model, we first had to choose a suitable architecture and hyperparameter set.

selecting the optimal network parametrization would require an extensive search over a large configuration space.


Clearly projections contain useful information, as the tagging accuracy is well above chance.

they are riddled with noise and biases, which need to be accounted for to improve performance.


As a preprocessing step, pseudo-projectivization of the syntactic trees (Nivre et al., 2007) was used, which allowed an accurate conversion of even the non-projective syntactic trees into syntactic transitions.

the oracle conversion of semantic parses into transitions is not perfect despite using the M-SWAP action, due to the presence of multiple crossing arcs.


This setup bears similarity to other approaches which pipeline syntax and semantics, extracting features from the syntactic parse to help SRL.

unlike other approaches, this model does not offer the entire syntactic tree for feature extraction, since only the partial syntactic structures present on the syntactic stack (and potentially the buffer) are visible at a given timestep.


(2015) proposed an efficient joint model for CCG syntax and SRL, which performs better than a pipelined model.

their training necessitates CCG annotation, ours does not.


This can be a reasonable assumption in text or speech transcription (Toselli et al., 2007;Rodríguez et al., 2007) where the output sequence is generated monotonically respect to the input data.

it has always be an important handicap for translation due to the intrinsic reordering involved in the process.


Obviously, it does not qualify as real-time.

we expect an important reduction in response time after implementing our approach in a more efficient language.


This reduction in typing effort came at the expense of a larger amount of mouse actions required to validate correct segments of the suggested translations.

since mouse actions are cheaper than typing full words, we can expect this exchange to reduce overall user effort.


Traditionally, these models provide point estimates and are evaluated using metrics like Mean Absolute Error (MAE), Root-Mean-Square Error (RMSE) and Pearson's r correlation coefficient.

in practice QE models are built for use in decision making in large workflows involving Machine Translation (MT).


Most kernel methods tie all lengthscale to a single value, resulting in an isotropic kernel.

since in GPs hyperparameter optimisation can be done efficiently, it is common to employ one lengthscale per feature, a method called Automatic Relevance Determination (ARD).


Intuitively, if two models produce equally incorrect predictions but they have different uncertainty estimates, NLPD will penalise the overconfident model more than the underconfident one.

if predictions are close to the true value then NLPD will penalise the underconfident model instead.


That is, a named entity recognizer is first applied to identify mentions of interest, and then a wikifier is used to ground the extracted mentions to Wikipedia entries.

to this traditional pipeline, we show that the ability to ground and disambiguate words is very useful to NER.


They do NER using Wikipedia category features for each mention.

their method for wikifying text is not robust to ambiguity, and they only do monolingual NER.


If it fails, we find the corresponding English Wikipedia title via interlanguage links and then query the API with the English title.

freeBase does not contain entities in Yoruba, Bengali, and Tamil, so the first step will always fail for these three languages.


(2015) turn to convolution network directly based on the original word2vec (Mikolov et al., 2013a).

they pay little attention to design effective word and entity representations.


This feature is directly counted from Wikipedia's anchor fields and measures the link probability of an entity e given a mention m. Prior is a strong indicator (Fader et al., 2009) to select the correct entity.

it is unwise to take prior as a feature all the time because prior usually get a very large weight, which overfits the training data.


The best performing solutions are supervised, discriminative learning methods which learn transliteration models from parallel transliteration corpora.

such corpora are available only for some language pairs.


If the switch is turned on, the decoder produces a word from its target vocabulary in the normal fashion.

if the switch is turned off, the decoder instead generates a pointer to one of the word-positions in the source.


Although this dataset is smaller and more complex than the Gigaword corpus, it is interesting to note that the Rouge numbers are in the same range.

our switching pointer/generator model as well as the hierarchical attention model described in Sec.


This implementation has the advantage of simplicity as it requires minimal changes to the training and deployment code, but we note that a more complex implementation utilizing sparse matrices and sparse matrix multiplication could potentially yield speed improvements.

such an implementation is beyond the scope of this paper.


Indeed these senses are presupposed by listeners according to linguistics theories (Segal et al., 1991;Murray, 1997;Levinson, 2000;Sanders, 2005;Kuperberg et al., 2011).

asr and Demberg (2015) finds that DCs are more often dropped for the discourse relation Chosen alternative (the relation typically signalled by the DC 'instead'), if the context contains negation words, which are identified cues for this relation.


Asr and Demberg (2012; 2015) attribute the corpus statistics to the UID hypothesis, which explains that expected, predictable relations are more likely to be conveyed implicitly, and thus more ambiguously, to maintain steady information flow.

there are explicit 'causal' and 'continuous' relations and some Chosen Alternative are marked even argument 1 is negated.


e U (imp;s,C) = e U (null;s,C) + e U (args;s,C) (9) The amount of information that the null DC provides for the discourse relation is defined similarly as in Equation (8): On the other hand, the informativeness of arguments, I(s; arg, C) is quantified by negative surprisal in RSA.


We also plan to evaluate the effectiveness of the model in applications, such as natural language generation or machine translation tasks.

as discourse presentation differs across genres (Webber, 2009) and mediums (Tonelli et al., 2010), the model can be applied to predict the explicitation of discourse relations from, for example, news articles to spoken dialogues.


In (Axelrod et al., 2011;Duh et al., 2013), the sizes of the in-domain data sets are 30K and over 100K sentences respectively.

we do not always have access to large or even medium amounts of in-domain data.


For example, in "I have a Dell desktop and a Macbook laptop", the words "Dell, laptop, Macbook, laptop" are from the computer domain, while the words "I, have, a, and" are general.

the topic of this sentence is decided by the domain specific words, not the general-domain words.


RNNs trained specifically to perform the agreement task can achieve very good average performance on a corpus, with accuracy close to 99%.

error rates increase substantially on complex sentences (Linzen et al., 2016, suggesting that the syntactic knowledge acquired by the RNN is imperfect.


The relevant studies such as clustering models and similarity computation in context (Fu et al., 2016;Wang, 2010) mainly focus on the metaphoric sense of each individual noun or adjectival phrase because the analyses are highly dependent on contextual information.

metaphoric senses of verbs are less touched because it is difficult to define regularities of their contextual information.


(2016) point out that the challenge arises from the highly context-dependent property of homonymies since the relations of senses are not unsystematic.

the senses of a polysemy form a systematic system, and thus CDSM has a better chance to detect metaphoric senses (Gutiérrez et al.


Figure 1 shows that verbs having the same radical are relatively similar to each other compared to verbs which belong to different radical groups.

the grouping by radicals does not work well in the metaphoric senses, as shown in the lower graph.


The incorporation of all the 17 features does improve the classification model by 1.70% in F-score.

group 1 has the best performance, outperforming the result when all the 17 features are used.


Since each of the syntactic conditions links to a particular aspect of a conceptual event, its change is an informative indicator of which sense, literal or metaphoric, is in use.

the conditions in Group 3 do not contribute much to detecting senses.


The formulation of Strehl and Ghosh (2003) is identical to ours.

there are several important differences.


This is an ambiguous result: on the one hand, a difference of 1.5% on the CoNLL dataset is non-negligible.

even the MUC-based method outperforms each individual component.


Here, the best MUC results (F = 73.35) are achieved with the top nine systems.

this MUC improvement comes at a high cost in B 3 and CEAF, leading to low MELA values even with the optimal parametrization.


coreference and entity relatedness) that are unavailable to local methods, and have significantly outperformed the local approach on standard datasets (Guo and Barbosa, 2014;Pershina et al., 2015;Globerson et al., 2016).

global approaches are difficult to apply in domains where only short and noisy text is available, as often occurs in social media, questions and answers, and other short web documents.


We find that both using pre-initialized embedding vocabularies, and the corrupt-sampling method employed are very important for properly training our model.

the gap between results of all systems tested on both CoNLL-YAGO and WikilinksNED indicates that mentions with noisy context are indeed a challenging test.


Deep learning has been successfully applied to many recent QA approaches and related tasks (Bordes et al., 2015;Hermann et al., 2015;He and Golub, 2016;Dong et al., 2015;Tan et al., 2016, inter alia).

large quantities of data are needed to train the millions of parameters often contained in these models.


Babelfy (Moro et al., 2014)) to a corpus and then using word2vec to learn sense embeddings over the pre-disambiguated text.

in their approach words are replaced by their intended senses, consequently producing as output sense representations only.


One important advantage of region representations is that they can distinguish words with a broad meaning from those with a more narrow meaning, and should thus in principle be better suited for tasks such as hypernym detection and taxonomy learning.

it is currently not well understood how such region based representations can best be learned.


(2011) using a sentiment lexicon to enhance embeddings for sentiment classification.

learning word embeddings with a particular target makes the approach less generic, also implying that customized adaptation has to be made whenever a new knowledge source is considered.


Typically to encode the entire vocabulary, the depth of the tree falls in a manageable range around 15 to 18.

different from learning context words, to encode a regularizer as shown on the right part of Figure 1, using hierarchical softmax is intractable due to exponential space demand.


The way we construct regularization matrix may be lossy, risking losing information that is explicitly delivered in the lexicon.

it provides us effective encodings for words, and also yields better learning performance empirically in our experiments.


Consequently, it can be observed from our experiment that unannotated knowledge, i.e., topic distributions, is not an effective source as a good guidance.

ppDB, which is of high quality of semantic knowledge, outperforms other types of information in most cases.


Empirical results show that with only character embedding features, the performance of our model outperforms CNN-CNN-MoT, and is close to LSTM-MoT.

there is still a big gap between character embedding and word embedding models, which could come from the fact that we use pretrained word embeddings, which helps improve the performance.


Taghipour and Ng (2016) tried to attend over words on their one-layer LSTM model, but failed to beat the baseline model that employs mean-over-time pooling, because of that text-level model contains a quite long sequence of words, which may weaken the effect of attention.

sentence-level model contains relatively short sequences of words, which makes attention more effective.


If you previously recognized the sentiment to be neutral or even positive, it is possible that the addition of this new word would cause the sentiment to change to negative.

it is hard to imagine a set of words to which adding the word drive-thru would change the sentiment in any direction.


While motivated differently, contrastive estimation (Smith and Eisner, 2005) is also related to matching.

ive estimation, negative training examples are synthesized by perturbing positive instances.


This phenomena has already been studied in script learning works (Chatman, 1980;Chambers and Jurafsky, 2008b;Ferraro and Van Durme, 2016;Pichotta and Mooney, 2016a;Peng and Roth, 2016).

modeling actions is not sufficient; participants in actions and their emotions are also important.


Recent research in the area has focused on neural approaches to summarisation, which can be very data-hungry.

few large datasets exist and none for the traditionally popular domain of scientific publications, which opens up challenging research avenues centered on encoding large, complex documents.


The introduction tends to be longer (average length of 72.1 sentences) than other sections, but still of a relatively simple level compared to the method (average length of 41.6 sentences), thus has more potential sentences for an author to use in highlights, giving the high Copy/Paste score.

it would also have more sentences which are not good summaries and thus reduce the overall average ROUGE score of the introduction.


We design this model to automatically generate mention embeddings and mention-pair embeddings that are used to construct cluster features for entity linking.

even though this model's success in coreference resolution is not our final objective, its success directly correlates to the success of entity linking because of the similarity between these two tasks.


Question q 1 is relevant with respect to it as it asks the same thing, and so is q 2 , which asks how much one should tip in a specific situation.

q 9 and q 10 are irrelevant: the former asks about what to wear at business meetings, and the latter asks about how to tip a kind of person who does not normally receive tips.


We train the network by minimizing the negative log-probability of the gold labels: The network described so far learns the abstract features through multiple hidden layers that are discriminative for the classification task, i.e., similar vs. non-similar.

our goal is also to make these features invariant across languages.


First, since here we are training with only 50% of the original training data, both FNN and CLANN-unsup yield lower results compared to before, i.e., compared to Table 1; this is to be expected.

the unsupervised adaptation, i.e., using the CLANN-unsup model, still yields improvements over the FNN model by a sizable margin, according to all three evaluation measures.


Our method achieves stateof-the-art F1 on the Jobs dataset.

even without such domain-specific supervision, the parser performs reasonably well.


Our grammar formalism can be related to synchronous CFGs (SCFGs) (Aho and Ullman, 1972) where the semantics and syntax are generated simultaneously.

instead of modeling the joint probability of the logical form and natural language utterance p(x, y), we model the factorized probability p(x)p(y|x).


Considering embeddings only, Levy and Goldberg (2014) proposed to learn word representations that incorporate syntax from dependency-based contexts.

we inject syntactic information by means of TKs, which establish a hard match between tree fragments, while the soft match is enabled by the similarities of distributed representations.


An alternative approach is to train the context embedding using neural networks on a sense annotated corpus, which can remap the word embeddings in a supervised fashion.

since there are not enough large disambiguated corpora, we need to approximate the word senses with coarse-grained information, e.g., the category of the context.


This shows that a more sophisticated interaction layer can help.

the differences are not substantial, indicating that this extension does not offer any systematic advantage.


It shows that more than half or more than a third of all questions in SQuAD or NewsQA, respectively, are (partially) answerable by a very simple neural BoW baseline.

the gap to state-of-the-art systems is quite large (≈ 20%F1) which indicates that employing 77.9 69.5 FastQA w/ beam-size k = 5 77.1 68.4 FastQAExt k = 5 78.9 70.8   Tables 2 and 3 clearly demonstrate the strength of the FastQA system.


This is in contrast to earlier studies (Sahu and Anand, 2017;Collobert and Weston, 2008) where pretrained embeddings Model i2b2-2010 DDI extraction Precision Recall F1 score Precision Recall F1 score SVM (Rink et al., 2011) 67    usually improved model performances by 3-4%.

this result aligns with the observations made in (Johnson and Zhang, 2015) and supports the argument for one-hot LSTMs.


The existing tool for automatic PID computation, CPIDR (Brown et al., 2008), is based on counting POS tags.

we noticed that the propositional structure of a sentence is very similar to its dependency structure, see the first column in Table 1.


When the repeated ideas are not counted (DEPID-R), the difference between groups becomes non-significant.

we were curious about why the association between the lower PID values and the AD diagnosis cannot be observed on DementiaBank.


4 On AMI data, the difference between group means is non-significant for both CPIDR and DE-PID values.

when the repeated ideas are excluded (DEPID-R), the mean PID for AD patients is significantly lower than for controls, as expected.


On AMI data, the SID performs surprisingly well, considering that the automatic ICUs were extracted from only 10 clusters and the number of clusters was not tuned to that dataset at all.

pID performs ca 5% better than SID in terms of all measures.


On DementiaBank dataset, cluster features alone do not perform too well and using cluster features together with PID and SID gives only minor improvements.

both the   Table 8: Classification results on DementiaBank (DB) and AMI using cluster features (C) combined with PID and SID.


On the open-domain dataset we found that the PID was more predictive than SID as expected.

the small number of automatically extracted cluster features underlying the SID, modeling the broad discussion topics, led to even better results.


In these efforts, the costs scale linearly in the number of instances, requiring significant investments for large datasets.

schema querification can generate an enormous amount of data for a fraction of the cost by labeling at the relation level; as evidence, we were able to generate a dataset 300 times larger than Simple QA.


We showed that relation extraction can be reduced to a reading comprehension problem, allowing us to generalize to unseen relations that are defined on-the-fly in natural language.

the problem of zero-shot relation extraction is far from solved, and poses an interesting challenge to both the information extraction and machine reading communities.


it can be understood in comparison with it: We can understand abstract change in terms of physical or local change.


Combining source and target data together into a single dataset is a simple way to jointly train for both domains.

this approach might not work well in the crosslingual case, i.e.


As expected, higher values of β yield lower precisions but higher recalls.

f 1 increases until  Table 2: Number of: "false anaphor" (fA, a non-anaphoric mention marked as anaphoric), "false new" (fN, an anaphoric mention marked as non-anaphoric), and "wrong link" (WL, an anaphoric mention is linked to a wrong antecedent) errors on the development set.


(2014a) in the sense that our resolvers incrementally add mentions to previously built clusters.


A disadvantage of using critics is that there is no guarantee that the critic converges to the true evaluation metric given finite training data.

our differentiable relaxations do not need to train, and the convergence is guaranteed as T → 0.


Yu and Jiang (2016) presented a convolutional NN that learns sentence embeddings using two auxiliary tasks (whether the sentence contains a positive or a negative domain independent sentiment word), purposely avoiding prediction with respect to a large set of pivot features.

to these works our model can learn useful cross-domain representations for any type of input example and in our cross-domain sentiment classification experiments it learns document level embeddings.


Our model is similar in structure to an autoencoder.

instead of reconstructing the input x from the hidden layer h(x), its reconstruction function r receives a low dimensional representation of the non-pivot features of the input (h(x np ), where x np is the non-pivot representation of x (Section 3)) and predicts whether each of the pivot features appears in this example or not.


The semantic role labeler achieves competitive performance on English, even without any kind of syntactic information and only using local inference.

when automatically predicted partof-speech tags are provided as input, it substantially outperforms all previous local models and approaches the best reported results on the English CoNLL-2009 dataset.


Semantic banks (e.g., PropBank (Palmer et al., 2005)) often represent arguments as syntactic constituents or, more generally, text spans (Baker  et al., 1998).

coNLL-2008 and shared tasks (Surdeanu et al., 2008;Hajic et al., 2009) popularized dependency-based semantic role labeling where the goal is to identify syntactic heads of arguments rather than entire constituents.


Moreover, as syntactic parsers are not reliable when used out-of-domain, standard (i.e., syntactically-informed) dependency SRL models are crippled when applied to such data.

our syntax-agnostic model appears to be considerably more robust: we achieve the best result so far on the English and Czech out-of-domain test set (77.7% and 87.2% F 1 , respectively).


Though LSTMs are likely to be affected by a similar trend, their states may be able to capture the uncertainty about the structure and thus let the  role classifier account for this uncertainty without the need to explicitly sum over potential syntactic analysis.

pathLSTM will have access only to the single (top scoring) parse tree and, thus, may be more brittle.


For example, if a word is a noun, it should take one of three tags from the case category: nominative (n), accusative (a), or genitive (g), while it should take "not applicable" (na) from the mood category since mood is not defined for nominals.

most of the previous approaches in Arabic did not utilize this information, applying one model for each task (Habash and Rambow, 2005;Pasha et al., 2014;Shahrour et al., 2015).


tional resources such as a morphological analyzer or a dependency parser, indicating the effectiveness of joint modeling of morphosyntactic categories.

the independent model gives an accuracy of 87.74%, which is 1.53% absolute worse than CamelParser.


(2016) proposed an approach in which they encode a sequence of possible morphosyntactic tags provided by a morphological analyzer using bi-directional LSTMs.

we provide an alternative way of encoding this information, as well as an analysis on the most influential categories in the encoded tag embeddings.


The rational for the separation is that different dialects have different affixes, make different lexical choices, and are influenced by different foreign languages.

performing reliable dialect identification to properly route text to the appropriate segmenter may be problematic, because conventional dialectal identification may lead to results that are lower than 90% .


Recently, it has become common to locally parametrize these models using rich features extracted by recurrent neural networks (such as LSTM), while enforcing consistent outputs through a simple linear-chain model, representing Markovian dependencies between successive labels.

the simple graphical model structure belies the often complex non-local constraints between output labels.


Outside of NLP, in tasks such as computer vision, certain classes of fully-connected graphical models over the output pixels have been used for multi-dimensional smoothing (Adams et al., 2010;Krähenbühl and Koltun, 2011), borrowing techniques for the graphics literature.

none of these models performs representation learning in the output space, as in the case of our proposed embedded latent-state model.


(Koo et al., 2010;Anzaroot et al., 2014;Vilnis et al., 2015), many based on dual decomposition (Rush and Collins, 2012).

the constraints are often fixed, and even when learned (Anzaroot et al., 2014;Vilnis et al., 2015), the learning is done simply on constraint weights generated from pre-made templates, the construction of which requires domain knowledge.


Table 1 shows that overall performance on the UMass Citation dataset using the embedded-state latent CRF (95.18) is marginally better than the baseline BiLSTM+CRF model (95.07).

examining the entities with the largest F1 score improvement in Table 2, we see that they are mostly within the VENUE section, which has longrange constraints with other sections, giving evidence of the model's ability to learn constraints from the citation dataset.


Recently, the RNN-based generators have shown improving results in tackling the NLG problems in task oriented-dialogue systems with varied proposed methods, such as HLSTM (Wen et al., 2015a), SCLSTM (Wen et al., 2015b), or espe-cially RNN Encoder-Decoder models integrating with attention mechanism, such as Enc-Dec (Wen et al., 2016b), and RALSTM .

such models have proved to work well only when providing a sufficient in-domain data since a modest dataset may harm the models' performance.


9 is differentiable, we can jointly optimize the parameter θ and variational parameter φ using standard gradient ascent techniques.

the KL divergence loss tends to be significantly small during training (Bowman et al., 2015).


Take, for example, the scr100 scenario in which the CrossVAE model mostly outperformed all the previous strong baselines with regard to the BLEU and the slot error rate ERR scores.

the previous methods showed extremely impaired performances regarding low BLEU score and high slot error rate ERR when training the models from scratch with only 10% of in-domain data (scr10).


The representation-focused approach (Huang et al., 2013) independently learns a representation for each ranking element (e.g., query and document) and then employ a similarity function.

the interactionfocused models are designed based on the early interactions between the ranking pairs as the input of network.


Our work studies on the strength of the dynamic relatedness between entities, hence we focus more on Pearson index.

traditional correlation metrics do not consider the positions in the ranked list (correlations at the top or bottom are treated equally).


Locations of social media users are important to many applications such as rapid disaster response, targeted advertisement, and news recommendation.

many users do not share their exact geographical coordinates due to reasons such as privacy concerns.


User network model can correctly geolocate only 18.4% of users.

our experiments show that excluding user network information declines the performance of the final model in terms of accuracy by 5.1%.


As reported in Table 2, our proposed approach achieves quite low median errors over the TWIT-TERUS and WNUT datasets (i.e., 40.1km and 0km, respectively).

there are some cases with large error distances, which make the mean errors much larger than median errors.


Our ablation study demonstrates that the location field highly contributes to the geolocation per-  Table 3: Performance breakdown for each component over WNUT dataset formance.

some prediction errors arise when location fields are incorrect.


To the best of our knowledge, there exists no prior work explicitly aiming at discovering thematic hierarchies in corpora.

the hierarchy-related effects are reported in some studies.


The pairwise ranking approach takes context into account.


PropBank semantic role annotation and the corresponding SemLink reference are constituents-based.

uD is a dependency formalism, and we employ a number of heuristics to align original PropBank annotations with the CoNLL-2009 datasets (Hajič et al., 2009) to recover the head node positions.


Let s 1 , s 2 ∈ S be two sentences: this rule is violated if, according to the model, a sentence s 1 contradicts s 2 , but s 2 does not contradict s 1 .

if we just use the final decision made by the neural NLI model, we can simply check whether the rule is violated by two given sentences, without any information on the degree of such a violation.


We can see that, in the case of rule R 1 (reflexivity of entailment), DAM and ESIM make a relatively low number of violations -namely 0.09 and 1.00 %, respectively.

in the case of cBiLSTM, we can see that, each sentence s ∈ S in the SNLI training set, with a 23.76 % chance, s does not entail itself -which violates our background knowledge.


Similar to these studies, our approach is also based on a Bi-LSTM CRF architecture.

considering the limited contexts within each individual sequence, we design two attention mechanisms to further incorporate topically related contextual information on both the document-level and corpus-level.


8 The impact on performance is fairly small indicating the robustness of both models.

after only two epochs of optimizing the selected words in the added sentence, the performance drops markedly under all variants of the sentence-level black-box attacks as displayed in the two bottom rows of  and incorrect answers (AddQA) is most detrimental and causes both models to perform at or even below chance level.


They reported to achieve 63% and 45% accuracies for coarse and fine-grained answer-type detection, respectively under 5-fold cross validation setup.

we manually create 5, 535 codemixed questions and train a CNN model that shows 87.21% and 83.56% accuracies for coarse and fine answer types, respectively, for the 5-fold cross validation.


(2017)'s system was mainly developed to answer pure factoid questions based only on the named entities denoting person, location and organization.

the datasets used in this experiment have different types of answers beyond the basic factoid questions.


Machine translation (MT) techniques greatly boost the productivity of the translation agencies (Arenas, 2008).

despite the recent advances achieved in this field, MT systems are still far to be perfect and make errors.


The SMT system was unable to handle well such unseen sentences.

iNMT systems usually respond much better to the human feedback than interactive SMT systems (Knowles and Koehn, 2016;Peris et al., 2017).


I would like to switch :)", the intention is not churny for the brand this tweet is addressed to.

in "@MARKE Internet langsamer als gedrosseltes.


They trained their models on a Twitter sentiment analysis corpus (Pak and Paroubek, 2010) which is composed of 15M data points with labeled sentiment.

to the best of our knowledge, there is no work that uses churn detection in the context of chatbot conversations.


2017), they weighted the representation of each word according to the position, and the words close to the aspect could be paid more attention.

this operation is not always reasonable and sometimes the adjunct word may be far away from the target word.


The most common approach to training NMT is to maximize the conditional log-probability of the correct translation given the source sentence.


Named-entity Recognition (NER) is an important task in the NLP field , and is widely used to solve many challenges.

in many scenarios, not all of the entities are explicitly mentioned in the text.


To an average human reader who is familiar with contemporary norms and trends, it is quite clear that Instagram app is discussed in the textual passage above.

it is not explicitly written, thus it is practically a latent entity.


Traditional approaches label each token in texts as part of named-entity, and achieve high performance (Ratinov and Roth, 2009;Passos et al., 2014;Chiu and Nichols, 2016).

these approaches are relying on the assumption that entities are necessarily mentioned in the text.


Therefore, a possible approach could possibly suggest that all of the entities should be predicted together as a single multi-task classification process.

this method is based on the assumption that all entities are necessarily related to one another (as presented in section 5.1).


Thus, there is a small difference between multitask and nonmultitask approaches in Table 2.

in Table 4 we examine the performance over the top-40 frequent entities, including very frequent entities (such ATP and ADP), and less frequent (such Oxygen, NADPH, NADP+ and water) as well.


The expected performance for Web applications is thus closer to the one observed on the Inspec and DUC2001 datasets, rather than on NUS.

on long documents, Multipartite outperforms all other methods.


As shown in Figure 1b, EmbedRank++ reduces the redundancy we faced with EmbedRank.

embedRank++ surprisingly results in a decrease of the F-score, as shown in Table 2.


The results obtained by GREEDY can be arbitrarily bad.

there are performance guarantees if the objective function f and the constraints I are "sufficiently nice" (Calinescu et al., 2011).


These models typically perform either "early" (input data are concatenated and pushed through a common model) or "late" (outputs of the last layer are combined together through linear or non-linear weighting) fusion.

our model does not fall into any of these categories directly as it is "iterative" in the sense that there are multiple fusions per decision, with an evolving belief state -the memory.


The very recent work (Zadeh et al., 2018a) also approaches multi-view learning with recourse to a system of recurrent encoders and attention mediated by global memory fusion.

fusion takes place at the encoder cell level, requires hard alignment, and is performed online in one sweep so it cannot be informed by upstream context.


Information about a specific semantic property can be stored latently over the dimensions of a semantic embedding model, such that the semantic prop-erty can be reliably decoded given an embedding vector, as tested in the previous section.

a stronger test of how closely an embedding model relates to human-elicited conceptual knowledge is to investigate whether the embedding dimensions encode interpretable, human-like semantic properties directly.


For instance, the Indo-European dataset has cognate cluster sizes ranging from 37 for meaning because to 1 for meaning name.

a non-parametric clustering method such as Chinese Restaurant Process (CRP; Gershman and Blei 2012) can form clusters directly from the data without the need for tuning the threshold.


The UPGMA algorithm is simple and yields reasonable results across various language families (List, 2012a).

uPGMA clustering algorithm is dependent on the threshold that needs to be tuned to obtain optimal performance (List et al., 2017b).


For these experiments, we use the dataset published as part of word2vec 5 , which consists of 8,869 semantic and 10,675 syntactic questions of this type (Mikolov et al., 2013).

word similarity measures the correlation 6 between the similarity scores produced by a model and a gold standard created by human annotators for a given set of word pairs.


More concretely, negative values of α are beneficial for the 15 https://github.com/lgazpio/DAM_STS centroid method up to certain point, bringing an improvement of nearly 4.5 points for glove, and the results clearly start degrading after that ceiling.

dAM is almost unaffected by negative values of α.


for filtering unacceptable generations at application time.

fluency evaluation of NLG systems constitutes a hard challenge: systems are often not limited to reusing words from the input, but can generate in an abstractive way.


They constitute a compromise between characters and words: On the one hand, they yield a smaller vocabulary, which reduces model size and training time, and improve handling of rare words, since those are partitioned into more frequent segments.

they contain more information than characters.


As can be seen, ILP produces the best output.


(2014) predicted the fluency (which they called grammaticality) of sentences written by English language learners.


Inducing sparseness while training neural networks has been shown to yield models with a lower memory footprint but similar effectiveness to dense models.

sparseness is typically induced starting from a dense model, and thus this advantage does not hold during training.


Also, stop words are the most frequent ones but are said to carry little information content.

table 3 confirms our initial hypothesis.


Traditional active learning (AL) methods for machine translation (MT) rely on heuristics.

these heuristics are limited when the characteristics of the MT problem change due to e.g.


.... Algorithmic Expert At a given AL state, the algorithmic expert selects a reasonable batch from the pool, D unl via: where m b b b φ φ φ denotes the underlying NMT model φ further retrained by incorporating the batch b b b, and B B B denotes the possible batches from D unl .

the number of possible batches is exponential in the size D unl , hence the above optimisation procedure would be very slow even for a moderately-sized pool.


see ; , where several heuristics for query sentence selection have been proposed, including the entropy over the potential translations (uncertainty sampling), query by committee, and a similarity-based sentence selection method.

active learning is largely under-explored for NMT.


For the Gītā dataset, the models CopyNet and PCRF-Seq2Seq report similar performances.

sahaśranāma is a noisier dataset, and we find that CopyNet outperforms all other models by a huge margin.


These probabilities are used as features in the log-linear ranker, and therefore the inverse parser affects the ranking results, albeit implicitly.

we should point out that the unsupervised training objective is relatively difficult to optimize, since there are no constraints to regularize the latent logical forms.


(2017), and this is likely due to the different visual features and max-of-hinges loss.

our Bilingual model with the additional c2c objective performs the best for German, whereas Gella et al.


They further use a pre-trained language model as a prior for their compression model to induce their compressed output to be grammatical.

their reported results are still based on models trained on at least 500k instances of paired data.


Wang and Lee (2018) train a generative adversarial network to encode references into a latent space and decode them in summaries using only unmatched documentsummary pairs.

in contrast with machine translation where monolingual data is plentiful and paired data scarce, summaries are paired with their respective documents when they exist, thus limiting the usefulness of such approaches.


The model specified above is supplied only with an unordered set of words with which to construct a shorter sentence.

there are typically many ways of ordering a given set of words into a grammatical sentence.


We found that ROUGE scores can be fairly uncorrelated with human evaluation, and in general can be distorted by quirks of the data set or model outputs, particularly pertaining to length, formatting, and handling of special tokens.

human evaluation can be more sensitive to comprehensibility and relevancy while being more robust to rewording and reasonable ambiguity.


Word embeddings are powerful tools that facilitate better analysis of natural language.

their quality highly depends on the resource used for training.


The intuition behind including only higher match-count n-grams in the training data is that they may be more valid segments of the raw text, as they appear several times in the same order.

we naturally lose information by pruning low match-count n-grams.


Having answered the first question, we will be able to quantify the effect of the fragmentation.

it is necessary to study the effect of the second parameter as well, in order to quantify the applicability of ngrams for embedding comprehensively.


In Factoid QA like air traffic information systems (ATIS) or dialogue systems, we answer the questions by extracting an entity from a structured database like relational databases or knowledge graphs.

in non-factoid systems, answers are extracted mostly from unstructured data like Wikipedia.


In contrast, (Xie and Eric, 2017) explicitly modeled candidate answers as sequences of constituents by encoding individual constituents using a chain oftrees LSTM (CT-LSTM) and tree-guided attention mechanism.

their formulation of constituents is more complicated than ours and as we will see, a direct use of constituents as answer chunk is much less complicated and yields better results.


One recent work (Yao, 2015) manually assigns a single numerical number to encode and derive phonetic similarity.

this single-encoding approach is inaccurate since the phonetic distances between Pinyins are not captured well in a one dimensional space.


Assuming twenty word pairs are provided as context per pair, the task quickly blows up to eighteen thousand annotations.

we observe that the phonetic similarity of Pinyin is greatly impacted by the pronunciation methods and the place of articulation -this allows us to improve the accuracy and simplify the annotation task.


In turn, MED struggles with representing accurate phonetic distances between initials, since most initials are of length 1, and the edit distance between any two characters of length 1 is identical.

dIMSIM encodes initials and finals separately, and thus even a 1-dimensional encoding (dIMSIM1) outperforms the other baselines.


However, as features of articulatory phonetics are manually assigned, these algorithms fall short in capturing the perceptual essence of phonetic similarity through empirical data (Kessler, 2005).

dIMSIM achieves high accuracy by learning the encodings both from high quality training data sets and linguistic Pinyin features.


Training frameworks are then used to learn the similarity.

the phonetic similarity used in these systems cannot be applied to Chinese words since Pinyin has its own specific characteristics, which do not easily map to English, for determining phonetic similarity.


News editorials are said to shape public opinion, which makes them a powerful tool and an important source of political argumentation.

rarely do editorials change anyone's stance on an issue completely, nor do they tend to argue explicitly (but rather follow a subtle rhetorical strategy).


As such, news editorials represent an important resource for research on argument mining (Mochales and Moens, 2011) and debating technologies (Rinott et al., 2015).

a single news editorial rarely changes the stance of a reader completely.


We captured our annotators' personality traits, too.

we primarily focus on nine political profiles from left to right (Doherty et al., 2017) in order to represent prior stance.


Accordingly, it meets our conditions of being of high argumentation quality.

figure 4 shows an excerpt of an editorial on global warming from the corpus.


Entity linking in long text has been well studied in previous work.

short text entity linking is more challenging since the texts are noisy and less coherent.


Several studies Huang et al., 2014) investigate collective tweet entity linking by pre-collecting and considering multiple tweets simultaneously.

multiple texts are not always available for collection and the process is time-consuming.


One straightforward way to combine multiple semantic matching signals is to apply a linear regression layer to learn a static weight for each matching signal (Francis-Landau et al., 2016).

we observe that the importance of different signals can be different case by case.


In the first example, the correct answer is 'Justin Trudeau' which contains the words of 'Canada' and 'trump' in its entity description.

m-CNN fails to capture this concrete matching information, since the concrete information of text might be lost after the convolution layer and maxpooling layer.


However, the soft-TF information in the descriptions of the two entities is similar.

m-CNN captures the whole meaning of the text and links the mention to the correct entity.


Note that the human studies and automatic evaluation are complementary to each other: while MTurk annotators are good at judging how natural and coherent a response is, they are usually not experts in the Ubuntu operating system's technical details.

automatic evaluation focuses more on the technical side (i.e., whether key activities or entities are present in the response).


Later approaches (Shang et al., 2015;Luong et al., 2015) applied Recurrent Neural Network (RNN)-based encoderdecoder architectures.

dialogue generation is considerably more difficult than language translation because of the wide possibility of responses in interactions.


A conversation between a pair of users often stops when the problem has been solved.

they might continue having a discussion which is not related to the topic.


If it attaches to the verb phrase saw a man, the first interpretation arises.

if it attaches to the noun man, the second interpretation is given.


On the other hand, if it attaches to the noun man, the second interpretation is given.

if the structural information is lost, we would have no way to disambiguate the two readings.


TreeLSTM achieves the state-of-the-art performance among the tree-structured models in various tasks, including natural language inference and sentiment classification.

there are non-tree-structured models on the market that outperform TreeLSTM.


To recapitulate, TreeRNN and TreeLSTM reflect the principle of compositionality but cannot capture the multiplicative interaction between two expressions.

cMS incorporates multiplicative interaction but violates the principle of compositionality.


As in CMS, the primary mode of semantic composition is matrix multiplication.

lMS improves on CMS in that it avoids associativity.


Topics tend to correspond to salient features, and are typically labelled with the most probable words according to the corresponding distribution.

while LDA only uses bag-of-words (BoW) representations, our focus is specifically on identifying and improving features that are modelled as directions in semantic spaces.


For the feature represented by the cluster {steep, climb, slope}, the top ranked object mountain is clearly relevant.

the next two objects -landscape and national parkare not directly related to this feature.


with any facts that relate to food but definitely not to geographical locations, sports, etc.

this assumption is not as damaging as long as P (t | s, z = f ) is almost zero for the z that P (z = f | s) should be ignored -the multiplication would result in zero anyway.


For PROFILEMEMORY + trained only on Per-sonaChat data, all types of sentences have similar effectiveness in predicting personality.

after joint learning with OpenSubtitles, only sentences from PersonaChat (which are most relevant to personalities) are able to predict noticeably accurate than other sentences.


Interestingly enough, there is no obvious correlation between how engaging the dialogue has been perceived and simple metrics like the length of the response or the number of asked questions.

it is strongly correlated with DISCOVERYSCORE, indicating that it indeed can be used as one of the automatic metrics for dialogues quality.


Some features are so distinctive that model can learn them easily.

a sentence may have more than one feature that can contribute to class prediction.


Given a source sentence, human translators are able to produce a set of diverse and reasonable translations.

although beam search for SEQ2SEQ models is able to generate various candidates, the final candidates often share majority of their tokens, except a few trailing ones.


Consider the trees tCO and t13 in Table 1. tCO is a supertag that does not use adjunction (this type of supertag is called an initial tree).

t13 modifies an internal VP node in another supertag (this type of supertag is called an auxiliary tree).


1 Deep neural networks have improved the accuracy of various natural language processing (NLP) tasks by performing representation learning with massive annotated datasets.

the annotations in NLP depend on the target language as well as the task, and it is unrealistic to prepare such extensive annotated datasets for every pair of language and task.


While the nearest neighbors of "excellent" and "terrible" are not semantically close, they all indicate positive and negative polarities in the respective domains.

the nearest neighbors of "economic" are noisy as they do not contribute to the task.


Another related task is Cross-lingual Lexical Substitution (Sinha et al., 2009): the model must provide plausible target language translations for the source language lexical item in the source language context.

our BTSR task: (1) directly evaluates token-level word representations without the need to predict sense labels from a sense inventory and (2) it contextualizes both the source query and the target candidates ensuring full sense disambiguation.


We follow these prior works in working with the CzEng, a Czech-English dataset (Bojar et al., 2016b), due to its size, diverse domain coverage, and rich syntactic variations , and to allow for a direct comparison in methodologies.

we propose a new approach to paraphrase generation designed to increase paraphrastic diversity, using a multi-step process: the first part of the pipeline generates a large number of candidate paraphrases through a random process, and the second part whittles them down to a much shorter list.


Some semantic changes during paraphrasing, especially omission, are not well-reflected by the (forward) probability p generate from the generating model.

a model running in the other direction can penalize this omission, as found by Goto and Tanaka (2017).


(2017), text-based retrieval models often handle misspellings poorly.

speech-based models are unlikely to suffer from similar problems because they inherently must deal with variation in the expression of words and utterances.


If we assume that S X and S Y are similar to each other in the LM's representation space, then A(Y | X) > 0 -i.e., encountering sentences with S X causes the LM to assign a higher probability to sentences with S Y .

if we assume that S X and S Y are unrelated to each other, then A(Y | X) = 0 -i.e., encountering sentences with S X does not cause the LM to change its probability for sentences with 3 A is shorthand for adaptation.


Unlike sentences with coordination, sentences with different types of RCs differ from each other at a surface level (see Table 1).

at a more abstract level they all share a common property: a gap.


Our analyses so far have demonstrated that sentences that belong to linguistically interpretable classes (e.g., sentences that match in reduction) are more similar to each other in the LMs' representation space than they are to sentences that do not belong to those classes (e.g., sentences that do not match in reduction).

it is unclear what properties of the sentences are driving this similarity between members of the class.


There was an increase in accuracy as the number of hidden units increased (see Figure 5b).

the similarity between object RCs and other types of RCs did not significantly correlate with agreement prediction; we therefore did not find any evidence for the hypothesis mentioned above.


We hypothesized that models' accuracy on subject verb agreement when preceded by object RCs would increase as the similarity between object RCs and the other types of RCs increased.


COCO, and in some cases reportedly surpass human-level performance as measured by n-gram based evaluation metrics (Bernardi et al., 2016).

recent work has revealed several caveats.


6 Therefore, one should not expect a model to generate a single caption with the concepts in a pair.

a model can generate a larger set of K captions using beam search or diverse decoding strategies.


(2019)) contain distributional information obtained from large-scale textual resources, which may improve generalization performance.

we do use them for this task because the resulting model may not have the expected paradigmatic gaps.


(2017), which is trained with a generative adversarial objective in order to generate more diverse captions.

these types are more equally distributed in the captions generated by BUTR+RR, as shown by the higher mean segmented type-token ratio (TTR 1 ) and bigram type-token ratio (TTR 2 ).


A major strength of these learned word embeddings is that they are able to capture useful semantic information that can be easily used in other tasks of interest such as semantic similarity and relatedness between pair of words (Mikolov et al., 2013a;Pennington et al., 2014;Wilson and Mihalcea, 2017) and dependency parsing (Chen and Manning, 2014;Dyer et al., 2015).

these models treat names and entities no more than the tokens used to mention them.


While the principles of federated learning are fairly generic, its methodology assumes that the underlying models are neural networks.

virtual keyboards are typically powered by n-gram language models for latency reasons.


They share many similarities with the vector space embeddings that are commonly used in natural language processing.

rather than representing entities in a single vector space, conceptual spaces are usually decomposed into several facets, each of which is then modelled as a relatively lowdimensional vector space.


Their approach relies on feature selection methods to find subsets of features that are predictive of particular class labels, based on a set of labelled training examples.

our focus in this paper is on unsupervised methods, as suitable training data is often not available.


horror and zombie) then the corresponding feature directions d a and d b will also be similar.

for paradigmatically similar words, such as horror and comedy, this should not be the case.


Looking more closely at the results of our main method IncAgg, it is interesting to note that large improvements are obtained for depth-1 decision trees, which shows that our facet subspaces make it easier to identify features that correspond to the categories from the corresponding classification problems.

large improvements can also be seen for SVMs, which shows that the actual decomposition of the space is also helpful.


The choice of linking vowel is partly determined by vowel harmony: back vowel stems select a or o whereas front vowel stems select e or ö.

for back vowel stems, it is largely unpredictable whether a or o is used (Siptár andTörkenczy 2000:224f., Vago 1980:110f.


Case syncretisms in inanimate (i.e., non-personal) nouns are found in many Slavic languages.

animacy is an inherent feature of nouns and cannot be predicted from the form of the lemma alone.


As a result, our final sample of twelve languages only includes two major language families, Indo-European and Uralic, the latter represented by Finnish and Hungarian.

this sample has some degree of grammatical diversity.


Bilingual word embeddings have been widely used to capture the correspondence of lexical semantics in different human languages.

the cross-lingual correspondence between sentences and words is less studied, despite that this correspondence can significantly benefit many applications such as crosslingual semantic search and textual inference.


Recently, language models (LMs) or language representation models are widely used in natural language understanding (NLU) tasks.

these LMs are usually trained on large unlabeled text corpora, while the finetuning process simply takes words or wordpieces as model input.


The researchers in Microsoft (Song et al., 2011) employed a big and rich probabilistic knowledgebase to machine learning algorithms, and got significant improvement in terms of tweets clustering accuracy.

such method needs huge human and material resources to build up a highquality and extremely wide-coverage knowledge base.


FastText  employed n-grams thus it could predict the zero-shot word embeddings.

n-gram is an arbitrary method of word segmentation.


Little performance boost is shown on SciTail, probably because sentences in SciTail are more focused on the expressions of concepts and knowledge, and thus key words are more about verbs and nouns rather than named entities.

as we unexpected, applying NPB-BPE * to SST-2 gains a 0.6% absolute increase.


The parameters in GPT are all the same for all wordpiece embeddings in the vocabulary.

linguistic features contain different levels of information compared with words and wordpieces.


Despite parallel syntax and overlapping vocabulary, the sentences above vary in numerous aspects of meaning: • The NPs her picture and the wall denote entities that stand in a certain locative relation to each other, as signaled by the preposition on.

• the relation between her speech (which is an event, not an entity) and security is a different one, TOPIC, despite being signaled by the same preposition.


Thus, the word may evoke 'nuclear disaster' in a reader in 2011 (e.g., driven by the Fukushima incident).

in 2012 the 'storm' sense may be more salient (e.g., driven by Superstorm Sandy).


They enable us to investigate how word meaning and relatedness between words change over time.

as it may take some time until an event's name is determined and referred to in newspapers, the paper's text may not have meaningful embeddings for those events.


Training an NLI model in this end-to-end manner assumes that any inference type involved in the sentence-level decision may be learned from the training data.

recent work created challenge datasets which show that these modelswhen trained on the original NLI datasets-fail when they need to make inferences pertaining to certain linguistic phenomena, often ones which are not sufficiently represented in the training data.


The TEED problem is essentially the same as that of Parallel Corpus Filtering (PCF), discussed in the previous section.

the usage scenario is quite different: in PCF, one is typically dealing with a very large collection of segment pairs, only a fraction of which are true translations; the PCF task is then to filter out pairs which are not proper translations, possibly with some tolerance for pairs of segments that do share partial meaning.


Globally, YiSi-2 clearly performs best at this task when using BWE's trained on domainspecific parallel data (PSC.bivec), even when there is very limited quantities of such data, as is the case here.

bERT models perform comparably to vector-mapped bWE's trained with indomain data (PSC.vecmap), and substantially better than bWE's trained on large quantities of generic, out-of-domain parallel data (WMT).


Recurrent neural network grammars (RNNGs) generate sentences using phrase-structure syntax and perform very well in terms of both language modeling and parsing performance.

since dependency annotations are much more readily available than phrase structure annotations, we propose two new generative models of projective dependency syntax, so as to explore whether generative dependency models are similarly effective.


This has already been explored in (Gu et al., 2018) in the context of Machine Translation.

our problem is simpler because the size of the output is always twice the size of the input, in other words we do not have to estimate the size of the output.


Compared to joint parsing systems working on both constituents and dependencies, our approach doesn't require external linguistic knowledge such as head percolation rules.

since derivations don't add new information, but merely offer a new vision of the problem, the potential accuracy gain is lower.


(2008) report the results of experiments which show that agreement between annotators is difficult to achieve, casting doubts on the reliability of the Project's codes.

in similar experiments, Lacewell and Werner (2013) report greater inter-annotator agreement, and claim that with ongoing training, annotators can produce reliable labels.


Some existing work alleviates this problem by directly incorporating coverage or fertility mechanisms to an NMT model (Tu et al., 2016;Feng et al., 2016;Kong et al., 2019).

the problem is that attention weights based coverage calculation for NMT is insensitive and sometimes even inaccurate to translation errors.


A policy gradient method is leveraged to co-train the NMT model and the discriminator.

those approaches rarely take account of translation adequacy.


Many linguists have formulated various constraints to define a general rule for code-switching (Poplack, 1978(Poplack, , 1980Belazi et al., 1994).

these constraints cannot be postulated as a universal rule for all code-switching scenarios, especially for languages that are syntactically divergent (Berk-Seligson, 1986), such as English and Mandarin since they have word alignments with an inverted order.


Difficulties in handling LDD have motivated the development of syntax-based MT (Yamada and Knight, 2001), that can effectively represent reordering at the phrase level, such as when translating between VSO and SOV languages.

syntaxbased MT models remain limited in their ability to map between arbitrarily different word orders (Sun et al., 2009;Xiong et al., 2012).


String-similarity metrics against a reference are known to be partial and coarsegrained aspects of the task (Callison-Burch et al., 2006), but are still the common practice in various text generation tasks.

their opaqueness and difficulty to interpret have led to efforts to improve evaluation measures so that they will better reflect the requirements of the task (Anderson et al., 2016;Sulem et al., 2018;Choshen and Abend, 2018b), and to increased interest in defin-ing more interpretable and telling measures (Lo and Wu, 2011;Hodosh et al., 2013;Choshen and Abend, 2018a).


The size of the sets allows using any MT evaluation measures to measure performance, and is thus a much more scalable solution than manual inspection, as is commonly done in challenge set approaches.

an automatic methodology has the side-effect of being noisier, and not necessarily selecting the most representative sentences for each phenomenon.


Preposition stranding is common in English and other languages such as Scandinavian languages or Dutch (Hornstein and Weinberg, 1981).

in German, it is not a part of standard written language (Beermann and Ik-Han, 2005), although it does (rarely) appear (Fanselow, 1983).


Dependency parsing has achieved new states of the art using distributed word representations in neural networks, trained with large amounts of annotated data Ma et al., 2018;Che et al., 2018).

many languages are low-resource, with small or no treebanks, which presents a severe challenge in developing accurate parsing systems in those languages.


Ideally, each word should find itself if it exists in the BERT vocabulary.


Cotterell and Heigold (2017) train charlevel taggers to predict morphological taggings for high/low resource languages jointly, alleviating OOV problems to some extent.

we focus on dealing with the OOV issue at subword level in the context of pre-trained BERT model.


Early studies on recurrent neural network (RNN)-based model analyze the translation quality with respect to the sentence length, and show that their models improve translations for long sentences, using the long short-term memory (LSTM) (Sutskever et al., 2014) or introducing the attention mechanism (Bahdanau et al., 2015;Luong et al., 2015).

koehn and knowles (2017) report that even RNN-based model with the attention mechanism performs worse than phrase-based statistical machine translation (koehn et al., 2007) in translating very long sentences, which challenges us to develop an NMT model that is robust to long sentences or more generally, variations in input length.


RNN or CNN-based NMT captures relative positions which stem from sequential operation of RNN or convolution operation of CNN.

position embeddings or positional encodings (vector representations of positions) are used to handle absolute positions in Transformer.


2018propose a RNN-based model, RNMT+, which is based on stacked LSTMs and incorporates some components from Transformer such as layer normalization and multi-head attention.

our model is based on Transformer and incorporates RNN into Transformer.


Transformer shows the worst performance among the four Transformer-based models on the sentences longer than those in the training data for any controlled length.

on the shorter sentences than those in the training data, RNN-Transformer scores almost the same as Transformer on the Middle and Long training data of ASPEC English-to-Japanese and also shows a larger drop than RNN-NMT at length of -24 on the Long training data of WMT2014 English-to-German.


Roy and Pentland (2002) were among the first to propose a computational model, known as CELL, that integrates both speech and vision to study child language acquisition.

cELL required both speech and images to be pre-processed, where canonical shapes were first extracted from images and further represented as histograms; and speech was discretised into phonemes.


We observed the same pattern for the word "fire hydrant" where both "fire hydrant" and "hydrant" are mapped to the same object.

figure 2b shows that when only prompted with /ÃÇ/ the network manages to find pictures of giraffes.


Adding the MFCC vectors corresponding to the transition from /Ç/ to /ae/ triggers a large difference in the embedding as the cosine similarity suddenly jumps to a higher value, showing it is getting closer to its final representation.

cosine similarity plateaus once /ae/ is reached up until final silence, suggesting the final /aef/ plays little to no role in the final representation of the word.


Recently, the creation of large-scale datasets (Bowman et al., 2015;wil;Khot et al., 2018) spurred the development of many neural models (Parikh et al., 2016;Nie and Bansal, 2017;Conneau et al., 2017;Balazs et al., 2017;Chen et al., 2017a;Radford et al., 2018;Devlin et al., 2018).

state-of-the-art models for NLI treat the task like a matching problem, which appears to work in many cases, but breaks down in others.


They propose DROP, a reading comprehension dataset focused on a limited set of discrete operations such as counting, comparison, sorting and arithmetic.

eQUATe features diverse phenomena that occur naturally in text, including reasoning with approximation, ordinals, implicit quantities and quantifiers, requiring NLI models to reason comprehensively about the interplay between quantities and language.


We note from the semantics and annotated examples, we expect this sense of grow to typically be literal.

in the CALIBRATABLE COS-45.6 class, it contains a Value role that moves along a scale by a certain Extent.