Now, the better a sentence is connected, the lower its weight.

s9 Apartheid is south Africa's policy of racial separation.


After applying WMVC to the graph in Figure 2, the cover C returned by the algorithm is {S 2 , S 4 , S 7 , S 9 } (highlighted in Figure 2).

the existing summarization methods do not utilize its potential fully.


'do with him order', is ambiguous between the last two and may mean either 'Deal with him' (R1) or 'Clean up with him' (R0).

but why should we care about semantic roles at all?


In this and the accompanying paper Jaworski and Przepiórkowski 2014 we suggest an answer in the negative and propose to approximate semantic roles on the basis of syntactic and morphosyntactic information.

in Zwierzę jest leczone z tych chorób 'An animal is treated for these diseases', in the VerbNet experiment the animal was marked as Beneficiary (by 3 annotators), as Patient (×3) and as Source (×1), and in the Sowa experiment -as Beneficiary (×2), as Patient (×2), as Recipient (×2) and as Result (×1).


2 Each sentence in Europarl was written in one of the official languages of the European Parliament and translated to all of the other languages.


We have found that increasing the number of these artificial projections that are used in training an SRL system does not improve performance as might have been expected when creating such a resource.

we must be careful of drawing too many conclusions because in addition to the difference in dependency schemes, the training data used to train the parsers as well as the parsers themselves are different.


RTE is the task of deciding whether a long text T entails a shorter text, typically a single sentence, called hypothesis H. It has been often seen as a classification task (see (Dagan et al., 2013)).

cSTKs are defined over a chunk-based syntactic subtrees where terminal nodes are words or word sequences.


As usual, a tree kernel, although written in a recursive way, computes the following general equation: (1) In our case, the basic similarity K F (t i , t j ) is defined to take into account the syntactic structure and the distributional semantic part.

productions of the initial subtrees are complete.


Given a simple transition, a process can be viewed as simply an iteration of ν (Fernando, 2009).

more robust parsing will afford us the opportunity to expand the diversity of predicates that the software can handle as well (mc-Donald and Pustejovsky, 2014).


This 3-vector is computed as a quaternion for rendering purposes.

we intend to expand the object library to include more complex inanimate objects (tables, chairs, or other household objects) as well as animate objects.


Additional contextual information from videos (e.g., scene locations) should help improve performance, especially on tougher videos (e.g., videos involving children chases).

we define the maximum likelihood probabilitiesP , derived from relative frequencies f , for the unigrams, bigrams, and trigrams as follows: for all mental state labels m, activities, and actor types in our queries.


Upon review of the video, we agreed that one child did indeed look annoyed.

for recall, the reward given for each color in the mystery bag is capped by the number of pencils of that color in the response bag.


The pruned distribution is renormalized to yield the final response distribution.

this would not account for the probability of r in the gold standard distribution, G. An analogy might help here: Suppose we have an unknown "mystery bag" of 100 colored pencils that we will try to match with a "response bag" of pencils.


The set ∆ can then be derived from Γ using the following natural deduction rules: 4 • Initialize ∆ with lambda terms (sets) that have no outscoped sets in Γ: • Add constraints to appropriate sets in ∆: • Add constraints of supersets as constraints on subsets in ∆: For example, the graph in Figure 2 can be translated into the following lambda calculus expression (including quantifiers over eventualities in the source graph, to eliminate unbound variables): The semantic dependency representation defined in this paper assumes semantic dependencies other than those representing continuations are derived compositionally by a categorial grammar.

this notation will be used in Section 4 to define constraints in the form of equations.


When used to guide a statistical or vectorial representation, it is possible that this local context will allow certain types of inference to be defined by simple pattern matching, which could be implemented in existing working memory models.

this definition assumes a Generalized Categorial Grammar (GCG) (Bach, 1981;Oehrle, 1994), because it can be used to distinguish argument and modifier compositions (from which restrictor and nuclear scope sets are derived in a treestructured continuation graph), and because large GCG-annotated corpora defined with this distinction are readily available (Nguyen et al., 2012).


An agent who is told that an entity with height h is tall adds that observation to its knowledge base without questioning the reliability of the speaker.

our setting so far offers a straightforward solution to this: If a new entity x : T with height h is referred to as tall, the agent adds h to its set of observations Ω T tall and recomputes µ tall (Human), for instance using RH-R as defined in (13).


Contrary to what has been argued in the literature (Rapp, 2002; Sahlgren, 2006)-that bag-of-words models based on secondorder statistics mainly capture paradigmatic relations and that syntagmatic relations need to be gathered from first-order models-we show that second-order models perform well on both paradigmatic and syntagmatic relations if their parameters are properly tuned.

the results displayed in table 2 show that dimensionality reduction with SVD improves the performance of the models for all datasets but GEK.


We can thus explore the empirical question of whether all these related phenomena can be tackled together, with a single model accounting for all of them.

most asymmetric measures proposed in the literature build upon the distributional inclusion hypothesis, stating that "if u is a semantically narrower term than v, then a significant number of salient distributional features of u is included in the feature vector of v as well" (Lenci and Benotto, 2012).


We collect both membership and typicality ratings because we expect them to have different implications for sound entailment.

being able to track the impact that modifiers have on heads should thus have a positive effect on important tasks such as recognizing textual entailment, paraphrasing and anaphora resolution (Androutsopoulos and Malakasiotis, 2010;Dagan et al., 2009;Poesio et al., 2010).


The highly significant correlations show that the measures do capture to some extent the patterns of variance in the data.

as a control, we also present ratings for unmodified h as an instance of c (we will use them below to test similarity measures on their ability to capture the direction of the membership relation, and to zero in on the effect of modification vs. more general membership/typicality effects).


Finding meaningful combinations among unattested or infrequent phrases was not an easy task and there was not always a perfect candidate.

we are thus also providing a novel evaluation of compositional models and asymmetric measures on a challenging task where they could potentially be very useful.


We treat the top 10K most frequent lemmas as context elements.

we think that it is more productive for computational systems to handle modifier-triggered disambiguation as a special case of the more general class of modification effects, than to engage in the quixotic pursuit to determine, a priori, what's the boundary between a word-sense and a "pure" modification effect.


With the LSTM-RNN model, the tanh function, in general, worked best whereas the sigmoid function was the worst.

positive, negative, or neutral) is available, we put a softmax layer (see Equation 1) to compute the probability of assigning a class to it.


We use a neural network which consists of a weight matrix W 1 ∈ R d×d for left children and a weight matrix W 2 ∈ R d×d for right children to compute the vector for a parent node in a bottom up manner.

the complexities of the two models are dominated by the matrix-vector multiplications that are carried out.


Vector adaptation methods modify a traditional (i.e.

it is somewhat surprising that dimensionality reduction and integration of semantic spaces do not help in improving performance.


We believe that this is the case because implicit SRL, as discussed in Section 2.3, can rely less on syntactic features but must make predictions on the basis of semantic and discourse features, which are more comparable across target parts of speech.

the main question is whether the addition of the (much smaller) SEMEVAL corpus to GERBERCHAI can improve performance.


Table 2), which is shaped by subcategorization, is a likely candidate for changess across domains, due to sense shifts.

all previous studies on the SEMEVaL dataset used the FrameNet annotation, and without access to the actual predictions we cannot directly compare our predictions to theirs.


The experiments presented in this paper are geared towards the identification of: (1) all 4 types unified under a single label and (2) the "Displacement" type of CMCs (1 of the 4 types).

the most marked improvement is in the WEB models (both CMC and DISPLACE) and the BN model's DISPLACE label classification.


Statistically significant change from the Baseline feature set is marked with a †.

if the semantic interpretation is strictly based on the expected semantics of the verb and its arguments, it fails to include the relevant information from the CMC.


(Tou Ng et al., 1999) propose to use annotator agreement to cluster senses, reporting higher interannotator agreement after clustering.

scale 4 artifact 6 3 a flat surface at right angles to a plumb line 3 artifact 5 2 indicator that establishes the horizontal when a bubble is centered in a tube of liq.


For example, WordNet senses #3 and #4 are grouped under the stative supersense, although the definition and use of the two senses are completely different.


Another problem with BioScopeScopeSpan annotations stems from the requirement that such annotations should have contiguous spans.

as shown in Table 1, our CuePredicate tagger obtained F-measures in the range of state-of-the-art results on negation cue detection using the BioScope (90-96% F-measure (Velldal et al., 2012)).


predicate nodes corresponding to the deep syntactic subject: observe how activator is 'subj' both to do and activate.

we now build a supervised learning system which, given a CuePredicate in a sentence, will identify its corresponding NegatedPredicate.


Since our task is different-negated predicate detection as opposed to negated span detectionwe report the Percentage of Correct Scope Predicates (PCSP) obtained in our experiments.

whether to include a subject into a verb scope (e.g.


We make no claim that the above list is exhaustive or that there would not be exceptions to these rules.

we discuss these two assumptions in turn.


As already explained (Section 2), we take the terms "belief" and "factuality" to refer to the same phenomenon underlyingly (with perhaps different emphases).

in the quote above, Saurí and Pustejovsky (2012) (apart from distinguishing factuality from truth) also make the point that the writer's communicative intention of making the reader believe she has a specific belief state does not mean that she actually has that cognitive state, since she may be lying.


We use 6 different word relatedness benchmarks to evaluate NESA.

every seed concept has a ranked list of 20 related Wikipedia concepts.


This transformation allows the computation of the correlation weights between the concept dimensions.

to obtain the DiSER based relatedness scores between Wikipedia concepts, we use Entity Relatedness Graph (EnRG) 4 (Aggarwal et al., 2015), which is a focused related entities explorer based on DiSER scores.


ESA represents the semantics of a word with a high dimensional vector over the Wikipedia concepts.

it shows that the distributional representation of the article title captures the semantic information better than considering only the corresponding article content.


Table 1 shows that the top 5 Wikipedia concepts retrieved for "football" and "soccer" do not share any concept, however, the concepts may exhibit relatedness to each other.

most of the knowledge-based measures use the taxonomic relations for computing word relatedness.


This paper explores the utility of homophily within joint models for document-level semantic classification, focusing specifically on tasks which are not associated with any explicit graph structure.

2-grams and 5-grams are the next best, with a statistically significant 3.37% absolute gain over the content-only baseline.


We also replaced our term length factor, TL in equation 2, with KP-miner's boosting function for multi-words.

as these results show, on this dataset of relatively short documents, TextRank outperforms KP-Miner for k>2.


We compare our approach with two other unsupervised algorithms that utilize this heuristic: KP-Miner and KX-FBK.

longer n-grams are generally more likely to be keyphrases.


This procedure leads to an assimilation of density values in the graphs G k as shown in Table 1: for the 10 languages, the relative standard deviation in network density decreases by about 23%.

we decided to lemmatize only words in the reference language and kept full-forms for all source languages.


For frame-semantic parsing, no such restriction in entity type exists.

this bias affects both, frame-semantic parsing and event extraction.


As in event extraction, frames occur within sentences and have triggers and roles (called lexical units and frame elements).

their system gives a higher precision for both subtasks.


A word pair is represented as a vector of features set up with the most meaningful patterns of context and filled in with information extracted from the graph representation of the corpus.

word pair representations Using the graph model G and the set of contextual patterns automatically acquired P, each word pair (x, y) is represented as a binary distribution over each pattern from P. Rather than using the input corpus to identify contexts of occurrence for the word pair (x, y) and match those with the acquired patterns, GraCE uses paths connecting x and y in G. All the paths between x and y up to three edges are extracted from G. These paths are then matched against the feature patterns from P and the word pair (x, y) is represented as a binary vector encoding non-zero values for all the features matching the pair's paths extracted from G, and zero otherwise.


The K&H dataset contains only instances from three domains and is imbalanced between the number of instances across domains and relation types.

given two word pairs, (w 1 , w 2 ) and (w 3 , w 4 ), if w 1 is lexically similar to w 3 and w 2 to w 4 (i.e., are pair-wise similar) then the pairs are said to have the same semantic relation.


However, (Turney, 2006b;Turney, 2008a) showed that relational similarity cannot be improved using the distributional similarity of words.

the novelty of this system stands in the graphbased representation.


these approaches are limited because a relation may be expressed in many ways, depending on the domain, author, and writing style, which may not match the originally identified patterns.


Alignments can be established at the word level, phrase level (MacCartney et al., 2008), or dependency level (Dinu and Wang, 2009).

1 We implement the algorithm within an open source TE development platform (Padó et al., 2015).


One feature of the Dinners from Hell corpus that bears further inspection in future work is the fact that its stories contain many violations of the restaurant script.

first, the ordered pmi model, pmi(e i , e) + n i=k+1 pmi(e, e i ) (4) where C(e 1 , e 2 ) is asymmetric, i.e., C(e 1 , e 2 ) counts only cases in which e 1 occurs before e 2 .


Though many early AI systems employed hand-encoded scripts, more recent work has attempted to induce scripts with automatic and scalable techniques.

our results suggest that applying these techniques to a domain-specific dataset may be reasonable way to learn domain-specific scripts.


General surveys on EL can be found in (Cornolti et al., 2013) and (Rao et al., 2013).

mapping files: Evaluating EL to Wikipedia requires making sure that we consider the same set of target entities for each EL system, since the versions of Wikipedia deployed within each system may differ.


using the XML frame file structure of Propbank, we defined the automatic generation of frame files, combining data from Propbank-Br and from Propbank, as shown in Fig.3.


Only 109 of the 1453 senses identified in Portuguese did not have an equivalent verb sense in English identified in Propbank.

this approach is part of a previous decision to invert the process of implementing a Propbank project, by first annotating a core corpus and only then generating a lexical resource to enable further annotation tasks.


With the recent extension of PropBank SRL to nominal and adjective predicates, preposition relationships, light-verb constructions, and abstract meaning representation (Bonial et al., 2014;Banarescu et al., 2013), it may be time to revisit SP for SRL.

with this definition, we allow different role labels to share the same topic (though it does not encode role constraints quite like LinkLDA, ROOTH-LDA, etc).


We used 800 topics (w/ lemmatized headwords) tuning on the    5), our SP approach had a smaller (but still statistically significant) absolute F1 gain, with most of the gain coming from core argument type improvements.

the margin of improvement is a modest 0.4 F1 point (on WSJ) over a baseline system with performance over 4 F1 points lower than the top system in CoNLL-2005 (Carreras and Màrquez, 2005).


Handling adjectives, adverbs and words with different POS tags: To get the best out of all Word-Net similarity measures, we exploited the relationships between different forms of the terms in Word-Net to find the noun form of the terms in the entity models and pruned sentences before calculating the similarity.

the term similarity is computed by forming an ensemble using the standard WordNet similarity measures namely, WUP (Wu and Palmer, 1994), LCH (Leacock and Chodorow, 1998), Resnik (Resnik, 1995), LIN (Lin, 1998), JCN (Jiang and Conrath, 1997), as well as a predict vector-based measure Word2vec (Mikolov et al., 2013) and a morphology-based similarity metric Levenshtein 1 as: where t 1 and t 2 are input terms and M is the set of above mentioned similarity measures.


Since the MCS showed low precision for the Tneg category in the previous experiment (Table 2), it is potentially introducing too much noise that the SVM is not able to linearly separate.

unfortunately, adjectives and adverbs are not arranged in a hierarchy, and terms with different part of speech (POS) tags cannot be mapped to the same hierarchy.


Negation detection is traditionally separated from the entity recognition task because negation indicating terms can be recognized separately from the phrases that contain explicit mention of an entity.

the first experiment evaluates the classification performance of our algorithm, MCS, and SVM.


An entity can have multiple definitions each explaining it using diverse vocabulary.

we deem some of the related algorithms to have good potential applicability for this task.


We used the NegEx algorithm (Chapman et al., 2001) to address the first type of negations.

we do not expect this limitation to have a major impact.


While the annotators have good agreement on annotating sentences in category TP, they agreed less on the categories Tneg and TN.

we included two strong algorithms from the closest related work as baseline solutions to the problem.


For the two English-Swedish equivalent sentences: Cologne is located on both sides of the Rhine River and Köln ligger på båda sidorna av floden Rhen, Wikifier, on the English version, identifies Cologne and Rhine river as named entities and links them respectively to the en.wikipedia.org/wiki/Cologne and en.wikipedia.org/wiki/Rhine pages, while NEDforia, on the Swedish text, produces a ranked list of entity candidates for the words Köln and Rhen shown in Table 4.

this differs from the predicate sense, open.02, having the meaning of something beginning in a certain state, such as a stock opening at a certain price.


It combines layout, discourse and terminological analyses to bridge the gap between the document layout and lexical resources.

table 4: Results for terms recognition vector-based strategies present interesting precisions, which seems to confirm a correlation between the lexical cohesion of terms and their likelihood of being involved in a relation.


Some attempts have been made for improving these results (Oh et al., 2009;Yamada et al., 2009).

since it is expected that terms involved in the relation share the same lexical field, we also consider the cosine similarity between the term vectors.


after applying λs.s(λx.x) to the semantic representation s i−1 , the semantic transition function t i is applied.


This latter limitation is particularly critical since it leads to significant degradation in performance when the trained system is applied to texts from new domains.

roleF1   For the Brown (OOD) dataset, the role subsystem F1 improves significantly with POS and dependency parse information (+2.72%) while the sense subsystem benefits less (0.96%).


An intermediate vector, δ, is calculated, which will contain the unnormalized log probability that any path through the trellis will pass through a particular state k for the particular word t. The δ vectors have a dimension of N, the number of tags, and they are re-used for the gradient calculation during back-propagation.

recent results using multilayer neural networks and pre-trained word embeddings have demonstrated high performance using a much smaller number of minimalist features.


It is straightforward to see that J T (I) is an antimonotone function.

we then evaluate the EFMP approach to confirm that it provides much higher performance figures and is on par with the state of the art.


Due to the tight coupling of multiple resolvers, a direct comparison with systems focusing on discourse deixis is hard.

negated parent/candidate We consider a verb token to be negated if it has a child connected with a negation label.


current coreference resolution systems ignore this phenomenon.


The degree to which two words are near-synonyms is proportional to the degree to which one can substitute for another in a given context (Inkpen and Hirst, 2006).

despite the lower overall emotionality of Hypothesis 2 All Pairs, our hypothesis that metaphorical instances are more emotional than the literal ones still holds.


Previous research on metaphor annotation identified metaphorical uses of words in text, thus analysing data for only one sense at a time.

focus Word 1: attack Sentence 1: I attacked the problem as soon as I was up.


But it is unable to distinguish between events and propositions and between predicate and sentence modifiers, and the formal interpretation of quantification in HLF can lead to contradiction (Schubert, 2015).

in this paper we limit ourselves to discussion of the "forward" version of gloss-derived axioms.


SRL systems (Täckström et al., 2015;Roth and Woodsend, 2014) have benefited NLP applications, and many approaches have been proposed to transfer semantic roles from English to other languages without further reliance on manual annotation (Kozhevnikov and Titov, 2013;Padó and Lapata, 2009).

our goal is to distinguish implicit roles from other translation shifts that cause poor alignment in SRL projection.


Accordingly, a transduction grammar is able to generate, translate or accept a transduction or a set of bisentences.

the English sentence can be divided into three major parts: "the Japanese islands", "run northeast to southwest" and "in the northwest part of the pacific ocean.".


OWL concepts and a set of SWRL rules then derive what the text implies about (the author's view of) reality and what the reader might make of it.

an important trait of their approach lies in the fact that these annotations are always relative to sources mentioned in the text, typically subjects or objects of source-introducing predicates, for instance, "said the minister".


for SE-WSI-CRP, we first learn word embeddings with word2vec 1 , then use them as pre-trained vectors to learn sense embeddings.


In a rigorous evaluation,  showed that neural word embeddings such as skipgram have an edge over traditional count-based models.

for instance, the word segmentation has 1105 positive dimensions in the vanilla configuration, but only 577 in the latter.


They also define a normalized modified purity and normalized inverse purity for evaluation, explained below.

the verb senses are generated through a DPMM, and we do not have a gold-label assignment of VerbNet classes to each sense.


For example, while an expected surge in unemployment is not a surge in unemployment, a policy that deals with an expected surge deals with a surge.

(3b) UNKNOWN Thrun predicted that leaps in artificial intelligence would lead to driverless cars on the roads by 2030.


In our model, alignment between Q and R and the prior for the author of R with respect to marker class m correspond to feature vectors a and b, respectively, with a first feature indicating marker presence and a second feature accounting for marker frequency.

in the current setting, if w 2 = 0, we obtain the original measure where only the presence of a marker is recorded, without taking into account frequency and hence post length.


4 Both LDA-based models (Misra et al., 2009;Riedl and Biemann, 2012) and GRAPHSEG rely on corpus-derived word representations.

the GRAPHSEG algorithm significantly outperforms the topictiling method (p < 0.05, Student's t-test).


All of these features are expected to be useful in stance classification as well.

observe that the word embedding features are beneficial for four out of five targets.


This approach was therefore specific to the domain of discussion threads.

we then perform a study of the employability of LSTM and CNN for this kind of text classification task.


This schema is composed of all types from diverse available ontologies.

the test collection is composed of a knowledge base of more than 350,000 entity categories obtained from the complete set of Wikipedia 2014 categories, but removing those containing non-ASCII characters.


The Yago project (Suchanek et al., 2007) shares unary properties associating hundreds of thousands of descriptive categories manually created by the Wikipedia community to DBpedia entities (Auer et al., 2007).

the problem resides in the fact that places and demonyms have a high relatedness with common nouns.


To measure the efficacy of vector arithmetic in a manner controlled for variances in prior vector similarity, we propose a baseline, defined for each analogy as the best ranking between the word most similar to w 2 and the word most similar to w 3 : For the above example, as dog is the most similar word to dogs, there is no improvement to be made upon baseline.

categories are grouped into a "semantic" and a "syntactic" subset, and results are often reported averaged over each rather than by category.


During preprocessing we handled removal of punctuation.

• We demonstrate improvements to an integrated path-based and distributional model, showing that our morphology-aware neural network model, AntNET, performs better than state-of-the-art methods for antonym detection.


For example, from the paraphrase pair (caught/not escape), we can derive the antonym pair (caught/escape) by just removing the negating word 'not'.

automated methods have been proposed to determine for a given term-pair (x, y), whether x and y are antonyms of each other, based on their occurrences in a large corpus.


For algorithms based on sentence order, it is not clear whether the problem lies in the lack of wider collections of texts in some languages, or rather on the maximum amount of information about polarity that is learnt through a sentence-level distributional hypothesis.

we decoded the unsupervised representations multiple times through different initialisation of the MLP weights, hence we report both the mean value and its standard deviation.


Context-aware models outperform context-agnostic models by up to 3 points 5 .

this yields three d-dimensional vectors for w l ( w l,max , w l,min , w l,mean ), and three for w r ( w r,max , w r,min , w r,mean ).


We require a dataset to study hypernymy detection in context to satisfy the following desiderata: (1) the dataset should make it possible to assess the sensitivity of context-aware models to contexts that signal different word senses, and (2) the dataset should help quantify the extent to which models detect the asymmetric direction of hypernymy, rather than symmetric semantic similarity.

we propose to detect hypernymy between word meanings described by specific contexts.


Our D-TopWords model will give more weights to the probability whether a sequence can be a word or not, and the TopWords model will more reliable on the external information.

the proposed model can extract these two types of words jointly.


Recent efforts have, however, proposed semi-automated methods for resolving these inconsistencies (Chan et al., 2017).

the value of ρ was set to 50, and the LFN had a single hidden layer containing 1000 neurons with the tanh activation function.


(2014b) our aim is to create dense input features to allow neural network architectures, as well as other machine learning algorithms, to be trained on them.

the second link is from token 6-7 in the gold standard which matches the model's prediction.


Existing affect datasets are predominantly categorical.

• Some tweets have words clearly indicative of the focus emotion and some tweets do not.


Within the NLP community, Best-Worst Scaling (BWS) has thus far been used only to annotate words: for example, for creating datasets for relational similarity (Jurgens et al., 2012), word-sense disambiguation (Jurgens, 2013), word-sentiment intensity (Kiritchenko et al., 2014), and phrase sentiment composition (Kiritchenko and Mohammad, 2016).

a promising avenue of future work is to test theories of emotion composition: e.g, whether optimism is indeed a combination of joy and anticipation, whether awe if fear and surprise, and so on, as some have suggested (Plutchik, 1980).


Best results for each column are shown in bold: highest score by a feature set, highest score using a single lexicon, and highest score using feature set combinations.

most work on automatic emotion detection has focused on categorical classification (presence of anger, joy, sadness, etc.).


In this work we demonstrate how online Deep Active Learning can be integrated with standard neural network based dialogue systems to enhance their open-domain conversational skills.

the weight λ associated with this metric is tuned to aggressively promote diversity between the first tokens of each of the K generated sequences, thereby avoiding similar beams like I don't know and I really don't know.


The first predicts all negative attachments, which yields an accuracy of 85.8% on the test set (with F1 of 0).

then, for each example e, we compute the Ad-dCos lexical substitutability between w p and the target word in context C e .


Target-Only gets very high scores on FrameNet dataset.

we use cross-validation on the training set and we observe the model performs better when we update the word vectors which is different from the preceding experimental setup.


All dependents related to the target, their POS tags, dependency relations, lemmas, NE tags and the target itself will be extracted as features.

the number of training instances per target is limited.


Frame semantics currently used in NLP have a rich history in linguistic literature.

the combined 20 properties of Reisinger et al.


Also, its sharing of training data among relations should lead to more reliable learning for infrequent relations.

in this paper, we assess to what extent relational attributes of entities are easily accessible from word embedding space.


In addition, all subjects were native speakers of Standard American English, while half the participants in the CXD corpus were native speakers of Mandarin Chinese.

we compared the cosine similarity of the turn and the question to the cosine similarity of any previous identified matches.


(2) so that the predicate matrix does not learn a direct transformation from an argument vector to a phrase vector, but rather a difference between these vectors: This justifies the addition of predicate vector in (1).

we investigate the application of PLF to Croatian language, a Slavic language with relatively free word order.


Table 2 shows that our model outperforms the baseline Word2Vec Skip-gram model (in fifth row from bottom).

the models are trained using Stochastic Gradient Descent (SGD).


our system first performs clustering of verb arguments to identify their possible semantic roles and then computes the level of association between a given argument role and the verb, thus deriving the structure of the semantic frame in which the verb participates.


Titov and Khoddam (2015) proposed a reconstruction-error minimization approach using a log-linear model to predict roles given syntactic and lexical features and a probabilistic tensor factorization model to identify argument fillers based on the role predictions and the predicate.

these works point out the coverage limitations of the hand-constructed FrameNet database, suggesting that a data-driven frame acquisition method is needed to enable the integration of frame semantics into real-world NLP applications.


8 To ensure the quality of workers, we applied a qualification test and required a 99% approval rate for at least 1,000 prior tasks.

finally, we rank the predicate pairs according to the number of instances in which they were aligned ( §3.4).


We extract propositions from news tweets using PropS , which simplifies dependency trees by conveniently marking a wide range of predicates (e.g, verbal, adjectival, nonlexical) and positioning them as direct heads of their corresponding arguments.

to demonstrate the added value of our resource, we show that even in its current size, it already contains accurate predicate pairs which are absent from the existing resources.


At the current stage of the resource there is a long tail of paraphrases that appeared few times.

we run PropS over dependency trees predicted by spaCy 6 and extract predicate types (as in Table 1) composed of verbal predicates, datives, prepositions, and auxiliaries.


(2017) evaluated an unstructured QA system against semantic parsing benchmarks.

note that TF-IDF is by far the most impactful feature, leading to a large drop of 12 points in performance.


Secondly, we introduce a semantic composition weight that is used to model the reading times of metonymic sentences reported in previous experimental studies and to predict the covert event in a binary classification task (Section 3).

we totally extracted 1,043,766 events that include at least one of the words of the evaluation datasets.


Competition between different adposition construals accounts for many of the alternations that are near-paraphrases, but potentially involve slightly different nuances of meaning (e.g., "talk to someone" vs. "talk with someone"; "angry at someone" vs. "angry with someone").

scenes of emotion and perception (Dirven, 1997;Osmond, 1997;Radden, 1998) provide a compelling case for the construal analysis.


Every token was initially labeled by at least two independent annotators, and differences were adjudicated by experts.

the semantics of the phrase also includes EXPE-RIENCER-thus, it seems inappropriate to force a choice between EXPERIENCER and POSSESSOR for this token.


We gratefully acknowledge the Alexander von Humboldt Foundation for an Anneliese-Maier prize and grant to Pelletier, and the Deutsche Forschungsgemeinschaft (KI-759/5) grant to Kiss, for their support of the work reported here and our other reports.

there are six of these tests in all, four syntactic and two semantic, chosen for their relevance to various of the issues that are salient in the studies of +MASS and +COUNt terms.


b. invention: the process of generating some new type of thing vs. the actual kind of thing that has been generated.

without a detailed account of the wide range of senses of the component nouns, it will be impossible to give the desired group of semantic rules.


COMPLETE requires that each of the nodes ofē i in R(p) have been identified, these nodes match up with the corresponding external nodes of the subgraph J, and that the subgraphs I and J are edge-disjoint.

(strings) by restricting the form of the productions in the grammars.


We count instantiations of COMPLETE for an upper bound on complexity (McAllester, 2002), using similar logic to (Chiang et al., 2013).

given a graph g, a path in g from a node v to a node v is a sequence The length of this path is k. A path is terminal if every edge in the path has a terminal label.


Then, to introduce distractors, we sample a random nominal from a unigram noise distribution.

according to his model, the predicate embeddings are distributed as: where W is a K × d matrix of weights and σ V is a hyperparameter.


In practise, the likelihood component is optimized using negative sampling with EM for the latent slots.

while this step is not strictly required, we found that it leads to generally better results than random initialization given the relatively small size of our predicate-argument training corpus.


Since this is a data driven approach, we identify non-core roles as well, if they occur with predicates often enough in the data.

for example, we have relational indicators such as "L-on", "R-before", "L-because", "R-None", etc.


In order to circumvent the manual creation of patterns, nu-merous approaches have been investigated to derive patterns automatically.

for our experiments on this corpus, we divided the full set of patterns in the corpus (around 500 per relation) into two equally-sized portions, one for creating training data for the RTE engine, and one for evaluating pattern selection methods.


For the marriage relation, the results not only show that our RTE engine adaptations yielded a much higher recall (with almost no loss in precision) than the original implementation (thanks to an increased number of relevant alignments), but also that pattern selection can in fact benefit from the graph structure: Entailment graphs created using MultiAlignAdapted achieved much better performance than a selection based on pair-wise entailment computation using the same RTE model.

it cannot express that the relation expressed by pattern P3 entails the relation expressed by pattern P1, but not vice versa.


Tree-based LSTM models have been shown to often perform better than purely sequential bi-LSTMs (Tai et al., 2015;Miwa and Bansal, 2016), but depend on parsed input.

adding genre information does not help, but adding attention within the local clause yields an improvement of 2.44 percentage points (pp).


For writing samples dataset, syntactic features are found to be the most successful in classification whereas for the less structured Twitter dataset, a combination of features performed the best.

negative sentiment could also be prominent in other psychiatric diseases such as post-traumatic stress disorder (PTSD) , as such, by itself, it may not be a discriminatory feature for schizophrenia patients.


We thank Dr. Michael Roth for conducting the semantic role labelling evaluation using features from our model to see whether it is beneficial.

we design a multi-task model (NNRF-MT) which shares the event participant embedding for the event context and tackles role-filler prediction and semantic role classification simultaneously.


The thematic fit judgements from the tasks discussed in section 6 only contain ratings of the fit between the predicate and one role-filler.

accuracy p-value NNRF-MT 89.1 -RoFa-MT 94.8 < 0.0001 ResRoFa-MT 94.7 < 0.0001 Table 1: Semantic role classification results for the three multi-task architectures.


Because we use random parameter initialisation, to observe its effect to our evaluations, we train 10 instances of each model and report average performance (we do not use these 10 models as an ensemble method such as labelling by majority voting).

other event participants contained in a clause can affect human expectations of the upcoming role-fillers.


Alternatively to applying nonparametric operators on word embeddings to create sentence embeddings, recurrent neural networks can learn the optimal weight matrix that can produce an accurate sentence embedding when repeatedly applied to the constituent word embeddings.

our experimental results show that the proposed NWS scores outperform baseline methods, previously proposed word salience scores and sentence embedding methods on a range of benchmark datasets selected from past SemEval STS tasks.


We use SGNS, CBOW and GloVe word embedding learning algorithms to learn 300 dimensional word embeddings from the Toronto books corpus.

likewise, for a randomly selected sentence S j / ∈ {S i−1 , S i , S i+1 }, the expect similarity between S j and S i would be low.


Using t and h(s i , s j ) above, we compute the cross-entropy error E(t, (S i , S j )) for an instance (t, (S i , S j )) as follows: Next, we backpropagate the error gradients via the network to compute the updates as follows: Here, we drop the arguments of the error and simply write it as E to simplify the notation.

in principle, we can learn NWS scores using any pre-trained set of word embeddings.


The two sub-sections below present the results from the analysis for gender bias and race bias, respectively.

the bias can originate from any or several of these parts.


The Equity Evaluation Corpus and the proposed methodology to examine bias are not meant to be comprehensive.

8 Each point (L, M, G) on the plot corresponds to the difference in scores predicted by the system on one sentence pair.


(2018) continued the work in this direction recently and applied convolutional neural networks to cross-lingual EL.

table 3 lists the main selected hyperparameters for the VCG model 6 and we also report the results for each model's best configuration on the development set in table 2.


We extract all entities that are mentioned in the question from the SPARQL query.

the parameter sharing between mention encoding and entity label encoding ensures that the representation of a mention is similar to the entity label.


We extract gold entities for each question from the SPARQL query and map them to Wikidata.

such approaches can not handle entities of types other than those that are supplied by the named entity recognizer.


(2014), we expect concrete target words to have significantly less diverse context dimensions than abstract target words, as the former should co-occur within a restricted set of context words.

more recently, embodied theories of cognition have suggested that word meanings are grounded in the sensory-motor system (Barsalou and Wiemer-Hastings, 2005;Glenberg and Kaschak, 2002;Hill et al., 2014;Pecher et al., 2011).


In this work, we provide a detailed characterisation of the distributional nature of abstract and concrete words across 16,620 English nouns, verbs and adjectives.

we expect concrete words to occur in a limited set of different contexts because there is only a restricted amount of words that have a high probability to fit a specific context.


For future work, we would like to evaluate the performance of EmoWordNet on larger datasets and we would like to improve the accuracy of the recognition model.

while sentiment is usually represented by three labels namely positive, negative or neutral, several representation models exist for emotions such as Ekman representation (Ekman, 1992) (happiness, sadness, fear, anger, surprise and disgust) or Plutchik model (Plutchik, 1994) that includes trust and anticipation in addition to Ekman's six emotions.


Mohammad and Turney (2013) presented challenges that researchers face for developing emotion lexicons and devised an annotation strategy to create a good quality and inexpensive emo-tion lexicon, EmoLex, by utilizing crowdsourcing.

the second approach is cheap and results in large scale emotion lexicons but with lower accuracy compared to manually developed emotion lexicons in terms of accurately representing the emotion of the term.


So it is not surprising that cross-language extensions of such models (crosslanguage word embeddings) rapidly gained popularity in the NLP community (Vulić and Moens, 2013), proving their effectiveness in certain crosslanguage NLP tasks (Upadhyay et al., 2016).

the task is to predict the similarity score for a word a in language A and a word b in language B.


Similarly, we observed that increasing the number of interaction types M does not guarantee consistent performance gain.

for each Implementation-wise, we vertically stack G for M times to constructG ∈ R M d×3 , and use each row g i as input to Eq.


We are grateful to Tagyoung Chung and Valeria de Paiva for helpful comments on presentation, though they should not be held responsible for the views expressed.

but the need to assert a negative rarely arises: better to wait until the corresponding incompatible positive is known, or as a last resort make up a positive fact that is incompatible with negative (e.g.


This is not directly comparable to existing work because it is a unique setup; but we note that this is likely a difficult task because of the large output space.

in our preliminary work, we experimented with CNN and RNNbased architectures but their performance was inferior to the model described here both in terms of accuracy and speed.


Problems Our end-to-end system is sound in the sense that it polarizes the correctly input semantic representations.

we are able to take simple sentences all the way through.


The ability of the system to successfully cluster instances varies from one gold frame category to another one.

for input strings in each cluster E i , we instantiate a new G i and perform parameter fitting and splitting independently of other E i s. The corresponding probabilistic G i is initialized by assigning random parameters to its rules and then smoothing them ( § 3.1.3) by the fitted parameters for G. We apply the aforementioned process several times, until the number of independently generated clusters is at least twice as large as |T v |.


Then with (1, 2)=before and (2, 3)=vague, we can see that (1, 3) cannot be uniquely determined, but it is restricted to be selected from {bef ore, vague} rather than the entire label set.

3 Joint Learning on F and P In this work, we study two learning paradigms that make use of both F and P. In the first, we simply treat those edges that are annotated in P as edges in F so that the learning process can be performed on top of the union of F and P. This is the most straightforward approach to using F and P jointly and it is interesting to see if it already helps.


Edges are presented one-byone and the annotator has to choose a label for it (note that there is a vague label in case the TempRel is not clear or does not exist).

in practice, every annotation comes at a cost, either time or the expenses paid to annotators, and as more edges are annotated, the marginal "benefit" of one edge is going down (an extreme case is that an edge is of no value if it can be inferred from existing edges).


3 While incorporating transitivity constraints in inference is widely used, Ning et al.

since annotators are not required to label all the edges in these datasets, it is less labor intensive to collect P than to collect F. existing TempRel extraction methods only work on one type of datasets (i.e., either F or P), without taking advantage of both.


(1), as was done by (Bramsen et al., 2006;Chambers and Jurafsky, 2008;Denis and Muller, 2011;Do et al., 2012;Ning et al., 2017).

p does not contain any vague training examples, so System 3 would only predict specific TempRels, leading to a low precision.


In this work, we adopt a corpus linguistics approach in a similar way to Antoniak and Mimno (2018), Hamilton et al.

the POS also has a high impact on the model trained.


concerning these two last features (polysemy and entropy) experiments confirm that distributional semantics has more difficulty in representing the meaning of words that appear in a variety of contexts.


For ACL and PLOS we also noticed that words belonging to the transdisciplinary scientific lexicon remained stable (conjunctive adverbs such as nevertheless, moreover, furthermore and scientific processes such as hypothethize, reason, describe).

we are aware that this measure assess only a part of what has changed from one model to the other based on the number of neighbors observed.


This setup is more realistic than the random and lexical split setups, in which the classifiers can benefit from memorizing verbatim words (random) or regions in the vector space (lexical split) that fit a specific slot of each relation.

this setup is more realistic than the random and lexical split setups, in which the classifiers can benefit from memorizing verbatim words (random) or regions in the vector space (lexical split) that fit a specific slot of each relation.


2016where we apply an long short term memory (LSTM) (Hochreiter and Schmidhuber, 1997) layer over the characters of a word and then concatenate the output with a word embedding to create a word representation that combines both character-level and word-level information.

hand-crafting these lexicons is time-consuming and expensive and so these resources are often either unavailable or sparse for many domains and languages.


The score p(l|w) of the words shown for SNLI deviate strongly, regardless of the label.

we find that this baseline can perform above the majority-class prior across most of the ten examined datasets.


We collected annotations for 4586 contextcontinuation pairs, collecting the following three criteria for each pair: • Overall quality (O): a subjective judgment by the annotator of the quality of the continuation, i.e., roughly how much the annotator thinks the continuation adds to the story.

we develop simple systems based on recurrent neural networks and similarity-based retrieval and train them on the ROC story dataset .


We attempted to replace the PMI features by similar features based on word embedding similarity, following the argument that skip-gram embeddings with negative sampling form an approximate factorization of a PMI score matrix (Levy and Goldberg, 2014).

we use the initial data release of 45,502 stories.


Our model can be applied to any new terms in any domain, given some context of the term usage and their domainagnostic definitions.

this is expected to generate more indicative features than a similar work (Shwartz et al., 2016), which simply concatenated distributional models with path-based models; • HYPERDEF employs definitions to provide richer information for the terms.


One possible explanation for comparably low performance of most similar assertion is that the past assertions are not sufficient to make a meaningful prediction.

we attribute this to the systematic error made by the SVM when trying to 10 https://github.com/muchafel/judgmentPrediction  predict the similarity of assertions that have a negative agreement score.


We will refer to such utterances as assertions.

with an accuracy of about 85% the most similar user strategy performs best.


In Figure 1, we show this case for binary (yes/no) predictions on individuals and argue that this can be also generalized to probabilistic predictions on groups of people.

we capture the wording of assertions by different ngram features.


for the SNN, the predictions follow a similar pattern: resembling a learning curve, the performance increases rapidly with increasing n, but then plateaus from a certain number of assertions.


Furthermore, the entity embeddings tend to be very similar to the embeddings of relations in which they frequently participate.

we applied L 2normalization on the generated embeddings.


Please note that a direct comparison with IKRL (Paper) is not possi-   ble since we do not have access to the same set of negative samples.

such a representation suffers from limited discriminativeness and can be considered a main source of error for different KG inference tasks.


We have illustrated the techniques by looking at the roles in the TRIPS framework.

semantic roles have a long history, originating in linguistics as thematic roles (e.g., Fillmore, 1968;Dowty, 1991) and widely adopted in computational linguistics for semantic representations because of their compatibility with frame-based and graph-based (i.e., semantic networks) representations of meaning.


Note that based on the definitions of the roles, it is common that the same role appears more than once in a sentence.

for example, VerbNet would treat "the table" as the DESTINATION without explicit representation of the spatial relation (under).


There is an argument to this verb that is the judgement, but this is not signaled by an elided argument or an indefinite pronoun.

loosely speaking, when +causal obtains (for an AGENT o ag ), there exists some scale sc * such that a change of o ag on the sc * scale would entail a change of the AFFECTED o aff on the sc _E scale.


For short-text classification we use the Stanford sentiment treebank (TR) 3 , customer reviews dataset (CR) (Hu and Liu, 2004), subjective dataset (SUBJ) (Pang and Lee, 2004) and movie reviews (MR) (Pang and Lee, 2005).

to address the feature sparseness problem encountered in short-text and cross-domain classification tasks, we propose a novel method that computes related features that can be appended to the feature vectors to reduce the sparsity.


To overcome feature-sparseness in training and test instances, we expand features that are cores by their corresponding peripheral sets.

can be used to compute the weights assigned to the edges.


For short-text classification (Figure 2a), the accuracy increases for k ≥ 100 (TR and CR obtain the best for k = 1000).

we decompose a feature-relatedness graph into core-periphery (CP) structures, where a core feature (a vertex) is linked to a set of peripheries (also represented by vertices), indicating the connectivity of the graph.


Then, the detected peripheral vertices may be densely interconnected because they belong to the same community.

we use the Kernighan-Lin's algorithm (Kernighan and Lin, 1970) to find a good (but generally a suboptimal) solution.


The contexts we care about are those that are shared.

depending on whether we are talking about bed sizes, these two items are either closely related or completely unrelated, and thus context dependent.


Experiments on the development set showed that target word BERT representations and USE sentence embeddings are the best-suited for WiC.

following this procedure, context2vec produces a ranking of candidate substitutes for each target word instance in the Usim, CoInCo and WiC datasets, according to their fit in context.


Apart from featurization, loss functions and normalization of probability are also design choices available.

we can featurize both inputs and outputs -in our case, contexts and words.


However, we hypothesize that each relation may contain useful information about the others, and training on only one relation inevitably neglects some relevant information.

in the multitask setting, relations are presented to the model in the order they are listed in the result tables within each batch.


Our representation transfer framework is very similar to their approach, although we use a simpler loss function.

mono-lingual evaluation of sentence representation models can be found in Hill et al.


We surmise that only a subset of semantic features were learned by the InferSent objective given the specific characteristics of the SNLI training sets.


(2018a) decomposes the Parsing Time Normalizations task into two subtasks: a) time entity identification using a character-level sequence tagger which detects the spans of characters that belong to each time expression and labels them with their corresponding time entity; and b) time entity composition using a simple set of rules that links relevant entities together while respecting the entity type constraints imposed by the SCATE schema.

pre-trained language models (LMs) such as ELMo (peters et al., 2018), ULMFiT (Howard and Ruder, 2018), OpenAI GpT (Radford et al., 2018), Flair (Akbik et al., 2018) and Bert (Devlin et al., 2018) have shown great improvements in NLp tasks ranging from sentiment analysis to named entity recognition to question answering.


The SemeEval task description paper (Laparra et al., 2018b) has more details on dataset statistics and evaluation metrics.

there is a need to study pre-trained contextualized character embeddings, to see if they also yield improvements, and if so, to analyze where those benefits are coming from.


The structure-based representation, however, seems to capture bot variability more effectively, i.e.

all hidden layers consists of ReLU activation units, and are regularized using dropout rate of 0.5.


During training, the input to this network is the representation of a conversation (either contentbased or structure-based), and the ground truth is a one-hot vector of the bot that handled this conversation.

in addition, if the bot was annotated as not-production, the experts had to provide a list of reasons for their choice (e.g., repeating users ids, repeating bot response, etc.).


We maintain a 60%-40% train-test split over the corpus, and average accuracy over 5 runs, varying the number of topics between 5 and 25.

using Tf-Idf weighting in conjunction with a Word2Vec representation helps alleviate issues that the individual representations face when used independently.


We therefore ran a second set of qualitative experiments in which LDA topics were used to derive clusters of similar documents.

first, we associate each word w ∈ W with a set of documents, D(w), based on their similarity in the embedding space.


We presented MCScript2.0, a new machine comprehension dataset with a focus on challenging inference questions that require script knowledge or commonsense knowledge for finding the correct answer.

some answers can still be read off the text, if other parts of the texts contain the same information as the hidden target sentences.


As a more balanced and relevant test set, we use noun pairs (666 total) from the SimLex999 semantic similarity dataset (Hill et al., 2015).

the employed WSD algorithm starts with building a graph where the nodes are the WordNet synsets of the words in the input sentence.


We did not compare our approach to the GraphSAGE embeddings (Hamilton et al., 2017b) and Graph Convolutional Networks (Schlichtkrull et al., 2018), since they make use of input node features, which are absent in our setup.

additionally, graph embeddings can be of importance in privacy-sensitive network datasets, since in this setup, explicitly storing edges is not required anymore.


Particularly, in the case of WordNet, each node (synset) has 36 synsets in its V 2 on average, and half of the nodes do not have any neighbors at all.

raw WordNet similarities are still the best, but the path2vec models are consistently the second after them (and orders of magnitude faster), outperforming other graph embedding baselines.


We use negative sampling to form a training batch B adding n negative samples (s ij = 0) for each real (s ij > 0) training instance: each real node (synset) pair with zero similarities, where v k and v l are randomly sampled nodes from V .

we additionally used human-annotated semantic similarities from the same SimLex999.


We consider the domain adversarial training network (Ganin et al., 2016) (DANN) on the user factor adaptation task.

we want to keep the feature space such that the features are predictive of document classes in a way that is invariant to demographic shifts.


Different demographic groups can show substantial linguistic variations, especially in online data (Goel et al., 2016;Johannsen et al., 2015).

because of this, the topics all look very similar and are hard to interpret, so we do not show the topics themselves.


In several cases, the algorithm has discovered "parallel" relations.

many datasets suffer from paths sparsity, lack of enough paths connecting source target pairs, resulting in poor performance for many relations.


informative about causality or related probabilities), then they should be in the same form as that knowledge: the alternative, to keep rule-like topoi apart from knowledge about rule-based(ish) systems, is counter-intuitive.

both intuitively and according to experimental evidence, positive acceptability judgements can still be made without fore-knowledge of such a connection.


Compared to traditional pair-wise temporal relation representations, temporal dependency trees facilitate efficient annotations, higher inter-annotator agreement, and efficient computations.

our training data consists of two parts.


The performance on *-*-idf runs suggest that random word selection is as good or better.

from Table 1, we observe that adding noise is necessary for good performance, as we see that the various noise strategies consistently improve performance over the baseline on both the datasets.


So are other image transformations such as translation, rotation, removing color and so on.

to reduce computational overhead, we filtered out entity mentions which were greater than length 5 from the Ontonotes dataset (4 respectively for CoNLL), and contexts which were greater than length 59 or smaller than length 5 (40 and 3 respectively for CoNLL).


We say that a mention is a person-mention if the head of the mention is a PER named entity, and we say that the name of the person-mention is the PER named entity that is its head.

we restrict our attention to GPE entities that satisfy the following requirements: (1) they occur in the GeoNames database and (2) they are not countries.


For each dataset, highest score for each dataset is bolded and is underlined if the difference between it and the other model's score is statistically significant (p < 0.20 per a stratified approximate randomization test similar to that of (Noreen, 1989)  the system must determine whether a given pronoun refers to one, both, or neither of two given names.

(Again, we use the gold NER to identify GPE names in the CoNLL text.)


first names with male proportion less than or equal to 0.5 in the gazetteer) by F. We remove all names occurring in training from L, M, and F. We use the spaCy dependency parser (Honnibal and Johnson, 2015) to find the heads of each mention.

changing the name of an organization while ensuring that it is compatible with nominals in the cluster is nontrivial without a finer semantic typing.


We explore three classification models: logistic regression, instance-based learning, and prototypical neural networks (Snell et al., 2017).

for each event expression the semantic classifier generates 50 semantic features.


Our findings indicate that debiasing methods that need explicit set of words to be debiased are unlikely to be effective in removing all stereotypelike data.

we collected human judgments about a person's Big Five personality traits formed solely from information about the occupation, nationality or a common noun description of a hypothetical person.


(2019) have demonstrated the capacities of contextualized word embeddings across a wide variety of tasks, including SPRL.

model Ablations All ablation experiments are conducted with markerEB in the multi-label formulation.


The significance test results summarized in Table 4 are unambiguous: for many proto-role properties, ensembling helps to improve performance significantly (SPR1: 14 18 cases; SPR2: 7 14 cases; significance level: p < 0.05).

in addition, our model as a single model instance (Marker) is outperformed by RUD'18's approach both in the regression and in the multi-label setup.


We thank our three anonymous reviewers for helpful suggestions.

downward monotonicity inferences are interesting in that they allow to replace a phrase with a more specific one and thus the resulting sentence can become longer, yet the inference is valid.


Notice that this is not a case of epistemic uncertainty.

of our treatment of VP negation, the universal quantifier ("all") and the existential quantifier ("some") are not interdefinable, as they are in classical logic.


Other research has also identified demographic keys closely associated with vulgarity: Wang et al.

that data were designed for use in social science research, not natural language processing research, and thus there were several challenges in working with the data as they were collected, including: • The comments were saved in PDFs, and the metadata referenced each comment by a number that was drawn (not typed) into the PDF beside the comment.


In particular, to extract information, Clause-Based Open IE systems (Del Corro and Gemulla, 2013;Angeli et al., 2015;Schmidek and Barbosa, 2014) reduce a complex sentence into simpler sentences using linguistic patterns.

we use a pretrained Semantic Role Labelling model 4 based on a Bi-directional LSTM network (He et al., 2017) with pre-trained ELMo embeddings (Peters et al., 2018).


A manual examination of the results also showed the same trend.

nevertheless, it is important to assess how our system is performing.


3 At prediction time, in both the discriminative and the generative cases, we find the most likely label sequence using Viterbi decoding.

while widely differing in the specific model structure and learning objective, all of these approaches achieve excellent results.


While widely differing in the specific model structure and learning objective, all of these approaches achieve excellent results.

there is considerable variation between languages: Spanish has the highest coverage with over 90%, while Turkish, an agglutinative language with a vast number of word forms, has less than 50% coverage.


As for the HMM, Y(x) is not necessarily the full space of possible tag-sequences; specifically, for us, it is the dictionarypruned lattice without the token constraints.

as we will observe next, coupling the dictionary constraints with token-level information solves this problem.


We use sentences from the TACOS corpus and record their timestamps.

correspond to low-level actions, and each sentence is aligned with the last of its associated low-level actions.


In this way, we replace z c by several z E i that can be handled by our bounding strategy.

the first three types of features are firstly introduced by McDonald et al.


Our work differs in the use of joint learning and inference approaches.

learning parameters were tuned using cross-validation on the training set: the margin δ is set to 1, the GEN lEX margin δ l is set to 2, we use 6 iterations (8 for experiments on SAIl) and take the 250 top parses during lexical generation (step 1, Figure 5).


(2011) used a graphical model semantics representation to learn from instructions paired with demonstrations.

figures 3 and 4 include a sample of our seed lexicon.


Our inference includes an execution component and a parser.

we show that, given only a small seed lexicon and a task-specific executor, we can induce high quality models for interpreting complex instructions.


We also separated clitics from their base word.

first, they switched from MLE to a Bayesian approach, estimating a probability distribution over model parameters θ and dependency trees T given the training corpus C and a prior distribution α over models: Headden et al.


(2010), both trained and tested on the length 10 training data from the CoNLL-X Shared Task.

a direct comparison between dependency treebanks and dependencies produced by CCG is more difficult (Clark and Curran, 2007), since dependency grammars allow considerable freedom in how to analyze specific constructions such as verb clusters (which verb is the head?)


The algorithm first initializes G to Φ and X to SU .

existing supervised approaches seldom exploit the intrinsic structure among sentences.


The comparative results in Table 5 clearly show that while our vanilla seed lexicon performs comparably to off-the-shelf lexicons on our data, the paraphraser-expanded lexicon with sentitment profiles outperforms OpinionFinder, General Inquirer, and SentiWordNet.

tESt set contains the 43 agreed double-annotated sentences, and additional 238 sampled from the 500 single-annotated sentences, 281 sentence in total.


We produce 4 features, two for each polarity: (1) the number of words such that 0 ≤p pos w < 0.4; (2) the number of words such that 0.4 ≤p pos w ≤ 1; similarity for the negative polarity.

we implemented a WordNet (Miller, 1995) based expansion that uses the 3 most frequent synonyms of the top sense of the seed word (WN-e).


In the following, we provide a more formal characterization of the strong and weak generative power of ITSG with respects to context-free grammar (CFG) and TSG.

6 This restriction is compensated for by the existence of the Forward Substitution operation, which has no analog in the Earley algorithm.


A syntax-based noise model may achieve better performance in detecting and correcting child word drops.

we then compute the total arc weight of all paths through F ST train by relabeling all input and output symbols to and then reducing F ST train to a single state using epsilon removal (Mohri, 2008).


We run EM for 100 iterations, at which time the log likelihood of all sentences generally converges to within .01.

our model has 6,718 parameters, many more than the ESL model's 187.


a ↷ b denotes that a tree b is attached to a tree a. rated in standard graph-based models.

• PP-Attachment features: when a parent word is a preposition, we define tri-gram features with the parent word and POS tags of grand parent and the rightmost child.


This issue of "misleading" WP links becomes even more prominent when the links from the full articles are used as edges (LM); while the increase in recall is relatively small the precision drops substantially.

this is by far the high-est value across all resources (see table 4).


Wikipedia (WP) is a freely available, multilingual online encyclopedia.

intuitively not all links in an article are equally meaningful.


E.g., the two senses of letter "The conventional characters of the alphabet used to represent speech" and "A symbol in an alphabet, bookstave" (taken from WN and WKT, respectively) are clearly equivalent and should be aligned.

for the SR configuration, we decided to retain only the category links and the links within the first paragraph of the article.


Though there is some empirical work on competitive assignments in the computer science education literature (Lawrence, 2004;Garlick and Akl, 2006;Regueras et al., 2008;Ribeiro et al., 2009), they generally measure student satisfaction and retention rather than the more difficult question of whether such assignments actually improve student learning.

we ensured that our data met this criterion.


This transformation does not exactly preserve meaning, but still captures the most important relations.

from the following sentences: All of these require knowledge of lexical semantics (e.g.


They do not require any knowledge of lexical semantics, meaning we can evaluate the formal component of our system in isolation.

almost all errors are due to incorrectly predicting unknown -the system makes just one error on yes or no predictions (with or without gold syntax).


The terminal features are indicator features for each lexicon entry, as shown in the top row of Figure 4.

1 Weakly supervised training estimates parameters for LSP using queries annotated with their denotations in an environment (Figure 1c).


In this example, the denotation is the set of "things to the right of the blue mug," which does not include the blue mug itself.

each held-out sentence z i is parsed to produce a logical form , which is marked correct if it exactly matches our manual annotation i .


These definitions and related prepositions provide a starting point to identify senses that can be merged across prepositions.

for the governor and object, we have a set of type labels, comprised of one element for each type category.


All the probabilities in the above formulas are computed by Equation 3.

this subject will be one of our future work topics.


With the same settings as before, we run the Gibbs sampler for 1000 iterations and utilize the final U-tree structure to build a string-to-tree translation system.

with SCFG, we have to discard all the internal nodes (i.e., flattening the Utrees or rules) to express the same sequence, leading to a poor ability of distinguishing different U-trees and production rules.


For a formal definition, see Johnson et al.

the template that obtained the highest score is then chosen.


The most notable differences are in Turkish, where all models perform far worse on the test than dev set.

first, we trained 5 samplers on the 50k training set with labelled set added, and used the labelled data to choose the best template for each inferred grammar.


The next two rules collect a new dependent with a gap and embed it within the gap of our lower tree, creating a new dependency.

oEi; j ; p 0 ; q; j =LR > oEp; p 0 C 1; ; ; p 0 C 1 0 oEi; j ; p; q; j U˚a j !


Alternatively, one can easily also use fully-automatic bootstrapping techniques based on seed word pairs (Hearst, 1992;Chklovski and Pantel, 2004;Yang and Su, 2007;Turney, 2008;Davidov and Rappoport, 2008).

good but not great) to reveal order information between a pair of adjectives (Sheinman and Tokunaga, 2009).


Gibbs sampling is another effective algorithm for unsupervised learning problems.

aERs on the testing set are listed in Table 3 6.


The head-modifier cohesion term ℎ is used to penalize the distortion probability according to relationships between the head node and its children (modifiers).

finally, we conclude this paper and mention future work in Section 7.


The model assumes each source word is assigned to exactly one target word, and defines an asymmetric alignment for the sentence pair as 1 = 1 , 2 , … , , … , , where each ∈ [0, ] is an alignment from the source position j to the target position , = 0 means is not aligned with any target words.

in our model, the word distance is calculated based on the previous node in BUTorder rather than the previous word in the original sentence.


They do not overlap each other, so the head-modifier cohesion is maintained.

the training procedure is very timeconsuming, and they trained the model with only 100 hand-annotated sentence pairs.


We can use auxiliary symbols to denote the head phrase position in a CFG rule.

(2011) respectively evaluated their graph-based and transitionbased parsers; Zhang and Clark (2011) evaluated CoNLL-test UAS (Li et al., 2012) 83.23% Graph+Tran+FlatH+BinH+BinLR 87.23% CTB5-test UAS (Li et al., 2011) 80.79% (Hatori et al., 2011) 81.33% (Zhang and Clark, 2011) 81.21% Graph+Tran+FlatH+BinH+BinLR 84.65% Table 6: UAS of different models on the test data.


Before proceeding to the SMT experiments, we evaluate the performance of the WaW reordering model in isolation.

6 According to how the DL, pruning parameters, and ϑ are set, we can actually aim at different targets: with a low DL, loose pruning parameters, and ϑ=0 we can try to speed up search without sacrificing much translation quality.


This is an important result, considering that the jump length, strongly correlating with the jump likelihood, is not directly known to our model.

the need of specific reordering rules makes the method harder to apply to new language pairs.


As regards efficiency, the new model makes decoding time increase by 8%.

we report statistically significant improvements in the reordering of verbs, which is where the impact of our method is expected to concentrate (+0.7, +0.8 and +1.0 KRS-V on eval08-nw, eval09-nw and vs-09, respectively).


We do this calculation only for certain types of syntactic relations-a) nouns and their adjective modifiers, b) verbs with adverb modifiers, c) adjacent nouns in a noun phrase and d) verb and subject pairs.

we identified the relations for texts in our corpus using the AddDiscourse tool (Pitler and Nenkova, 2009).


Several other properties of science writing could also be relevant to quality such as the use of humor, metaphor, suspense and clarity of explanations and we plan to explore these in future work.

if it appears only with 'who' in all noun phrases, it is animate.


For each pair, we generate function similarity features, Fun(x i , x j , k, p), where k and p vary as they did with domain space.

section 6 discusses the implications of the results.


We utter about one metaphor for every ten to twenty-five words, or about six metaphors a minute (Geary, 2011).


c. Max idf of matching base form: max idf of a word corresponding to a matching base.

furthermore, users could provide other kinds of feedback such as moving items between clusters, allowing for the opportunity for relevance feedback; classifier uncertainty could be surfaced to the user, and active learning approaches could be used to ask the teacher for specific labels.


This process is iterated until the clusters converge.

finally, Pulman and Sukkarieh (2005) compare hand-authored patterns with machine learned patterns based on simple word-based features, and find that the hand-crafted patterns perform better.


We show results in terms of grading progress with a small "budget" of human actions, both from our method and an LDA-based approach, on a test corpus of 10 questions answered by 698 respondents.

instead of classifying individual answers as being right or wrong, we propose to automatically find groupings and subgroupings of similar answers from a large set of answers to the same question, and let teachers apply their expertise to mark the groups.


If t p is correct, we simply apply it and move to t p (c) without changing the model parameters (line 11).

they are still only guaranteed to be correct on a subset of the possible configurations.


(2006), using a greedy arc-eager transitionbased system with pseudo-projective parsing.

interestingly, however, for both these languages there is nevertheless a small improvement in the joint PM score, indicating that the JOiNT model in general does a better job at selecting a valid complete morphological description than the SiMPLETAG model.


Turning to the results for SIMPLETAG, we note that our results are consistent with those reported by Bohnet and Nivre (2012), with small but consistent improvements in POS and UAS/LAS (and in the compound metrics PM and PMD) for most languages.

we use the open-source morphological analyzer OMorFi (Pirinen, 2011) and word clusters derived from the entire Finnish wikipedia.


The story is more complicated for seen words with known translations: if we limit ourselves to "high confidence" translations, there is a lot to be gained by improving the scores in translation models.

but this is based on intersected phrase tables, from which we removed seen and sense distinctions, and in which there is no competition between phrases from the OLD and NEW systems.


Similarly, in the Science domain, the French word 'finis,' when it should have been translated as 'finite,' was translated incorrectly due to a sense error 27 times.

in contrast to that work, we do not directly align the hypothesis and reference translations but, rather, pivot through the source text.


For an evaluation of these topics see Section 4.3.1.

given the top M words ) for a topic t, the coherence of that topic can be calculated with the following formula: , where D(v) is the number of documents containing v and D(v, v ) is the number of documents containing both v and v .


Although labels have a one-to-one correspondence with languages, the label distribution does not actually correspond directly to the language proportion, because the distribution estimates the proportion of byte n-gram sequences associated with a label and not the proportion of bytes directly.

we propose a method that concurrently detects that a document is multilingual, and estimates the proportion of the document that is written in each language.


17 Code from (Burkett et al., 2010) is obtained through personal communications.

in a multilingual setting, coming up with effective constraints require extensive knowledge of the foreign 1 language.


We distinguish the two senses in a preprocessing step: if the other captions of the same image do not mention children, but refer to teenaged or adult women, we assign girl the woman-sense.

this variety of descriptions associated with the same image is what allows us to induce denotational similari-ties between expressions that are not trivially related by syntactic rewrite rules.


This suggests that learning this pattern correctly requires considerably more input than for the other patterns.

by combining different kinds of cues, e.g.


The sole exception in this respect is Doyle and Levy (2013) who added stress cues to the Bigram model , demonstrating that this leads to an improvement in segmentation performance.

we find that phonotactic cues to word-boundaries interact with stress cues, indicating synergistic effects for small inputs and partial redundancy for larger inputs.


If together with its neighbors it can match the selected label, the assignment is finalized.

results: Table 4 compares the results for two baseline systems-standard EM (method 1), and a previously reported system using model minimization (method 2) for the same task.


We propose a new method for unsupervised tagging that finds minimal models which are then further improved by Expectation Maximization training.

this data was created by semi-automatically converting the Penn treebank to CCG derivations (Hockenmaier and Steedman, 2007).


• We show how to efficiently parallelize the algorithm while preserving approximation guarantees.

previous work (Ravi et al., 2010a;Ravi et al., 2010b) recognized this challenge and employed two phase heuristic approaches.


We also write r(σ[i]) to denote the root of t Informally, the LR-spine parser uses the same transition typologies as the arc-standard parser.

for the arc-standard parser, the computation takes cubic time in the size of the largest of the left and right input stacks.


The path from t's root to node σ L [1] is called the spine of t. • Every node of t not in the spine is a dependent of some node in the spine.

our novel dynamic oracle algorithms rely on arguably more complex structural properties of computations, which are computed through dynamic programming.


This could not be applied to the other two disfluency detectors, so we cannot test those differences for significance.

speakers tend be disfluent in bursts: if the previous word is disfluent, the next word is more likely to be disfluent.


Deriving patient history timelines from clinical notes also involves these types of assumptions, but there are special demands imposed by the characteristics of the clinical narrative.

similarly, mentions of symptoms or disorders reflect occurrences in a patient's life, rather than abstract entities.


For unguided, general-purpose annotation, the number of relations that could be annotated grows quadratically with the number of events and times, and the task quickly becomes unmanageable.

based on the known progression-of-care for colon cancer, we can infer that the colonoscopy occurs first, biopsies occur during the colonoscopy, pathology happens afterwards, a diagnosis (here, adenocarcinoma) is returned after pathology, and resection of the tumor occurs after diagnosis.


eMRG with both Binary and Ternary Edges: If there are more than one entity mentions and at least one attribute mention in a sentence, an eMRG can potentially have both binary and ternary edges.

if not, the sentence is marked as off-topic.


In practice, however, differ-ent types of sentiment-oriented relations frequently coexist in documents.

a mention-based relation graph (or MRG ) represents a collection of mention-based relation instances expressed in a sentence.


An evidentiary mention-based relation graph, coined eMRG , extends an MRG by associating each edge with a textual evidence to support the corresponding relation assertions (see Figure 2).

since instance sets of sentiment-oriented relations (ssoRs) are the expected outputs, we can obtain ssoRs from MRGs by using a simple rule-based algorithm.


Then (y, h) corresponds to an eMRG, and (a, c) ∈ (y, h) is a labeled edge a attached with a textual evidence c. Given a labeled dataset D = {(x 1 , y 1 ), ..., (x n , y n )} ∈ (X × Y) n , we aim to learn a discriminant function f : X → Y × H that outputs the optimal eMRG (y, h) ∈ Y(x) × H(x) for a given sentence x.

these methods fall short of extracting comparative relations based on domain dependent information.


week") and a linear interpolation of "base all" and "base one" ("int one all").

algorithmically, our work comes closest to the online dynamic topic model of Iwata et al.


Additionally, we also compare with a base model whose counts decay exponentially ("base exp").

our experiments show that we are still able to learn reasonable parameter estimates by optimizingB.


1 We point out that our definition is unconstrained in terms of what to link, i.e., unlike Wikification and WSD, we can link overlapping fragments of text.

• The Senseval-3 dataset for English all-words WSD (Snyder and Palmer, 2004), which contains 899 nouns to be disambiguated using WordNet.


The results obtained by UKB show that the high performance of our unified approach to EL and WSD is not just a mere artifact of the use of a rich multilingual semantic network, that is, Ba-belNet.

the aim of EL is to discover mentions of entities within a text and to link them to the most suitable entry in a reference knowledge base.


Notwithstanding these differences, the tasks are similar in nature, in that they both involve the disambiguation of textual fragments according to a reference inventory.

but while the two tasks are pretty similar, they differ in a fundamental respect: in EL the textual mention can be linked to a named entity which may or may not contain the exact mention, while in WSD there is a perfect match between the word form (better, its lemma) and a suitable word sense.


On Wikipedia our results range between 71.6% (French) and 87.4% F1 (English), i.e., more than 10 points higher than the current state of the art (UMCC-DLSI) in all 5 languages.

we show that the semantic network structure can be leveraged to obtain state-of-the-art performance by synergistically disambiguating both word senses and named entities at the same time.


The only knowledge we use is a simple translation lexicon, that is, a list of translation pairs without translation probabilities, as shown in Figure 2.

our Croatian evaluation is a synonym choice task parallel to Exp.


It is proposed that the temporal model on social media needs to account for two factors: imitation and recency (Leskovec et al., 2009).

for evaluation, we compute the f1 score over the test set, as defined earlier (Guo et al., 2013).


Suppose that a marketing firm is interested in the sentiment about some product on Twitter.

next, we examine how these errors can be recovered in +T+L.


For the binning approach, we tune the bin size on the development set, ranging from 10 minutes to 1 day for time, and from 10 × 10 sqkm to 1500 × 1500 sqkm for location.

retrieving tweets via keyword queries inevitably mixes different entities.


The TempEval competitions (Verhagen et al., 2007;Verhagen et al., 2010;UzZaman et al., 2013b) aimed to improve coverage by annotating relations between all events and times in the same sentence.

the set of temporal relations are a subset of the 13 original Allen relations (Allen, 1983).


This can be seen in the performance of the ClearTK-TimeML system, which achieved the top performance (36.26) on the sparse relation task of TempEval 2013, but performed dramatically worse (15.8) on our dense relation task.

only a small subset of possible pairs are labeled.


While linguistic information overall effectively reflects the meaning of all concept types, we show that features encoding syntactic patterns are only valuable for the acquisition of abstract concepts.

in future work we aim to extend our experiments to concept types such as adjectives and adverbs, and to develop models that further improve the propagation and combination of extra-linguistic input.


8, which shows all the values at 15% evidence ratio.

for a Wikipedia editor to become an administrator, a request for adminship (RfA) 3 must be submitted, either by the candidate or by another community member.


(Note that at most one of the two sum factors of Eq.

assume we want to predict the sign of the edge (u, v) in the LOO setting, and that u, v, and w form a triangle.


It seems hard to improve by much on a sentiment model that achieves an AUC/ROC of 0.88 on its own; the Wikipedia corpus offers an exceptionally explicit linguistic signal.

we sketch a proof of this theorem in Appendix A: The objective function of Eq.


HITs were automatically approved after fifteen minutes.

then By normalizing (and rounding), 11 Although the majority vote on i is for category 1, the estimated probability that the category is 1 is only 0.11, given the adjustments for annotators' accuracies and biases.


Each item has sense labels from up to twenty-five different annotators, collected through crowdsourcing.

we conclude that a modelbased label derived from many Turkers is preferable to a label from a single trained annotator.


The MSM2013 corpus has 4 types of named entities, person (PER), location (LOC), organization (ORG), and miscellaneous (MISC).

qu and Liu (2012) used the pairwise CRF with a 4-connected neighborhood system (2D CRF) as their graphical model, where each vertex in the graph represents a sentence pair, and each edge connects adjacent source sentences or target sentences.


All the 4 taggers are trained using linear chain CRFs with perceptron training.

they could not be handled efficiently by belief propagation for spanning trees.


ee Congruence Constraints: To ensure that ch CKY cell has at most one symbol we require e also require that here R h = {r 2 R : r = h !

6 The number of positions is equal to the number of p types k=0$ two variables have been discussed by Clarke and pata (2008).


(dissonance(due( to(generaliza&on(error( Completely(wrong( Extraneous(informa&on( Vision(detec&on(error( Human:"A"delighGul"clock" in"the"town"centre"of"St" Helier"with"the"iconic"Jersey" cow"at"the"base."

pruning Case (2)/(3): Deletion of the left/right child respectively.


For example, S i = {NN,NP, ...} if p i corresponds to an "object" (noun-phrase).

note that some test instances include rules that we have not observed during training.


To give a specific example, their operator literal to constant is equivalent to having named entities for larger text chunks in our case.

the denotation 1 of the NL query graph is {AUSTIN}.


As seen in these examples, locally non-linear models (Eq.

this technique is sometimes referred to as binning and is closely related to quantization.


We introduce some additional terminology and notation that we will use in the proofs.

the set of derivation trees of a CCG can be formally defined as in Figure 2.


This sharing is the key to the polynomial runtime.

this idea actually underlies several parsing algorithms for equivalent mildly context-sensitive formalisms, such as tree-Adjoining Grammar (Joshi and Schabes, 1997).


The relative proliferation of inference rules, combined with the increase in their complexity, makes, in our own opinion, the specification of the V&W parser more difficult to understand and implement, and calls for a more articulated correctness proof.

for every degree d 0 there are 2 d forward rules and 2 d backward rules.


Preposition features are active if the NP is immediately preceded by a preposition.

table 14 compares models trained on native and learner data in their best configurations based on the training data.


The Illinois system improves its F1 from 31.20 to 42.14 on revised annotations.

it does this by generating additional artificial errors using the error distribution from the training set.


Practical systems, however, should be tuned for good precision to guarantee that the overall quality of the text does not go down.

when a very large native training set such as the Web1T corpus is available, it is often advantageous to use it.


We present MULTIP (Multi-instance Learning Paraphrase Model), a new model suited to identify paraphrases within the short messages on Twitter.

for example, the word pair "next new" in two tweets about a new player Manti Te'o to a famous former American football player Junior Seau: • Manti bout to be the next Junior Seau • Teo is the little new Junior Seau further note that not every word pair of similar meaning indicates sentence-level paraphrase.


For example, the word pair "next new" in two tweets about a new player Manti Te'o to a famous former American football player Junior Seau: • Manti bout to be the next Junior Seau • Teo is the little new Junior Seau Further note that not every word pair of similar meaning indicates sentence-level paraphrase.

y i is then set by aggregating all z i through the deterministic-or operation.


The additional images provide two important benefits: (1) an increasing degree of challenge to keep the player's interest, (2) more image interactions to use in producing the annotation.

in essence, accurate players with high α are more likely to be shown mystery gates that annotate pictures from U , whereas completely inaccurate players are prevented from adding new annotations.


The second type of gate, referred to as a mystery gate, shows three images that are potentially related to the clue.


continuously revises the annotation during gameplay based on which pictures players spare, the first analysis assesses how the accuracy changes with respect to the length of one Ka-boom!

puzzle Racer reduces the annotation cost to ≤27% of that required by crowdsourcing.


Each flight contains one randomly-selected picture for each of a word's n senses and n distractor images.

in our first future work, we plan to develop new types of video games for textual items as well as extend the current games for new semantic tasks such as selectional preferences and frame annotation.


Probabilistic modeling is a useful tool in understanding unstructured data or data where the structure is latent, like language.

vARIATIONAL runs fifty iterations, with the same truncation level as in ONLINE.


In this case, an NER system based on surface statistics alone would likely predict that Freddie Mac is a PERSON.

table 3 shows the results of adding each interaction factor in turn to the baseline and removing each of the three interaction factors from the full joint model (see Figure 4).


5 Its state merging criterion is based on the defining property of ISL functions: two input strings with the same suffix of length (k − 1) have the same tails.


A function f is ISL iff there is some k such that f can be described with a SFST for which This theorem helps make clear how ISL functions are Markovian: the output for input symbol a depends on the last (k − 1) input symbols.

each element of the left projection of S is of length (k + 1) and each element of its right projection is at most of length (k + 1) the size of the characteristic sample is in , which is clearly polynomial in the size of the target transducer.


Using a semi-Markov CRF, we model the conditional distribution over all possible opinion segmentations given the input x: where θ denotes the model parameters, y and f denotes a feature function that encodes the potentials of the boundaries for opinion segments and the potentials of transitions between two consecutive labeled segments.

semi-Markov CRFs (Sarawagi and Cohen, 2004) (henceforth semi-CRF) have been shown more appropriate for the task than CRFs since they allow contiguous spans in the input sequence (e.g.


16 Another source for phonetic problems occurs, if the prefix boundary splits the word in a way that leads to another pronunciation pattern compared to the solution word as in this example.

note that the automatic prediction is compared with the actual error rates, not the human predicated ones.


We check if the same solution also occurs in another gap to account for repetition.

our model has been developed for the difficulty prediction of English C-tests.


In addition, an expert policy π must be specified which is an oracle that returns the optimal action for the instances in S, akin to an expert demonstrating the task.

• Utterances that are within the scope of the system but too complex to be represented by the proposed MRL, e.g.


For these reasons, we converted the MR expressions into a node-argument form.


The datasets developed in the recent dialog state tracking challenge (Henderson et al., 2014) also consist of dialogs between a user and a tourism information system.

3 are not meaningful since fewer dialogs are being used for training and testing in the cross-scenario setup.


The loss function in DAGGER is only used to compare complete outputs against the gold standard.

since π rand is progressively ignored, the effect of such actions is reduced.


spaghettibutter) from the Wall Street Journal test set are unseen in training.

the best variants use different composition matrices based on the distance of the candidate head from the PP (HPCD, HPCDN).


(2008) demonstrate that using WordNet semantic classes benefits PP attachment performance.

this can be integrated in the composition architectures in the following way: for each candidate head, represented by a vector h ∈ R n , concatenate a vector representing the word following the candidate.


This paper takes some key steps towards facilitating reasoning about quantities expressed in natural language.

for our classifiers, we use a sparse averaged perceptron implemented with the SNOW framework (Carlson et al., 1999).