However, instead of looking at individual words, we extract representations of semantic relations from sets of pairs T ρ = {(t i , ρ, r i )} instantiating the relation ρ.

for each relation ρ, LinM computes the average (or centroid) difference vector over the set of training pairs: the predictedr for an input (t, ρ) is the sum of the target vector and the relation's prototype.


(2014), PLF has been shown to work better than some other CDSMs in modeling semantic similarity.

good results were obtained on ANVAN (adjective-noun-verb-adjectivenoun) phrases, where PLF outperformed both simple CDSMs (due to its higher expressiveness) as well as the higher-order Lexical Function model (Baroni and Zamparelli, 2010).


The key difference between MSWE and other models is that we induce the weights of senses while jointly learning the word and sense embeddings.

we train a topic model (Blei et al., 2003) to obtain the topic-to-word and document-to-topic probability distributions which are then used to infer the weights of topics.


Table 2 shows that our model outperforms the baseline Word2Vec Skip-gram model (in fifth row from bottom).

on the RW dataset, MSWE obtains a significant improvement of 2.92 in the Spearman's rank correlation (which is about 8.5% relative improvement).


We report accuracies in Table 4 and show that MSWE achieves better results in comparison with the baseline Word2Vec Skip-gram.

mSWE reaches the accuracies of around 69.7% model Accuracy (%) Pennington et al.


baselines, in spite of the availability of only small amounts of training data.

we also demonstrate the relevance of event ordering information provided by script knowledge.


To test the model in a scenarioindependent setting, we perform additional experiments based on a cross-validation with the 10 scenarios as one fold each and exclude the script features.

we repeatedly train our model on 9 scenarios and evaluate on the remaining scenario, without using any information about the test scenario.


We extract propositions from news tweets using PropS , which simplifies dependency trees by conveniently marking a wide range of predicates (e.g, verbal, adjectival, nonlexical) and positioning them as direct heads of their corresponding arguments.

we run PropS over dependency trees predicted by spaCy 6 and extract predicate types (as in Table 1) composed of verbal predicates, datives, prepositions, and auxiliaries.


We apply the distributional model of sentence comprehension presented in the previous section to account for psycholinguistic data about metonymic sentences.

we predict that metonymic sentences will have higher Sem-Comp scores than non-coercion sentences, because they do not comply with the semantic preferences of the event-selecting verb.


semantically coherent with respect to the participants that are overtly realized.

the inferred covert event is the event that maximizes the SCW of the global event structure representing the interpretation of the sentence.


If we are constrained to one label per argument, where should the line be drawn between STIMU-LUS and TOPIC in cases of overlap?

should the semantic representation emphasize the semantic commonality between all of the examples in (2), or between (2d) and (3)?


Thus, "works by Puccini" and "works of Puccini" are paraphrases, but present subtly different portrayals of the relationship between Puccini and his works.

these paraphrases are not identical in meaning because the preposition carries with it different nuances of construal.


(2013) but suitably extended to incorporate the latent frame slots and their associated weights.

we define the probability for a single predicate-argument pair (p i , a i ) as: where represents the element-wise multiplication operator.


In practise, the likelihood component is optimized using negative sampling with EM for the latent slots.

we use hard EM, to select a single slot before taking gradient steps with respect to the model parameters.


In what follows, we detail experimental results on two quantitative evaluation tasks: at the local and global levels of predicate-argument structure.

we evaluate on pseudo disambiguation of selectional preference, and semantic frame lexicon overlap.


To obtain the triples for this task we use the test set of the CoNLL 2008 shared task data.

for every verbal predicate mention in the data we select a random nominal word from each of its arguments phrase chunks to obtain a true predicate-argument word pair.


Our evaluation framework is, in many ways based on the metrics used in unsupervised SRL, except applied at the "type" lexicon level rather than the corpus-based "token" cluster level.

given a gold frame lexicon Ω with K * real argument slots (i.e.


For our experiments, we used the original MultiAlign implementation as well as an adapted version, in which we made some changes to the WordNet aligner.

unlike in the original implementation, which only considered the first sense of each WordNet entry, we extended this to cover all senses.


Our experimental results suggest that meaningful pattern-based entailment graphs can be constructed automatically and that the derived knowledge is in fact valuable for selecting useful relation extraction patterns.

entailment graph based filtering can help achieve higher precision than methods which do not take into account the asymmetric nature of semantic relations.


We extend our GRU model with a neural attention mechanism to capture the most relevant words in the input clauses for classifying SE types.

we adapt the implementation of attention used in Rocktäschel et al.


As a first approach, we use semantic role labeling (SRL).

we use Semafor (Das et al., 2010) tool to generate semantic role labels of the writings and then calculate the frequency of the labels as features for the classifier.


Finally, we compute dense semantic features by computing clusters based on global word vectors.

for LabWriting dataset, we use word vectors trained on Wikipedia 2014 dump and Gigaword 5 (Parker, 2011) which are generated based on global word-word co-occurrence statistics (Pennington et al., 2014).


Hence, we use beam search with a fixed beam size (100) for inference.

in each step, we only expand the ten most promising candidates so far given by the current score.


and  demonstrated effects of typical locations of a word's referent in language processing.


We propose Neural Word Salience (NWS) scores, unlike heuristics, are learnt from a corpus.

we learn word salience scores such that, using pre-trained word embeddings as the input, can accurately predict the words that appear in a sentence, given the words that appear in the sentences preceding or succeeding that sentence.


To overcome the above-mentioned disfluencies in existing word salience scores, we propose an unsupervised method that first randomly initialises word salience scores, and subsequently updates them such that we can accurately predict the words in local contexts.

we train a two-layer neural network where in the first layer we take pre-trained word embeddings of the words in a sentence S i as the input and compute a representation for S i (here onwards referred to as a sentence embedding) as the weighted average of the input word embeddings.


2 This assumption enables us to use a sentence-ordered corpus for selecting both positive and negative training instances required for learning NWS scores.

the model is trained using the two adjacent sentences to S i -{i − 1, i + 1} as positive examples, and K=2 negative examples not in {i − 1, i + 1}.


Note that only 7,807 books in this corpus are unique.

for 2,098 books there exist one duplicate, for 733 there are two and for 95 books there are more than two duplicates.


Following the success of Inverse Document Frequency (IDF) in filtering out high frequent words in text classification tasks (Joachims, 1998), we define Inverse Sentence Frequency (ISF) of a word as the reciprocal of the number of sentences in which that word appears in a corpus.

iSF is computed as follows: iSF(w) = log 1 + no.


From Table 2, we see that GloVe is the best among the three word embedding learning methods compared in Table 2 for producing pre-trained word embeddings for the purpose of learning NWS scores.

nWS scores reports best results with GloVe embeddings in 10 out of the 18 benchmark datasets, whereas with CBOW embeddings it obtains the best results in the remaining 8 benchmark datasets.


We used the EEC as a supplementary test set in a recent shared task on predicting sentiment and emotion intensity in tweets, SemEval-2018 Task 1: Affect in Tweets (Mohammad et al., 2018).

1 we wanted to test a hypothesis that a system should equally rate the intensity of the emotion expressed by two sentences that differ only in the gender/race of a person mentioned.


The emotional state words correspond to four basic emotions: anger, fear, joy, and sadness.

for each of the emotions, we selected five words that convey that emotion in varying intensities.


They are treated just like regular nodes.

they are not sources and do not count towards the function graph's placeholder count.


In the recent years, EL on Twitter data has emerged as a branch of entity linking research.

eL on tweets was the central task of the NeeL shared task from 2014 to 2016 (Rizzo et al., 2017).


The dataset is meant to test the ability of the system to understand different wordings of the same question.

the paraphrases include various references to the same entity, which creates a challenge for an entity linking system.


Simplified VCG To test the effect of the end-toend context encoders of the VCG network, we define a model that instead uses a set of features commonly suggested in the literature for EL on noisy data.

we employ features that cover (1) frequency of the entity in Wikipedia, (2) edit distance between the label of the entity and the token n-gram, (3) number of entities and relations immediately connected to the entity in the KB, (4) word overlap between the input question and the labels of the connected entities and relations, (5) length of the n-gram.


In this work, we provide a detailed characterisation of the distributional nature of abstract and concrete words across 16,620 English nouns, verbs and adjectives.

we investigate the following questions: (1) What is the distribution of concreteness in the contexts of concrete and abstract target words?


(2013) and Hoffman and Woollams (2015), we aim to show that concrete words are easier to predict than abstract words.

we expect higher entropy values for abstract than for concrete contexts, indicating that on average, we need more information to uniquely encode an abstract word than a concrete word (Shannon, 2001).


3 Further, to consider multiple types of elementwise interaction, we use a set of M weight matrices per dimension.

for each Implementation-wise, we vertically stack G for M times to constructG ∈ R M d×3 , and use each row g i as input to Eq.


But this looks ideal for dealing with negation, disjunction and hypotheticals in natural language.

the named graph above asserts neither that John likes Mary nor that John doesn't like Mary.


Despite similarities between our method and those proposed previously to address unsupervised semantic role induction (Carreras and Marquez, 2005;Lapata, 2010, 2011;Swier and Stevenson, 2004), our method differs from them in that we attempt to include frame head grouping information for inducing roles associated to them.

these methods leave out the problem of sense/frame grouping in their models.


First, we note that TB-Dense is actually not fully annotated in the strict sense because only edges within a sliding, two-sentence window are presented.

distant event pairs are intentionally ignored by the designers of TB-Dense.


Thus, they did not get any information from the annotated edges in P and any improvement came from bootstrapping alone.

system 6 is the standard bootstrapping and system 7 is the constrained bootstrapping.


This setup is more realistic than the random and lexical split setups, in which the classifiers can benefit from memorizing verbatim words (random) or regions in the vector space (lexical split) that fit a specific slot of each relation.

on BLESS and K&H+N, one domain is held out for testing whilst the classifiers are trained and validated on the remaining domains.


prescribe the sufficient conditions of such a claim, we argue for an obvious necessary, or at least desired condition: that interesting natural language inference should depend on both premise and hypothesis.

a baseline system with access only to hypotheses (Figure 1b) can be said to perform NLI only in the sense that it is understanding language based on prior background knowledge.


Our notion of relevance is closely related to coherence; when thinking of judging a continuation, we believed that it would be more natural for annotators to judge the relevance of the continuation to its context, rather than judging the coherence of the resulting story.

coherence is a property of a discourse, while relevance is a property of a continuation (in relation to the context).


We found this to outperform other methods.

the next best method used gated recurrent averaging networks (GRANs; Wieting and Gimpel, 2017), followed by long short-term memory (LSTM) networks (Hochreiter and Schmidhuber, 1997), and followed finally by word averaging.


In this work, we pick one of those knowledge resources -WordNet -to collect training data.

our positive instances consist of (i) all direct hypernymy pairs, and (ii) switched terms from the original hyponymy pairs.


In addition, the instructions state that assertions should be explicit, relevant to an issue, self-contained, and only contain a single position.

the use of co-reference or hedging indicated by words such as perhaps, maybe, or possibly was not permitted.


This paper starts from the belief that if we are going to base a representation on semantic roles, they should have consequences independent of the predicate or event they are used in.

we will explore three different aspects that identify criteria one might want in a theory of semantic roles: 1.


Loosely speaking, when +causal obtains (for an AGENT o ag ), there exists some scale sc * such that a change of o ag on the sc * scale would entail a change of the AFFECTED o aff on the sc _E scale.

there is some property of the AGENT such that when this property is changed, regardless by what means, some property of the AFFECTED will vary accordingly.


This indicates that the semantic roles appear to be consistent across the lexicon and, furthermore, are identifiable by the semantic properties induced by the events in the definitions.

they meet the criterion of derivability!


Frame Semantics has been a staple of artificial intelligence and cognitive linguistics since its first formulation in the '70s (Fillmore, 1976).

frame semantics has been widely adopted as a theoretical backbone for the interpretation of natural language, in order to represent its meaning with formal structures suited for computation.


The aim of this experiment is to show that our measure of frame instance relatedness correlates with the semantic relatedness of the text that evokes the frame.

we use textual similarity as a proxy for human judgment of relatedness between frame instances.


Cluster size 5 Noise_makers (the Noise_maker is an artifact used to produce sound, especially for musical effect) Role Noise_maker (this FE identifies the entity or substance that is designed to produce sound) Guitar (a stringed instrument usually having six strings; played by strumming or plucking) Cluster size 40 Frame type Substance (this frame concerns internally undifferentiated Substances) Role Substance (the undifferentiated entity which is presented as having a permanent existence) Concept Sand (a loose material consisting of grains of rock or coral) Cluster size 3 Part_inner_outer (This frame concerns Parts of objects that are defined relative to the center or edge of the object Center (an area that is approximately central within some larger region) Role Whole (an undivided entity having all its Parts) Concept Pond (a small lake)  the hierarchical clustering approach, because the number of clusters is not necessary to be known a priori.

we used the version implemented in the SciPy library 4 .


To overcome this feature sparseness problem, we propose a novel method based on graph decomposition to find candidate features for expanding feature vectors.

we first create a feature-relatedness graph, which is subsequently decomposed into core-periphery (CP) pairs and use the peripheries as the expansion candidates of the cores.


To address the feature sparseness problem encountered in short-text and cross-domain classification tasks, we propose a novel method that computes related features that can be appended to the feature vectors to reduce the sparsity.

we decompose a feature-relatedness graph into core-periphery (CP) structures, where a core feature (a vertex) is linked to a set of peripheries (also represented by vertices), indicating the connectivity of the graph.


To overcome feature-sparseness in training and test instances, we expand features that are cores by their corresponding peripheral sets.

for each core i ∈ C, we sort its peripheries P i by their coreness values and select the top-k ranked peripheries as the expansion features for a core i if it appears in a document.


For example, we want our solution to expand the set {pluto, mickey}-both Disney characters-to other Disney characters.

the context mickey should determine what is considered 'similar' to pluto, rather than being biased by the more dominant sense of pluto, to determine that neptune is similar to it.


There is a sense, however, that the problem is overdetermined: in many such problems, people can solve it even if the first term is not shown.

people easily answer "What is the glove for the foot?".


In developing it we proposed a new measure of association between words and contexts and demonstrated its utility in set expansion and a hard version of the analogy problem.

our results show that sparse representations deserve additional careful study.


We are thankful to all the reviewers for their helpful comments and critiques.

ido Dagan, Yoav Goldberg, Omer Levy, Praveen Paritosh, and Chris Waterson gave us insightful comments on earlier versions of this write up.


In this work, we look beyond syntactic properties of the text and incorporate semantic properties to the tree-structured LSTM model.

we utilize the network topology offered by a tree-structured LSTM and incorporate semantic features induced by AMR formalism.


Table 1 also shows mTreeLSTM consistently outperforms .0 (1.0) 86.7 (0.7) 45.6 (1.1) 85.6 (0.5) TreeLSTM(C) 49.8 (0.8) 87.9 (0.9) 46.3 (0.7) 85.8 (0.5) TreeLSTM(D) 46.9 (0.2) 85.5 (0.4) 46.0 (0.3) 85.0 (0.4) TreeLSTM(A) n/a n/a 44.4 (0.2) 82.9 (0.6) mTreeLSTM(A) n/a n/a 45.2 (0.5) 83.2 (0.5) mTreeLSTM(D) 47.5 (0.7) 85.7 (0.1) 46.7 (0.8) 85.7 (0.8) TreeLSTM on the same tree structures in both settings -Whenever a tree structure is applicable to both mTreeLSTM and TreeLSTM, the performance of mTreeLSTM with that tree structure is better.

in phrase-level setting, mTreeL-STM (D) outperforms TreeLSTM (D) and similarly in root-level setting, mTreeLSTM (D) and mTreeLSTM (A) perform better than TreeLSTM (D) and TreeLSTM (A), respectively.


Consequently, we can systematically analyze the problem of training word embeddings using lessons from the XML literature.

we can featurize both inputs and outputs -in our case, contexts and words.


We will use the word subsampling probability g(y) above to define the cost for the pair (x, y) as the expected frequency of the pair in the training set.

we use the subsampling frequency from word2vec to define g(y) as follows: 2 Here, n(y) is the context-independent frequency of label y in the whole corpus and α is a hyperparameter.


If the label features φ(y) consists only of indicators for the label (i.e., the target word) and the context features ψ(x) is the average of the indicators for the words in the context, then these two loss functions are identical.

cBOW is an instance of our model that minimizes local log loss, and uses these specific features.


For the XML problem of word prediction, the label density is small enough that the problem can be approximated as a multiclass problem.

because |L| is large, the average value of |Y| |L| is small enough that it is close to 1 |L| .


In our modeling framework, the local hinge loss can be defined as: Here, ∆(y, y ) represents the Hamming distance between the ground truth y and any other label y .

it takes the value 1 when y = y and 0 otherwise.


We surmise that only a subset of semantic features were learned by the InferSent objective given the specific characteristics of the SNLI training sets.

the model was not pushed to preserve the full semantic content since only a small subset of features were useful for entailment relationships.


(2018)'s analysis of the effective context size of a word-based language model, we present an ablation study to measure performance when contextual information is removed.

when evaluating models, we retain only the characters in a small window around each time entity in the dev and test sets, and replace all other characters with padding characters.


Note that nodes connected vertically by negative edges represent alternative understandings of an utterance.

in our example, an utterance can be identified as Account operation, Store information or Technical problem, etc.


Nodes connected by horizontal positive edges represent specializations of the analysis.

after the utterance is classified as Technical problem, moving along the positive edge will check whether the utterance expresses a Headset problem, or (moving again vertically along negative edges) a HD problem or, alternatively, a Battery problem.


During training, the input to this network is the representation of a conversation (either contentbased or structure-based), and the ground truth is a one-hot vector of the bot that handled this conversation.

given a conversation c, the network predicts which bot handled c using softmax, that is a distribution over the bots.


When representing a path, we consider the type of the node it is mapped to, in the corresponding bins in the bin vector: each bin maintains 4 counters, one counter for each type of node mentioned above (success, failure, regular and uninvolved).

the mapping of the first user utterance "I'm having some issues with my headset" to the bin vector is as follows: • The first node in the bot graph that is visited is Make a payment (1).


Common bin vector As explained, to capture comparable behavior across bots, we create one common bin vector using the average depth and average width for each level of the bots graphs.

first, based on the average depth, we define the number of sections to be 7.


Results Table 2 summarizes the classification results for all models.

the BOT2VEC-S outperforms all other models with a relative improvement of 16.4%.


We introduce a novel topic modeling approach based on constructing a semantic set cover for clusters of similar documents.

our approach first clusters documents using their Tf-Idf representation, and then covers each cluster with a set of topic words based on semantic similarity, defined in terms of a word embedding.


Second, given the clustering, we attempt to find a set of words for each cluster that forms a description of the cluster.

we use a word embedding, along with a document representation in the same semantic space, to cover each cluster with a small set of topic words that are semantically similar to the documents.


Semantic similarity between words is roughly captured by cosine similarity in the embedded space.

we first represent documents in the same embedding space as words, and define the problem of the choice of topic extraction as a set cover problem instance.


We then generate the collection of topic words for a given cluster using LDA in a standard way.

we choose the n most probable words in the associated LDA topic, where n is set as the number of topic words produced by the set cover.


We therefore ran a second set of qualitative experiments in which LDA topics were used to derive clusters of similar documents.

we clustered all documents based on their associated likelihood given a topic; that is, a document i was assigned to an LDA topic j which maximizes P (i|j).


We present a new data collection method and corpus that results in a larger number of challenging questions that require script knowledge.

we define a revised question collection procedure, which ensures a large proportion of nontrivial commonsense questions.


This choice of task allows us to assess the importance of each type of visual information in human semantic processing.

we perform two experiments: (1) using the Visual Genome (Krishna et al., 2016) dataset of images where objects are manually annotated with bounding boxes, and (2) using images retrieved from Google Image Search and automatically segmenting them using a Faster R-CNN (FRCNN) model (Ren et al., 2015).


The amount of pairwise similarity calculations is then linear in the number of nodes times the average number of neighbors in V 2 , which is much better.

in the case of WordNet, each node (synset) has 36 synsets in its V 2 on average, and half of the nodes do not have any neighbors at all.


We then describe a neural model for the task of document classification that adapts to demographic factors using a multitask learning framework (Section 3).

the model is trained to predict the values of the demographic attributes from the text in addition to predicting the document label.


Then within each demographic factor (e.g., gender), we calculate the percentage of top 1,000 features that overlap across the different attribute values in that factor (e.g., male and female).

if S 0 is the set of top features for one attribute and S 1 is the set of top features for another attribute, the percent overlap is calculated as |S 0 ∩ S 1 |/1000.


It also finds surprising and most probably idiosyncratic patterns.

for the StateHasLake relation, from the paths found, some very unexpected ones had groundings for the given training data: Agric.P rod.GrowingInStateOrP rov.


Overall, we rewrite 5,989 positive interpretations for 5,770 unique negations.

the procedure in Section 4 yields a positive interpretation for 77% of negations.


Out of all dependency types, advmod and xcomp (adverbial modifier and open clausal complement respectively) have the highest ratios of known rewrites, and nsubj (nominal subject) has the most unknown answers.

the easiest placeholders to rewrite are those whose syntactic function is adverbial modifier or open clausal complement, and the most challenging are those whose syntactic function is nominal subject.


In this section, we present baseline results with existing seq2seq models.

we experiment with a basic seq2seq model , two seq2seq models with attention (Luong et al., 2015;, and Google's neural machine translation (NMT) system (Wu et al., 2016), which is also seq2seq model with attention and arguably the most complex.


Stalnaker (1970) proposed that the probability of a conditional and the conditional probability of the consequent on the antecedent are one and the same, in what is usually referred to as the Equation.

the overall probability P (if this doesn't get interesting then I'm going home) is the same as the conditional probability P (I'm going home This doesn't get interesting).


Likewise for any subtypes p in members of e.links, t.links, e.probs and t.probs.

the asymmetric aspect of the merge is at the level of the indexed nodes, not the fields containing them.


We take this approach by modifying the test set so that each PER and GPE (person and geopolitical entity) named entity is different from those seen in training.

we ensure that there is no leakage of PER and GPE named entities from the training set into the test set.


We focus on PER and GPE named entities because they are two of the three most common entity types and because in general when replacing a PER or GPE name with another name, it is easy to not change the true coreference structure of the document.

changing the name of an organization while ensuring that it is compatible with nominals in the cluster is nontrivial without a finer semantic typing.


The adversarially trained system performs significantly better over the entire dataset in comparison to the previous systems, and the difference is consistent between genders.


Here, we present results that show that human stereotypes exist even for much more nuanced judgments such as personality, for a variety of person identities beyond the typically legally protected attributes and that these are similarly captured in word representations.

we collected human judgments about a person's Big Five personality traits formed solely from information about the occupation, nationality or a common noun description of a hypothetical person.


Nevertheless, the prediction score density for gold parses is situated more on the right hand side than most others.

we find that in the out-of-domain data gold parses tend to be assigned above-average scores.


Additionally, identifying the head word of an argument may be more important than getting the boundaries of the span exactly right.

span-based SPRL models may be more robust when confronted with slightly erroneous spans compared to dependencybased models which may be vulnerable to false positive heads.


On the other hand, when only the predicate marker is ablated, the performance decreases only slightly (SPR1: -1.2, SPR2: -1.7).

it appears to be of paramount importance to provide our model with indicators for the argument position in the sentence, but it is of lesser importance to point at the predicate.


We first identify premise-hypothesis pairs from the MNLI dataset that contain antonym pairs (e.g., dirty appears in p and clean in h) and generate all possible patterns of negation with the two mutation strategies: swapping antonyms and adding explicit negation.

we use each of lexical negation, explicit negation, and their combination to mutate the premise and/or the hypothesis.


We discard sentences in which the majority label does not agree with our expected label.

we only include mutated sentences with a majority label of unnatural and unmutated sentences with a majority label of natural.


Except for the use of scaled judgments, our instructions follow the MNLI guidelines.

our instructions said to assume that the sentences co-refer and that the first sentence (p) states a true fact, describes a scenario, or expresses an opinion, and to then indicate how likely it is that the second sentence (h) is also true, describes the same scenario, or expresses the same opinion.


Image-Caption Matching We train on imagecaption pairs from the MSCOCO dataset (Lin et al., 2014) to minimize the cosine distance between representations of the sentence and corresponding image.

we encode the sentence with the BiLSTM encoder.


First, the finegrained annotations in the PMB facilitate our automatic monotonicity-driven construction of inference problems.

semantic tokenization and WordNet (Fellbaum, 1998) senses make narrow and broad concept substitutions easy while the syntactic analyses in Combinatory Categorial Grammar (CCG, Steedman, 2000) format and semantic tags (Abzianidze and Bos, 2017) contribute to monotonicity and polarity detection.


When compared among linguistic phenomena, the improvement by adding HELP was better for upward and downward monotone.

all models except models trained with HELP failed to answer 68 problems for monotonicity inferences with lexical replacements.


Recent work in NLP aims to build more robust systems using adversarial methods (Alzantot et al., 2018;Chen & Cardie, 2018;Belinkov & Bisk, 2018, i.a.).

elazar & Goldberg (2018) attempted to use adversarial training to remove demographic attributes from text data, with limited success.


Here, we study the effect of jointly modeling the speech act and target referent of each utterance, in order to exploit the task dependencies.

this paper is an application of discourse analysis to the pragmaticsrich domain of political science, to determine the intent of every utterance made by politicians, and in part, automatically extract pledges at varying levels of specificity from campaign speeches and press releases.


Our work comes close to the work done in the area of Open Information Extraction (IE) (Niklaus et al., 2018).

to extract information, Clause-Based Open IE systems (Del Corro and Gemulla, 2013;Angeli et al., 2015;Schmidek and Barbosa, 2014) reduce a complex sentence into simpler sentences using linguistic patterns.


We initially used a corpus of Descriptions components from ScreenPy (Winer and Young, 2017), in order to study linguistic patterns in the movie script domain.

we used the "heading" and "transition" fields from ScreenPy's published JSON output on 1068 movie scripts scraped from IMSDb.


2), the "Saarbrücken Corpus of Textually Annotated Cooking Scenes" (TACOS).

the TACOS corpus provides: • A collection of coherent textual descriptions for video recordings of activities of medium complexity, as as a basis for empirical discourse-related research, e.g., the selection and granularity of action descriptions in context • A high-quality alignment of sentences with video segments, supporting the grounding of action descriptions in visual information • Collections of paraphrases describing the same scene, which result as a by-product from the text-video alignment and can be useful for text generation from videos (among other things) • The alignment of textual activity descriptions with sequences of low-level activities, which may be used to study the decomposition of action verbs into basic activity predicates We expect that our corpus will encourage and enable future work on various topics in natural language and video processing.


It also enables the model to be learned while executing instructions, for example by trying to replicate actions taken by humans.

we show that, given only a small seed lexicon and a task-specific executor, we can induce high quality models for interpreting complex instructions.


We incorporate the execution procedure described above with a linear weighted CCG parser, as described in Section 4, to create a joint model of parsing and execution.

we execute logical forms in the current state and observe the result of their execution.


The Prosodic Bootstrapping hypothesis (Gleitman and Wanner, 1982) points out that prosodic phrases are often also syntactic phrases, and proposes that language-acquiring infants exploit this correlation.

if infants can learn about prosodic phrase structure using word duration (and fundamenin a model of language acquisition, gold tags certainly are not.


However, it actually specifies a probability distribution over the decision to back off (B) or not back off (¬B), and we can use different notation to reflect this view.

λ stop (•) and λ choose (•) will represent our backoff distributions for the Stop and Choose decision, respectively.


Second, the numerator and denominator are scaled by the function exp(ψ(•)), which reduces the probability of rare events.

the P choose distribution is estimated using expectations for each arc a dp,h,dir from head h to dependent POS tag d p in direction dir, and the update equation for P choose from iteration n to n + 1 is: where h is the head POS tag for the backoff distribution, and the head (word, POS) pair for the no backoff distribution.


In this work the size of the grammar and the number of productions are fixed and small, but we present the formulation as infinite to allow for easy extension in the future.

this framework allows for extensions which grow the grammar during parsing/training or fully lexicalize the productions.


We present experimental results in which we vary the value of α Y as a function of the number of outcomes allowed by the grammar for argument categories or the corpus in the case of terminal symbols.

we set α Y = n p for conditioning contexts with n outcomes.


However, the Slovene dependency treebank seems to follow a substantially different annotation scheme.

the gold standard annotation of the 1,000 sentences in the Slovene development set treats many of them as consisting of independent sentences (often separated by punctuation marks that our system has no access to), so that the average number of roots per sentence is 2.7: je is mehko soft rekla said When our system is presented with these short components in isolation, it oftentimes analyzes them correctly, but since it has to return a tree with a single root, its performance degrades substantially.


We also experimented with more standard methods, using WordNet and distributional similarity (Beigman Klebanov et al., 2012;Esuli and Sebastiani, 2006;Kim and Hovy, 2004;Andreevskaia and Bergler, 2006;Hu and Liu, 2004;Kanayama and Nasukawa, 2006;Strapparava and Valitutti, 2004;Kamps et al., 2004;Takamura et al., 2005;Turney and Littman, 2003;Hatzivassiloglou and McKeown, 1997).

we implemented a WordNet (Miller, 1995) based expansion that uses the 3 most frequent synonyms of the top sense of the seed word (WN-e).


Moreover, our parser presupposes a restricted inventory of fragments in the grammar (the ones allowed by an ITSG) as opposed to the general CFG rules allowed by the Earley algorithm.

the Backward Substitution operation is more limited than the corresponding Prediction step of the Earley algorithm: only lex-first fragments can be introduced using Backward Substitution, and therefore left recursion (allowed by the Earley algorithm) is not pos-sible here.


(2008) formulated their approach as stacking of parsers where the output of the first-stage parser is provided to the second as guide features.

they used a transition-based parser for the first stage and a graph-based parser for the second stage.


The connections between dual methods and the online algorithms have been previously discussed.

shalev-shwartz and singer (2006) connects the dual methods to a wide range of online learning algorithms.


As a consequence, we note that the syntactic paths π are not tied to any consistency constraint other than each of the paths being a well-formed sequence of dependencies linking the predicate to the argument.

the optimal solution in this case does not guarantee that the set of paths from a predicate to all of its arguments satisfies tree constraints.


This will allow to compute subgradients efficiently.

the subgradient of D at a point λ is: where Whenever π is consistent with y the subgradient will be zero and the method will converge.


This both defeats the purpose of identifying the relations to aid natural language understanding and makes the prediction task harder than it should be: using the standard word sense classification approach, we need to train a separate classifier for each word because the labels are defined per-preposition.

we cannot share features across the different prepositions.


In the following discussion, for a labeled example (x, y * ), we refer to the missing part of its structure as h(y * ).

h(y * ) is the assignment to the arguments of the relation and their types.


We use the score to cluster groups of similar words using a greedy set-covering approach.

we randomly select a word which is not yet in a cluster as the center of a new cluster and add all words whose score is greater than σ to it.


This comparison verifies that our Bayesian tree induction model is effective in shifting the tree structures away from complex minimal rules, which tend to negatively affect translation.

we show an example of a binary parse tree and our U-tree in Figure 8.


Other than for dependency parsing, our results have also implications for mildly context-sensitive phrase structure formalisms.

the algorithm of 5 can be adapted to parse a subclass of lexicalized tree-adjoining grammars, improving the result by Eisner and Satta (2000) from O.n 7 / to O.n 6 /.


In contrast, grammarbased approaches rely on linguistic grammars (in either dependency or constituency formalisms) to shape the search space for possible syntactic analysis.

cFG-based dependency parsing exploits a mapping between dependency and constituency representations and reuses parsing algorithms developed for cFG to produce dependency structures.


To overcome this limitation, we propose several strategies to acquire pseudo grammars only from dependency annotations.

dependency trees are converted to pseudo constituency trees and PCFGs can be extracted from such trees.


bagging substantially improves both HMMLA and PCFGLA models, at least for Chinese POS tagging and constituency parsing.


We can use auxiliary symbols to denote the head phrase position in a CFG rule.

some categories may be splitted into subcategories according to if they are head phrases of their parent nodes or which children are their head phrases.


Pseudo-grammar-based parsing models behave very differently with grammar-based models.

they increase the diversity of model candidates for parser ensemble.


Our main contribution lies in the novel integration of existing techniques and algorithms in parsing and grammar induction to offer a complete solution for simultaneously modeling grounded language at the sentence and discourse levels.

we: (1) use the Earley algorithm to exploit the special structure of our grammars, which are deterministic or have at most bounded ambiguity, to achieve approximately linear parsing time; (2) suggest a rescaling approach that enables us to build a PCFG parser capable of handling very long strings with thousands of tokens; and (3) employ Variational Bayes for grammatical inference to obtain better grammars than those given by the EM algorithm.


An Earley parser constructs left-most derivations of strings, using dotted productions to keep track of partial derivations.

each state in an Earley parser is represented as [l, r]: X→α .


To extend the Inside Outside (IO) algorithm (Baker, 1979) to the Earley context, Stolcke introduced inner and outer probabilities which generalize the inside and outside probabilities in the IO algorithm.

the inner probability of a state [l, r]: X→α .


To tackle the underflow problem posed when parsing discourses ( §3.3), we borrow the rescaling concept from HMMs (Rabiner, 1990) to extend the probabilistic Earley algorithm.

the probability of each Earley path is scaled by a constant c i each time it passes through a scanned state generating the input symbol x i .


12 The large performance differences for child.eyes are consistent with the hypothesis that caregivers are following in, or discussing the object that children are interested in -even control-ling for the continuity of discourse, a confound in previous analyses.

the importance of child.eyes in the discourse model suggests that this cue encodes useful information in addition to the intersentential discourse topic.


In this paper, we proposed a novel integration of existing techniques in parsing and grammar induction to offer a complete solution for simultaneously modeling grounded language at the sentence and discourse levels.

we used the Earley algorithm to exploit the special structure of our grammars to achieve approximately linear parsing time, introduced a rescaling approach to handle very long input strings, and utilized Variational Bayes for grammar induction to obtain better solutions than the Expectation Maximization algorithm.


The main novelties with respect to the TSP model are the mixed word-POS templates (rows 16-17) and the shallow syntax features.

we use the chunk types of i, j and their context (18-19), as well as the chunk head words of i and j (20).


As shown in Table 5, early pruning achieves the best results overall: despite the high DL, we report  no loss in BLEU, METEOR and KRS, but we actually see several improvements.

the gains on the blind test eval09-nw are +0.3 BLEU, +0.2 ME-TEOR and +0.2 KRS (only METEOR is significant).


As for our enhanced system, it achieves similar gains as in the medium-scale scenario: that is, BLEU and METEOR are preserved or slightly improved despite the very high DL, while all the reordering scores increase.

we report statistically significant improvements in the reordering of verbs, which is where the impact of our method is expected to concentrate (+0.7, +0.8 and +1.0 KRS-V on eval08-nw, eval09-nw and vs-09, respectively).


Our idea is to compute usualness scores based on frequency with which a particular pair of words appears in the background.

we compute the conditional probability of the auxiliary word given the main word as the score for likelihood of observing the pair.


We utter about one metaphor for every ten to twenty-five words, or about six metaphors a minute (Geary, 2011).

a metaphor is a mapping of concepts from a source domain to a target domain (Lakoff and Johnson, 1980).


But many pairs thus extracted are not metaphors, for example, Malaysia is a tropical country.

pairs extracted by the "is a" pattern contains at least two types of relations: the literal isA relations and the metaphor relations.


Furthermore, from Γ H , we know Ferrari is a sports car, that is, (Ferrari, sports car) ∈ Γ H , we can then infer that Ferrari to beast is very likely a metaphor mapping.

let (x, y) be a pair we are concerned with.


Instead of trying to grade answers completely automatically, we attempt to leverage the abilities of both the human and the machine.

instead of classifying individual answers as being right or wrong, we propose to automatically find groupings and subgroupings of similar answers from a large set of answers to the same question, and let teachers apply their expertise to mark the groups.


When we instead used our learned similarity metric and looked at the fraction of items that are classified as having no similar items, it dropped rapidly and flattened out to an average of 2-3% (Figure 9 below).

we can expect 97% of responses to cluster nicely with others, and the rest to end up in our miscellaneous cluster ݇.


We say that a complete non-deterministic oracle is correct if and only if, for every projective dependency tree T with yield W , every configuration c that is reachable from INITIAL(W ), and every transition t ∈ o d (c, T ), min A:c;A C(A, T ) = min A:t(c);A C(A, T ), where c ; A signifies that the parse A is reachable from c, a notion that will be formally defined in the next subsection.

even if the gold tree T is no longer reachable itself, the best tree reachable from t(c) has the same cost as the best tree reachable from c. In addition to a cost function for arc sets and trees, it is convenient to define a cost function for transitions.


We now present a procedure for deriving a dynamic oracle for any arc-decomposable system.

first of all, we can define a non-deterministic oracle as follows: we allow all transitions after which the goal tree is still reachable.


In an arc-decomposable system, we know that any intersection of the set of individually reachable arcs with a projective tree is tree consistent, and therefore also reachable.


However, as statistical parsing techniques have been applied to more and more languages, it has also been observed that typological differences between languages lead to new challenges.

it has been found over and over again that languages exhibiting rich morphological structure, often together with a relatively free word order, usually obtain lower parsing accuracy, especially in compar-ison to English.


Another hypothesis from the literature is that the high type-token ratio resulting from large morphological paradigms leads to data sparseness when estimating the parameters of a statistical parsing model (Tsarfaty et al., 2010;Tsarfaty et al., 2013).

for many words in the language, only a subset of its morphological forms will be observed at training time.


In general, however, it is more effective to add the lexical constraints in the form of features, or soft constraints, than to apply them as hard constraints and discard all analyses that are not licensed by the lexicon.

this is a useful strategy when the lexical resource is not completely compatible with the annotation in the training set, as seen in the case of German and (to a lesser extent) Czech.


As a result, we believe that WADE yields results which are informative despite the inevitable automatic alignment errors.

because alignments between a test and reference set are held constant in a system comparison, such errors should impact all analyses in the same way.


It seems that, for the most part, there is relatively little disagreement in the orderings of high-impact entities between different impact measures.

most highly prominent entities tend to be highly rated by most measures.


To generate name disambiguation data, we use the pseudoword method of (Gale et al., 1992).

we choose two or more completely random authors and conflate them by giving all instances of both authors the same name.


We follow a long tradition of older (Finch and Chater, 1992;Schütze, 1993;Schütze, 1995) and newer (Huang and Yates, 2009) work on creating distributional features for POS tagging based on local left and right neighbors.

the i th entry x i of f left (w) is the weighted number of times that the indicator word c i occurs immediately to the left of w: where c i is the word with frequency rank i in the corpus, freq (bigram(c i , w)) is the number of times the bigram "c i w" occurs in the corpus and we weight the non-zero frequencies logarithmically: tf(x) = 1 + log(x).


In summary, when comparing different DA methods caution should be exercised in the choice of domains.

the effect of unknown tags should be made transparent and the gold standards should be analyzed to determine whether the task addressed in the TD differs significantly in some aspects from that addressed in the source domain.


Our purpose in this section is to investigate the ability of automatic systems to select this subset of bilingual documents.

given a collection of documents retrieved for a target language, the task is to identify the documents that contain text in English in addition to the target language.


A i is the set of English words aligned to Chinese word i. φ i,l j are defined for all position i such that A i = ∅.

the constraint function applies only to Chinese word positions that have at least one aligned English word.


This allows us to both start from a stronger baseline model and to investigate how the role of stress cues interacts with other aspects of the model.

we find that phonotactic cues to word-boundaries interact with stress cues, indicating synergistic effects for small inputs and partial redundancy for larger inputs.


These 1000 utterances will be segmented by the model after it has performed inference on its input, without making any further changes to the lexicon that the model has induced.

the model will have to segment each of the test utterances using only the lexicon (and any additional knowledge about co-occurrences, phonotactics, and stress) it has acquired from the training portion of the corpus during inference.


Unlike for the colloc3-nophon models, we also see a clear impact of input size.

at 100 utterances the addition of stress cues leads to an 8 -10% improvement, depending on whether or not the USC is assumed, whereas for the colloc3nophon model we only observed a 3 -8% improvement.


This effect disappears starting from around 1000 utterances; for inputs of size 1000 and larger, the net-gain of stress drops from roughly 10% to a 3-4% improvement.

while we did not notice any relationship between input size and impact of stress cues for the colloc3-nophon model, we do see such an interaction for the combination of phonotactics and stress cues which, taken together, lead to a larger relative gain in performance on smaller inputs than on large ones.


Acknowledgments We wish to thank the anonymous reviewers.

we are indebted to one of them for two important technical remarks.


It then restores the words both preceding and formerly governed by i to the stack.

the word on top of the stack and Pass me the red rectangle uh I mean square The dotted edges are difficult for an incremental parser to avoid, but cannot be part of the final parse if it is to be a projective tree.


In practice, however, differ-ent types of sentiment-oriented relations frequently coexist in documents.

we found that more than 38% of the sentences in our test corpus contain more than one type of relations.


Our model also exploits observable context variables to capture temporal variation that is otherwise difficult to capture using only text.

for the applications we consider, we use stock market data as exogenous evidence on which the language model depends.


Throughout this paper, we use the term features for x.

at each round t, the learner receives x t and makes a predictionŵ t , by choosing a parameter vector α t ∈ R M .


The intuition behind the model is that the probability of a word appearing at day t depends on the background log-frequencies, the deviation coefficients of the word at previous timesteps β 1:t−1 , and the similarity of current conditions of the world (based on observable features x) to previous timesteps through f (x t , x k ).

f is a function that takes ddimensional feature vectors at two timesteps x t and x k and returns a similarity vector f (x t , x k ) ∈ R M (see §6.1.1 for an example of f that we use in our experiments).


Instead, we derive an upper bound for this quantity and minimize that upper bound.

using Jensen's inequality, the variational upper bound on the negative log likelihood is: we use mean-field variational inference where the variables in the variational distribution q are completely independent.


µ t,v is: is: Although we require iterative methods in the E-step, we find it to be reasonably fast in practice.

4 we use the L-BFGS quasi-Newton algorithm (Liu and Nocedal, 1989).


Additionally, we also compare with a base model whose counts decay exponentially ("base exp").

the counts from previous timesteps decay by exp(−γs), where s is the distance between previous timesteps and the current timestep and γ is the decay constant.


We evaluate the perplexity on unseen dataset to evaluate the performance of our model.

we use per-word predictive perplexity: Note that the denominator is the number of tokens up to timestep T .


Thanks to processes like metaphor and grammaticalization, these are (to various degrees) semantically opaque, structurally fossilized, and/or statistically idiosyncratic.

idiomatic expressions may be exceptional in form, function, or distribution.


In our comparison to other methods we focus on those models that can also compute fixed, continuous vectors for sentences.

we compare to the RNN model on constituency trees of Socher et al.


The set of edges E I connects related meanings and is populated as follows: we add an edge from (v, f ) to (v , f ) if and only if f = f and v ∈ semSign v (see lines 9-11).

we connect two candidate meanings of different fragments if one is in the semantic signature of the other.


The results obtained by UKB show that the high performance of our unified approach to EL and WSD is not just a mere artifact of the use of a rich multilingual semantic network, that is, Ba-belNet.

it is not true that any graphbased algorithm could be applied to perform both EL and WSD at the same time equally well.


They generally obtain the highest accuracy (or correlation, respectively), but only cover a relatively small part of the datasets.

the models with a quality higher than the DM variants (11 in Exp.


First, microblogs usually embed rich meta-data, most notably spatiotemporal (i.e., spatial and temporal) signals.

all tweets are associated with a timestamp, and many can be mapped to a location.


cation information, the two teams can be easily distinguished.

based on our labeled data, San Antonio Spurs accounts for 91% of the "spurs" tweets in the US, whereas it only accounts for 8% in the UK.


• We demonstrate the effectiveness of our framework through extensive quantitative experiments.

we improve F1 by more than 10 points over the existing state of the art.


Given a tweet, the system generates a set of candidate anchors based on the lexicon.

each tweet is tokenized into individual words, keeping each punctuation as a separate token.


ASSUMPTION: Given an entity e, how e is expressed (m, a), and when or where e is published (t, l), are conditionally independent.


Intuitively, the expression (m, a) of a given entity e is stable across most times and locations (t, l), which could be attributed to the imitation nature (Leskovec et al., 2009) of social media.

as stories of e propagate on the social media, users over different t and l often imitate each other (disregarding cross-lingual scenarios).


The goal of this section is to describe our base system that does not consider spatiotemporal signals, i.e., to model P (e|m, a).

we adopt an end-to-end entity linking system (E2E), which is designed to jointly detect mentions and disambiguate entities.


Once the assignment of entities has been generated by the previous step, we can re-estimate it by maximizing the objective function with arg max θ tl m,a log P (e|θ tl ).

we are looking for the maximum likelihood estimate (MLE) of θ tl , given the (previously inferred) entity assignments {e}.


For some time t and location l, the objective function Ω({e}, θ tl ) sums over all messages at t, l. These messages themselves are unlabeled, but some supervision is provided by the base system indirectly.

even though we do not know the ground truth entity assignments in the messages, we can leverage the predictions from the base system to update the entity assignments.


The intuition can be captured by the following optimization: where ∈ (0, 1) is a regularization parameter, W is an affinity matrix such that W ij measures the "closeness" of δ i and δ j , and D is a diagonal matrix such that D ii = j∈I W ij .

we design the affinity matrix as follows: where d 0 > 0 and γ < 0 are parameters, and d(•) is a symmetric distance function for bins.


We generalize their idea to both temporal and spatial signals, and extend it to entity linking.

(7) can be interpreted as a result of imitation: tweets mentioning entity e in δ i are imitated from those in δ j .


Finally, while modeling time and location jointly is more expressive than modeling each signal separately, the joint approach also creates much more bins (and hence more multinomial parameters), making data scarcity a severe issue.

jointly we have |∆ T | × |∆ L | bins, but separately we only have |∆ T | + |∆ L | bins.


For evaluation, we test if the system can correctly identify the presence or absence of the query entity in every tweet.

we compute the F1 score over the test set.


3a, using either temporal (+T) or spatial (+L) signal can indeed improve entity linking over the base systems.

the improvements in IR-driven evaluation are more significant since the chosen entities are known to be influenced by time or location.


5a reveals that +T+L is consistently helpful to different types of entities.


8, the intensity based on keyword "spurs" roughly matches that of San Antonio Spurs based on entity linking, but significantly differs from Tottenham Hotspur F.C.

"spurs" only accounts for the former entity, even though it is also a standard anchor for the latter.


The architecture avoids inconsistent graphs by inferring all transitive relations from each sieve's output before the graph is passed on to the next one.

the n th sieve cannot add an inconsistent edge that is inferrable from the previous n − 1 sieves.


1 is intuitive, but, as with many combinatorial optimization problems, it is hard to find a good solution.

we sketch a proof of this theorem in Appendix A: The objective function of Eq.


The crucial observation is that TLSG defines vertex costs (implicitly all-zero) and edge costs, and asks for vertex signs, whereas TRIANGLE BALANCE defines edge costs and triangle costs, and asks for edge signs.

vertices (edges) in TLSG correspond to edges (triangles) in TRIANGLE BALANCE, and our proposed reduction transforms an original TLSG instance into a TRIANGLE BALANCE instance in which each edge corresponds to exactly one original vertex, and each triangle to exactly one original edge.


A probabilistic model to fit many annotators' observed labels produces much more information about the annotated corpus.

there will be a confidence estimate for each ground truth label.


Agreement: Pairwise agreement A m,n between two annotators m, n ∈ 1:J is defined as the proportion of items i ∈ 1:I for which the annotators supplied the same label, where I(s) = 1 if s is true and 0 otherwise.

a m,n is the maximum likelihood estimate of chance of agreement in a binomial model.


Chance-Adjusted Agreement: Agreement coefficients measure the proportion of observed agreements that are above the proportion expected by chance.

given an estimate A m,n of the probability that two annotators m, n ∈ 1:J will agree on a label and an estimate of the probability C m,n that they will agree by chance, chance-adjusted agree- Chance agreement takes into account the prevalence of the individual labels in 1:K. it is defined to be the probability that a pair of labels drawn at random for two annotators will agree.


Conditional Random Fields (Lafferty et al., 2001) (CRFs) are popular models for many NLP tasks.

the linear chain CRFs explore local structure information for sequence labeling tasks, such as part-of-speech (POS) tagging, named entity recognition (NER), and shallow parsing.


Formally, the decoding problem in fully connected pairwise CRFs can be formulated as follows: Note that for any edge (u, v), adding a constant ψ uv to all of its related patterns will not change the optimal solution of the problem.

the optimal Y for the following problem is irrelevant to The reparameterization keeps the optimality of the problem and plays an important role for graph representation, as we will show later.


One is a set of independent vertices, and the other is a pseudo-Boolean optimization problem.

we can redefine g λ (Z) as A sufficient condition for g λ (Z) to be graph representable is that coefficients of all non-linear terms are non-negative (Freedman and Drineas, 2005).


As the MSM2013 competition allows to use extra resources, we use several additional datasets to generate rich features.

we trained two POS taggers and two NER taggers using extra datasets.


Let g λ (Z) = g 1 (Z) + g 2 (Z) + g 3 (Z) + g 4 (Z) where For g 2 (Z), since coefficients of all terms are nonnegative, we can use the fact to reduce g 2 (Z) into an equivalent quadratic form (Freedman and Drineas, 2005).

max Z g 2 (Z) is equivalent to  which is graph representable because coefficients of all the quadratic terms are non-negative.


In this paper, we build a semantic parser that does not require example annotations or question-answer pairs but instead learns from a large knowledge base (KB) and web-scale corpora.

we exploit Freebase, a large community-authored knowledge base that spans many sub-domains and stores real world facts in graphical format, and parsed sentences from a large corpus.


In the final part of our correctness proof we now prove the completeness of the deduction system in Figure 5.

we show the following stronger statement: For every derivation tree and for every derivation context with a signature I satisfying the arity bounds for items of Figure 5, the deduction system infers the corresponding item I .


Similarly, the adapted priors (singular|plural) and (plural|singular) are 0.034 and 0.016, respectively.

the unadapted prior probability for "plural" is three times lower than for "singular", which does not change much with adaptation.


5 It directly combines both discriminativelytuned latent features and surface lexical features into a SVM classifier.

the latent representation of a pair of sentences v 1 and v 2 is converted into a feature vector, We also introduce a new baseline, LEXLATENT, which is a simplified version of LEXDISCRIM and easy to reproduce.


The factors we implement for these purposes closely resemble the factors employed for latent semantic clusters in .

that structure is as follows: the features between the type variables for mentions i and j does not come into play unless i and j are coreferent.


However, belief propagation would still be quite computationally expensive if run on the full factor graph as described in Section 3.

the factors in Section 3.2.2 and Section 3.2.3 are costly to sum over due to their ternary structure and the fact that there are quadratically many of them in the number of mentions.


The restrictions on segment sequences represented in natural language data amount to 'missing' transitions in the initial prefix tree transducer that is built from that data.

the transducer represents a partial, not a total function.


Automatic extraction of opinions from text has attracted considerable attention in recent years.

significant research has focused on extracting detailed information for opinions at the finegrained level, e.g.


In this section, we present our approaches for the joint modeling of opinion expression extraction and attribute classification.

given a sentence, our goal is to identify the spans of opinion expressions, and simultaneously assign their polarity and intensity.


This confirms the informativeness of the extracted opinion expressions.

using the opinion expressions extracted by JI-LOSS gives the best perfor-mance among all the baselines in all evaluation criteria.


Starting from a sample size of about 70 instances, the learning curve proceeds as   Figure 6: Learning curve for 10-fold cross-validation with increasing size of training data, results are averaged over 100 runs Figure 7 shows that our prediction approach produces a few strong outliers for the test data.

it strongly underestimates the error rate for some very easy gaps.


These tendencies confirm the results in Polajnar and Clark (2014) and Kiela and Clark (2014).

the finding that sparse association measures (with negative values clamped to zero) achieve the best performance can be connected to the positive impact of context selection highlighted by Polajnar and Clark (2014): ongoing work targets a more specific analysis of their "thinning" effect on distributional vectors.


For these reasons, we converted the MR expressions into a node-argument form.

all predicates introducing entities (isA) and most predicates introducing relations among entities (e.g.


We denote a given sentence by x and the set of prepositions in x by P R(x).

p R(x) is the set of words whose pOS tags are prep.


We create clusters from our unsupervised corpora using the Liang (2005) implementation of Brown's algorithm, and add features in the spirit of (Koo et al., 2008).

we add full and prefixed bit strings for the head, preposition, and child, as well as bi-lexical versions for head-child pairs.


Probabilistic graphical models like Markov Logic networks (Richardson & Domingos, 2006;Domingos et al., 2007;Domingos & Lowd, 2009) or factor graphs (Loeliger, 2008;Koller & Friedman, 2009) take into consideration constraints such as transitivity, while spectral clustering methods (Luxburg, 2007) implicitly consider transitivity in the underlying eigenspace decomposition, but suffer from high computational complexity.

all methods need to precompute features for the data points and similarity values between all pairs of data points.


To assess whether these two chains should be combined, i.e., they both refer to the same entity, we compute semantic features by tapping into knowledge bases (KB's).

we harness labels and properties from freebase.com entries, for possibly matching entities, to enrich the features of a mention group.


We make only weak assumptions on matching men-tions against KB entities, by filtering on confidence and merely treating semsum's as features rather than relying on perfectly mapped entities.

our CROCS method handles the four dimensions of knowledge enrichment as follows: Enrichment Target: We use per-document mention groups, after the local CR step, as target.


, w n ), we seek to predict a set of argument spans labeled with their semantic roles.

we assume that there is a set of candidate spans S that could potentially serve as arguments of t. we derive S with a high-recall rule-based algorithm that looks at the (dependency) syntactic context of the predicate word t, as described in §6.3.


This corpus consists of nominal predicate-argument structure annotations in addition to verbs.

we use version 12 downloaded from http://cemantix.


Both the tagger and the parser, as well as the frame identification and role labeling models (see Tables 1  and 2), have features based on word clusters.

we use the clusters with 1000 classes described by Turian et al.


We thank Ryan McDonald, Emily Pitler, Slav Petrov and Fernando Pereira for their detailed comments.

ryan pointed out a simplification that improved on our original dynamic program formulation.


This models the intuitive characteristic that subtopics belonging to similar supertopics (encoded by δ) should come from similar priors over their word distributions (since they will have similar β values).

children of a supertopic are topically related -they are likely to share words.


For variables that we prefer to be binary but have softened to continuous variables using sparse Beta or Dirichlet priors, we can straightforwardly strengthen the preference to be binary by modifying the objective function to favor the prior more heavily.

under a Dirichlet(ρ<1) prior we will introduce a scaling parameter τ t ≥ 1 to the prior log likelihood: τ t log P (β) with partial derivative τ t ρ−1 β kc , which adds extra weight to the sparse Dirichlet prior in the objective.


Deficient modeling has been found to be useful for a wide range of NLP tasks (Klein and Manning, 2002;May and Knight, 2007;Toutanova and Johnson, 2007).

we factor the conditional probabilities in Eq.


We next model the complexity prediction task, following a Bayesian approach.

we learn to assign complexity levels to solutions based on their linguistic makeup.


In these models, complexity is captured through a discrete set D of L levels and a total ordering between the levels reflects their relative complexity.

each complexity level is parametrized by a unigram language model which captures words likely to occur in solutions with that level.


Nevertheless, the models are able to rate solutions on a global scale while accommodating problem-specific ordering sequences.

we can compute the expected complexity of the solution set for problem i, using the inferred distribution over levels Table 5 shows the complexity of different problems as predicted by the position model (with 10 levels).


The difference makes training our model more indirect than previous approaches to grounded meaning.

our model introduces a new layer of uncertainty that describes what category the speaker uses.


We assume that speakers' choices mirror established patterns.

the model learns a measure of availability for each color term that tracks how frequently speakers tend to use it when it is applicable.


The most likely label of this posterior is the maximum likelihood estimate (MLE).

we track how often the MLE color label is what the user actually said as 5 the histograms are of size (90,10,10), (45,5,5), and (1,1,1) across Hue, Saturation, and Value with interpolation weights of 0.322, 0.643, and 0.035 respectively.


However, the differences in the table, though numerically small, are significant (by Binomial test) at p < .02 or less.

the fact that LUX wins T OP 1 hints that its representations enable better generalization than HM or GM.


• We describe an algorithm to exploit parallel news streams to cluster sentences that belong to the same event relations.

we propose the temporal negation heuristic to avoid conflating co-occurring but nonsynonymous phrases.


We train another version of Universal Schemas R13P on the parallel news streams NS13.

entity pairs from different NewsSpikes are used as different rows in the matrix.


However, this approach still requires substantial human effort in constraint generation.

lexical constraints were constructed by creating a detailed word list for each information structure category.


In contrast, we do not use manually specified guidance in constraint learning.

we construct constraints from latent variables (topics in topic modeling) estimated from raw text rather than applying maximum likelihood estimation over observed variables (fluents and temporal expressions) in labeled data.


We use Contrastive Estimation (Smith and Eisner, 2005) to efficiently learn this model in an unsupervised manner.

we require that each word has greater support among its bounded set of candidate parents than an artificially constructed neighboring word would.


Here, W is a set of words and Z is the set of candidates for words in W, that includes the parents as well as their types.

a candidate is a (parent, type) pair, where the type variable keeps track of the type of morphological change (or the lack thereof if there is no parent) as we go from the parent to the child.


We include other features to help make this distinction.

we include features such as word vector similarity and the presence of the parent in the observed wordlist.


We also analyze the various affixes produced by the model, and compare with the gold affixes.

we plot the frequency distributions of the affixes 16 obtained from the gold and 15 Note that the candidate probability distribution may have more than a single peak in some cases.


The authors did not specify training and smoothing of their model, so we only adopt their definition of the context window, and use the same neural network architecture as for our other models.

we use two neural networks: one for left dependents, and one for right dependents.


Considering the systems tuned on BLEU, we observe that the 5-gram NNLM and RDLM are best in terms of BLEU and TER, but that RDLM is the only winner 9 according to HWCM f and METEOR.

we observe a sizable gap of 0.6 HWCM f points between the NNLM and the RDLM systems, despite similar BLEU scores.


The mapping is performed by first inducing a function f proj : is a vector representation of an image tagged with an object or an attribute (such as dog or metallic), and w i ∈ R d 2 is the linguistic vector representation of the corresponding word.

the mapping function can subsequently be applied to any given image v i ∈ V to obtain its projection w i ∈ W onto linguistic space: we consider two mapping methods.


Linguistic Space We construct distributional vectors from text through the method recently proposed by , to which we feed a corpus of 2.8 billion words obtained by concatenating English Wikipedia, ukWaC and BNC.

5 we used the CBOW algorithm, which induces vectors by predicting a target word given the words surrounding it.


We use a subset of the Pascal VOC 2008 dataset.


The first model is a straightforward adaptation of the alignment model in §4.2, where the representations of questions and answers are changed from bags of words to "bags of syntactic dependencies" (Surdeanu et al., 2011).


This method follows the same algorithm as PR U , but uses NNLM similarities of terms to define non-uniform teleportation probabilities (following the intuition that teleportation likelihood should be correlated with semantic similarity of terms).

we obtain the ith row of T by computing the pairwise cosine similarities of term i's NNLM vector with every other term's vector, and taking the softmax of these similarities to produce a probability distribution, v(i).


These models represent each word as a ddimensional vector of real numbers, and vectors that are close to each other are shown to be semantically related.


Furthermore, analysis by Levy and Goldberg (2014c) shows that word2vec's SGNS is implicitly factorizing a word-context PMI matrix.

the mathematical objective and the sources of information available to SGNS are in fact very similar to those employed by the more traditional methods.


While the focus of the presentation in the wordembedding literature is on the mathematical model and the objective being optimized, other factors affect the results as well.

embedding algorithms suggest some natural hyperparameters that can be tuned; many of which were already tuned to some extent by the algorithms' designers.


In many cases, changing the setting of a single hyperparameter yields a greater increase in performance than switching to a better algorithm or training on a larger corpus.

word2vec's smoothing of the negative sampling distribution can be adapted to PPMI-based methods by introducing a novel, smoothed variant of the PMI association measure (see Section 3.2).


Levy and Golberg (2014c) show that SGNS's corpuslevel objective achieves its optimal value when: Hence, SGNS is implicitly factorizing a wordcontext matrix whose cell's values are PMI, shifted by a global constant (log k): SGNS performs a different kind of factorization from traditional SVD (see 2.2).

the factorization's loss function is not based on L 2 , and is much less sensitive to extreme and infinite values due to a sigmoid function surrounding w • c. Furthermore, the loss is weighted, causing rare (w, c) pairs to affect the objective much less than frequent ones.


Here, we provide a different interpretation of its effect on the cosine similarity function.

we show that adding context vectors effectively adds firstorder similarity terms to the second-order similarity function.


We note that in the SVD-based factorization, the resulting word and context matrices have very different properties.

the context matrix C SVD is orthonormal while the word matrix W SVD is not.


The contradictory results in (Baroni et al., 2014) stem from creating word2vec embeddings with somewhat pre-tuned hyperparameters (recommended by word2vec), and comparing them to "vanilla" PPMI and SVD representations.

shifted PMI (negative sampling) and context distribution smoothing (cds = 0.75, equation 3in Section 3.2) were turned on for SGNS, but not for PPMI and SVD.


the effects of fine-tuning word embeddings are usually limited in NLP applications.


A proposed combined classifier simply and straightforwardly makes decisions based on the agreed decisions of the base classifiers, namely the SSL classifier and DL classifier.

if the base classifiers agree on the NE type of a certain word, then it is annotated by an agreed NE type.


Our system incorporates the provided entity links into the syntactic parse provided that they are consistent with the parse structure.

we require that each mention is either (1) a constituent in the parse tree with syntactic category N or N P or (2) a collection of N/N or N P/N P modifiers with a single head word.


The correct answers from the pool are then used to evaluate the precision and recall of each system.

we compute average precision (AP) for each question and report the mean average precision (MAP) across all questions.


We also introduce a new comparative approach to simplification corpus analysis.

we assemble a new simplification corpus of news articles, 1 re-written by professional editors to meet the readability standards for children at multi-Not Aligned (17%) [NORM] The soprano ranges are also written from middle C to A an octave higher, but sound one octave higher than written.


Our main contribution is that we propose two new latent feature topic models which integrate latent feature word representations into two Dirichlet multinomial topic models: a Latent Dirichlet Allocation (LDA) model (Blei et al., 2003) and a onetopic-per-document Dirichlet Multinomial Mixture (DMM) model (Nigam et al., 2000).

we replace the topic-to-word Dirichlet multinomial component which generates the words from topics in each Dirichlet multinomial topic model by a twocomponent mixture of a Dirichlet multinomial component and a latent feature component.


We see that λ = 1.0 gives the highest NPMI score.

using only the latent feature model produces the most coherent topic distributions.


uations, our models perform better than the baseline models.

on the small N20small and Twitter datasets, when the number of topics T is equal to number of ground truth labels (i.e.


We assume the candidate c and the sentential context s are conditionally independent if both the mention m and its type t are given.

p(c|m, t, s) = p(c|m, t), the RHS of which can be estimated by renormalizing p(c|m) w.r.t.


For incorporating it in VINCULUM, we seek a document-wise assignment of entity links that maximizes the sum of the coherence scores between each pair of entity links predicted in the document d, i.e.

where φ is a function that measures the coherence between two entities, M for a mention m and each of its candidates, we compute a score, coh(c) = Since both measures take values between 0 and 1, we denote the coherence score coh(c) as p φ (c|P d ), the conditional probability of an entity given other entities in the document.


Freebase API offers a better coverage than the intra-Wikipedia dictionary but is less efficient than CrossWikis.

freebase API needs a larger cut-off value to include the gold entity in the candidate set.


The gradient for the classification parameters therefore depends on the margin value between gold label and all other labels.

taking one component of A y , a y,1 , as an example, the derivative of the objective for where is the delta function.


(2011), is to train a recursive neural network (RNN) whose structure is defined by a binarized parse tree.

they trained their RNN as an unsupervised autoencoder.


For phrase p, we compute its vector g(p) through recursive computation on the parse.

if phrase p is the yield of a parent node in a parse tree, and phrases c 1 and c 2 are the yields of its two child nodes, we define g(p) recursively as follows: where f is an element-wise activation function (tanh), [g(c 1 ); g(c 2 )] ∈ R 2n is the concatenation of the child vectors, W ∈ R n×2n is the composition matrix, b ∈ R n is the offset, and n is the dimensionality of the word embeddings.


E.g., for choosing t 1 for a given x 1 , x 2 : where X b ⊆ X is the current mini-batch.

we want to choose a negative example t i that is similar to x i according to the current model parameters.


Hyperparameters were tuned using the wordsim-353 (WS353) dataset (Finkelstein et al., 2001), specifically its similarity (WS-S) and relatedness (WS-R) partitions (Agirre et al., 2009).

we tuned to maximize 2×WS-S correlation minus the WS-R correlation.


We build the Boolean Distributional Semantic Model (BDSM) by mapping real-valued vectors from a distributional semantic model into Booleanvalued vectors, so that feature inclusion in Boolean space corresponds to entailment between words (or sentences).

we optimize the mapping function so that, if two words (or sentences) entail each other, then the more specific one will get a Boolean vector included in the Boolean vector of the more general one.


To quantify this difference, we compared the similarity scores (cosines) produced by the two models.

we first created a list of pairs of semantically related words using the following procedure.


Synonyms and hypo/hypernyms have significant denotational overlap, and they move closer to each other after mapping.

synonyms significantly gain in similarity between BS and DS (p<0.01), whereas hyponym-hypernym pairs, while not differing significantly in average similarity across the spaces, change from being weakly significantly lower in cosine than all other pairs in DS (p < 0.05) to being indistinguishable from the other pairs in BS.


The distribution G q ag (t) for q ∈ N of an AG is a sample from this distribution over distributions.

where H q (t) denotes the base distribution over T q .


We formulate the noisy-channel model as a PCFG and encode the substitute, split, and delete operations as grammar rules.

for l ∈ L, where l ∈ L are the start symbols as well as the nonterminals of the PCFG.


Finally, we assign each discovered PLU l ∈ L to an HMM, π l , which is used to model the speech realization of each phonetic unit in the feature space.

to capture the temporal dynamics of the features associated with a PLU, each HMM contains three emission states, which roughly correspond to the beginning, middle, and end of a phonetic unit (Jelinek, 1976).


We exploit Markov chain Monte Carlo algorithms to generate samples from the posterior distribution over the latent variables.

we construct three Markov chain kernels: 1) jointly sampling d i , o i , u i , 2) generating new samples for v i , z i , and 3) updating π.


The acoustic component of the FullDP system was initialized by using the output of the DPHMM model for each lecture.

we made use of the HMMs, the phone boundaries, and the PLU that the DPHMM model found as the initial values for π, z i , and v i of the FullDP system.


Since our model also looks for the best segmentation in the same hypothesis space, by initializing the boundary variables around the optimum, our model should simply maintain the segmentation.

as shown in Table 2, the full systems also achieve about the same performance as the -AM systems for the first four lectures, with the overall largest performance difference being 0.4%.


Furthermore, this establishes a setting to systematically analyze the contribution of the underlying structure to performance, while fixing parameters such as preprocessing and features.

we analyze approaches to coreference resolution and point out that they mainly differ in the structures they operate on.


where φ : X × H × Z → R d is a joint feature function for inputs and candidate outputs.


In our framework, we can represent the mention pair model as a labeled graph.

let the set of edges be all backwardpointing edges, i.e.


The mention ranking approach can be represented as an unlabeled graph.

we allow any graph with edges A ⊆ {(m j , m i ) | j > i} such that for all j there is exactly one i with (m j , m i ) ∈ A (each anaphor has exactly one antecedent).


We work in a setting that corresponds to the shared task's closed track (Pradhan et al., 2012).

we make use of the automatically created annotation layers (parse trees, NE information, ...) shipped with the data.


We focus on a set of features that reflect lexical and structural properties of the text, and have been shown to be effective for supervised classification of translationese (Volansky et al., 2015).

we use function words (FW), more precisely, the same list that was used in previous works on classification of translationese (Koppel and Ordan, 2011;Volansky et al., 2015).


We measure the similarity between a class X (either O or T) and a cluster C i using the Jensen-Shannon divergence (JSD) (Lin, 1991) on the respective probability distributions.

we define the distance between the language models as the square root of the divergence value, which is a metric, often referred to as Jensen-Shannon distance (Endres and Schindelin, 2003): The assignment of the label X to the cluster C 1 is then supported by both C 1 's proximity to the class X and C 2 's proximity to the other class: C 2 is assigned the complementary label.


We apply this procedure to the outcome of all clustering experiments (per domain, using various features), achieving overall precision of 100%.

the labeling procedure yields prefect accuracy not only on Europarl and Hansard texts that were not used for generation of O and T prototypical examples, but also on unseen Literature and TED datasets.


We first merged equal numbers of O and T chunks from two corpora: 800 chunks each from Europarl and Hansard, yielding 1,600 chunks, half of them O and half T. We applied the clustering algorithm of Section 5 to this dataset; the result was a perfect domain-driven separation of all Europarl and Hansard chunks, yielding poor (chance-level) translationese accuracy.

we obtained two clusters, one consisting of Europarl chunks and the other of Hansard chunks, independently of their O-vs.-T status.


A language's lexicon M and morphophonology S are deterministic, in that each morpheme has a single underlying form and each word has a single surface form.

the point of the language-specific distributions M φ and S θ is to aid recovery of these forms by capturing regularities in M and S. S θ constitutes a theory of the regular phonology of the language.


A probabilistic treatment also allows relatively simple learning methods (e.g., Boersma and Hayes (2001)) since inference never has to backtrack from a contradiction.

our method searches a continuous space of phonologies S θ , all of which are consistent with every mapping S. we always have S θ (s | u) > 0 for all u, s, so our current guess of S θ is always capable of explaining the observed words, albeit perhaps with low probability.


This corresponds to the "Richness of the Base" principle advocated by some phonologists (Prince and Smolensky, 2004), and seems to yield good generalization for us.

we say all URs of the same length have the same probability, and the length is geometrically distributed with mean (1/φ)−1.


So our finite network can be viewed as a subgraph of an infinite graph.

we make no closed-vocabulary assumption, but implicitly include (and predict the surface forms of) any unobserved words that could result from combining morphemes, even morphemes not in our dataset.


Approximate inference then proceeds by loopy BP on this model.

reduplication is not regular if unbounded, but we can adopt morphological doubling theory (Inkelas and Zoll, 2005) and model it by having U concatenate two copies of the same morph.


However, we actually used a (0,2,1) topology, to allow features that also look at the "upcoming" input phoneme that immediately follows the edit's input (/@/ in Figure 2).

for each natural class, we also included contextual versions of each EDIT or BIGRAM feature, which fired only if the "upcoming" input phoneme fell in that natural class.


We substituted a simplified phonology model where S θ (s | u) just decays exponentially with the edit distance between s and u; the decay rate was learned by EM as usual.

this model uses only the COPY feature of section 6.


The contrast analysis method (Tesar, 2004;Merchant, 2008) in effect uses constraint propagation (Dechter, 2003).

it serially eliminates variable values (describing aspects of the URs or the constraint ranking) that are provably incompatible with the data.


The adopted SRL system has been developed for PropBank/NomBank-style role labeling and we make several changes to adapt it to FrameNet.

we change the argument labeling procedure from predicate-specific to frame-specific roles and implement I/O methods to read and generate FrameNet XML files.


General vs. Document-specific Vectors We also assessed the impact of adapting vectors to documents (see Table 6).

we compared a version of the Framat +context model without any vectors against a model using the adaptation technique presented in Section 5.1 and a simpler alternative which obtains GloVe representations trained on the Wikipedia corpus and FrameNet texts.


Applying Haussler's formulation to trees we reach a general formula for a tree kernel between two trees t 1 and t 2 , namely where F is the set of all tree fragments, c 1 (f ) and c 2 (f ) return the counts for fragment f in trees t 1 and t 2 , respectively, and w(f ) assigns a weight to fragment f .

we can consider the kernel a weighted dot product over vectors of fragment counts.


For small sizes the variance in the resulting hyperparameter values is large but as soon as we reach 200 instances we are able to retrieve the original values with high confidence.

in an ideal setting 200 instances are enough to learn the kernel.


Dowty offers a solution to the mapping problem couched not in terms of fine-grained fragmented thematic roles, but in terms of what Dowty analogizes to 'prototype' concepts constructed over finegrained role properties.

the roleproperties are features such as whether the participant in question causes the event to happen, or whether the participant changes state.


If lexical argument structure is organized around proto-roles, then we predict that we will find this organization reflected in corpora, and that token-level annotations of verb meanings would benefit from observing this organiza- tion.

an annotation strategy that takes the proto-role hypothesis seriously would annotate verbs for properties such those shown in Table 1.


Kako did not use actual or attested sentences of English, but rather focused on 'nonce'based tasks.

he constructed stimuli by taking constructed sentences of English containing the target verbs, and replacing noun positions with nonce words like dax.


In the remainder of this section we describe the general setup of the experiments.

we describe a pro-  cess for arriving at the specific fine-grained property questions we ask, the creation of the data set that we ask the questions about, the task that Mechanical Turkers are presented with, and the manner in which we analyze and display the results.


We find this gradient by backpropagation.

we treat the entire parser (approximations and all) as a differentiable circuit, as others have done for loopy CRFs (Domke, 2010; Stoyanov et al., 2011; Domke, 2011; Stoyanov and Eisner, 2012).


(2011), our aim is to minimize expected loss on the true data distribution over sentence/parse pairs (X, Y ): Since the true data distribution is unknown, we substitute the expected loss over the training sample, and regularize our objective in order to reduce sampling variance.

we aim to minimize the regularized empirical risk, given by (8) with J(θ; How slow is exact inference for dependency parsing?


In some other settings, exact inference is NPhard.

non-projective parsing becomes NP-hard with even second-order factors (McDonald and Pereira, 2006).


Our proposed L 2 training is able to close the gap, just as intended.

we succesfully train a few iterations of an approximate O(n 3 ) algorithm to behave as well as an exact O(n 4 ) algorithm.


Our system, hereafter referred to as Plato, is designed to be resilient to irrelevant features and can be seen as a selective extension of the naïve Bayes model.

we assume that most of the context features of a mention are irrelevant to its disambiguation.


Our selective context model assumes that most features that appear in the context of a mention are not discriminative for entity disambiguation.

we make a simplifying modeling assumption that exactly one context feature is relevant for disambiguation, and the remaining features are drawn from a background distribution.


We consider event mentions as the data points and encode the similarity (or compatibility) between event mentions as priors for event clustering.

we use a log-linear model to estimate the similarity between a pair of event mentions ( where ψ is a feature vector, containing a rich set of features based on event mentions i and j: (1) head word string match, (2) head POS pair, (3) cosine similarity between the head word embeddings (we use the pre-trained 300-dimensional word embeddings from word2vec 1 ), (4) similarity between the words in the event mentions (based on term frequency (TF) vectors), (5) the Jaccard coefficient between the WordNet synonyms of the head words, and (6) similarity between the context words (a window of three words before and after each event mention).


To this end, we borrow ideas from imitation learning for structured prediction (Daume et al., 2009;Ross et al., 2011;Goldberg and Nivre, 2013;.

we cast agenda-based semantic parsing as a Markov decision process, where the goal is to learn a policy, that given a state (i.e., the current chart and agenda), chooses the best next action (i.e., the parse to pop from the agenda).


The AGENDA baseline uses an agenda-based parser to approximate the gradients of (2).

we update parameters as in FIXE-DORDER, but search for K root derivations using the agenda-based parser, described in Algorithm 1 (where we pop the highest scoring derivation).


Naturally, the number of featurized derivations in training increases as decreases.

nOSTREAM results in a 2.5x increase in number of featurized derivations compared to no unrolling ( = 10 2 ), and 1.5x increase compared to = 10 −2 , which is the chosen value.


A circle graph is an undirected graph that represents the intersection graph of a set of chords of a circle.

for every circle graph G we can draw a circle and a set of chords of that circle such that the chords are in one-to-one correspondence with the vertices of G and two chords cross each other if and only if the corresponding vertices are adjacent in G. An example of a circle graph and a corresponding chord drawing is given in Figure 3.


For example, this is the case for keyword-based search queries (Sajjad et al., 2012) or short dialogue utterances (Zettlemoyer and Collins, 2007).

taking the example of search queries, users tend to omit some of the function words and grammatical constructs in the language to make a more concise query.


In this work, we simulate the keyword query K by altering the original question S to make it more closely match the style of keyword queries.

following the analysis of Leveling (2010), we make two changes to the original queries: stop word deletion, and word order shuffling.


We use a stop word list (Buckley et al., 1993), making a few subjective changes to make the simulated keyword output more realistic.

we add "give" and "show," which often occur in statements such as "give me ..." or "show me ..." but are unnatural in keyword queries.


In this work, we construct the tri-synchronous grammar by transforming the basic SCFG for semantic parsing G into a 3-SCFG.

we first assume that the source question γ s and target MR γ t of the original SCFG become the two outputs γ 2 and γ 3 of the new 3-SCFG grammar.


Bold indicates a significant gain over both Direct and Tri-LM for keyword input according to bootstrap resampling (Koehn, 2004)  First, in this section, we examine the effect of the proposed method on accuracy of parsing ambiguous keyword queries.

in Table 3 we show the baseline "Direct" method of training a standard SCFG-based semantic parser, the proposed method without language model verification "Tri-LM," and the proposed method using the Questions language model with NNLM reranking "Tri+LM."


An exhaustive enumeration over T quickly becomes impractical as problem complexity increases and n = |V ∪ {x}| grows.

on Monday, 375 students went on a trip to the zoo.