This creates some similarity between texts from different clusters, blurring the lines between them.

louvain and MCl follow a similar pattern on PAS: both perform at their peak on the most modular similarity metric (dependency), but louvain handles slightly less modular similarity metrics nearly as well as the most modular one, while MCl quickly falters.


k i is the degree of vertex i, so 2m is the expected number of edges between i and j in a random graph.

for instance, when multiple puzzle-composers write crossword puzzle clues for the same word, they will try to write creative, unique clues to make the puzzle interesting and challenging; clues for "star" could be "Paparazzi's target" or "Sky light."


Note also that the CLUE unigram tf-idf clusters appear tighter than the PAS unigram tf-idf clusters.

word2vec, LSTM autoencoders, and skip-thoughts ought to perform better than the n-gram-based methods and dependency count method when applied to CLUE, but should enjoy no advantage when applied to PAS.


Future work will explore further how the goals of short text authors translate into measurable properties of the texts they write, and how measuring those properties can help predict which similarity metrics and clustering methods will combine to provide the best performance.

we conclude both that CLUE is more tightly clustered than the other datasets and that surface-form-based measures yield tighter clusters than semantics-based measures.


In this paper, we propose a generative topic embedding model to combine the two types of patterns.

these two types of patterns are complementary.


Taking the logarithm of both sides, we obtain where m ik = L i j=1 δ(z ij = k) counts the number of words assigned with the k-th topic in 6 Variational Inference Algorithm Given the hyperparameters α, γ, µ, the learning objective is to find the embeddings V , the topics T , and the word-topic and document-topic distributions Here the hyperparameters α, γ, µ are kept constant, and we make them implicit in the distribution notations.

hinton and Salakhutdinov (2009) proposed to model topics as a certain number of binary hidden variables, which interact with all words in the doc-ument through weighted connections.


We could use these semantic centroids and the number of words around them to represent a document.

our method finished the 100 EM iterations in 2 hours.


In addition, on the two datasets, even the fastest Alias sampling in (Das et al., 2015) took over 2 hours for one iteration and 10 days for the whole 100 iterations.

we fix t i1 = 0, referring to it as the null topic.


A document's PLR is the average rank of the documents to which it has explicit positive links, among all documents, so lower PLR is better.

sBM assumes that each node belongs to only one block and each link exists with a probability that depends on the block assignments of its connecting nodes.


While topic assignments are updated iteratively, the weight vectors and matrix η, τ , and ρ are optimized in each global iteration over the whole corpus using L-BFGS (Liu and Nocedal, 1989).

the NN topic   Table 3: Average Association Scores of Topics splits into two topics and the proportions are not assigned to the same topic, which greatly brings down the link prediction performance.


As such sentiment lexicons that are built from standard dictionaries cannot adequately capture the informal language in social media text.

arabic is a morphologi-cally rich language where one lemma can have hundreds of surface forms; this complicates the tasks of Sa.


The proposed approach can not only correctly determine the author of a disputed document but also provide a way for measuring the confidence level of the authorship decision for the first time.

this approach applies an HMM in order to classify each sentence in document C into a class corresponding to its co-author.


Their work was tested on documents with up to 400 transitions among authors and the accuracy of their approach was not sensitive to the setting of parameters, in contrast with the approach in Giannella (2015).

each q i symbolizes the most likelihood author of observation o i , for 2.


For each segment, create a D-dimensional vector where the i th element in the vector is one (zero) if the i th element in the selected word set does (not) appear in the segment.

we perform our approach on New York Times articles.


We make this simplifying assumption for tractability, like many neural network approaches (Devlin et al., 2014).

in the case of this dataset, meaning representations always consist of a root production with two children: a trigger and an action.


They require larger datasets to outperform sparse linear approaches with careful feature engineering, as evidenced in work on syntactic parsing (Vinyals et al., 2015).

that requires segmenting the arguments into those that are general across all users, and those that are specific to the recipe's author.


The scheme was introduced to prevent reviewers from overlooking annotation issues due to passive approval.

tokens with automatically filled annotations included an additional # sign in a seventh column of each word's annotation.


To further assure annotation quality, each annotated sentence was reviewed by two additional annotators.

the EWT annotates possessive pronoun UPOS as PRON rather than DET, which leads the UPOS results in this setup to be lower than the PTB POS results.


Our best explanation for why the incorporation of preceding fMRI measurements is beneficial to our model, is that the grammatical function of a token may be predictable from a reader's cognitive state while reading preceding tokens.

wiktionary Of the 1,411 word types in the corpus, we find that 1,381 (97.84%) are covered by the wiktionary dump made available by Li et al.


RCNN achieves a better performance at learning features along the shortest dependency path, compared with some common neural networks.

sDP is a special structure in which every two neighbor words are separated by a dependency relations.


(2013) addressed the ontology mismatch problem (i.e., two ontologies using different vocabularies) by first parsing a sentence into a domainindependent underspecified logical form, and then using an ontology matching model to transform this underspecified logical form to the target ontology.

in this paper, we produce logical forms for each sentence rewritings using an agenda-based semantic parser , which is based on the lambda-DCS proposed by Liang (2013).


In Setting 2, our model outperforms the baseline on all sources except NW and BC, where the F-scores drop insignificantly by 0.1% for both sources.

hence, while unsupervised resolvers have achieved stateof-the-art performance, these weaknesses together suggest that it is very challenging to scale these models up so that they can achieve the next level of performance.


(2015), who applied domain-adaptation to the AES task using EasyAdapt (EA) (Daume III, 2007).

this is because for ctAP rank we assume that scores across tasks are not necessarily comparable.


learning a weight vector w such that w(x i − x j ) > δ, when essay i has a higher gold score than essay j, where δ is the one-sided margin 3 (Joachims, 2002;Chapelle and Keerthi, 2010).

one of the problems hindering the wide-scale adoption and deployment of AES systems is the dependence on prompt-specific training data, i.e.


We calculate Spearman's ρ using the ordinal gold score and the realvalued prediction on the original prompt-specific scoring scale of each prompt.

using the full representation allows the learning algorithm to capture both general and task-specific characteristics jointly.


(2) and (3), we propose an approximate solution to the problem defined by Eq.

7 http://nlp.stanford.edu/software/ CRF-NER.shtml 8 As F -1 score is usually used as the metric for sequence labeling, sentence level accuracy is more informative here.


to better capture the context information on both sides, two GRUs with opposite directions can be combined to form a bidirectional GRU (BiGRU).


For example, "Harry Potter" in "Who created the character Harry Potter" is more likely than "the character", "character Harry", etc.

we use the same subset of Freebase (FB5M) as our knowledge base so that the results are directly comparable.


To illustrate the effectiveness of this idea, we parameterize p κ (w|q) with a general-purpose neural labeling model, which consists of a word embedding layer, two layers of BiGRU, and a linearchain conditional random field (CRF).

lately, several large-scale generalpurpose KBs have been constructed, including YAGO (Suchanek et al., 2007), Freebase (Bollacker et al., 2008), NEll (Carlson et al., 2010), and DBpedia (lehmann et al., 2014).


By considering all ngrams, this approach usually achieves a high recall rate.

basically, it takes the element-wise average of the word embeddings of the question to be the question representation.


Based on the result, we conclude that BiGRU plays a crucial role in our focused pruning module.

a main difference among several approaches lies in the neural networks proposed to represent questions and KB elements.


Recognizing subcat frame similarity requires a common standardized format for the otherwise incomparable frames.

most of all, the linking to GermaNet and FrameNet shows that it is indeed possible to narrow down meaning components for Levin classes.


This work has been supported by the Volkswagen Foundation as part of the Lichtenberg-Professorship Program under grant No.

"Aspectuals": the inability to take thatclauses and to-infinitives in the past tense.


These classes contain many AO-selecting verbs that were not covered by Levin's classification.

the cross-lingual linking of our classes to Verb-Net senses requires an additional translation step, which we describe first.


When the syntactic parser fails to output a parse tree for a sentence, we encode the sentence with the sequential encoder by setting h (phr) root = 0.

our encoder differs from the original Tree-LSTM in the calculation of the LSTM units for the leaf nodes.


In SMT, it is known that incorporating syntactic constituents of the source language into the models improves word alignment (Yamada and Knight, 2001) and translation accuracy (Liu et al., 2006;Neubig and Duh, 2014).

effects of the proposed beam search Table 4 shows the results on the development data of proposed method with BlackOut (K = 2000) by the simple beam search and our proposed beam search.


(2015a) refined the attention model so that it can dynamically focus on local windows rather than the entire sentence.

the existing NMT models do not allow us to perform this kind of alignment.


Coordination is a frequent and important syntactic phenomena, that pose a great challenge to automatic syntactic annotation.


For example, the annotator provided the following annotation: "(The [humiliation]) are beyond measure", treating the determiner "The" as a shared modifier.

for the rest of the cases, we distinguish between NP phrases and non-NP phrases.


Later on, variance decreased and converged with more observations, which can be understood as the formation of the new social norms among users.

but in practice, users will generate their opinionated data with different paces, such that we have to postpone model adaptation until all the users have at least one observation to update their own adaptation matrix.


In this work, we assume that a topic model provides a reasonable proxy for the concepts a person might identify in a technical corpus.

there are several difficulties with this approach: (1) It is quite labor-intensive to manually generate a concept graph; (2) we expect only moderate agreement between graphs produced by different experts, who have different ideas of what concepts are important and distinct and which concepts are important to understanding others; and (3) the concept graphs we learn from a collection of documents will differ significantly from those we imagine, without these differences necessarily being better or worse.


The backoffs b LL are generated in suffix order, since each context produces a backoff value.


The similarity between two problems is calculated by the weighted Jaccard similarity between their corresponding vectors.

inference: After we train the model to get parameter vector ν, the predicated gold answer is selected from the candidate number subsets by maximizing ν •f (x i , y ik ).


Examples of these approaches were presented by Mihalcea and Faruque (2004).

they generally fail in providing substantial improvements in WSD performance.


... what was supposed to be a fantasy sports car ride at @entity3 turned deadly when a @entity4 crashed into a guardrail .

this way we model probability s i that the answer to query q appears at position i in the document d. In a functional form this is: Finally we compute the probability that word w is a correct answer as: where ) is a set of positions where w appears in the document d. We call this mechanism pointer sum attention since we use attention as a pointer over discrete tokens in the context document and then we directly sum the word's attention across all the occurrences.


Compared to our work these models were trained with Dropout regularization (Srivastava et al., 2014) which might improve single model performance.

our model combines the best features of the architectures mentioned above.


While the task is easy to evaluate, one can vary the context, the question Document: What was supposed to be a fantasy sports car ride at Walt Disney World Speedway turned deadly when a Lamborghini crashed into a guardrail.

empirical evaluation has shown that both models perform almost identically on the CNN and Daily Mail datasets.


However, there are approaches such as discriminative reranking (Collins and Koo, 2005) to rerank the output of an existing system that offer a means for further improving the performance of our SLL+RLL model.

from Table 2, we find that SLL+RLL model performs worse with respect to the opinion expression entities and opinion holder entities.


Our parser can insert empty elements at an arbitrary position to realize empty element detection.

there is an important   difference between ATTACH and REDUCE.


In the rare case that S has full column rank (if we have many training examples), then the left inverse of S exists, and there is exactly one consistent mapping, the true one (M * = S † T ), but we do not require this.


Finally, these values are forwarded to the fully-connected layer with softmax which generates the probabilistic distribution over the topic labels for a given input.

we present a problem definition of dialog topic tracking in Section 2.


Secondly, the use of external knowledge could be a key to success in dialogue topic tracking, as proved in the previous studies.

secondly, no correlation between different topics is considered neither in the topic decision process nor in each topic-specific sub-system operated independently from the others.


Finally, the generated errors from the models were categorized into the following error types:   Figure 7: Distributions of errors generated from the best model of each architecture.

the segmentation performances even with the best models were still very limited especially for inter-categorical transitions.


When the L1 lexico-semantic features are added to the model, we observe an absolute increase in the accuracy ranging from 0.57% (for RU subj ) to 1.43% (for ES dobj ).

the system has both a higher coverage in error detection (a rise in recall) and a higher precision.


The learners selected Chinese distractors, which matched their L1, 29.9% of the time over the three sessions.

the items produced jointly by these automatic methods, in both expert and learner evaluations, rivalled the quality of human-authored items.


It is important to note that the Sordoni dataset offers up to 10 references per message, while the Twitter Persona dataset has only 1 reference per message.

(2011) by rescoring out-puts using a SEQ2SEQ model conditioned on conversation history.


A variety of approaches have been proposed with the purpose of finding effective statistical information through the network Henderson et al., 2011;Tang and Liu, 2011).

let where η is a key parameter that balances the weights of the two objectives.


The stochastic gradient descent with back-propagation is performed using Adadelta update rule (Zeiler, 2012).

in the context of concept normalisation, Limsopatham and Collier (2015a) showed that effective performance could be achieved by mapping the processed social media messages and medical concepts using the similarity of their embeddings.


In addition, the choice of embeddings has a marked impact on the achieved performance.

these results show that the semantics captured in word embeddings are useful for both CNN and RNN approaches for medical concept normalisation.


words surrounding the input phrase); however, such contexts are not available in any of the three datasets.

the MetaMap system of Aronson (2001) applied a rule-based approach using pre-defined variants of terms when mapping texts to medical concepts in the UMLS Metathesaurus 3 .


We use a CNN architecture with a single convolutional and pooling layer, as shown in Figure 1.

we have motivated the importance of semantics when normalising medical concepts in social media messages.


Finally, Section 7 provides concluding remarks.

we investigate the use of techniques from two families of deep neural networks, i.e.


When compared with DNorm and P-MT with GNews baselines, which are the most effective existing baselines, we observe that both CNN and RNN significantly (p < 0.05, paired t-test) outperform the two baselines for all of the three datasets.

the GNews embeddings benefit both CNN and RNN more than the BMC embeddings, which is in line with the previous finding that GNews is more useful than BMC for the EmbSim and P-MT baselines.


Although it is appealing to apply our approach to dealing with real-world non-parallel corpora, it is time-consuming and labor-intensive to manually construct a ground truth parallel corpus.

they introduce a corpus-level latentvariable translation model in a non-parallel scenario and develop a training algorithm that alternates between (1) using a parallel lexicon to extract parallel phrases from non-parallel corpora and (2) using the extracted parallel phrases to enlarge the parallel lexicon.


The new translation probabilities can be obtained by Counts of target-to-source length and translation models can be calculated in a similar way.

the availability of large-scale, wide-coverage corpora still remains a challenge even in the era of big data: parallel corpora are usually only existent for resourcerich languages and restricted to limited domains such as government documents and news articles.


where l is one-hot representation of the groundtruth label l;l is predicted probabilities of labels; C is the class number.

experiment results demonstrate that our proposed architecture is more effective.


We neglected such time-related phrases when defining C 0 .

indeed, job dissatisfaction poses serious health risks and has even been linked to suicide (Hazards Magazine, 2014).


Due to memory constraint in GPUs, we limit our source and target sequences to a maximum length of 150 each, i.e., we backpropagate through at most 300 timesteps from the decoder to the encoder.

to address that concern, we propose another approach called the separate-path target generation.


For example, English POS taggers benefit from carefully designed word spelling features; orthographic features and external resources such as gazetteers are widely used in NER.

our end-to-end model slightly improves this model by 0.01%, yielding a state-of-the-art performance.


In the past few years, non-linear neural networks with as input distributed word representations, also known as word embeddings, have been broadly applied to NLP problems with great success.

for the tasks of POS tagging and NER we try to share as many hyper-parameters as possible.


Common vector representations include lexical Vector Space Models and Latent Semantic Analysis (LSA) (Yannakoudakis, 2013).

additionally, if any of the test or example response utterances are short, then their topic vector representations may not be robustly estimated.


The essence of the standard approach to topic detection is to assess the semantic distance D sem between the test response w and the aggregate example response w q by approximating it using a vector distance metric D vec between vector representations of the response f w and the topic f q .

the candidate randomly selects with uniform probability P(w|q) a response w from the pool W q .


The RNNLM could be used in a KNN-style approach, where it associates each example response with its individual topic vector, using L(q) as a distance metric.

• SC-EX3: Talk about a company in your local town which you admire.


Our approach does away with the need for post processing, and avoids complications caused by reordering of morphemes in previous approaches.

second, the lexical translation tables, trained on compound split German data, which form the basis of our generation approach.


Table 4 shows a few example cases from the experiment with stratified 80%-20% train-test split.

as far as we know, these features are being introduced in NLP tasks like textual sarcasm detection for the first time.


Many traditional, feature-based relation classification models extract features from both sequences and parse trees (Zhou et al., 2005).

aCE04 defines the same 7 coarse-grained entity types as aCE05 (Doddington et al., 2004), but de-fines 7 coarse-grained relation types: PYS, PER-SOC, Employment / Membership / Subsidiary (EMP-ORG), aRT, PER/ORG affiliation (Other-aFF), GPE affiliation (GPE-aFF), and Discourse (DISC).


Argument mining research has studied different data-driven approaches for separating organizational content (shell) from topical content to improve argument component identification, e.g., supervised sequence model (Madnani et al., 2012), unsupervised probabilistic topic models (Séaghdha and Teufel, 2014;Du et al., 2014).

the authors first use topic prompts in development data of unannotated persuasive essays to semi-automatically collect argument and domain seed words.


The baseline achieves significantly higher F1:Support than WINDOW model.

we also extract discourse relations between each pair of source context sentence and target context sentence.


tures that are extracted by exploiting context sentence windows and writing topic to improve relation prediction.

three experts identified possible argument components of three types within each sentence in the corpus (MajorClaim -writer's stance toward the writing topic, Claim -controversial statements that support or attack MajorClaim, and Premiseevidence used to underpin the validity of Claim), and also connected the argument components using two argumentative relations (Support and Attack).


Due to the data sparsity issues (Bauer and Koller, 2010), we use unlexicalized probabilities.

in the open action policy , we choose open actions according to their heuristic values, instead of just their tree probabilities.


Given an ex- isting sentence tree, an auxiliary tree can be applied in a three-step process called "adjunction".

we must have at least one Algorithm 4 calcReward (Algorithm 2, line 13) score −= C 4 |S.sentence| score /= C 5 |B| 11: end if 12: return score valid binding, as this indicates that our partial sentence is factual (with respect to the world); however, more than one binding means that the sentence is ambiguous, so a penalty is applied.


TA emphasizes documents that are both informative to the classifier and representative of a topic; if a document is not representative, the surrounding context of a topic will be less useful.

topic Models topic models (Blei et al., 2003) automatically induce structure from a text corpus.


The labels assigned in different conditions and the automatic label of the document's prominent topic construct the candidate labels for the document.

when time is very  limited, using traditional active learning (LA) is preferable to topic overviews; users need time to explore the topics and a subset of documents within them.


If a merged label gets a "best" or "worst" vote, we split that vote across all the identical instances.

establishing the label set remains difficult.


LSTMs have been successfully applied to various tasks, such as speech recognition (Graves et al., 2013), machine translation (Luong et al., 2015), and natural language generation (Wen et al., 2015).

word embeddings were set to size 300 and initialised using the publicly released pretrained word2Vec vectors .


The ability to condition on the previous label also does not provide much help on this task -there are only two possible labels and the errors are relatively sparse.

there has been limited work on more general error detection systems that could handle all types of errors in learner text.


This is in contrast to Ir-  2014, who experimented with Elman-type networks and found some improvements using multiple layers of Bi-RNNs.

the performance of the bidirectional recurrent network (Bi-RNN) is somewhat lower, especially on the test set.


Roth and Lapata (2015) recently introduced features that model the influence of discourse on role labeling decisions.

at test time, we select the structure with the highest overall score, which we compute as the geometric mean of the global regression and all argument-specific scores.


As our starting point, we use the "words-asclassifiers" model recently proposed by Kennington and Schlangen (2015).

for uniquely referring expressions ("the red cross"), what is required is to pick the most likely candidate from the distribution: the prediction of an expression such as "the brown shirt guy on right" is computed by first getting the responses of the classifiers corresponding to the words, individually for each object.


The strengthen mapping is similar to the hard tanh in neural network.

in this paper, we propose two improvements based on the event extraction baseline JET.


Entailment annotations are then provided for a subset of sentences from the document corpus, based on a Lucene filter for each hypothesis.

we trained the systems on the training set of each domain and tested on the test set.


Once the positive instances are exhausted, more negative instances are added, resulting in some class imbalance that hurts performance (even though more training data is being added overall).

a variation of this task, dubbed textual entailment search, has been the focus of RTE-5 and subsequent challenges, where the goal is to find all sentences in a corpus that entail a given hypothesis.


We post a feature that aggregates matched elements of detected frames.

our active learning experiments demonstrate that we can match (and even beat) ENT using only 6.6% of the training data in the clinical domain, and only 5.8% of the training data in the newswire domain.


instead, we use binary question-answering (true / false questions) as an intuitive formulation of non-restrictive modification.


Most relevant to this work is the comparison between Open-IE and Semantic Role Labeling (Carreras and Màrquez, 2005).

future work can use our annotated corpus to develop classifiers that deal better with prepositional and adjectival modifiers, which require deeper semantic analysis.


γ i is drawn from a Beta distribution with the priors η 0 and η 1 (Step 9).

multilingual topic models other than BiLDA (Section 2) have been proposed for document-aligned comparable corpora.


We collected French articles that correspond to the English articles used in the experiments in Section 5, from the French Wikipedia database dump (2 June 2015) based on inter-language links.

(2010) projected monolingual models based on PLSA or Principal Component Analysis into a shared multilingual space with the constraint that document pairs must map to similar locations.


(2014), in exploring vector calculations that realize logic operations.

the equation 5is intended to degrade long paths which pass through several high-valency nodes.


this simply means that there is no change to the existing formulation of the LStM.


Website privacy policies are often ignored by Internet users, because these documents tend to be long and difficult to understand.

our preliminary analysis has shown that the policy vocabularies associated with certain annotations are very distinctive (e.g., the Do Not Track category or financial information as a data type, for example), lending themselves to automatic identification.


By contrast, in the CL learning regime, the incremental loss is computed as − log p (y t ), where p (y t ) is normalized only over the values of y t that are possible continuations once the grammar-induced constraints are taken into account, ignoring whatever weights the RNN predictor may (wrongly) believe should be put on impossible continuations.

when the grammar is known (in fact, as soon as the CFG core of the grammar is known), two properties of the DS hold (we omit the easy algorithms underlying these properties; they involve using a prefix of the DS for constructing a partial derivation tree in a top-down fashion): 1. knowing the DS uniquely identifies the derivation tree.


The word embedding layer has 100 dimensions, which is also the case of the hidden layer of the LSTM u l,t .

the search is based on a so-called "floating parser" (Pasupat and Liang, 2015), a modification of a standard chart-parser, which is able to guide the search based on the similarity features.


The main disadvantage of simple concatenation is that word embeddings are commonly used to initialize words in DNN systems; thus, the high-dimensionality of concatenated embeddings causes a great increase in training parameters.

gloVe contributes most in SimLex-999, WS353, MC30 and MEN; word2vec contributes most in RW and word analogy tasks.


Second, recurrent neural network models have been used to extend the range of neural features beyond a local window .

the modified trees are always structurally different instead of only with different dependency labels compared to the original one, which guarantees structured diversity.


Besides, CRFs can only employ word-level features, which are either hand-crafted or extracted with deep neural networks, such as window-based neural networks (Collobert et al., 2011) and bidirectional Long Short-Term Memory networks (BI-LSTMs) .

we also denoise our gazetteers by removing overlapped entities and using BBN Pronoun Coreference and Entity Type Corpus (weischedel and Brunstein, 2005) as filters 10 .


The explosion caused people to evacuate the building.

we show that these features improve causal inference by an 11.05 point increase in F-measure over a naive baseline in 6.


It is easy for an annotator to overlook a causal relation when reading through a large quantity of text.

these examples do not contain standard causal connectives and occur infrequently in the data, so the lexical semantic features help to identify them.


In comparison, in the task of identifying discourse relations in the PDTB these features yield an 75.33 F-score and 85.85% accuracy in distinguishing between discourse and non-discourse usage  and an accuracy of 93.67% when distinguishing between discourse classes.

the the overall accuracy is still much lower than if we use informative features.


Many similarity scores are of the following form comparing sentences W and W : where σ(w, w ) is a score 1 between 2 words and Z is a normalizer ensuring the score is between 0 and 1.

the words can not be substituted in the following sentence: The baker made a cake.


The memory required is for storing embeddings of entities and relations, and the time is for scoring 2(η + 1) triples for every fact in the training KB.

the overall time for this method per training iteration is O 2d(η + 1)E kb T We should note that whether this method or the one of Guu et al.


1 Our methods are similar to past work on KBC (Mintz et al., 2009;Nickel et al., 2011;Lao et al., 2011;Nickel et al., 2012;Gardner et al., 2014;West et al., 2014), particularly methods based on distributed representations and neural networks (Socher et al., 2013;Bordes et al., 2013;Bordes et al., 2014a;Bordes et al., 2014b;Yang et al., 2014;Gu et al., 2015;Toutanova et al., 2015).

we want to design and train models that define a function score(t 1 , R, t 2 ) that provides a quality score for an arbitrary tuple t 1 , R, t 2 .


With no initial lexicon and longer contextdependent texts, our situation is exacerbated.

moreover, they can be used to bootstrap the full model.


To do this, the model predicts a t at each step using a simple two-way softmax classifier whose input is the state of the tracking LSTM: The above model is nearly the simplest viable implementation of a transition decision function.

on pairs with premises of 20 or more words, the RNN's 76.7% accuracy, while SPINN-PI reaches 80.2%.


In this section, we will get into the details of Experiment II: decoding dependency relation from word representation.

despite its popularity, character-based approach also comes out as an equal competitor (Santos and Zadrozny, 2014;Kim et al., 2016;Ling et al., 2015b;Ling et al., 2015a;Faruqui et al., 2016;Ballesteros et al., 2015) .


Current research has gaps for understanding model behaviours towards language typological diversity as well as the utility of context and form for different languages.

for word sentiment feature, we use the manually annotated data collected by Dodds et al.


we select three representative types of models and design a series of experiments to reveal the universals and specifics of various word representations on decoding linguistic properties.


What is the role of word frequency in meaning change?

qualitatively, we found SGNS to be most useful for discovering new shifts and visualizing changes (e.g., Figure 1), while SVD was most effective for detecting subtle shifts in usage.


The Google N-Gram datasets are extremely large (comprising ≈6% of all books ever published), but they also contain many corpus artifacts due, e.g., to shifting sampling biases over time (Pechenick et al., 2015).

: Top-10 English words with the highest semantic displacement values between the 1900s and 1990s.


), and annotations with respect to y verb (coarse-(C) and fine-grained (F) labels per temporal anchor: before, during and after).

but determining for how long the agent of drive remains at the destination depends on the arguments of drive: from [John] AGENT [drove] v [home] DESTINATION [after an exhausting work day] TIME , it is reasonable to believe that John will be located at home overnight.


This strongly suggests that our conclusions can be extended to lemmas with low LEXSEMTM frequency, though more work is needed to confirm this.

these experiments are all performed using the BNC dataset (see Section 4.1).


In addition, previous work by Lau et al.

because we found the usages contained substantially fewer tokens on average compared to the BNC dataset, we included two sentences rather than one on either side of the target lemma location where possible (giving 5 sentences in total), which gave a better match in usage size.


Target word: lonely 9Context: "Again, he left that up to you.

in order to test for strong biases in the data, we constructed Sup-CBOW, a baseline model weakly tailored to the task at hand, consisting of a simple neural network that takes as input a bag-ofword representation of the passage and attempts to predict the final word.


For a given passage, 1. one human subject guessed the target word based on the whole passage (comprising the context and the target sentence); if the guess was right, 2. a second subject guessed the target word based on the whole passage; if that guess was also right, 3. more subjects tried to guess the target word based on the target sentence only, until the The subjects in step 3 were allowed 3 guesses per sentence, to maximize the chances of catching cases where the target words were guessable from the sentence alone.

although co-reference seems to play a big role, only 0.3% of target words are pronouns.


When estimating an HMM of a low order with data which was generated from a higher order model, EM does quite poorly.


Our models yield high performance on treebanks of varying sizes from 5,000 sentences (Hebrew and Swedish) to 40,472 sentences (German).

these differences are narrowed, and in some cases, overcome, when the number of latent states is optimized.


Bottom: Comparison against state-of-the-art baselines utilizing the POS tags.

because the connections for any specific parsing decision are fixed given the derivation, we can still extract examples for training off-line by unrolling the network structure from gold derivations.


In addition to lexical features we extracted a set of stylistic features including emoticons, elongated words, capitalization, repeated punctuation, number of hashtags and took into account the clauselevel negation (Pang et al., 2002).

we also demonstrated that it is possible to accurately predict many perceived demographic traits of Twitter users based solely on the emotional contrast between them and their neighbors.


It's Friday #joy, they still demonstrate that a hashtag is a reasonable representation of real feelings (González-Ibáñez et al., 2011).

these are simply features of the text a user produces, and make limited use of the social embedding of the user in the network.


We provide experiments that evaluate all three components of LOBBYBACK.

the two clustering algorithms differ somewhat in precision.


We assume that π is a K-vector (where we take the limit as K → ∞) where π k describes the population proportion of documents that contain latent feature k. We suppose that π is generated by the stick-breaking construction (Doshi-Velez et al., 2009).

the random assignment of texts to individuals induces a distribution over a probability measure on treatment vectors Z, f (Z) = this requires that every combination of treatment effects is possible from the documents in our corpus.


Algorithm 1 Bandit Expected Loss Minimization 1: Input: sequence of learning rates γ t 2: Initialize w 0 3: for t = 0, .

the result can be explained by considering important empirical factors such as the variance of stochastic updates.


For the text game "Machine of Death", we restrict an episode to be no longer than 500 steps.

it is desirable to use two networks with different structures to handle state/action texts, respectively.


As the result, the selective read often takes rigid move and tends to cover consecutive words, including UNKs.

different from the canonical encoderdecoder architecture, the attention-based Seq2Seq model revisits the input sequence in its raw form (array of word representations) and dynamically fetches the relevant piece of information based mostly on the feedback from the generation of the output sequence.


Since the filled substrings have no overlaps in DS-II, the performance of RNNSearch drops significantly as it cannot handle words unseen in training data.

the attention mechanism was first introduced to Seq2Seq  to release the burden of summarizing the entire source into a fixed-length vector as context.


(2015) introduced a heuristics to postprocess the translated sentence using annotations on the source sentence.

cOPYNET, with its ability to handle the OOV words with the copying mechanism, performs however slightly better with the word-based variant.


They assume homogeneous labels and expect the aligned features across domains to behave similarly for the prediction task.

single source adaptation (Chen et al., 2009;Ando and Zhang, 2005;Daumé III, 2009) primarily aims at minimizing the divergence between the source and target domains either at instance or feature levels.


This transferable knowledge is accumulated as an auxiliary training set for an algorithm to learn the target domain classification task followed by suitable transformation of the auxiliary training instances.

transfer learning techniques allow domains, tasks, and distributions used in training and testing to be different, but related.


Unfortunately, we do not currently know of a way to smooth these bit-string encodings (also found by WORD2VEC).

this is a clear win for lower-resource languages.


The number of training iterations was set to 5 (no tuning) and we set α = 1 and β = 1 (no tuning).

a systematic comparison of these models is missing from the literature, making it difficult to analyze which approach is suitable for a particular NLP task.


2013c, but we include a much wider range of lexical semantic relations, especially those standardly evaluated in the relation classification literature.

this is interesting from a DIFFVEC point of view, since it shows that the lexical semantics of one word in the pair can overwhelm the semantic content of the DIFFVEC (something that we return to investigate in §5.4).


Reducing α further to 1 × 10 −4 , however, results in lower BLEU scores.

10suggests there is no need to differentiate ∆(y, y (s) ), MRT allows arbitrary nondifferentiable loss functions.


Between languages that share an alphabet, names can often be copied from source to target text.

if alphabets differ, transliteration is required (Durrani et al., 2014).


One avenue of future research is to learn the optimal vocabulary size for a translation task, which we expect to depend on the language pair and amount of training data, automatically.

when alphabets differ, like in English→Russian, the subword models do much better.


• Word+NTN: We use the neural tensor defined in (8) to capture the semantic interaction scores between every word embedding pair, the rest of the method is the same as our proposed method.

the rest of the method is the same as our model.


The work of Siegel and McKeown (2000) is a major inspiration in computational work on modeling these linguistic phenomena, introducing the use of linguistic indicators (see Section 5.1).

such a pipeline approach is not competitive with the model trained directly on SE types (see Section 6.3).


Experiments showed that the resulting bilingual comparable corpus has better performance than a monolingual corpus, indicating that image understanding using a resource-rich language benefits a resource-poor language.

existing work in the cross-lingual setting is usually formulated as follows.


There are five sentences about a hyper-linked image, and several sentences require fixes in order to satisfy the conditions below.

the CIDEr-D score was about 4% higher than that for the monolingual baseline.


Im-ageNet (Deng et al., 2009)) or retrieved from the web using search engines, as illustrated in Fig 1.

(2011) first find leaf nodes and then use lexical patterns to find intermediate terms and all the attested hypernymy links between them.


Moreover, we leverage the distributed representations of images and words to construct compact and effective features.

the feature vector f is used to measure the local semantic consistency between category x n and its parent category x n as well as its siblings c n \x n .


We apply logistic regression to train a separate classifier for each attribute.

after summarizing the annotations from three different annotators, each sentence is represented by a 18-dimension causality vector.


If d = 0, it seems intuitive to interpret the second term as minimizing the summary redundancy, which is in accordance to previous works.

rOUGE-N relies on reference summaries which forces us to perform an estimation step.


The high agreement shows that the annotation scheme provides an effective way of consistently annotating learner corpora with phrase structures.

a replacement type error is an error where a word should be replaced with another word.


HLR combines a psycholinguistic model of human memory with modern machine learning techniques, and generalizes two popular algorithms used in language learning technology: Leitner and Pimsleur.

the top two HLR variants are not far behind, and they also reduce MAE compared to Leitner by least 45%.


What constitutes "comprehensible input" for a learner varies as their knowledge of the L2 increases.

the reordering positions are calculated using the word and phrase alignments obtained from Moses.


We rely on just 6 features at present (see section 6 for future work), although each is complex and real-valued.

reordering Clue ( Figure 2): A German substring is moved into a more English-like position.


Computational approaches also proved beneficial for theoretical research in second language acquisition (Jarvis and Pavlenko, 2008).

we are interested in the reasons for the differences between translations and originals, on one hand, and native and non-native language, on the other.


Autoencoders are trained using a local unsupervised criterion (Vincent et al., 2010;Hinton and Salakhutdinov, 2006;Vincent et al., 2008).

cSAE-LGc performs significantly better than the baselines, including the semi-supervised approach in Rothe and Schütze (2015).


Denoising autoencoders (DAEs) reconstruct an input from a corrupted version of it for more effective learning (Vincent et al., 2010).

injecting context information improves the reconstruction power of our model by providing more information.


For retrieval, our unsupervised approach that merely ranks inputs with respect to the cosine similarity between their hidden representations shows comparable performance with the state-of-the-art supervised models and in some cases outperforms them.

we use Non-Negative Matrix Factorization (NMF) (Lin, 2007): Given a training set with n instances, i.e., X ∈ R v×n , where v is the size of a global vocabulary and the scalar k is the number of topics in the dataset, we learn the topic matrix D ∈ R v×k and context matrix C ∈ R k×n using the following sparse coding algorithm: where each column in C is a sparse representation of an input over all topics and will be used as global context information in our model.


We argue that quality is too complex to be represented by a single number and should be instead decomposed into a set of simpler variables that capture the different linguistic and narrative aspects of online news.

this shows that the text comprehension and writing style are aspects that are more relevant than sentiment.


Sentimentality: Sentimentality is a measure of the total magnitude of positive or negative statements made in the article regarding an object or an event.

we can identify benchmarks for characterising news article quality in an automatic and scalable way and, thus, predict poor writing before a news article is even published.


Note CoNLL testa  216  4791  CoNLL testb  231  4485  TAC2014 DEL test  138  2817   Table 1: Document and linkable mention counts for CoNLL and TAC2014 DEL datasets.

the association is performed using article titles, redirections, disambiguation pages, and textual anchors.


Given a named entity mention, we are thus able to retrieve the named entity mentions which are most similar in the distributional vector space.

we add an exponent coefficient to the probabilities, thus allowing to control the contribution of each model.


Historically, MT metrics have been evaluated by how well they correlate with human annotations (Callison-Burch et al., 2010;Machacek and Bojar, 2014).

these three metrics were chosen because they present three very different approaches for quantifying semantic similarity, namely: sentence alignments, compression redundancies, and edit rates.


Our language model is implemented on a GPU, but its general design (and much of the actual code) is likely to be useful to other hardware that supports SIMD parallelism, such as the Xeon Phi.

let Σ be a our vocabulary, Σ n the set of all n-grams over the vocabulary, and Σ [N ] the  Federico et al.


As the inference is intractable, we propose to sample the full search space to improve the efficiency.

√ " means the corpus is included in the training data and × means not included.


The test perplexities (Table 2) and validation learning curves (Figures 2, 3, and 4) show that the competitiveness of softmax diminishes with larger vocabularies.

figure 7 shows validation perplexity versus the number of iterations for a week of training.


This figure shows that a softmax model with 1024 hidden units in the last layer could perform better than the 512-hidden unit model with a longer training horizon.

for neural models, we train 11-gram models for gigaword and bil-lionW; for PTB we train a 6-gram model.


As future work, we plan to examine the use of a voting scheme for combining the output of complementary DSMs.

compositionality identification is a first step towards complete semantic interpretation in tasks such as machine translation (to translate non-compositional compounds as a unit), word sense disambiguation (to avoid assigning a sense to parts of non-compositional compounds), and semantic parsing (to identify complex predicates and their arguments).


We measured all runtimes on an Intel Xeon E7-8857 CPU at 3 GHz using Java 8.

classical grammar formalisms such as TAG (Joshi and Schabes, 1997) and ccG (Steedman, 2001) have been equipped with expressive statistical models, and high-performance parsers have become available (clark and curran, 2007;Lewis and Steedman, 2014;Kallmeyer and Maier, 2013).


This is a crucial property, because entailment reflects what information is and is not known; a representation y entails a representation x if and only if everything that is known given x is also known given y.


To facilitate the creation of ASL content that can easily be updated or maintained, we have investigated technologies for automating the synthesis of ASL animations from a sparse representation of the message.

similar to the Profile HMM, the CPM has strict left-to-right transition rules, constrained to only move forward along a sequence.


Using a very standard, and in fact somewhat dated sentiment analyzer, we are regularly able to garner annualized returns over twice that percentage, and in a manner that highlights two of the better design decisions that Zhang and Skiena (2010) made, viz., (1) their decision to trade based upon numerical SVM scores rather than upon discrete positive or negative sentiment classes, and (2) their decision to go long (resp., short) in the n best-(worst-) ranking securities rather than to treat all positive (negative) securities equally.

the oracle strategy computes the value of the stock h days into the future.


This paper provides an example of how to test that utility.

we trade based upon the raw SVM score itself, rather than its relative rank within a basket of other securities as Zhang and Skiena (2010) did, and we experimentally tune a threshold for that score that determines whether to go long, neutral or short.


We believe this contribution has the potential to affect future research on additional NLP problems.

the final undirected tree will have a designated root vertex.


Moreover, the occurrence of non-topic words such as emotional, sentimental, functional and even meaningless words are very common in microblog posts, which may distract the models from recognizing topic-related key words and thus fail to produce coherent and meaningful topics.

the various C variables refer to counts excluding the message m on conversation tree t. For each message m on a tree t, we sample the leader switcher y t,m and topic assignment z t,m according to the following conditional probability distribution: where g(s, k, t, m) takes different forms depending on the value of s: For each word n in m on t, the sampling formula of its background switcher is given as the following: where To evaluate our LeadLDA model, we conducted experiments on real-world microblog dataset collected from Sina Weibo that has the same 140character limitation and shares the similar market penetration as Twitter (Rapoza, 2011).


In the next step, we plan to exploit fine-grained discourse structures, e.g., dialogue acts (Ritter et al., 2010), and propose a unified model that jointly inferring discourse roles and topics of posts in context of conversation tree structures.

due to the severe sparsity of short text posts, we modify the calculation of commonly-used topic coherence measure based on word co-occurrences in messages tagged with the same hashtag, named as hashtag-document, assuming that those messages discuss related topics 8 .


Therefore, (Zeng et al., 2015) combines at-least-one multi-instance learning with neural network model to extract relations on distant supervision data.

kBs due to the lack of human-annotated training data.


Similar to Markov Logic Networks (MLNs) (Richardson and Domingos, 2006), it uses weighted first-order logic formulas to compactly encode complex undirected probabilistic graphical models.

for example, in the first sentence, He and hospital are the arguments, and in the second sentence, Aeroplanes and London are the fEs.


However, in this situation, H1 can help to rectify it because the majority of LUs under Clothing are not ambiguous words.

event Sample in FN Attack Attack Aeroplanes bombed London.


In this case, our assumption is that before finding the 70 seashells Joan had an empty hand.

such knowledge is not always present in ConceptNet which results in error.


inferring entailment relations based on the entailments generated by INS and DEL editsdoes not enforce that all of the ANs fit into one of the five entailment relations defined by natural logic.

we also train a standard unigram model (BOw).


The symmetric reverse entailment can also occur, in which the N is a subset of the set denoted by the AN.

this is not necessarily true that running entails playing-maybe the dogs are being (1) FraCas p No delegate finished the report on time.


an author who claims the prison system in a lot of countries is outdated would Agree Somewhat with the first part of Table 1's second prompt).

identifying stancetaking text in the midst of nonstancetaking sentences in a potentially long essay, as we will see, is by no means a trivial task.


After that, we count the number of times the identified stancetaking path is negated.

to do this, we semantically parse the sentence using SEMAFOR (Das et al., 2010).


To ensure consistency in annotation, we randomly select 100 essays (187 instances) for annotation by both annotators.

we train a multiclass classifier on our data set using a feature set composed solely of unigram, bigram, and trigram features, each of which indicates the number of times the corresponding n-gram is present in the associated essay.


The analysis result shows that the new metric gives a more balanced evaluation result towards the human intuition of the segmentation quality.

'S' indicates that the word consists of only a single character.


the Krippendorff's α for all begin point annotations is 0.629 and for all end point annotations it is 0.737.


The TimeBank-Dense Corpus annotated all TLINKs between Event-Event, Event-Time, and Time-Time pairs in the same sentence and between succeeding sentences as well as all Event-DCT and Time-DCT pairs.

for the majority of events, the most informative temporal expression is not in the same or in the preceding / succeeding sentence.


They used CoNLL-train and Lang-8 as non-native data and English Wikipedia as native data.

an MT system can easily handle interacting and complex errors where replacements involve a sequence of words.


This is achieved by having two hidden components in the intermediate layer, as schematically shown in the fig 1.

more significantly, the proposed models obtain significantly improved performance on the disease name classification task.


We use a corpus that consists of texts published in The Guardian daily newspaper that is actively used by the authorship attribution community in cross-domain studies (Stamatatos, 2013;Sapkota et al., 2014;Sapkota et al., 2015).

there are a handful of authorship attribution studies that explore cross-domain authorship attribution scenarios (Mikros and Argiri, 2007;Goldstein-Stewart et al., 2009;Schein et al., 2010;Stamatatos, 2013;Sapkota et al., 2014).


One of the typical examples that satisfy Claim E is an idiomatic expression.

(0.4%) date-DAT woman-ACC ask ACC-DAT: Josei-wo dêto-ni sasou.


(2.5%) affection-ACC word-DAT feel (φ I feel the affection in φ your words.)

the third claim, by Miyagawa (1997), asserts that both the DAt-ACC order and ACC-DAt order are canonical for all cases.


However, most of them required either manual analyses or measurements of human characteristics such as brain activities or reading times for each example.

in addition, arguments are often omitted in Japanese when we can easily guess what the omitted argument is or we do not suppose a specific object.


That is, even if there are several possible word orders for essentially same sentences consisting of the same elements, only one of them is regarded as the canonical word order and the others are considered to be generated by scrambling it.

there are a number of known features that affect word order.


While it is difficult to say whether a word order is canonical or not from one specific example, we can consider that a word order would be canonical if it is overwhelmingly dominant in a large corpus.

in the case of pass-type verbs, the accusative argument is the subject of its corresponding inchoative sentence as shown in Example (7).


Claim D: The canonical word order varies depending on the semantic role and animacy of the dative argument.

if we presume that there is only one canonical word order for Japanese double object constructions, this result suggests that the DAT-ACC order is the canonical one, as claimed by Hoji (Claim A).


To inspect Matsuoka (2003)'s claim that the DAT-ACC order is canonical for show-type verbs, whereas the ACC-DAT order is canonical for passtype verbs, we investigated the proportions of the ACC-DAT order for several pass-type and showtype verbs.

these claims can be verified by investigating the distribution of word orders for each verb.


We thank Dongdong Zhang, Junwei Bao, Zhirui Zhang, Shuangzhi Wu and Tao Ge for helpful discussions.

the encoder-decoder system (Enc-Dec) does not generate the translation of "白 色 的"/"white", and fails to preserve the correct meaning of Source1, shown in Table 1.


As with many NLP approaches, most of the previous EL research have focused on English, mainly because it has many NLP resources available, it is the most prevalent language on the web, and the fact that the English Wikipedia is the largest among all the Wikipedia datasets.

the system displays a robust and strong empirical evidence by not only outperforming all stateof-the-art English EL systems, but also achieving very good performance on multiple Spanish and Chinese entity linking benchmark datasets, and it does so without the need to switch, retrain, or even translate, a major differentiating factor from the existing multi-lingual EL systems out there.


Moreover, we could obtain 84.2 F1 score, using the ranking based loss function (dos Santos et al., 2015) and fine-tuning of the distributed representations initially trained by GAC.

the dataset is publicly available on the Web site 1 .


Although these reports described good performance on the respective tasks, we are unsure of the generality of distributed representations trained for a specific task such as the relation classification.

here, the length of a relational-pattern pair is defined by the maximum of the lengths of two patterns in the pair.


Experiments show that the new dataset does not only enable detailed analyses of the different encoders, but also provides a gauge to predict successes of distributed representations of relational patterns in the relation classification task.

we compare RNN, LSTM, and GRU empirically with the same training data and the same training procedure.


It is closely related to the task of coreference resolution (Van Deemter and Kibble, 2000), for which a myriad of solutions have been proposed (Clark and Manning, 2015;Peng et al., 2015;Wiseman et al., 2015;Björkelund and Farkas, 2012;Lee et al., 2011;Stoyanov et al., 2010;Ng, 2008;Bergsma and Lin, 2006;Soon et al., 2001).

it represents a form of negation of group membership (e.g.


∈ measures how similar E is to sentence ′ and then ∑ ( ′ , ) ∈ is the largest value that ∑ ( ′ , ) ∈ can achieve.

the diversity of the summary is enhanced.


For topic modeling, we adopted the most popular LDA to discover topics in the two datasets, respectively.

it is usually very hard for a user to understand the discovered topics based only on the multinomial distribution of words.


The whole sentence is rooted at a designated special symbol ROOT, thus the dependency graph for a sentence is constrained to be a rooted, directed tree.

for prefix, we directly use the last LSTM hidden vector in prefix to represent it, which equals to subtract a zero embedding.


Based on firstorder model, higher-order models (McDonald and Pereira, 2006;Carreras, 2007;Koo and Collins, 2010;Ma and Zhao, 2012) factor a dependency tree into a set of high-order dependencies which bring interactions between head, modifier, siblings and (or) grandparent into their model.

distance features are encoded as randomly initialized embeddings.


In these models, subgraphs are usually represented as high-dimensional feature vectors which are then fed into a linear model to learn the feature weights.

it computes the forward hidden sequence − → h , the backward hidden sequence ← − h and the output sequence v by iterating the forward layer from t = 1 to n, the backward layer from t = n to 1 and then updating the output layer: l is the dimensionality of LSTM hidden vector.


Though all the components contribute to the model, the primary one contributes the most due to the exponential effect (exp(•)).

to this end, knowledge graph embedding has been proposed to project entities and relations into continuous vector spaces.


TransM (Fan et al., Figure 2: Visualization of multiple relation semantics.

different semantics are characterized by different components in our em-bedding model.


Note that, in the Wikipedia page of the topic entity, we may collect more than one sentence that contain a candidate answer.

suppose the pair (e gold , r gold ) represents the gold entity/relation pair for a question q.


We first introduce these neural models from a high level description, and then elaborate them in more details.

we propose to evaluate results from 3 different evaluation metrics.


As a strongly competitive baseline of the neural model LSTM-RNN, the perplexity performance gets boosted in the generation process, which indicates that neural models show strong ability for language generation.

as mentioned, generally people regard Chinese couplets as a reduced form of Chinese poetry and there are several large Chinese poem datasets publicly  (Jiang and Zhou, 2008;Yan et al., 2013;He et al., 2012).


On the one hand, this dataset has a high demand on various reasoning capacities: over 50% of the questions require multiple sentences to answer and also the questions come in assorted categories (what, why, how, whose, which, etc).

various types of memory networks (Sukhbaatar et al., 2015;Kumar et al., 2016) have been shown effective on these tasks, and Lee et al.


(2014b) elaborate on this methodology, showing that the statistical effect is robust to different choices of form-distance and semantic-distance metrics.

behavioral and corpus studies have confirmed the validity of localized form-meaning patterns manifested in limited subsets of the lexicon.


From this, they conclude that the observed systematicity of the lexicon is not a consequence only of small pockets of sound symbolism, but is rather a feature of the mappings from sound to meaning across the lexicon as a whole.

the final lexicon was composed of 4,949 word types.


The correlation between the Word2Vec semantic distances and the orthographic edit distances in our corpus was r = 0.0194, similar to the correlation reported by Monaghan et al.

to previous studies, we study formmeaning systematicity using a kernel regression framework.


Examining a sample of these pairs suggests that they are usually near-synonyms, i.e., it is not that clear whether one term is truely more general than the other or not.

we represent each edge by the concatenation of its components' vectors: dir represent the embedding vectors of the lemma, part-of-speech, dependency label and dependency direction (along the path from x to y), respectively.


All instances in our dataset, both positive and negative, are pairs of terms that are directly related in at least one of the resources.

to the original paper, in which the number of each term's contexts is fixed to N , in this adaptation it was set to the minimum between the number of contexts with LMI score above zero and N .


Deep neural networks provide a powerful mechanism for learning patterns from massive data, achieving new levels of performance on image classification (Krizhevsky et al., 2012), speech recognition , machine translation (Bahdanau et al., 2014), playing strategic board games (Silver et al., 2016), and so forth.

with the recent success of deep neural networks in a vast variety of application domains, it is increasingly desirable to incorporate structured logic knowledge into general types of networks to harness flexibility and reduce uninterpretability.


It is also difficult to encode human intention to guide the models to capture desired patterns, without expensive direct supervision or ad-hoc initialization.

on SST2, MVCNN (Yin and Schutze, 2015) (Row 5) is the only system that shows a slightly better result than ours.


Once again, this speaks to the remarkable conservativeness of the Icelandic language.

if the second regression is a significant improvement over the first, then it can be said that x Granger-causes y, indicating that past values of x significantly predict future values of y over and above any predictive power of y's own past values.


Teaching a system how to respond appropriately in a task-oriented SDS is non-trivial.

figure 1 is an example of a dialogue task which is typically set for users who are being paid to converse with the system.


Also, Ultes and Minker (2015) demonstrated that there is a strong correlation between expert's user satisfaction ratings and dialogue success.

the Obj=Subj system is relatively poor compared to the others.


We extract the following features on a window ±3 tokens centered at the current focus token: word, cluster, character n-gram up to length 3.

it additionally also has transition features of the word, cluster and character n-gram up to length 3 on both endpoints of the transition.


We assume that there is a one-to-one mapping between decision sequences d 1:j−1 and states s j : that is, we essentially assume that a state encodes the entire history of decisions.

consider a locally normalized model with scores ρ(d 1: Then it is easily verified that for all x 1:n , d 1:n .


In our final experiments with full lookahead, the globally normalized model achieves 94.01% accuracy, compared to 93.07% accuracy for a local model with beam search.

(1998) describe global training of neural network models for structured prediction problems.


The proof that P G P L gives a clear illustration of the label bias problem.

we use averaged stochastic gradient descent with momentum, and we tune the learning rate, learning rate schedule, momentum, and early stopping time using a separate held-out corpus for each task.


The solver is used with its default parameters.

the addition of backward flowing information improves accuracy considerably.


The decision to make an attachment, the direction of attachment, and the label of the attachment is made by a classifier.

in our projection method, the topological field features of tokens in the embedded clause reflect the topological structure of the embedded clause.


Most of these techniques involve randomness, which can be undesirable in certain situations (e.g., when experiments need to be exactly reproducible).

one solution is to approximate the matrix with some Y ∈ R l×d where l n. Many matrix approximation techniques have been proposed, such as random projection (Papadimitriou et al., 1998;Vempala, 2005), sampling (Drineas and Kannan, 2003;Rudelson and Vershynin, 2007;Kim and Snyder, 2013;Kim et al., 2015b), and hashing (Weinberger et al., 2009).


We aim to learn sentence representations from as many unlabeled queries as possible via principal component analysis (PCA): specifically, learn a projection matrix for embedding a bag-of-words vector into a lowdimensional dense feature vector.

pCA has been a workhorse in representation learning, e.g., inducing features for face recognition (Turk et al., 1991).


it is prohibitive to even store the data in memory due to its sheer size, let alone apply conventional batch algorithms.


2 We excluded count vectors from the GCCA experiments for computational efficiency since they performed similarly to TF-IDF representations in initial experiments.

we also consider all possible combinations of views obtained by concatenating views before applying PCA, and concatenating PCA-projected views.


BOW: We take the bag-of-words (both count and TF-IDF weighted) representation of all tweets made by users in that view (ego, mention, friend, or follower) following the above pre-processing.

a user is represented by a summation of the one-hot encodings of each neighboring user in their social network.


CNN achieves the best implicit polarity and aspect recognition in T41-test dataset.

total 9 major aspects based on commonsense knowledge, including Dining, Staff, Maintenance, Check In, Cleanliness, Comfort, Amenities, Location and Value for Money, are considered.


Our model favors features which are more likely to appear in target examples, while DA NN seeks domain invariant features.

we  Figure 2: Impact of the noise parameter p on the average accuracy for the 12 Amazon adaptation tasks.


We found that in this setting, we did need slightly more input features.

the latter is significantly more challenging than the former under the shift-reduce paradigm because: 1 shift (I) 6 pro (NP) 2 pro (NP) 7 adj 3 shift (like) 8 pro (S) 4 pro (VP) 9 adj 5 shift (sports) Figure 5: Shift-Promote-Adjoin parsing example.


Re-estimated models with three different values of n ∈ {1, 10, 100} were evaluated under three settings: phrase table re-estimation, interpolation, and BiLM re-estimation.

another relevant line of research relates tuning (weight optimisation), where our work lies between forced decoding (Wuebker et al., 2010) and the bold updating approach of (Liang et al., 2006).


Thus forced alignment is a reestimation technique where translation probabilities are calculated based on their frequency in best-scoring hypotheses instead of the frequencies of all possible phrase pairs in the bitext.

we thank Arianna Bisazza and the anonymous reviewers for their comments.


On top of this basic seq2seq model, we implemented a simple beam search for decoding (Sutskever et al., 2014;Bahdanau et al., 2015).

greedy into strings X is a restaurant offering italian and indian takeaway in the city centre area near X.


A comparison of the two approaches goes in favor of the joint setup: Without the reranker, models generating trees produce less semantic errors and gain higher BLEU/NIST scores.

while some generators keep this division and use a two-step pipeline (walker et al., 2001;Rieser et al., 2010;Dethlefs et al., 2013), others apply a joint model for both tasks (wong and Mooney, 2007;Konstas and Lapata, 2013).


(2) is that it allows for detecting words w j whose embeddings have negative coefficients α w j for a target word w i (we consider p = 1 in the remainder).

across time, several wellknown examples of meaning change in English have been documented.


, w |V | (t) tend to be different across t, by our very postulate, so that there is non-identity of these 'reference points' over time.

our analysis also finds that most core words' meanings decay linearly in time.


We evaluate our phonetic clustering by computing the same pairwise F-score on pairs of vowel tokens.

both phonological and phonetic variability probably contribute to the difficulty of the real task.


We compute this metric on the subset of words for which the segmentation system performs correctly (it is not clear how to count "misses" and "false alarms" for tokens which were mis-segmented).

the model's sparsity preference cannot encourage such moves.


Taking advantage of word semantic representation, our model can get rid of hand-crafted features and thus be easily adapted to multiple languages.

it is a crucial and quite challenging sub-task of event extraction, because the same event might appear in the form of various trigger expressions and an expression might represent different event types in different contexts.


In our model, the loss function is the cross-entropy error of event trigger identification and trigger classification.

we use multiple convolutional filters with different widths to produce local context representation.


Almost all of these cases (98%) are symmetric: each of the conjoined VPs has the same types of indexed arguments.

: Wendy (ran 19 miles) and (walked 9 miles) as well as the spans of each component of the argument-cluster coordinations, including the inner span of each argument: The bracketing in this set follow our proposed ACC bracketing, and we refer to it as ACC OU R .


This highlights how unnatural the original ACC representation is for the parser: it predicts the alternative representation more often than it predicts the one it was trained on.

as we show in the experiments section, the performance of the Berkeley Parser on ACC structures significantly improve when applying our transformations prior to training.


R2: However, you cannot choose yourself.

to make the annotation process feasible given the resources available, we selected a subset of 100 source sentences for the experiment.


Firstly, we conducted experiments on the fixed size of data collection by using 4 settings of our proposed approach.

for the query time of fAST E setting costs only 19.12s while baseline strategy costs 897.5s for obtaining the top-1 value.


In addition, the number of candidates accessed has increases not quickly which indicate our proposed approach scales well.

in addition, the number of candidates accessed has increases not quickly which indicate our proposed approach scales well.


It is also possible that the TransE model is simply not powerful enough to achieve a filtered mean rank lower than 90, no matter the initialization strategy.

table 3 summarizes the experimental results, compared to baseline and state-of-the-art relational models.


A natural next step is to attempt to integrate the two approaches in order to develop a representation that is informed by both unstructured text and a structured knowledge base (Faruqui et al., 2015;Xu et al., 2014;Fried and Duh, 2015;Yang et al., 2015).

this model uses two vectors to represent each entity and relation; one to represent the meaning of the entity, and one to construct a mapping matrix dynamically.


For the WordNet and Freebase datasets, we use entity descriptions which are readily available (see Table 2).

reducing the number of correct entities that are poorly ranked may not significantly affect the hits@10, since this only considers entities near the top of the ranking.


Such clustering is usually performed through manually defined semantic and syntactic features defined over argument instances.

similar to ours, L&G and Arg2vec both encode dependency relations in the embeddings.


(2015) alleviated the large vocabulary issue by proposing an approach that partitions the training corpus and defines a subset of the full target vocabulary for each partition.

(2015), we dump the alignments for each sentence, and replace UNKs with the word-to-word dictionary or the source word.


But our single system is still about 2 points behind of the best phrase-based system (Durrani et al., 2014).

only uses top 30k most frequent words for both V o and V y , and replaces all other words as unknown words (UNK).


The hierarchical extension describes the probabilities over the current word given various contexts consisting of up to n − 1 words.

in this section we give an overview of the data sets we use for the English and Flemish-Dutch experiments.


This difference is explained by the addition of skipgrams of length 3 and 4, and the bigrams and unigrams derived from these skipgrams.

we are interested in the distribution over word sequences induced by the PYP, which has a tractable form, and is sufficient for the purpose of language modelling.


These many-to-one mappings are overlooked when systems use only length or frequency as a proxy for simplicity.

all of the existing methods were evaluated using the implementations as provided in the LEXenstein toolkit.


Many downstream applications that use social media, such as relation extraction (Bunescu and Mooney, 2005) and entity linking Ratinov et al., 2011), rely on first identifying mentions of entities.

it enables propagating the gradients back into the LSTM to adjust the parameters.


The fact that the Lite Speech2Sign platform is intended primarily for use by sign language experts who may only have modest skills in computer science has dictated several key design decisions.

the first was to make the process of developing, uploading, compiling, and deploying web-based speech applications simple to invoke, so that these operations could be performed without detailed understanding of the underlying technology.


Sampling from a class n-gram provided many complementary word forms, not easily generated by the other models.

in this work, an approach comprising of unsupervised morph segmentation and statistical language models was suggested.


(2015) used labeled images from the Web to learn bilingual lexicons based on visual features, with features derived from deep convolutional neural networks (CNNs) leading to the best results (Kiela et al., 2015).

the ID-weighted BLL models reach (near-)optimal BLL results across a variety of language-vision combinations without any fine-tuning.


(2016) explored the task of mind change by focusing on delta awarded comments using their CMV data.

location The position of c in the tree generated by rc.


In order to evaluate the costs and benefits of gathering semantic parse labels, we created the WEBQUESTIONSSP dataset 1 , which contains semantic parses for the questions from WEBQUESTIONS that are answerable using Freebase.

we designed a staged, dialog-like user interface (UI) to improve the labeling efficiency.


We employ dropout on the embedding layer, LSTM layer and the penultimate layer.

although bidirectional RNN has access to both past and future context information, the range of context is limited due to the vanishing gradient problem.


(2014b), ignoring tweets which are less than 7 tokens.

the methods are timeconsuming, requiring language and domain exper-tise.


As a point of comparison, Collobert et al.

in our work, we use recurrent instead of convolutional networks, but our main contribution is observing that we obtain better performance by having POS task supervision at a lower layer.


Multiple-tasks supervision at the outermost layer has a strong tradi-tion in neural net models in vision and elsewhere (Caruana, 1993;Zhang and Zhang, 2014;Yim et al., 2015).

this suggests having task supervision for low-level tasks at the lower bi-RNN layers.


Our model also performs considerably better than the MTL model in Collobert et al.

as shown below and in other recent work using bi-RNNs for sequence tagging, we can still produce competitive tagging accuracies, because of the richness of the representation v i that takes the entire input sequence into account.


This selection process resulted in 12 semantic classes covering between 2 and 27 prepositions each (cf.

note that this baseline is more informed than an entirely random baseline, because the information about the number of gold-standard classes for each preposition is very helpful.


Most interestingly, a qualitative analysis zoomed into the assignment behaviour of the soft clustering approaches, and revealed different attitudes towards predicting ambiguity.

while k-Means/prob defines almost all preposition types (43 out of 48) as ambiguous, NMF only defines 28 out of 46 prepositions as ambiguous.


(2) Based on string edits between f and f , f i is recognized.

in addition, our approach is simpler and easier to adopt than WANG.


We conduct experiments (Section 3) on English-Chinese (EN-ZH) and English-French (EN-FR) TM data.

this property makes it hard to build a lattice from subtraction, as -different from the addition -subtraction does not directly produce a constraint for an input phrase.


As expected, all features correlate with age and income in the same direction.

we quantify the number of hedges (Hyland, 2005) and abstract words 1 used, and the ratio of standalone numbers stated per user as these are indicators of specificity (Pennebaker et al., 2003;Pitler and Nenkova, 2008).


Each data set consists of users and their historical record of tweet content, profile information and trait level features extracted with high precision from their profile information.

to these document-level studies, we adopt a user-centric approach to measuring stylistic differences.


This would open the door to massive studies of the relationships between optimism, pessimism, and a range of online behaviors.

in order to build computational models to differentiate between the optimistic and pessimistic users, we use five different methods from the scikit-learn python library 3 : Naive Bayes (NB), Nearest Neighbor (NN), Decision Tree (DT), Random Forest Classifier (RFC), Gradient Boosting Classifier (GBC) and Stochastic Gradient Descent (SGD).


We tune an adaptation trick by considering predicted class scores as augmented features for target data.

more recently, domain adaptation has been tackled with word embedding techniques or deep learning.


The model does not use textual information, but unlike the previous methods it uses information on unadopted items.

the original CCMC algorithms adopt Singular Value Thresholding (SVT) method, which converges much faster than CCMC-Hazan.


Since the exact segmentation guidelines likely varied across these systems-in particular, we found that Stanford CoreNLP often left off the title in names such as Mr. Smithand we didn't want to focus on these issues, we did not require exact matches of our name segmentation; instead, we consider the entire name as PERSON or LOCATION if any of the tokens were tagged as such (names with both tags were considered OTHER).

to build a training set, we pass through the corpus and each time we come across a common name in a particular document, we build a feature vector corresponding to all the contexts in that document, with the label taken from the clustering.


(2008) also use real and fake reviews and LMs, but in almost exactly the opposite setup: they select features that have high discriminative power in distinguishing real from fake reviews to include in their LMs.

this is equivalent to a situation where the judge can only see the text, not the meta information.


Initially, we would like to establish whether conditioning LMs on demographic information has any effect on detection.

they do not use an adversarial setup, but focus on the use of an item-response model to detect fake-review writers.


Bilingual phrase pairs are consistently the best evidence of semantic equivalence in our experiments.

tuning results confirmed the importance of negative sampling and distance function in our models: in J bi , increasing k consistently helps the bilingual models, whereas the correlation score for monolingual models degrade for k > 10.


In this work, we evaluate the quality of the monolingual representations learned with a variant of the bilingual compositional model of Hermann and Blunsom (2014), when viewing translations in a second language as a semantic annotation as the original language text.

they can be noisy due to automatic sentence alignment and one-to-many mappings, and bag-of-word representations of sentence meaning are likely to be increasingly noisier as segments get longer.


Note that in cases where an event candidate can potentially belong to multiple event types, one can replace the softmax output node with a set of binary output nodes or a sigmoid to allow for multi-label prediction for each event candidate.

compared to cNN, our FBRNN model achieved better recall but the precision is lower.


Existing event extraction work often heavily relies on a rich set of hand-designed features and utilizes existing NLP toolkits and resources (Ji and Grishman, 2008;Patwardhan and Riloff, 2009;Liao and Grishman, 2010;McClosky et al., 2011;Huang and Riloff, 2012;Li et al., 2013a;Li et al., 2013b;Li et al., 2014).

despite this limitation, FBRNN achieved competitive result on Rich ERE with only 0.8% difference from the best reported system in the TAC 2015 competition.


These include IBC (IBC, 2016), ACLED (Raleigh et al., 2010), EDACS (Chojnacki et al., 2012), UCDP (Gleditsch et al., 2002), and GTD (GTD, 2015).

there is thus a tradeoff which must be struck between linguistic richness and the quality of annotation.


BTM directly models the generation of biterms (pairs of words) in the whole corpus.

more specifically, LCTm models each topic as a distribution over the latent concepts, where each latent concept is a localized Gaussian distribution over the word embedding space.


To show that our model can better predict topic proportions of documents containing OOV words than other topic models, we conducted an experiment on a classification task.

inspired by this, we propose a latent concept topic model (LCTM) that infers topics based on documentlevel co-occurrence of references to the same concept.


In this case, if we directly apply the rounding method to certain precisions (e.g., 8 bits), the update of word vectors will always be zero.

keeping embedding vectors for hundreds of thousands of words for repeated use could take its toll both on storing the word vectors on disk and, even more so, on loading them into memory.


Denials and corrections of these viral pieces of information might often come late and without the sufficient effect to prevent the harm that the rumours can produce (Lewandowsky et al., 2012).

research has been done on modeling dynamics of rumour propagation (Lukasik et al., 2015b).


We use both fine-grained and course-grained POS tags, e.g., from the POS tag Npmsi, we generate three tags: Npmsi, N and Np.

paid troll comments: We collected them from the leaked reputation management documents, which included 10,150 paid troll comments: 2,000 in Facebook, and 8,150 in news community forums.


We selected 650 non-troll comments for the paid trolls, and other 578 for the mentioned trolls as follows: for each paid or mentioned troll comment, we selected a non-troll comment at random from the same thread.

the most helpful feature again is metadata, but this time it is less helpful (excluding it yields a drop of 5 points vs. 8 points before).


We say phrase pair x j covers word f ik if x j appears in f i , e i and f ik is contained in the subsequence that equals p j .

the horizontal axis is the number of phrase pairs contained in the table, and the vertical axis is phrase table size.


If we naively apply the algorithm to situations where M is very large, then the algorithm may not work in reasonable running time.

f i and e i are represented as sequences of words f i = (f i1 , .


We run MERT for 10 times to obtain 10 different feature weights.

the method can be impractical if the phrase pairs consist of longer phrases.


Table 2 OOV ACC columns), especially for languages like Arabic, Farsi, Hebrew, Finnish.

note that these results are not strictly comparable as they use the earlier UD v1.1 version.


(3)) penalizes more heavily reconstruction error of frequent cooccurrences, improving on PPMI-SVD's L 2 loss, which weights all reconstruction errors equally.

the LexVec loss function has two terms We minimize eqs.


The aim was to compare the accuracy of the bilingual models with respect to the monolingual ones, when there is no variation on the parsing algorithm between them.

this clearly   surpasses the amount of significant gains to be expected by chance, and applying the Benjamini-Hochberg procedure (Benjamini and Hochberg, 1995) to correct for multiple comparisons with a maximum false discovery rate of 20% yields 8 significant improvements in LAS and UAS.


More strikingly, there are many cases where bilingual parsers outperform monolingual ones, even in this evaluation on purely monolingual datasets.

other languages that tend to get improvements in this way are French and Portuguese.


We find a positive effect of within-topic position for leaders, and a reliably negative effect for followers (except SL of BNC follower), which confirms the observation of convergence trend (See Table 1).

it postulates that speakers will strive to keep information density approximately constant.


For example, Adel and Schütze (2014) integrated coreference chains extracted from large corpora into a skip-gram model to create word embeddings that identified antonyms.

as gold standard resource, we used the English dataset described in (Roth and Schulte im Walde, 2014), containing 600 adjective pairs (300 antonymous pairs and 300 synonymous pairs), 700 noun pairs (350 antonymous pairs and 350 synonymous pairs) and 800 verb pairs (400 antonymous pairs and 400 synonymous pairs).


The intuition behind the lexical contrast information in our new weight SA is as follows.

due to interchangeable substitution, antonyms and synonyms often occur in similar contexts, which makes it challenging to automatically distinguish between them.


Most relevant to this work is the comparison between Open-IE and SRL.

in cases of multiple answers referring to the same entity, annotators are asked to provide the most specific answer, otherwise (if the answers refer to different entities), the annotators are asked to list all of the answers.


To conclude this section, the entire comparison measurements are summarized in Table 1.

a restrictive modifier is an integral part of the meaning of the containing NP, and hence should not be removed, as in "She wore the necklace that her mother gave her".


(2013c) The linear transformation model is constructed as follows.

this makes our approach highly scalable and portable.


However, it should be also stressed that our evaluation was very strict: a retrieved collocate candidate was considered as correct only if it formed a collocation with the base, and if it belonged to the target semantic gloss.

for the outcome of each experiment, we computed both precision (p) as the ratio of retrieved collocates that match the targeted glosses to the overall number of obtained collocates for each base, and Mean Reciprocal Rank (MRR), which rewards the position of the first correct result in a ranked list of outcomes: where Q is a sample of experiment runs and rank i refers to the rank position of the first relevant outcome for the ith run.


2/3 can be combined to train an orthogonal transformation matrix.

training this model requires about 500GB of RAM.


For instance, while it has been noted that modeling verb similarity is indeed a difficult problem in English as evidenced by lower correlation scores on SimLex (see Fig.

tion of word2vec which is capable of learning from arbitrary (word, context) pairs.


We generated claims for 67 topics, extracted from debatabase motions (http://idebate.org) for which we have previously annotated relevant Wikipedia articles (for the benefit of the n-TLs construction; see Section 2.1).

it is interesting and challenging to synthesize arguments in an open domain.


By performing inference over the vector representations of the words, their model is encouraged to group words that are semantically similar, leading to more coherent topics.

the variational distributions for z, π, µ have the following parametric forms, where Dir denotes the Dirichlet distribution and ϕ, θ, ψ and λ are the parameters we need to optimize the ELBO.


Specifically, we investigate the topic interpretability by examining the "coherence" of a topic generated by topic modeling algorithms, i.e., how close topic words are within a topic.

assuming that random words represent random topics, we can name the above method "betweentopic" measure.


The insight of this measure is that a good topic is one whose top few words are distant, or highly separate, from randomly-selected words.

if the Turkers are confused about which second word to choose, then their answers will be scattered amongst the remaining five words.


We use two settings for the classification experiments using libSVM (Chang and Lin, 2011) for both argument component and relation identification.

table 6: Correlation of LR (10 fold CV) with predicted results.


we made no unrealistic assumptions as to the availability of parallel texts and preprocessing tools for the target languages.


This group contains features relating to the fixations of the words in near proximity of the token.

token-level aggregation leads to more reliable feature values.


Then we sample a relatively small number (e.g.

seqCor and phrase-F 1 -multiplied, despite identifying artificial datasets, cannot discriminate between real systems: seqCor fails with the top half systems, phrase-F 1 -multiplied is bad at finding differences in the bottom half of the list.


In this case we repeat the process with a larger (e.g.

the widely used weighted average of F 1scores for the two classes is not suitable as it will be dominated by F 1 -OK due to labels imbalance.


The most likely reason is that NLP research has not directly involved human subjects.

even if we address all of the above concerns and do not intend any harm in our experiments, they can still have unintended consequences that negatively affect people's lives (Jonas, 1984).


Partial cognacy, however, is a frequently recurring phenomenon, especially in language families with productive derivational morphology.

with a total of three different clustering algorithms (UPGMA, Markov Clustering, and Infomap), we thus carried out twelve tests on complete cognacy (three for each of our four approaches), and six additional tests on pure partial cognate detection, in which we compared the suitability of SCA and LexStat as string similarity measures.


Park and Im (2016) proposed a novel approach for multimodal representation learning which uses adversarial back-propagation concept.

the detailed statistics about all the datasets are listed in table 1.


Adversarial networks have recently surfaced and are first used for generative model (Goodfellow et al., 2014).

more concretely, compared with SP-mTL, ASP-mTL achieves 4.1% average improvement surpassing SP-mTL with 1.0%, which indicates the importance of adversarial learning.


(2016c) first utilizes different LSTM layers to construct multi-task learning framwork for text classification.

the task-invariant features can appear both in shared space and private space.


Figure 2a illustrates the fully-shared model.

most existing work on multi-task learning (Liu et al., 2016c,b) Figure 1: Two sharing schemes for task A and task B.


Graph-based parsers solve a structured prediction problem in which the goal is learning a scoring function over dependency trees such that correct trees are scored above all others.

hence, the independence assumptions of the BL tagger apparently lead to more 'local' errors such as exact argument span identification (cf.


We believe that this is due to a mixture of the following: (1) 'capacity', i.e., model complexity, of the parsersthat is, risk of overfitting; and (2) few, but very long sequences on essay level-that is, little training data (trees), paired with a huge search space on each train/test instance, namely, the number of possible trees on n tokens.

there are 5338 relations, most of which are supporting relations (>90%).


Moreover, from a machine learning perspective, pipeline approaches are problematic because they solve subtasks independently and thus lead to error propagation rather than exploiting interrelationships between variables.

dependency Parsing methods can be classified into graph-based and transition-based approaches (Kiperwasser and Goldberg, 2016).


"people" is the domain, "person" is the type and "parents" is the property.

the model always puts a reasonable amount of probability on a program with high reward during training.


We evaluate performance using the offical evaluation script for WEBQUESTIONSSP.

training neural networks for semantic parsing (Zelle and Mooney, 1996;Zettlemoyer and Collins, 2005;Liang et al., 2011) or program induction, where language is mapped to a sym-⇤ Work done while the author was interning at Google † Work done while the author was visiting Google x: Largest city in the US ⇒ y: NYC  Figure 1: The main challenges of training a semantic parser from weak supervision: (a) compositionality: we use variables (v0, v1, v2) to store execution results of intermediate generated programs.


(2) MNRE(CNN)-R has similar performance as compared to MNRE(CNN) when the recall is low.

there are 176 relations including a special relation NA indicating there is no relation between entities.


Results on WEBQUESTIONS are summarized in Table 3.

for further comparison, we built a simple baseline which identifies predicates based on the output of the Stanford POStagger  following the ordering VBD VBN VB VBP VBZ MD.


In the absence of anaphora and composite binary predicates, conversion algorithms exist between FunQL and λ-DCS.

the non-terminal is popped as well, after which a composite term representing the entire subtree, e.g., border(texas), is pushed back to the stack.


The first feature is the likelihood score of a grounded representation aggregating all intermediate representations.

the questions were created by randomly removing an entity, thus producing sentence-denotation pairs (Reddy et al., 2014).


is reportedly trying to get impregnated by djimon now .

since the key idea of our model is to capture salient meaning for the task at hand rather than strictly obey syntax, we would not expect the  Table 8: Evaluation of predicates induced by SCANNER against EASYCCG.


As ∇ uc Q equals we see that (7) follows.

recall that Skip-Gram chooses U and V to maximize where This objective can be rewritten using the pairwise cooccurence statistics as where H(•) denotes the entropy of a distribution.