Overall, the addition of these hand-crafted features to our best systems do not provide improvements as high as one might have hoped.

we investigate three well-known word embeddings, namely Collobert and Weston (Collobert and Weston, 2008), hierarchical log-bilinear model (Mnih and Hinton, 2007) and Hellinger Principal Component Analysis (Lebret and Collobert, 2014), in addition to Brown clusterbased and standard one-hot representations.


In the spirit of this approach, we add an additional component to the network architecture, capturing the bag-of-words for the entire document.

as seen in the comparison between lines B1 and D1, discourse depth weighting offers substantial improvements over the bag-of-words approach for lexiconbased sentiment analysis, with raw improvements of 4−5%.


The goal of feature representation learning is to obtain dense features that capture feature correlations between the source and target domains.

our experiments test the utility of the two domain adaptation methods, using the Penn Discourse Treebank (Prasad et al., 2008) and some extra-training data collected from a external resource.


In this work, we focus on the following differences between spoken dialogues and written texts as sources for Wikification.

while the intersection strategy contributed to produce more precise outputs than the others even including the case with manual filtering, the other proposed approach with union achieved more gain in recall with slightly better F-measure than the former one.


Firstly, at least two speakers are engaged in a dialogue session, while the texts in newswire or microblogs are mostly written by a single author.

if the value for in-dialogue reference is positive, the searching is restricted to the set of concepts linked with the previous mentions in the same session.


According to the analyzed results, the criteria in selecting concept candidates is determined, and then a ranking is performed on the filtered candidates to identify the concept that is the most relevant to the mention.

we could expect to improve the understanding capabilities of applications based on Wikification and broaden the coverage of the contents in knowledge-bases, if Wikification is successfully performed also for human-human spoken conversations.


all of these demonstrate that our model is suitable for implicit DRR.


Maybe RAE needs more labeled training data for better results.

in the future, we will verify our model on other languages, for example, Chinese and Arabic.


The tokens are labeled with the IOB scheme, i.e., every token is annotated as inside an AU (I), beginning of an AU (B) or outside an AU (O).

since argumentation theory offers a wide range of models, we performed a pre-study on a held-out development set to develop the annotation scheme.


The system shown in Figure 2 is part of a smart house (Motooka et al., 2010) that is used to capture multimodal information.

a word is considered to express a category when the MI between the word and category is large.


We focus on person identities, because person names might be mentioned when describing facts.

modeling local coherence alone is not adequate to distinguish discourse elements in persuasive essays.


For each sentence t containing pronouns, we introduce a constraint indicator variable c protop,t,t−1 , and add the ILP constraints: 4 A word that evokes a semantic frame (event) in a sentence.

we also suggest an approach to compute back pointers in coreference resolution globally.


Let U be the set of binary indicator variables corresponding to the values assigned to the back pointers.

but while CoNLL 2011 training data (OntoNotes) includes a few different source domains (newswire, weblogs, etc.


The right tree is the actual edit tree we use in our model, the left tree visualizes what each node corresponds to.

the alignment of umgeschaut -umschauen is: Each alignment pair constitutes a feature in our model.


many deterministic WFSAs will not exhibit this behaviour (eg.


Here we describe a single LSTM layer and refer to Graves et al.

during testing, the learned weights are used in a forward step to compute a prediction over the labels.


Every character in our corpus has a label corresponding to 0, 1, or 2 diacritics, in the case of the gemination marker combining with another diacritic.

lSTM AiEotabara Almudiyru AlEAm~u l " Aln~ahAri " juborAn t w yoniy Ain Alt~a$okiylAti AlqaDA}iy~apa jA'at litamoyiyEa milaf~i maHaT~api Al " Aim .


Missing or substituted letters are the most common mistakes (10.81%), and further 6.42% of the errors is due to punctuation errors in the original.

even for the biggest 437.6Mword training corpus, incorporating the morphological analyzer with ranking yielded a relative error rate reduction of 39.74%, reducing the word error rate from 1.56% to 0.94%.


For each dimension, the pooling layer chooses the maximum value from the nodes that are pooled to it.

(2) How can we determine weights, W i , for dependency trees, where nodes may have different numbers of children?


Word embeddings are 300 dimensional, pretrained ourselves using word2vec (Mikolov et al., 2013)  To train our model, we compute gradient by back-propagation and apply stochastic gradient descent with mini-batch 200.

rNNs bury illuminating information under a complicated neural architecture.


All the datasets except QC, the best number of groups is equal to or smaller than our calculated upper bound.

a recurrent neural network (RNN) (Elman, 1990) is able to process a sequence of arbitrary length by recursively applying a transition function to its internal hidden state vector h t of the input sequence.


Fast-to-Slow (F2S) Strategy Intuitively, when we accumulate the short-term memory to a certain degree, we store some valuable information from the short-term memory into the long-term memory.

• NBOW The NBOW sums the word vectors and applies a non-linearity followed by a softmax classification layer.


The first set of experiments uses only the Interviews video clips (62 in total), and the results are shown in the left column of Table 7.

in the MUMiN scheme, facial displays consist of several different facial expressions associated with eyebrows, eyes, gaze, and mouth.


This suggests that truth-tellers show similar word usage when interviewed on a real scenario.

the final set of transcriptions contain 7835 words, with an average of 66 words per transcript.


And in order to filter out less useful features for the center vector of the positive class, we performed feature selection in the document space using the classic information gain method (Yang and Pedersen, 1997) to empirically choose the most effective 100 features for the positive class.

cBS-L is very different because we perform supervised classification.


Next, s examples are sampled from the unlabeled data into S, and C 1 , C 2 are used to obtain predictions for these s examples.

wan (2009) used co-training for cross-lingual sentiment classification of product reviews, where English and Chinese features were considered as two independent views of the data.


Moreover, in Blum and Mitchell (1998), the co-training algorithm moves p highest confidence positive examples and n highest confidence negative examples from S to L, where p : n represents the class distribution in the original labeled training set (i.e., if there are 10 positive examples and 90 negative examples in the labeled set L, then p = 1 positive and n = 9 negative examples are moved to the labeled set at each iteration of co-training).

moreover, a great range of classifiers were used for this task, including: Naïve Bayes (Lewis and Ringuette, 1994), Knearest neighbors (Yang, 1999) and Support Vector machines (Joachims, 1998).


For example, words like learning, multi-agent or interface are more important in the content view.

we experimented with both TF and TF-IDF scores, using different classifiers (Support Vector Machine, Naïve Bayes Multinomial, and simple Naïve Bayes classifiers), but Naive Bayes Multinomial with TF performed best.


Many search engines follow the intuition that the anchor text pointing to a page is a good descriptor of its content, and thus anchor text terms are used as additional index terms for a target webpage.

for the content of a paper, we use its title and abstract as it is commonly used in the literature (Lu and Getoor, 2003); for the citation contexts, we use both the cited and citing contexts, as described in the previous section.


One needs a larger cultural context to figure out the subtle humorous meaning expressed in these two sentences.

the wordplay by itself is not enough to trigger the comic effect, thus not equivalent to the humor anchors for a joke.


Our data is drawn from Part 1 of the Fisher English speech corpus (Cieri et al., 2004c), which contains audio recordings (Cieri et al., 2004a) and manual transcriptions (Cieri et al., 2004b) of telephone conversations.

the learned representations we consider explain which words appear in a document d via a latent, lowerdimensional representation θ (d) .


Our results and those of prior work show that using whole-conversation documents instead of individual-side documents make the topic ID task easier.

we apply l 2 normalization rather than tf-idf weighting to the speech data because it is dense and tf-idf is thus inappropriate.


Many model components of competitive statistical machine translation (SMT) systems are based on rather simplistic definitions with little linguistic grounding, which includes the definitions of phrase pairs, lexicalized reordering, and n-gram language models.

as discussed in Sections 1 and 2, we would like to test how much a PBSMT can benefit from an additional syntax-based LM.


The LRscores of the baseline are compared to the best performing BiSLM system with respect to BLEU, for each of the language pair.

dependency relations are preserved, as they are defined based on thematic relations between words.


(2012), who also propose a language model based on a suffix tree which scales well with m but poorly with the corpus size (requiring memory of about 20× the training corpus).

ehsan Shareghi and Gholamreza Haffari are grateful to National ICT Australia (NICTA) for generous funding, as part of collaborative machine learning research projects.


The number of iterations of the L-BFGS algorithm is set to 500.

for short) and testing datasets (Tes.1, Tes.2 for short).


There are three main tasks for relational similarity.

(graduate)" can be "c n ê (Obama), o Ú (President), u (from), M Ã (Harvard)", while a dependency context of the same word can be "1991c/ADV, o Ú/SBV, u/CMP, { AE /POB u" 1 , where "ADV, SBV, CMP, POB" indicate adverbial modifier, subject, complement and prepositional object, respectively.


We study an alternative way of making use of syntactic dependencies, by using them to prune the vocabulary-sized search space of analogy detection.

for example, the comparative pairs "good:better" and "rough:rougher" are analogous, and the past tense inflections "see:saw" and "return:returned" are analogous.


This problem is very much re-  lated to the object recognition task in computer vision, but instead of using just visual data and labels, these cross-modal models also utilize textual information (Socher et al., 2014;Frome et al., 2013).

nearest neighbors for the linguistic model tend to be of a more abstract nature: where we find mouth and throat as auditory neighbors for language, the linguistic model gives us concepts like word and dictionary; while auditory gossip sounds like maids and is something you might do in the corridor, it is linguistically associated with more abstract concepts like news and newspaper.


This did not result in significant performance improvements.

* † ‡ differences statistically significant.


It is a function of the main verb and a select group of complements (it may differ per verb which ones are important).

habitual sentences tolerate exceptions: (2a) is true even if Bill does not drink a coffee after every lunch.


The input to the model are instances of arguments (e.g., window, the burglar in sentence (1)) and the output is a grouping of these instances into clusters such that each cluster contains arguments corresponding to a specific semantic role and each role corresponds to exactly one cluster.

dimensionality reduction was performed by the t-SNE library.


Slightly better results with the C&C parser is explained by employing the parser in the learning phase.

a tableau method (Beth, 1955) is a popular proof procedure and nowadays many formal logics have their own version of it (D'Agostino et al., 1999).


Traditional machine learning algorithms often train a classifier utilizing the labeled data for CSC.

considering the data in practice, the sentiment orientation y has only two forms, positive or negative.


We show the effectiveness of introducing emotional words to solve the CSC problem.

we rebuild the data with the help of emotional lexicon cutting out the nonemotional words.


However, we also wish to investigate how well individual annotators perform.

there is no discernible difference between the two annotator groups in terms of error margins.


Trained annotators are more correlated with one another (ρ = 0.90, α = 0.67, κ = 0.34) than the crowdsource annotators (ρ = 0.75, α = 0.52, κ = 0.09).

the model achieves better results using annotator rather than author ratings, even when controlling for rating variance as a confounding factor.


M AE M values are all similar across languages, again confirming what has been observed on agreement.

we describe Trip-MAML and report experiments aimed at defining a first baseline.


Given a training corpus with hand-annotated sentiment polarity labels, following Kim (2014), we train a deep convolutional neural network (CNN) on it.

in this way, we used the last output layer of the CNN only for training, but for actual decision-making, we replaced it with much more sophisticated classifiers, namely, with SVM or MKL.


With this tweak, the best result was obtained.

the parameter selection for decision level fusion produced suboptimal results.


We provide a case study of automatically categorizing annoying behaviors using #petpeeve Tweets.

the best results for each section are highlighted in bold.


In the qualitative analysis, we study the language use in these tweets, with a special focus on the fine-grained categories and the geographic variation of the language.

we briefly review a supervised approach of using sparse mixed-effects topic model to visualize the topical words to analyze this behavior data.


Batches of 100 are the maximum for M-CT and T-Lex, but we go up to 1000 with A-CT. As expected, the performance of the classifiers trained on individual tweets is poor (see Figure 1).

as all linguistic stegosystems rely on automatic paraphrasing, and generally require a source of data, it is conservative to assume that the attacker has access to this data source.


Various studies within the emerging field of Computational Social Science (Lazer et al., 2009) have found that Twitter tends to be a good reflection of society (Lamb et al., 2013;.

they were not annotated, because it was unclear whether participants used these 'default' motivations on purpose.


UMass has no initialization and performs the fastest when the number of documents is kept low.

they did not perform a quantitative evaluation of their approach.


Let R be a set of rumours, each of which consists of tweets discussing it, is the complete set of tweets from all rumours.

we formulate the rumour classifica-  tion problem as transfer learning, where unseen rumours (or rumours with few initial tweets observed) are classified using already known rumours -a much harder and more practical setting.


In experiments on 16 statistical properties about countries from Freebase we show that our approach identifies simple statistical claims about properties with 60% precision, while it is able to verify these claims without requiring any explicit supervision for either tasks.

best performances were achieved for relations such as population which have a wide range of values that is well separated from the rest, while percentage rates were usually harder for the opposite reason.


We then proceed to verify the value claimed in text for the property of this entity against the value known in a knowledge base such as Freebase and return a score reflecting the accuracy of the claim (absolute percentage error in the example).

it is important to distinguish between them.


We also studied the impact of the size of the training data in the generalization of our system, by training the system in {100, 200, .

since rules (or tree mappings) would become hidden variables, this generalized rule extraction may require faster methods to enumerate plausible rules.


The coordination classification and candidate extraction components are described in Section 5.

these three features are denoted as F eats(i, j, l, m).


Coordination is a common syntactic phenomena, appearing in 38.8% of the sentences in the Penn Treebank (PTB) (Marcus et al., 1993), and in 60.71% of the sentences in the Genia Treebank (Ohta et al., 2002).

for example, in: "The Jon Bon Jovi Soul foundation [was founded in 2006] and 1 [exists to combat issues that force (families) and 2 (individuals) into economic despair]."


This architecture is similar to Siamese Networks, which are used for learning similarity functions in vision tasks (Chopra et al., 2005).

in our symmetry component we opted to not rely on predicted tree structures, and instead use the simpler option of representing each conjunct by its sequence of POS tags.


In other words, LEN is a stronger constraint than DEP in that the structures penalized by DEP are only a subset of structures penalized by LEN.

this bracket tree is not binarized; for example, we extract (X a b (X c d)) from the tree a b c d. table 2 shows the results, and we can see that DEP always performs the best, showing that DEP leads to the models that find better constituent structures.


Our main empirical finding is that though two biases, avoiding center-embedding and favoring shorter dependencies, are conceptually similar (both favor simpler grammars), often they capture different aspects of syntax, leading to different grammars.

we take an EM-based simplest approach, and multiply the original model by a constraint factor f (z, x) ∈ [0, 1] to obtain a new model: and then optimize θ based on p (z, x|θ).


We treat ξ as a hyperparameter in our experiments, and in practice, we find that this relaxed constraint leads to higher performance.

there are more than one (binary) CFG parses corresponding to a given dependency tree.


Whether this is due to differences in labeling or to intrinsic properties of these events, the result is that they are much harder to label because tense and aspect are less available than for events realized as finite verbs.

we show that the temporal status of these events is difficult to classify because local tense and aspect cues are often lacking, time expressions are insufficient, and the linguistic contexts have rich semantic compositionality.


In total, 2907 English and 2807 Spanish event phrases exist in the relevant sentences and were annotated.

row 1 of Tables 6 and 7 shows TIPSem results.


These verbs project the events in the lower clause into the future.

previous temporally annotated corpora focus on a wide variety of events.


(2013), which uses 7 tags to handle entities that can be discontiguous and overlapping.

noticing that many named entities contain other named entities inside them, Finkel and Manning (2009) proposed a model that is capable of extracting nested named entities by representing the sentence as a constituency parse tree, with named entities as phrases.


When humans skip a word, they are able to identify it using parafoveal preview (Rayner, 2009).

in Table 4, we give the simulated fixation probabilities and the human fixation probabilities estimated from the Dundee corpus for the tags of the Universal PoS tagset (Petrov et al., 2012), using the PoS annotation of Barrett et al.


We evaluate the rationales using the mean average precision (MAP) of retrieval.

the SVM baseline performs poorly, achieving around 30% accuracy.


Figure 2 shows the performance of our joint dependent model when trained to predict the sentiment of all aspects.

beyond learning to understand or further constrain the network to be directly interpretable, one can estimate interpretable proxies that approximate the network.


That is, the joint probability p(z|x) factors according to The component distributions p(z t |x) can be modeled using a shared bi-directional recurrent neural network.

this is because it is easier to assess the rationale once produced rather than directly guide how it is obtained.


We set the cardinality regularization λ 1 between values {2e − 4, 3e − 4, 4e − 4} so the extracted rationale texts are neither too long nor too short.

the rationale for a given sequence x can be equivalently defined in terms of binary variables where each z t ∈ 0, 1 indicates whether word x t is selected or not.


nothing out of the ordinary here , but a good brew s9ll .

target rationales are never provided during training; the intermediate step of rationale generation is guided only by the two desiderata discussed above.


At each step t, LSTM emits output h t and three key vectors k t , e t and a t simultaneously.

to get an intuitive understanding of what is happening when we use shared memory to predict the class of text, we design an experiment to compare and analyze the difference between our models and vanilla LStM, thereby demonstrating the effectiveness of our proposed architectures.


Adding in the lookup baseline (LU) to the bestperforming causal model does not improve performance (compare lines 10 and 12), suggesting that the bidirectional causal embeddings subsume the contribution of the LU model.

levy and Goldberg (2014) have modified the algorithm of Mikolov et al.


Additionally, embeddings have been customized to distinguish functional similarity from relatedness (Levy and Goldberg, 2014;Kiela et al., 2015).

causal convolutional Neural Network Model (ccNN): Each of the previous models have at their root a bag-of-words representation, which is a simplification of the causality task.


For example, the predicate People.Person.Profession expects its subject to be of Person type and its object to be of Profession type.

no explicit type inference is exploited.


However, the information required to execute this logical form and answer the question must be extracted from the diagram.

finally, we have released our data and code to facilitate further research.


The cycle count features help the model learn that food webs are typically, but not always, acyclic and the arrow reuse features aim to prevent the model from predicting two different EATS instances on the basis of a single arrow.

eATS should be a learned function of the environment.


• The embedding layer should also be different depending on the role of context word.

a hit is achieved when journalist check spelling has a higher score than mechanic check spelling, setting the query to the model as journalist check and mechanic check and finding a score for spelling in that context.


Given the model and a sentence, we can perform exact inference using dynamic programming and get the optimal sense sequence of the sentence.

most of the previous work has the problems of learning sense embeddings based on imperfect word embeddings as well as ignoring the dependency between sense choices of neighboring words.


This could be a limitation of our strategy, since the number of connectives in the PDTB is fixed.

although there is often a lack of consensus among linguists as to the exact definition of discourse connectives, they nevertheless form a closed class.


This case shows the effects of multiple hops.

the results of sentence "great food but the service was dreadful!"


Past research has proposed many techniques to extract opinion targets (we will just call them targets hereafter for simplicity) and also to classify sentiment polarities on the targets.

modifier Extraction: These modifiers are identified from the corpus d using three syntactic rules.


Later Yang and Eisenstein (2014) further showed that their proposed structured dropout noise strategy can dramatically improve the efficiency without sacrificing the accuracy.

figure 1 shows an example sentence x, its modified version g(x) and the labels u and v for x.


(2007), we employ weighted log-likelihood ratio (WLLR) to select the most positive and negative words in both domains as pivots.

we randomly select 100 sentences from the development set and mix them with the training set.


As a general tendency, the performance of all approaches worsens as sentence length increases.

we believe this choice provides an optimal trade-off between fully manual error analysis (Farrús Cabeceran et al., 2010;Popović et al., 2013;Daems et al., 2014;Neubig et al., 2015), which is very costly and complex, and fully automatic error analysis (Popović and Ney, 2011;Irvine et al., 2013), which is noisy and biased towards one or few arbitrary reference translations.


In doing so, we begin by studying different translation strategies available in the multi-way, multilingual model in Sec.

to handle those N source and M target languages, the model consists of N encoders and M decoders.


This allows for more flexible forms of anchors with weak supervision.


for each state h ∈ H, let its set of anchor words be A(h) is the set of unambiguous words that always take the label h. This can be estimated from the labeled dataset D L by collecting the most frequent unambiguous words for each label.


To test the generality of different approaches, we thus conduct experiments on a newly released data set, DRAW-1K 8 (Upadhyay and Chang, 2016), which covers more than 200 templates and contains 1,000 algebra word problems.


2 The dataset has not been made public at the time of publication.

compared to the results in Table 3, note that when using noisy implicit signals, it cannot obtain the same level of results, even though we had more training problems (2,000 mined problems instead of 514 problems).


These practical issues make it difficult to use the crawled equations for explicit supervision directly.

the reason is that in both Implicit and Pseudo settings, the algorithm needs to select one from many derivations that match the labeled solutions, and use the selected derivation to update the model.


Figure 1 illustrates the main differences between implicit and explicit supervision signals.

the implicit supervision signals may be incomplete and noisy, and using the solutions alone to guide the training procedure might not be sufficient.


The second set of experiments evaluate performance on abstractive summarisation (Table 2).

in this section, we introduce the auto-encoding sentence compression model ( Figure 1) 1 in the framework of variational auto-encoders.


In Figure 3, we present the validation perplexity to compare the abilities of the three models to learn the compression languages.

for the parameters φ in the compression model, we firstly define the learning signal, Then, we update the parameters φ by: this gradient estimator has a big variance because the learning signal l(s, c (m) ) relies on the samples from q φ (c|s).


The most common family of variational autoencoders relies on the reparameterisation trick, which is not applicable for our discrete latent language model.

the ASC model is granted the ability to explore unlimited unlabelled data resources.


f gru t is the probability of a non-agenda-item token, f new t is the probability of an new item reference token, and f used t is the probability of a used item reference.

nn baselines are not successful in generating a goal-oriented text that follows the given agenda: compared to Checklist+ (83.4%), they use substantially less % of the given ingredients (40% -58.2%) while also introducing extra ingredients not provided.


Also transformations of verbal features are likely to intercept simplification operations in a language like Italian with a rich inflectional paradigm.

in ATS, these techniques are typically applied to already existing parallel corpora; in this case the task of aligning the original sentence to its corresponding simple version can be tackled by applying similarity metrics that consider the TF/IDF score of the words in the sentence (Barzilay and Elhadad, 2003;Nelken and Shieber, 2006;Coster and Kauchak, 2011) or methods taking into account also the order in which information is presented (Bott and Saggion, 2011).


The experimental result demonstrates the effectiveness of multi-view model and proves that word sequence view and utterance sequence view can bring complementary information for each other.

the utterance-level discourse information and dependencies have been left out in these studies since they view the context and response as word sequences.


An important aspect of natural language understanding involves recognizing and categorizing events and the relations among them.

we add either of the NER labels if we detect such a named entity.


Formally, we define S pair (a = 4 See supplementary materials for the full list of examples.

note that there may be missing event arguments (nIL).


For event co-reference, we compare all systems using standard F1 metrics: MUC (Vilain et al., 1995), B 3 (Bagga and Baldwin, 1998), Entity-based CEAF (CEAF e ) (Luo, 2005) and BLANC (Recasens and Hovy, 2011).

we convert each word to a vector of 2 4 + 2 6 + 2 10 = 1104 dimensions.


However, the term embedding learning method proposed in (Yu et al., 2015) only learns through the pairwise relations of terms without considering the contextual information between them.

as also shown in Table 3, our approach has achieved high accuracy for both general and specific domain datasets, while in Yu's method, there is a huge difference in accuracy between these domain datasets.


(2015) proposed a super Taxonomies which serve as the backbone of structured knowledge are useful for many NLP applications such as question answering (Harabagiu et al., 2003) and document clustering (Fodeh et al., 2011).

the experimental results in table  5 show that our approach achieves the best performance compared to other state-of-the-art methods for all the datasets.


Another such example is shown in the Plant domain, where our ap-proach recognizes 'lily' as a hyponym of 'flowering plant', but WordNet places them in different subtrees incorrectly 3 .

the input is a 3d-dimensional vector where d is the dimension of term embeddings, x and y are two terms used to check whether x is a hypernym of y or not, and v x , v y are the term embeddings of x and y respectively.


For the general domain datasets, we have conducted two experiments to evaluate the performance of our proposed approach.

many references 1, 2 consider 'wild sheep' as a species of 'sheep'.


The experimental results are given in Table 4.

the offset feature is not so important for the Word2Vec model.


2 Data and source code available from the following link: www.taln.upf.edu/taxoembed.

the performance of tAXOEMBED is evaluated by conducting several experiments, both automatic and manual.


Alternatively, recent advances in OIE based on disambiguation and deeper semantic analysis (Nakashole et al., 2012;Grycner and Weikum, 2014;Delli Bovi et al., 2015b) have shown their potential to construct taxonomized disambiguated resources both at node and at relation level.

the hypernym detection phase has gathered much interest not only for taxonomy learning but also for lexical semantics.


This expansion step results in much larger sets W * and K * , where |W * | = 18,291,330 and |K * | = 15,362,268.

our model does not perform particularly well on media and physics.


This leads to an enormous number of word combinations.

it is theoretically able to capture the word polysemy.


Representation quality in semantics is usually evaluated using intrinsic datasets of human similarity and relatedness judgments.

for each of the network architectures, we see a marked improvement of multi-modal representations over uni-modal linguistic representations.


In Section 4 we detail the conditions on SETGAMMA under which SWVP converges, and in Section 5 we describe various SETGAMMA implementations.

the proof for #mistakes-SWVP is given below.


As noted in the introduction, this binary representation is motivated by feature-based theories of lexical semantics (Katz and Fodor, 1963).

sUMEMBED We compare against the baseline described in § 3.3, which can be viewed as a reimplementation of the compositional model of Botha and Blunsom (2014).


Note that the main contribution of our method is not to perform better than other methods in terms of correlation and nDCG to the gold scores, but to result in highly interpretable and robust estimates discussed later.


To maintain quality, evaluation tasks implemented using crowdsourcing should be simple.

existing methods, however, only yield system-level scores.


In contrast, DCU loses 34.5% and win 34.8% of the comparisons and discriminate the other systems successfully.

iRT models are highly interpretable and are supported by theoretical and empirical studies.


We used nDCG because we are often interested in ranks and scores, especially in MT competitions such as the WMT translation task.

many previous studies focused on pairwise comparisons instead of absolute evaluations.


Our second experiment is carried out on a synthetic dataset where each new source sentence is a concatenation of neighboring source sentences in the original test sets.

experiments on both Chinese-english and english-German translation tasks show that VNMT achieves significant improvements over several strong baselines.


h e , into the computation of s j , which is denoted by the bold dashed red arrow in Figure 2 (c).

there are three essential components in VNMT (The detailed architecture is illustrated in Figure 2): • A variational neural encoder transforms source/target sentence into distributed representations, which is the same as the encoder of NMT ) (see section 3.1).


The development and test data have been manually aligned at the word level, annotating alignments between source and target words in the corpus as either "sure" (S) or "possible" (P ) alignments, as described in (Och and Ney, 2003).

in the context of the original task, we have that English is the target language while French is the source.


These models share parameters but generate words using different recipes, as discussed above.

for a sentence pair (e, f ) with |e| = l and |f | = m, in HMM decoding we aim to find (a 1 .


By further introducing the relational knowledge among word-senses, the training for rare wordsenses will be linked to the training of their related word-senses.

such questions are a bit more difficult than the Analogy-I questions since the analogical relation cannot be observed directly from the questions, but need to be searched for in the word pair combinations from the candidate answers.


The model maintains two sets of vectors stored in a hidden state tape used to interact with the environment (e.g., computing attention), and a memory tape used to represent what is actually stored in memory.

a dropout rate of 0.5 was applied to the neural network classifier.


In practice, however, sequence-level networks are met with at least three challenges.

the model induces undirected relations among tokens as an intermediate step of learning representations.


GRAPHQUESTIONS contains 500 graph queries, 2,460 sentence-level paraphrases, and 5,166 questions 2 .

literal classes such as Datetime are represented as diamonds, and other classes are rounded rectangles.


A QA dataset explicitly specifying such question characteristics allows for fine-grained inspection of system performance.

the benefits of this study are two-fold.


removing it does not change the answer q K .

although questions generated from a KB may not follow the distribution of user information needs, it has the advantage of explicit question characteristics, and enables programmatic configuration of question generation.


We build a feature for each combination of translation direction and method, and train a model that learns optimal feature weights.

each method builds a probability distribution for every question word, expressing the translation space in the collection language.


(2014), but employs more flexible semiparametric density models.

the motivation for this problem is an unobtrusive biometric setting in which a user is observed during access to a document, but no specific challenge protocol requiring the user's time and attention is carried out.


This indicates that using highquality word-vector embeddings can have a drastic impact on performance.

two large-scale studies that reveal how sentiment depends on both social and historical context.


In addition, W h , W v , W p , W x , w are the parameters of attention.

in this section, we will review related works on aspect-level sentiment classification and neural networks for sentiment classification briefly.


3 Attention-based LSTM with Aspect Embedding 3.1 Long Short-term Memory (LSTM) Recurrent Neural Network(RNN) is an extension of conventional feed-forward neural network.

most of these studies focus on building sentiment classifiers with features, which include bag-of-words and sentiment lexicons, using SVm (mullen and Collier, 2004).


Aspect information is vital when classifying the polarity of one sentence given aspect.

simply averaging the word embeddings of a target phrase is not sufficient to represent the semantics of the target phrase, resulting a suboptimal performance.


This method requires sufficient data for modeling desired relations.

as will be shown in exper- Domain  Training  Test  Total  Restaurant  3,041  800  3,841  Laptop  3,045  800  3,845  Total  6,086 1,600 7,686 We evaluate our model on the dataset from SemEval Challenge 2014 task 4 (subtask 1), which includes reviews from two domains: restaurant and laptop 3 .


To overcome the limitations of existing methods, we propose a novel model, named Recursive Neural Conditional Random Fields (RNCRF).

during backpropagation, the error for like is propagated through a top-down manner to revise the representation h food .


This task was first studied by Hu and Liu (2004a;2004b), followed by Popescu and Etzioni (2005), Zhuang et al.

they also separate neural network training with structured prediction.


There can be exponential number of phrase sequence possible for each sentence of arbitrary length len(len = 1 for words; len > 1 for phrases) starting at an index st ∈ [0, We observed that most of opinion expression are centered around the head aspect thereby causing the space of potential opinion expressions to be quite sparse.

the thread on fine grained opinion expressions Choi et al., 2006;Breck et al., 2007) focus on subjective expression extraction which are generic instead of aspect specific.


The MPQA 2.0 corpus  has some labeled opinion expressions, but they are generic subjective expressions as opposed to aspect-sentiment phrases (see Section 1) we find in reviews.

the thread of research in (Brody and Elhadad, 2010;titov and McDonald, 2008;Zhao et al., 2010;Mei et al., 2007;Jo and Oh, 2011) focus on extracting and grouping aspect and opinion words via generative models but lack the natural aspect opinion correspondence (e.g., in the manner they appear in sentences).


We note that the Topic Coherence (TC) metric in (Mimno et al., 2011) which is often used to approximate coherence in unigram topic models as it correlates with human notions of coherence, uses codocument frequency of individual words in topics.

dataset Statistics: For CRF training, we created a phrase labeled dataset of aspect opinion phrases using product reviews from Amazon across 4 domains each spanning 4 head aspects.


# of coherent topics: Figure 2 (rightmost chart) shows the number of coherent topics produced by each model.

all the above works focus on generic subjective expressions as opposed to aspect specific opinionsentiment phrases.


The thread on fine grained opinion expressions Choi et al., 2006;Breck et al., 2007) focus on subjective expression extraction which are generic instead of aspect specific.

in opinion mining, it is often desirable to mine the aspect specific opinion expressions (or aspectsentiment phrases) containing both the aspect and the opinion.


The emotion intensity is normalized to make d y x ∈ [0, 1] and ∑ y d y x = 1 to constitute the emotion distribution.

coverage evaluates how many steps are needed to move down the ranked emotion list so as to cover all the relevant emotions of the example.


As mentioned in section 1, some emotions often co-occur such as joy and love, and some rarely co-exist such as joy and anger.

dimensional approaches (Russell, 2003) emphasize the fundamental dimensions of valence and arousal in understanding emotional experience, which have long been studied by emotion theorists.


BP-MLL is derived from the famous backpropagation algorithm through employing a novel error function capturing the characteristics of multi-label learning, i.e., the emotions belonging to a sentence should be ranked higher than those not belonging to that sentence (Zhang and Zhou, 2006).

the third term is employed to incorporate such prior knowledge.


We thank the AMT Turkers who completed our annotation task.

an item refers to a pair of sentences to which humans or NLP systems assign a label (entailment, contradiction, or neutral).


We were unable to build an IRT model that accurately modeled ability to recognize entailment with the obtained response patterns.

such comparison may use model selection indices such as Akaike Information Criterion (AIC) and Conditional Bayesian Information Criterion (CBIC) and should also take into account the interpretablility of the loading pattern that links items to factors.


wise loss function used in ranking (Lee and Lin, 2013).

with word2vec, we train the Skip-Gram with Negative Sampling (SGNS) model since it produces state-of-the-art performance, and is widely used in the NLP community (Mikolov et al., 2013b).


Note that the time complexity of the first stage is O(|Ω|) since the cost of each update in Lines 8-10 is independent of the size of the corpus.

to see this, supposeÛ := {û w } w∈W andV := {v c } c∈C minimizes (9).


The improvement from using the fMRI data of the temporal lobe is the most significant among the four lobes, but the highest correlation coefficient is achieved when utilizing the fMRI data of whole brain.

the BoVW representation of an image was a vector of 8000 dimensions.


The alignments are accomplished using an aligner from JAMR.

the constant in the O is relatively large.


After that we can calculate the posterior estimate of and (line 14 in Table 2).

though this group of work is attractive, if relevant items cannot be fetched in the first place, regardless of the complex re-ranking process, the results still tend to be unsatisfactory.


The deliciousT140 dataset is made up by 144,574 unique URLs, all of them with their corresponding social tags retrieved from del.icio.us.

a Wikipedia snapshot was obtained on the 14/08/2014, which contained a collection of 4,634,369 articles.


In the sampling procedure, we need to calculate the conditional distribution !,!

in this paper we extend personalized web search using social data by exploiting an external knowledge base to enhance the user profile.


5 We also report a gold(x,y) = (a,b) measure: where a is the percentage of gold standard arcs included in trees produced by algorithm x but not by y, and b is the corresponding number for y and x.

the third author was partly supported by a research grant from the Microsoft/technion research center for electronic commerce: Context Sensitive Sentence Understanding for Natural Language Processing.


Unlike modern approximation algorithms for our problem, our algorithm is greedy and deterministic.

in experiments, our algorithms improve the runtime of the TurboParser by a factor of up to 2.41.


For these languages, we first trained the parser on the training set and then used the development set for hyperparameter tuning (|B|, s, α, β, and γ 1 , .

it supports global parameter training based on the comparison between an induced tree and the gold tree.


Dependency parsing is instrumental in NLP applications, with recent examples in information extraction (Wu and Weld, 2010), word embeddings (Levy and Goldberg, 2014), and opinion mining (Almeida et al., 2015).

every other aspect of the parser -feature set, pruning algorithm, cost-augmented MIRA training (Crammer et al., 2006) etc., is kept fixed but our algorithms replace the inference algorithms: Chu-Liu- Edmonds ((Edmonds, 1967), first order) and dual-decomposition (higher order).


It is different from the traditional sequence classification tasks, whose the number of outputs are usually same as the input.

for training, testing, and development, we randomly selected 20 million, 50,000, and 50,000 n-grams respectively.


The results are 1 https://code.google.com/p/word2vec 2 https://github.com/SUTDNLP/ZPar  shown for the multi-view learning model.

the aforementioned methods on heterogeneous annotations are investigated mainly for discrete models.


There have been a large number of studies on English, a morphologically simple language, that show that the effect of normalization, in particular stemming, is different for different applications.

for example, present indicative forms of the German verb "brechen" (to break) are mapped to four different stems ("brech", "brich", "bricht", "brecht").


For instance, Karlgren and Sahlgren (2001) analyze the impact of morphological analysis on creating word representations for synonymy detection.

we use the preprocessed and tokenized Wikipedia datasets of .


Then we fix r = 16 and vary λ from 0.99 to 0.99999.

this work is also closely related with multi-task learning, which aims to jointly learn multiple related tasks with the benefit of using interactive features under a share representation (Ben-David and Schuller, 2003;Ando and Zhang, 2005;Parameswaran and Weinberger, 2010).


Their case study on POS tagging shows that the coupled model outperforms the guide-feature method.

this work makes the following contributions: (1) We propose and systematically compare two ways for realizing context-aware pruning, i.e., online and offline pruning.


RA-CNN consistently outperforms all of the baseline models, across all five datasets.

when training new classification systems, it is natural to acquire supervision at both the document and sentence level, with the aim of inducing a better predictive model, potentially with less effort.


One major application of our approach is to improve the recom-mendation quality in the target service using auxiliary data obtained from another seemingly irrelevant service.

first, the topic vectors and the user latent factors are not required to have equal dimensionalities, which allows for the optimal dimensionality to be chosen in both models.


To address the coldstart issues commonly present in a collaborative ltering (CF) system, most existing crossdomain CF models require auxiliary rating data from another domain; nevertheless, under the cross-website scenario, such data is often unobtainable.

as described previously, we have observed that the hypotheses encoded by the LDa-MF model is too strong as the PMF latent factor is enforced to inherit certain mathematical properties from the LDa topic vector.


we expect that the source and target services contain heterogeneous information (e.g.


For the baseline, we utilize the classifier-based baseline developed by Zou et al.

section 4 illustrates the experimental settings, and reports the experimental results and analysis.


The annotated scope of the cue "appear" is "appear to represent a stone".

we determine whether the candidate token is in the scope of the current cue according to its F 1 .


It is mainly attributed to the shorter scopes of negation cues.


This difference is partly due to the critical nature of medical domain, and partly due to the nature of clinical texts and entities therein.

we believe, exact CRF-LSTM models described here require more training samples than the baseline Bi-LSTM to achieve a comparable recall for labels that are complex or "difficult to detect".


Here F L (t) , contains all factors formed between the variables from the group {y t−m , ..., y t−1 } and y t .

this format allows sequence labeling algorithms to label the relevant information that should be extracted.


The value X[i, j, 3] = |P ij | , P ij = P i ∩ P j ; P i is the product set reviewed by reviewer α i .

they identify three categories of spam: fake reviews (also called untruthful opinions), reviews on the brand only, and nonreviews.Stepping studies focus on studying fake reviews because of its difficulty to be detected.


As we mentioned in Section 2.2, the tensor decomposition embeds the informations over all the relations collectively, and removes the noise of the original data by learning through the global loss function.

in this section, we propose our method (shown in Figure 1) in detail.


Bidirectional LSTMs (Graves and Schmidhuber, 2005) have been shown to learn improved representations of sequences by encoding a sequence from left to right and from right to left.

table 7 further lists baselines reported by Mohammad et al.


A tweet is labelled as positive if one of the positive expressions is detected, else negative if a negative expressions is detected.

such an approach cannot capture the stance of a tweet with respect to a particular target, unless training data is available for each of the test targets.


In order to make it compatible, we use the same test set with 40 newswire articles, the same development set with 30 other documents and the same training set with the remaining 529 documents.

nC-Cnn is 2% better than B- Rnn (nguyen et al., 2016a), the state-of-theart system that only relies on the context information within the sentences of the trigger candidates.


From each training set, we hold out 10% as the development set.

in our experiments, we find that parameter update is necessary for the bottom linear layer because of covariate shift (Sugiyama et al., 2007), which is caused by discrepancy in the distribution between the source and target domains.


Furthermore, we have shown the great potential of using subevents to improve event detection performance.

furthermore, subevent phrases and general event phrases often have the same grammatical structure and may share common words, which makes automatically differentiating between event phrases and subevent phrases within a document even more difficult.


There are some subevents that are unambiguous.

most current research is specifically related to social media applications, like Twitter, in terms of both its definition of subevents and methodologies.


These include: cross-modal linear regression (LinReg, (Dinu et al., 2015)), non-linear regression (NLin-Reg, (Lazaridou et al., 2014;Socher et al., 2013)),  ES-ZSL (Romera-Paredes and Torr, 2015), and a max-margin cross-modal energy function method (CME, (Akata et al., 2013;Frome et al., 2013)).

results may be unreliable when used with polysemous words, or words with large appearance variability.


We propose the task of "sequencing" in a set of image-caption pairs, with the motivation of learning temporal common sense.

we experiment with 2 networks -(1) A language-alone unary model (Skip-Thought+MLP) that uses a Gated Recurrent Unit (GRU) proposed by Cho et al.


Therefore, a forgetting mechanism to "forget" less critical historical information, as is employed in LSTM (controlled by the forget gate f t ), becomes necessary.

in HRL, two independent signals, h LST M  The final representation h HRL T is obtained by the mean pooling of the two "sub" hidden states: h HRL T is then used for higher level tasks such as predicting the sequence label.


Inspired by the success of the gating mechanism of LSTM and the residual connecting mechanism with enhanced information flow of ResNet, we further propose two Gated Residual Recurrent models leveraging the strengths of the two mechanisms.

when it comes to sequence classification, the scenario is quite different in that there is a new input at every time step.


This is a question that needs to be addressed in future research.

the regularization parameters were tuned on the development set over a logarithmic scale between 10 −3 to 10 3 .


Empirically, for non-convex objectives, different approaches may arrive at different solutions.

hence, both Lemma 2.1 and Theorem 2.3 of this paper apply, and our framework also generalizes multi-task regularized learning.


The relation between their work and the feature augmentation technique was also briefly discussed in the paper of Daumé III (2007).

for convex loss functions, our objective (Equation 2) is also convex, and all approaches should share the same solution.


(2016), who show that neural network-based models are suboptimal to use for smaller amounts of data.

these dimensionalities have been reported to perform well for the respective models (Landauer and Dumais, 1997;Sahlgren et al., 2008;Mikolov et al., 2013;Österlund et al., 2015).


The CO model underperforms in all frequency ranges.

our results also show that none of the standard DSMs work well in situations with small data.


The framework of MGE is sketched in Figure 1.

this task evaluates the effectiveness of embeddings in capturing linguistic regularities between pairs of words, in the form of " 伦敦 (London) : 英国 (England) ≈ 巴黎 (Paris) : 法国 (France)".


The assigned score represents how much more probable a correction is than the original document.

this  approach can incorporate KB tuples flexibly, even when values of some attributes are missing.


In addition, our model shows results competitive with the best single models of the competition, while requiring no expensive hand-crafted features or external resources, thereby demonstrating its language and domain independence.

we show that the hierarchical model outperforms two non-hierarchical baselines, obtains results competitive with the state-of-the-art, and outperforms the state-of-the-art on five multilingual, multi-domain datasets without any handengineered features or external resources.


Neural network-based approaches include an LSTM that determines sentiment towards a target word based on its position (Tang et al., 2015) as well as a recursive neural network that requires parse trees (Nguyen and Shirai, 2015).

the sentence-level forward and backward LStMs receive the sentence starting with the first and last word embedding x 1 and x l respectively.


By leveraging the structure of the review, our model is able to inform and improve its sentiment predictions as evidenced in Table 2.

it is less expressive than ours, as it only extracts features from the preceding and subsequent sentence without any notion of structure.


For all similarity scores, we divide them by square of distance between the two words.

sarcasm is a form of verbal irony that is intended to express contempt or ridicule.


All entries in the tables are higher than the simple unigrams baseline, i.e., F-score for each of the four is higher than unigrams -highlighting that these are better features for sarcasm detection than simple unigrams.

our paper deals with the question: Can word embedding-based features when augmented to features reported in prior work improve the performance of sarcasm detection?


The function captures the distance between the label for a user and her tweet.

the goal of the task was to evaluate stance classification systems, which used minimal labeling on phrases.


Crowdsourced Annotation The GVDB is built and updated through a continuously running crowdsourced annotation pipeline.

reasoning about the causes and outcomes of gun violence is typically dominated by politics and emotion, and data-driven research on the topic is stymied by a shortage of data and a lack of federal funding.


However, for the past 25 years, research in this area has been, in the best case, massively underfunded (Roth et al., 1993) and in the worst case, actively blocked by federal legislation (Kassirer, 1995;Frankel, 2015;Bertrand, 2015).

in the future, the database will also include full within-document coreference annotation, with all mentions of a shooter/victim being flagged as such, and will incorporate visual data, so that within-article images are tagged with relevant information which may not be communicated by the text alone (e.g.


To build a suitable data set 2 for fluency detection, we first identified 1000 Twitter users who, according to the Twitter LID system, have tweeted in Russian and at least one additional language.

social media platforms like Twitter produce data sets in which individual documents are extremely short, and language use is idiosyncratic: LID performance on such data is dramatically lower than on traditional corpora (Bergsma et al., 2012;Carter et al., 2013).


Finally, with the newly proposed features, the parser trained on ASR texts outperforms the parser trained on the transcription texts with the improvement of 1.5% and 0.7% for UAS and disfluency detection, respectively.

leftArcBackoff is not actual action performed by our parser, rather it is used to provide the common feature representation which both leftArc and leftArcError can "back off" to.


To avoid this problem, we instead attach them to the head of the deleted token.

when compared with the case of (Train, Test) = (T rans, T rans), we observe significant decreases in performance in both of the tasks conducted on ASR texts.


Hope your next vacation will be a good one.

a separate, CNN-based sentiment analysis module recognizes sentiments from speech recognition results, with 82.5 Fmeasure on human-machine dialogues when trained with out-of-domain data.


Figure 1 shows the likelihood LL obtained by varying {λ 1 , λ 2 }.

under such a setting, robust models are expected to be effected less from the inherent variation.


Instead, approximate rank constraints are utilized with a Trace Norm (Fazel et al., 2001) or Max Norm (Srebro and Shraibman, 2005).

but for the Robust Gram removing the rare words actually means a significant decrease in useful information, and the performance starts to degrade towards the SG performance.


Our SimpleScience approach outperforms the original approach by Biran et al.

we use word embeddings to extract simplification rules from a novel parallel corpora that contains scientific publications and wikipedia.


Embeddings identify words that share context in an unsupervised, scalable way and are more efficient than constructing co-occurrence matrices (Biran et al., 2011).

we created an antonym set consisting of the 304 terms from the biology, chemistry, and physics categories in Wictionary for which at least one antonym is listed in WordNik.


The equation represented by the tree can be generated as follows.

we call the problem of mapping noun phrases to variables the problem of grounding variables.


Example (2) is also a simple negated clause, and 4 out of 5 interpretations receive high scores, capturing valid positive meaning.

note that splitting instances randomly would not be sound: training with some interpretations generated from a negation, and testing with the rest of interpretations generated from the same negation would be an unfair evaluation.


These variations do not reflect features attested in the literature; rather, they appear to be purely orthographic variations highly specific to AAE-speaking communities online.

the approach requires a number of ad-hoc thresholds, cannot capture authors who only occasionally use demographically-aligned language, and cannot differentiate language use at the message-level.


This is different from previous work, where a max-over-time pooling operation was used to find the most activated feature in the time series.

their model differs from ours in that it uses a max-pooling layer that picks the most activated feature across time, thus ignoring temporal information, whereas we explicitly avoid doing so.


Network structures: We used both feed-forward networks with tanh non-linearities and LSTM (Hochreiter and Schmidhuber, 1997) networks.

for the main experiments, we evaluate on two corpora: the Penn Treebank (PTB) data set prepared by Mikolov et al.


Maximum likelihood estimation: n-gram models predict the next word based on the previous N -1 words.


The PTB corpus uses the standard vocabulary of 10k words, and for the ASPEC corpus we use a vocabulary of the 20k most frequent words.

the model learns to set the mixture proportions of the δ elements to near zero and rely mainly on the count-based n-gram distributions.


in addition, we add words from popular rhyme pairs 7 (like do/you and go/know) to the list of related words with a low topic similarity score.


Sinalefa merges vowels across word boundaries.

• Each line is punctuated with comma or period, except for the 4th, 8th, 12th, and 14th lines, which are punctuated with period.


This approach enables the incorporation of rich context when mapping between consecutive dialogue turns  in a way not possible, for example, with MT-based dialogue models (Ritter et al., 2011).

it is not clear how well MLE approximates the real-world goal of chatbot development: teaching a machine to converse with humans, while providing interesting, diverse, and informative feedback that keeps users engaged.


Our first model, Bidirectional Long Short-Term Memory (BLSTM) network contains two LSTM blocks (forward and backward), each with 64 hidden units on the output.

we can thus conclude that workers tend to prefer explanations based on labels from the reason units and using the annotation process presented in this section is reliable.


It has often been suggested (Asher, 1993;Hardt, 1997;Hardt and Romero, 2004) that the antecedent selection process is very closely tied to discourse relations, in the sense that there is a strong preference or even a requirement for a discourse relation between the antecedent and ellipsis.

analysis of the annotated corpus data reveals surprising ambiguity in the identification of the antecedent for sluicing.


Our MCNN architecture was inspired by a Siamese architecture (Chopra et al., 2005), which we extend to a multi-column network and replace its similarity measure with a softmax function at its top.

dEPTREE The third set consists of four columns.


In such a case, an antecedent probably exists among the most salient candidates in the embedded clause.

our experimental results show that the proposed method achieved better precision than the strong baselines in a wide range of recall levels.


Although the existing works also exploited such word sequences, they used only particular types of information from them as features based on the researchers' linguistic insights.

this is because the zero anaphoric relations are exclusive; a zero anaphor does not refer to more than one antecedent.


In the explanation based on Centering Theory, the most salient candidate (e.g., the candidate marked with wa (topic marker)) is selected as /"4-"57$3"4 !8:*&% "7$97$!,#*"% %&'89""534:, "# "$<.*,/"57%4!

we minimized such feature engineering due to using an MCNN.


We first use a sample of 2000 sentences, out of which we use 200 as a development set and the rest as a test set.

the model we propose here, which combines fast_align and k-means clustering using DtW and DBA, outperforms both a very strong naive baseline and a neural attentional model, on three tasks of various sizes.


The Abstract Meaning Representation (AMR) (Banarescu et al., 2013) shares UCCA's motivation for defining a more complete semantic annotation.

uCCA (universal Conceptual Cognitive Annotation) (Abend and Rappoport, 2013) is a cross-linguistically applicable scheme for semantic annotation.


We will use the example "man bites dog" to illustrate typical examples of why a structural node should be labelled as "Bad": incorrect ordering ("dog bites man"), deletion ("man bites") and insertion ("man bites biscuit dog").

these provide no insight into possible errors, and do not scale well with sentence length.


The baseline NER system is an MEMM model (CRF cannot handle such a big size of training data, since our NER system has 51 entity types, and the number of features and training time of CRF grow at least quadratically in the number of entity types).

a model could still make mistakes if its features favor a wrong entity type, which happens more frequently for unseen entities as we have observed in our experiments.


These results confirm our hypothesis in §4.3 that U is better at capturing bilingual relations and V is better at capturing monolingual relations.

previous attempts had expensive resource requirements, difficulty incorporating monolingual data or were unable to handle polysemy.


The translationw i is selected from the possible translations of w i listed in the dictionary.

u is bound to better at bilingual lexicon induction task and V is better at monolingual word similarity task.


While word ordering is a somewhat synthetic task, it has two interesting properties for our purposes.

we also experiment with constraining the BSO model during training, as described in Section 4.2, by defining the succ function to only allow successor sequences containing un-used words in the source sentence.


We define this distillation for a sentence as, where V is the target vocabulary set.

this process is nonideal for two reasons: (1) unlike for standard knowledge distribution, it doubles the size of the training data, and (2) it requires training on both the teachergenerated sequence and the true sequence, conditioned on the same source input.


In this paper, we propose and investigate four methods for controlling the output sequence length for neural encoder-decoder models.

(Rush et al., 2015) 26.55 7.06 22.05 ABS+ (Rush et al., 2015) 28.18 8.49 23.81 RAS-Elman (Chopra et al., 2016) 28.97 8.26 24.06 RAS-LSTM (Chopra et al., 2016) 27.41 7.69 23.06 opportunity to use the Kurisu server of Dwango Co., Ltd. for our experiments.


Instead, we define the legitimate range of the sequence by setting minimum and maximum lengths.

figure 1 shows the statistics of the corpus.


Figure 2: The encoder-decoder architecture we used as a base model in this paper.

the memory cell of LSTM networks is suitable for this endeavour, as it is possible for LSTMs to learn functions that, for example, subtract a fixed amount from a particular memory cell every time they output a word.


However, this approach does not guarantee the label proportion observed in the annotated data (which in this case is not well-separated as shown in Section 4).

we can splitT at the boundary of labeled and unlabeled data as follows: where Y 0 is the shape of Y at iteration 0.


For example, the formula, F ather(x, y) ∧ F ather(y, z) ⇒ Grandf ather(x, t), has a low score when x and y both have several sons, but it obviously is the most effective to infer Grandf ather.

finally, we dive deep into the formulas generated by our approach to analyze the characters of our approach.


The potential Φ(r(i, j), ρ) measures an implicit relationship between two triplets in the KB, so the total number of parameters is the square of the K-B size.

the computation complexity of MLN increases exponentially with the scale of a KB.


As the key and value are the same in this case, this is identical to a standard MemNN and this approach has been used in several papers (Weston et al., 2016;Dodge et al., 2016).

unfortunately KBs often suffer from being too restrictive, as the schema cannot support certain types of answers, and too sparse, e.g.


Window representations for MemNNs have been shown to work well previously (Hill et al., 2016).

such structured memory is not employed in any existing attention-based neural network architecture for QA.


it should be noted that the advantage of working from raw documents in real applications is that data sparsity is less of a concern than for a KB, while on the other hand the KB has the information already parsed in a form amenable to manipulation by machines.


We have extended models for discovering latent personas to simultaneously cluster documents by their "casts" of personas.

the DPM assumes that each document has a unique distribution (θ i ) from which its personas are drawn.


We also acquired the personality and value scores of each Turker based on the IPIP personality and PVQ value survey.

for example, STOP is a Natural Language Generation (NLG) system that generates tailored smoking cessation letters based on a user's responses to a four-page smoking questionnaire (Reiter et al., 2003); PER-SIVAL customizes the content of search summaries based on its relevance to a given patient's health record (McKeown et al., 2003); MATCH (Johnston et al., 2002) is a multimodal dialogue system that tailors the content of its responses based on a user's restaurant preferences; M-PIRO (Burenhult, 2002) tailors the words and the complexity of museum object descriptions for different audiences (e.g.


Since customizing a message based on an interaction of all ten traits can be very challenging for a Turker, we used a simplified trait profile in our study.

we employed constrained clustering in this analysis.


• We combine this distributed model with a feedforward neural network that learns non-linear combinations of surface features.

for each input tweet, we consider any n-grams of the tweet that match the lexicon as mention candidates.


Tweet entity linking Previous work on entity linking mainly focuses on well-written documents (Bunescu and Pasca, 2006;Cucerzan, 2007;Milne and Witten, 2008), where entity disambigua-tion is usually performed by maximizing the global topical coherence between entities.

in addition, 800 tweets from the NEELtrain dataset are sampled as our development set to perform parameter tuning.


For the word embedding size, we find 600 offers marginal improvements over 400 but requires longer training time.

the parameters of the composition model are Θ 2 = {W (u,e) , W (m,e) , E (u) , E (w) , E (e) }.


(2013) leverage social relations for sentiment analysis by exploiting a factor graph model and the graph Laplacian technique respectively, so that the tweets belonging to social connected users share similar label distributions.

these approaches often yield unsatisfactory performance on Twitter messages, due to the short and noisy nature of the tweets.


For each input tweet, we consider any n-grams of the tweet that match the lexicon as mention candidates.

we use the Twitter API to crawl the friends of the NEEL users (individuals that they follow) and the mention/retweet links are induced from their most recent 3,200 tweets.


(2016) follow a very similar architecture to , replacing the LSTMs with Gated Recurrent Units .

the following approaches minimize the use of features while still maintaining a monolingual focus.


This category includes models that approximate (1) based on the Markov dependence assumption of order N −1.

in Figure 5, we show the variation of this temporal correlation for the local and global states of the proposed LSRC network in comparison to RNN and LSTM for various values of the distance d (up to 3000).


The AoG learned based on the tight integration with language yields significantly better performance than the loose integration (over 0.2 gain on F-score, p < 0.001).

the computer vision based clustering on average produces more than 30 clusters when all the 36 training examples are used.


Thus, if there are other good pairs not too much worse than the oracle, the margin for such tuples will not be tightly enforced.

learning a visual representation for noun-preposition-noun triplets ("car on road").


A ED network first encodes an input x into a vector which is then used to initialize a recurrent neural network (RNN) for decoding the output y.

each y i is a discrete symbol from a finite dictionary V of size m. Typically, m is large.


For a significant fraction of cases (27% for Reddit, and 21% for Subtitles), the local margin is positive while the global margin is negative.

we find the EE model is largely better that the ED model.


Our model uses a second RNN encoder to represent the output as another fixed length vector.

our model is During training we use maximum likelihood to estimate θ given a large set of valid input-output pairs where each y i belongs to Y which in general is much larger than W. Our main challenge during training is that Y is intractably large for computing Z.


Very few works try to interpret recurrent neural networks in NLP.

an improvement over the lower bound models indicates that the encoder learns syntactic information, whereas a decline from the upper bound model shows that the encoder loses certain syntactic information.


From the last two lines in Table 1, we can see that RNNSearch-Mono-Autoencoder can also improve the translation quality by more than 1.0 BLEU points when using the most related monolingual data.

in this work, we investigate the usage of the source-side large-scale monolingual data in NMT and aim at greatly enhancing its encoder network so that we can obtain high quality context vector representations.


Systems with better training data, can be tuned with BLEU without suffering this "disabling" effect, but will reach non-optimal performance.

our main contributions are presented in sections 3 and 4 where we investigate the interaction of parameter tuning towards the M 2 metric with task-specific dense and sparse features.


NUCLE consists of 1,414 essays written by Singaporean students who are nonnative speakers of English.

most of the improvement over the CoNLL-2014 shared task of these works stems from using the parameter tuning tools we introduced in Junczys-Dowmunt and Grundkiewicz (2014).


The exact variety of the NMT model depends on how we calculate η i used as input.

pBMT and other traditional SMT methods tend to rarely make this kind of mistake.


The fact that the two models start from and converge to very different points, yet have similar training set performances, indicates that our architecture 13.9 14.7 13.8 13.7 14.4 and training algorithm are able to reach a good minimum of the training objective regardless of the initialization.

in this paper, we describe a method for substantially improving NMT results on these languages.


For training we use a minibatch size of 128, hidden state size of 1000, a target vocabulary size of 15K, and a source vocabulary size of 30K.

we explore transfer learning for the child model with parent models that only use the English side of the French-English corpus.


For example, for relatedness, lexical similarity is 2.6 MAP points more informative 10 than distributed representations.

in the figure, q stands for the new question, q is an existing related question, and c is a comment within the thread of question q .


This demonstrates the robustness of the character-level model to unseen entities.

we find that a two-layer LSTM boosts joint accuracy by over 6%.


There are other studies focused on joint learning with sentiment (Luo et al., 2015;Mohtarami et al., 2013), emotion in tweets or blog (Hasegawa et al., 2013;Qadir and Riloff, 2014;Ou et al., 2014;Quan and Ren, 2009), and emotional lexicon construction Staiano and Guerini, 2014;Mohammad and Turney, 2013).

it means that the consideration of terminal node improves the performance of the tree kernel significantly.


This is why some studies even incorporate cause extraction with target identification to improve performance.


"I was assigned to work at the Barbershop in Danyang, Nanjing.

we first present a dataset we built using SINA city news.


We bring in User Product Attention to capture the crucial components over different semantic levels for sentiment classification.

we also thank all anonymous reviewers for their insightful suggestions.


Sentiment analysis aims to analyze people's sentiments or opinions according to their generated texts and plays a critical role in the area of data mining and natural language processing.

in sentence level, we also use a attention mechanism with user vector u and product vector p in word level to select informative sentences to compose the document representation.


recently, a number of neural models have been developed and achieved new levels of performance (Kim, 2014;Socher et al., 2013;Lei et al., 2015).


This section introduces the proposed framework that enables joint learning of knowledge components and their weights with the neural network models.

without interaction between the knowledge and neural network learning, the pipelined method yields inferior results.


Hence, the value of p i denotes the extent of semantic relatedness between y i and y t .

the relatively low performance of their system shows that the usage of glosses in WordNet and the automated disambiguation have not resulted in accurate sense representations.


PredPatt derives from the system described by Rudinger and Van Durme (2014).

participles and imperatives were excluded, as well as verbs in embedded clauses, in questions, and under negation or auxiliaries.


We filtered amici who have participated in fewer than 5 briefs 11 and merged regional chapters of amicus organizations together (i.e., "ACLU of Kansas" and "ACLU of Kentucky" are both labeled "ACLU").

we used Metropolis within Gibbs, a hybrid MCMC algorithm, to sample the latent parameters from their posterior distributions (Tierney, 1994).


The results we present on the different auxiliary categories (see Tables 2 and 4) are obtained from training a single classifier over the entire dataset and then testing on auxiliaries from each category, with the ALL result being the accuracy obtained over all of the test data.

not only do we improve upon the stateof-the-art accuracy, but we also expand the scope of VPE-detection to include to-VPE without causing a significant decrease in accuracy.


Yara Parser (Rasooli and Tetreault, 2015) 9 A transition-based parser that uses beam search training and dynamic oracle.

we ignore punctuations when parsers are trained with the Tweebank data, because punctuations are not annotated in the tweets with their dependencies.


Recently, such techniques were used to build state-of-the-art syntactic parsers, and have demonstrated the effectiveness of deep neural architectures for decision making in lineartime dependency parsing (Chen and Manning, 2014;Dyer et al., 2015;Andor et al., 2016;Kiperwasser and Goldberg, 2016).

we set the number of epochs T = 3, regularization coefficient 2 = 10 −6 , learning rate 10 http://deeplearning4j.org/ Parser P R F CKY (Artzi et al., 2015) 67   µ = 0.05, skipping term γ = 1.0.


The generated set Y approximates an intermediate state of a shift-reduce derivation.

the number of concatenated vectors and stack elements used is for illustration.


For example, for toys kids, we are pretty sure as a web query, its intent is to look for toys for kids, instead of the other way around.

from clicked sentences that contain crude oil price, we de-rive crude dep ← − −price, but from those that contain crude price, we derive crude amod ← −− −price.


We define text synopses and dialogues as S where S i and T i are the set of sentences from synopses and dialogues of the i-th episode.

− TREM: Our TREM model proposed in Section 4 extracts sentences that can both summarize the current episode and prompt the next episode with two contingency factors.


These abundant textual resources enable us to study a novel, yet challenging task: automatic text recap extraction, illustrated in Figure 1.

intuitively, our model learns W by assuming each sentence in D i+1 can be reconstructed by contingent sentences from D i : in the equation, we first convert every description sentence to its distributed representation using the pre-trained skip-thought model proposed by Kiros et al.


|c| is the number of words in c. We model plot contingency on the concept level as well as on the sentence level.

under assumption (b), the text recap should capture the connections between two consecutive episodes.


We use this feature as indicator for component or object identification.

the approach itself is presented in Section 5 before it is evaluated in Section 6.


Since the approach should be able to distinguish between (ontopic) requirements and off-topic comments, we acquired software domain-specific off-topic sentences, too.

(2010), aims at the modeling of requirements by comparing transformation techniques in such models.


In L 1 and L2 regularizations, features are considered as independent, which makes sense without any additional prior knowledge.

we split the training set in a stratified manner to retain the percentage of classes.


M is an indicator matrix of size L × V , where L is the sum of the total sizes of all groups, and its ones show the link between the actual weights θ and their copies v. Following Yogatama and Smith (2014b) a constrained optimization problem is formed, that can be transformed to an augmented Lagrangian problem: Essentially, the problem becomes the iterative update of θ, v and u: Convergence Yogatama and Smith (2014b) proved that ADMM for sparse overlapping group lasso converges.

the overall expected risk (Vapnik, 1991) is the weighted sum of two components: the empirical risk and a regularization penalty term, expression referred to as "Loss+Penalty" by Hastie et al.


So up to three operations can be returned per word.

we select a total of 95 out of these 128, following four criteria: (i) reuse should not consist of an exact literal copy of a Bible verse (skipping six instances), (ii) reuse should be recognizable by our expert (skipping ten instances), (iii) the reference frame should be within five Bible verses (comparable with sentences) to avoid too much noise in our data to ensure a comparable length to the original Bible verse (skipping nine instances), and (iv) reuse instances should not exceed a length of 40 tokens (1-2 sentences), again to cut the long tail and avoid too much noise (skipping eight instances).


In the context of massive digitization projects, it can identify relationships between text excerpts referring to the same source.

efforts to automatically process ancient texts are made around the Perseus Digital Library project (Crane, 1985), among others.


The top 2% of the features (or 1% for the sentence-in-passage with coreference, since the feature set size is doubled) were selected automatically.

each sentence was compared to five others, which created 300 comparisons in each task.


We found that 19% of pairs had only a vocabulary distinction, and 65% of pairs could be judged correctly either by vocabulary or syntax.

this work was supported in part by Dept.


There exist many automated essay scoring systems (Shermis and Burstein, 2013) and some of them are being used in high-stakes assessments.

since the essays are usually long, consisting of hundreds of words, the learnt vector representation might not be sufficient for accurate scoring.


However, the two sentences are unequal in length, thus it is hard to know what the extra phrase 'wi-fi and' should be compared against.

while true positive cases for both NLD and PD can exist, there are not likely to happen in prac-tice since textual differences in PD tend to be much higher than in NLD.


(2) If the in-domain data is too small, the probability distribution learnt by the indomain NNJM will be overfitted.

we re-implement their method by incorporating this regularization term into the log likelihood objec-tive function with self-normalization, L (Equation 2), during adaptive training.


This opens up the possibility of further exploration of sub-word level translation units e.g.

there is overlap between the vocabulary of these languages to varying degrees due to cognates, language contact and loanwords from Sanskrit (throughout history) and English (in recent times).


Neural Machine Translation Recent advances in neural machine translation (NMT) (Bahdanau et al., 2014;Devlin et al., 2014) using the framework of sequence to sequence learning (Sutskever et al., 2014) have demonstrated the effectiveness of deep learning models at capturing and translating language semantics.

our work, however, is closest to Kushman and Barzilay (2013).


Evaluation Metric To accurately evaluate our model, we perform a functional equality check called DFA-Equal.

our method is similar in that we require only description and regex pairs to learn.r we treat the problem as a direct translation task without applying any domain-specific knowledge.


When assigning these 100 documents to 9 other annotators, the amount of positive instances increases to 1,258 keyphrases, which means that labels for 889 keyphrase candidates, or 17% of the original negative candidates when training on annotations by a single annotator, can be considered noise and relabeled.

instead of assuming the noise to be random, we assign weights depending on the document and the candidate.


Using distant supervision we obtained examples of correct and incorrect anchoring of event mentions to entities and temporal expressions.

we further improve this result by 3.2 F 1 -score points using joint inference.


model (60% for the ATT model, 73% for the MCB model) which is more than the respective average accuracy on the entire VQA validation set (54.13% for the CNN+LSTM model, 57.02% for the ATT model, 60.36% for the MCB model).

our motive is simply to better understand current generation models via their behaviors, and use these observations to guide future choices -do we need novel model classes?


In this paper, we develop novel techniques to characterize the behavior of VQA models.

producing the same response across images seems to be statistically favorable.


The network then essentially becomes a multi-task model with two loss functions.

further, LSTMs are also effective as language models (LMs) (Sundermeyer et al., 2010).


This LSTM layer encodes the video sequence.

our approach differs in two 1 The LM was trained to achieve a perplexity of 120  Figure 2: Illustration of our late and deep fusion approaches to integrate an independently trained LM to aid video captioning.


We tried to test variations of this measure with our representation, but we were not able to achieve satisfying results.

proximity between word vectors is taken as an index of meaning similarity, and vector cosine is generally adopted to measure such proximity, even though other measures have been proposed (Weeds et al., 2004;Santus et al., 2016).


Table 2 presents the results of the cross-domain experiment, whereby we train a model on MR and test on CR, and vice versa, to measure the robustness of the different regularization methods in a more real-world setting.

finally, the model minimizes the loss of the cross-entropy between the ground-truth and the model prediction, L = CrossEntropy(y true , y pred ), for which we use stochastic gradient descent.


Conventional training seeks to minimise the difference between y true and y pred .

figure 1 is a plot of the training and test accuracy at varying points during training over SST.


Where our proposed approach differs is that we train models using first-order derivatives of the training loss as part of a regularization term, necessitating second-order derivatives for computing the gradient.

and In other words, minimising local noise sensitivity is equivalent to minimising the Frobenius norm of the Jacobean matrix of partial derivatives of the model outputs wrt its inputs.


However, α i is required to be positive in Dir because otherwise the normalization factor becomes divergent.

our modified Dirichlet distribution (mDir) is defined as follows.


In order to alleviate these issues, we introduce an RNN-LM that utilizes both character-level and word-level inputs.

since the word embedding part is independent from the language modeling part, our model retains the flexibility to change the architecture of the language modeling part.


Here, φ agree is a feature of agreement constraint, which assigns each alignment direction to a sign (i.e., +1 or -1).

to reduce computational cost, the probabilities of phrasal rules P (A → ⟨Y /Z⟩) and P (A → [Y /Z]) are not trained, which are set to 0.5 following Saers et al.


As described in Section 2, the ITG constraint is imposed in the E-step of the EM algorithm, not in decoding steps.

table 2 shows the results of word alignment evaluations 8 , where none denotes that the model has no constraint.


Due to the ITG constraint, the proposed method can symmetrize two directional alignments based on their structural coherence.

for the sentences that are not contained in the training corpus, the word alignments are calculated using the emission, transition and fertility tables trained with the constraint.


These wrong alignments disturb the phrase extraction, because excessively long phrase pairs are extracted by bridging the gaps in wrong alignments or simply no phrase pairs are extracted from wrong gappy alignments.

φ agree is defined as follows: where − → Z and ← − Z are sets of possible alignments generated by source-to-target and target-to-source alignment models, respectively.


We noticed that tuning the number of features to keep made little or no difference in the absolute ordering of the different DS models.

simLex-999 A word pairing task meant to specifically target similarity rather than the more broad "relatedness" (Hill et al., 2015).


The brain is the locus of our language ability, and so brain images can be used to ground linguistic theories.

averaging the results together may be misleading, as the fMRI and MEG result patterns are different.


Our models are based on simple convolutional neural networks directly on raw text, avoiding any manual identification of complex sentiment or syntactic features, while performing better than such feature-based models from previous work.

we find activations that also react to indefinite pronouns such as something/somebody.


The authors acknowledge support from the Templeton Religion Trust (grant TRT-0048) and Bioibérica.

beyond sentiment, this paper investigates how expressions of personal well-being translate between English and Spanish on Twitter.


Prior studies that have examined the role of reviews as a form of interaction between the reader and the author have documented the first two categories extensively (Campbell et al., 2016;Magnifico et al., 2015;Lammers and Marsh, 2015;Black, 2006).

2011, who find a greater increase in mentions of Mr. Darcy relative to Elizabeth in a different corpus of Pride and Prejudice fanfiction, and supports claims that fanfiction authors may delve deeper into characters that receive less attention in the canon (Jwa, 2012;Thomas, 2011).


Both Levin's and CCA-based verb classes are derived from syntactic behavior.

we then build matrix S by multiplying x by u and matrix O by multiplying z by v to get the verbs by CCA-components from subjectview, and verbs by object components from objectview respectively.


For instance, Levin's "admire" verb class contains both 'love' and 'hate".

these analyses are performed out of context-syntactic and semantic-which may change the magnitude or even direction of such relationships.


It should also be noted that the results reported in (Klebanov et al., 2014) are not the same, due to the mentioned differences in the implementation of the features and possibly other differences in the experimental setup (e.g., data filtering, pre-processing and meta-parameter optimization).

2014, we use the same classifier, i.e., logistic regression, in the implementation bundled with the scikit-learn package (Pedregosa et al., 2011).


As shown by the feature ablation experi-ments, one of the main reasons for the performance difference between N and B is the ability of the former to model domain information.

for the genres "academic" and "fiction", combining N and B features improves classification performance over B, and the difference is always statistically significant.


At test time, the model combines the embedding representation of a new bill with the trained ideal vector of a congressperson and generates a prediction for how the congressperson will vote on the bill.

they are trained using the complete vote matrix for the bill, which indicates how each congressperson voted on each bill.


An ideal point model would likely capture the "enterprise" dimension, but not the "science" one, and would not be able to distinguish between libertarians like Ron Paul (R-TX) who are against both "corporate welfare" and government science funding, conservative budget hawks like Jeff Flake (R-AZ) who favor business but are skeptical of government funding of science, and es- Figure 2: Relative likelihood of congresspeople in the 110th Congress voting for a bill containing only the word "Enterprise" versus only the word "Science."

1 our dataset consists of all votes on the full-text (not amendments) of bills or resolutions from the 106th-111th Congress, six of the most recent Congresses for which bill texts are readily available.


By using the proposed technique, we maximize the reuse of existing data labeled for different domains and applications.

in ALARM domain, the slot indicating time is subdivided into sub-slots such as start time representing starting time, duration representing duration for an alarm.


For example, ALARM domain, there are 6 shared slots among 8 slots.

building new domains and tasks that need a separate set of models is a bottleneck for scaling to a large number of domains and experiences.


(Sutskever et al., 2011;Song and Roth, 2015)).

rather than derive the vector from the entire article (which can be long), we only use the top summary section.


While we use the loss function commonly used for binary classification during training, we ultimately want to frame our question as a ranking problem.

we leverage Mechanical Turk to link each ingredient to the appropriate wikipedia article.


We show how two computational measures can be used to distinguish between semantic changes caused by cultural shifts (e.g., technological advancements) and those caused by more regular processes of semantic change (e.g., grammaticalization or subjectification).

some well-known regular linguistic shifts include rarer parts of speech like adverbs (included in our case studies below).


Each file corresponds to a document D. In a document, one line corresponds to one log given by a user.

the dynamic model predicts correct addressees most robustly.


To test this, we retrained the model with only hypernymy as positive examples, and all other relations as negative.

these H-features are aggregate indicators of hypernymy across a large corpus.


For instance, 5 VN classes did not have any member verbs included, 22 VN classes had only 1 verb, and 6 VN classes had 2 verbs included in the current set.

these datasets do not provide a representative sample of verbs (Hill et al., 2015).


The other systems did not report that measure.

we selected the 1000 most frequent word-Net types in YAGO3 (incl.


First, we randomly chose 100 semantically typed relational phrases with at least three words (to focus on the more interesting multi-word case, rather than single verbs).

dEFIE's synonyms often do not fit the semantic type signature of the given relational phrase and are thus incorrect.


Chunk similarity: Every aligned chunk is associated with a relatedness score between zero and five, denoting the range from unrelated to equivalent.

1, with the variables associated with unaligned chunks being scored zero.


We post-processed the predicted chunks to correct for errors using the following steps: 1.

then, we optimize the full objective (all three loss terms) on the combined dataset.


The rows 1 and 5 in the table correspond to the in-domain settings and match the results of typed alignment and score in Table 2  When the headlines data is tested on the images section, we see that there is the usual domain adaptation problem (row 3 vs row 1) and using target images sentence data does not help (row 4 vs row 3).

the typed similarity evaluation (denoted by typed score) is the more stringent evaluation that measures F1 of the alignment edge labels, but penalizes them if the similarity scores do not match.


The goal of inference is to find the assignment to the inference variables that maximizes total score.

the implicit assumption in such approaches is that better constituent alignments can lead to better identification of semantic relationships between sentences.


In the remaining part of the paper, we refer to x i ∈ R p as the token embedding for word at position i.

we use stochastic gradient descent with the Adam algorithm proposed in (Kingma and Ba, 2015).


The proposed BiAtt-DP parses a sentence in a linear order via sequentially querying the memory component that stores continuous embeddings for all headwords.

as we can see, the loss function (3) tries to minimize the distance between the golden alignment and the intersection of the two directional attention alignments at every time step.


First, we show that parser bias entails substantial overestimation of parser performance.

one participant reviews an annotation generated by the Turbo tagger and parser (Martins et al., 2013).


Experiments on non-expert dependency annotation from scratch were previously reported for French, suggesting low agreement rates (79%) with an expert annotation benchmark (Gerdes, 2013).

most such evaluations were conducted us-ing annotation setups that can be affected by an anchoring bias (Carroll et al., 1999;Rambow et al., 2002;.


We then use the original sentence as p and the extracted complement as h: e.g.


quantifiers, all primitive expressions in first-order logic are confined to constant symbols of type E and predicates of type E ⇒ Prop, E ⇒ E ⇒ Prop, and so on.

each problem has one or more premises, followed by a hypothesis.


Data The dataset used to train and evaluate our model consists of pairs of colors and descriptions collected in an open online survey (Munroe, 2010).

the representation f of a color (h, s, v) is given bŷ The Fourier representation is inspired by the use of Fourier feature descriptors in computer vision applications (Zhang and Lu, 2002).


We also test the REINFORCE policy gradient algorithm (Williams, 1992).

most measures of coreference resolution performance do not decompose over local decisions, which means the utility of a particular decision is not known until all other decisions have been made.