Using the dependency tree as illustrated in Figure 2, SEDT is able to identify the subject phrase correctly, namely "The Annual Conference".

biDAF failed to anwer the question correctly and selected a noun phrase as the answer.


(2017) investigated a similar approach to generate embeddings for out-of-vocabulary words from their definitions and applied it to a number of different tasks.

their method mainly focuses on modeling generic concepts and is evaluated on tasks that do not require the understanding of world knowledge specifically.


The double encoder architecture mentioned above considers each context independently.

since each document consists of a sequence of contexts, the knowledge carried by other contexts in C could also provide useful information for the decision process of C i .


Note that alternatively, one could aggregate information about the past predictions through other means like policies or soft attention.

this would introduce extra complexities to the learning process.


We experimented with different configurations of defining contexts and entity definitions, such as expanding the context windows by including sentences that come before and after the one where blank is found, as well as taking more than one sentence out of the entity description.

results on validation set show that increasing the context window size and the definition size had very little impact on accuracies, but drastically increased the training time of all models.


Moreover, these datasets have the attractive quality that the answer is a short snippet of text within the paragraph, which narrows the search space of possible answer spans.

many of these models rely on large amounts of human-labeled data for training.


There has also been work on generating high-quality questions (Yuan et al., 2017;Serban et al., 2016;Labutov et al., 2015), but not how to best use them to train a model.

we use the two-stage SynNet to generate data tuples to directly boost performance of a model on a domain with no annotations.


We then transfer the MC model originally learned on the source domain to the target domain t using SGD on the synthetic data.

since the synthetic data is usually noisy, we alternatively train the MC model with mini-batches from x s and x t , which we call data-regularization.


An exception is the Dolphin18K dataset (Huang et al., 2016) which contains 18,000+ problems.

this dataset has not been made publicly available so far.


Ideally, we would generate all the paraphrases of q.

since this set could quickly become intractable, we restrict the number of candidate paraphrases to a manageable size.


These are then used by the other layers of the network to measure Q/AP relatedness.

convolution Tree Kernels (cTK) can be applied to relational structures built on top of syntactic/semantic structures derived from Q/AP text (Tymoshenko et al., 2016a).


Additionally, to have more reliable results, it is standard practice to apply n-fold cross-validation.

we cannot do this on the training (TRAIN) sets, since the embeddings learned in Sec.


It is possible to argue that part of the improvements of refinement models over STAGG in Table 3 may be due to model ensembling.

the performance gap between QUESREV and the alternative solutions enables us to isolate this effect for query revision approach.


As previously discussed, we can define the relevancy of a question in terms of the validity of its premises for an image, so we extract premises from each question Q and must find a suitable irrelevant image I − .

there are certainly many images for which one or more of Q's premises are false and an important design decision is then how to select I − from this set.


This reduces the size of the caption training set from 82,783 images to 70,194 images.

the complete caption training set is tokenized as a bag of words per image, and made available as image tag training data.


In addition, they have distinct visual patterns which are robust for current vision systems to recognize.

activity attributes are more difficult to conceptualize as they involve varying levels of abstractness, which are also more challenging for computer vision as they have less distinct visual patterns.


In addition to the attributes presented here, we also crowdsourced attributes for the emotion content of each verb (e.g., happiness, sadness, anger, and surprise).

we found these annotations to be skewed towards "no emotion", since most verbs do not strongly associate with a specific emotion.


From the numbers given in Section 3, variation in the number of chats in NALCS was much higher than LMS, which one may expect to have a critical effect in the language model.

our results seem to suggest that the L-Char-LSTM model can pickup other factors of the chat data (e.g.


Current reinforcement-based text generation works use traditional phrase-matching metrics (e.g., CIDEr, BLEU) as their reward function.

these metrics use undirected ngram matching of the machine-generated caption with the ground-truth caption, and hence fail to capture its directed logical correctness.


VerbNet (Kipper et al., 2008), the largest English verb argument structure resource, 1 organizes verbs and classes into a shallow hierarchy, but its structure has been handcrafted incrementally over time (starting with seminal work by Levin, 1993).

recently-developed, state-of-the-art machine learning methods offer a unique alternative approach to constructing such a hierarchy.


These results provide additional psychological evidence for the effects associated with Verb-Net's coarse distinctions: for unattested verbframe pairs, participants tend to assign a higher compatibility rating when the verb has sibling VerbNet classes that can take the frame.

the range of compatibility judgments is highly variable across all three categories, and BHC's finer-grained predictions fail to account for much of this variability.


In this example, the visual term "having fun" is also generated by the baseline NMT model, making it clear that at times what seems like a translation extracted exclusively from the image may have been learnt from the training text data.

none of the two baselines translated "Mexikanischen Setting" as "Mexican restaurant", but four out of the five multi-modal models did.


The summation expresses the goal of learning a behavior parame-terized by natural language instructions.

contextual Bandit Setting to most policy gradient approaches, we apply the objective to a contextual bandit setting where immediate reward is optimized rather than total expected reward.


While a simpler setup, for example decomposing the problem to source and target prediction and using a planner, is likely to perform better, we aim to minimize task-specific assumptions and engineering of separate modules.

to better understand the problem, we also report results for the decomposed task with a planner.


Persons with MCI show symptoms across several cognitive domains, where global cognitive ability, episodic memory, perceptual speed, and executive functioning are most clearly affected.

the performance of persons with MCI over-lap greatly with the performance of healthy controls, which highlights the complexity and heterogeneity of the diagnosis (Bäckman et al., 2005).


Clearly, we expect these new features to be somewhat correlated with each other, since function words tend also be high-frequency words.

many content words are also labeled as high-frequency in our methodology, such as bil (English: car) and potatis (English: potato).


In many systems, vague or none is also included as another relation type when a TLINK is not clear or missing.

current systems usually use a reduced set of relation types, mainly due to the following reasons.


We could of course sidestep the problem by exhaustively annotating the entire document corpus, by annotating all mentions of entities and checking relations between all pairs of mentions.

that would be a laborious and prohibitively expensive task: using the interfaces we've developed (Section 6), it costs about $15 to annotate a single document by non-expert crowdworkers, resulting in an estimated cost of at least $1,350,000 for a reasonably large corpus of 90,000 documents (Dang, 2016).


Even with large beam sizes, the locally normalized model underperforms these approaches.

by increasing model flexibility and performing search during training, the globally normalized model is able to recover from search errors and achieve much of the benefits of scoring all possible spans.


Second, its sequence-to-sequence neural architecture allows it to handle either one-hot symbolic input or dense vectors of acoustic features.

existing models are typically designed for "clean" symbolic input, then retrofitted with additional mechanisms to cope with acoustics.


Sparse text-based representations with linear context (DISTRIB and DEPS) significantly outperform some dense semantic representations.

no dense semantic models significantly outperform DISTRIB and DEPS.


Most of the above studies focus on synchronous multi-modal content, i.e., in which images are paired with text descriptions and videos are paired with subtitles.

we perform summarization from asynchronous (i.e., there is no given description for images and no subtitle for videos) multi-modal information about news topics, including multiple documents, images and videos, to generate a fixed length textual summary.


Specifically, when speech transcriptions are not considered, the informativeness of the summary is the worst.

adding speech transcriptions without guidance strategies decreases readability to a large extent, which indicates that guidance strategies are necessary for MMS.


The utterance "This movie is sick" can be ambiguous (either positive or negative) by itself, but if the speaker is also smiling at the same time, then it will be perceived as positive.

the same utterance with a frown would be perceived negatively.


When annotators are asked for objective judgments about a text (e.g., POS tags), the broader context in which the text is situated is often irrelevant.

many NLP tasks focus on inference of factors beyond words and syntax.


We ask an-notators to determine whether a given Twitter user supports Donald Trump or Hillary Clinton.

inferring something about a user from a single tweet that she writes may prove difficult.


2010have been proposed (Guan et al., 2017;Tian and Zhu, 2012;Wauthier and Jordan, 2011;Passonneau and Carpenter, 2014).

our work is most similar to efforts outside the domain of NLP, where Dai et al.


While some annotation tasks only require information from short texts, in many others, we can elicit higher-quality labels by providing annotators with additional contextual information.

asking annotators to consider too much information would make their task slow and burdensome.


According to the Law of Innovation, polysemy was claimed to correlate positively with meaning change.

our analysis showed that polysemy is highly collinear with frequency, and as such, did not demonstrate independent contribution to semantic change.


Including user factors as direct features is beneficial when there is a linear relationship with the class label, such as with gender and sarcasm use.

user-factor adaptation can capture more complex relationships between user groups and their language expression.


Modeling users has a long history of successful applications in providing personalized information access (Dou et al., 2007;Teevan et al., 2005) and recommendations (Guy et al., 2009;Li et al., 2010;Morales et al., 2012).

this work models users to better understand their content via language processing tasks following ideas from demographics-aware and domain adaptation.


For example, in the sentence: (a) The two justices have been attending federalist society events for years, our model correctly disambiguated justices with the WordNet sense justice 3 n (public official) rather than justice 1 n (the quality of being just), and the corresponding softmax distribution was heavily biased towards words and senses related to persons or groups (commissioners, defendants, jury, cabinet, directors).

in the sentence: (b) Xavi Hernandez, the player of Barcelona, has 106 matches, the same model disambiguated matches with the wrong WordNet sense match 1 n (tool for starting a fire).


Search systems are often focused on providing relevant results for the "now", assuming both corpora and user needs that focus on the present.

many corpora today reflect significant longitudinal collections ranging from 20 years of the Web to hundreds of years of digitized newspapers and books.


In some sense, one can see the relation -both are war movies, and Lucas and Spielberg have worked together.

inglourious Basterds was created at 2009 and was directed by Quentin Tarantino.


From this point, we propose a novel inter-weighted layer to measure the importance of each word.

the more similar two sentences are, the more probably we can align each word of sentence S with several words of sentence T , and vice versa.


As sentence length grows, RNN will suffer from gradient vanishing problem.

gated mechanism, such as Long Short Term Memory(LSTM) (Hochreiter and Schmidhuber, 1997) is introduced to address it.


The PWIM  is still competitive on WikiQA but gets an inferior performance on TrecQA.

our models both have state-of-the-art performances on three datasets which demonstrates our models have excellent generalization ability in different datasets.


Recently, several methods have been developed to induce taxonomies from text corpora (Wu et al., 2012;Yang, 2012;Luu et al., 2014).

this task is far from being solved for three reasons: i) Text corpora may vary in size, topic and quality.


(2014); Wang and He (2016), they suffer from extremely low recall for the Chinese language.

distributional approaches use word representations derived from contexts, independent of its hyponym or hypernym.


The ambiguity issue has been addressed in a few systems (Anke et al., 2016b;Wu et al., 2012;Ponzetto and Navigli, 2009) by word sense disambiguation.


TAXI crawls a domainspecific corpora and employs lexico-syntactic patterns and substrings for domain is-a relation extraction.

the potential of distributional methods is not fully exploited as only one system uses such techniques.


The form of compositional function involves many kinds of neural networks, such as recurrent neural networks (Hochreiter and Schmidhuber, 1997;Chung et al., 2014), convolutional neural networks (Collobert et al., 2011;Kalchbrenner et al., 2014), and recursive neural networks (Socher et al., 2013;Tai et al., 2015;Zhu et al., 2015).

these methods show an obvious defect in representing idiomatic phrases, whose semantics are not literal compositions of the individual words.


Due to its importance, some previous work focuses on automatic identification of idioms (Katz and Giesbrecht, 2006;Li and Sporleder, 2009;Fazly et al., 2009;Peng et al., 2014;Salton et al., 2016).

challenge remains to take idioms into account to improve neural based semantic representations of phrases or sentences.


Differing from previous work, which evaluating idiom detection as a standalone task, we want to integrate idiom understanding into sentiment classification task.

most of existing sentiment datasets do not cover enough idioms or related linguistic phenomenon.


The result is a lexicon where each entry contains a logical form template and a set of possible phrases for triggering the template.

we have avoided binding grammar rules to particular phrases in order to handle lexical variations.


These generative approaches aim to solve the modeling problem of assigning higher probability mass to outputs that use reoccurring parts.

our learning algorithm uses caching as a way to constrain the search space for computational efficiency; the probabilities of the candidate outputs are assigned by a separate discriminative model.


This is indeed a pretty simple trick, and per-feature standardization (with zero mean) is also a standard data preprocessing method.

it is not self-evident that this kind of standardization shall be applied on pre-trained word embeddings before using them in deep neural networks, especially with the obvious downside of rendering the word embedding algorithm's loss function sub-optimal.


For example, since we generalize all the predicates to a compact form <pred-xx>, for irregular verbs like "became" ⇒ become-01, simply stemming the inflected verb form will not give us the correct concept even if the sense is predicted correctly.

since CAMR uses the alignment table to store all possible concept candidates for a word, adding our predicated label as a feature could potentially help the parser to choose the correct concept.


to existing pipeline methods which first consider individual candidate answers separately and then make a prediction based on a threshold, we propose an end-to-end deep neural network framework, which is trained by a novel group-level objective function that directly optimizes the answer triggering performance.


An alternative pipeline approach is to first solve P 2 and then P 1 , i.e., first determine whether there's a correct answer in the candidate set and then rank all candidates to find a correct one.

as we will show using state-of-theart Multiple Instance Learning (MIL) algorithms in Section 4, P 2 by itself is currently a very challenging task, partly because of the difficulty of extracting features from a set of candidate answers that are effective for answer triggering.


In such cases, perhaps a multi-word embedding like Doc2Vec (Le and Mikolov, 2014) would be more appropriate.

if this were indeed the issue, one would expect vector offsets to perform equally poorly.


Given the asymmetric nature of word associations, KL-divergence seemed to be a natural fit.

it is vastly outperformed by even cosine similarity on the same set of embeddings.


Therefore, in order to learn the relationship representation between two arguments, we propose an attention mechanism that can select out the most important part from two arguments and perform the information interaction between two arguments.

one common issue involved in implicit discourse relationship identification is the lack of labeled data.


Therefore, the proper representations of ZPs are required so as to take advantage of semantic information when resolving ZPs.

representing ZPs is challenging because they are merely gaps that convey no actual content.


Meanwhile, considering that the antecedents of a ZP provide the necessary information for interpreting the gap (ZP), it is a natural way to express a ZP by its potential antecedents.

only some subsets of candidate antecedents are needed to represent a ZP 1 .


The semantic nature of discourse relations makes discourse parsing a difficult task.

the recent introduction of distributed representations of discourse units has seemingly led to significant improvements, with a claimed relative error reduction of 51% on fully labelled structures.


This avoids introducing discourse units that were not part of the original annotation, which a preliminary binarization of trees would have induced.

rST-Parseval considers approximately twice as many nodes as the original Parseval would on binarized trees (at most 2n − 2 nodes for n EDUs, compared to n − 1 attachments in a binary tree), and the relation labels of most nuclei are redundant with the nuclearity of a node and its sister (SPAN for a nucleus whose sisters are satellites, and the same label as its sisters for a nucleus whose sisters are nuclei).


Both aspects artificially raise the level of agreement between RST trees, especially when using manual EDU segmentation.

all the parsers in our sample except (Sagae, 2009;Heilman and Sagae, 2015) predict binary trees over manually segmented EDUs and evaluate them against right-heavy binarized reference trees.


Since they are trained on the same gold-standard data, one would expect RNN-GOLD to perform similarly to MAXENT.

in the case of the RNN-gold, the 50 tokens window may actually not have enough words to be filled with, because the gold-standard data is composed of the sentence with the it-pronoun and the three previous sentences, which in addition tend to be short.


Selectional preferences have long been claimed to be essential for coreference resolution.

they are mainly modeled only implicitly by current coreference resolvers.


Work on the automatic acquisition of selectional preferences has shown considerable progress (Dagan and Itai, 1990;Resnik, 1993;Agirre and Martinez, 2001;Pantel et al., 2007;Erk, 2007;Ritter et al., 2010;Van de Cruys, 2014).

today's coreference resolvers (Martschat and Strube, 2015;Wiseman et al., 2016;Clark and Manning, 2016a, i.a.)


captain and pilot, as well as ship and airplane have high paradigmatic similarity, i.e., they are seman-tically similar and occur in similar contexts.


In this way, the coreference resolver could learn to use selectional preferences mainly for mentions that are more likely to be anaphoric.

given that the F 1 score of current anaphoricity determiners or singleton detectors is only around 85 percent (Moosavi and Strube, 2016a, 2017), the effect of using system anaphoricity scores might be small.


RAPIER (Califf and Mooney, 2003) constraints are similar to our attributes, but are basic (surface form, POS tag, and hypernyms only), and expanding them will exponentially increase its complexity.

adding attributes to GrASP only increment runtime linearly (see Section 3.2).


Error Analysis The classifier has a small tendency towards labeling sentences with the majority class other.

sampling the training set yielded worse results for all classes.


To be more specific, in terms of accuracy, precision, recall, F1 and AUC, the average improvement for the baseline features are 4.33%, 10.30%, 4.32%, 11.01% and 10.40%, respectively.

we observe that the precision of U-GR+AF, although gives the second highest score among all feature combinations, is lower than that of UGR; we leave it for future work.


Up to this point, we focused on the task of extracting links between ACs.

recent work has shown that joint models that simultaneously try to complete multiple aspects of the subtask pipeline outperform models that focus on a single subtask (Persing and Ng, 2016;Stab and Gurevych, 2014b;Peldszus and Stede, 2015).


* indicates that this bin does not contain any major claim labels, and this average only applies to claim and premise classes.

we do not disable the model from predicting this class: the model was able to avoid predicting this class on its own.


One thing to notice here is that lattice nodes can have multiple predecessor states.

hidden states in LSTMs and other sequential RNNs are conditioned on only one predecessor state (h j in left column of Table 1), rendering standard RNNs unsuitable for the modeling of lattices.


This is not surprising, given that the lattice training data includes lattices of varying density, including lattices with very few paths or even only one path.

without fine-tuning on lattices, using lattices as input performs poorly (lattice/R and lattice/R+1).


More seriously, because the translation function is smooth, infrequent pairs tend to be wrongly seen as noise in the training process and so are largely ignored by the model.

to this, the conventional SMT approach is based on statistics of words and/or phrases, which, in principle, is a symbolic method that uses a discrete model and involves little parameter sharing.


This idea has been adopted in early research into neural-based MT methods, where neural models were utilized to improve SMT performance (Zhang et al., 2015).

this seems to be counterintuitive, as intuitively learning general rules should be the first step, rather than first memorizing special cases and then learning general rules.


Secondly, the results show that with both datasets, the lexical approach (NMT-L) can improve NMT performance, showing that using SMT knowledge helps NMT.

the improvement seems less significant than reported in (Arthur et al., 2016).


Gradual fine-tuning The second dynamic data selection technique, see Figure 1b, is inspired by the success of domain-specific fine-tuning (Luong and Manning, 2015;Zoph et al., 2016;Sennrich et al., 2016a;Freitag and Al-Onaizan, 2016), in which a model trained on a large general-domain bitext is trained for a few additional epochs only on small in-domain data.

rather than training a full model on the complete bitext G, we gradually decrease the training data size, starting from G and keeping only the top n sentence pairs for the duration of η epochs, where the top n pairs are defined by their CED s scores.


The results in Table 4 show that the bitext ranking plays a crucial role in the success of data selection.

the results also show that even in the absence of an appropriate bitext ranking, dynamic data selection-and in particular gradual fine-tuning-is still superior to static data selection.


We explain this result as follows: Compared to static selection, both sampling and gradual fine-tuning have better coverage due to their improved exploration of the data.

sampling also suffers from a surprise effect of observing new data in every epoch.


Overall, the fluency of the MT output improves when NMT is used, and the number of lexical choice errors is also reduced.

state-of-the-art NMT approaches based on an encoder-decoder architecture with an attention mechanism as introduced by  exhibit weaknesses that sometimes lead to MT errors which a phrase-based MT system does not make.


In (Tang et al., 2016), the NMT decoder is modified to switch between using externally de-fined phrases and standard NMT word hypotheses.

only one target phrase per source phrase is considered, and the reported improvements are significant only when manually selected phrase pairs (mostly for rare named entities) are used.


When we use the LM trained on extra monolingual data, we get total improvements of 1.0% BLEU and 2.3% BLEU with the hybrid approach.

when we add this language model and a word penalty on top of the pure NMT system and tune scaling factors with MERT, we get small improvements (last row of Table 2) only on product descriptions.


Phrases play an important role in natural language understanding and machine translation (Sag et al., 2002; Villavicencio et al., 2005).

it is difficult to integrate them into current neural machine translation (NMT) which reads and generates sentences word by word.


Machine translation (MT) models typically require large, sentence-aligned bilingual texts to learn good translation models (Wu et al., 2016;Sennrich et al., 2016a;Koehn et al., 2003).

for many language pairs, such parallel texts may only be available in limited quantities, which is problematic.


BPR+MNN uses the same MNN seed translations as MNN, obtained from unannotated monolingual data of English and the foreign language, to learn the linear mapping between their embedding spaces.

unlike MNN, BPR+MNN uses the mapped word vectors to predict ranking in a supervised manner with BPR objective.


Instance weighting has been widely applied to phrase-based machine translation domain adaptation.

it is challenging to be applied to Neural Machine Translation (NMT) directly, because NMT is not a linear model.


In recent years, neural encoder-decoder models (Kalchbrenner and Blunsom, 2013; have significantly advanced the state of the art in NMT, and now consistently outperform Statistical Machine Translation (SMT) (Bojar et al., 2016).

their success hinges on the availability of sufficient amounts of parallel data, and contrary to the long line of research in SMT, there has only been a limited amount of work on how to effectively and efficiently make use of monolingual data which is typically amply available.


One downside of this approach is the significantly increased training time, due to training of a model in the reverse direction and translation of monolingual data.

we propose to train NMT models from scratch on both bilingual and target-side monolingual data in a multi-task setting.


Interestingly, the improvements from monolingual data are additive to the gains from ensembling of 3 models with different random seeds.

the use of synthetic parallel data still outperforms our approach both in single and ensemble systems.


While separating out a language model allowed us to carry out multi-task training on mixed data types, it constrains gradients from monolingual data examples to a subset of source-independent network parameters (σ).

synthetic data always affects all network parameters (θ) and has a positive effect despite source sequences being noisy.


With this additional gating mechanism, the final syntactic GCN computation is formulated as The inability of GCNs to capture dependencies between nodes far away from each other in the graph may seem like a serious problem, especially in the context of SRL: paths between predicates and arguments often include many dependency arcs (Roth and Lapata, 2016).

when graph convolution is performed on top of LSTM states (i.e., LSTM states serve as input to GCN) rather than static word embeddings, GCN may not need to capture more than a couple of hops.


The link embedding i for the ith question token is computed as: e ∈Eτ ∪{∅} exp s(e , i) For WIKITABLEQUESTIONS, we ran the entity embedding and linking module over every entity.

this approach may be prohibitively expensive in applications with a very large number of entities.


Dynamic programming on denotations (DPD) is an automatic procedure for enumerating logical forms that execute to produce a particular value; it leverages the observation that there are fewer denotations than logical forms to enumerate this set relatively effi-ciently.

many of these logical forms are spurious, in the sense that they do not represent the question's meaning.


(2015) explores visual concept learning from few examples, and presents encouraging results for one-shot learning by learning representations over Bayesian programs.

none of these address the issue of learning from natural language.


the probability of the latent statement evaluation values z can be parametrized using a probabilistic semantic parsing model (with associated parameters θ p ).


If the length of d is 1, all the weights in W d have value 1, and γ is a linear activation, then this formula is equivalent to a regular cosine similarity.

we use a larger length for d to capture more features, use tanh as the activation function, and optimise the weights of W d during training, giving the framework more flexibility to customise the model for the task of metaphor detection.


Techniques have been developed to categorize questions based on the nature of these information needs in the context of the TREC QA challenge (Harabagiu et al., 2000), and to identify questions asking for similar information (Shtok et al., 2012;Zhang et al., 2017;Jeon et al., 2005); questions have also been classified by topic (Cao et al., 2010) and quality (Treude et al., 2011;Ravi et al., 2014).

our work is not concerned with the information need central to QA applications, and instead focuses on the rhetorical aspect of questions.


Types 6 and 7 are much more combative: in type 6 questions the asker explicitly attempts to force the minister to concede/accept a point that would undermine some government stance, while type 7 contains condemnatory questions that prompt the minister to justify a policy that is self-evidently bad in the eyes of the asker.

type 2 constitutes tamer narrow queries that require the minister to simply report on non-partisan matters of policy.


11 In the unanswerable and unanswered tasks, we find that the BOW features do not perform significantly better than a random (50%) baseline.

the latent question features produced by our framework bring additional predictive signal and outperform the baseline when combined with BOW (binomial p < 0.05), achieving accuracies of 66% and 62% respectively (compared with 55% and 50% for BOW alone).


These methods typically require historical data for fitting model parameters, and may be sensitive to issues such as concept drift (Fung, 2014).

our approach does not rely on historical data for training; instead we forecast outcomes of future events by directly extracting users' explicit predictions from text.


With the increasing number of hops, the performance improves.

when the number of hops is larger than 3, the performance decreases due to overfitting.


training on multiple datasets has shown promising results (Collobert and Weston, 2008).

multitask learning requires access to the emoji dataset whenever the classifier needs to be tuned for a new target task.


Note that word coverage can be a misleading metric in this context as for many of these small datasets a word will often occur only once in the training set.

all of the words in the pretraining vocabulary are present in thousands (if not millions) of observations in the emoji pretraining dataset thus making it possible for the model to learn a good representation of the emotional and semantic meaning.


A naive model of yielding v C could utilise the attention mechanism over h t , deriving a weighted sum according to user information.

dynamic memory networks have been shown highly useful for deriving abstract semantic information compared with simple attention, and hence we follow Sukhbaatar et al.


A straightforward approach to predicting the rating score of a product is to take the average of existing review scores.

the drawback is that it cannot reflect the the variance in user tastes.


In order to integrate user preferences into the rating, we instead take a user-based weighted average of existing rating scores, so that the scores of reviews that are closer to the user preference are given higher weights.

existing ratings can be all different from a users personal rating, if the existing reviews do not come from the user's neighbours.


Constraint tightening: In subproblem P 2 , we consider a vertex and all of its adjacent arcs of positive weight.

we know that our optimal solution must satisfy tree-shape constraints (5).


Pop (2002) also used Lagrangian relaxation -in the non directed case -where a single subproblem is solved in polynomial time.

the relaxed constraints are inequalities: if the dual objective returns a valid primal solution, it is not a sufficient condition in order to guarantee that  Table 3: Dependency parsing and tagging results.


Theoretically, the refinement can be made until there is no update in the scoring matrix.

experimental results show that comparable performance can be achieved with no more than twice high-order refinements (see Section 3).


Their greedy algorithm breaks the parsing into a sequence of local steps, which correspond to choosing the head for each modifier word (one arc at a time) in the bottom-up order relative to the current tree.

we employed the global inference algorithm to change the entire tree (all at a time) in each refinement step, which makes the improvement more efficient.


We believe this is because unsupervised parsers have relatively low accuracy and forcing them to reconcile would not lead to better parses.

joint decoding during training helps propagate useful inductive biases between models and thus leads to better trained models.


their published hyperparameters and preprocessing.

rather than selecting the final model based on reranking performance, we instead perform early stopping based on development set perplexity.


Parameterized by the encoding parameters Λ and the reconstruction parameters Θ, our NCRF-AE consists of the encoder and the decoder, which together forms the log-likelihood a highly non-convex function.

a careful observation shows that if we fix the encoder, the lower bound derived in the E step, is convex with respect to the reconstruction parameters Θ in the M step.


This system will fail to capture non-projective TAG derivation structures.

as noted in Section 2, there is almost no non-projectivity in TAG derivation structures of English.


WSJ Section 00 does not have any non-projective sentences.

wSJ Sections 01-22 contain 0.6% of non-projective sentences in dependency grammar (Chen and Manning, 2014), an order of magnitude more than non-projectivity for TAG.


Setup 3 considers the most query  Figure 3: Entity-relation table entity pairs in total since multi-token entities are split into their comprising tokens.

setup 3 represents a more realistic scenario than setup 1 or setup 2 because in most cases, entity boundaries are not given.


6 Our results are best comparable with  since we use the same setup and traintest splits.

their model is more complicated with a lot of hand-crafted features and various iterations of modeling dependencies among entity and relation classes.


However, their model is more complicated with a lot of hand-crafted features and various iterations of modeling dependencies among entity and relation classes.

we only use pre-trained word embeddings and train our model end-to-end with only one iteration per entity pair.


This demonstrates the strength of neural representation learning for end-to-end relation extraction.

miwa and Bansal (2016)'s model is trained locally, without considering structural correspondences between incremental decisions.


First, Miwa and Bansal (2016) rely on external syntactic parsers for obtaining syntactic information, which is crucial for relation extraction (Culotta and Sorensen, 2004;Zhou et al., 2005;Bunescu and Mooney, 2005;Qian et al., 2008).

parsing errors can lead to encoding inaccuracies of tree-LSTMs, thereby hurting relation extraction potentially.


Our method can avoid the problem since we do not compute parser outputs.

the computation complexity is largely reduced by using our method since sequential LSTMs are based on inputs only, while the dependency path LSTMs should be computed based on the dynamic entity detection outputs.


Embedding methods have shown state-ofthe-art results on several benchmark datasets.

by construction, these benchmark datasets differ from data in real KGs.


First, benchmark datasets have largely been restricted to the most frequently occurring entities in the KG.

in most KGs, entities are associated with a sparse set of observations.


Through sampling, FB15K rebalances Freebase, increasing the diversity of entities and relations.

wordNet and wN18 have similar diversity statistics.


A sparse, high-quality set of extractions may be insufficient to learn meaningful embeddings.

the benefit of incorporating additional, unreliable facts may also be questionable.


Unspecialized word embeddings (Mikolov et al., 2013;Pennington et al., 2014) capture general semantic properties of words, but are unable to differentiate between different types of semantic relations (e.g., vectors of car and driver might be as similar as vectors of car and vehicle).

we often need embeddings to be similar only if an exact lexico-semantic relation holds between the words.


Distributional and path-based models have been used to discriminate between multiple lexicosemantic relations, including hypernymy and meronymy, at once .

as pointed out by (Chersoni et al., 2016), distributional vectors and scores based on their comparison fail to discriminate between multiple relation types at once.


(2015) propose dataset splits with no lexical overlap between the train and test portions.

model's performance in a lexically-split setting is an overly pessimistic estimate of models' true performance -in a realistic scenario, the model will occasionally make predictions for pairs involving some of the concepts from the training set.


Although existing RE systems have achieved promising results with the help of distant supervision and neural models, they still suffer from a major drawback: the models only learn from those sentences contain both two target entities.

those sentences containing only one of the entities could also provide useful information and help build inference chains.


(Zeng et al., 2014;dos Santos et al., 2015) adopt CNN in this task, and (Zeng et al., 2015;Lin et al., 2016) combine attention-based multiinstance learning which shows promising results.

these above models merely learn from those sentences which directly contain both two target entities.


Hence we only apply dropout to s i for PCNN.

even with a dropout rate of 0.5, RNN still performs well.


(2010) have applied it to create relation extraction datasets for a large-scale KB.

to our dataset, their data contains a single relation instance per sentence.


Much effort has been made in reducing the influence of noisy sentences within the bag, including methods based on at-least-one assumption (Hoffmann et al., 2011;Ritter et al., 2013;Zeng et al., 2015) and attention mechanisms over instances (Lin et al., 2016;Ji et al., 2017).

the sentence level denoise methods can't fully address the wrong labeling problem largely because they use a hard-label method in which the labels of entity pairs are immutable dur-ing training, no matter whether they are correct or not.


Wrong corrections like Case 1 fail to distinguish similar relations (both Nationality and place of birth are relations between people and locations) between entities because of their similar sentence patterns.

wrong corrections like Case 1 are rare (5/39) in our experiments.


ResNet has won the ImageNet ILSVRC 2015 classification task, and achieved state-of-theart performances in many computer vision tasks.

the effect of residual learning on noisy natural language processing tasks is still not well understood.


Empirically, we evaluate on the NYT-Freebase dataset (Riedel et al., 2010), and demonstrate the state-of-the-art performance using deep CNNs with identify mapping and shortcuts.

to popular beliefs in vision that deep residual network only works for very deep CNNs, we show that even with a moderately deep CNNs, there are substantial improvements over vanilla CNNs for relation extraction.


Some relation types are more lexical by nature: relations such as dog is an animal; a teacher works at a school; a car is kept at a parking lot, can be identified out of context.

many relations are contextual; they are time-anchored or tied to extra-linguistic, situational context.


The input-output combination method still has an advantage, and concatenation and multiplication also perform well.

the advantages over the baseline are less significant than when the number of clusters was identical to the standard.


Our measurement approach can also be defined as 'local' to some extent: the linear projections that we learn are mostly based and evaluated on the nearest neighborhood data.

this method is different in that its scope is not single words but pairs of typed entities ('location' and 'armed group' in our case) and the semantic relations between them.


The cumulative baseline results are slightly better, probably simply because they are trained on more data.

they still perform much worse than the models trained using incremental updates.


If a token appears in the ingredients, we set z v = 1 and z v = 0 otherwise.

we can train the model in a fully supervised fashion, i.e., we can obtain the probability of it may be not be accurate.


Recurrent Neural Network (RNN) language models recently outperformed traditional n-gram LMs across a range of tasks (Jozefowicz et al., 2016).

an important practical issue associated with such neural-network LMs is the high computational cost incurred.


The NEG objective function is considered a simplification of the NCE's objective, unsuitable for learning language models (Dyer, 2014).

in this study, we show that despite its simplicity, it can be used in a principled way to effectively train a language model, based on PMI matrix factorization.


Much research has been done on subword-level and subword-aware 1 neural language modeling when subwords are characters (Ling et al., 2015b;Kim et al., 2016;Verwimp et al., 2017) or morphemes (Botha and Blunsom, 2014;Qiu et al., 2014;Cotterell and Schütze, 2015).

not much work has been done on syllable-level or syllable-aware NLM.


These models can be spiced up with the most recent regularization techniques for RNNs (Gal and Ghahramani, 2016) to reach state-of-the-art.

to make our results directly comparable to those of Kim et al.


It seems that syllable-aware language models fail to outperform competitive character-aware ones.

usage of syllabification can reduce the total number of parameters and increase the training speed, albeit at the expense of languagedependent preprocessing.


The core of our model closely corresponds to the structure of the RMN (as technically described in Section 3.2).

we provide the model with distinct types of informative input for each view, and, reformulate the objective in a way that jointly optimizes parameterizations of all views to encode distinct information (cf., Section 3.3).


We therefore restrict our quantitative analysis to the Amazon data set.

we also report qualitative results on the Gutenberg corpus, demonstrating that our model induces meaningful novel representations for corpora of varying size and diversity.


Publicly-available labeled datasets are scarce for many NLP tasks, and crowdsourcing services such as Amazon Mechanical Turk 1 (AMT) offer researchers a quick, inexpensive means of labeling their data.

workers employed by these services are typically unfamiliar with the annotation tasks, and they may have little motivation to perform high-quality work due to factors such as low pay and anonymity.


To illustrate such a case, consider the example in Figure 3: In our study, all 5 workers incorrectly selected yes as answer.

(perhaps somewhat counterintuitively to nonexperts) under the PropBank paradigm it is the "phone representative" that provide explicit help in this sentence, not "Vanguard."


In particular, even at similar levels of expert involvement, it outperforms the HYBRID all5 approach.

we also note that with an F 1 -score of 0.94, our approach does not yet reach the quality of gold annotated data.


We focused on maximizing the accuracy of dependency parsing on the development data in our experiments.

the sizes of the training data are different across the different tasks; for example, the semantic tasks include only 4,500 sentence pairs, and the dependency parsing dataset includes 39,832 sentences with word-level annotations.


Task-oriented learning of low-level tasks Each task in our JMT model is supervised by its corresponding dataset.

it would be possible to learn low-level tasks by optimizing highlevel tasks, because the model parameters of the low-level tasks can be directly modified by learning the high-level tasks.


At first sight, this task appears formidable, as it would imply that a bilingual semantic space can be constructed by using monolingual corpora only.

the existence of structural isomorphism across monolingual embedding spaces points to the feasibility of this task: The transformation exists right there only to be discovered by the right tool.


For example, we choose c = L 2 2 for the EMDOT approach to obtain a closed-form solution to the subprogram (10), otherwise we would have to use gradient-based solvers.

the WGAN approach calls for c = L 2 because the Kantorovich-Rubinstein duality takes a simple form only in this case.


Similar to ours, the underlying idea is to match cross-lingually at the level of distribution rather than word.

the distributions considered in that work are the hidden states of neural embedding models during the course of training.


Each decoding step in the unfolded network computes a single attention weight vector.

ensemble decoding would compute one attention weight vector for each of the K input models.


Srinivas and Babu (2015) propose to add the outgoing weights of j to the weights of a similar neuron i to compensate for the removal of j.

we have found that this approach does not work well on NMT networks.


0.8 BLEU better than the single system (a)).

using the data-bound version of our shrinking algorithm (Sec.


For example, unlike recursive neural networks (Socher et al., 2013), GCNs do not require the graphs to be trees.

in this work we solely focus on dependency syntax and leave more general investigation for future work.


(2016) use convolution in both the encoder and the decoder; they make use of dilation to increase the receptive field.

to both approaches, we use a GCN informed by dependency structure to increase it.


In simultaneous translation/interpretation, which has recently been studied in the context of neural machine translation (Gu et al., 2016), the decoding objective is formulated as a trade-off between the translation quality and delay.

when a machine translation system is used as a part of a larger information extraction system, it is more important to correctly translate named entities and events than to translate syntactic function words.


Psycholinguistic Features: Psychological differences are useful for our problem, because professional journalists tend to express opinion conservatively to avoid unnecessary arguments.

satirical news includes aggressive language for the entertainment purpose.


The task is mostly based on lexical cues and specific reporting verbs that are the signal for the majority of direct quotations.

in the case of quotation attribution the task is to find the source, cue, and content of the quotation, whereas in our case, for a given citing paragraph and reference we simply assess which text fragment is covered by the reference.


When comparing the sequence classifier CSP S to the plain classifier CSP C, we see a marginal difference of 1.3% for F1.

it will become more evident later that classifying jointly the text fragments for the different span buckets, outperforms the plain classification model.


CSP C achieves a similar performance in this case.

for the cases where the span is at the sub-sentence level or across multiple sentences, the performance of baselines drops drastically.


For larger spans, for instance, for (1, 2], we are still slightly better, with roughly 3% less erroneous span when comparing CSP C and CS.

only in the case of spans with > 2, we perform below the CS baseline.


Despite, the smaller erroneous span, the CS baseline never includes more than one sentence, and as such it does not include many erroneous spans for the larger buckets.

it is by definition unable to recognize any longer spans.


For instance, Daxenberger and Gurevych (2012) categorized edits based on whether edits affect the text meaning, resulting in syntactic edit categories such as file deletion, reference modification, etc.

simply understanding the syntactic revision operation types does not provide the information we seek: why do editors do what they do?


We found similar trends for interactions between previous quality and elaboration and verification, which are essential for articles in the starting stages.

the positive slopes for simplification, wikification and process suggest that, as articles increase in quality, simplifying articles' content, adding proper links or reorganizing their structure becomes more important.


In a semi-supervised setting, our method of reusing (V)RAE encodings in a reading comprehension framework is effective, with SWEAR-PR reaching an accuracy of 66.5 on 1% of the dataset against last year's state of the art of 71.8 using the full dataset.

these methods require careful configuration and tuning to succeed, and making them more robust presents an excellent opportunity for future work.


For language, the direct analogue would be to paraphrase the input (Madnani and Dorr, 2010).

high-precision paraphrase generation is challenging, as most edits to a sentence do actually change its meaning.


Standard evaluation is overly lenient on models that rely on superficial cues.

adversarial evaluation reveals that existing models are overly stable to perturbations that alter semantics.


In recent years, many methods have been proposed for commonsense machine comprehension.

these methods mostly either focus on matching explicit information in given texts (Weston et al., 2014;Wang and Jiang, 2016a,b;Zhao et al., 2017), or paid attention to one specific kind of commonsense knowledge, such as event temporal relation (Chambers and Jurafsky, 2008;Modi and Titov, 2014;Pichotta and Mooney, 2016b;Hu et al., 2017) and event causality (Do et al., 2011;Radinsky et al., 2012;Hashimoto et al., 2015;Gui et al., 2016).


To identify whether a hypothesis is reasonable, we need to consider all possible inferences.

in human reasoning process, not all inference rules have the same possibility to be applied, because the more reasonable inference will be proposed more likely.


Rosenthal and McKeown (2012) set out to explore this direction by conducting cross-domain experiments for detecting claims in blog articles from LiveJournal and discussions taken from Wikipedia.

they focused on relatively similar datasets that both stem from the social media domain and in addition annotated the datasets themselves, leading to an identical conceptualization of the notion of claim.


For the related important sentence detection task in text summarization,  therefore propose a two-stage approach (Lee and Liu, 2003;Elkan and Noto, 2008) to augment the set of known summaryworthy sentences.

we adopt a conservative approach rather than predict too many sentences as being question-worthy: we pair up source sentences with their corresponding questions, and use just these sentence-question pairs to training the encoder-decoder model.


In fact, the notion of "attention" has gained popularity recently in neural network modeling, which has improved the performance of many tasks such as machine translation (Bahdanau et al., 2015;Luong et al., 2015).

very few previous works employ attention mechanism to tackle MDS.


These features help them select better category-specific content for the summary.


However, few previous works employ attention mechanism to tackle the unsupervised MDS problem.

our attention-based framework can generate summaries for multi-document summarization  We propose a cascaded neural attention based unsupervised salience estimation method for compressive multi-document summarization.


Celikyilmaz and Hakkani-Tur (2010) estimated scores for sentences based on their latent characteristics using a hierarchical topic model, and trained a regression model to extract sentences.

they only use the latent topic information to conduct the sentence salience estimation for extractive summarization.


However, they only use the latent topic information to conduct the sentence salience estimation for extractive summarization.

our purpose is to model and learn the latent structure information from the target summaries and use it to enhance the performance of abstractive summarization.


For example, our result for S(1) "Wuhan wins men's soccer title at Chinese city games" matches the "Who Action What" structure.

the standard decoder StanD ignores the latent structures and generates some loose sentences, such as the results for S(1) "Results of men's volleyball at Chinese city games" does not catch the main points.


Some existing studies employ parallel corpora as artificial reference summaries (Woodsend and Lapata, 2010;Cheng and Lapata, 2016).

preparing such large volumes of reference summaries manually is sometimes costly.


With respect to the predictor component in the proposed model, we use an encoder-decoder architecture modeled by recurrent neural networks (Kim et al., 2016) based on recent neural extractive summarization approaches (Cheng and Lapata, 2016;Nallapati et al., 2017).

our summarization framework is applicable to all models of sentence extraction using distributed representation as inputs.


The proposed neural multi-task learning model, NN-ML, is significantly inferior to NN-SE and LREG.

nn-ML-CL, the proposed model with curriculum learning, is superior to all  other models.


This result shows that merely introducing multi-task learning does not positively influence on sentence extraction.

curriculum learning overcomes the difficulty of multitask learning; thus, document classification has positive effects on sentence extraction.


Table 5 shows that 85.7% of sentences extracted using NN-ML-CL correspond to changes of revenues and profits.

only 40.0% of sentences extracted by NN-SE correspond to these parameters, which indicates that document classification supports extraction of sentences related to the revenue and profit change, and contributes to the improvement.


NN-ML-CL extracted sentences that mention the good revenue performance of air-conditioners in Asia and Europe, which is the same as that in the gold summary.

nn-SE extracts sentences mentioning the bad revenue performance of fluorine resin and medical equipment, which are not described in the gold summary.


To overcome this, several studies have used artificial reference summaries (Sun et al., 2005;Svore et al., 2007;Woodsend and Lapata, 2010;Cheng and Lapata, 2016) compiled by collecting documents and corresponding highlights from other sources.

preparing such a parallel corpus often requires domain-specific or expert knowledge depending on the domain (Filippova et al., 2009;Parveen et al., 2016).


We first crawled 18121 English Wikinews and their source news articles via the associated URLs.

many Wikinews articles have very few source news articles and they are very short, and moreover, the URLs for many of the source news are out of date.


One common practice (Lee et al., 2012) to approach CD coreference task is to resolve event coreference in a megadocument created by concatenating topic-relevant documents, which essentially does not distinguish WD and CD event links.

intuitively, recognizing CD coreferent event pairs requires stricter evidence compared to WD event linking because it is riskier to link two event mentions from two distinct documents rather than the same document.


Like this work, several studies have considered both WD and CD event coreference resolution task together.

to simplify the problem, they (Lee et al., 2012;Harabagiu, 2010, 2014) created a meta-document by concatenating topic-relevant documents and treated both as an identical task.


However, only very small weights were assigned to the similarity and distance scores calculated using context embeddings.

in the classifier trained with cross-doc coreferent event mention pairs, the highest weight was assigned to the cosine similarity score calculated using context embeddings of two event mentions.


In neural text generation such as neural machine translation, summarization, and image captioning, beam search is widely used to improve the output text quality.

in the neural generation setting, hypotheses can finish in different steps, which makes it difficult to decide when to end beam search to ensure optimality.


We therefore propose a novel and simple beam search variant that will always return the optimalscore complete hypothesis (modulo beam size), and finish as soon as the optimality is established.

another well-known problem remains, that the generated sentences are often too short, compared to previous paradigms such as SMT (Shen et al., 2016).


Combining these two Theorems, it is interesting to note that our method is not just optimal but also faster.

optimal-score hypothesis, though satisfying in theory, is not ideal in practice, since neural models are notoriously bad in producing very short sentences, as opposed to older paradigms such as SMT (Shen et al., 2016).


In an extreme case, an agent may be micro-managed by a human user who uses the neural model to enforce grammar and style (e.g., a level of politeness, or a type of humor), while driving the content directly (e.g., by expressing In this manner, the neural model becomes an authoring tool, rather than an independent chat-bot.

in fully automated agent systems, the agent may be influenced by a knowledge database, or some other artificial information system, while running in a pre-set style or a style deemed best based on the course of the conversation.


Here, we follow a similar process as in (Wen et al., 2015) which generates multiple target hypotheses with stochastic sampling based on p(T |S), and then ranks them with the objective function 2 above.

as also observed by (Shao et al., 2017), step-by-step naive sampling can accumulate errors as the sequence gets longer.


This makes the model gradually adapt to the noisiness of the predicted label.

this method still relies upon a single current label, and, by omitting the distribution over the possible labels, this model loses information about the current stage.


However, this method still relies upon a single current label, and, by omitting the distribution over the possible labels, this model loses information about the current stage.

we propose to condition the next label on a predicted distribution of the current label.


(2016b) use keyword retrieval confidence as a reward.

it is widely acknowledged that manually defined reward functions can't possibly cover all crucial aspects and can lead to suboptimal generated utterances.


Training is formalized as a game in which the generative model is trained to generate outputs to fool the discriminator; the technique has been successfully applied to image generation.

to the best of our knowledge, this idea has not achieved comparable success in NLP.


This is similar to our work in that we also evaluate using the previous utterances, and the predicted DAs for them.

our modeling approaches are all based on DNNs, as de-scribed in more details in Section 3, and the interaction between utterances and DA labels is modeled in the hierarchical models in a more principled way.


(2011)'s model, individual reply structure paths from the first utterance to terminal utterances are teased apart into separate sequential conversations by duplicating utterances.

this method counts the same utterance multiple times and requires an aggregation method for making a final decision of the DA for each utterance.


The baseline models show a large variance in performance depending on the characteristics of the corpus.

our model has a low variance between the corpora, because the content word filtering, distinction between utterance-level and sentence-level DAs, and speaker preferences complement one another to adapt to different corpora.


One of the prevailing methods to build the system is using the generative Sequence-to-Sequence (Seq2Seq) model through neural networks.

the standard Seq2Seq model is prone to generate trivial responses.


(2017) developed several teaching strategies answering "how" the human teacher guide the learning process.

those previous teaching methods exclude "when" to teach from concern.


By investigating the real work mode in a call center, this framework makes a reasonable assumption that human teacher has access to the extracted dialogue states from the dialogue state tracker as well as the system's dialogue act, and can also reply in the same format.

there are two major problems in the previous framework.


In addition to early teaching, the teaching heuristics can be broadly divided into two categories: teacher-initiated heuristics and studentinitiated heuristics (Amir et al., 2016).

the teacher-initiated approaches require the constant long-term attention of the teacher to monitor the dialogue process (Torrey and Taylor, 2013;Amir et al., 2016), which is costly and impractical for real applications.


A straight way is to set a ratio threshold α, and consider it to be failure prognosis when T (s t ) < αR succ .

this assumes that the numerical scale of Q succ is consistent through the training period, which is not always the case.


if the success rate over a training process rises and falls and sometimes is below the threshold, it is risky.


In the seq2seq approach, the decoder network therefore only has to keep track of where it is in the output, and the content to generate can be transformed from the relevant parts in the source via the attention mechanism .

in conversation response generation, the prompt turn may be short and general (e.g., "what do you have planned tonight"), while an appropriate response may be long and informative.


This algorithm behaves similarly to standard beam search when the categorical distribution used during the process is sharp ('peaked'), since the samples are likely to be the top categories (words) .

when the distribution is smooth, many of the choices are likely.


In the case of reranking whole target sequences y, this becomes the marginal P (y), which corresponds to the same diversitypromoting objective used in (Li et al., 2015).

we found that our approximation works better in terms of N-choose-1 accuracy (see Section 5.2), which suggests that its value may be closer to the true conditional probability.


Our evaluation set is therefore not from the same distribution as our training set.

since our goal is to produce good general conversation responses, we found it to be a good general purpose evaluation set.


It is clear enough that this model progresses much slower, so we terminated it early.

it is surprising that the glimpse model with K = 10 progresses faster than the baseline model with only source-side attention, because the model is trained on examples with decoder-length fixed at 10, while the average response length is 38 in our data set.


There is a growing demand for intelligent personal assistants, mainly in the form of dialogue agents, that can help users accomplish tasks ranging from meeting scheduling to vacation planning.

most of the popular agents in today's market, such as Amazon Echo, Apple Siri, Google Home and Microsoft Cortana, can only handle very simple tasks, such as reporting weather and requesting songs.


A related but different extension to singledomain dialogues is multi-domain dialogues, where each domain is handled by a separate agent (Lison, 2011;Gasic et al., 2015a,b;Cuayáhuitl et al., 2016).

to compositedomain dialogues studied in this paper, a conversation in a multi-domain dialogue normally involves one domain, so completion of a task does not require solving sub-tasks in different domains.


We consider this paper to be the most complete study to date, across metrics, systems, datasets and domains, focusing on recent advances in data-driven NLG.

to previous work, we are the first to: • Target end-to-end data-driven NLG, where we compare 3 different approaches.


ness for the TGEN system on BAGEL (ρ = 0.33, p < 0.01, see Figure 1).

the wps metric (amongst most others) is not robust across systems and datasets: Its correlation on other datasets is very weak, (ρ ≤ .18) and its correlation with in-formativeness ratings of LOLS outputs is insignificant.


These results are quite promising, and suggest that neural models are a good fit for text generation.

the statistics of these datasets, shown in Table 1, indicate that these datasets use relatively simple language and record structure.


(2016) introduced the WIKIBIO dataset, which is at least an order of magnitude larger in terms of number of tokens and record types.

as shown in Table 1, this dataset too only contains short (single-sentence) generations, and relatively few records per generation.


(2015) to parameterize this probability; full details are given in the Appendix.

importantly, we note that the (s, y 1:T ) pairs typically used for training data-to-document systems are also sufficient for training the information extraction model presented above, since we can obtain (partial) supervision by simply checking whether a candidate record lexically matches a record in s. 1 since there may be multiple records r ∈ s with the same e and m but with different types r.t, we will not always be able to determine the type of a given entity-value pair found in the text.


(Hidayat, 2012) noted that in Facebook, users mostly preferred inter-sentential mixing and showed that 45% of the mixing originated from real lexical needs, 40% was used for conversations on a particular topic and the rest 5% for content clarification.

(Das and Gambäck, 2014) showed that in case of Facebook messages, intra-sentential mixing accounted for more than half of the cases while inter-sentential mixing accounted only for about one-third of the cases.


We also have a significant amount of unsupervised status update data (100k+ users).

the supervised datasets which have the SUD ground truth are pretty small, ranging from 896 for the intersection of the likes, status updates and SUD (LikeStatusSUD in Table 1) to 3508, which is the intersection of the likes and SUD (LikesSUD in Table 1).


0.69 0.64 0.67 Table 8: Results obtained on the test set for the source identification task Most of the errors of the algorithm are due to information sources not recognized as NEs (in particular, when the source is a Twitter user), or NEs that are linked to the wrong DBpedia page.

in order to draw more interesting conclusions on the most suitable methods to address this task, we would need the increase the size of the dataset.


They first detect argumentative sentences, and second identify premises and claims.

none of them is neither interested in distinguishing facts from opinions nor to identify the arguments' sources.


In recent studies, embeddings v IN are usually used for measuring the similarity between words.

given the characteristics described in the previous paragraph and SGNS's equivalence with shifted positive pointwise mutual information (Levy and Goldberg, 2014), if we want to measure to what extent word w t tends to co-occur with w k in the training data, then we should use the similarity of In this study, we show the importance of using v OU T in a task where we need to see if a word matches its context.


with uniform weights, we set α w j to be 1 for all w j in the context.


Close in spirit to our investigation, Schofield and Mehr (2016) train a number of classifiers over movie scripts for determining the gender of individual (and pairs) of speakers as well as the expected length of their relationships.

we focus on understanding how the gender of a given character implicitly relates to features that track their control over their own path (agency) and the world around them (power).


Advances in Natural Language Processing are leading to a point when text generation methods are deployed at scale.

in the quest to make these applications more likable, effective and hence more usable, these methods should consider a way to adapt themselves to the person or type of persons they are interacting with (Bates, 1994;Loyall and Bates, 1997) e.g., a student may learn better from a tutoring agent that expresses similar traits to himself (Baylor and Kim, 2004).


For instance in (Quinn et al., 2010), a topic model for legislative speech is defined.

those works study topics one at a time whereas a set of co-referenced topics is more relevant since it constitutes the core of a candidate's political program.


Following the tradition of sentiment analysis, many have proposed methods to automatically assess the quality of recommendations or comments based on subjective ratings of their usefulness (Liu et al., 2007;Siersdorfer et al., 2010;Becker et al., 2012;Momeni et al., 2013) or of persuasiveness (Wei et al., 2016).

information thought to be useful does not always prove so, and subjective ratings may be driven by biases.


MOOF forecasters were also linked directly to the form to update their forecast from the comment to minimize outside influences between the reading of the comment and prediction update.

as the forecasters were not in a laboratory, other Web browsing behavior in other tabs could not be controlled.


This model is then used to predict the quality of the comments in the 1/10th sample and compared to the true quality for those comments (using Pearson correlation in this case).

many of our scores for change in forecaster accuracy (benefit) are based simply on one change and thus quite unreliable.


Most studies listed thus far found length of comment to be the dominant predictor, with other features providing minimal benefit.

a few studies (including our own) have found this baseline can be overcome.


Extracting irregularly-formed bursty phrases as described in the previous paragraph is difficult since no restriction can be used anymore to filter out incomplete N-grams.

they are rare and little influence the overall accuracy even if they are correctly extracted.


In the field of trend detection on microblogs, this ignoring-minority strategy has become a de facto standard.

it always fails to extract irregularly-formed bursty phrases.


Owing to a good segmentation algorithm, it can potentially detect bursty phrases other than noun phrases, uni-grams, bi-grams, and tri-grams with high accuracy.

it is still possible to miss irregularly-formed bursty phrases because they are likely to be segmented incorrectly.


Among comparative methods, the segmentation-based method best achieved the min-z-score since it did not restrict the form of phrases.

it was still possible to miss very irregular phrases due to segmentation mistakes and the min-z-score was less than that of the proposed method.


Neural networks have achieved state-ofthe-art performance on several structuredoutput prediction tasks, trained in a fully supervised fashion.

annotated examples in structured domains are often costly to obtain, which thus limits the applications of neural networks.


The explicit supervision signals can be viewed as a source of immediate rewards, as we can often instantly know the correctness of the current action.

the implicit supervision can be viewed as a source of delayed rewards, where the reward of the actions can only be revealed later.


In , the authors demonstrated that labeling semantic parses is possible and often more effective with a sophisticated labeling interface.

collecting answers may still be easier or faster for certain problems or annotators.


Natural language processing research has largely adopted the outlined hierarchical models for mining arguments from text (Stab and Gurevych, 2014;Habernal and Gurevych, 2015;Peldszus and Stede, 2016).

the adequacy of the resulting overall structure for downstream analysis tasks of computational argumentation has rarely been evaluated (see Section 2 for details).


Recently, we generalized the model in order to make flows applicable to any type of information relevant for argumentation-related analysis tasks .

flows capture only sequential structure, whereas here we also model the hierarchical structure of overall argumentation.


To this end, we represent labels and positions as follows: Labels The tree kernel approaches in natural language processing discussed in Section 2 include text (usually words) in the leaf nodes.

we label each node v ∈ V with the type of the associated argument unit only.


In fact, we found that larger ηs do not delay most training instances in the first few iterations.

once the network obtains a reasonably high performance, schedulers start delaying instances for longer durations.


Most of them are based on the co-occurrence information of words and their contexts.

it is still an open question what is the best definition of context.


Furthermore, we systematically evaluate three word embedding models: CSG, CBOW and GLoVe.

c is thus odel; cond the numally larger generalize ords conpendencyntexts cap--word conence "Ausscope".


For example, in Figure 1, the context "scientist/nsubj" can only be predicted by word "discovers".

most of the word is connected to several contextual words.


In contrast, linear context type captures topical similarity with the "help" of unbound representation.

the above findings come with a major caveat: a lot seems to depend on the particular dataset, in addition to the model and context type.


The performance of unbound linear context and unbound DEPS context is similar.

for most models and categories, bound representation seems to outperform unbound representation.


Their system thus requires chunker or constituent parser.

we investigate the usefulness of syntactic information derived from dependency parses, and we extend their work in also comparing our results to the use of only POS tags and words.


As done in that study, we do sequence prediction using a neural network.

we extend their work significantly in reporting results for intra-sentential segmentation, in comparing more settings concerning the availability of information (tokenisation, POS tags), and in including syntactic information into our systems.


First, all previous systems were evaluated on the same set of 38 documents that initially contains 991 sentences -and more precisely on each sentence of this set for intra-sentential results.

soricut and Marcu (2003) do not consider sentences that are not exactly spanned by a discourse subtree (keeping only 941 sentences in the test set), and sporleder and Lapata (2005) only keep the sentences that contain intra-sentential EDUs (608 sentences).


The example actions not only make the learning safer but also can be directly used by the training of the student policy.

there are costs to the teaching of a human teacher.


dropout can be observed as claimed in (Gal and Ghahramani, 2016).

dropout dQN 1 seems to suffer premature and sub-optimal convergence, while our proposed dropout dQN 32, whose decision is based on multi votes (algorithm 1), can result in improvement of efficiency and better final performance.


(2016), which also makes predictions on a dataset derived from the IQ2 debates.

their work analyses speech signals, as opposed to textual data.


Considering applause as a sign of endorsement is not controversial, but laughter could be viewed as more ambiguous.

consider the audience of the debates: the debates air on the Bloomberg network and National Public Radio, suggesting a higher level of maturity of the audience, which is less likely to laugh at the participants, rather than at their jokes.


3 Although the level of statistical significance is reported to be 99%, CIs in Figure 1(a) show that the proportion of each bootstrap distribution was substantially underestimated leading to overly narrow CI limits for both SAME and DIFF.

figure 1(b) shows CIs resulting from an accurately computed proportion of 95% of the same bootstrap distribution, where even at the lower level of 95% significance (as opposed to 99%) CIs for SAME and DIff now overlap, reversing the conclusion of strong reference bias.


These sentences are designed to dive deep into linguistic phenomena of interest, and to provide a much finer-grained analysis of the strengths and weaknesses of existing technologies, including NMT systems.

this strategy also necessitates that we work on fewer sentences.


Leveraging zero-shot learning to learn mapping functions between vector spaces of different languages is a promising approach to bilingual dictionary induction.

methods using this approach have not yet achieved high accuracy on the task.


to advances in methods for extracting general purpose knowledge Nakashole et al., 2013;Wijaya et al., 2014), the use of semantic knowledge has been explored for several natural language tasks (Nakashole and Mitchell, 2015;Yang and Mitchell, 2017).

for bilingual dictionary induction, and more generally, machine translation, the role of semantic knowledge has not been fully explored.


This is partly due to a better prediction of non-TQs, represented by the high corresponding F-scores for CL-SEQ for all three language pairs.

it is also linked to a better labelling of grammatical and lexical TQs, which can be seen by the high labelling precision (P*) in the context of high recall (R).


The matrix reveals that for predicted lexical tags and non-TQs, the majority were correctly classed into these coarse-grained classes.

grammatical tags proved more difficult to predict, the majority being classed as non-TQs, most likely a result of the fact that no such tag question was present on the German source side.


(2008) and Naim and Gildea (2015) propose models that can use orthographic similarities.

the model proposed by (Naim and Gildea, 2015) is only capable of producing a parallel lexicon and not translation.


This shows that our system is able to exploit regular sound correspondences to filter out a substantial number of false cognates, such as lexical borrowings or chance resemblances.

the overall contribution of the specific models is relatively small.


For example, "sickness" is difficult for our general model to associate with "bitterness, pain."

there are many instances where our system is successful in identifying non-obvious semantic similarity, often thanks to the word vector features of our model.


There are aspects of cognate identification that can only be detected by human experts, such as cognates that have undergone extensive phonetic and semantic changes, or large-scale lexical borrowing between languages.

we believe that our system represents a step towards automated cognate identification, and will prove a useful tool for historical linguists.


Uyghur is not supported by Google Translate, and the Uyghur Wikipedia has less than 3,000 articles.

the smallest Wikipedia size language in our test set is Yoruba, with 30K articles.


This benefit for English NLP has motivated the development of VerbNets for languages such as Spanish and Catalan (Aparicio et al., 2008), Czech (Pala and Horák, 2008), and Mandarin (Liu and Chiang, 2008).

end-to-end manual resource development using Levin's methodology is extremely time consuming, even when supported by translations of English VerbNet classes to other languages (Sun et al., 2010;Scarton et al., 2014).


Approaches which aim to learn verb classes automatically offer an attractive alternative.

existing methods rely on carefully engineered features that are extracted using sophisticated language-specific resources (Joanis et al., 2008;Sun et al., 2010;Falk et al., 2012, i.a.


K-Means is outperformed for each target language, confirming the superiority of spectral clustering established in prior work, e.g., (Scarton et al., 2014).

we find results with another clustering algorithm, hierarchical agglomerative clustering with Ward's linkage (Ward, 1963), on par with spectral clustering (1.4 points on average in favour of spectral, which is better on 4 out of 6 languages).


They clearly indicate that cross-lingual synonymy constraints are useful for both relationship types (compare the scores with XLing), with strong gains over the nonspecialised distributional space.

the inclusion of VerbNet information, while boosting classification scores for target languages and (trivially) for EN, deteriorates EN similarity scores across the board (compare XLing+VN against XLing in Tab.


This work has proven the potential of transferring lexical resources from resource-rich to resourcepoor languages using general-purpose cross-lingual dictionaries and bilingual vector spaces as means of transfer within a semantic specialisation frame-work.

we believe that the proposed basic framework may be upgraded and extended across several research paths in future work.


"Swimming in the lake" has no natural endpoint, as also shown by the linguistic test presented in (2).

"swimming across the lake" will necessarily be finished once the other side is reached.


Variance reduction by additive control variates has implicitly been used in doubly robust techniques (Dudik et al., 2011;Jiang and Li, 2016).

the connection to Monte Carlo techniques has not been made explicit until Thomas and Brunskill (2016), nor has the control variate technique of optimizing the variance reduction by adjusting a linear interpolation scalar (Ross, 2013) been applied in off-policy learning.


(2014), containing 1,391 positive pairs and 4,294 negative pairs.

the number of positive pairs is not sufficient for our propose.


As Figure 1 shows, in ACE data, for 70% of relations, two mentions are embedded in each other or separated by at most one word.

in SF, more than 46% of query, filler entity pairs are separated by at least 7 words.


One domain for which automated analysis is particularly useful is Internet security: researchers obtain large amounts of text data pertinent to active threats or ongoing cybercriminal activity, for which the ability to rapidly characterize that text and draw conclusions can reap major benefits (Krebs, 2013a,b).

conducting automatic analysis is difficult because this data is outof-domain for conventional NLP models, which harms the performance of both discrete models (McClosky et al., 2010) and deep models (Zhang et al., 2017).


Roughly 60% of posts in the two forums contain multiple annotated tokens that are distinct beyond stemming and lowercasing.

we analyzed 100 of these multiple product posts across Darkode and Hack Forums, and found that only 6 of them were actually selling multiple products, indicating that posts selling multiple types of products are actually quite rare (roughly 3% of cases overall).


And as expected, the LCRF baselines yields relatively lower results compared to the other models, since it cannot predict overlapping mentions.

11 such results give us some idea on how much performance increase we can gain by properly recognizing overlapping mentions by looking at the results of LCRF (single), which in this case can be up to 9.7 points in F 1 -score in ACE-2004.


We believe it is possible that this slower convergence is due to the spurious structures issue in mention hypergraphs, which causes the objective function to be more complex to optimize.

some further analyses on the convergence issue and the impact of different ways of exploiting features (over different hyperedges) for the hypergraph-based models are needed.


In recent years, many text and language understanding tasks have been advanced by neural network architectures.

despite recent work, competitive ED systems still largely employ manually designed features.


MinIE's dictionary mode also makes use of the corpus frequency of constituents.

to ReVerb, MinIE uses frequency to inform minimization (instead of to prune) and applies it to subjects and arguments as well.


Past research has addressed citation sentiment (Athar and Teufel, 2012b,a), citation networks (Kas, 2011;Gabor et al., 2016;Sim et al., 2012;Do et al., 2013;Jaidka et al., 2014), summarization (Abu-Jbara and Radev, 2011) and some analysis of research community (Vogel and Jurafsky, 2012;Anderson et al., 2012;Luan et al., 2012Luan et al., , 2014bLevow et al., 2014).

due to scarce hand-annotated data resources, previous work on information extraction (IE) for scientific literature is very limited.


An analysis of confusion patterns show that the most frequent type confusions are between PROCESS and MATERIAL.

we observe that ULM+GRAPHFEAT* can greatly reduce the confusion, with 3.5% relative improvement of PRO-CESS and 3.6% relative improvement of PROCESS over ULM+GRAPHINTERP on token level.


For example, in biomedical domain, the number of unseen biomedical entity mentions (such as disease names, chemical names), their abbreviations or acronyms, as well as multiple names of the same entity is growing fast with the rapid increase of biomedical literatures and clinical records.

the performance of a learning based NER system relies heavily on data annotation, which is quite expensive.


Therefore, in many special domains, only trained models or APIs are available, while their training data are private and inaccessible.

due to insufficient labeled training data, deep models usually fail to behave normally in such domain, and state-of-the-art methods in these domains are usually dominated by rule based deductive methods or shallow model with hand-crafted features.


For this dataset, state value function of RLIE-A3C converged at 48 minutes.

from Figure 4, we observe that accuracies converge to the final values much before that.


Motivated by the observation that named entities are highly related to linguistic constituents, we propose a constituent-based BRNN-CNN for named entity recognition.

to classical sequential labeling methods, the system first identifies which text chunks are possible named entities by whether they are linguistic constituents.


Finally, the system makes predictions for a sentence by collecting the constituents whose most probable classes are named entity classes.

nodes whose ancestors are already predicted as named entities are ignored to prevent predicting overlapping named entities.


Given these features, O(D) prediction is simple and parallelizable across the length of the sequence.

feature extraction may not necessarily be parallelizable.


It may also have better sample complexity, as it imposes more prior knowledge about the structure of the interactions among the tags (London et al., 2016).

it has worse computational complexity than independent prediction.


These outperform similar systems that use the same features, but independent local predictions.

the greedy sequential prediction (Daumé III et al., 2009) approach of Ratinov and Roth (2009), which employs lexicalized features, gazetteers, and word clusters, outperforms CRFs with similar features.


In order to disambiguate "India" to the correct entity, a linking system would need to utilize both the local context (played and match), and the document context (to identify the sport).

the model needs to represent context such that the semantics are preserved, e.g.


(2005) released a corpus covering four different kinds of OCRed text comprising German and Bulgarian.

in 2017 the corpus was untraceable and no recent re-search relating to the data could be found.


the 17th century or named entities cannot be found in a dictionary and can therefore not be covered with any of the approaches mentioned above.

especially named entities are crucial for the automatic or semi-automatic analysis of narratives e.g.


We henceforth call this unknown set test unk (text 6).

the second set contains parts of the same texts as the training, thus specific vocabulary might have been introduced already.


Given a sequence [x 1 , x 2 , ..., x T ] where x t is the input embedding of element t, the state of Bi-GRU at position t is: where h f t and h bt are the states of the forward and backward GRU at position t. The final sequence embedding is either the concatenation of h f T and h b1 or simply the average of h t .

attentive Sequence Encoder directly using [h f T , h b1 ] for sequence encoding often fails to capture all the information when the sequence is long, while using the average of h t also has the drawback of treating useless elements equally with informative ones.


The improvements made by using relevant law articles actually indicates the nature of the civil law system that judgements are made based on statutory laws.

if we only use the extracted relevant articles to make prediction (SVM art and NN art 4 ), the performance becomes worse.


In LSA, entropy is not as directly applicable: vectors in W = U Σ can be arbitrarily realnumbered.

we still want to access a similar basic concept, the amount a vector representation of a document is concentrated in a few dimensions.


However, the amount of decay depends on the proportion of the corpus replicated: the smaller the proportion size, the more dramatic the decay.

the loss for singular documents increases only slightly with more copies, though more for higher proportions of duplication.


Figure 6 demonstrates that, as before, perplexity is significantly higher as template repetition increases when there is a small number of topics K = 5.

as the number of topics in-  Figure 6: LDA training perplexity for REUSL 2.5k with different types of templated text repetition.


For LSA, templated repetition has no apparent effect on the loss of untemplated texts.

the effect for templated texts is less straightforward.


Repeating one document will likely only affect one or a few components regardless of how many repetitions occur.

if there are many different repeated documents, more components will be used to model them, which worsens the model fit more as the number of unique repeated texts increases.


In a topic model, it may be easy to identify the templated text based upon it appearing in one topic.

if there is a concern that there is systematic use of text templates in documents (such as page headers or publication information) that may be too close to the language model, the ngram removal approach inspired by Citron and Ginsparg (2015) is an expensive but straightforward way to ensure these strings are detected and deleted.


Existing models may lack signals for the following outlier detection steps and hence cannot be directly plugged in.

it is possible to adapt certain models to the outlier detection task.


The semantic regions learned from the Embedded vMF Allocation model provide a set of candidates frequently mentioned by documents in the corpus.

not all of them are semantic focuses of the corpus -some are too general to distinguish outlier and normal document.


To estimate this, we first need to calculate the probability of each word being drawn from the semantic focuses.

it is then possible to estimate the expected percentage of words not drawn from semantic focuses in each document as the outlierness: due to the noisiness in text data, this assumption oversimplifies the characterization of outlier documents.


Figure 1 shows an example of causal features that temporally causes Facebook's stock rise in August.

it is difficult to understand how the statistically verified factors actually cause the changes, and whether there is a latent causal structure relating the two.


A simple choice for traversing a graph are the traditional graph searching algorithms such as Breadth-First Search (BFS).

the graph searching procedure is likely to be incomplete (low recall), because simple string match is insufficient to match an effect to all its related entities, as it misses out in the case where an entity is semantically related but has a lexically different name.


By this way, the relevant videos to the threads can be recommended to learners, and the chaotic threads in discussion forums can also be well grouped.

it is challenging to identify the relevant video clips for threads without labeling data.


To our knowledge, (Jiang et al., 2017) also proposes an unsupervised learning model (called NOSE) for the task of thread-subtitle matching within MOOC settings.

nOSE needs to build a heterogeneous textual network beforehand and may suffer from heterogeneous issue, which our model can avoid.


Choices for privacy controls, which are the most actionable pieces of information in these documents, are frequently "hidden in plain sight" among other information.

the nature of the text and the vocabulary used to present choices provide us with an opportunity to automatically identify choices, a goal that we focus upon in this paper.


The unbalanced distribution of the opt-out labels allowed us to manually verify and correct labels in the positive class.

correcting errors in the much larger negative class (of 12K instances) was a challenge, since comprehensive manual verification was infeasible.


The BCT model serves as a state-of-the-art transition-based baseline.

it requires that the training data contains both syntax trees and disfluency annotations, which reduces the practicality of the algorithm.


4 We can see that AMU16 SMT is the current state of the art on CoNLL, with an F 0.5 of 49.49.

cAMB16 SMT generalises better on FcE and JFLEG: 52.90 and 52.44 F 0.5 respectively.


A selectively chosen example is the replacement from "discontinous" to "discontinuous", which never occurs in training.

similar errors of low edit distance also occur once in the dev set and never in training, but the CHAR+BI+DOM model filtered against the NUCLE corpus, hurt effectiveness for the phrase-based models.


Similarly, we solved 75% of the Inconsistency errors including lexical, tense and definiteness (definite or indefinite articles) cases.

we also observe that our system brings relative 21% new errors.


In this case, transfer learning methods still show better accuracies than target-only approaches on average.

the performance gain is weakened compared to using 1,280 labeled training sentences and there are some mixed results.


When training with only 32 tag-labeled sentences, which is an extremely low-resourced setting, transfer learning methods still showed better accuracies than target-only methods on average.

not using the common BLSTM in transfer learning models showed better performance than using it on average.


Recently Lin and Parikh (2016) have leveraged visual question answering models to encode images and descriptions into the same space.

all of this work is targeted at monolingual descriptions, i.e., mapping images and descriptions in a single language onto a joint embedding space.


Linguistic parsers can also be too slow for real-time applications.

an RNN can detect entities in the question with high accuracy and low latency.


(2015) similarly assume that the answer to a question is at most two hops away from the target entity.

they do not propose how to obtain the target entity, since it is provided as part of their dataset.


These inner products are negative, indicating that the context vectors point in the opposite direction from the word vectors.

the GloVe context vectors have essentially the same relationship to the mean of the word vectors as the word vectors themselves.


From a memory perspective, one multilingual BTS model will take less space than separate FF models.

from a runtime perspective, a pipeline of our models doing language identification, word segmentation, and then POS tagging would still be faster than a single instance of the deep LSTM BTS model, by about 12x in our FLOPs estimate.


(2016) considered relation between opinion words and aspect words in a supervised model named RNCRF.

rNCrF tends to suffer from parsing errors since the structure of the recursive network hinges on the dependency parse tree.


CMLA (Wang et al., 2017a) used a multilayer neural model where each layer consists of aspect attention and opinion attention.

cMLA merely employs standard GRU without extended memories.


For example, domain adaptation is studied for sentiment classification (Glorot et al., 2011) and parsing (McClosky et al., 2010), just to name a few.

there is very little work on domain adaptation for word embedding learning.


Embeddings learned from such an approach were shown to be able to improve the performance on a cross-domain sentiment classification task.

this model fails to learn embeddings for many words which are neither pivots nor non-pivots, which could be crucial for some downstream tasks such as named entity recognition.