The right plot shows the cell states of LSTM unit3 (x-axis) and unit4 (y-axis).

table 2 shows how R 2 increases as beam search augments the subset of units.


A combination, A → T, shows that we optimize A only first, then we fix A and only update T part.

experimental results in Table 1 show some interesting results.


For that reason, we use the English-Italian dataset on the same task provided by  2 .

for that purpose, we use the translation induction task introduced by Mikolov et al.


These multilingual interactions facilitated by MT, such as reading nonnative listing descriptions or conversing with a foreign seller, are integral to the user experience.

a generic engine, trained on commonly available parallel text, translates clutch as an "automotive clutch."


Thus, each backtrack will require at maximum 2l − 1 predictions.

heuristic backtracking with cutoff outperformed greedy decoding, and reduced transitions by an additional 50%.


The scoring function is a linear model f (x, y) = θ Φ(x, y, z) and is trained with an early update structured perceptron to match both a given order and syntactic tree.

let Z represent all projective dependency parse trees over M words.


The correct parse, under our scheme, is [un [[lick] able]].

by contrast, the parse [[un [lick]] able] contains the span unlick, which is not a word in the lexicon.


Luckily, we can exploit the structure of the problem to efficiently compute the partition function, where T (u) is the set of all trees under the grammar that have yield u.

similarly, Adaptor Grammars (Johnson et al., 2006), a non-parametric Bayesian generalization of PCFGs, have been applied to the unsupervised version of the task (Botha and Blunsom, 2013;sirts and Goldwater, 2013).


If a is a subspan of another answer a and their votes differ by at most one (See Example 7 in Table 2), it is unlikely that both a and a are correct.

the exception to this is with copular predicates, where we include the span of the argument in the question (see Example 4 in table 2).


Local factors reflect how important an event is in its own right.

further subtasks included the normalisation of temporal expressions and the creation of links between events and temporal expressions.


Many models have been proposed for KB embedding (Nickel et al., 2011;Bordes et al., 2013;Socher et al., 2013  2013; Wang et al., 2014a;Zhao et al., 2015), entity type and relationship domain (Guo et al., 2015;Chang et al., 2014), and relation path (Lin et al., 2015a;Gu et al., 2015).

the di-  mension of the embedding n is set in the range of {20,50,100}, the margin γ 1 and γ 2 are set in the range {1,2,4,10}.


these methods solely rely on triple facts but neglect temporal order constraints between facts.


For instance, RL-Extract is 11.4% more accurate than the basic Maxent extractor on City and 7.1% better on NumKilled, while also achieving gains of more than 5% on the other entities on the Shootings domain.

the DQN, in which the Q-function is approximated using a deep neural network, has been shown to learn better value functions than linear approximators (Narasimhan et al., 2015;He et al., 2015) and can capture non-linear interactions between the different pieces of information in our state.


, w n , we compute hidden states h t and cell states c t in the forward LSTM for 1 < t ≤ n: where σ is the logistic sigmoid, • is the componentwise product, and x t denotes a learned word embedding for w t .

instead of computing the full recursion, we reuse the existing latent states of the children and compute s global (e) with an incremental forward pass over a single recursive unit in the neural network.


We refer to these models as locally factored models.

our model scores a hyperedge e by combining the score from the local model with a global score that conditions on the entire parse at the head node: In s global (e), we first compute a hidden representation encoding the parse structure of y = HEAD(e).


In the en-ja/vague case, the model actually underperforms its monolingual counterpart.

translations of the minority language into a major language are more easily acquired.


then we handcraft derivation rules (e.g., create, creation, creator) and extract potentially derivational words from the training corpus.


(2008, 2011 formally prove that sequences of steps in the edgefactored GBDP (Eisner, 1996) can be used to emulate any individual step in the arc-hybrid system (Yamada and Matsumoto, 2003) and the Eisner and Satta (1999, Figure 1d) version.

• Combining exact decoding with global training -which is also enabled by our minimal feature set -with an ensemble of parsers achieves 90.22 UAS on the Chinese Treebank and 95.33 UAS on the Penn Treebank: these are, to our knowledge, the best and secondbest results to date on these data sets among "purely" dependency-based approaches.


Higher-order MST models associate higher-order score functions with the construction of individual dependencies.

given a sentence s = w 0 w 1 • • • w n−1 of length n, the vertices, i.e.


Edges e 1 and e 2 cross if e 1 and e 2 have distinct endpoints and exactly one of the endpoints of e 1 lies between the endpoints of e 2 .

our previous work showed that 1-endpoint-crossing, pagenumber-2 (1EC/P2) graphs are an appropriate graph class for modelling semantic dependency structures .


The learning process of REHESSION is summarized as below.

among four tasks, the relation classification of Wiki-KBP dataset has highest label quality, i.e.


Thus, the reliability of a labeling function is not consistent among different pieces of instances.

8 efficiently, we adopt the stochastic gradient descent algorithm to update {W, v, v * , l, t} iteratively, and o c * is estimated by maximizing J T after calculating z c .


The remaining of this paper is organized as follows.


The drawback of above approach is that it considers the contribution of each event on the chain is same.

there are several important hyper-parameters in our models, and we tune their values using the development dataset.


It is noted that the anchors from sentence search can also be found in entity search.

in addition, features such as the number of sentences containing the entity from sentence search which provide query sensitive prior probability contribute to the ranking process.


Since these are crucial to question-answering, we use a small lexicon to refine the tags for determiners (DET), adverbs (ADV) and pronouns (PRON) to DET:WH, ADV:WH and PRON:WH, respectively.

we perform experiments on question answering against Freebase and provide German and Spanish translations of the webQuestions and GraphQuestions datasets to facilitate multilingual evaluation.


This research is supported by a Google PhD Fellowship to the first author.

this work is limited to English, and cannot process dependency graphs, which allow handling complex phenomena such as control.


In all of these cases, the model for composing embeddings of subword units into word embeddings is learned by optimizing an objective over a large unlabeled corpus.

unlike prior work, MIMICK does not require retraining on the original word embedding corpus; instead, learning is performed at the type level.


The immediate challenge is that without tokens, we have no alignment between the languages anymore.

we compute the relative position x of each pivot that occurs in the verse and apply a Gaussian filter (σ = 6 where the unit of length is the character), i.e., we set p(x) ≈ 0.066 (0.066 is the density of a Gaussian with σ = 6 at x = 0) and center a bell curve around x.


The labels whose ratio is more than 3% are nn, amod, prep, pobj, dobj, nsubj, num, det, advmod, and poss.

the encoder then takes the word embedding v enc (w i ) and the hidden state h (2) i as the input to a uni-directional LSMt: is the hidden state corresponding to w i .


During the beam search step, we use the following scoring function for a generated word sequence y = (y 1 , y 2 , .

these results suggest that our model can be improved by a small amount of parsing and tagging datasets in different domains.


The intuition is that since the initial state is responsible for the translation of whole target sentence, it should at least contain information of each word in the target sentence.

due to the large amount of parameters and relatively small training data set, the end-toend learning of an NMT model may not be able to learn the best solution.


Our architecture handles many of the linguistic phenomena needed to solve this task, including object references and spatial regions.

offset scores o t are assigned based on the context the word x t occurs in, which allows the model to incorporate information from words such as right or under in its decisions.


(2011) and led to the creation of shared tasks such as SpaceEval (Pustejovsky et al., 2015).

for our evaluation, we restrict the set {y 1 , .


The hidden layer is subject to drop-out and elastic net regularisation (with equal l 1 and l 2 shares).

a mixture density network ("MDN": Bishop (1994)) is a latent variable model where the conditional probability of p(y|x) is modelled as a mixture of K Gaussians where the mixing coefficients π and the parameters of Gaussian distributions µ and Σ are computed as a function of input using a neural network: In the bivariate case of latitude/longitude, the number of parameters of each Gaussian is 6 (π , which are learnt in the output layer of a regular neural network as a function of input x.


To further reduce the number of spans to consider, we only keep up to λT spans with the highest mention scores and consider only up to K antecedents for each.

we factor the model over unary mention scores and pairwise antecedent scores, both of which are simple functions of the learned span embedding.


The ROUGE evaluations on DUC 2003 topic-focused summarization task and DUC 2004 generic summarization task show the good performance of our method, which has the best ROUGE-2 recall among the graph-based ranking methods.

the information coverage of summary is a determinant, which requires that the summary sentences should cover diverse aspects of information.


Other details can be found in Supplementary Materials.

the ASN corpus serves as the test corpus.


The first feed-forward layer size is set to a value in {400, qlog-U(200, 800)}, the second to a value in {1024, qlog-U(400, 2000)}.

to nominal anaphora, abstract anaphora is difficult to resolve, given that agreement and lexical match features are not applicable.


The direct use of matrix is reported to bring improved results and higher speed.

sGNs is proven to be equivalent with factorizing pointwise mutual information (PMI) matrix (Levy and Goldberg, 2014c).


This model relies on the principle "You shall know a word by the company it keeps" -Firth (1957).

2 https://en.wikipedia.org/wiki/Alan_Turing (2014) combine several sources (syllables, POS tags, antonyms/synonyms, Freebase relations) and incorporate them into a CBOW model.


(2016) uses internal additional information from the corpus to solve the latter drawback.

there is a non-zero probability that w i and w t are related.


Instead of adding mean value of character glyph features of the contexts, it adds mean value of glyph feature of the target word (tarG), as shown in Fig.8.

there is no ambiguity of radicals, so each radical is assigned with one embedding r. We denote r k as the radical embedding of character c k .


It is common for a Chinese learner to comprehend the meaning of a word from these graphical components.

updating model parameters for word w i would also change the word representation of word w j .


However, in practice, since we have so much bitext to draw from, we can easily find data in which they do not match exactly.

we performed this procedure separately for each of the 10 language/data pairs.


26206717), China 973 Fundamental R&D Program (No.

for example, (water) is the semantic component of characters (lake) and (sea), (horse) is the phonetic component of characters (mother) and (scold) where both and are pronounced similar to .


We quantitatively evaluate the quality of word embeddings learned by our model on word similarity evaluation and word analogy tasks.

wUBI input method uses rules to group Chinese characters into meaningless clusters which can fit the alphabet based keyboard.


If we encounter a word such as "preparedness" for which we do not have a representation or our representation is not reliable, we can identify any reliable form of the word, say "prepared" and apply <null,ness> operator on it, resulting in a representation for "preparedness".

embeddings generated by our approach should be evaluated by plugging into end-level tasks such as machine translation, POS tagging etc.


Moreover, we select aligned sub-sentence pairs between paraphrases to enlarge the training corpus.

the code and data will be publicly released.


Distributional models (DS: Turney and Pantel (2010); Clark (2012); Erk (2012)), and in particular neural network approaches (Bengio et al., 2003;Collobert et al., 2011;Huang et al., 2012;Mikolov et al., 2013), do not fare well in the absence of large corpora.

finally, human similarity responses are given for each probe with respect to the nonce.


The PMI-based weighting function is critical here since it helps to highlight the more surprising events in original word-context matrix.

more recently, (Levy and Goldberg, 2014) has showed that the objective function of SGNS is essentially seeking to minimize the difference between the models estimate and the log of cooccurrence count.


The conventional approach required an average of 177 minutes to train, evaluate, and select an embedding set for each test corpus.

high-quality embeddings can be expensive to train.


The dimensionality of the original embedding space should be retained and used for learning the manifold to guarantee the best re-embedding.

dimensional reduction on the original space will bare a cost in performance.


other popular candidates, like GloVe (Pennington et al., 2014) and CBoW (Mikolov et al., 2013a), require more sense identities to be selected as input and thus not suitable for our scenario.


For example, in the TOEFL-80 dataset, a question shows {(Q) enormously, (A) appropriately, (B) uniquely, (C) tremendously, (D) decidedly}, and the answer is (C).

instead, the proposed approach, MUSE, enables joint training on sense selection and representation learning.


To address the above issues, we apply the Qlearning algorithm (Mnih et al., 2013).

mUSE enables linear time sense identity decoding with a sense selection module and purely senselevel representation learning with a sense representation module.


For flat networks with a single LSTM-layer, around 150 recurrent units yielded the best performance.

gradient normalization has a better theoretical justification and rescales the gradient whenever the norm goes over a threshold.


the network will not be able to distinguish between trigrams at the beginning, in the middle, or at the end of a word, which can be crucial information for some tasks.

there is a good chance that these configurations require less tuning when applied to new tasks or domains.


An alternative is the sparsemax transformation (Martins and Astudillo, 2016): The sparsemax maintains most of the appealing properties of the softmax (efficiency to evaluate and backpropagate), and it is able to generate truly sparse distributions.

it is possible to have fewer steps (or more, if we want the decoder to be able to "self-correct").


the re-training is obviously inefficient since it has to process the entire training data received thus far whenever new training data is provided.


In addition, we are interested how well the metric learned for one target domain transfers to other settings.

selecting individual examples based on a domain similarity measure performs only as good as chance.


a relatively linear mapping between source and target tokens.

the typical attention mechanism used in these architectures computes a new attention context at each decoding step based on the current state of the decoder.


By visualizing attention scores we demonstrate that our models learn distinct, meaningful alignments.

we avoid looking back at the source sequence once it has been encoded.


Though the mapping is simple, it is not one-toone (a group of characters can correspond to a single phoneme) nor deterministic (the same character can map to different phonemes depending on the context).

the explanation graphs for two such examples, shown in Figure 8 (left and center), suggest strong associations between the genderneutral but stereotype-prone source tokens (nurse, doctor, charming) and the gender-carrying target tokens (i.e.


Using features that test arbitrary combinations of tag components would make feature selection much more difficult, as the number of possible patterns grows combinatorially with the number of components.

even in moderate size applications, the parameter set can be very large and contain dozen of millions of features, due to the introduction of sequential dependencies in the model.


Online topic models (Hoffman et al., 2010;Bryant and Sudderth, 2012;Paisley et al., 2015) optimize the global parameters of interest using stochastic gradient ascent.

boyd-Graber is also supported by IIS-1564275 and IIS-1652666.


ADAGRAD is a popular learning rate method designed for online optimization problems with high dimension and sparse gradients.

the learning rates will not monotonically decrease, but vary in a certain range.


In addition, since ADADELTA computes the moving average of updates, and ADAM needs to compute the bias-corrected gradient estimate, they require more intricate implementations.

since the update progresses slowly, online LDA needs more iterations to break the equilibrium.


When using ADA-GRAD (Duchi et al., 2011) with online LDA, the update rule for the each learning rate is where ρ 0 is a constant, and a very small guarantees that the learning rates are non-zero.

it may seem reasonable to apply ADA-GRAD to optimize online topic models.


Since the same words in Figures  3a and 3b have different memory slices due to position weighting and augmented offset feature, as described in Section 3.3, our model predicts opposite sentiments on the two persons.

the model is trained by minimizing the cross entropy plus an L 2 regularization term: where C is the sentiment category set, D is the collection of training data, y ∈ R |C| is a one-hot vector where the element for the true sentiment is 1, f (x; θ) is the predicted sentiment distribution of the model, λ is the weight of L 2 regularization term.


For example, in "I bought a mobile phone, its camera is wonderful but the battery life is short", there are three opinion targets, "camera", "battery life", and "mobile phone".

for simplicity, we notate a target as one word here, where necessary, we will elaborate how to handle phrase-form targets, e.g.


One might want to use the length of the path from the specific word to the target in the dependency tree as the distance, which is a worthwhile option to try in the future work, given the condition that dependency parsing on the input text is effective enough.

our framework introduces a novel way of applying multiple-attention mechanism to synthesize important features in difficult sentence structures.


The predicted reading time is then used to build an attention layer in neural sentiment analysis models.

standard RNN suffers from the gradient vanishing problem (Bengio et al., 1994) where gradients may grow or decay exponentially over long sequences.


We use two real world datasets: (a) hotel reviews from TripAdvisor (Wang et al., 2010), and (b) restaurant reviews from yelp.com.

the fundamental assumption of topic models is the independence of topics even in the same document.


The Analyze-Words snapshot of each user was taken at the time of that user's tweet in the dataset.

first, as outlined above, it shows how different kinds of context -from the linguistic to the psychological -can be usefully incorporated to yield improved detection.


The proposed model is a mixture model with multinomial distributions as component models (i.e., models of topics), similar to Probabilistic Latent Semantic Analysis (Hofmann, 1999).

sion is around 50%, see Table 2, this is more than two times better than random guess which is about 19% and second that our features can be used to filter out some useful reviews.


In subsection 5.1, we describe one way of obtaining these aspects, but first we describe the estimation methodology.

in funny reviews, the authors tend to use referential humor, in which specific concepts or entities are referenced to produce comedic effects, which we call aspects.


HSSWE (hard) has slight superiority over HSSWE (hard) in Semeval 2013, 2014 and 2016, but HSSWE (hard) is better on Semeval2015.

the key of this method is to build a lexical graph by calculating the PPMI between words.


The reason is that direct parallel data can be used for transferring the sentiment information from the source language to the target language directly.

xiaojun Wan is the corresponding author.


The detail of these symbols would be illustrated in the following part.

(2014) apply convolutional neural network for modeling sentences and Li et al.


In order to measure the performance of our model, here we use three metrics.

some work take user information into consideration.


The scalars λ 1 and λ 2 are hyperparameters.

the posterior distribution of the hidden variables for each word w dv,i (i-th word in venue document d v ) is calculated as follows (to simplify, we use Θ to refer to s,z ) (1) where n u,c (0) and n u,c (1) are the numbers of times that c = 0 and c = 1 are sampled for user u, respectively, and we have n u,c n u,c (0) +n u,c (1).


The valence represents the degree of positive and negative sentiment, where values of 1, 5 and 9 respectively denote most negative, neutral and most positive sentiment.

section 2 describes the proposed word vector refinement model.


They provided ordering among polar adjectives that bear the same semantics using a corpus-based approach.

most of the literature on intensity ranking consists of manual approaches or corpus-based approaches.


For example, the polarity vector (0.9, 0.1) means that 90% of the words in the existing lexicon that contains the character are positive and the other 10% are negative.

in the first stage, a Spy technique is used to identify the set of reliable N class instances (or examples) RN from U .


Below, we first evaluate sentiment words discovery (Step 1) and then polarity classification (Step 2).


A potential reason is that most active learning algorithms require a substantial 'seed set' of data for learning a basic classifier, which can then be used for active data selection.

2 As each instance arrives, an agent must decide the action to take, namely whether or not the instance should be manually annotated.


As the model, we use a standard linear chain CRF (Lafferty et al., 2001) for the first two sets of experiments, while for cold-start case we use a basic RNN classifier with the same multilingual embeddings as before, and a 128 dimensional hidden layer.

given the dearth of data in the low-resource setting, this assumption can make standard approaches infeasible.


Through this differentiable component, the loss of the discriminator is properly propagated to the generator and guide the tuning of the latter's parameters.

firstly, when training G, we replace the objective function given in Equation 5 with the l 2 -loss between A r and Ar, to maintain a reasonable scale of the gradient.


This experimental setting is due to the following two reasons.

in this paper we adopt a CNNbased discriminator as shown in Figure 1.


This suggests that our model is better at encoding latent representations of long texts since the amount of information in the latent vector does not decrease as the length of a text fragment grows.

i want to see you so much @userid #FollowMeCam ... @userid @userid @userid @userid @userid ... Why do i start the day today?


We split  -six or more "funny votes" is funny -three or fewer funny votes is not funny Otherwise the word was discarded.

a player can see only the story's title and the list of blanks with hints.


As a result, doc2vec (Le and Mikolov, 2014) doesn't perform well on Information Retrieval as described in (Ai et al., 2016a) and (Roy et al., 2016).

empirically, compared to T W e, SCDV reduces document vector formation, training and prediction time significantly.


Since it is easier to linearly separate in high dimension, especially with logistic regression, it is not surprising that increased embedding sizes lead to increased performance for almost all models.

(pos) SUBJ 10k subjectivity/objectivity 2 "A movie that doesn't aim too high , but doesn't need to."


if both fail, then we check whether A ′ contradicts B ′ , which amounts to proving the negation of the original conclusion, namely A ′ ⇒ ¬B ′ and B ′ ⇒ ¬A ′ .


There are two reasons for this phenomenon: negations tend to be omitted in nonlogic-based features such as TF-IDF and the proportion of word overlap is high.

5.0 4.9 4.1 Yes A rock is being climbed by a person with a rope, which is pink.


Again, by applying inference rules, we obtain the following sets of premises P and subgoals G: Here, the two sub-goals G 5 and G 6 do not match any of the premises, so the attempted proof of B ′ ⇒ A ′ fails.

we set the baseline of the relatedness score to 5 when the gold entailment label was yes and to 3 when the label was no or unknown.


During this period, we have summarized a list of highfrequency corresponding patterns among the three guidelines (see Appendix A), and have also written a simple program to automatically detect inconsistent annotations of given words in different training datasets, so that annotators can use the outputs of the program to decide ambiguous cases, which we find is extremely helpful for annotation.

this paper defines MWS as a strict hierarchy structure, and propose a supervised learning framework for the problem.


The morphological features vector can be constructed through various constructs, representing morphological and/or subword units.

in this paper we also study the effect of using word-level vs character-level embeddings on the overall morphological tagging problem, especially in light of the various subword and morphological features that we utilize.


Morphological analyzers help reduce sparsity by providing several out-of-context morpheme-based analyses for words, but they usually introduce ambiguity by returning multiple analyses for the same surface form.

in the example above the case of the noun mqAlthA 'her article' is dependent on the voice reading of the verb, which determines if the noun is the subject or object of the verb.


Inflectional morphology primarily marks semantic features that are necessary for syntax, e.g., gender, tense and aspect.

for example, English is labeled as morphologically impoverished, whereas German and Russian are considered morphologically rich, e.g., see the introduction of Tsarfaty et al.


The radical structure of a Chinese character does not follow any systematic process, requiring an incomplete dictionary mapping between characters and radicals to take advantage of this information (Sun et al., 2014;Yin et al., 2016).

first, as mentioned earlier, the compositional structure is fundamentally different between Chinese and Korean.


Many previous works have also considered morphemes to augment lexical models (Luong et al., 2013;Botha and Blunsom, 2014;Cotterell et al., 2016).

we believe that the true benefit of this architecture will be more evident in speech processing as jamo letters are definitions of sound in the language.


Tagging those languages requires a morpho-D a n is h D u tc h E n g li s h G e rm a n -1 G e rm a n -2 G e rm a n -3 Is la n d ic N o rw e g ia n S w e d is h -1 S w e d is h -2 B -P o rt u g .

taggers for Slavic languages often make use of additional resources such as morphological dictionaries, which we intentionally do not include to avoid human-crafted resources that are not available for all languages.


Especially for the morphologically fine-grained tagsets this might have been problematic, as it is doubtful if all PoS tags of a morphological tagset do even occur on 50k tokens.

table 1 shows the fine-grained annotated corpora we collected by screening the literature.


Rather, our method jointly learns multiple taggers and forces them to share features-a true transfer learning scenario.

to the fact that F 1 is a softer metric in that it assigns credit to partially correct guesses.


Our work addresses a different task, but the undergirding philosophical motivation is similar, i.e., attack low-resource NLP through multi-task transfer learning.

namely, we define each word embedding as the concatenation we run a bidirectional LSTM over the character stream.


In particular, our context window size is five.

as shown in Table 5, using word-context character (WCC) embeddings and multi-view wordcontext character embeddings both give significantly higher accuracy improvements compared with other semi-supervised methods.


For out-of-domain data, 20K raw sentences of Zhuxian is used.

their results cannot be compared directly with ours because they use partial labeled URL link data from Chinese Wikipedia data for training.


We performed a grid search over (C, classifier) ∈ {0.5, 1, 5, 10, 50, 100} × {one-vs-one, one-vs-rest} of linear SVM using the train set for each vector representation, and 2 We used {ja,zh}wiki-20170220-pages-articles1.xml in https://dumps.wikimedia.org 3 In this experiment, we defined the frequent n-grams as the union of the top-kn frequent ngrams, where n and kn are the pre-specified numbers.

word segmentation, that often requires manually annotated resources, is difficult and expensive, and unavoidable errors in word segmentation affect downstream tasks.


The turkers' performance is 85% for RACE-M and 70% for RACE-H.


the integrated approach also allows us to handle novel classes of anaphoric phenomena by framing anaphora resolution as an operation on a parse forest decorated with implicits (Section 5.2).


This encourages us to build a semantic parser that produces an explicit representation of what the question is asking, if we want to make quantitative progress on the question set.

4 System Overview Figure 1 shows a high-level view of our QA system.


aligns nouns to variable slots {x 1 , x 2 , ...} which leads to a huge hypothesis space and does not perform as well as the number slot alignment only method proposed later by (Zhou et al., 2015).

first we parse the equation to a hierarchical tree.


Although the methods proposed in the paper are demonstrated using syntactic trees, we note that similar approaches can be used to encode other types of tree structured information such as knowledge graphs and ontology relations.

instead of using a fixed embedding, we use Convolutional Neural Networks (CNN) to model character level embedding, which values can be changed during training (Kim, 2014).


They used logistic regression with pos tagging information (Rajpurkar et al., 2016) and provided a strong baseline for all subsequent models.

biDAF failed to anwer the question correctly and selected a noun phrase as the answer.


We experimented with different configurations of defining contexts and entity definitions, such as expanding the context windows by including sentences that come before and after the one where blank is found, as well as taking more than one sentence out of the entity description.

we measure the performance on our entity prediction task using the accuracy; that is, the number of correct entity predictions made by the model divided by the total number of predictions.


(2017) investigated a similar approach to generate embeddings for out-of-vocabulary words from their definitions and applied it to a number of different tasks.

this suggests that models that only rely on the surrounding context information may not be able to correctly predict the missing entities.


Our first model, DOUBENC, combines information derived from two encoders: one for the text passage being read, and one for the entity description.

we parse the HTML texts of the web pages and ex- 238,025 (97.1%) Table 2: Statistics for the augmented version of the Wikilinks dataset, where n represents the entity frequency in the corpus.


Transfer learning (Pan and Yang, 2010) has been successfully applied to numerous domains in ma-chine learning, such as machine translation (Zoph et al., 2016), computer vision, (Sharif Razavian et al., 2014), and speech recognition (Doulaty et al., 2015).

the rise of large-scale human annotated datasets with over 100,000 realistic question-answer pairs such as SQuAD (Rajpurkar et al., 2016), NewsQA (trischler et al., 2016), and MSMARCO (Nguyen et al., 2016), has led to a large number of successful deep learning models (Lee et al., 2016;Seo et al., 2017;Dhingra et al., 2017;Wang and Jiang, 2016).


We use LSTMs with hidden states of size 150 for the answer module vs. those of size 100 for the question module since the answer module is less memory intensive than the question module.

to train the answer synthesis module, we further augment the human-annotated labels of each paragraph with tags from a simple NER system 6 because labels of answers provided in the train set are underspecified, i.e., many words in the paragraph that could be potential answers are not labeled.


Hence, the synthesis of a (question, answer) tuple is needed.

labeling data to train a model in the new domain is expensive and time-consuming.


This paper presents a deep neural solver to automatically solve math word problems.

by using single layer LSTMs with 128 nodes and a symmetric window of length 3, our model achieves 99.1% accuracy.


In order to answer questions about temporal relations, it is not particularly harmful to introduce spurious events, but missing an event makes it impossible to answer any question related to it.

although our system was not optimized for such a paradigm, and this data is quite different in terms of both the annotation scheme and the evaluation method, we obtain state-of-the-art results on this corpus as well.


The model is factored such that the bottom LSTM layer receives only language input, consisting of the embedded previous word.

if the constraints are non-trivial (i.e.


Visual Captioning: Most recent approaches to image captioning (Vinyals et al., 2015b;Xu et al., 2015) have used CNN-LSTM structures to generate descriptions.

we evaluate the precision and recall of our generated summaries (output by the photo selector) compared to human selections (the combined set   of both human-selected 5-photo stories).


During training, due to the GPU memory constraints, we unfold the LSTM cell 16 times.

1), and the largest live-streaming platform for video games.


Verbs can only be used with a few specific arrangements of their arguments (syntactic frames).

figure 4a shows that VerbNet's 3 categories predict differences in the mean coercion ratings of verb-frame pairs (f = 43.46, p < 0.001).


Barak et al., 2016;Ambridge and Blything, 2015;Barak et al., 2014;Parisien and Stevenson, 2010;Perfors et al., 2010), respectively.

the range of compatibility judgments is highly variable across all three categories, and BHC's finer-grained predictions fail to account for much of this variability.


The multi-modal models also had problems translating the German "trinkt Shots" (drinking shots).

multi-modal MT has only recently been addressed by the MT community in the form of a shared task .


We model the agent policy π with a neural network.

while this process may introduce noise for instructions that specify specific trajectories (e.g., move SRI two steps north and .


The reward function where m (i) is the length ofē (i) .

to better understand the problem, we also report results for the decomposed task with a planner.


It was previously thought that during reading aloud, the semantic level of information did not need to be activated, but rather letters could be matched directly to phonemes and then articulated.

the eye-tracking experiments were carried out in a quiet lab environment.


In , CAEVO was reported on the straightforward evaluation metric including the vague TLINKs, but the temporal awareness scores were used here, which explains the difference between line 11 in Table 4 and what was reported in .

we have chosen CAEVO as the baseline system to evaluate the significance of the proposed ones.


Both the official TAC-KBP evaluation and the on-demand evaluation we propose use microaveraged precision and recall as metrics.

we only con- Figure 3: Median pooling bias (difference between pooled and unpooled scores) on the top 40 systems of TAC KBP 2015 evaluation using the official and anydoc scores.


sider instances selected in the first part of this process.

existing datasets (Angeli et al., 2014;Adel et al., 2016) have relied on the output of existing systems, making it likely that they exhibit the same biases against novel systems that we've discussed in this paper.


It then selects the top k = l d /n windows by averaging similarity and discards all other terms in the document.

the DUEt model (Mitra et al., 2017) is a hybrid approach that combines signals from a local model for relevance matching and a distributed model for semantic matching.


For example, as a pioneering work, MatchPyramid is mainly motivated by models developed in computer vision, resulting in its disregard of certain IR-specific considerations in the design of components, such as pooling sizes that ignore the query and document dimensions.

owing to space constraints, we omit the plots for PACRR-firstk.


Specifically, a model's retrieval quality can be examined by checking across a range of individual document pairs, namely, how likely a model can assign a higher score for a more relevant document.

for example, given the query "dog adoption requirements", unigram matching signals correspond to the occurrences of the individual terms "dog", "adoption", or "requirements".


Figure 3: The ERR@20 of re-ranked QL with PACRR-kwindow when applying different hyperparameters: l d , n s and l g .

we end up with l g − 1 matrices C n lq×l d ×n f , and the original similarity matrix is directly employed to handle the signals over unigrams.


First, as a cognitive model, it relates the kinds of learning biases used in previous work to the wider literature on working memory.

finally, we find that, despite pre-training, the system may settle into an initial state where the phonological network simply embeds the characters and the utterance network learns a character LM.


For instance, compare the top left subplot, with 0 dropout and good performance with H p = 20, H u = 100, to subplot 3,3, with optimal performance at H p = 80, H u = 200.

for values in between, however, the lines are relatively far apart, showing that the real words are easier for the network to remember.


The only difference was that we measured the similarity between the two brain activation vectors using Pearson's correlation coefficient, following Anderson et al.

(2008), we also compare the models in their ability to make accurate predictions when the two test words are exemplars of the same semantic category 11 .


Our main contributions are as follows: • We design an MMS method that can automatically generate a textual summary from a set of asynchronous documents, images, audios and videos related to a specific topic.

with the help of  the local community, MSF's   emergency  teams  focus on  searching.


Equations 8,9 and 10 are all monotone submodular functions under the budget constraint.

when summarizing with textual and visual modalities, performances are not always improved, which indicates that the models of image caption, image caption match and image alignment are not suitable to MMS.


A spoken opinion such as "I think it was alright .

table 6 shows examples taken from the CMU-MOSI dataset.


Additionally, we report the average log-loss (the negative log-likelihood, according to the classifier, of the true label).

in our dataset, every item is labeled in 6 conditions (every |C i | = 6), and within every context, every item is labeled by at least 3 annotators (every |A c i | ≥ 3).


We ask an-notators to determine whether a given Twitter user supports Donald Trump or Hillary Clinton.

the first contribution of this paper (Section 3) is to show that, at least for political stance detection on twitter, displaying different kinds of context to annotators yields significantly different annotations for the same user.


Item i has feature vector X i and a "true" label Y i ∈ V .

researchers have provided a range of contexts (or more broadly, information conditions) to annotators in an attempt to balance annotators' exposure to the data needed for accuracy with reasonable costs in terms of time, money and cognitive load.


This tests whether simply modeling differences in annotators' error rates, as Raykar et al.

throughout this work the labels we use are taken from the set {"Support Trump / Oppose Clinton" = −1, "Neutral / I don't know" = 0, "Oppose Trump / Support Clinton" = 1}.


We rewrite (7) for the two conditions: 1. w has one meaning: 2. w has two meanings: With some algebraic manipulations, it can be shown that M (1) > M (2) if the following holds: Thus when (8) holds, the average cosine distance between two samples of a certain word w gets larger as w acquires more meanings.

here we validate the use of the shuffled condition as a proper control.


word2vec (Mikolov et al., 2013) -perhaps the most popular model of word meaning representation, implicitly factorizes the values of the wordcontext PMI matrix.

the same analysis is repeated in the shuffled condition, see Fig.


(Blei et al., 2003;Mikolov et al., 2013), have inherent biases.

since semantic change may result from the interaction of many factors, small effects may be a priori more credible than large ones.


We derive real-valued demographics and personality scores using the models introduced in section 3.1.

our work builds on these ideas and explores the general task of user-factor adaptation.


For example, in the sentence: (a) The two justices have been attending federalist society events for years, our model correctly disambiguated justices with the WordNet sense justice 3 n (public official) rather than justice 1 n (the quality of being just), and the corresponding softmax distribution was heavily biased towards words and senses related to persons or groups (commissioners, defendants, jury, cabinet, directors).

a sketch of our bidirectional LSTM tagger is shown in Figure 1.


10 All these supervised systems were trained on the standardized version of Sem-Cor.

in the sentence: (b) Xavi Hernandez, the player of Barcelona, has 106 matches, the same model disambiguated matches with the wrong WordNet sense match 1 n (tool for starting a fire).


we define the word context of a specific embedding of a year y for a word w i , to be {w c ∈ C y i }.


In this work, we leverage the temporal variation in word relatedness to understand, and better accommodate, intent.

most popular search engines return results about Sharon's death in 2011, when he was no longer prime minister.


Temporal relatedness learning can be used for various NLP and IR-related tasks.

we need to detect peaks, as well as periods of continuous peaks (i.e., steps).


We would also like to thank the anonymous reviewers for their helpful comments.

in summary, our contributions are as follows: 1.


Since we have to analyse a pair of sentences, the weights should be related to not only the sentence itself, but also its partner.

ablation experiments (Mueller and Thyagarajan, 2016) illustrates a 0.04 degradation of Pearson's r without data argumentation strategy.


In our opinion, It is also possible to solve the problem put forward by Levy et al.


As shown in (Fu et al., 2014;Li et al., 2015;Wang and He, 2016), learning Chinese is-a relations is fundamentally challenging due to flexible language expressions.

additionally, other combinations of vectors are mentioned in the literature, such as vector sum x + y, and dot-product x • y .


Several issues in current benchmarks and methods have already been pointed out by previous works.


If the search succeeds, then the semantic parser parameters are updated as usual.

the result is a lexicon where each entry contains a logical form template and a set of possible phrases for triggering the template.


Orthogonal Matrix Factorization (OrMF) (Guo et al., 2014) is the extension of WMF, with an additional objective to obtain nearly orthogonal dimensions in matrix factorization to discount redundant information.

samsung's Galaxy Note 7 Phones AND replacement phones have been going up in flames due to the defective batteries  Table 4: A representative set of tweets linked by a URL in streaming data (generally poor readability).


We compared the quality of paraphrases extracted by our method with the closest previous work (BUCC-2013) (Xu et al., 2013), in which a similar phrase table was created using Moses from monolingual parallel tweets that contain the same named entity and calendar date.

for the (vec) version, vectors of a pair of sentences v 1 and v 2 are converted into one feature vector, , by concatenating the element-wise sum v 1 + v 2 and absolute differ- We also provide the (sim) variation, which directly uses the single cosine similarity score between two sentence vectors.


We denote the state vectors of the decoder RNN as (d 1 , d 2 , .

it is the knowledge of paraphrasing, at lexical, syntactic, and semantic levels, that will be transferred across domains.


Perfeature standardization does not preserve cosine similarity, nor does it fix the large macro variance problem.

for the example in figure 1, the paraphrase model may have learned the mapping from "play for" to "whose team is" in a source domain.


We expect to see many of these ideas to be applied in the future.

every logical form will be converted into a unique canonical utterance deterministically (Figure 1).


In order to enforce structural consistency, most existing work applies different types of structural constraints during inference.

the performance gain introduced by our integrated model is relatively small compared to our FrameNet results.


For example, since we generalize all the predicates to a compact form <pred-xx>, for irregular verbs like "became" ⇒ become-01, simply stemming the inflected verb form will not give us the correct concept even if the sense is predicted correctly.

it is non-trivial to formalize concept identification as a sequence labeling problem because of the large concept label set.


Since the code from (Yang et al., 2015) is available, we use it (rather than (Jurczyk et al., 2016)) to assist our analysis.

as we will show using state-of-theart Multiple Instance Learning (MIL) algorithms in Section 4, P 2 by itself is currently a very challenging task, partly because of the difficulty of extracting features from a set of candidate answers that are effective for answer triggering.


As such, despite their drawbacks, word associations may provide a more flexible, more holistic view of mental semantics.

it is vastly outperformed by even cosine similarity on the same set of embeddings.


At the sentence level, Long-Short Term Memory Networks (LSTMs; Hochreiter and Schmidhuber 1997) learn representations for EDUs and larger constituents, whereas at the document level, LSTMs learn representations for entire sentences.

given a constituent sequence C, the probability of forming the structure sequence Y str is written as: where the u str t represents the unary potential score of structure node l t = y t , depending on constituents c t and c t+1 .


We also perform multi-class classification on PDTB and CoNLL-2016.

our method captures both the discourse relationships through interactions between discourse arguments and the complementary knowledge through interactions between annotated and unannotated data.


Figure 5 shows the performance of four binary classification on four top level classes influenced by different share weights w in Wshare multi-task framework.

under the framework of multi-task and gated mechanism, the main and auxiliary tasks are capable of not only sharing the parameters of learning argument pair representation but also interacting the representations learning from each other.


This work was supported by the Swedish Research Council under project 2012-916 Discourse-Oriented Statistical Machine Translation.

a * indicates significance at p < 0.001 using McNemar's χ 2 test.


Feature ablation tests revealed that while combining all features is beneficial for the prediction of the anaphoric and pleonastic classes, the same is not true for the event class.

since the two models have different strengths, in future work we plan to enrich the training data with re-training instances from the silver data where the two systems agree, in order to reduce the amount of noise, following the example of Jiang et al.


In this paper, we address the problem of predicting one of three functions for the English pronoun 'it': anaphoric, event reference or pleonastic.

the inclusion of semantic features, which we designed as indicators of eventness, appears to be harmful (Figure 1).


Work on the automatic acquisition of selectional preferences has shown considerable progress (Dagan and Itai, 1990;Resnik, 1993;Agirre and Martinez, 2001;Pantel et al., 2007;Erk, 2007;Ritter et al., 2010;Van de Cruys, 2014).

the increase in the precision error is the highest for common nouns.


We now investigate the effect of incorporating selectional preferences, implicitly and explicitly, in coreference resolution.

they are mainly modeled only implicitly by current coreference resolvers.


an attribute a (e.g., obj): (i) grow right -add a as another term in the pattern (i.e., [noun][obj]); (ii) grow inside -add a as another attribute to the last term of p, making it more specific (i.e., [noun,obj]).

next, we define the alphabet of attributes that will be used to compose longer patterns (lines 2-3).


Similar to (O'Mahony and Smyth, 2010), we consider a review as helpful if and only if at least 75% opinions for the review are positive, i.e.

we observe that the precision of U-GR+AF, although gives the second highest score among all feature combinations, is lower than that of UGR; we leave it for future work.


Up to this point, we focused on the task of extracting links between ACs.

this work presents the first neural network-based approach to link extraction in argument mining.


This extra layer seems to be crucial for improving performance on this task.

we refer to encoding hidden layers as e, and decoding hidden layers as d. The PN uses a form of content-based attention (Bahdanau et al., 2014) to allow the model to produce a distribution over input elements.


Our lat2seq framework uses this network as encoder, computing the attention over all lattice nodes.


This uses the phrase table produced by SMT to improve NMT.

this seems to be counterintuitive, as intuitively learning general rules should be the first step, rather than first memorizing special cases and then learning general rules.


When we use the LM trained on extra monolingual data, we get total improvements of 1.0% BLEU and 2.3% BLEU with the hybrid approach.

the first row shows results with all models enabled.


In the training phase, given a parallel sentence (x, y), NMT models the conditional probability as follows, where y i is the target word emitted by the decoder at step i and y <i = y 1 , y 2 , ..., y i−1 .

to explore the phrase generation in NMt beyond the word-byword generation framework, we propose a novel architecture that integrates a phrase-based SMt model into NMt.


In contrast, phrases are much better than words as translation units in SMT and have made a significant advance in translation quality.

the training corpora consisted of about 1.25 million sentence pairs 3 with 27.9 million Chinese words and 34.5 million English words respectively.


Furthermore, in order to enhance the ability of the NMT decoder to effectively select appropriate target phrases, we modify the encoder of NMT to make it fit for exploring structural information of source sentences.

if the model selects the word generation mode, it generates a word by the NMT decoder as in the standard NMT framework.


Experiment results show that the proposed model significantly outperforms the conventional attention-based NMT by 1.07 BLEU points on multiple NIST test sets.

phrases are much better than words as translation units in SMT and have made a significant advance in translation quality.


While some work (Luong et al., 2015b;Luong and Manning, 2016;Wu et al., 2016) has achieved state-of-the-art results using deep networks, others (Jean et al., 2015;Chung et al., 2016;Sennrich et al., 2016b) have produced similar results with far shallower ones.

we hope that this allows us to isolate the effect of various hyperparameter changes.


We construct a new test set (CROWDTEST) for a larger set of 27 languages from crowdsourced dictionaries (Pavlick et al., 2014).

bPR optimizes (Rendle et al., 2009): where is the logistic sigmoid function, ⇥ is the parameter vector ofx u,i,j to be trained, and ⇥ is its hyperparameter vector.


The encoder and the decoder are unidirectional single-layer LSTMs.

as more variance is injected, the models degrade quickly but still improve from the pre-trained models.


These attractive properties were not studied in A2C's original paper (Mnih et al., 2016).

in any realistic scenario, human feedback will vary from its average, and so the reward that our algorithm receives will be a perturbed variant of sentence-BLEU.


Numerous approaches have been proposed for compression and inference speedup of neural networks, including but not limited to low-rank approximation (Denton et al., 2014), hash function , knowledge distillation (Hinton et al., 2015), quantization (Courbariaux et al., 2015;Han et al., 2016;Zhou et al., 2017) and sparsification (Han et al., 2015;Wen et al., 2016).

interpolated Sample: replace the official said the grenade explosion did not cause any casualties or damage .


The speedup of vocabulary selection is only about 30%.

we edit S toward Y , which can be seen as interpolation.


Besides this, we also tried to learn the batch weighting dynamically.

for the batch weight tuning, one way is to fix the weights for several systems and select the best-performed system on the development data.


The proposed methods were evaluated by adapting WMT corpora to IWSLT (mainly contains TED talks) corpora.

nMT was trained with 0% in-domain data at first and then with batch weighing method for further training (Luong + bw in Table 4).


(2017) which investigates a variant of it for continual learning (learning a new task while preserving performance on previously learned task) rather than domain adaptation.

our baseline fine-tuning models use the same hyperparameters, except that the learning rate is 4 times smaller and the validation frequency for early stopping 4 times higher.


(a) Baseline model with a single-layer decoder RNN and attention (b) Addition of a source-independent LM layer that feeds into the source-dependent decoder (c) Multi-task setting next-word prediction from both layers; green softmax layers are shared.

the use of synthetic parallel data still outperforms our approach both in single and ensemble systems.


(2017) and in most previous work, we process individual predicates in isolation, so for each predicate, our task reduces to a sequence labeling problem.

as the semantic representations are closely related to syntactic ones, we exploit syntactic information in our model.


Entity Linking Identifying the entities mentioned in a question is a critical subproblem of semantic parsing in broad domains and proper entity linking can lead to large accuracy improvements (Yih et al., 2015).

we use wIKITABLEQUESTIONS to evaluate our parser as this data set exhibits both a broad domain and complex questions.


Our semantic parser extends recent neural semantic parsers by enforcing type constraints during logical form generation, and by including an explicit entity embedding and linking module that enables it to identify entity mentions while generalizing across tables.

many of these logical forms are spurious, in the sense that they do not represent the question's meaning.


For the CCG grammar, we manually compile a domain lexicon containing a list of trigger words mapped to their syntactic categories and associated logical predicates.

while this is not a conditional likelihood Predicate Description and evaluation stringVal Returns string value corresponding to a text span in the statement getPhraseMention Looks for matching tokens or phrases in a target text, and return true if an exact match is found.


The second term decouples over evaluations of individual statements In turn, since we never observe the correct interpretation l for any statement, but only model its evaluation z j , we marginalize over all interpretations whose evaluations in a context x matches z j (similar to Liang et al.

humans can efficiently learn about new concepts and phenomena through language.


Therefore we aggregate an entity's mention-level labels with a deterministic disjunction: At test time, z i is latent.

populating an entity-event database: From a corpus of news articles D (test) over timespan T , extract the names of persons killed by police during that same timespan (E (pred) ).


We further show that the types of questions specific legislators tend to ask vary with whether they are part of the governing or opposition party, consistent with wellestablished accounts of partisan differences (Cowley, 2002;Spirling and McLean, 2007;Eggers and Spirling, 2014).

q: Will the Government now apologise for their complacent decision to scrap the future jobs fund, [given that] long-term youth unemployment is rising?


Consequently, scholars have studied parliamentary questions in detail, in terms of their origins (Chester and Bowring, 1962), their institutionalization (Eggers and Spirling, 2014) and their importance for oversight (Proksch and Slapin, 2011).

sachdeva and Kumaraguru (2017) studied another rhetorical aspect by examining linguistic attributes distinguishing serviceable requests addressed to police on social media from general conversation.


Richer linguistic representations, as well as more judicious ways of weighting different fragments and motifs, could enable us to capture a wider range of possible surface and rhetorical forms, especially in settings where phrasings are potentially less structured by institutional conventions.

if two questions are phrased differently but answered in similar ways, the parallels exhibited by their answers should reflect commonalities in the askers' intentions.


All these approaches have been largely domainspecific and rely on a small set of labelled data for supervised model learning.

depending on the type of word, we follow a different generative process.


Assuming an asymmetric Dirichlet prior for x is parametrized by γ = [γ b , γ z , γ a ] for background, topic and argument words, it is modified by a transformation matrix λ, γ new = λ×γ , where λ is defined by: The conditional probability for the switch variable x is modified by simultaneously considering the POS tag g for the word at position t: where an additional term is added to the RHS of Equation 1.

cPT mixed perspectives about Israel/Palestina and other viewpoints about GcSE and mortgages.


For instance, (a) in Table 1 strongly asserts the win of Natalie Portman over Meryl Streep, whereas (b)   uncertainty.

the mind" in the first example is not trivial to capture.


Naively applying the original deep memory network or convolutional network for emotion cause extraction outperforms all the baselines except the convolutional multi-kernel method.

in our model, the training epochs are set to 20.


the weight of word w i in the i-th position considers both the previous word w i−1 and the following word w i+1 by a convolutional operation: For the first and the last word in a clause, we use zero padding, w 0 = w k+1 = 0, where k is the length of a clause.


Existing work in emotion analysis mostly focuses on emotion classification  and emotion information extraction (Balahur et al., 2013).

we can obtain the precision, recall, and F-measure by comparing the proposed keywords with the correct keywords by: Since the reference methods do not focus on the keywords level, we only compare the performance of Memnet and ConvMS-Memnet in Table 6.


training on multiple datasets has shown promising results (Collobert and Weston, 2008).

we only save a single tweet for the pretraining per unique emoji type regardless of the number of emojis associated with the tweet.


We leverage this advantage in building our model.

multi-task learning has been recognised as a strength of neural network models for natural language processing (Collobert et al., 2011;Henderson et al., 2013;Zhang and Weiss, 2016;Chen et al., 2016a), where hidden feature layers are shared between different tasks that have common basis.


By utilizing global features and linguistic priors, their approach achieves stateof-the-art performance.

it can be seen that our method achieves the best results on average.


We evaluated our model on seven languages from the PASCAL Challenge on Grammar Induction (Gelling et al., 2012).

their approach uses an approximate parsing algorithm, which has no theoretical guarantee.


Given a word, we consider a fixed size window w (another hyperparameter) of words around it.

some researchers have attempted to address the limitations of (Chen and Manning, 2014) by augmenting it with additional complexity.


We find that using a recursive model with a memory cell improves results by 1.12 F-score, and thus we settle for the TreeLSTM composition function.

the initial configuration consists of a buffer filled with words and an empty stackpresented as the axiom in Figure 1.


The second approach-approximation on the parsing algorithm-consists of an approximate search for the most probable parse.

as an example, in Figure 2a word w1 should shift and swap many times to get to word w5 in order to construct constituent C. a Lazy oracle would postpone swapping until constituent B is built so that only one swap operation over node B would be enough for word w1 to get to word w5.


When l is small (l < 4), the general trend is that larger l leads to better result.

as shown in Table 3, the POS-tag information plays the most important role in our model.


In this paper, we propose a new learning strategy that learns a generative model and a discriminative model jointly based on the dual decomposition method.

due to the strong independence assumption in most generative models, it is difficult for these models to utilize context information that has been shown to benefit supervised parsing.


Once equipped with a coarse model, we use it for search reduction in the following manner.

these fast-tracked candidates completely bypass competition with potentially high-scoring OPEN or CLOSE successors, allowing for higher-quality results in practice with minimal overhead.


Most relevant to this work are structured prediction models capturing dependencies between decisions, either by modeling the dependencies between the hidden representations of connected decisions using RNN/LSTM Katiyar and Cardie, 2016), by explicitly modeling the structural dependencies between output predictions (Durrett and Klein, 2015;Lample et al., 2016;Andor et al., 2016), or by combining the two approaches (Socher et al., 2013;Wiseman and Rush, 2016).

in our model, illustrated in Figure 1b, the encoder is a CRF model with neural networks as its potential extractors, while the decoder is a generative model, trying to reconstruct the input.


The one exception is the treatment of long distance whmovement in TAG.

normally, in dependency parsing, PP attachment is resolved by the parser.


For the supertagger, each direction of LSTM computation involves two layers, and each LSTM contains 512 units.

in addition, it has long been observed that the derivation tree can also be interpreted linguistically as a dependency tree (Rambow and Joshi, 1997), if certain assumptions are made about the shape of the elementary trees in the grammar (Frank, 2001).


Finally, we concatenate the vectors associated with the relevant elements from the stack and buffer into a 2dk dimensional vector and feed it to the network to obtain a probability distribution over the possible transition actions.

building a robust, broad-coverage TAG parser has proven difficult.


We also obtain comparable accuracy of 89.44% on Section 23.

the results of partial derivations are not identified with atomic lexical entries, resulting in sparser data.


We show several topperforming systems in the table as well, where M&B (2016) refers to Miwa and Bansal (2016), who exploit end-to-end LSTM neural networks with local optimization, and L&J (2014) and M&S (2014) refer to Li and Ji (2014) and Miwa and Sasaki (2014), respectively, which are both globally optimized models using discrete features, giving the top F-scores among statistical models.

we verify this by examining the sentence-level accuracies, where one sentence is regarded as correct when all the labels in the resulted table are correct.


At the ith step, we determine the label l i of the next table slot based on the current hypothesis T i−1 .

this demonstrates the strength of neural representation learning for end-to-end relation extraction.


It consists of two sub-tasks, namely entity detection, which recognizes valid entities, and relation classification, which determines the relation categories over entity pairs.

the method of Miwa and Bansal (2016) must consider tree LSTM formulations that are specific to grammar formalisms, which can be structurally different (Tai et al., 2015).


We show the sizes of all dataset variants used in our experiments in Table 1.

via dual tensors, we explicitly capture the function that transforms unspecialized embeddings to specialized embeddings that are better suited to detect the asymmetric relation of interest.


In the first example, the test sentence does not directly express the relation spouse, the proof of this relation appears in a further context in NYT.

we keep the words which appear more than 100 times in the corpus as vocabulary.


We assume that we can make a prediction about (h, t), once we know the information of (h, e) and (e, t).

then, the similarity o i is calculated as follows: if r i gets more similar to (r A + r B ), the conditioned predicting probability of r i will become larger.


There are also other works for regularizing classifiers by adding random noise to the data, such as dropout (Srivastava et al., 2014) and its variant for NLP tasks, word dropout (Iyyer et al., 2015).

loss Function: For an entity pair, we compute the probability of relation r by P (r | X; θ) = softmax(As r +b), where A is the projection matrix and b is the bias.


Hence we only apply dropout to s i for PCNN.

the statistics of the two datasets are summarized in table 1.


We feed the concatenated vector to the softmax layer in Eq.

later, Miwa and Bansal (2016) have implemented an end-to-end neural network to construct a context representation for joint entity and relation extraction.


Case study shows that soft-label corrections are of high accuracy.

most bags which are soft-labeled as NA are still welllabeled in our experiments.


2 Sequences: bi-directional LSTM based models considering all combinations of two sequences taken from POS tags, dependency and word forms sequences.

baseline III: a neural network classifier based on event embeddings for both event mentions that were learned using bidirectional LSTMs (Kiperwasser and Goldberg, 2016).


Early works on temporal relation classification Mani et al.

our approach first extracts a sequence of context words that indicates the temporal relation between two events, which well align with the dependency path between two event mentions.


for the temporal relation dispute after rel 3 buildup, the context between the two events is long, complex and involves another event (invasion) as well, which makes it challenging for any individual feature or feature combinations to capture the temporal relation.


The generalization ability of feature based prediction models will be badly hurt, when the features do not match between testing and training.

in the algebra view of likelihood, variance parameter is just the weight of training instance (i.e., the interaction between "entity pair and feature" or "entity pair and label"), measuring the importance for its contribution to the total likelihood.


The θ is obtained from the stickbreaking process (Blei and Jordan, 2004), with where β k are independent draws from the beta distribution β(1, α).

in this mechanism, for each y i,j in noisy corpus, it allows 1 → 0 (noisy feature) for features, and allow 1 → 0 (label with no supportive features) or 0 → 1 (incomplete label) for labels.


We evaluate our method on two widely used datasets as shown in Table 1 with the same setting in (Fan et al., 2014).

the corresponding relation instance (Senate-of(Barack Obama, U.S.)) is missing in the knowledge base.


Our aim is to find out whether the Location-Insurgent projection learned on the first model is able to reveal conflicts that appear in 2001.

then, we used the 137 Location-Insurgent pairs derived in Section 2 to learn a projection matrix from the embeddings for locations to the embeddings for insurgents.


This means we will have one model saved after sequential training for the years up to 2000, and one saved after year 2001.

as CBOW proved to consistently outperform Skipgram for our tasks, we only report results for CBOW, due to limited space.


In our case we have multiple unitary matrices U = (U 1 , ..., U p ).

absolute Phone Error Rate is not so important here and it can be certainly possible to devise recognisers with better performances by applying more sophisticated techniques.


Turn Encoder: We further encode the sequence {h y 1 , h x 1 , ..., h y i , h x i } with another LSTM encoder.

rather than a sequence of vectors, we now operate over a table.


Let p(w|c) be the probability of a word w given a context c that represents its entire preceding context, and let p(w) be a 'noise' word distribution (e.g.

we show that with minor modifications to word2vec's algorithm, we get principled language models that are closely related to the well-established Noise Contrastive Estimation (NCE) based language models.


ported by the following experiment: We took word embeddings produced by both models on the English PTB, and applied PCA to them.

they do not consider syllables, and they experiment with relatively small models on small data sets (0.6M-1.4M tokens).


In this work, we encode semantic features into convolutional layers by initializing them with important n-grams.

none of them outperforms the others on all datasets.


In textual data, the features extracted by CNNs are n-grams.

we respectively use uni, bi and tri-gram centroid vectors to fill the filters.


CNN-non-static 1 (short for CNN) proposed by Kim (2014) is used as our baseline, which consists of one embedding layer, one convolutional layer, one max pooling layer, and one fully connected layer.

in fields like computer vision, features extracted by filters are more difficult to interpret.


Let G = (V, E) be an undirected and unweighted graph consisting of a set V of vertices and a set E of edges between them.

a path is a walk without repetition of nodes.


Among the three tree prior construction algorithms, the two-level is the best on the 20News-Groups corpus.

6 We also include the features of BOW and the average word vector for the document (BOW+VEC).


Their removal led to a correlation reduction of 10.2% (rounded) and 6.2% (continuous) relative to using all features.

spam and malicious workers were identified during data collection using a filtering algorithm that compared annotations with those completed by "potentially good annotators" (P GA).


To further complicate matters, some workers may produce spam or malicious responses.

we posit that although annotator quality is an important factor in predicting accurate aggregations, the interplay between it and other factors is more nuanced.


Annotations indicate the degree of emotion in an emotion-headline pair (Affect (Emotion)) and the overall positive or negative valence of a headline (Affect (Valence)).

it is desirable to devise more effective aggregation strategies that consider factors such as label distribution and worker quality, while still avoiding manual adjudication of all instances.


The study was conducted in a span of three weeks.

a task thus is defined as follows: Definition 1 annotation Task: a task consists of a sentence, a human readable question regarding a predicted label, and a set of answer options.


Typical tasks include selecting answers from a set of candidates (Fossati et al., 2013), marking text passages that contain specific semantic roles (Feizabadi and Padó, 2014), and constructing question-answer pairs (He et al., 2015(He et al., , 2016.

as the correctness column in Table 6 shows, the selection of tasks by TaSKROUTER is more appropriate for the crowd in general.


These experiments confirm our initial postulation that it is possible to train a high quality classifier to detect expert-required tasks.

as  tors agreed in 1,801 out of all 2,549 tasks (71%).


The potential for leveraging multiple levels of representation has been demonstrated in various ways in the field of Natural Language Processing (NLP).

the second task is textual entailment, which requires one to determine whether a premise sentence entails a hypothesis sentence.


We focused on maximizing the accuracy of dependency parsing on the development data in our experiments.

the previous state-ofthe-art result in Yin et al.


The probability values are predicted by the POS layer, and thus no gold POS tags are needed.

in the target tasks used in our model, sharing the word representations and the output label embeddings is more effective than just stacking the bi-LSTM layers.


We can enjoy the best of both worlds by incorporating the merits of both approaches: First the WGAN approach locates a good neighborhood of the parameter space, and then, starting from a reasonable initialization, the EMDOT approach efficiently explores the neighborhood to achieve enhanced performance.

the use of Wasserstein GAN addresses this problem and allows our simple architecture to be trained successfully.


The EMDOT approach is attractive for several reasons: It is consistent for training and testing (the equivalence between the ground distance c = L 2 2 and cosine dissimilarity), compatible with the orthogonal constraint, mathematically sound (without much assumption and approximation), guaranteed to converge, almost hyperparameter free, and fast in speed (the alternating subprograms have either effective approximate solvers or closed-form solutions).

the typology dissimilarity is computed from features in the WALS database (Dryer and Haspelmath, 2013).


Linguistic universals manifest themselves at various levels of linguistic units.

we can try to find the transformation that minimizes the earth mover's distance.


Our main motivation for unfolding is to obtain a single network with ensemble level performance which can be shrunk with the techniques in Sec.

unfolding is an alternative to ensembling of multiple neural networks with the same topology.


A typical scenario of neural machine translation starts with training a model to maximize its log-likelihood.

as the quality of the critic's approximation of the decoding objective has direct influence on the actor's learning, we make ten updates to the critic before each time we update the actor once.


(2016b) propose hierarchical attention network to understand both attended words and sentences for sentiment classification.

for writing stylistic features, it is suggested that informative writing has more nouns, adjectives, prepositions and coordinating conjunctions, while imaginative writing has more verbs, adverbs, pronouns, and pre-determiners (Rayson et al., 2001).


we exclude these websites in this work because it requires experts to annotate the news articles.


[47] Sagan led home a group of four riders almost a minute behind, […] behind Sánchez.

the non-ML competitors introduce a minimum of ∆ w (CS) = 182%, with MRFs having the highest error.


Although there are over 5.5 million articles in the English Wikipedia, fewer than 0.2% have been evaluated by Wikipedians as good articles and around 92% have been evaluated as start or stub class articles, Wikipedia's two lowest quality categories.

moreover, edit types simply extracted from an edit is inadequate in outlining the correct intentions, for instance, adding a sentence could be Clarification, Elaboration, or Vandalism.


Even though the comments sometimes signal the editors' intents, they are not infallible, editors may fail to complete the comment field, may only label one of the multiple edit intentions for a single revision, or write comments that are inaccurate, irrelevant, or incomplete.

for example, syntactic edit type taxonomies cannot tell the difference between simplifying a paragraph and maliciously damaging that paragraph, since both involve deleting a sentence.


Similarly, Zanzotto and Pennacchiotti (2010) used edits as training data for textual entailment recognition, and Recasens et al.

the positive slopes for simplification, wikification and process suggest that, as articles increase in quality, simplifying articles' content, adding proper links or reorganizing their structure becomes more important.


We also evaluate the model in a semi-supervised setting by downsampling the WikiReading training set to create increasingly smaller amounts of supervision, while leaving the full unlabeled document corpus to train a sequence autoencoder on document windows.

table 6 shows the results of semi-supervised reviewer models.


We return immediately if the model's argmax prediction has 0 F1 score.

table 5 shows the results of evaluating the four main models on adversarial examples generated by running either ADDSENt or ADDANY against each model.


ADDSENT, our main adversary, adds grammatical sentences that look similar to the question.

6 The results of evaluating these models are shown in Table 6.


For example, 41.5% of BiDAF Ensemble successes had a 4-gram in common with the original paragraph, compared to only 21.0% of model failures.

high-precision paraphrase generation is challenging, as most edits to a sentence do actually change its meaning.


By modeling and exploiting heterogeneous knowledge during commonsense reasoning, our method can achieve more accurate and more robust performance than traditional methods.

compared with Narrative Event chain Model, our model achieves a 16.3% accuracy improvement by considering richer commonsense knowledge, rather than only narrative event knowledge.


Then we model how to choose inference rules for a specific reasoning context.

we mine three types of commonly used commonsense knowledge, including: 1)Event narrative knowledge, which captures temporal and causal relations between events; 2)Entity semantic knowledge, which captures semantic relations between entities; 3)Sentiment coherent knowledge, which captures sentimental coherence between elements.


We also remove annotations which are solely links to external resources, and do not provide useful textual annotations.

we analyze the performance of translation and retrieval models on this task, measuring performance with both automated and human evaluation.


The task of evaluating lyric annotations was difficult for CrowdFlower collaborators as was apparent from their evaluation of the task.

in ALA the system must often include additional information to clarify niche terminology and abstract concepts.


In fact, the notion of "attention" has gained popularity recently in neural network modeling, which has improved the performance of many tasks such as machine translation (Bahdanau et al., 2015;Luong et al., 2015).

let a be the magnitude of the columns in A o and a ∈ R m .


Based on the parsed constituency tree for each input sentence as described in Section 2.3.1, we extract the noun-phrases (NPs) and verb-phrases (VPs).

we use a weighted combination of the context vectors and the original outputs with λ c ∈ [0, 1].


Automatic summarization is the process of automatically generating a summary that retains the most important content of the original text document (Nenkova and McKeown, 2012).

the deterministic decoder hidden state h d t is calculated using two layers of GRUs.


We employ Variational Auto-Encoders (VAEs) (Kingma and Welling, 2013;Rezende et al., 2014) as the base model for our generative framework which can handle the inference problem associated with complex generative modeling.


StanD: new zealand women are having more children birth rate hits highest level in ## years.

the standard decoder StanD ignores the latent structures and generates some loose sentences, such as the results for S(1) "Results of men's volleyball at Chinese city games" does not catch the main points.


For the financial report corpus, we used financial reports published every quarter during 2013-2016 by Japanese exchange listed companies.

sentences related to good revenue performance are extracted.


News summaries in previous studies are usually very short, and most of them consist of about one or two hundred words.

the average number of words of Wikinews in the test set is 598 and the average number of total words of their source news articles is 2136.


(2012), and furthermore, in end-to-end parsing, the superiority of our parser is more pronounced comparing to the previously best parser of Hernault et al.

the majority of the conventional discourse parsers are not end-to-end: they rely on gold EDU segmentations and pre-trained tools like Stanford parsers to generate features.


WD Classifier: the neural network based WD classifier essentially inherits the features that have been shown effective in previous event coreference studies (Ahn, 2006;, including both features for event words and features for their arguments.

this is especially common among reporting event mentions and light verb mentions.


This provides us with a system that can talk about almost anything, being able to respond to most prompts, but in a recognizable Star Wars style.

it is difficult to directly decode with objective P (T |S)/P (T |∅), because this objective removes the influence of the target-side language model.


Answers question is used as source S, and its answer is treated as target T .

(See section 7.3) we use the selective-sampling method to generate candidates for all our experiments; this (a) samples stochastically then (b) selects using a learned objective from data.


In contrast, we propose to condition the next label on a predicted distribution of the current label.

there have been many works on DA classification applied to these two datasets; some focus on textual data (Kalchbrenner and Blunsom, 2013;Stolcke et al., 2000), while others explore speech data (Julia et al., 2010).


kalchbrenner and Blunsom (2013) model a DA sequence with a recurrent neural network (RNN) where sentence representations are constructed by means of a convolutional neural network (CNN); Ji et al.


Training is formalized as a game in which the generative model is trained to generate outputs to fool the discriminator; the technique has been successfully applied to image generation.

for each partially-generated sequence Y t = y 1:t , the discriminator gives a classification score Q + (x, Y t ).


Regarding the effect of context, in general there is slightly more gain when more context is used, as in BLSTM, or larger windows in CNN.


and the locutionary act of an utterance, which includes the ostensible surface-level meaning of the words.

accuracy of Da recognition in terms of homogeneity, completeness, and v-measure on both corpora is summarized in Table 5.


(2011)'s model, individual reply structure paths from the first utterance to terminal utterances are teased apart into separate sequential conversations by duplicating utterances.

one of the reasons is that the DA of an utterance depends not only on the preceding DA, but also on the speaker's personal style (Appling et al., 2013) or preferences for certain DAs.


The higher the η value, the more words are likely to be generated from a foreground topic, and thus the more function words are included in foreground topics, leaving background topics with content words.

learned topics tend to be characteristic of short sentences that are similar enough to form the separate topics, and as a result, long sentences may be assigned the same topic regardless of the DA actually performed.


Table 3: The explicit introducing-content cases of our HGFU model.

also, the Stochastic Language Generation in Dialogue method (Wen et al., 2015a) adds additional features in each gate of the neural cel- In open domains, Xing et al.


As for such content-introducing method, there are two aspects that need to be taken into consid-   Wen et al., 2015a,b;Xu et al., 2016).

(2015b) apply an additional control cell to gate the dialogue act (DA) features during the generation process to ensure the generated repliesexpressthe intended meaning.


We observe that the switch corresponds with the heat map: the generated word is more correlated with the cue word when the switch is open.

the cue word added on the first step of the neural networks can still influence the generation of the later steps.


Our Risk Index integrate these two contents of riskiness.

the teacher can choose among three teaching strategies corresponding to different configurations of switches in a wiring diagram as Figure 1 shows: the left switch is a Single-Pole, Double-throw (SPDt) switch, which controls whether the answer is made by the system (connected to 1) or given by the teacher as an example (connected to 2).


In this paper, we propose two scalar criteria: Risk Index (RI) and Hitting Time (HT).

the teacher-initiated approaches require the constant long-term attention of the teacher to monitor the dialogue process (Torrey and Taylor, 2013;Amir et al., 2016), which is costly and impractical for real applications.


Our evaluation set is therefore not from the same distribution as our training set.

the evaluation itself is done by human raters.


In the seq2seq approach, the decoder network therefore only has to keep track of where it is in the output, and the content to generate can be transformed from the relevant parts in the source via the attention mechanism .

the full conditional probability follows from the product rule, as: We propose to implement target-side attention by augmenting the attention mechanism to include the part of the target sequence already generated, i.e., we include y [0:i−2] in the arguments to the attention function: Attention(h i−1 , y [0:i−2] , x).


The second example has y 3 , y 4 , y 5 as input of the decoder and y 4 , y 5 , y 6 as the output of the decoder, and so on.

we found that our approximation works better in terms of N-choose-1 accuracy (see Section 5.2), which suggests that its value may be closer to the true conditional probability.


(Sordoni et al., 2015;Vinyals and Le, 2015).

the semantics of the utterance to be generated.


We note that while interactional synonymy relations in dialogue should be accounted for by semantic grammars or formal models of dialogue structure (such as DS-TTR (Eshghi et al., 2012), or KoS (Ginzburg, 2012)), lexical synonymy relations have to be learned from data.

for direct comparison, and for simplicity of exposition, we do the same here: we apply the method described for creating a user simulation (section 3.2.1), this time for the system side, resulting in a 'system simulation'.


(Napoles et al., 2016), once they become publicly available.

following previous work (Vedantam et al., 2015;Kilickaya et al., 2017), we mainly concentrate on WBMs.


The Stanford parser score is not designed to measure grammaticality, however, it will generally prefer a grammatical parse to a non-grammatical one.

• Quality: How do you judge the overall quality of the utterance in terms of its grammatical correctness and fluency?


In Example 1, LOLS generates a grammatically correct English sentence, which represents the meaning of the MR well, and, as a result, this utterance received high human ratings (median = 6) for informativeness, naturalness and quality.

in general, we find consistent differences in inter-annotator agreement per system and dataset, with lower agreements for LOLS than for RNNLG and TGEN.


We can train with the following reconstruction loss for a particular b i : x∈{e,m,t} where p k is the k'th predicted distribution over records, and where we have modeled each component of r independently.

in the late 2000s and early 2010s, a number of systems were proposed that did both (Liang et al., 2009;Angeli et al., 2010;Kim and Mooney, 2010;Lu and Ng, 2011;Konstas and Lapata, 2013).


ROTOWIRE data it achieves over 90% accuracy on held-out data, and recalls approximately 60% of the relations licensed by the records.

the dataset consists of two sources of articles summarizing NBA basketball games, paired with their corresponding box-and line-score tables.


In this context, we believe the problem of generating multi-sentence summaries of tables or database records to be a reasonable next-problem for neural techniques to tackle as they begin to consider more difficult NLG tasks.

as neural systems begin to move toward generating longer outputs in response to longer and more complicated inputs, however, the generated texts begin to display reference errors, intersentence incoherence, and a lack of fidelity to the source material.


The writing is colloquial, but relatively well structured, and targets an audience primarily interested in game Atlanta was in desperate need of a win and they were able to take care of a shorthanded Miami team here .

as shown in Table 1, this dataset too only contains short (single-sentence) generations, and relatively few records per generation.


Since we are interested in the frequency of the English word only when it appears as a foreign word we do not consider the ( In Example I the English word "film" is surrounded by mostly Hindi words.

the overall macro and micro precision/recall as shown in table 4 further strengthens our observation that U U R is a better metric than the baseline.


To capture these variations, we introduce the task of demographicaware word associations.

computational linguistics has traditionally taken the "one-size-fits-all" approach, with most models being agnostic to the language of the speakers behind the language.


They were able to show that the regional meaning of words can successfully be carried by word embeddings, for example the word "wicked" was most similar to the word "evil" in Kansas, while in Massachusetts, it was most similar to "super" (based on the cosine similarity of the words' vectorial representation).

6 This ensures that the maximum amount of data will be available for the selected users.


After removing duplicate mentions (Step 3), the number is reduced to 3,481; restrictions (4a) and (4b) further reduce the number to 3,143 and 2,696 respectively.

note that removing duplicate mentions (Step 3) does not reduce the number of relationships targeted, it simply avoids duplicate pairs.


Our experimental results show that the model leveraging the context embedding outperforms other methods and provide us with findings, for example, on how to construct context embeddings and which corpus to use.

given the characteristics described in the previous paragraph and SGNS's equivalence with shifted positive pointwise mutual information (Levy and Goldberg, 2014), if we want to measure to what extent word w t tends to co-occur with w k in the training data, then we should use the similarity of In this study, we show the importance of using v OU T in a task where we need to see if a word matches its context.


In the Youngest experiment, both true age and predicted age are negatively correlated with self-rated confidence (true age: Pearson r = −0.218, p-value < 10 −8 , predicted age: Pearson r = −.246, p-value < 10 −10 ), showing that raters believe their guess is easier when encountering younger users.

as proven by the Opposite experiment, posts written by males can be selected such that they are perceived as written by females.


About Extracted Topics Figure 2 displays a visualization of all main topics.

for instance in (Quinn et al., 2010), a topic model for legislative speech is defined.


This case study, based on topic signatures, shows that our method is able to derive each candidate's recurrent topics.

nouns, adjectives and foreign words were kept while verbs and personal nouns were removed.


is there any evident difference between bursty incomplete N-grams and bursty phrases of irregular forms?


9 MMRN-JOINT outperforms REIN-FORCE and its variant, REINFORECE+, which re-normalizes the probabilities of the sampled candidate sequences.

3, followed by the reward and learning algorithm in Sec.


The explicit supervision signals can be viewed as a source of immediate rewards, as we can often instantly know the correctness of the current action.

its result is also better than the state-of-the-art STAGG system.


In contrast, in scenarios like semantic parsing, the whole search space is controlled by the agent itself.

together the semantic parse in λ-calculus is shown in the top of Figure 2.


The constraint implies that the "best" action sequence should rank higher than any other sequence by a margin computed from rewards as R(h * ) − R(h).

the superior performance of MMRN-JOINT shows that the joint learning plays a crucial role in addition to the choices of architecture.


(2014) and we ourselves  do similar to assess the quality of persuasive essays, and Beigman Klebanov et al.

to the local structure of single arguments employed by all these approaches, we study the impact of the global overall structure of complete monological argumentative texts.


For example, excluding the number of parameters used in the word embeddings, our trained CNN sentence encoder has 3 million parameters, while the skip-thought vector of  contains 40 million parameters.


i kept checking the time every few minutes , hoping it would be five oclock .

any word from V w2v can be mapped into V cnn for encoding sentences.


Figures 2(a) and 2(b) show average network retention across networks for the fully and partially trained recall points respectively.

correctly classifies) a previously-seen training instance, and • Delay since last review of a training instance is the difference between the recall point and the last review point of the training instance.


This is because of the tradeoff between delaying more instances and creating stronger networks.

a heuristic scheduler such as Leitner system is suboptimal as its hard review schedules (i.e.


Most of them are based on the co-occurrence information of words and their contexts.

note that in practice, one should nOT use DEPS context for POS-tagging and chunking tasks, since their labels are used in  parsing the source corpus.


Levy and Goldberg (2014a) propose dependency-based context (DEP-S) for CSG.

similarly to the previous evaluation of sequence labeling tasks, instead of building complex deep neural networks, we use a simpler classification method called Neural Bag-of-Words (Li et al., 2017) to directly evaluate the word embeddings: texts are first represented by the sum of their word vectors, then a Logistic Regression Classifier (the same as that in previous subsection)   (Pang and Lee, 2005), customer product reviews (CR) (Nakagawa et al., 2010), and subjectivity/objectivity classification (sUBJ) (Pang and Lee, 2004).


The number of iterations i with 1 ≤ i ≤ 20 is tuned on the development set for each experiment.

7 We also evaluate our model across languages.


In preliminary experiments we considered using the multilingual system UD-Pipe 3 (Straka et al., 2016) to segment document into sentences in an effort to use tools available in multiple languages.

interestingly, Fisher and Roark (2007) investigate the utility of parse-derived features for the task.


The model then outputs a set of choices o.

we then annealed the learning rate by a factor of 5 each epoch.


Secondly, we introduce a new form of planning for dialogue called dialogue rollouts, in which an Figure 1: A dialogue in our Mechanical Turk interface, which we used to collect a negotiation dataset.

across multiple rounds, learning to walk away without a deal may incentivize partners to cooperate more in future.


It corresponds to the right switch in Figure 1(a).

although dropout was suggested as an ad-hoc technique, recently it was theoretically proven that the dropout training in deep neural networks is an approximate Bayesian inference in deep Gaussian processes (Gal and Ghahramani, 2016).


• Dropout DQN 1 and Dropout DQN 32: They both have a dropout layer after each hidden layer.

these methods were previously quite unstable (Mnih et al., 2013).


There are two types of models that use the flow features: LR Flow and LR Flow*.

f C pre and f C post refer to Equations 1/2 and 8, respectively.


's results suggest that the interaction of talking points between debate teams can possess strong predictive power.

we use LOO evaluation, resulting in a training set of 107 examples.


Besides increased time and resources required by the alternative, human evaluation of systems, human assessment of MT faces additional challenges, in particular the fact that human assessors of translation quality tend to be highly inconsistent.

translations that encounter reference-dissimilarity bias are expected to appear in the lower-right quadrant of Figure 5(a), receiving an unfairly low GEN-REF score combined with a high POSt-EDIt score.