We hypothesize that some of the errors made by the local classifier could be corrected by employing a global joint model that performs a collective classification taking into account the conversational dependencies between sentences (e.g., adjacency relations).

unlike synchronous conversations (e.g., phone, meeting), modeling conversational dependencies between sentences in asynchronous conversation is challenging, especially in those where explicit thread structure (reply-to relations) is missing, which is also our case.


There exist large corpora of utterances annotated with speech acts in synchronous spoken domains, e.g., Switchboard-DAMSL or SWBD (Jurafsky et al., 1997) and Meeting Recorder Dialog Act or MRDA (Dhillon et al., 2004).

such large corpus does not exist in asynchronous domains.


We notice that the two sentences in comment C 4 were mistakenly identified as Statement and Response, respectively, by the B-LSTM p local model.

by considering these two sentences together with others in the conversation, the global CRF (FC-FC) model could correct them.


Linear chain (for sequence labeling) and tree structured CRFs (for parsing) are the common ones in NLP.

speech act recognition in asynchronous conversation posits a different problem, where the challenge is to model arbitrary conversational structures.


As our corpus is also annotated with this information, we also trained separate models for these subtasks and assigned the SE type label accordingly.

such a pipeline approach is not competitive with the model trained directly on SE types (see Section 6.3).


Such image representation has been successfully applied in various vision tasks.

the category name t is represented by its word embedding v t ∈ R b , a low-dimensional dense vector induced by the Skip-gram model  which is widely used in diverse NLP applications too.


When comparing to Bansal2014, our model with only word embedding-based features underperforms theirs.

when introducing visual features, our performance is comparable (pvalue = 0.058).Furthermore, if we discard visual features but add semantic features from Bansal et al.


As a result, we have pushed all estimation steps into supervised ML components, which leaves the subset selection step fully principled.

we found in our experiments that even a simple heuristic yields a decent approximation of .


Therefore, we continue our experiments in the following sec-tions with ILP-R only.

sBL-R offers a nice trade-off between performance and computation cost.


We observe that the model trained on ROUGE-2 is performing better than the model trained on ROUGE-1, although learning the ROUGE-2 scores seems to be harder than learning ROUGE-1 8 The symbol * indicates that the difference compared to the previous best baseline is significant with p ≤ 0.05. scores (as shown in table 2).

errors and approximations propagate less easily in ROUGE-2, because the number of bi-grams in the intersection of two given sentences is far less.


Maximizing JSD can not be solved exactly with an ILP because it can not be factorized into individual sentences.

applying an efficient greedy algorithm or maximizing a factorizable relaxation might produce strong results as well (for example, a simple greedy maximization of Kullback-Leibler divergence already yields good results (Haghighi and Vanderwende, 2009)).


Specifically, In order to prove the formula (20), we have to find an expression for the (k) that gives to g the correct contribution to the formula: First, we observe that g does not appear in the terms that contain the intersection of more than k sentences.

specifically, (t) is not affected by g if t ≥ k. g is affected by all the (t) for which t ≤ k. Given that g appears in the sentences {s i 1 , .


As related work, Foster (2007a;2007b) and Foster and Andersen (2009) propose a method for creating a pseudo-learner corpus by artificially generating errors in a native corpus with phrase structures.

the resulting corpus does not capture various error patterns in learner English.


Thus, they cause significant modifications to PTB-II, which violates (P2).

a preposition normally constitutes a prepositional phrase with another phrase (although not normally with an adjective phrase).


(correctly, The lunch I ate was delicious.).

according to the superficial forms and local contexts, the phrase I ate lunch would form an S: S NP I VP ate lunch the relations of the S to the rest of the constituents are not clear.


The high agreement shows that the annotation scheme provides an effective way of consistently annotating learner corpora with phrase structures.

one might argue that the annotation does not represent the characteristics of learner English well because it favors consistency (and rather simple annotation rules) over completeness.


At first sight, this does not seem so surprising because ϕ never appears in the native corpus.

the rules actually show in which syntactic environment missing heads tend to occur.


In one study, after a day of cramming he could accurately recite 12-syllable sequences (of gibberish, apparently).

he could achieve comparable results with half as many practices spread out over three days.


This is reasonable, since they were largely developed during the 1960s-80s, when people would have had to manage practice schedules without the aid of computers.

the recent popularity of large-scale online learning software makes it possible to collect vast amounts of parallel student data, which can be used to empirically train richer statistical models.


He referred to his method as graduated-interval recall, whereby new vocabulary is introduced and then tested at exponentially increasing intervals, interspersed with the introduction or review of other vocabulary.

this approach is limited since the schedule is pre-recorded and cannot adapt to the learner's actual ability.


The Leitner method did yield the highest AUC 7 values among the algorithms we tried.

the top two HLR variants are not far behind, and they also reduce MAE compared to Leitner by least 45%.


2014, where the focus has been to build a system to help learners retain new vocabulary.

much of the existing work on incidental learning is found in the education and cognitive science literature rather than NLP.


Table 1 depicts statistics of the dataset.

4 to other learner corpora such as ICLE (Granger, 2003), EFCAMDAT (Geertzen et al., 2013) or TOEFL-11 , this corpus contains translations, native, and nonnative English of high proficiency speakers.


Thus broader (not only local) context is needed to judge their semantic similarity.

we don't know the reason for improvement on the A-A category as, in context, adjective interpretation is often affected by local context (e.g., the nouns that adjectives modify).


On one hand, we can offer valuable insights with respect to what constitutes an engaging, good quality news article.

we can identify benchmarks for characterising news article quality in an automatic and scalable way and, thus, predict poor writing before a news article is even published.


Templates have the advantage that the generation system does not have to deal with the internal syntax and coherence of each template, and can instead focus on document-level discourse coherence and on local coreference issues.


Duma and Klein (2013) extract templates from Wikipedia pages aligned with RDF information from DBPedia, and although they do not explicitly mention aligning multiple templates to the same set of RDF templates, the possibility seems to exist in their framework.

we are interested in extracting paraphrasal templates from non-aligned text for general NLG, as aligned corpora are difficult to obtain for most domains.


We intend this figure as the closest thing to recall that we can conceive for mining paraphrases.

keep in mind that it is not a comparable figure across the methods, since different corpora are used.


Although the parsing models outperform MARMOT, the improvements in F 1 are not significant.

all systems fare considerably worse on WSJ * which confirms that the orthographic clues in newspaper text suffice to segment the sentences properly.


Efforts have been made to create standard metrics (Papineni et al., 2001;Lin, 2004;Denkowski and Lavie, 2014;Vedantam et al., 2014) to help advance the state-of-the-art.

most such popular metrics, despite their wide use, have serious deficiencies.


Historically, MT metrics have been evaluated by how well they correlate with human annotations (Callison-Burch et al., 2010;Machacek and Bojar, 2014).


In a sense, W2V-AVG does well on passive sentences for the wrong reasonsrather than understanding that the semantics are unchanged, it simply observes that most of the words are the same.

we still see the trend that performance goes down as the number of reference  sentences increases.


When we fully saturate our professional-grade CPU, using all sixteen cores and sixteen hyperthreads, KenLM is about twice as fast as gLM.

our CPU costs nearly four times as much as our GPU, so economically, this comparison favors the GPU.


Swedish and Danish also transfer well to each other, while English transfers best to Dutch, which the former is most closely related to among the languages compared here.

there are also some cases of unrelated source languages performing best: Using Danish as source language gives the highest performing models for both Bulgarian and Czech.


Unfortunately, parallel corpora are usually only available for a handful of researchrich languages and restricted to limited domains such as government documents and news reports.

sMT is capable of exploiting abundant target-side monolingual corpora to boost fluency of translations.


There are significant gaps between k = 1 and k = 5.

keeping increasing k does not result in significant improvements and decreases the training efficiency.


Important model parameters that have been studied include the choice of association and similarity measures (Curran and Moens, 2002) and the use of subsampling and negative sampling techniques (Mikolov et al., 2013c).

the particular effects may be heterogeneous and depend on the task and model (Lapesa and Evert, 2014).


This shows that pre-trained source embeddings can be extremely helpful in bootstrapping multilingual ones.

8 the performance of the Joint w/ Aux system with 1 regularization is rather disappointing.


The same is visible in Figure 2, where these supersense embeddings are more central, with closer neighbors.


As a consequence, condensing the invhom is much less helpful.

the sibling-finder algorithm excels at maintaining state information within each elementary tree, yielding a 1000x speedup over the naive bottom-up algorithm when it was cancelled.


(2014), who try a number of unsupervised and semi-supervised models, and use the same testing methodology and hyponymy data.

note that their word embeddings are different.


For instance, an utterance in AirTicketBooking dataset, "Tomorrow afternoon, about 3 o'clock" corre-sponds to the latent state "Time Information".

by carefully examining words in dialogues we can observe that not all words are generated from the latent states (Ritter et al., 2010;Zhai and Williams, 2014).


(2015) generate text for spoken dialogue systems with a two-stage approach, comprising an LSTM decoder semantically conditioned on the logical representation of speech acts, and a reranker to generate the final output.

we design an end-to-end attention-based model for source code.


This is in part because, unlike SQL, C# code contains informative intermediate variable names that are directly related to the objective of the code.

sQL is more challenging in that it only has a handful of keywords and functions, and summarization models need to rely on other structural aspects of the code.


Using a very standard, and in fact somewhat dated sentiment analyzer, we are regularly able to garner annualized returns over twice that percentage, and in a manner that highlights two of the better design decisions that Zhang and Skiena (2010) made, viz., (1) their decision to trade based upon numerical SVM scores rather than upon discrete positive or negative sentiment classes, and (2) their decision to go long (resp., short) in the n best-(worst-) ranking securities rather than to treat all positive (negative) securities equally.

we trade based upon the raw SVM score itself, rather than its relative rank within a basket of other securities as Zhang and Skiena (2010) did, and we experimentally tune a threshold for that score that determines whether to go long, neutral or short.


Differences in the amount or degree of improvement might arguably be rescalable, but Section 4.3 shows that such intrinsic measures are not even accurate up to a determination of the delta's sign.

the results reported here should not be construed as an indictment of sentiment analysis as a technology or its potential application.


Indeed, U-MSTuf-lep outperforms U-MST in all 20 setups in D-UAS evaluation and in 15 out of 20 setups in U-UAS evaluation (in one setup there is a tie).

the improvement this procedure provides is much more noticeable for D-UAS, with an averaged improvement of 2.35% across setups, compared to an averaged U-UAS improvement of only 0.26% across setups.


Distant supervised relation extraction has been widely used to find novel relational facts from text.

distant supervision inevitably accompanies with the wrong labelling problem, and these noisy data will substantially hurt the performance of relation extraction.


To address this issue, (Mintz et al., 2009) aligns plain text with Freebase by distant supervision.

distant supervision inevitably accompanies with the wrong labelling problem.


It means the embedding of the set S is the average of all the sentence vectors: It's a naive baseline of our selective attention.

selective Attention: the wrong labelling problem inevitably occurs.


2For both CNN and PCNN, the AVE method is comparable to the ATT method in the One test setting.

when the number of testing sentences per entity pair grows, the performance of the AVE methods has almost no improvement.


Solving a word problem in general, requires several such applications in series or parallel, generating multiple equations.

in this research, we restrict the problems to be of a single equation which requires only one application.


Since, the template involves two sets, there is a 3 n−3 factor present in the formula of N change .

any application of change concept with gains and losses slots containing a collection of variables can be broken down into multiple instances of change concept where the gains and losses slots accepts only a single variable by introducing more intermediate unknown variables.


", it is important to know that 'skateboard' and 'marbles' are toys but 'shorts' are not.

such knowledge is not always present in ConceptNet which results in error.


Literary theory suggests that it should be possible, because fictional character names function as expressions of experience, ethos, teleology, values, culture, ideology, and attitudes of the character.

work in literary theory, psychology, linguistics and philosophy has studied fictional names by analysing individual works or small clusters of closely related works, such as those of a particular author.


The analysis showed that the features that calculate the emotional load of fictional names based on SentiWordNet contribute to the classification task.

we believe that there is still room for improvement for the performance of this feature mainly towards the optimization of the selection threshold in order to reduce the degree of false positive matches as well as the addition of more lexical resources for example WordNet Affect or LIWC.


Our stancetaking pathbased features that we identified as intuitively having a connection to the Disagree Strongly class together cover only 51% of Disagree Strongly instances, meaning that it is in principle impossible for our system to identify the remaining 49%.

our decision to incorporate only features that are expected to have fairly high precision for some class was intentional, as the lesson we learned from the Faulkner-based system is that it is difficult to learn a good classifier for stance classification using a large number of weakly or non-predictive features.


To evaluate a word segmenter, the standard metric consists of precision p, recall r, and an evenly-weighted F-score f 1 .

with the successive improvement of performance, state-of-the-art segmenters are hard to be distinguished under the standard metric.


We can see from Figure 2c that OOV generally has high difficulty.

a lot of OOV is relatively easy for segmenters.


One problem is that the number of possible TLINKs grows quadratic with the number of event mentions, therefore most annotation studies concentrate on links for mentions in the same or in adjacent sentences.

as our annotation study shows, this restriction results for 58% of the event mentions in a less precise information when the event took place.


The input gate by allowing incoming signal to alter the state of the memory cell, regulates proportion of history information memory cell will keep.

the output gate regulates what proportion of stored information in the memory cell will influence other neurons.


Cross-Domain Authorship Attribution Almost all previous authorship attribution studies have tackled traditional (single-domain) authorship problems where the distribution of the test data is the same as that of the training data (Madigan et al., 2005;Stamatatos, 2006;Luyckx and Daelemans, 2008;Escalante et al., 2011).

there are a handful of authorship attribution studies that explore cross-domain authorship attribution scenarios (Mikros and Argiri, 2007;Goldstein-Stewart et al., 2009;Schein et al., 2010;Stamatatos, 2013;Sapkota et al., 2014).


Therefore, the authors suggest their use in authorship attribution should be done with care.

the study did not attempt to construct authorship attribution models where the source and target domains differ.


Most previous work on part-of-speech tagging, sentiment analysis, and dependency parsing also had only binary-valued pivot features.

for authorship attribution, all features are count-based, so translation from a pivot feature value to a binary classification problem is not trivial.


2 Authorship attribution typically relies on countbased features.

the classic SCL algorithm assumes that all pivot features are binary, so that it can train binary classifiers to predict pivot feature values from non-pivot features.


Thus, while these analyses are reliable for the examples they focus on, they cannot be generalized to other examples.

the trend of actual usage can be collected automatically from a large corpus.


(2.5%) affection-ACC word-DAT feel (φ I feel the affection in φ your words.)

since the ACC-DAT order is overwhelmingly dominant in the case that the verb is sasou (ask), its dative argument is dêto (date), and its accusative argument is josei (woman) as shown in Example (3), the ACC-DAT order is considered to be canonical in this case.


I-TOP him-DAT camera-ACC showed c: φI -NOM him-DAT camera-ACC showed (I showed him a camera.)

when an argument represents the topic of the sentence (TOP), the topic marker wa is used as a postpositional particle, and case particles ga and wo do not appear explicitly.


In the case of show-type verbs, the dative argument of a causative sentence is the subject of its corresponding inchoative sentence as shown in Example (6).

in the case of pass-type verbs, the accusative argument is the subject of its corresponding inchoative sentence as shown in Example (7).


As predicted, we can alleviate this problem by using a dependency parser.

the accu-racy of the state-of-the-art Japanese dependency parser is not very high, specifically about 92% for news paper articles (Yoshinaga and Kitsuregawa, 2014), and thus, inappropriate examples would be extracted even if we used one.


president-NOM degree-ACC conferred (The president conferred a degree on φ someone .)

if we use all extracted examples that include only one of the dative and accusative arguments for calculating R DAT-only , the value is likely to suffer from a bias that the larger R ACC-DAT is, the larger R DAT-only becomes.


Through a preliminary investigation on Kyoto University Text Corpus 7 , we confirmed the effect of this constraint to avoid the bias.

r ACC-DAT is calculated as follows: where N DAT-ACC/ACC-DAT is the number of example types that include both the dative and accusative arguments in the corresponding order.


LIEL makes use of new as well as well-established features in the EL literature.

we make sure to use only non-lexical features.


NEREL (Sil and Yates, 2013) is a powerful joint entity extraction and linking system.

by construction their model is not language-independent due to the heavy reliance on type systems of structured knowledgebases like Freebase.


SASH is expected to perform well when the number of dimensions is large, since FLANN and NGT perform worse as the number of dimensions becomes larger.

nGT would be a better choice since most existing pre-trained embeddings (Turian et al., 2010;Mikolov, 2013;Pennington et al., 2014a) have a few hundred dimensions.


This fast can be acceptable assuming an empirical rule that a corpus follows a power law or Zipf's law.

graphs (a), (b), and (c) have quite different tendencies.


The results of GN and GV embeddings in graphs (a) and (c) show a similar tendency to those of Wikipedia embeddings in Figure 8.

the overall performance for the analogy task using GV embeddings is unexpectedly high, contrary to the results for the search task in Figure 6 (c).


For each encoder that requires training, we tried 0.025, 0.0025, and 0.00025 as an initial learning rate, and selected the best value for the encoder.

to the presentation of Section 3, we compose a pattern vector in backward order (from the last to the first) because preliminary experiments showed a slight improvement with this treatment.


vector of the content word and to ignore the semantic vector of the preposition.

input gates close and forget gates open when the current word is 'be' or 'a' and scanned words form a noun phrase (e.g., "charter member of"), a complement (e.g., "eligible to participate in"), or a passive voice (e.g., "require(d) to submit").


Although these reports described good performance on the respective tasks, we are unsure of the generality of distributed representations trained for a specific task such as the relation classification.

this paper demonstrated the contribution of distributed representations trained in a generic manner (with the Skip-gram objective) to the task of relation classification.


(2015) to score dependency arcs using neural network model.

different from their work, we introduce a Bidirectional LSTM to capture long range contextual information and an extra forward LSTM to better represent segments of the sentence separated by the head and modifier.


(2015) use distributed representations obtained by averaging word embeddings in segments to represent contextual information of the word pair, which could capture richer syntactic and semantic information.

their method is restricted to segment-level since their segment embedding only consider the word information within the segment.


Previous neural network models (Pei et al., 2015;Pei et al., 2014;Zheng et al., 2013) normally set context window around a word and extract atomic features within the window to represent the contextual information.

context window limits their ability in detecting long-distance information.


(2015), our basic model occasionally performs worse in recovering long distant dependencies.

this should not be a surprise since higher order models are also motivated to recover longdistance dependencies.


Knowledge bases such as Wordnet (Miller, 1995) and Freebase (Bollacker et al., 2008) have been shown very useful to AI tasks including question answering, knowledge inference, and so on.

traditional knowledge bases are symbolic and logic, thus numerical machine learning methods cannot be leveraged to support the computation over the knowledge bases.


A dot denotes a triple and its position is decided by the difference vector between tail and head entity (t − h).

since TransE adopts the principle of t − h ≈ r, there is supposed to be only one cluster whose centre is the relation vector r. results show that there exist multiple clusters, which justifies our multiple relation semantics assumption.


All remaining hsICUs are recalled by both the control and dementia models, indicating that the hsICU topics are discussed by both groups.

to assess whether they are discussed to the same extent, i.e.


Interestingly, all of the control cluster words are used in the same contexts by both healthy participants and those with dementia.

the average number of times these words are used per transcript is significantly higher in the control group (1.07, s.d.


In recent years, there have been several strands of work which attempt to collect human-labeled data for this task -in the form of document, question and answer triples -and to learn machine learning models directly from it (Richardson et al., 2013;Berant et al., 2014;Wang et al., 2015).

these datasets consist of only hundreds of documents, as the labeled examples usually require considerable expertise and neat design, making the annotation process quite expensive.


(2016) also experiment on these two datasets and report competitive results.

our single model not only still outperforms theirs, but also appears to be structurally simpler.


Monroe and Potts (2015) uses learning to improve the pragmatics model.

we use pragmatics to speed up the learning process by capturing phenomena like mutual exclusivity (Markman and Wachtel, 1988).


Broadly speaking, both the localized tests of individual phonosemantic sets and the global analyses of phonosemantic systematicity challenge the arbitrariness of the sign.

they attribute responsibility for non-arbitrariness differently.


From this, they conclude that the observed systematicity of the lexicon is not a consequence only of small pockets of sound symbolism, but is rather a feature of the mappings from sound to meaning across the lexicon as a whole.

it is possible that their methods may not be sensitive enough to find localized phonosemantic sets.


The validation set was used to tune the threshold for classifying a pair as positive, as well as the maximum number of each term's most associated contexts (N ).

to the original paper, in which the number of each term's contexts is fixed to N , in this adaptation it was set to the minimum between the number of contexts with LMI score above zero and N .


1 There is clearly also a practical demand for multilingual image captions, e.g., automatic translation of descriptions of art works would allow access to digitized art catalogues across language barriers and is thus of social and cultural interest; multilingual product descriptions are of high commercial interest since they would allow to widen e-commerce transactions automatically to international markets.

while datasets of images and monolingual captions already include millions of tuples (Ferraro et al., 2015), the largest multilingual datasets of images and captions known to the authors contain 20,000 (Grubinger et al., 2006) or 30,000 2 triples of images with German and English descriptions.


We rely on the CNN framework (Socher et al., 2014;Simonyan and Zisserman, 2015) to solve semantic classification and disambiguation tasks in NLP with the help of supervision signals from visual feedback.

we consider image captioning as a different task than caption translation since it is not given the information of the source language string.


For clarity, we focus on K-way classification, where Y = ∆ K is the K-dimensional probability simplex and y ∈ {0, 1} K ⊂ Y is a one-hot encoding of the class label.

our method specification can straightforwardly be applied to other contexts such as regression and sequence learning (e.g., NER tagging, which is a sequence of classification decisions).


This sheds light from another perspective on why our framework would work.

we found in our experiments (section 5) that to produce strong performance it is crucial to use the same labeled data x n in the two losses of Eq.


(2) 6: until convergence Output: Distill student network p θ and teacher network q ning over multiple examples), requiring joint inference.

as mentioned above, p is more lightweight and efficient, and useful when rule evaluation is expensive or impossible at prediction time.


By incorporating the bi-gram transition rules (Row 2), the joint teacher model q achieves 1.56 improvement in F1 score that outperforms most previous neural based methods (Rows 4-7), including the BLSTM-CRF model (Lample et al., 2016) which applies a conditional random field (CRF) on top of a BLSTM in order to capture the transition patterns and encourage valid sequences.

our method implements the desired constraints in a more straightforward way by using the declarative logic rule language, and at the same time does not introduce extra model parameters to learn.


Once again, this speaks to the remarkable conservativeness of the Icelandic language.

these small differences are in fact reliable: There are significant slopes in both regressions (positive differences: R + = .161, p + < .001; negative differences: R − = −.164, p − < .001).


The analyses reported exclude these 552 points.

keeping these outlying values in the regression, both key effects remained significant, but the slope estimates were less trustworthy.


This results in a mismatch between the Obj and Subj ratings.

relying on subjective ratings alone is also problematic since crowd-sourced subjects frequently give inaccurate responses and real users are often unwilling to extend the interaction in order to give feedback, resulting in unstable learning (Zhao et al., 2011;Gašić et al., 2011).


This removes the need for the Obj check during online policy learning and the resulting policy is as effective as one trained with dialogues using the Obj = Subj check.

a user simulator will only provide a rough approximation of real user statistics and developing a user simulator is a costly process (Schatzmann et al., 2006).


This measure was later used as a reward function for learning a dialogue policy (Rieser and Lemon, 2011).

as noted, task completion is rarely available when the system is interacting with real users and also concerns have been raised regarding the theoretical validity of the model (Larsen, 2003).


This shows that dialogue length was one of the prominent features in the dialogue representation d. It can also be seen that the longer failed dialogues (more than 15 turns) are located close to each other, mostly at the bottom right.

there are other failed dialogues which are spread throughout the cluster.


At first glance, it might appear that a locally normalized model used in conjunction with beam search or exact search is able to revise earlier decisions.

the label bias problem (see Bottou (1991), Collins (1999) pages 222-226, Lafferty et al.


For every amount of lookahead k, we can construct examples that cannot be modeled with a locally normalized model by duplicating the middle input b in (7) k + 1 times.

only a local model with scores ρ that considers the entire input can capture any distribution p(d 1:n |x 1:n ): in this case the decompo- increasing the amount of context used as input comes at a cost, requiring more powerful learning algorithms, and potentially more training data.


Finally, Watanabe and Sumita (2015) introduce recurrent components and additional techniques like max-violation updates for a corresponding constituency parsing model.

our model does not require any recurrence or specialized training.


As we can see in Table 4, the separated verb prefix is always in the RK.

an adverbial modifier as in (1-b) is rarely in the RK (0.35% of the adverbs cases in the TüBa-D/Z).


For the baseline, SVM without embedding (w/o Embed) achieved 91.99% accuracy, which is already very competitive.

the models with word embedding trained on 6 billion tokens (6B-50d) and 840 billion tokens (840B-300d) (Pennington et al., 2014) achieved 92.89% and 93.00%, respectively.


Finding domain invariant features is critical for successful domain adaptation and transfer learning.

in the case of unsupervised adaptation, there is a significant risk of overfitting on source training data.


To further demonstrate the advantage of our idea of minimal features with bidirectional sentence representations, we extend our work from dependency parsing to constituency parsing.

the latter is significantly more challenging than the former under the shift-reduce paradigm because: 1 shift (I) 6 pro (NP) 2 pro (NP) 7 adj 3 shift (like) 8 pro (S) 4 pro (VP) 9 adj 5 shift (sports) Figure 5: Shift-Promote-Adjoin parsing example.


A notable shortcoming of this approach is that the translation model probabilities thus calculated from the training bitext can be unintuitive and unreliable (Marcu and Wong, 2002;Foster et al., 2006) as they reflect only the distribution over the phrase pairs observed in the training data.

from an SMT perspective it is important that the models reflect probability distributions which are preferred by the decoding process, i.e., phrase translations which are likely to be used frequently to achieve better translations should get higher scores and phrases which are less likely to be used should get low scores.


Thus forced alignment is a reestimation technique where translation probabilities are calculated based on their frequency in best-scoring hypotheses instead of the frequencies of all possible phrase pairs in the bitext.

one limitation of forced alignment is that only the phrase translation model can be re-estimated since it is restricted to align the source sentence to the given target reference, thus fixing the choice of reordering decisions.


Another relevant line of research relates tuning (weight optimisation), where our work lies between forced decoding (Wuebker et al., 2010) and the bold updating approach of (Liang et al., 2006).

our approach specifically proposes a novel method for training models using oracle BLEU translations.


(2010) address this problem by using a leave-one-out approach where they modify the phrase translation probabilities for each sentence pair by removing the counts of all phrases that were extracted from that particular sentence.

in our approach, we do not impose a constraint to produce the exact translation, instead we use the highest BLEU translations which may be very different from the references.


For the linear interpolation of the re-estimated phrase table with the baseline, forced decoding shows only a slight improvement for MT06, MT08 and MT09 and still suffers from a substantial drop for MT05.

oracle-BLEU re-estimation shows consistent improvements for all test sets with a maximum gain of up to +0.7 for MT06.


For an additional anal-    ysis, we experimented with the interpolation of both the re-estimated phrase table (forced decoding and oracle-BLEU) with the baseline.

improvements achieved with this interpolation did not surpass the best result obtained for the oracle-BLEU re-estimation.


This implies that only in combination with the original phrase table does forced-decoding with leave-one-out outperform the baseline.

oracle-BLEU re-estimation by its own not only performs better than forced decoding, but also gives a performance equal to forced decoding with leave-oneout when interpolated with baseline phrase table.


A comparison of the two approaches goes in favor of the joint setup: Without the reranker, models generating trees produce less semantic errors and gain higher BLEU/NIST scores.

with the reranker, the string-based model is able to reduce the number of semantic errors while producing outputs significantly better in terms of BLEU/NIST.


, w |V | (t) tend to be different across t, by our very postulate, so that there is non-identity of these 'reference points' over time.

as we may assume that the meanings of at least a few words are stable over time, we strongly expect the vectorsw i (t) to be suitable for our task of analysis of meaning changes.


We note that this linearity concerns the core vocabulary of languages (in our case, words that occurred at least 100 times in each time period), and, in the case of TISS graphs, is an average result; particular words may have drastic, non-linear meaning changes across time (e.g., proper names referring to entirely different entities).

our analysis also finds that most core words' meanings decay linearly in time.


In the segmentation literature, several previous systems learn lexical items from variable input (Elsner et al., 2013;Daland and Pierrehumbert, 2011;Rytting et al., 2010;Neubig et al., 2010;Fleck, 2008).

these models use pre-processed representations of the acoustics (phonetic transcription or posterior probabilities from a phone recognizer) rather than inducing an acoustic category structure directly.


Previous approaches heavily depend on languagespecific knowledge and pre-existing natural language processing (NLP) tools.

compared to English, not all languages have such resources and tools available.


For example, in S2, when predicting the type of a trigger candidate " release", the forward sequence information such as "court" can help the classifier label "release" as a trigger of a "Release-Parole" event.

for feature engineering methods, it is hard to establish a relation between "court" and "release", because there is no direct dependency path between them.


R1: But all this is beyond your control.


Since the baseline strategy should access all the short texts in each size of data collection, which means in 1k size of BNC data collection, the baseline strategy access all these 1k candidates.

our proposed strategies under different settings only access small size candidates to obtain the results.


The exact nature of this tradeoff isn't clear, and is an interesting avenue for future work.

there are potential reasons for the results discrepancy betweeen mean rank and hits@10.


Such clustering is usually performed through manually defined semantic and syntactic features defined over argument instances.

the representation based on these features are usually sparse and difficult to generalize.


As analyzed in Section 3.2, the element-wise difference matching layer does not add to model complexity and can be absorbed as a special case into simple concatenation.

explicitly using such heuristic yields an accuracy boost of 1-2%.


For most sentence models including TBCNN, the overall complexity is at least O(n).

an efficient matching approach is still important, especially to retrieval-and-reranking systems (Yan et al., 2016;Li et al., 2016).


We begin by establishing a CRF baseline (#1) and show that adding segmentation features helps (#2).

adding those features to the full model (with embeddings) in Peng and Dredze (2015) (#3) did not improve results (#4).


Sampling from a class n-gram provided many complementary word forms, not easily generated by the other models.

it became successively harder to improve the OOV reduction rates by a combination of three models.


Speech-based, demographic and morphological features unequivocally contributed to performance.

the effect of semantic features seems less obvious as they harm performance taken as a whole but some individual semantic features are useful for the system, as shown by the results achieved with just using significant features.


Recently, deep learning methods provide an effective way of reducing the number of handcrafted features (Socher et al., 2012;Zeng et al., 2014).

these approaches still use lexical resources such as WordNet (Miller, 1995) or NLP systems like dependency parsers and NER to get high-level features.


The tagger is greedy: the tagging decisions are independent of each other.

as shown below and in other recent work using bi-RNNs for sequence tagging, we can still produce competitive tagging accuracies, because of the richness of the representation v i that takes the entire input sequence into account.


LSTM (-3) LSTM(3-3) LSTM(1-3) 1.298 1.034 0.990 Our deep single task model increases performance over this baseline by 30%.

we see that when we predict both POS and the target task at the top layer, Rademacher complexity is lower and close to a random baseline.


These rules contain typed relations and can also be applied in a context-sensitive manner.

ignoring the types and applying the inference rules out of context worked better on our dataset, perhaps because Berant et al.


Current data-to-text systems assume that the underlying data is precise and correct -an assumption which is heavily criticised by other disciplines concerned with decision support, such as medicine (Gigerenzer and Muir Gray, 2011), environmental modelling (Beven, 2009), climate change (Manning et al., 2004), or weather forecasting (Kootval, 2008).

simply presenting numerical ex-pressions of risk and uncertainty is not enough.


This is a 44% average increase in game score.

multi-modal vs. NLG-only: there is no significant difference between the NLG only and the multi-modal representation, for game score.


Confidence: For confidence, the multi-modal representation is significantly more effective than NLG only (p < 0.01, effect = 17.7%).

as Table 2 shows, although adults did not feel very confident when presented with NLG only, they were able to make better decisions compared to being presented with graphics only.


Demographic factors: We further found that prior experience on making decisions based on risk, familiarity with weather models, and correct literacy test results are predictors of the players' understanding of uncertainty, which is translated in both confidence and game scores.

we found that the education level, the gender, or being native speaker of English does not contribute to players' confidence and game scores.


Post-edited data are often used in incremental MT frameworks as additional training material.

often this does not fully exploit the potential of these rich PE data: e.g., PE data may just be drowned out by a large SMT model.


These procedures search over a vast space of translations, much larger than is considered by the NMT beam search.

the Hiero context-free grammars that make efficient search possible are weak models of translation.


As expected, all features correlate with age and income in the same direction.

some features and groups are more predictive of one or the other (depicted above or below the principal diagonal in Figure 2).


As we can see, it is possible to predict the most pessimistic 14 users with perfect accuracy.

some of the most likely optimistic users actually belonged to another class based on the ground truth labels.


Since this is a generative model, it can not only be used to assign probability to observed sentences, but also generate new sentences based on the model parameters.

extra-linguistic information, if available, can improve classification performance (Volkova et al., 2013;Hovy, 2015), and fakereview detection models often also exploit metainformation about the author and their behavior , looking for irregularities.


The numbers are generally in a high range, which is encouraging, since it means that our models can detect fake reviews fairly reliably.

we also see that the conditional models introduced here quickly become significantly harder to detect than the regular LM.


Even in MT (Ling et al., 2015b), where authors use the character transformation presented in (Ballesteros et al., 2015;Ling et al., 2015a) both in the source and target.

they do not seem to get clear improvements.


The concatenation of these max values already provides us with a representation of each word as a vector with a fixed length equal to the total number of convolutional ker-nels.

the addition of two highway layers was shown to improve the quality of the language model in (Kim et al., 2016) so we also kept these additional layers in our case.


In the target size we are still limited in vocabulary by the softmax layer at the output of the network and we kept the standard target word embeddings in our experiments.

the results seem to show that the affix-aware representation of the source words has a positive influence on all the components of the network.


statistical phrase-based, are that the translation is faced with trainable features and optimized in an end-to-end scheme.

there still remain many challenges left to solve, such as dealing with the limi-tation in vocabulary size.


Parallel Sentences occur naturally, and provide training examples that are more consistent with downstream applications.

they can be noisy due to automatic sentence alignment and one-to-many mappings, and bag-of-word representations of sentence meaning are likely to be increasingly noisier as segments get longer.


The best bilingual compositional representations are better than non-compositional Glove embeddings (Pennington et al., 2014), but worse than compositional Paragram embeddings (Wieting et al., 2016).

paragram initialization requires large amounts of text and human word similarity judgments for tuning, while our models were initialized randomly.


The closest model to LCTM is Gaussian LDA (Das et al., 2015), which models each topic as a Gaussian distribution over the word embedding space.

the assumption that topics are unimodal in the embedding space is not appropriate, since topically related words such as 'neural' and 'networks' can occur distantly from each other in the embedding space.


(Petterson et al., 2010) exploits external word features to improve the Dirichlet prior of the topic-word distributions.

both of the models cannot handle OOV words, because they assume fixed word types.


Also, we see that LCTM performs comparable to LFLDA and nl-cLDA, both of which incorporate information of word embeddings to aid topic inference.

as we will see in the next section, LCTM can  better handle OOV words in held-out documents than LFLDA and nl-cLDA do.


We see that LCTM-UNK outperforms other topic models in almost every setting, demonstrating the superiority of our method, even when OOV words are ignored.

the fact that LCTM outperforms LCTM-UNK in all cases clearly illustrates that LCTM can effectively make use of information about OOV to further improve the representation of unseen documents.


Consequently, word embeddings have been commonly used in the last few years for lexical similarity tasks and as features in multiple, syntactic and semantic, NLP applications.

keeping embedding vectors for hundreds of thousands of words for repeated use could take its toll both on storing the word vectors on disk and, even more so, on loading them into memory.


This suggests that mentioned trolls are very similar to paid trolls (except for their reply rate, time and day of posting patterns).

using just mentions might be a "witch hunt": some users could have been accused of being "trolls" unfairly.


The original phrase table requires 2.8GB memory.

the 90% pruned table only requires 350MB of memory.


On the one hand, it uses SGNS window sampling, negative sampling, and stochastic gradient descent (SGD) to minimize a loss function that weights frequent co-occurrences heavily but also takes into account negative co-occurrence.

since PPMI generally outperforms PMI on semantic similarity tasks (Bullinaria and Levy, 2007), rather than implicitly factorize a shifted PMI matrix (like SGNS), LexVec explicitly factorizes the PPMI matrix.


(3)) penalizes more heavily reconstruction error of frequent cooccurrences, improving on PPMI-SVD's L 2 loss, which weights all reconstruction errors equally.

as it does not penalize reconstruction errors for pairs with zero counts in the co-occurrence matrix, no effort is made to scatter the vectors for these pairs.


de en es fr id it ja ko pt-br sv de 16 1 14 14 14 13 10 12 14 0 en 45 1 1 1 1 1 1 1 5 es 24 14 14 13 10 12 15 0 fr 14 14 13 10 12 14 0 id 14 13 10 12 14 0 it 13 10 12 13 0 ja 763 10 10 0 ko 20 12 0 pt-br 15 0 sv 25 While all these experiments have been performed on sentences with gold PoS tags, preliminary experiments assuming predicted tags instead show analogous results: the absolute values of LAS and UAS are slightly smaller across the board, but the behavior in relative terms is the same, and the bilingual models that improved over the monolingual baseline in the gold experiments keep doing so under this setting.

table 2 shows the performance of the monolingual and bilingual models under the universal tags only configuration.


By starting a new topic in conversation, the leading speaker brings novelty to the existing context, which often involves relatively long and complex utterances.

the following speaker has to accommodate this change of context, by first producing short acknowledging phrases at the early stage, and gradually increase his contribution as the topic develops.


For computation, each word is represented by a weighted feature vector, where features typically correspond to words that co-occur in a particular context.

dSMs tend to retrieve both synonyms (such as formal-conventional) and antonyms (such as formal-informal) as related words and cannot sufficiently distinguish between the two relations.


Furthermore, in MTE we can expect shorter texts, which are typically much more similar.

in cQA, the question and the intended answers might differ significantly both in terms of length and in lexical content.


Most of these papers concentrate on providing advanced neural architectures in order to better model the problem at hand.

our goal here is different: we extend and reuse an existing pairwise NN framework from a different but related problem.


", the non-reduced argument "Obama, the newly elected president" can be reduced to the minimal argument "Obama", as both specify the same answer to the role question "who flew to Russia?".

a restrictive modifier is an integral part of the meaning of the containing NP, and hence should not be removed, as in "She wore the necklace that her mother gave her".


", in which "The tall boys and girls" would reduce to two overlapping arguments: {"The tall boys", "The tall girls"}.

non-distributive conjuncts cannot be split.


Semantic classification of collocations has been addressed, for instance, in (Wanner et al., 2006;Gelbukh and Kolesnikova., 2012;Moreno et al., 2013;.

to the best of our knowledge, our work is the first to automatically retrieve and typify collocations simultaneously.


So we perform better even though we only use 8 of 300 dimensions!

the greatest advantage of UD is that we only need 100GB of RAM, 80% less than W2V.


In sum, our comparison provides no clear answer to the question posed by the title of this paper.

it shows conclusively that different context types yield semantic spaces with different properties, and that the optimal context type depends on the actual application and language.


A full discourse parser typically consists of a pipeline of classifiers: explicit and implicit DCs are first classified and then processed separately by 2 classifiers (Xue et al., 2015).

the pragmatic listener of the RSA model considers if the speaker would prefer a particular DC, explicit or implicit, when expressing the intended sense.


Recent distributional representations of words exhibit semantic consistency over directional metrics such as cosine similarity.

neither categorical nor Gaussian observational distributions used in existing topic models are appropriate to leverage such correlations.


POET improves the results for all tag pairs for CELEX.

initial experiments indicated that it is not effective for SIGMORPHON16 because its training sets are not large enough.


Many previous approaches have been effectively applied to CWS problem (Lafferty et al., 2001;Xue and Shen, 2003;Sun et al., 2012;Sun, 2014;Cheng et al., 2015).

these approaches incorporated many handcrafted features, thus restricting the generalization ability of these models.


Intuitively, extra hidden layers are able to improve accuracy performance.

it is common that extra hidden layers decrease classification accuracy.


An ideal metric should be oriented towards the recall for the "BAD" class.

the case of F 1 -BAD score shows that this is not the only requirement: in order to be useful the metric should not favour pessimistic labellings, i.e., all or most words labelled as "BAD".


One of the most reliable ways of comparing metrics is to measure their correlation with human judgements.

for the word-level QE task, asking humans to rate a system labelling or to compare the outputs of two or more QE systems is a very expensive process.


synthetic systems from real ones and in the task of discriminating among real systems, despite the fact that its d scores are not the best.

f 1 -BAD is not far behind: it has high values for d scores and can identify synthetic datasets quite often.


For instance, people estimate the size of cities they recognize to be larger than that of unknown cities (Goldstein and Gigerenzer, 2002).

the same holds for individuals/groups/characteristics we research.


The differences are very striking, with LexStat-Partial outperforming its nonpartial counterpart by up to four points, and SCA-Partial outperforming the classical SCA variant by almost five points.

2 we do not find strong differences in the performance of the cluster algorithms.


Neural network models have shown their promising opportunities for multi-task learning, which focus on learning the shared layers to extract the common and task-invariant features.

in most existing approaches, the extracted shared features are prone to be contaminated by task-specific features or the noise brought by other tasks.


Recently, neuralbased models for multi-task learning have become very popular, ranging from computer vision (Misra et al., 2016;Zhang et al., 2014) to natural language processing (Collobert and Weston, 2008;Luong et al., 2015), since they provide a convenient way of combining information from multiple tasks.

most existing work on multi-task learning (Liu et al., 2016c,b) Figure 1: Two sharing schemes for task A and task B.


For the positive sentence "Five stars, my baby can fall asleep soon in the stroller", both models capture the informative pattern "Five stars" 6 .

sP-MTL makes a wrong prediction due to misunderstanding of the word "asleep".


Contrary to models that operate on the argument component level, we find that framing AM as dependency parsing leads to subpar performance results.

less complex (local) tagging models based on BiL-STMs perform robustly across classification scenarios, being able to catch longrange dependencies inherent to the AM problem.


Moreover, from a machine learning perspective, pipeline approaches are problematic because they solve subtasks independently and thus lead to error propagation rather than exploiting interrelationships between variables.

to this, we investigate neural techniques for end-to-end learning in computational AM, which do not require the hand-crafting of features or constraints.


We observe that programs with three expressions use a more limited set of properties, mainly focusing on answering a few types of questions such as "who plays meg in family guy", "what college did jeff corwin go to" and "which countries does russia border".

programs with two expressions use a more diverse set of properties, which could explain the lower performance compared to programs with three expressions.


(2016) further employ sentence-level attention mechanism in neural relation extraction, and achieves the state-of-the-art performance.

most RE systems concentrate on extracting relational facts from mono-lingual data.


Our model falls under the second class of approaches where utterances are first mapped to an intermediate representation containing natural language predicates.

rather than using an external parser (Reddy et al., 2014 or manually specified CCG grammars (Kwiatkowski et al., 2013), we induce intermediate representations in the form of predicate-argument structures from data.


This property makes FunQL logical forms convenient to be predicted with recurrent neural networks (Vinyals et al., 2015;Choe and Charniak, 2016;.

funQL is less expressive than lambda calculus, partially due to the elimination of variables.


One of the great promises of semantic analysis (over more surface forms of analysis) is its cross-linguistic potential.

while the theoretical and applicative importance of universality in semantics has long been recognized (Goddard, 2011), the nature of universal semantics remains unknown.


If we do not discriminate the unknown words and assign different unknown words with the same token unk , it would be impossible for us to know what is the exact word that unk represents for in the real test.

when using our proposed unknown word processing method, if the model predicts a unkX as the answer, we can simply scan through the original document and identify its position according to its unknown word number X and replace the unkX with the real word.


The distinction between discourse modes is expected to be clarified conceptually by considering their different communication purposes.

there would still be specific ambiguous and vague cases.


Even the simple features like length can judge that short essays tend to have low scores.

when the lengths of essays are close, AES would face greater challenges, because it is required to deeper understand the properties of well written essays.


The attention scores a i are determined by a dot product between h i with each z j , followed by a softmax over the source sequence: In preliminary experiments, we did not find the MLP attention of (Bahdanau et al., 2015) to perform significantly better in terms of BLEU nor perplexity.

we found the dot-product attention to be more favorable in terms of training and evaluation speed.


Motivated by the hypothesis that the this may be due to the decoder depending on the length of the source sentence (which it cannot determine without position embeddings), we explicitly provided a distributed representation of the input length to the decoder and attention module.

this did not cause a change in attention patterns nor did it improve translation accuracy.


Deep Neural Networks (DNNs) have provably enhanced the state-of-the-art Neural Machine Translation (NMT) with their capability in modeling complex functions and capturing complex linguistic structures.

nMT systems with deep architecture in their encoder or decoder Rnns often suffer from severe gradient diffusion due to the non-linear recurrent activations, which often make the optimization much more difficult.


Both systems rely on features based on a set of alignments produced using bi-lexical cues and hand-written rules.

our models train directly on parallel corpora, and make only minimal use of alignments to anonymize named entities.


During decoding we find the most likely sequence of instructions z given x, which can be performed with a stack-based decoder.

it is important to refer that each generated instruction must be executed to obtain v i .


To create a themed corpus about 'love', for instance, we would aggregate love poetry to train the model, which would thus learn an implicit representation of love.

this forces us to generate poetry according to discrete themes and styles from pretrained models, requiring a new training corpus for each model.


The path structure T 1 where B is shared by two predicates (mission and operator) will favour the use of a participial or a passive subject relative clause.

the branching structure T 2 will favour the use of a new clause with a pronominal subject or a coordinated VP.


Through a gated matching layer, the resulting question-aware passage representation effectively encodes question information for each passage word.

recurrent networks can only memorize limited passage context in practice despite its theoretical capability.


Those that are automatically generated from natural occurring data can be very large (Hill et al., 2016;, which allow the training of more expressive models.

they are in cloze style, in which the goal is to predict the missing word (often a named entity) in a passage.


Once the source sequence is encoded, another decoding RNN model is to generate a target sequence through the following prediction model: where s t is the RNN hidden state at time t, the predicted target word y t at time t is typically performed by a sof tmax classifier over a settled vocabulary (e.g.

30,000 words) through function g. The prediction model of classical decoders for each target word y i share the same context vector c. a fixed vector is not enough to obtain a better result on generating a long targets.The attention mechanism in the decoding can dynamically choose context c t at each time step (Bahdanau et al., 2014), for example, representing c t as the weighted sum of the source where the function ρ use to compute the attentive strength with each source state, which usually adopts a neural network such as multi-layer perceptron (MLP).


For the automatically collected WIKISUGGEST dataset, noisy question-answer pairs were problematic, as discussed in Section 3.

the models frequently guessed the spurious answer.


There are several tailor-made languages designed for querying KBs, such as SPARQL (Prudhommeaux and Seaborne, 2008).

to handle such query languages, users are required to not only be familiar with the particular language grammars, but also be aware of the architectures of the KBs.


(2015) make use of the context and the type of the answer.

the representation of the question end is oligotrophic.


Figure 1 (a) as either house or building, ignoring the lexical similarity between these two names.

humans seem to be more flexible as to the chosen level of generality.


It would be interesting to test other IC models on T3 and compare their results against the ones reported here.

note that IC-Wang is 'tailored' for T3 because it takes as input the whole sentence (minus the word to be generated), while common sequential IC approaches can only generate a word depending on the previous words in the sentence.


Ideally, an AI system should acquire such knowledge through direct physical interactions with the world.

such a physically interactive system does not seem feasible in the foreseeable future.


When considered in verb implications, size, weight, strength, and rigidness concern individual-level semantics; the relative properties implied by verbs in these dimensions are true in general.

speed concerns stage-level semantics; its implied relations hold only during a window surrounding the verb.


Relatedly, previous work uses natural language inference to infer new facts from a dataset of commonsense facts that can be extracted from unstructured text (Angeli and Manning, 2014).

we focus on a small number of specific types of knowledge without access to an existing database of knowledge.


Another novel aspect is that our dynamic oracle is approximate, i.e., based on efficiently-computable approximations of the loss due to the complexity of calculating its actual value in a non-monotonic and non-projective scenario.

this is not a problem in practice: experimental results show how our parser and oracle can use non-monotonic actions to repair erroneous attachments, outperforming the monotonic version developed by Gómez-Rodríguez and Fernández-González (2015) in a large majority of the datasets tested.


This implies that the calculation of the two non-monotonic upper bounds is less efficient than the linear loss computation in the monotonic scenario.

a non-monotonic algorithm that uses the lower bound as loss expression is the fastest option (even faster than the monotonic approach) as the oracle does not need to compute cycles at all, speeding up the training process.


Sheshadri and Lease (2013) survey and benchmark methods.

these models are almost all in the binary or multiclass classification setting; only a few have considered sequence labeling.


due to the technical difficulty of the joint approach with CRFs, they resorted to strong modeling assumptions.


proposed HMM models for aggregating crowdsourced discourse segmentation labels.

they did not consider the general sequence labeling setting.


The posteriors can then be easily evaluated: In the standard M-step, the parameters are estimated using maximum likelihood.

we found a Variational Bayesian (VB) update procedure for the HMM parameters similar to (Johnson, 2007;Beal, 2003) provides some improvement and stability.


Needless to say, the ability to obtain this annotated data for many languages is limited.

we can expect that for most languages we can obtain large amounts of unlabeled surface forms that may allow for semi-supervised learning over this unlabeled data (entirety of Fig.


timization methods such as stochastic gradient descent (SGD) (Robbins and Monro, 1951); stochastic methods of this type are particularly important for training with large data sets.

this approach often provides a maximum a posteriori (MAP) estimate of model parameters.


pSGLD and dropout have similar behavior: they explore the parameter space during learning, and thus coverge slower than RMSprop on the training dataset.

the learned uncertainty alleviates overfitting and results in lower errors on the validation and testing datasets.


The reduction of computational costs has been early designed by imposing a budget (Dekel and Singer, 2006;Wang and Vucetic, 2010), that is limiting the maximum number of SVs in a model.

in complex tasks, such methods still require large budgets to reach adequate accuracies.


Cho and Saul (2009) introduced a family of kernel functions that mimic the computation of large multilayer neural networks.

such kernels can be applied only on vector inputs.


Notice that other low-dimensional approximations of kernel functions have been studied, as for example the randomized feature mappings proposed in Rahimi and Recht (2008).

these assume that (i) instances have vectorial form and (ii) shift-invariant kernels are adopted.


An intuitive approach is to search the most similar existing review for the new review, then take the found reviewer's behavioral features as the new reviewers' features (detailed in Section 5.3).

there is abundant behavioral information in the review graph ( Figure 1), it is difficult for the traditional discrete manual behavioral features to record the global behavioral information (Wang et al., 2016).


Using the methods described in Section 2, from the corpus, we collected a number of instances of users' preferences regarding various topics.

twitter users do not necessarily express preferences for all topics.


This result again indicates that our proposed method reasonably utilizes known preferences to complete missing preferences.

the performance of the majority baseline decreased as it received more information regarding the users.


Employing a single axis (e.g., liberal to conservative) or a few axes (e.g., political parties and candidates of elections), these studies provide intuitive visualizations and interpretations along the respective axes.

this study is the first attempt to recognize and organize various axes of topics on social media with no prior assumptions regarding the axes.


Empirical results here probably reflected the underlying assumptions that PCA treats missing elements as zero and not as missing data.

in the present work, we properly distinguish missing values from zero, excluding missing elements of the original matrix from the objective function of Equation 2.


And DS for RE assumes that if two entities have a relationship in a known knowledge base, then all sentences that mention these two entities will express that relationship in some way (Mintz et al., 2009).

when we use DS for RE to EE, we meet following challenges: Triggers are not given out in existing knowledge bases.


The Dynamic Multi-pooling Convolutional Neural Networks (DMCNNs) is the best reported CNN-based model for event extraction  by using human-annotated training data.

our automatically labeled data face a noise problem, which is a intrinsic problem of using DS to construct training data (Hoffmann et al., 2011;Surdeanu et al., 2012).


For example, if k = 2, we will get 25, 797 sentences labeled as people.marriage events and we will get 534 labeled sentences, if k = 3.

when we set k = 1, although more labeled data are generated, the precision could not be guaranteed.


(2014) extended the distant supervision approach to fill slots in plane crash.

the method can only extract arguments of one plane crash type and need flight number strings as input.


They use five time types and assign one of them to each word, which is similar to SynTime in the way of defining types over tokens.

they focus only on the type of date, while SynTime recoginizes all the time expressions and does not involve learning and runs in real time.


For bag level models, since reliable and less reliable sentences are all aggregated into a sentence bag, we can not determine which bag is reliable and which is not.

bag level models can still build a curriculum by changing the content of a bag, e.g., keeping reliable sentences in the bag first, then gradually adding less reliable ones, and training with Equation 5, which could benefit from the prior knowledge of data quality as well.


This work treats code generation as a sequence-tosequence modeling problem, and introduce methods to generate words from character-level models, and copy variable names from input descriptions.

unlike most work in semantic parsing, it does not consider the fact that code has to be well-defined programs in the target syntax.


Metrics As is standard in semantic parsing, we measure accuracy, the fraction of correctly generated examples.

because generating an exact match for complex code structures is nontrivial, we follow Ling et al.


2), with the number of productions in the grammar remaining unchanged.

dJANGO has a broader domain, and thus unary closure results in more productions in the grammar (237 for dJANGO vs. 100 for HS), increasing sparsity.


An alternative approach that we follow here is to independently train the embeddings for each language on monolingual corpora, and then learn a linear transformation to map the embeddings from one space into the other by minimizing the distances in a bilingual dictionary, usually in the range of a few thousand entries (Mikolov et al., 2013a;Artetxe et al., 2016).

dictionaries of that size are not readily available for many language pairs, specially those involving less-resourced languages.


Such representations have found a place in many semantic applications but there is no clear consensus as to the best representation.

with the rise of supervised machine learning techniques, a new requirement has come to the fore: the ability of human annotators to quickly and reliably generate semantic representations as training data.


2 https://github.com/MiuLab/TC-Bot Our work is motivated by the neural GenQA (Yin et al., 2016a) and neural enquirer (Yin et al., 2016b) models for querying KBs via natural language in a fully "neuralized" way.

the key difference is that these systems assume that users can compose a complicated, compositional natural language query that can uniquely identify the element/answer in the KB.


RL-Soft achieves a success rate of 74% on the human evaluation and 80% against the simulated user, indicating minimal overfitting.

all agents take a higher number of turns against real users as compared to the simulator, due to the noisier inputs.


It is also effective to control the hypothesis length by the minimum and maximum lengths to some extent, where the minimum and maximum are selected as fixed ratios to the length of the input speech.

since there are exceptionally long or short transcripts compared to the input speech, it is difficult to balance saving such exceptional transcripts and preventing hypotheses with irrelevant lengths.


Finally, we achieved 1xRT with one-pass decoding when using a beam width around 3 to 5, even though it was a single-threaded process on a CPU.

the decoding process has not yet achieved realtime ASR since CTC and the attention mechanism need to access all of the frames of the input utterance even when predicting the first label.


English texts produced by native speakers of a variety of languages have been used to reconstruct phylogenetic trees, with varying degrees of success (Nagata and Whittaker, 2013;Berzak et al., 2014).

to language learners, however, translators translate into their mother tongue, so the texts we studied were written by highly competent native speakers.


This does not undermine the force of translation universals: we demonstrated how explicitation, in the form of cohesive markers, can help identify translations.

it may be possible to define classi-fiers implementing other universal facets of translation, e.g., simplification, which will yield good separation between O and T. explicitation fails in the reproduction of language typology, whereas interference-based features produce trees of considerable quality.


Similarly, MORSE performs better on Turkish with a 7% absolute margin in terms of F1 score.

morfessor surpasses mORSE in performance on Finnish by a large margin as well, especially in terms of recall.


In the literature, several deep and complex neural networks have been proposed for this task, assuming availability of relatively large amounts of training data.

the associated computational complexity increases as the networks go deeper, which poses serious challenges in practical applications.


A number of models (Simonyan and Zisserman, 2015;He et al., 2015He et al., , 2016Conneau et al., 2016) increase the number of feature maps whenever downsampling is performed, causing the total computational complexity to be a function of the depth.

we fix the number of feature maps, as we found that increasing the number of feature maps only does harm -increasing computation time substantially without accuracy improvement, as shown later in the experiments.


This approach alleviates the problem of vocabulary gaps between source and target to a certain degree.

this translation model is unable to handle semantic meaning.


(2016) proposed a joint-layer recurrent neural network model to extract keyphrases from tweets, which is another application of deep neural networks in the context of keyphrase extraction.

their work focused on sequence labeling, and is therefore not able to predict absent keyphrases.


We see that both models can generate phrases that relate to the topic of information retrieval and video.

most of RNN predictions are high-level terminologies, which are too general to be selected as keyphrases.


RNN and CopyRNN are supervised models, and they are trained on data in a specific domain and writing style.

with sufficient training on a large-scale dataset, we expect the models to be able to learn universal language features that are also effective in other corpora.


Siskind, 1996;Frank et al., 2007;Fazly et al., 2010).

several studies presented models that learn from sensory rather than symbolic input, which is rich with regards to the signal itself, but very limited in scale and variation (e.g.


The pattern observed for the two datasets is slightly different: due to the systematic conversion of words to synthetic speech in COCO, using the number of time steps for this dataset yields the highest R 2 .

this feature is not as informative for predicting the utterance length in Flickr8K due to noise and variation in human speech, and is in fact outperformed by some of the features extracted from the model.


This conclusion seems contradictory to the perspective of interactive alignment at the first glance.

here we are starting with a very highlevel model of dialogue that has does not refer to linguistic devices.


Ideal output responses should be both coherent and diverse.

most models end up with generic and dull responses.


Xing et al., (2016) maintain topic encoding based on Latent Dirichlet Allocation (LDA) (Blei et al., 2003) of the conversation to encourage the model to output more topic coherent responses.

many attempts have also been made to improve the architecture of encoderdecoder models.


On the contrary, the model with only KLA learns to encode substantial information in latent z when the KL cost weight is small.

after the KL weight is increased to 1 (after 5000 batch), the model once again decides to ignore the latent z and falls back to the naive implementation.


HCNs also use an RNN to accumulate dialog state and choose actions.

hCNs differ in that they use developer-provided action templates, which can contain entity references, such as "<city>, right?".


SL and RL correspond to different methods of updating weights, so both can be applied to the same network.

there is no guarantee that the optimal RL policy will agree with the SL training set; therefore, after each RL gradient step, we check whether the updated policy reconstructs the training set.


This construction intersects the languages of G and F (r s ), but because F (r s ) accepts all strings over the alphabet, the languages of G and G will be the same (namely, all REs for o u ).

the weights in G are inherited from F (r s ); thus the weight of each RE in L(G ) is the edit cost from r s .


That is to say, the error signal back propagated from the word sequence would not affect the structural label RNN, and vice versa.

in the Hierarchical RNN encoder, the error signal back propagated from the word sequence has a direct impact on the structural label annotation vectors, and thus on the structural label embeddings.


Our approach is similar to theirs by using source-side phrase parse tree.

our Mixed RNN system, for example, incorporates syntax information by learning annotation vectors of syntactic labels and words stitchingly, but is still a sequenceto-sequence model, with no extra parameters and with less increased training time.


(2016) design a few experiments to investigate if the NMT system without external linguistic input is capable of learning syntactic information on the source-side as a by-product of training.

their work is not focusing on improving NMT with linguistic input.


"NMT" refers to the translation result from a conventional NMT model, which fails to capture the long distance word relation denoted by the dashed arrow.

it is not trivial to build and leverage syntactic structures on the target side in current NMT framework.


the agreement between annotators is not high and the set of assigned labels do not adhere to a specific theory of emotion.


This study analyzed user-level political ideology through Twitter posts.

to previous work, we made use of a novel data set where finegrained user political ideology labels are obtained through surveys as opposed to binary self-reports.


This knowledge is then used as input to PSL Model 1 via the rule: UNIGRAM F (T, U) !FRAME(T, F) (shown in line 1 of Table 3).

not every tweet will have a unigram that matches those in this list.


In preliminary experiments for the hybrid models, we found that selecting the same vocabulary of 30K words for the source and target based on combined frequency was better (.003 in F 0.5 ) and use that method for vocabulary selection instead.

there was no gain observed by using such a vocabulary selection method in the baseline.


We conjecture that terms in scientific literature are often overloaded (e.g., a tree could be a parse tree or a decision tree), necessitating some abstraction when representing ideas.

news stories are more self-contained and seek to employ consistent usage.


The procedure generally builds on steps designed to assemble the SICK corpus, which contains pairs of English sentences annotated for semantic relatedness and entailment, because we aim at building a comparable dataset.

the implementation of particular building steps significantly differs from the original SICK design assumptions, which is caused by both lack of necessary extraneous resources for an investigated language and the need for language-specific transformation rules.


The standard measure of inter-annotator agreement in various natural language labelling tasks is Cohen's kappa (Cohen, 1960).

this coefficient is designed to measure agreement between two annotators only.


As we aim at building an evalua-tion dataset which is comparable to the SICK corpus, the general assumptions of our procedure correspond to the design principles of the SICK corpus.

the procedure of building the SICK corpus cannot be adapted without modifications.


The presented procedure of building a dataset was tested on Polish.

it is very likely that the annotation framework will work for other Slavic languages (e.g.


The question is answerable simply by noticing one sentence, without needing to fully understand the content of the text.

consider the second example from MCTest (Richardson et al., 2013), which was written for children and is easy to read.


We applied these metrics only to sentences that needed to be read in answering questions.

because these metrics were proposed for human readability, they do not necessarily correlate with those used in RC systems.


This appears to have been caused by the automated sourcing methods, which may generate a separation between the contents of the context and question (i.e., web segments and a search query in MS MARCO, and a context article and question article in Who-did-What).


Although its questions are created automatically, they are sophisticated in terms of knowledge reasoning.

the automated sourcing method must be improved to exclude nonsense questions.


one problem is that the dataset contained nonsense questions.


Most existing RC datasets use such texts because of their availability.

narrative texts may have a closer correspondence to our everyday experience, involving the emotions and intentions of characters (Graesser et al., 1994).


QA4MRE required the longest distance because readers had to look for clues in the long context texts.


This transforms the parsing problem back into a sequence-to-sequence problem, while making it easy to force the decoder to take only actions guaranteed to produce well-formed outputs.

transition-based models do not admit fast dynamic programs and require careful feature engineering to support exact search-based inference (Thang et al., 2015).


These models enjoy a number of appealing formal properties, including support for exact inference and structured loss functions.

previous chart-based approaches have required considerable scaffolding beyond a simple well-formedness potential, e.g.


As with the chart parsing formulation, we also use a margin-based method for learning under the topdown model.

rather than requiring separation between the scores of full trees, we instead enforce a local margin at every decision point.


(2016b) and Cai and Zhao (2016), we use word context on top of character context.

words play a relatively less important role in our model, and we find that word LSTM, which has been used by all previous neural segmentation work, is unnecessary for our model.


Because this is a significant problem for neural language and translation models, there are a number of methods proposed to resolve this problem, which we detail in Section 2.2.

none of these previous methods simultaneously satisfies the following desiderata, all of which, we argue, are desirable for practical use in NMT systems: Memory efficiency: The method should not require large memory to store the parameters and calculated vectors to maintain scalability in resource-constrained environments.


The hierarchical softmax (Morin and Bengio, 2005) predicts each word based on binary decision and reduces computation time to O(H log V ).

this method still requires O(HV ) space for the parameters, and requires calculation much more complicated than the standard softmax, particularly at test time.


Sampling-based approximations (Mnih and Teh, 2012;Mikolov et al., 2013) to the denominator of the softmax have also been proposed to reduce calculation at training.

these methods are basically not able to be applied at test time, still require heavy computation like the standard softmax.


We can easily obtain the maximum-probability bit array from q by simply assuming the i-th bit is 1 if q i ≥ 1/2, or 0 otherwise.

this calculation may generate invalid bit arrays which do not correspond to actual words according to the mapping between words and bit arrays.


This is expected, as described in Section 3, because using raw bit arrays causes many one-off estimation errors at the output layer due to the lack of robustness of the output representation.

hybrid-N and Binary-EC models clearly improve BLEU from Binary, and they approach that of Softmax.


Neural machine translation (MT) models obtain state-of-the-art performance while maintaining a simple, end-to-end architecture.

little is known about what these models learn about source and target languages during the training process.


The improved translation quality is often attributed to better handling of non-local dependencies and morphology generation (Luong and Manning, 2015;Bentivogli et al., 2016;Toral and Sánchez-Cartagena, 2017).

little is known about what and how much these models learn about each language and its features.


(2016), who use hidden vectors from a neural MT encoder to predict syntactic properties on the English source side.

we focus on representations in morphologically-rich languages and evaluate both source and target sides across several criteria.


Recently, a number of approaches to multimodal sentiment analysis, producing interesting results, have been proposed (Pérez-Rosas et al., 2013;Wollmer et al., 2013;Poria et al., 2015).

there are major issues that remain unaddressed.


Normally, doing something similar, i.e., monotonous or repetitive might be perceived as negative.

the nearby utterances "It engages the audience more", "they took a new spin on it", "and I just loved it" indicate a positive context.


The textual modality, combined with non-textual modes, boosts the performance in IEMOCAP by a large margin.

the margin is less in the other datasets.


The performance was poor for audio and textual modality as the MOUD dataset is in Spanish while the model is trained on MOSI dataset, which is in English language.

notably the visual modality performs better than the other two modalities in this experiment, which means that in cross-lingual scenarios facial expressions carry more generalized, robust information than audio and textual modalities.


For the fusion, the hierarchical fusion framework was used.

information from neighboring utterances, e.g., "And I really enjoyed it" and "The countryside which they showed while going through Ireland was astoundingly beautiful" indicate its positive context and help our contextual model to classify the target utterance correctly.


For example, the utterance "who doesn't have any presence or greatness at all" was classified as positive by the audio classifier (as "presence and greatness at all" was spoken with enthusiasm).

the textual modality caught the negation induced by "doesn't" and classified it correctly.


The lexicon designers then manually contructed lexicons for each category, augmenting their intuitions by using distributional statistics to suggest words that may have been missed (Pennebaker et al., 2015).

we follow the approach of Biber (1991), using multidimensional analysis to identify latent groupings of markers based on co-occurrence statistics.


The chance rate would be 50%, so this supports Hypothesis I and 9 It is possible that inflections are semantically similar, because by definition they are changes in the form of a word to mark distinctions such as tense, person, or number.

different inflections of a single word form might be used to mark different stances (e.g., some stances might be associated with the past while others might be associated with the present or future).


The relatively modest effect sizes are unsurprising, given the short length of the texts.

these differences lend insight to the relationship between formality and politeness, which may seem to be closely related concepts.


In addition to facilitating discovery of topical trends (Gardner et al., 2010), topic modeling is used for a wide variety of problems including document classification (Rubin et al., 2012), information retrieval (Wei and Croft, 2006), author identification (Rosen-Zvi et al., 2004), and sentiment analysis (Titov and McDonald, 2008).

the most compelling use of topic models is to help users understand large datasets (Chuang et al., 2012).


Once Q is constructed, topic recovery requires O(KV 2 + K 2 V I), where K is the number of topics, V is the vocabulary size, and I is the average number of iterations (typically 100-1000).

traditional topic Anchor Top Words in Topics  backpack  backpack camera lens bag room carry fit  cameras equipment comfortable  camera  camera lens pictures canon digital lenses  batteries filter mm photos  bag  bag camera diaper lens bags genie smell  room diapers odor   Table 1: Three separate attempts to construct a topic concerning camera bags in Amazon product reviews with single word anchors.


The anchor word "backpack" may seem strange.

this dataset contains nothing about regular backpacks; thus, "backpack" is unique to camera bags.


This is in harmony with our quantitative measurements of topic coherence, and may be the result of our stopping criteria: when users judged the topics to be useful.

100% of our users feel that the topics created through interaction were better than those generated from Gram-Schmidt anchors.


Recurrent networks (RNNs) (Elman, 1990) have recently become very popular for sequence tagging tasks such as entity extraction that involves a set of contiguous tokens.

their ability to identify relations between non-adjacent tokens in a sequence, e.g., the head nouns of two entities, is less explored.


During training, we pass the gold label embedding to the next time step which enables better training of our model.

at test time when the gold label is not available we use the predicted label at previous time step as input to the current step.


Thus, for our approach described so far if we only compute the argmax on our objective then we limit our model to output only one relation label per token.

from our analysis of the dataset, an entity may be related to more than one entity in the sentence.


We can also use Sparsemax (Martins and Astudillo, 2016) instead of softmax which is more suitable for sparse distributions.


Table 4 clearly shows that the overall performance of all methods is lower for verb similarity.

the improvement using both signed clustering as well as thesaurus look is also larger.


based on manually engineered semantic grammars (Jia and Liang, 2016)) and learning through direct interaction with users (e.g., where a single user teaches the model new concepts (Wang et al., 2016)).

there are unique advantages to our approach, including showing (1) that non-linguists can write SQL to encode complex, compositional computations (see Fig 1 for an example), (2) that external paraphrase resources and the structure of facts from the target database itself can be used for effective data augmentation, and (3) that actual database users can effectively drive the overall learning by simply providing feedback about what the model is currently getting correct.


All three of these languages are modeled after natural language to simplify parsing.

none of them is used to query databases outside of the semantic parsing literature; therefore, they are understood by few people and not supported by standard database implementations.


GUSP (Poon, 2013) creates an intermediate representation that is then deterministically converted to SQL to obtain an accuracy of 74.8% on ATIS, which is boosted to 83.5% using manually introduced disambiguation rules.

it requires a lot of SQL specific engineering (for example, special nodes for argmax) and is hard to extend to more complex SQL queries.


To date, most efforts to leverage discourse information to detect salient content from dialogues have focused on encoding gold-standard discourse relations as features for use in classifier training (Murray et al., 2006;Galley, 2006;McKeown et al., 2007;Bui et al., 2009).

automatic discourse parsing in dialogues is still a challenging problem (Perret et al., 2016).


This makes the problem computationally easier by enabling the use of maximum spanning tree-style parsing ap- proaches.

argumentation in the wild can be less well-formed.


Possible link types are reason and evidence, and propo-sition types are split into five fine-grained categories: POLICY and VALUE contain subjective judgements/interpretations, where only the former specifies a specific course of action to be taken.

tEStIMONY and FACt do not contain subjective expressions, the former being about personal experience, or "anecdotal."


Xiao and Cho (2016) extended that architecture by inserting a recurrent neural network layer between the convolutional layer and the classification layer.


(2010) developed a two-step approach by first predicting implicit connectives whose sense is then disambiguated to obtain the relation.

the pipeline approach usually suffers from error propagation, and the method itself has relied on hand-crafted features which do not necessarily generalize well.


Our experiments will demonstrate that ∆ ρ can be accurately learned from data.

the features we used for this are not factorizable over the edges of the latent trees.


Note that Björkelund and Kuhn (2014) perform inexact search on the same latent tree structures to extend the model to non-local features.

to our approach, they use beam search and accumulate the early updates.


Lexical resources such as dictionaries and gazetteers are often used as auxiliary data for tasks such as part-of-speech induction and named-entity recognition.

discriminative training with lexical features requires annotated data to reliably estimate the lexical feature weights and may result in overfitting the lexical features at the expense of features which generalize better.


Particle filtering is typically used in online settings, including word segmentation (Borschinger and Johnson, 2011), to make decisions before all of x has been observed.

we are interested in the inference (or smoothing) problem that conditions on all of x (Dubbin and Blunsom, 2012;Tripuraneni et al., 2015).


If monthly is listed in (only) the adjective lexicon, this tells us that P ADJ sometimes generates monthly and therefore that H ADJ may also tend to generate other words that end with -ly.

for us, P ADV (monthly) > 0 as well, allowing us to still correctly treat monthly as a possible adverb if we later encounter it in a training or test corpus.


our method is different from the above methods in that we apply domain adaptation techniques to the outputs of a generative model rather than a natural data domain.


Various neural models based on attention mechanisms (Wang and Jiang, 2016;Seo et al., 2016;Xiong et al., 2016;Dhingra et al., 2016;Kadlec et al., 2016;Trischler et al., 2016b;Sordoni et al., 2016;Cui et al., 2016; have been proposed to tackle the tasks of question answering and reading comprehension.

the performance of these neural models largely relies on a large amount of labeled data available for training.


model policy puts near-uniform probability over the decisions at each time step.

this causes shorter programs to have orders of magnitude higher probability than longer programs, as illustrated in Figure 2 and as we empirically observe.


In REINFORCE, a common solution is to sample from an -greedy variant of the current policy.

mmL exploration with beam search is deterministic.


In our experiments, J MML performs significantly better than J RL (Table 4).

while J MML assigns uniform weight across examples, it is still not uniform over the programs within each example.


More uniform upweighting across reward-earning programs leads to higher accuracy and fewer spurious programs, especially in SCENE.

no single value of β performs best over all domains.


On the depicted training example, BS-MML actually achieves higher expected reward / marginal probability than RANDOMER, but it does so by putting most of its probability on a spurious programa form of overfitting.

rANDOMEr spreads probability mass over multiple rewardearning programs, including the correct ones.


This could be because their method is very aggressive in dealing with the history (as explained later in the Experiments section).

our method has a better way of handling history (by passing context vectors through an LSTM recurrent network) which gives us the flexibility to forget/retain some portions of the history and at the same time produce diverse context vectors at successive time steps.


Comparison with baseline diversity models: The baseline diversity model M1 performs at par with our models D1 and SD1 but not as good as D2 and SD2.


Due to the difficulty of abstractive summarization, the great majority of past work has been extractive (Kupiec et al., 1995;Paice, 1990;Saggion and Poibeau, 2013).

the recent success of sequence-to-sequence models (Sutskever  Figure 2: Baseline sequence-to-sequence model with attention.


In general, ROUGE-1 and ROUGE-2 were considered as the baselines for validating the performance of AP because these variants strongly correlate with human evaluation methods (Owczarzak et al., 2012a,b).

the comparison could be repeated with ROUGE-3, ROUGE-4 and ROUGE-BE, which have been found to predict manual Pyramid better than ROUGE-1 and ROUGE-2 (Rankel et al., 2013).


Some previous works apply this framework to summarization generation tasks Gu et al., 2016;Gulcehre et al., 2016).

abstractive sentence summarization is different from MT in two ways.


The most widely used metric for evaluating such dialogue systems is BLEU (Papineni et al., 2002), a metric measuring word overlaps originally developed for machine translation.

it has been shown that BLEU and other word-overlap metrics are biased and correlate poorly with human judgements of response quality (Liu et al., 2016).


For example, the two responses in Figure 1 are equally appropriate given the context.

if we simply change the context to: "Have you heard of any good movies recently?


For training VHRED, we use a context embedding size of 2000.

we found the ADEM model learned more effectively when this embedding size was reduced.