To do this, we employ a CNN-based model to locally disambiguate the entities.

sentence E1.2 is linked correctly to the country "United States", and thus is given a low d value..


To achieve the above goal, we first infer the representations of entity descriptions using relation representation as attention: where r ∈ d w is the representation of the relation mention by averaging all the hidden vectors of BiLSTM, h i is the hidden representation of w i , and W e ∈ d w×2×h is a trained parameter matrix.

the parameters of our model are optimized using the stochastic gradient descent (SGD) algorithm.


Concretely, we employ BiLSTM model (Schuster and Paliwal, 1997;Graves and Schmidhuber, 2005) with mutual attention mechanism  to learn representations for relation mentions and entity descriptions.

(ii) We propose a mutual attention mechanism which exploits the textual representations of relation and entity to enhance each other (Section 3.2).


This section presents our accurate text-enhanced knowledge graph representation learning framework.

several illustrative triples from the test set of FB15K are listed in Table 4.


In our research, we propose an automatic phrase abduction mechanism to inject phrasal knowledge during the proof construction process.

in the presence of two or more variables with the same predicate, there might be multiple possible variable unifications.


There is one critical reason that the word-toword axiom injection described in Section 3.2 fails to detect phrase-to-phrase correspondences.

then, the negation in G is removed by applying the introduction rule (¬-I) to G. Here, False is the propositional constant denoting the contradiction.


We represent logical formulas using graphs, since this is a general formalism that is easy to visualize and analyze.

this work was supported by JSt CRESt Grant Number JPMJCR1301 and AIP Challenge Program, Japan.


Moreover, we find that it can accurately identify useful key phrases such as officials declared the video, according to previous reports, believed will come, president in his tweets as supporting pieces of evidence, and proved a hoax, shot down a cnn report, would be skeptical as opposing pieces of evidence.

the rationale behind using the similarity matrix is that in our memory network model, as Figure 3 shows, we seek a transformation of the input claim such that s = M × s in order to obtain the closest facts to the claim.


Next, the relevant parts of the input are identified in F , and afterwards they are used by G to update the memory.

memory networks are exactly designed to remember previous information.


After a set of preliminary experiments, we select the beam size from {3, 4, 5, 6}.

less expressive models may be preferred as they may learn more abstract representations that will generalize better to outof-domain data.


We replace the appearance of a word with its refined word vector.

we will base our semantic distance δ on a slight variation of the DESM similarity metric.


The results of the LogMap system show a quite similar behavior with the experiments conducted in the conference dataset.

the basic idea is that some alignments that were omitted by the Stable Marriage solution were very close to the optimal alignment and they should also be included in the final alignment set.


In any case, however, we retain a small precision which indicates a semantic similarity and conceptual association coalescence.

despite the use of refined word vectors, we cannot completely avoid the problems that come from the semantic similarity and conceptual association coalescence.


In this section, we introduce our proposed architecture for RNNs.

we report BLEU score results together with number of parameters of recurrent layers.


The model is almost same as the one used in De-En machine translation, which is a two layers RNNSearch model, except that the embedding size is 512, and the LSTM hidden size in both encoder and decoder is 512.

it fails to capture dependency cross different groups.


The group recurrent layer in Equation 4 and 5 can be re-formulated as From the reformulation, we can see group recurrent layer is equivalent to standard recurrent layer with block-diagonal sparse weight matrix.

training such big models usually takes days or even weeks of time even if using tens of GPU cards.


mutations beyond adding "not" to the original).

of these, 28.51% (n = 274) had the correct predicted label but did not satisfy the requirements for evidence.


Vlachos and Riedel (2014) constructed a dataset for claim verification consisting of 106 claims, selecting data from fact-checking websites such as PolitiFact, taking advantage of the labelled claims available there.

for instance, it would be interesting to test how approaches similar to natural logic inference (Angeli and Manning, 2014) can be applied, where a knowledge base/graph is constructed by reading the textual sources and then a reasoning process over the claim is applied, possibly using recent advances in neural theorem proving (Rocktäschel and Riedel, 2017).


We partitioned the annotated claims into training, development and test sets.

despite the rising interest in verification and fact checking among researchers, the datasets currently used for this task are limited to a few hundred claims.


By the end of this section a bipartite relation graph like Figure 2 will be constructed, with one node set being textual relations T , and the other being KB relations R. The edges are weighted by the normalized co-occurrence statistics of relations.

for the embedding model, the mini-batch size is set to 128, and the state size of the GRU cells is 300.


Similar to previous work (Riedel et al., 2010;Zeng et al., 2015), we use two settings for evaluation: (1) Held-out evaluation, where a subset of relational facts in KB is held out from training (Table 1), and is later used to compare against newly discovered rela- tional facts.

the results in Figure 6 show that pairwise ensemble of existing relation extraction models does not yield much improvement, and GloRE brings much larger improvement than the other models.


We want the coherence score between e c and e p to be close to 1, while the score for e c and e n should be close to 0.

this task is closely related to our argument cloze task.


For example, given three nodes, e1, e2, and e3, a local method can possibly classify (e1,e2)=before, (e2,e3)=before, and (e1,e3)=after, which is obviously wrong since before is a transitive relation and (e1,e2)=before and (e2,e3)=before dictate that (e1,e3)=before.

time is an important dimension of knowledge representation.


In practice, e.g., in the TBDense dataset , roughly 30%-40% of the P-Before pairs are T-After.

the tempRels between events can be represented by an edge-labeled graph, where the nodes are events, and the edges are labeled with tempRels (Chambers and Jurafsky, 2008;Do et al., 2012;Ning et al., 2017).


In Example 2, where we show the complete sentences, the task has become much easier for humans due to our prior knowledge, namely, that explosion usually leads to casualties and that people usually ask before they get help.

very limited attention has been paid to generating such a resource and to make use of it; to our knowledge, the TEMPROB proposed in this work is completely new.


"Marshmellooooo" with trailing 'o's) and amplifies character-level features intsead (e.g.


At each token t and for each gold label at the previous step g k t−1 , our network takes the hidden representation from the previous layer z from the previous time step, and computes: Unlike the encoder LSTM, this decoder LSTM is single-directional and bifurcates when multiple gold labels are present.

gold-standard annotations are shown in red.


Each of these baselines uses mention-specific features encoding relative position of each token to the two target entities being classified, whereas our model aggregates over all mention pairs in each sentence.

this could be ameliorated by integrating our model into open relation extraction architectures such as Universal Schema (Riedel et al., 2013;Verga et al., 2016b).


These baselines allow us to compare the representation power independent of the classifier.

we use our prepositional representations as simple features to a standard classifier on this task.


It is a variant of the popular particle filtering method that tracks the state of a physical system in discrete time (Ristic et al., 2004).

this suggests that an accurate HMM language model p(x) would require very large kas would a generative OOHMM model p(x, y) of annotated language.


We investigate the feasibility of neural syntactic generative models with structured latent variables in which exact inference is tractable.

the sentence is encoded left-to-right by an LStM RNN taking the word embedding of the last predicted word as input at each time step, independent of t. the RNN hidden states h 0:n represent each sentence position in its linear context.


In cases of spurious ambiguity arcs are added as soon as possible.

unsupervised training based on this model cannot learn to predict arc directions.


We observe the same pattern for the embedding space mapping approach for noise reduction against the narrow window embeddings.

2 The pipeline so far results in a more consistent version of the text, which we use to learn the final embeddings upon.


Depending on whether this is an example of a zero copula construction, or a clausemodified noun, either annotation is plausible.

authors of this paper manually corrected the parsed data and finally achieved 3,550 labeled tweets.


We witnessed a small number of tweets that contain multi-token words (e.g., Y O, and R E T W E E T) but didn't combine them for simplicity.

when it comes to informal, unedited, user-generated text, the guidelines may leave many annotation decisions unspecified.


So far, not much has been shown about how neural networks are able to compensate for the removal of the structures used in past models.

shuffling does cause performance to degrade relative to the base parser even when the unshuffled win-  dow is moderately large, indicating that the LSTM is propagating information that depends on the order of words in far-away positions.


8 As COMBO results show, the representations induced from different corpora are somewhat complementary.

unlike hierarchical softmax, CSS only affects training, that is, at test time we simply use the entire support instead of the approximation.


The unobserved term can be either real zero (two words shouldn't be co-occurred even when we use very large corpus) or just missing in the small corpus.

in the experiments, we evaluate all models with u avg .


Our expanded lexicon,  which roughly preserves that ratio, includes about 800 adjectives in total.

there have been investigations examining features on various datasets (Nobata et al., 2016;Samghabadi et al., 2017), however, these studies always trained and tested on the same domain.


Due to space limitations, we cannot list the other classifiers.

many of our features should also be applicable to other languages.


The results from our statistical analysis validate our original hypothesis that power relations do correlate with the level of commitment people express in their messages.

since verbosity of a participant can be highly correlated with each of these feature values (we found it to be highly correlated with subordinates (Prabhakaran and Rambow, 2014)), we added token count as a control variable to the linear regression.


These approaches are motivated from an information extraction perspective, for instance in aiding tasks such as knowledge base population.

for example, let us consider two sentences: I need the report by tomorrow vs.


Figure 1 pictorially demonstrates these results by plotting the difference between the mean values of each commitment feature (here normalized by token count) of superiors vs. subordinates, as a percentage of mean feature value of the corresponding commitment feature for superiors.

tHR thread structure (e.g., reply rate) DIA Dialog act tagging (e.g., request count) ODP Overt displays of power LEX Lexical ngrams (lemma, POS, mixed ngrams) None of the features used in POWERPREDIC-tOR use information from the parse trees of sentences in the text in order to accurately obtain the belief labels, deep dependency parse based features are critical (Prabhakaran et al., 2010).


This approach supports explanations based on interpretable features (e.g., words) even when the underlying representation may be less interpretable.

we measure local fidelity by deleting words in the order of their estimated importance for the prediction.


For example, compare the random approach with the omission approach on true positives with ten word explanations.

it is unclear to what extent they correspond with human-based evaluations.


We compute cosine similarity of the key-terms defined for each selected topic and topics discovered by the topic models over the years.


Following monolingual coherence evaluations (Lau et al., 2014), we present topic pairs to bilingual CrowdFlower users.

as expected, CNPMI outperforms INPMI regardless of reference corpus overall, because INPMI only considers monolingual coherence.


After training an estimator (Section 4.2), we calculate Pearson's correlation between Wikipedia's CNPMI and the estimated topic coherence score (Table 3).

topic models are often used as a feature extraction technique for downstream machine learning  applications, and topic model evaluations should reflect whether these features are useful (Ramage et al., 2009).


Crosslingual Gap (GAP) A low CNPMI score could indicate a topic pair where each language has a monolingually coherent topic but that are not about the same theme (Topic 6 in Figure 1).

hao has been supported under subcontract to Raytheon BBN Technologies, by DARPA award hR0011-15-C-0113.


Prior work has also em-phasized explainability.

we remove those note-label pairs for which no -gram has a score greater than 0, which gives an "unfair" advantage to this baseline.


This fact limits the usefulness of these tools for problems involving styles of language not represented in large annotated training sets.

existing NLI datasets like SNLI have facilitated substantial advances in modeling, but have limited headroom and coverage of the full diversity of meanings expressed in english.


All three models achieve accuracy above 80% on the SNLI test set when trained only on SNLI.

in the mixed setting, we use the full MultiNLi training set and randomly select 15% of the SNLi training set at each epoch, ensuring that each available genre is seen during training with roughly equal frequency.


We do not report all these results for brevity and clarity of presentation.

each model uses a different strategy to exploit the hierarchical relationships encoded in these constraints (their approaches are discussed in Section 5).


Specifically, we use BiLSTMs layers with 512 units each.

we also demonstrate that the proposed parser achieves state-of-the-art performance in the downstream tasks of Parsing Evaluation using Textual Entailments (PETE) and Unbounded Dependency Recovery.


In this case, UDR would pick the relation (what, hope, pobj).

for this reason, when a lemma of a verb is a non-be copula, 10 we add arcs involving the word to the copula adjoining into the copula.


Copulas A copula is usually treated as a dependent to the predicate both in our TAG grammar (adjunction) and UDR.

an example is the UDR relation (those, stayed, nsubj) "in the other hemisphere it is growing colder and nymphs, those who stayed alive through the summer, are being brought into nests for quickening and more growing" where our parser yields (those, alive, 0).


Ideally, the transfer performance could be estimated by training a MNet on task i and directly evaluating it on task j.

adaptive ROBUSTTC-FSL although the RO-BUSTTC-FSL improves over baselines on intent classification, the margin is smaller compared to that on sentiment classification, because the intent classification tasks are more diverse in nature.


Additionally, since the observed entries of Y are generated based on high and low enough performance, it is safe to assume that most observed entries are correct and only a few may be incorrect.

the k-th metric of the cluster is thus to build a predictor M with access to only a limited number of training samples, we make the prediction probability by linearly combining prediction from learned cluster-encoders: where f k is the learned (and frozen) encoder of the k-th cluster, {↵ k } K k=1 are adaptable parameters trained with few-shot training examples.


The analysis complements the evidence, lower resolutions have higher IG at early chunks, whereas higher 6 The only exception to this is R1, which has the lowest overall performance.

late stages require to exploit as much evidence as possible to make accurate predictions.


maining three are used as source domains.

for all three of our experiments, we use = 0.05 and k = 5 (See Algorithm 1).


To make fair comparisons, the previous experiments follow the standard settings in the literature, where the widely adopted Amazon review dataset is used.

each domain has a development set of 200 samples, and a test set of 400 samples.


1 PBLM learns the connection between easy -an adjective that is often used to describe kitchen appliances, but not books -and great.


There also exists an unlabeled set containing large amounts of collected samples without annotation.

the selection of samples in existing co-training methods is based on a predetermined policy, which ignores the sampling bias between the unlabeled and the labeled subsets, and fails to explore the data space.


Performance improvements in these more recent models are mainly due to using better image features such as those obtained by Region-based Convolutional Neural Networks (R-CNN), or using reinforcement learning (RL) to directly optimize metrics such as CIDEr, or using more complex attention mechanisms (Gan et al., 2017) to provide a better context vector for caption generation, or using an ensemble of multiple LSTMs, among others.

1, the state updating equations are: pt Here The initial state p 0 is the zero vector.


∀l ∈ [1..L] : e l = P osition Encoder(y c l ) (9) We then put a distribution over the candidate responses conditioned on the summarized dialog history h his (Equation 10).

in Figure 5, the user already mentioned that he/she wants to find a "cheap" restaurant, but the GMN and QRN seem to "forget" this information.


Our RNN units use a different gate arrangement than that used by RAN.

(2017), we construct a templatized set of responses.


Our experiments show that: (1) the encoder with character attention achieves significant improvements over the standard word-based attention-based NMT system and a strong character-based NMT system; (2) incorporating source character information into the decoder by our multi-scale attention mechanism yields a further improvement, and (3) our modifications also improve a subword-based NMT model.

many recent studies have focused on using character-level information in neural machine translation systems.


The standard encoder operates purely on (sub)words or characters.

table 5(a) shows the translation of an OOV word 通 信 业 (tong-xin-ye, telecommunication industry).


(c)  Prior works show a trend of designing more expressive attention mechanisms (as discussed in Section 2).

the decoder layers {z l } follow similar structure, while getting extra representations from the encoder side.


when a retrieved source sentence is not very similar with the input sentence (e.g.


With this result in hand, we propose a method for more directly capturing contextual information that may help disambiguate difficult-to-translate homographs.

lastly, we show sample translations of the baseline system and our proposed model.


In this work, we take the policy gradient training strategies following .

to their approach, we apply the CNN-based discriminator for the machine translation task.


The basic idea is to augment the MTL training objective with additional terms, so that the identity of a task cannot be predicted from its data items by the representations resulted from the shared encoder/decoder RNN layers.

nMT is notorious for its need for large amounts of bilingual data (Koehn and Knowles, 2017) to achieve reasonable translation quality.


Implementation We compare the proposed models with two strong baselines from SMT and NMT: • Moses (Koehn et al., 2007): an open source phrased based translation system with default configuration.

for the MAP feeding style, we optimize u i according to the loss function in Eq.


(2017) enhance attention through a planning mechanism.

there is a significant difference between our approach and their approaches.


In some languages (e.g., Hungarian, in the top left) Lematus performs very well on unseen words even with little training data, while in others (e.g., Arabic, along the bottom) it performs poorly despite relatively large training data.

swim) given one of its inflected variants (e.g.


(2017) used their model for machine translation, while we work on lemmatization.

when making predictions we use beam-search decoding with a beam of size 12.


For example, one might expect that for languages with more training data, the system would learn better generalizations and lemmatization accuracy on unseen words would be higher.

evaluation To evaluate models, we use test and development set lemmatization exact match accuracy.


Lemming (which proves to be the strongest baseline) also consists of two log-linear components (a classifier for lemmatization and a sequence model for tagging), which are combined either using a pipeline (first tag, then lemmatize) or through joint inference.

we reduced the number of hidden units to 100 and the encoder and decoder embedding size to 300.


Some years later, many researchers incorporated machine learning algorithms to their systems, but there was still a strong dependency on external resources and domain-specific features and rules (Tjong Kim Sang and De Meulder, 2003).

the most important differences of our approach and previous works are i) the use of phonetics and phonology (articulatory) features at the character level to model SM noise, ii) consistent BLStMs for character and word levels, iii) the segmentation and categorization tasks, iv) a multitask neural network that transfers the learning without using lexicons or gazetteers, and v) weighted classes to handle the inherent skewness of the datasets.


The surface form F1 metric  supports that intuition as well.

we emphasize that both datasets present significantly different challenges and, thus, some relevant aspects in CoNLL 2003 may not be that relevant in the WNUT 2017 dataset.


Furthermore, observe that even our vanilla model (LSTM) significantly outperforms the equivalent "FOR-WARD" models by (Gangal et al., 2017) 12 that use greedy and beam-search decoding (1.90 and 2.37 vs 1.75).

such word formation patterns are also evident in other languages (Štekauer et al., 2012) and it is an open question as to whether similar models generalize to other languages as well.


Since multiple sequences of abstract morphemes may in general give rise to a single output form, 1 we marginalize these, i.e., where GEN(a) gives the surface word form produced from the morpheme sequence a.

to these previous works, we demonstrate the utility of incorporating morphological information in these open-vocabulary models.


One reason is that our model The sparse instances on the high nested levels could be another reason that resulted in the zero performances on the last flat NER layer.

cRFs are used to globally predict label sequences for any given sequences.


Additionally, 17% of the errors belong to type error.

sophia Ananiadou acknowledges BBsRC BB/P025684/1 Japan Partnering Award and BB/M006891/1 Empathy.


In deep learning models, such purpose is often achieved with a soft attention mechanism.

it is noteworthy that DR-BiLSTM (Single) performs better than ESiM in more frequent categories.


However, the model would suffer from the absence of similarity and closeness measures.

u i ) and its context depending on the other sentence (e.g.


State-of-the-art FrameId systems rely on pretrained word embeddings as input (Hermann et al., 2014).

we assign sentences to 100 buckets based on their IDs and create a 70/15/15 split for training, development, and test sets based on the bucket order.


Regarding FrameId, to the best of our knowledge, multimodal approaches have not yet been investigated.

the initial situation for the German case is more difficult.


They use a linear program to jointly predict frames and arguments at test time.

in semantic dependencies, the head of an arc is analogous to the target in frame semantics, the destination corresponds to the argument, and the label corresponds to the role.


The cross-task score is given by The computation of s c is described in §4.4.

this leads to potentially very slow inference.


Gold supervision for frame-semantic parses comes from the FrameNet lexicon and corpus (Baker et al., 1998).

as new annotation efforts cannot be expected to use the same original texts as earlier efforts, the utility of this approach is limited.


They filter out noise and select relevant in-domain examples jointly, using similarities between sentence embeddings obtained from the encoder of a bidirectional neural MT system trained on clean indomain data.

we generate negative examples automatically as described in Section 3, and sample a subset to maintain a 1:5 ratio of positive to negative examples.


We detect semantic divergence by computing the cosine similarity between sentence embeddings in a bilingual space.

candidate negative examples are generated starting from the positive examples {(e i , f i ) ∀i} and taking the Cartesian product of the two sides of the positive examples{(e i , f j )∀i, j s.t.


We optimized content alignment on the development set against manual alignments.

content selection and generation models (base encoder-decoder and MTL) were trained for 20 epochs with the ADAM optimiser (Kingma and Ba, 2014) using a learning rate of 0.001.


Block sizes were set to 40 (base), 60 (MTL) and 50 (RL).

in our scenario, the input data varies from one entity (e.g., athlete) to another (e.g., scientist) and properties might be present or not due to data incompleteness.


Instantiations of this framework include the widely-used attention-based sequenceto-sequence model (Bahdanau et al., 2015), in which f enc and f dec are implemented by an RNN architecture using LSTM (Hochreiter and Schmidhuber, 1997) or GRU (Chung et al., 2014) units.

the prevalent approach to training such a model is to update all the model parameters using all the examples in the training data (over multiple epochs).


other sequence of vectors used to produce output symbols step by step.

in many applications and for many natural language datasets, there exist multiple underlying distributions, characterizing a variety of language styles.


First, the amount of training data in these two languages is smaller than that in English.

although in this work we focus only on generating descriptions in one language, we hope that this dataset will also be useful for developing models which jointly learn to generate descriptions from structured data in multiple languages.


Intuitively, when a human writes a description from a table she keeps track of information at two levels.

this standard model is too generic and does not exploit the specific characteristics of this task.


This operation is also called the pointer sum attention ).

the final score m for a metric v is obtained by averaging over the test set: (4) Since there are multiple correct answers A, we take the highest scoring answerâ at each instance, as done in Rajpurkar et al.


The Ubuntu dataset, however, has diverse contents related to issues in using the Ubuntu system.

the LtC module provided additional performance improvements when combined with both RDE and HRDE models, as it added latent topic cluster information according to dataset properties.


They also tried using long short-term memory (LSTM) (Hochreiter and Schmidhuber, 1997), bidirectional LSTM and ensemble method with all of those neural network architectures and achieved the best results on the Ubuntu Dialogues Corpus dataset.

mobile, office, photo, tv/video, accessories, and home appliance as top-level categories, and specific categories like galaxy s7, tablet, led tv, and others are used.


We also consider unsupervised transfer learning where the correct answer to each question in the target dataset is not available.

although transfer learning has been successfully applied to various applications, its applicability to Qa has yet to be well-studied.


For +BOW style models, we take the matrix that compresses the text's word frequency vector, then score each word by computing the l 1 norm of the column that multiplies it, with the intuition that important words are dotted with big vectors in order to be a large component of e. Motivation.

both elicit lexicons by considering learned weights or attentional scores.


Most of this prior work uses lexical, syntactic, discourse, and dialog interactive features (Stab and Gurevych, 2014;Habernal and Gurevych, 2016;Wei et al., 2016), power dynamics (Rosenthal and Mckeown, 2017;Moore, 2012), or diction (Wei et al., 2016) to study discourse persuasion as manifested in argument.

and  propose an algorithm for counterfactual inference which bear similarities to our Adversarial Selector (Section 3.2), Imai et al.


The tweet's tokens can draw from 1 of 3 topics, which is good but a bit constraining.

similar to the symptom stage, this stage contributes to false positives because it also occurs with normal routine network problems, not just malicious acts.


Services can have trouble for a variety of reasons, not necessarily DDoS attacks.

a detection system may desire to optimize recall at the expense of precision, thus choosing a lower λ and forcing the system to predict attacks more often.


Yelp reviews: As a primary example of a large dataset with intrinsic labels, we make use of the Yelp10 dataset, treating the source location of the review as the label of interest.

estimating label proportions in a target corpus is a type of measurement that is useful for answering certain types of social-scientific questions.


Although they are not typically thought of in the context of estimating proportions, several methods have been proposed to deal directly with the problem of covariate shift, including kernel mean matching and its extensions (Huang et al., 2006;Sugiyama et al., 2011).

we select regularization strength via grid search, choosing the value that leads to the lowest average calibration error across training / held-out splits.


For our DCA models, we initialize the number of agents before training, and partition the document among the agents (i.e., three agent → three paragraphs).

similar to single agent model, it missed to capture the player who scored the only goal in the game.


For both Twitter and Weibo dataset, we randomly sample 80% for training, 10% for development, and the rest 10% for test.

avgEmb is the worst encoder on Twitter dataset, as word order is crucial in English.


Also, they only use internal information in the input text and ignore external knowledge in conversation context.

features learned for keyphrases and non-keyphrases are different, which can thus benefit keyphrase tagger to correctly distinguish keyphrases from non-keyphrases.


Understandably, the proposed approach is comparable in the preference to both the baselines for the major topic (0.5667) -since most standard summaries will cover the primary topic of the input.

early works on abstractive summarization were focused on sentence compression based approaches (Filippova, 2010;Berg-Kirkpatrick et al., 2011;Banerjee et al., 2015) which connected fragments from multiple sentences to generate novel sentences for the summary or template based approaches that generated summaries by fitting content into a template (Wang and Cardie, 2013;Genest and Lapalme, 2011).


Formally, we write the step-wise reward s(y t |y 1:t 1 , Y ⇤ ) as follows: where N (Y,Ỹ ) represents the occurrence of sub-sequenceỸ in whole sequence Y .

the uniform bridge adopts a uniform distribution U (Y ) as constraint.


Together with the generator network, three GBN systems are constructed: 1).


The average compression ratio of the gold compression for input sentence was 39.8.

since seq2seq is more expressive than Tagger, we built HisAN on the baseline seq2seq model.


For the last storyline "Egypt election", it starts in Day 20 and continues beyond the end of May.

to explore the efficiency of the proposed approach, we conducted an experiment by comparing the proposed approach NSEM with DSEM.


If further increasing S, the precision/recall have slight change and the F-measure become relatively stable.

it is reasonable to assume that the title h and its main body d of a document share a similar storyline distribution.


This problem can be formulated as an STKP on a star graph, whose vertex and edge sets are {r, r 1 , .

the results on approximation ratios and ROUGE 1 scores imply that our method compares favorably with the ILP-based method in terms of empirical performance.


Different from the naive greedy algorithm explained in Section 3, the above greedy algorithm is performed on the set of all rooted paths, P. Thus, even if beneficial vertices are far from r, rooted paths that include such beneficial vertices are considered as candidates to be chosen in each iteration.

the constraint that appears in DR-submodular maximization is somewhat easier to deal with than that of our problem; exploiting this, Soma and Yoshida (2017) developed a polynomial-time algorithm that achieves roughly 1 2 -approximation.


This approach is known to have three advantages: its applicability to many useful submodular objective functions, the efficiency of the greedy algorithm, and the provable performance guarantee.

, R K ⊆ V and function C e (S), which counts the number of times that n-gram e occurs in summary S ⊆ V , the ROUGE n score function is defined as We applied our method to compressive summarization tasks with the three kinds of objective functions: the coverage function, the one with rewards, and ROUGE 1 .


The more questions a system can answer, the better it is at summarizing the document as a whole.

sentence Encoder A core component of our model is a convolutional sentence encoder which encodes sentences into continuous representations.


We found that there are many candidate summaries with high ROUGE scores which could be considered during training.

we are not aware of any attempts to use reinforcement learning for training a sentence ranker in the context of extractive summarization.


For instance, after losing power in a second, 2004 coup Haiti's Jean Bertrand Aristide was forced into exile in South Africa.

8 Filtering the original dataset in this manner 9 yields 17,529 positive and 30,266 negative sentences.


We refer to these 3-tuples as "matching".

the details of how textual summaries might actually be presented to users are often ignored.


Graph-based Features Our graph-based features are similar to those described in Gorinski and Lapata (2015).

as can be seen, MLE outperforms Lib across attributes, and is superior to ZeroR by a large margin.


Biasing the sentence selection slightly to longer sentences is therefore promising.

the results are a first confirmation of the previously described intuition that predicting precision scores can be better than predicting recall scores.


The word drop probability is normally set to 0.25, since using a higher probability may degrade the model performance (Bowman et al., 2016).

the VHRED is trained to predict a next target utterance x t conditioned on the context h cxt t which encodes information about previous utterances {x 1 , .


The variational posterior is a factorized Gaussian distribution where the mean and the diagonal variance are predicted from the target utterance and the context as follows: where A known problem of a VAE that incorporates an autoregressive RNN decoder is the degeneracy that ignores the latent variable z.

the VHRED will not encode any information in the latent variable, i.e.


We followed the methodology presented in (Jovita et al., 2015) claiming that the best answer given by the system has the highest similarity value between the customer turn and the agent answer.

if, though, we can automatically detect the worst conversations (in our experience, typically under 10% of the total), the focus can be on fixing the worst problems.


We did not encounter any unsupported intent errors leading to customer rephrasing, which affected the ability of the rule-based model to classify those conversations as egregious.

in (Sarikaya, 2017) they reported on how the different reasons affect the users' satisfaction.


The effectiveness of CNNs for representing text has already been addressed in previous studies.

sHCNN outperforms most of the baselines even if only low-level (L) or high-level (H) representations are exploited.


This graphic model views the paths as discrete latent variables and relation as the observed variables with a given entity pair as the condition, thus the path-finding module can be viewed as a prior distribution to infer the underlying links in the KG.

during testing, we use a beam of 5 to approximate the whole search space for path finder.


DeepPath is trained to find paths more efficiently between two given entities while being agnostic to whether the entity pairs are positive or negative, whereas MIN-ERVA learns to reach target nodes given an entityquery pair while being agnostic to the quality of the searched path 1 .

in practice, we found that the KL-reward term log p β qϕ causes severe instability during training, so we finally leave this term out by setting w KL as 0.


The samples of FB15k-237 (Toutanova et al., 2015) are sampled from FB15k (Bordes et al., 2013), here we follow DeepPath (Xiong et al., 2017) to select 20 relations including Sports, Locations, Film, etc.

chains of reasoning and compositional reasoning only learn to predict relation given paths while being agnostic to the path-finding procedure.


For solving this, many dense annotation schemata are proposed to force annotators to annotate more or even complete graph pairs.

it can possibly provide useful information for the future processes of classification or time inference.


If the observation of a part of vague TLINKs induced as non-VAGUE TORDERs in the new data can be found, it will be the evidence.

tORDERs show extremely rare VAGUE labels in Event-DCt pairs.


ELDEN then adds edges connecting entities in E + to entities in E. This is done by processing a web text corpus looking for mentions of entities in E + , and linking the mentions to entities in KG G = (E + , F ).

densification is used for relation inference in the former methods whereas it is used for entity coherence measurement in EL-DEN.


TEMPLATEBASED combines the content with the target attribute markers in the retrieved sentence by slot filling.

figure 1 shows an example in which the sentiment of a sentence can be altered by changing a few sentiment-specific phrases but keeping other words fixed.


RE-TRIEVEONLY scores well on grammaticality and having the target attribute, since it retrieves sentences with the desired attribute directly from the corpus.

retrieve: 3 of the 4 systems look through the corpus and retrieve a sentence x tgt that has the target attribute v tgt and whose content is similar to that of x.


They employ the embedding vector of the target word as features.

while research has focused on polarity for quite a long time, meanwhile this early focus has been shifted to more expressive emotion representation models (such as Basic Emotions or Valence-Arousal-Dominance).


Self-training has been successfully used in many NLP applications such as information extraction (Ding and Riloff, 2015) and syntactic parsing (McClosky et al., 2006).

for example, "I broke my arm" and "I got fired" are usually negative experiences, while "I broke a record" and "I went to a concert" are typically positive experiences.


A co-training model that simultaneously trains event expression and event context classifiers produced substantial performance gains over the individual models.

for the RNN, we used the example LSTM implementation from Keras (Chollet et al., 2015) github, which was developed to build a sentiment classifier.


Our gold standard data set is relatively small, so supervised learning that relies entirely on manually labeled data may not have sufficient coverage to perform well across the human need categories.

as with the other classifiers, the word embedding feature representations performed best, achieving an F1 score 54.4%, which is comparable to the F1 score of the LR Embed system.


Whereas education had almost negligible influence on the performance, the more extensive formal training in reasoning the participants had, the higher their score was.

a possible approach would be to categorize warrants using, e.g., argumentation schemes (Walton et al., 2008) and break down errors accordingly.


As arguments are highly contextualized, warrants are usually presupposed and left implicit.

if we treat a warrant as a simple premise, then the linked structure fits the intuition behind Toulmin's model, such that premise and warrant combined give support to the claim.


Real-world deceptive situations are highstakes, where there is much to be gained or lost if deception succeeds or fails; it is hypothesized that these conditions are more likely to elicit strong cues to deception.

linguistic features such as n-grams and language complexity have been analyzed as cues to deception Yancheva and Rudzicz, 2013).


The intuition behind this was that those turns were responses to follow up questions related to q1, and while the question detection and identification system discussed above did not identify follow up questions, we found that most of the follow up questions after an interviewer question q1 would be related to q1 in our hand annotation.

working with such data requires extensive research to annotate each utterance for veracity, so such datasets are often quite small and not always reliable.


Action types and arguments are listed in the first two columns of Table 5.

for all domains, we compare the rational listener and speaker against the base listener and speaker, as well as against past state-of-the-art results for each task and domain.


29.3% of S 0 's directions are correctly interpretable for Alchemy, vs. 83.3% of human directions).

in the SCONE domains, which have a larger space of possible outputs than SAiL, we extend the decoder by: (i) decomposing each action into an action type and arguments for it, (ii) using separate attention mechanisms for types and arguments and (iii) using state-dependent action embeddings.


CHES scores are used only for evaluation and not during training.

our target pos (calibrated RILE) is a continuous variable 0, 1 , where 1 indicates that a manifesto occupies an extreme right position, 0 denotes an extreme left position, and 0.5 indicates center.


In general, DAM and ESIM seem to be more robust than CE, with the latter's accuracy degrading to essentially random performance on the most challenging subsets.

given a sample E, we apply the transformation T in order to generate a transformed sample E T where the word pairs are swapped, similar to the procedure applied in Section 5.2 on SNLI control instances in order to generate their transformed instances counterpart.


Our results show that the models are robust mainly where the semantics of the new instances do not change significantly with respect to the sampled instances and thus the class labels remain unaltered; i.e., the models are insensitive to our transformation to input data.

in this work we aim to study how certain factors affect the robustness of three pre-trained NLi models (a conditional encoder, the DAM model (Parikh et al., 2016), and the ESiM model (Chen et al., 2017)).


On the other hand, much better performance is obtained for the DAM and ESIM models on I TH instances containing unseen word pairs, indicating these models have learned to infer hyper-nym/hyponym relations from information in the pre-trained word embeddings.

when the sample size is too small we apply Yate's correction or a Fisher test.


This alteration, at the instance level, yields new sets of instances which range from easy (the semantics and the label of the new instance are the same to those of the original instance) to challenging (both semantics and label of the new instance change with respect to those of the original instance), but all of them remain easy to annotate for a human.

the CE model performs quite poorly at both samples (0.508 and 0.48 accuracy points on E A and E TA ); this drop in performance, with respect to the in situ condition, suggests that the repeated sentence context is too different from the structure of the training instances for the CE model to generalize effectively.


It is currently estimated that over 1.5 billion people are learning English as a Second Language (ESL) worldwide.

it has been shown that key aspects of the reader's characteristics and cognitive state, such as mind wandering during reading (Reichle et al., 2010), dyslexia (Rello and Ballesteros, 2015) and native language (Berzak et al., 2017) can be inferred from their gaze record.


In 1, assuming the function word that is always available when the speaker plans to produce a relative clause, the speaker will produce that when the upcoming relative clause or the first part of its contents are not yet available.

we did not find consistent evidence for the effect of noun frequency on classifier choice.


The two frameworks, however, differ in their views on the theory of learning as we describe below: Curriculum learning is inspired by the learning principle that humans can learn more effectively when training starts with easier concepts and gradually proceeds with more difficult ones (Bengio et al., 2009).

to break the tie for these instances in the final ranking, we resort to their final loss values as follows: the Leitner System is inspired by the broad evidence in psychology that shows human ability to retain information improves with repeated exposure and exponentially decays with delay since last exposure (Cepeda et al., 2006).


The importance of error-free language resources cannot be overstated as errors can inversely affect interpretations of the data, models developed from the data, and decisions made based on the data.

leitner system is inspired by spaced repetition (Dempster, 1989;Cepeda et al., 2006), the learning principle that effective and efficient learning can be achieved by working more on difficult concepts and less on easier ones.


In the base-level learning equation, it can refer to both because of the decay parameter, d, which causes more recent presentations to be more important, with older presentations (signified by their time of presentation, t) becoming exponentially less relevant.

importantly, note that in both models, when we refer to the expected retrieval time or activation, we are referring to the target word, not any of the preceding words.


This framework was selected rather than newer or more task-specific frameworks as it is the same underlying memory model of ACT-R, which has been used to explain a wide variety of language phenomena (e.g., Vasishth and Lewis, 2004;Reitter et al., 2011), but also has been used to explain everything from decision-making (e.g., Marewski and Mehlhorn, 2011) to visual attention in graphical user interfaces (e.g., Byrne et al., 1999).

further, this type of buffering strategy could be part of the strategy that ferreira and Swets (2002) refer to, when they propose the incrementality of language production is under "strategic control."


We then experimented this model on our original length dataset.

is then queried to Elasticsearch that has indexed the selected scenes, and the scene with the highest relevance is retrieved.


2 Each sentence in the plot summaries (a) A dialog from Friends: Season 8, Episode 12, Scene 2.

it is the largest, if not the only, corpus for the evaluation of passage completion on multiparty dialog that still gives enough instances to develop meaningful models using deep learning.


Experimental results verified that the increased contexts are able to help producing more acceptable and diverse responses.

this idea is included in the "Reasoning box" in Figure 2.


The domain adaptation from document comprehension to dialog generation is feasible by taking the rich context of the speakers as a "document", current user's query as a "question" and the chatbot's response as an "answer".

in this attention-based model, the conditional probability in Equation 4 is defined as: where is a RNN hidden state in the decoder part for time j and c j = i h i is a context vector, a weighted combination of the annotation set memory (h 1 , ..., h Ts ) produced by encoding a source sentence s with length T s .


The validation costs begun converging in the third epoch for the three models.

the sentences for which the probabilities are being predicted as M r (response memory, specially corresponds to the chatbot's side) and the question set as M q (query memory, specially corresponds to the user's side).


The different metrics in this figure will be introduced in the following sections.

the data in Reddit movie category is originally high-quality.


Recent advances in extreme multi-label classification have proven to work well for large label spaces.

the support set S is chosen based on nearest neighbors and its selection process is discussed in Section 3.4.


EMR data is generally not available for public use especially if it involves textual notes.

fastXML (Prabhu and Varma, 2014) partitions the feature space using the nDCG measure as the splitting criterion.


Baker and Korhonen (2017) improve neural network training by incorporating hierarchical label information to create better weight initializations.

in this section we compare our work with prior state-of-the-art medical coding methods.


We then train a ridge regression model on these instances.

this measures how much variance in the dependent variable y is captured by the independent variables x.


The value is set as 1 at the start and gradually decreases to 0.1 after 3000 annealing steps.

we adopt Q-learning (Sutton and Barto, 1998) to optimize the neural agent, which uses a function Q(z, a) to represent Q-values and the recursive Bellman equation to perform Q-value iteration, when observing a new sample (z, a, z , r).


We leverage a small amount of manually labeled data, downloaded from financial analysis websites, for guiding evidence mining.

we have investigated a reinforcement learning method to automatically mine evidences from large-scale text data for measuring the correlation between a concept and a list of stocks.


Besides, MAP is roughly the average area under the precision-recall curve for a set of queries (Christopher et al., 2008).

formally, given a concept c, a set of m stocks {o i } m i=1 and n data sources and each D i j is a sequence of words w 1 , w 2 ...w |D i j | , we assume the relevant stocks of the concept are revealed in the data sources (e.g.


The training set contains approximately 42K sentences with 887K words.

binarization can be considered as a special case of quantization, which quantizes the parameters to pairs of opposite numbers.


Batch normalization is hard to apply to a recurrent neural network, due to the dependency over entire sequences.

it is important to find good binary embeddings.


This shows CMN's capacity to model speaker-based emotions.

initially both A and B are emotionally driven by their own emotional inertia.


To find the relevance of each memory m t λ 's context with q i , a match between both is computed.

the performance maintains a positive correlation with K. this trend supplements our intuition that the historical context acts as an essential resource to model emotional dynamics.


Needless to say, existence of some false positive patterns at the label level is imminent.

this passive exploration is a label-based analysis which is performed as a sanity check.


Previous work proposed an end-to-end time-aware attention network to leverage both contextual and temporal information for spoken language understanding and achieved the significant improvement, showing that the temporal attention can guide the attention effectively .

namely, in dialogue contexts, the recent utterances contain most information related to the current utterance, which is aligned with our intuition.


In other words, our model automatically learns that the convex timedecay attention is more suitable for modeling contexts from the dialogue data than the other two types.

the further analysis is discussed in Section 4.3.


We found them to be highly correlated: Pearson's r = .81.

since the intention is to measure the student's reading ability, any difference in performance that is not due to reading ability confounds the measurement.


In short, previous research suggests that multiple factors may affect phone and pause duration in a reading of a story: from the phonetic properties of individual segments to where the passages falls within the narrative structure.

we also note that the direction of the correlation was opposite to our expectation: more complex passages were in fact read faster.


To answer our first question, we looked at the distribution of the reading rate across the passages in the training set.

it has been long known that different segments have different intrinsic durations which account for a lot of variation in segmental durations (Peterson and Lehiste, 1960;Klatt, 1976;van Santen, 1992): for example, high vowels tend to be shorter than low vowels.


The speaker and listener are placed in a game environment in which they both see the three colors of the context and a chatbox.

second, the Chinese monolingual model does not include a common word for orange, but the bilingual model identifies 橙色 chéngsè 'orange'.


We identified the 10 most frequent color-related words in each language to translate.

as we mentioned earlier, our main goal with this work is to investigate the effects of bilingual training on pragmatic language use.


Thus, perplexity ceases to be a measure of language modeling ability and assumes the role of a proxy for the out-of-vocabulary rate.

perplexity is a common intrinsic evaluation metric for generation models.


However, this is an unfair comparison: the monolingual model's high perplexity is dominated by low probabilities assigned to rare tokens in the opposite-language data that it did not see.

both players can send messages through the chatbox at any time.


these kinds of reasoning problems have been well-studied in visual question answering settings (Johnson et al., 2017;Suhr et al., 2017).


(w, h) a group of people standing around a bus stop .

table 4 shows the CIDEr scores using the same setup as Section 3, but using representations with spatial information about individual object instances.


We note that the results with our simpler image representation are comparable to the ones reported in Yin and Ordonez (2017), which use more complex models to encode similar image information.

such approaches are hard to interpret and are dataset dependent (Vinyals et al., 2017).


CCA-based methods are popular within the IR community for learning multimodal embeddings (Costa Pereira et al., 2014;Gong et al., 2014).

we introduce an adaptive algorithm for characterizing the visual concreteness of all the concepts indexed textually (e.g., "dog") in a given multimodal dataset.


; if a segment gets assigned to any such terms, it is considered a misprediction.

if a character name was not mentioned in the dialogue, the annotators labeled it as "unknown."


Other work removed the dependency on a true character list by determining all names through coreference resolution.

it also significantly outperforms the usage of the visual and acoustic features combined, which have been frequently used together in previous work, suggesting the importance of textual features in this setting.


The metaclassifier then learns to predict whether a specific generated answer is correct or not.

we rank the pixels according to their spatial attention and then compute the correlation between the two ranked lists.


More generally, we compute a task specific weighting of all biLM layers: (1) In (1), s task are softmax-normalized weights and the scalar parameter task allows the task model to scale the entire ELMo vector.

we find that including ELMo at the output of the biRNN in task-specific architectures improves overall results for some tasks.


The relatively high performance of FULL-0 shows that substituting segment copying with attention maintains and even improves the system effectiveness.


The full model selects between generating query tokens and copying complete segments from previous queries.

attending on fewer utterances improves inference speed: FULL-0 is 30% faster than FULL.


be an RNN, s an input string of length n and 0 ≤ t ≤ n a time step.

• Minimization: Can we minimize the number of neurons for a given RNN?


Methods can then be trained on the union of the original and auxiliary dataset.

several problematic instances have been demonstrated, for example, word embeddings can encode sexist stereotypes (Bolukbasi et al., 2016;Caliskan et al., 2017).


Overall, the process of corpus creation went through several stages -claim extraction, evidence extraction and stance annotation -, which we describe below.

while there exist some datasets for Arabic (Darwish et al., 2017b), the most popular ones are for English, e.g., from SemEval-2016 Task 6 (Mohammad et al., 2016) and from the Fake News Challenge (FNC).


The dataset is publicly available at https://goo.gl/Bym2Vz.


a natural question is how to handle new AM datasets in a new domain and with sparse data.


The size of the hidden layer in the encoder and the decoder layers of the models based on the proposed hierarchical decoder (rows (b)-(h)) are 200 and 100, respectively.

some prior work applied additional technique such as reranker to select a better result among multiple generated sequences (Wen et al., 2015;Dušek and Jurčíček, 2016).


In this paper we presented the first neural poetry translation system and provided two novel methods to improve the quality of the translations.

human reference: For where a grave had opened wide, There was no grave at all: Only a stretch of mud and sand By the hideous prison-wall, And a little heap of burning lime, That the man should have his pall.


rating scales without numerical labels, similar to the visual analogue scales used by Belz and Kow (2011) or direct assessment scales of (Graham et al., 2013;Bojar et al., 2017), however, without given end-points.

nevertheless, human judges often fail to distinguish between these different aspects, which results in highly correlated scores, e.g.


In addition, the results in Table 3 show that RR, in combination with Setup 2, results in more consistent ratings than DA.

its ranking of informativeness is less clear than that of plain ME, which might be due to the different results for missed and added information (see Sec.


We set ς to 22 on Newsela, 33 on WikiSmall, and 77 on WikiLarge.

we randomly selected 40 sentences from each test set, and included human reference simplifications and corresponding simplifications from the systems above 2 .


In the Story Cloze Test, a system is presented with a 4-sentence prompt to a story, and must determine which one of two potential endings is the 'right' ending to the story.

in the absence of further context, a default prior is assumed -as implicitly encoded in skip-thought vectors trained on BookCorpus -that is often correct.


This only achieves a test-set accuracy of 55.2%.

no Context (nC) This model attempts to identify the 'right' ending of a story by ignoring the story context and looking only at examples of right and wrong endings.


The task is accompanied by the Rochester story (ROCstory) corpus.

we train a feed-forward network using skip-thought embeddings.


We use the standard precision, recall and F 1 metrics to evaluate and compare the perfor-mance.

we present the first steps towards a pragmatic approach based on linguistic annotation ( Figure 3).


The input of most NLP tasks, such as a sentence or a document could be represented as a 2D structure with word embedding (Mikolov et al., 2013).

it shows that the deep learning model can, to some extent learn the humorous meaning and structure embedded in the text automatically without human selection of features.


For the negative samples, we choose WMT16 2 English news crawl as our non-humorous data resource.

their algorithm of the study was based on the extraction of structural patterns and peculiar structure of jokes (taylor and Mazlack, 2004).


The latter models the user with manually selected features.

for example, the Tweet "I'm not sexist but I can not stand women commentators" is actually an instance of hate speech, even though the first half is misleading.


In our approach, we collect the user's historical posts through the Twitter API.

the agent can interactively make selections and update the inter-user representations step by step.


This has also been the tradition in works on parser robustness (Bigert et al., 2005;Foster, 2004).

to address this we propose a semantic RLM, USIM, that operates by measuring the graph distance between the semantic representations of the source and the output.


This domain knowledge is a scalar function that is represented in the form of certain set of rules, easily provided by humans.

in our approach the score function could be any complex non-differentiable function.


In this paper, additional lookup tables and transitions are defined to create concepts when needed, following the current trend (Damonte et al., 2017;Ballesteros and Al-Onaizan, 2017;.

if we predict a multi-sentence graph, but there is no punctuation symbol that splits sentences, we post-process the graph and transform the root node into an AND node.


LEFT and RIGHT-ARC handle cycles and reentrancy with the exception of cycles of length 2 (which only involve i and b).

discussion to related work that relies on ad-hoc procedures, the proposed algorithm handles cycles and reentrant edges natively.


In total, we obtained nine affective values for 2.2 million words.

(1) We used the 15 verb classes from Ger-maNet (Hamp and Feldweg, 1997;Kunze, 2000).


for example, somebody can participate in two events taking place at different locations only if they do not overlap temporally.


Parameters  The intuition behind this model is that word sequence is an important factor affecting the generation of our languages; a word should be biased associated with other words on its left or right side.

to test the effectiveness of the embeddings produced by DSG, we conduct experiments on semantic (word similarity evaluation) and syntactic (part-of-speech tagging) tasks.


Example 11 shows the advantage of combining the vector representation with orthographic distance, i.e., our model could find translations of sleddogs that have similar meaning, while in examples 12 and 13 orthographic distance helped to pick the correct translation which is the closest in terms of edit distance.

throughout the paper, we work with two lexicons.


Accuracies are comparable to previous work (which was on different language pairs).

we design a new task to evaluate bilingual word embeddings on rare words in different domains.


Using the Google dataset GL (Mikolov et al., 2013b)   the CosAdd method (Mikolov et al., 2013c) shown in (5).

we make a point of comparison to concatenation since it is the most comparable in terms of simplicity, whilst also providing a good baseline of performance on evaluative tasks.


While antonymy represents words which are strongly associated but highly dissimilar to each other, synonymy refers to words that are highly similar in meaning.

this IAA measure has been a common choice in previous data collections in distributional semantics (Padó et al., 2007;Reisinger and Mooney, 2010;Hill et al., 2015).


Again, the models show a similar behaviour in comparison to SimLex-999, where also the dLCE model outperforms the two other models, and the SGNS model is by far the worst.

antonyms and synonyms often occur in similar context, as they are interchangeable in their substitution.


Dataset Splits Dima (2016) showed that a classifier based only on v w 1 and v w 2 performs on par with compound representations, and that the success comes from lexical memorization (Levy et al., 2015): memorizing the majority label of single words in particular slots of the compound (e.g.

the model was only tested on a small dataset and performed similarly to previous methods.


• Sentence: It is clear that I will never have another prime first experience like the one I had at Chompies.

we experimented with both a Maximum Entropy and a Binomial Naive Bayes binary classifier.


In this work we translated the SVF responses from Brazilian Portuguese to English.

evaluation is carried out at two levels of granularity: a rough-grained classification for the detection of a clinical condition in general (control vs. CI group), and a fine-grained classification for one of the three conditions (aMCD, mMCD and AD groups).


One is a control group with normal cognitive performance, and three are groups with clinical conditions according to assessment guidelines (de Paula et al., 2013;McKhann et al., 1984;Winblad et al., 2004): Amnestic Mild Cognitive Deficit (aMCD), Multi-domain Mild Cognitive Deficit (mMCD) and Alzheimer's Disease (AD).

given a sequence of semantically related words, a large number of switches from one semantic class to another has been linked to clinical conditions.


Our model significantly outperforms the only previous work on sluice resolution on available newswire corpora, but also has a number of advantages over this work.

we have presented a neural architecture for English sluice resolution and shown that it outperforms previous work on sluice resolution.


There are some important problems in the evaluation of word embeddings using standard word analogy tests.

once this assumption is built into systems, we still should put into question various details of the tests.


We find that MERGE is mostly responsible for combining spans of words to form a named entity in English parsing.


To this end, we design a neural network architecture, capable of tracking and updating the states of entities at the right time with external memory, making it a natural fit for the task.

when n = 2 (no unconstrained chains), model performance drops substantially to a F 1 of 76.6 ± 0.4.


Results A single NMT model achieves lower performance than the SMT baselines (Table 3).

using consistent training data and preprocessing ( § 2), we first create strong SMT ( § 3) and NMT ( § 4) baseline systems.


We consider various measures of both acoustic-prosodic and lexical entrainment, measuring the latter with a novel application of two previously introduced methods in addition to a standard high-frequency word measure.

it seems unlikely that alternative measures would yield fundamentally different outcomes, such as strong correlations across features.


In this paper, we present ConvKB-an embedding model which proposes a novel use of CNN for the KB completion task.

in these embedding models, valid triples obtain lower implausibility scores than invalid triples.


Online encyclopedias now make vast amounts of information and human knowledge available to internet users around the world in various languages.

then, we calculate the average distance for SCS, US, and MCS.


Entity recognition is a widely benchmarked task in natural language processing due to its massive applications.

there is no gain at all for NAM detection for both languages.


To the best of our knowledge, our work is one of the first applications of LN to any NLP task.

to simulate learning from large texts, we tuned hyper parameters on development, but ran the actual experiments on the train partitions.


Because the gold data does not contain examples involving sports, the baseline system mistakenly identifies a paraphrase of the above sentence as an attack event, and our system is not able to fix that mistake.

this process was repeated on a daily basis between January 2013 and February 2015, resulting in approximately 70 million sentences from 8 million articles.


For example, our system identifies the token shot in Bubba Watson shot a 67 on Friday as an attack event trigger.

we first group together paraphrases of event mentions.


• We develop the tree-based structure regularization methods and make progress on the task of relation classification.


We attribute the improvements to the simplified structures that generated by structure regularization.

understanding Chinese literature text is of great importance to Chinese literature research.


We established that syntactic patterns can markedly improve the extraction of patient, intervention and outcome descriptions in medical abstracts.

the performance gains could also be due to additional contextual information that bigrams and larger n-grams provide over unigrams alone, rather than their syntactic properties.


As an example of the importance of syntax in encyclopedic definitions, among the definitions contained in the WCL definition corpus (see Section 3.1), 71% of them include the lexico-syntactic To explore the potential of syntactic information, we represent dependency-based phrases by embedding them in the same vector space as the pretrained word embeddings introduced above.

in the early days of DE, rulebased approaches leveraged linguistic cues observed in definitional data (Rebeyrolle and Tanguy, 2000;Klavans and Muresan, 2001;Malaisé et al., 2004;Saggion and Gaizauskas, 2004;Storrer and Wellinghoff, 2006).


This is the idea of our dynamic oracle, which therefore is a correct dynamic oracle only with respect to a preset criterion for plane assignment, and not for all the possible plane assignments that would produce the gold dependency structure.

we present an efficient dynamic oracle to train the 2-Planar transition-based parser, which is correct with respect to a given plane assignment, and results in notable gains in accuracy.


2 Training transition-based parsers In transition-based parsing (Nivre, 2003), dependency trees are built incrementally, in a shiftreduce manner: to parse a sentence, a sequence of transitions (the derivation) is applied on the internal state of the parser (the configuration), consisting typically in a stack and a buffer of unprocessed words.

formally, if each configuration is associated with a UAS max , the maximum UAS value that can be achieved by any of its successor derivations, then the action cost is defined as the differ-ence between the current UAS max and the future UAS max (once the corresponding action has been applied).


1 Faster exact inference algorithms have been defined for some sets of mildly non-projective trees (e.g.

we present three novel variants of the degree-2 Attardi parser, summarized in Fig.


We selected those VMWEs whose frequency in TrC was higher than 9, i.e.


While their corpus frequency is relatively high at levels 0, 1 and 5, it is low at levels 2, 3 and 4.

2 shows the distribution of the resulting set S of 18 VMWEs into Tutin's levels.


The pragmatic listener L 0 is then defined in terms of this literal agent and a prior P (w) over possible images: The pragmatic speaker S 1 is then defined in terms of this pragmatic listener, with the addition of a rationality parameter ↵ > 0 governing how much it takes into account the L 0 distribution when choosing utterances.

in our incremental RSA, speaker models take both a target image and a partial caption pc.


Furthermore, to prevent word replacement that is incompatible with the annotated labels of the original sentences, we retrofit the LM with a label-conditional architecture.

", which is annotated with a positive label.


Contextual augmentation with a bidirectional RNN language model, when a sentence "the actors are fantastic" is augmented by replacing only actors with words predicted based on the context.

a word in the translated sentence is also replaced using a word alignment method and a rightward LM.


We also subsample German and French data to be equivalent to the size of Swahili, in order to compare training size effects.

the U.S. Government is authorized to reproduce and distribute reprints for governmental purposes notwithstanding any copyright annotation therein.


For the in-language case, we observe the cosine model outperforms the deep model.

compared to the modular approach, direct modeling is advantageous in that it focuses on learning translations that are beneficial for retrieval, rather than translations that preserve sentence meaning/structure in bitext.


We consider PropBank-style (Palmer et al., 2005) semantic role structures, or more specifi- cally their dependency versions (Surdeanu et al., 2008).

there are three possible directions (dir(u, v) ∈ {in, out, loop}): self-loop edges were added in order to ensure that the initial representation of node h v directly affects its new representation h v .


The forward RNN(x 1:t ) represents the left context of word x t , whereas the backward RNN(x n:t ) computes a representation of the right context.

in this work, we aim to fill this gap by showing how information about predicate-argument structure of source sentences can be integrated into standard attentionbased NMT models (Bahdanau et al., 2015).


Note that, when a translation is generated in the READ operation, the already committed target words remain unchanged, i.e.

generating translations for such incomplete sequences presents a considerable challenge for machine translation, more so in the case of syntactically divergent language pairs (such as german-English), where the context required to correctly translate a sentence, appears much later in the sequence, and prematurely committing to a translation leads to significant loss in quality.


Problem: In a stream decoding scenario, the entire source sequence is not readily available.

we were not able to improve upon our incremental decoder, with the results deteriorating notably.


It is also interesting to note that the performance of the baseline systems on out-of-domain data in both data conditions is the same, indicating that the in-domain data does not really help for the out-of-domain set.

a general domain system has been trained and tuned to offer general domain translation, but it needs to be adapted to specific domains in order to provide better quality.


We do not have access to aligned source words for gold constraints.

since there is no correspondence between constraints and the source words they cover, correct constraint placement is not guaranteed and the corresponding source words may be translated more than once.


Here we compare the results of InferSent (Conneau et al., 2017), a more involved representation that was found to provide a good sentence representation based on NLI data.

lorem ipsum dolor sit amet, consectetuer adipiscing elit.


We evaluate the model's performance using BLEU metric (Papineni et al., 2002).

the languages in each pair are similar in vocabulary, grammar and sentence structure (Matthews, 1997), which controls for language characteristics and also improves the possibility of transfer learning in multi-lingual models (in §7).


The numbers in the figure, separated by a slash, indicate how many times each n-gram is generated by each of the two systems.

interestingly, BE → EN does not seem to benefit from pre-training in the multilingual scenario, which hypothesize is due to the fact that: 1) Belarusian and Russian are only partially mutually intelligible (Corbett and Comrie, 2003), i.e., they are not as similar; 2) the Slavic languages have comparatively rich morphology, making sparsity in the trained embeddings a larger problem.


Finally, it is of interest to consider pre-training in multilingual translation systems that share an encoder or decoder between multiple languages (Johnson et al., 2016;Firat et al., 2016), which is another promising way to use additional data (this time from another language) as a way to improve NMT.

( §6) Q5 Do pre-trained embeddings help more in multilingual systems as compared to bilingual systems?


We additionally performed pairwise comparisons between the top 10 n-grams that each system (selected from the task GL → EN) is better at generating, to further understand what kind of words pre-training is particularly helpful for.

we can postulate that having consistent embedding spaces across the two languages may be beneficial, as it would allow the NMT system to more easily learn correspondences between the source and target.


Training the parameters of this latent-variable model will recover the posterior distribution over analyses of a form, p θ (t, , s | f ), which allows us to disambiguate counts at the type level.

the syncretic forms are bolded and colored by ambiguity class.


We used perplexity on development data to jointly choose 2 Our vocabulary and parameter set are determined from the lexicon.

(2) has 1 hidden layer, but we actually generalized this to consider networks with k ∈ {1, 2, 3, 4} hidden layers of d = 100 units each.


Together, these results show that there is significant potential for follow up work on developing innovative uses of QAMR and modeling their relatively comprehensive and complex predicate-argument relationships.

finally, we report simple neural baselines for QAMR question generation and answering.


We also add negative samples where the output is a special token and the input has w q , w a that never appear together.

we differ from QA-SRL in focusing on all words in the sentence rather than just verbs, and allowing free form questions instead of using templates.


To define a "community," volunteers manually selected a set of users that fit with a theme that they had familiarity with.

using agglomerative clustering, we found groups of words that centered around frequent words used in particular regions (foreign words, dialects) or cultures (sociolects), associated with hobbies or interests (specific sports, music genres, gaming), or polarizing topics (political parties, controversial issues).


Another problem that could be formulated is one in which the output summary is generated from multiple tables as proposed in a recent challenge (Wiseman et al., 2017) (this setting is out of the scope of this paper).

in addition to the standard BLEU (sBleu) (Papineni et al., 2002), a customized BLEU (cBleu) (Mei et al., 2016) has also been reported.


As future work, we propose to tackle general tabular summarization where the schema can vary across tables in the whole dataset.

mixed hierarchical attention model is faster than fully dynamic hierarchical attention.


Another alternative is update summarization, in which the summary should cover content in one set of documents, but not in another set that the user has already read (Dang and Owczarzak, 2008).

we call these facts targeted information units (TIU), because they are the pieces of information that must be part of the summary for the given information type.


One possibility is that simply encouraging workers to spend more time writing their summaries improved performance, but fitting a linear model we find the correlation coefficient between time and F 1 is −0.06, indicating no linear correlation between time and accuracy across conditions.

table 1: Performance for a range of metrics (defined in § 4.4) as the number of targeted information units and the condition vary (b: Baseline, h: Highlight, p: Pin-Refine).


We propose "targeted summarization" as an umbrella category for summarization tasks that intentionally consider only parts of the input data.

summaries for query-based summarization only cover parts of the text that are about the topic specified by a query (Rahman and Borah, 2015).


in this work, we investigate the generation of open domain Wikipedia summaries in underserved languages using structured data from Wikidata.


This research is partially supported by the Answering Questions using Web Data (WDAqua) project, a Marie Skłodowska-Curie Innovative Training Network under grant agreement No 642795, part of the Horizon 2020 programme.

with these works, we explore the generation of sentences in an open-domain, multilingual context.


every summary has salient tokens which should be rewarded with more weight.


For simplicity, we report the average over the 3 datasets.

the number of data points is relatively low and the learned θ might not be well-behaved (high θ scores for bad summaries) pushing the optimizer to explore regions of the feature space unseen during training where θ wrongly assumes high scores.


Since UD focuses on the relations between content words, UD triples are able to represent key components of sentences more directly.

for example, numbers tend to be put in the same clusters since our word embeddings place them close to each other.


To do this, we use a particular graph structure, called multipartite graph, to represent documents as tightly connected sets of topic related candidates (see Figure 1b).

unfortunately, these models suffer from several limitations: they aggregate multiple topic-biased rankings which makes their time complexity prohibitive for long documents 1 , they require a large dataset to estimate word-topic distributions that is not always available or easy to obtain, and they assume that topics are independent of one another, making it hard to ensure topic diversity.


TopicRank obtains the highest precision among the baselines, suggesting that its -one keyphrase per topic-policy succeeds in filtering out topic-redundant candidates.

among the features proposed to address this problem in the literature, the position of the candidate within the document is most reliable.


(2014) who attempt to understand the various dimensions that experts and non-experts consider while judging narrative similarity.

since some movies have multiple remakes, we obtain clusters of movie plots, each of which share the same narrative theme.


Emojis can encode different meanings, and they can be interpreted differently.

fastText represents a valid approach when dealing with social media content classification, where huge amounts of data needs to be processed and new and relevant information is continuously generated.


The network we used is composed of 101 layers (ResNet-101), initialized with pretrained parameters learned on ImageNet (Deng et al., 2009).

by modeling the semantics of emojis, we can improve highly-subjective tasks like sentiment analysis, emotion recognition and irony detection (Felbo et al., 2017).


Previous research such as (Fernández-González and Gómez-Rodríguez, 2012) and (Qi and Manning, 2017) proves that the widely-used projective arc-eager transition-based parser of Nivre (2003) benefits from shortening the length of transition sequences by creating non-local attachments.

it consists in a modification of the non-projective Covington algorithm where: (1) the Left-Arc and Right-Arc transitions are parameterized with k, allowing the immediate creation of any attachment between j and the kth leftmost word in λ 1 and moving k words to λ 2 at once, and (2) the No-Arc transition is removed since it is no longer necessary.


The resulting parser outperforms the original version and achieves the best accuracy on the Stanford Dependencies conversion of the Penn Treebank among greedy transition-based parsers.

there are still situations where sequences of No-Arc transitions are needed.


Language changes serve as a sign that a patient's cognitive functions have been impacted, potentially leading to early diagnosis.

each of these transcripts were then broken down by sentences and interruptions by the interviewer.


(2016) adopted a deep neural network language model.

instead, words that give structure to a sentence have the biggest impact on classification, such as definite articles and determiners (e.g., "the" and "that").


We focus on two neural encoder-decoder models for spelling normalization, comparing them against the memorization baseline and to previous results from Pettersson et al.

we conclude that future work should include more rigorous evaluation, including both intrinsic and extrinsic measures where possible.


More models adopt this thought to enhance the performance (Liu et al., 2015(Liu et al., , 2016.

gated mechanism in existing studies not only shows its convenience for training (Srivastava et al., 2015), but also behaves as tool to route the information (He et al., 2016).


For all the experiments, we employ Word2Vec (Mikolov et al., 2013) to initialized the word vectors, which is trained on Google News with 100 billion words.

under the scheme of proportional addition (Ruder et al., 2017;Misra et al., 2016), all the features are shared with the same weight between every pair of tasks.


The specific semantic parsing problem we study in this work is to map a natural language question to a SQL query, which can be executed against a given table to find the answer to the original question.

we evaluate our model on the wikiSQL dataset (Zhong et al., 2017).


So, in the one-dimensional case, the transformations that a word vector can do are restricted to translation.

the best variation learns unique embeddings for only the most frequent words and uses hard clustering for the rest.


This is likely due to the fact that SM was trained on the SALICON dataset; a higher correlation can probably be achieved by fine-tuning the salience model on the PASCAL Actions Fixation data.

(2016) report for their question answering ask, indicating that verb classification is a task that can be performed more reliably than Das et al.


If our models cannot localize and color the appropriate object, workers will be unable to select an appropriate image.

boyd-Graber is supported by NSF Grant IIS-1652666.


Inspired by a cognitive account of humor appreciation, we employ linguistic wordplay, specifically puns, in image descriptions.


Surprisingly, we find that models are robust to shuffling the word order and limiting the word categories to nouns and adjectives.

there has been increasing interest in modeling natural language in the context of a visual grounding.


GT+P results show that inter-modality constraints help in improving the results (about 2% F1) which indicates some of the visual relations successfully confirmed and boosted their corresponding relations in the text modality.

in the following we describe the information that we use from each modality (i.e.


In order to validate the superiority of the deep audio features in video captioning, we illustrate the performance of different audio features applied in the CM-ATT model in Table 2.

more specifically, we utilize the ResNet model for image classification (He et al., 2016) and the VGGish model for audio classification .


As shown in Figure 2, different from a stacked two-layer LSTM, the high-level LSTM here operates at a lower temporal resolution and runs one step every s time steps.

in formula, o e H j , h e H j = e H (f e H j , h e H j−1 ) , where e H denotes the high-level LSTM whose output and hidden state at j are o e H j and h e H j .


The same applies to named entity recognition: State-of-the-art systems are combinations of neural networks such as LSTMs or CNNs and conditional random fields (CRFs) (Strauss et al., 2016).

furthermore, Table 4 also presents the results of the NER models making use of the typeaggregated features instead of token-level gaze features.


On one hand, the quality improvement attributed to eye movement signals on lower-level tasks implies that such signals do contain linguistic information.

the best improvements on F 1score over the baseline models are significant under one-sided t-tests (p<0.05).


The impact of the eye-tracking features varies between the different combinations of datasets.

in total, they comprise 142,441 tokens with gaze information.


We present the benefits of eyetracking features by evaluating the NER models on both individual datasets as well as in cross-domain settings.

when evaluating the NER models in a cross-corpus scenario, the type-aggregated features lead to significant improvements.


The singular unit can thus use these gates to reliably store number information across long-range dependencies.

to gain further insight regarding the functioning of the syntax units, we next visualized their gate and cell dynamics during sentence processing.


In this section, we elaborate on the behavior of our model by conducting finer-grained analysis at queue-level and investigating the following questions in the context of challenges of semi-supervised learning.

we analyze our model from various perspectives to explain its improvement gain with respect to challenges of semisupervised learning.


Effective semi-supervised learning algorithms based on pretraining techniques (Hinton and Salakhutdinov, 2006;Bengio et al., 2007;Erhan et al., 2010) have been developed for text classification, deep belief networks (Hinton and Salakhutdinov, 2006), and stacked autoencoders Bengio et al., 2007).

the core difference between self-training algorithms is in the second step: data sampling policy.


In terms of DANs, we use FastText (Joulin et al., 2017) for its high per-formance and simplicity.

instances in the designated queues show considerably smaller similarity with training instances in both positive and negative classes, and, therefore, do not match well with training data.


We replace all target brand names with the keyword BRAND and other non-target brands with BRAND-OTHER for the purpose of our experiments.

if there is a large deviation among movement patterns of instances of the same queue, better data sampling policies could be developed, perhaps through finer-grained queue-level sampling.


That is, we examine language model behavior on artificially constructed sentences designed to expose behavior that is crucially dependent on syntactic state representations.

garden-pathing in LSTMs has recently been demonstrated by van Schijndel and Linzen (2018a,b) in the context of modeling human reading times.


The N400 is best predicted when the model is trained on that component independently, but every other ERP component prediction can be improved by including a second ERP component in the training.

we logtransform the self-paced reading time and the eyetracking measures.


Since this corpus does not include a designated test set, we randomly sampled and removed 200 utterances from training to use as a development set, and use the designated development data as a test set.

transferring the encoder parameters is where most of the benefit comes from.


There is a great deal of evidence that many phonological contrasts are perceptually available from a very early stage (Eimas et al., 1971;Moffitt, 1971;Trehub, 1973;Jusczyk and Derrah, 1987;Eimas et al., 1987).

but to prevent the acquisition process from being circular, the learner cannot operate solely on top-down information -the acoustic signal must provide some evidence for the phonemic categories.


A few simple examples are given below: Based on the similarity/differences between the reparandum and the repair, disfluencies are often categorized into three types: repetition (the first example), rephrase (the next example), and restart (the last example).

but it 's just you know leak leak leak everywhere people should know that that 's an option and i think you do accomplish more after that i mean [ it was + it ] interesting thing [ about gas is when + i mean about battery powered cars is ] containing 1126 files hand-annotated with disfluencies.


Our baselines include models with text-only, prosody cues only (raw), and innovation features only as inputs.

for these types of disfluencies, it makes sense that prosodic cues would not really be needed.


The second case is relevant to incident response as modelled by LORELEI (Strassel and Tracey, 2016), where there may only be a single target-language consultant available for which transcribed speech can be elicited, but the goal is to have an ASR model that generalizes to multiple speakers.

choosing as pretraining languages geographically proximal languages tends to help more than phonetically and phonologically similar but otherwise distant languages.


Multilingual pretraining of languages from a different language family and script benefitted from an explicit phoneme objective and adversarial objective when there was sufficient diversity in the pretraining languages.

although the characteristics of the speech are unique, it allows us to investigate multilingual models over many languages without the confounds of an overly noisy environment.


Optimal word frequency threshold is determined on dev set of each fold.

these features are extracted via sliding a window over the sentence, as displayed in Fig.


In this study, we hope to discover the ideal precision and recall tradeoff point regarding cognitive load in CAI terminology assistance and use this feedback to adjust the model.

we would expect loan words, words adopted from a foreign language with little or no modification, to be easier to recognize and translate for an interpreter.


The input to our model are mel-frequency cepstral coefficient (MFCC) audio features (Davis and Mermelstein, 1980) and the output is a sequence of words {y m } M m=1 , each of which is a symbol from the dictionary.

our collection induces the annotators to mainly abide to audio, hence, increasing the dependency of written text on the audio input as can be shown in our survey analysis in Figure 5.


As an initial exploration of this proposition, we perform various data analysis against the background of humor theories, and we train and examine classifiers to detect humorous edited headlines in our data.

(2015) analyzed cartoon captions in order to understand what made some funnier than others.


We use the attention based Transformer (Vaswani et al., 2017) architecture as our baseline.

bERT denoise the 15% of the tokens at random by replacing 80% of them with [MASK], 10% of them with a random word and 10% of them unchanged.


Copying mechanism was proved effective on text summarization tasks (See et al., 2017;Gu et al., 2016) and semantic parsing tasks (Jia and Liang, 2016).

the usage of the non-public CLC corpus (Nicholls, 2003) and self-collected non-public error-corrected sentence pairs from Lang-8 made their training data 3.6 times larger than the others and their results hard to compare.


Assuming a parsing model is a parameterized representation of a grammar, then we can expect those models to evolve in a similar way.

dravidian and Afro-Asiatic languages are not as consistent.


Faroese (fo) is morphologically rich and that should help, however its North-Germanic relatives are morphologically much simpler.

we thus make the simplifying assumption that a language grammar evolves only from an older stage and can be approximated by that previous stage.


Assuming that typology is homogeneous in a language family, the phylogeny should drive models to be typologically aware.

at the beginning, we initialize a new blank/random model that will be the basic parsing model for all the world languages.


our new training procedure can be applied to any task, so a future work would be to use it to perform phylogenetic POS tagging.


We thank Caio Corro, Giorgio Satta, Marco Damonte, as well as NAACL anonymous reviewers for feedback and suggestions.

we refer the reader to Table 5 of Appendix B for the full list of hyperparameters.


we used the standard split (sections 2-21 for training, 22 for development and 23 for test).


A naïve way of enforcing right-branching guarantee is to do a complete transformation of the subtree on the stack into a right-branching one.

in the worst case the method will reach the bottom of the tree, but often only 3 or 4 nodes need to be transformed to make the tree perfectly the right branching The worst case complexity of repairing the imperfection is O(n) which makes the complexity of the whole parsing algorithm O(n 2 ) for building a single derivation.


As visible on the Figure 3a side condition, the lower combinator must not be >B0.

having a special mechanism for right adjunction makes parser both more incremental and more accurate.


Formally, has the form: where t is the iteration number, T is the total number of training iterations, f is a monotonically increasing function, and we introduce two new hyper-parameters associated with the cyclical annealing schedule: • M : number of cycles (default M = 4); • R: proportion used to increase within a cycle (default R = 0.5).

this issue causes two undesirable outcomes: (i) an encoder that produces posteriors almost identical to the Gaussian prior, for all observations (rather than a more interesting posterior); and (ii) a decoder that completely ignores the latent variable z, and a learned model that reduces to a simpler language model.


to ensure that t 0 defines relation for all possible string pairs (x, y) ∈ Σ * × ∆ * , we add all arcs of the form a = (s 1 , s 1 , ω, σ, δ), ∀(σ, δ) ∈ Σ × ∆ to t .


B More analysis on the effectiveness of NFSTs B.1 Does feeding alignments into the decoder help?

in this paper, we propose neural finite state transducers (NFSTs), in which the weight of each path is instead given by some sort of neural network, such as an RNN.


The number of paths may be exponential in the size of T , or infinite if T is cyclic.

we use NACS (Bastings et al., 2018) in our experiment.


This shows that detecting speech acts is a very challenging task especially for domainindependent environments.

such a dataset can be fed into deep learning algorithms to yield better performance in detecting Answering issues.


We had to ensure the simplicity of the task to obtain high quality results.

traditionally, QA has been explored over large textual corpora (Cui et al., 2005;Harabagiu et al., 2001Harabagiu et al., , 2003Ravichandran and Hovy, 2002;Saquete et al., 2009) with answers being textual phrases.