Finally, ALGES uses discriminative models to score each candidate equation, using both local and global features (Section 7).

the recursive nature of our representation allows us to decompose the likelihood function p(t, w) into local scoring functions for each in- 3 Inspired by Semantically Augmented Parse Trees (Ge and Mooney, 2005) adapted to equational logic.


The properties of the target Qset are also extracted according to the rules of the Table 1.

the qnt property is set to unknown, the ent is set to the noun appearing after the words what, many or much in the target sentence, and the other properties are extracted as listed in Table 1.


In order to determine the effect of various components of our system on its overall performance, we perform the following ablations: No Local Model: Here, we test our method absent the local information (Section 7.1).

we generate equations using all ILP constraints, and score trees solely on information provided by the global model: No Global Model: Here, we test our method without the global information (Section 7.2).


That is, we generate equations using all ILP constraints, and score trees solely on information provided by the global model: No Global Model: Here, we test our method without the global information (Section 7.2).

we generate equations using only the hard constraints of ILP and score trees solely on information provided by the local model: No Qset Reordering: We test our method without the deterministic Qset reordering rules outlined in Section 5.


The utility of evaluating against treebank derived lexicons is that it allows us to evaluate on a large set of languages.

in the universal dependency treebanks v1.1 (Agić et al., 7 We use version 1.1 released in May 2015.


The model we use is a standard atomic sequence classifier, that classifies the morphological bundle for each word independent of the others (with the exception of features derived from these words).

we use a linear SVM model classifier with hand tuned features.


We make all code, training data, evaluation sets and both of our linguistic tools publicly available on-line for future research.

we propose the reverse dictionary task as a comparatively generalpurpose and objective way of evaluating how well models compose lexical meaning into phrase or sentence representations (whether or not they involve training on definitions directly).


Let φ = {φ 1 ...φ T } denote a T-dimensional random vector, where each φ t might for example correspond to a sense probability at time t. We define a prior which encourages smooth change of parameters at neighboring times, in terms of a first order random walk on the line (graphically shown in Figure 2, and the chains of φ and ψ in Figure 1(left)).

we define this prior as an intrinsic Gaussian Markov Random Field (iGMRF; Rue and Held 2005), which allows us to model the change of adjacent parameters as drawn from a normal distribution, e.g.


In this experiment we evaluate whether model induced temporal word representations capture perceived word novelty.

we adopt the evaluation framework (and dataset) introduced in Gulordava and Baroni (2011) 7 and discussed below.


We also apply our model in a supervised setting, i.e., by extracting features for classifier prediction.

we trained a multiclass SVM (Chang and Lin, 2011) on the training data provided by the SemEval organizers (for DTE tasks 1 and 2).


This analysis confirmed that native language was associated with judgments about economic change.

it indicated that changes in size affected English speakers' judgments and Spanish speakers' judgments differently (p < 0.001), with an increase in size increasing the odds (e β = 2.5) of a judgment of IMPROVED by Spanish speakers and decreasing the odds (e β = 0.44) of a judgment of IMPROVED by English speakers.


And, if so, which linguistic factors contribute to this notion ( §3.3)?

since formality varies substantially across genres (Li et al., 2013), we look at text from four different genres: News, Blogs, Emails, and com-  2.8 in addition to any customs duties that may be assessed.


3 We follow the protocol used in Lahiri (2015) in order to gather judgments on Amazon Mechanical Turk for the Email and Answers data.

we use a 7-point Likert scale, with labels from -3 (Very Informal) to 3 (Very Formal).


We look to existing theories of formality and of linguistic style matching to guide our analysis.

: • Formality is higher when the amount of shared context between speakers is low (Heylighen and Dewaele, 1999).


Perhaps more interestingly, we see a significant positive effect of the interaction between previous score and position.

the effect of prior post formality on current post formality becomes stronger later in a thread compared to at the beginning of a thread.


For a TSL 2 grammar G = T, S , T ⊆ Σ will be referred to as the tier, S the allowed tier substrings, and R = fac T-2 (Σ * ) − S as the forbidden tier substrings.

let H = Σ − T be the non-tier elements of G. For any elements σ i , σ j ∈ T , their tier adjacency with respect to a set l of strings refers to whether or not they may appear adjacent on the tier; formally, σ i , σ j are tier adjacent in l iff An exclusive blocker is a σ ∈ T which does not appear in R. ∀w ∈ R, σ ∈ fac 1 (w).


The intuition here is that the more divergent the distributional behaviour of source and target phrases across subdomains, the less certain we are whether this phrase pair is valid for the unknown target domain.

a translation rule with source and target phrases having two similar distributions over the latent subdomains is likely safer to use.


In the next Section, we aim to estimate distributions P (z|ẽ) and P (z|f ) for subdomain z over the source and target phrases respectively.

we aim at projecting phrases onto a compact (K − 1) dimensional simplex of subdomains with vectors: Each of the K elements encodes how well each source and target phrase expresses a specific latent subdomain in the training data.


(2009) and Cuong and Sima'an (2014a) who "embed" such a phrase-level model into a latent subdomain model that works at the sentence level.

we associate latent domains with sentence pairs rather than with phrases, and use the posterior probabilities computed for the sentences with all the phrases appearing in the corresponding sentences.


Finally, we use hard EM, as it has been found to yield better models than the standard soft EM on a number of different task (e.g., (Johnson, 2007)).

instead of standard 'soft' EM updates with phrase counts weighted according to the posterior P (z = i| e, f ), we use the 'winner-takes-all' approach: Here,ẑ e, f is the "winning" latent subdomain for sentence pair e, f : In practice, we found that using this hard version leads to better performance.


Overall, we can see that all domain-invariance features contribute to adaptation performance.

we observe the following: • Favouring the source-target coherence across subdomains (i.e., adding the feature D(ẽ,f )) provides a significant translation improvement of +0.3 BLEU.


The aligner computes for each word pair across Q and S (i) a semantic similarity score sim W ∈ [0, 1] using PPDB-a large database of lexical paraphrases developed using bilingual pivoting (Ganitkevitch et al., 2013).

it allows three different levels of similarity: 1 if the two words or their lemmas are identical, a value ppdbSim ∈ (0, 1) if the word pair is present in PPDB (the XXXL database) 2 , and 0 otherwise.


It can be interpreted as a variable that conjoins variables y and z together.

we introduce a post-processing step where the subexpression coord(x, y, z) ∧ arg 1 (e e , x a ) is replaced with arg 1 (e e , y a ) ∧ arg 1 (e e , z a ), and the x variable is removed.


To overcome this problem, we propose a novel method to utilize the redundancy and relationship between KBs as indirect supervision.

if two concepts in different KBs are determined to be the same, we can assume that one is the "gold label" for the other, and extract textual and relational features between them, making this pair an approximation of the real grounding instance.


The final score of a concept candidate is further adjusted by the matching between neighboring concepts in the KB and the concept candidates around the mention.

if a neighbor concept in the KBs also appears in the context of the mention, the score of the concept candidate is increased according to the initial score of the matched neighbor concept.


These two approaches only change the way CCMIS constructs training examples for linear ranking SVM.

only the ranking model is changed while other components (concept candidates, features, and learning algorithm) of the system are identical.


(2011) train a semantic parser in a question-answering domain using the denotation (or answer) of each question as the sole supervision.

impressive is their system's ability to learn complex linguistic structure not handled by earlier methods that use more direct supervision.


These sets of error tags are not easily interchangeable between different corpora.

two major GEC corpora have different taxonomies: the Cambridge Learner Corpus (CLC) (Nicholls, 2003) has 80 tags, which generally represent the word class of the error and the type of error (such as replace preposition, unnecessary pronoun, or missing determiner).


This is due to the nature of grammatical error correction, where there can be diverse correct edits for a sentence (Figure 1).

there is no single gold-standard correction.


The advantage of GLEU is that it penalizes n-grams in the system output that were present in the input sentence and absent from the reference.

a system loses credit for missing n-grams that should have been changed.


From that point, the action probabilities given state T j will sum to one, just as the action probabilities given state T k will sum to one, and there is no reason to expect that the most probable action from T j should be less probable than the most probable action from T k .

once an error occurs, search is of little help in recovering from it, since scores are based only on local decisions and not on any notion of state quality, and the error occurred precisely because an incorrect action was found to be more probable locally.


Once actions are scored for the parser's current state using a classifier, the score of a new state resulting from the application of a valid action to the current state can be computed as the product of the probabilities of all actions applied up to the new state in its derivation path.

the score of each new state is the score of the current state multiplied by the probability of the action applied to the current state to generate the new state.


For tractability, probabilistic graphical models are typically constrained by making conditional independence assumptions, thus imposing structure and locality on X ∪ Y.

in our models, we postulate that the following conditional independence assumptions hold: the label y i for the i th token directly depends only on the label y prev (i) of some previous token at position prev(i) and potentially on all input tokens.


The probability distribution encoded by the graphical model can then be expressed as follows: Here, F A (x A , y A ) denotes the factors of the model, each of which is of the following form: ensures that this distribution sums up to 1, while λ k are the parameters of the model, which we aim to learn from various annotated background corpora.

our inference objective then is to find the most probable sequence of labels y * when given the token sequence x as evidence: in our setting, we fix x = tok 1 , .


All of these provide features for our graphical model.

we harness dependency types between noun phrases (de Marneffe et al., 2006), like nsubj, dobj, prep in, prep for, etc.


Feature f 10 (ent i , tok i ) measures the weighted overlap similarity between the token contexts (tok-cxt) of token tok i and entity ent j .

we use a weighted generalization of the standard overlap coefficient, WO, between two sets X, Y of weighted elements, X k ∈ X and Y k ∈ Y : We set the weights to be tf-idf scores, and hence we obtain: f 10 (ent i , tok i ) = WO tok-cxt(ent i ), tok-cxt(tok i ) Entity-Entity Token Coherence.


Template f 12 (ent i , tok i ) generates a binary feature function that captures the coherence between an entity candidate ent i of token tok i and the domain d which the input text is classified into.


In addition to the competitors introduced in Section 5.2, we add a pipelined configuration of J-NERD , coined P-NERD.

we first run J-NERD in NER mode (thus only considering NER features f 1..7 and f 14..16 ).


Despite our description of the model as patternclustering, it is important to stress that we are inducing clusters in a context-sensitive way.

we are learning an encoder: a feature-rich classifier, which predicts a relation for a specific sentence and an entity pair in this sentence.


With the decoder learned and fixed, we trained the encoder part on unlabeled examples in T r 2 , while leveraging the previously trained decoder.

we optimize the objective (10) on T r 2 but update only the encoder parameters w. 4 In this setting the decoder provides a learning signal for the encoder.


Universal schema methods (Riedel et al., 2013) use factorization models to infer facts (e.g., predict missing entities), but they do not attempt to induce relations.

they consider each given context as a relation and induce an embedding for each of them.


• π(h) is the probability of generating h ∈ [m] in the first position of a sequence.

• t(h |h) is the probability of generating h ∈ • o(x|h) is the probability of generating an A-HMM is an HMM in which each hidden state h is associated with at least one "anchor" observation state that can be generated by, and only by, h. Note that the anchor condition implies n ≥ m. An equivalent definition of an A-HMM is given by organizing the parameters in matrix form.


Clearly, (11) is a special case with L = 1.

with the definition of Ω in the previous section, the dimension of Ω x is d = O(n) which can be difficult to work with when n m. Proposition 4.1 allows us to reduce the dimension as long as the final matrix retains the form in (3) and has rank m. we can multiply Ω by any rank-m projection matrix Π ∈ R d×m on the right side: if Ω satisfies the properties in Proposition 4.1, then so does ΩΠ with m-dimensional rows Since rank(Ω) = m, a natural choice of Π is the projection onto the best-fit m-dimensional subspace of the row space of Ω.


Since anchor discovery is inherently tied to parameter estimation, it is better to obtain anchors in a data-driven manner.

certain POS tags (e.g., X) appear quite infrequently, and the model is worse off by being forced to allocate a hidden state for such a tag.


But we do this on NONOVER (not on ORIG) to focus on what is critical for TE.

feature SYN is the number of word pairs in s 0 and s 1 that are synonyms.


Continuous space models of words, objects, and signals have become ubiquitous tools for learning rich representations of data, from natural language processing to computer vision.

there has been particular interest in word embeddings, largely due to their intriguing semantic properties (Mikolov et al., 2013b) and their success as features for downstream natural language processing tasks, such as named entity recognition (Turian et al., 2010) and parsing (Socher et al., 2013).


Recently, Levy and Goldberg (2014a) showed that linear linguistic regularities first observed with word2vec extend to other embedding methods.

explicit representations of words in terms of cooccurrence counts can be used to solve analogies in the same way.


The clearer foundation afforded by this perspective enables us to analyze word embedding algorithms in a principled task-independent fashion.

we ask whether word embedding algorithms are able to recover the metric under specific scenarios.


If we rewrite the bias terms as a i = a i − || x i || 2 2 and b j = b j − || c j || 2 2 , we obtain the equivalent representation: Together with Lemma 1, we recognize this as a weighted multidimensional scaling (MDS) objective with weights f (C ij ).

splitting the word vector x i and context vector c i is helpful in practice but not necessary under the assumptions of Lemma 1 since the true embedding x i = c i = x i /σ and a i , b i = 0 is a global minimum whenever dim( x) = d. GloVe can recover the true metric provided that we set d correctly.


Stemming is a popular way to reduce the size of a vocabulary in natural language tasks by conflating words with related meanings.

stemming aims to convert words with the same "stem" or root (e.g "creative" and "creator") to a single word type ("create").


Additionally, a variety of work exists in the general field of stemmer evaluation, though much of it centers on the information retrieval community.

the work of Harman (1991) highlights some of the fundamental issues of strong stemming, including the potential positive effect of light stemmers like the S-removal stemmer.


Consequently, they only consider closely related Indo-European languages for which high-quality tokenization can be obtained with simple heuristics.

we argue that recent approaches to cross-lingual POS tagging and dependency parsing are biased toward Indo-European languages, in particular the Germanic and Romance families.


However, the variance in gold edge length is about 1.2 times the deviation of predicted edge length.

gold edges are often longer and more far-reaching.


In order to expose the parser to more errors, we follow an aggressive-exploration scheme: we sometimes follow incorrect transitions also if they score below correct transitions.

when the score of the correct transition is greater than that of the wrong transition but the difference is smaller than a margin constant, we chose to follow the incorrect action with probability p agg (we use p agg = 0.1 in our experiments).


and the gradient of the log-probability works out to: For the complete derivation, see Appendix C. The parameters θ of the adjustment function are learned by maximizing the Poisson log-probability, using stochastic gradient ascent.

for each feature-target pair (f, t) we compute the gradient in Eq.


we configured the following feature extractors: We then built a model that uses both these features and regular 5-grams (SNM5-skip), as well as one that only uses the skip-gram features (SNM5-skip (no n-grams)).


(16) expresses the general update rule for both type of examples, we first have to separate it into updates for negative and positive examples and then adapt accordingly.


When investigating possible causal factors for observed phenomena, one powerful method, if available, consists in intervening on the postulated causes to observe possible changes in the observed effects.

if A causes B, then changing A or properties of A should result in an observable change in B.


We propose such a method based on word order permutations which allows us to manipulate word order properties analogously to familiar wordlevel properties and study their effect on parsing performance.

given a dependency treebank, we obtain new synthetic data by permuting the original order of words in the sentences, keeping the unordered dependency tree constant.


We calculate the overall entropy of arc directions in a treebank conditioned on the relation type defined by the dependency label Rel and the PoS tags of the head H and the child C: Dir in (1) is the order between the child and the head in the dependency arc (Left or Right).

we compute the entropy of arc direction H for each observed tuple (rel, h, c) independently and weigh them according to the tuple frequency in the corpora.


Another challenge is due to the fact that formal quantitative approaches to studying word order variation cross-linguistically are just beginning to appear and not all word order features relevant for parsing performance have been identified.

our results suggest that the relative order between the children (and not only the order between heads and their children) should be taken into account (Section 4.2).


Artificially permuted treebanks were previously used in Fong and Berwick (2008) as stress-test diagnostics for cognitive plausibility of parsing systems.

fong and Berwick (2008) permuted the order of words in the English Penn Treebank to obtain 'unnatural' languages.


First we will investigate more properties related to word order.

we will apply the method to the non-projectivity property.


For each token in the match, the feature is en-   coded in BIOES annotation (Begin, Inside, Outside, End, Single), indicating the position of the token in the matched entry.

b will not appear in a suffix-only partial match, and E will not appear in a prefix-only partial match.


To the best of our knowledge, our best models have surpassed the previous highest reported F1 scores for both CoNLL-2003 and OntoNotes.

with no external knowledge other than word embeddings, our model is competitive on the CoNLL-2003 dataset and establishes a new state of the art for OntoNotes, suggesting that given enough data, the neural network automatically learns the relevant features for NER without feature engineering.


Then word vectors are obtained by lowrank SVD on this matrix, or a related matrix with term reweightings.

the PMI matrix is found to be closely approximated by a low rank matrix: there exist word vectors in say 300 dimensions, which is much smaller than the number of words in the dictionary, such that where ≈ should be interpreted loosely.


we propose a probabilistic model of text generation that augments the log-linear topic model of Mnih and Hinton (2007) with dynamics, in the form of a random walk over a latent discourse space.


Researchers have tried to understand why vectors obtained from the highly nonlinear word2vec models exhibit linear structures (Levy and Goldberg, 2014a;Pennington et al., 2014).


To solve these tasks, we use linear algebraic queries.

9 first normalize the vectors to unit norm and then solve "a:b::c:??"


Our model is tailored to capturing semantic similarity, more akin to a log-linear dynamic topic model.


Going beyond previous work, we make di-rect modifications to four key components in the SMT pipeline: 1 1) two novel simplification-specific tunable metrics; 2) large-scale paraphrase rules automatically derived from bilingual parallel corpora, which are more naturally and abundantly available than manually simplified texts; 3) rich rule-level simplification features; and 4) multiple reference simplifications collected via crowdsourcing for tuning and evaluation.

we report the first study that shows promising correlations of automatic metrics with human evaluation.


This explains why they are more suitable than BLEU to be used in training the simplification models.

sARI provides a balanced and integrative measurement of system performance that can assist iterative development.


Our algorithm, on the other hand, makes a more indirect use of the source of prior knowledge, by changing the co-occurence matrix on which we do singular value decomposition.

we believe that our algorithm is more stable to cases in which words for the task at hand are unknown words with respect to the source of prior knowledge.


The functions N and O defining the RNN are parameterized by parameters θ (matrices and vectors), which are trained from data.

one is usually interested in using some of the outputs y i for making predictions.


The RNN is trained such that the encoding y i is good for the prediction task.

the RNN learns which aspects of the sequence x 1 , .


• Is the modifier m compatible with the already existing modifiers of h?

is m a good subtree to connect as an outer-most modifier in the subtree h?


In order to expose the parser to more errors, we employ a cost augmentation scheme: we sometimes follow incorrect actions also if they score below correct actions.

when the score of the correct action is greater than that of the wrong action but the difference is smaller than the margin constant, we chose to follow the wrong action with probability p aug (we use p aug = 0.1 in our experiments).


However, we find that this is not the case.

we find that both counselor groups are very similar in how they select conversations from the queue (picking the top-most in 60.1% vs. 60.3%, respectively), work similar shifts, and handle a similar number of conversations simultaneously (1.98 vs. 1.83).


We investigate this question by looking for language differences between positive and negative conversations.

we compute a distance measure between the language counselors use in positive conversations and the language counselors use in negative conversations and observe how this distance changes over time.


Observing that successful counselors are better at adapting to the conversation, we next examine how counselors differ and what factors determine the differences.

domain experts have suggested that more successful counselors are better at handling ambiguity in the conversation (Levitt and Jacques, 2005).


We show that the linguistic aspects of crisis counseling explored in previous sections have predictive power at the level of individual conversations by evaluating their effectiveness as features in classifying the outcome of conversations.

we create a balanced dataset of positive and negative conversations more than 30 messages long and train a logistic regression model to predict the outcome given the first x% of messages in the conversation.


Removing the need for computing these values at query time leaves only pattern search and a few floating point operations in order to compute language model probabilities (see §4.3) which can be done cheaply.


Even better, we could treat linguistic structure discovery as a supervised learning problem.

we could train a system to extract features from the surface of a language that are predictive of its deeper structure.


In this paper, we select a substrate language S represented in the UD treebanks, and systematically reorder the dependents of some nodes in the S trees, to obtain trees of a synthetic language S .

we choose a superstrate language R V , and write S = S[R V /V] to denote a (projective) synthetic language obtained from S by permuting the dependents of verbs (V) to match the ordering statistics of the R V treebanks.


However, if the plot shows a mean of 57, an interdecile range from 53 to 61 actually means that the middle 80% of experiments were within ±4 percentage points of the mean UAS for their target language.

(before computing this range, we adjust each UAS score for target T by subtracting the mean UAS from the experiments with target T , and adding back the mean UAS from all 10,000 experiments (e.g., 57).)


The second was a TensorFlow-based RNN with an attention mechanism (Mnih et al., 2014), using an overall architecture similar to that used in a system for end-to-end speech recognition (Chan et al., 2016).

we used a 4-layer pyramidal bidirectional LSTM reader that reads input characters, a layer of 256 attentional units, and a 2-layer decoder that produces word sequences.


This method is to find how much more evidence for t 1 → t 2 is found on the Web than for t 2 → t 1 .

a list of manually constructed taxonomic patterns (e.g., "t 2 is a t 1 ") is queried with a Web search engine to estimate the amount of evidence for t 1 → t 2 from the Web.


• Baseline 1: The taxonomy is updated with the newly obtained data every two days, but does not use any temporal information in either taxonomic relation identification (Section 4.2.2) or taxonomy induction (Section 4.2.4).

step 1 (update existing taxonomy) in section 4.2.4 is excluded since we are not using any temporal information, so there is no updating of the weights of the existing taxonomy using the decay function.


The control rate ξ in Equation (1) and decay rate λ in Equation (6) affect the contribution of old and new data.

smaller values for the control rate and decay rate will allow newer data to contribute more evidence of taxonomic relations than older data, whereas larger values will cause the old and new data to have similar evidence contribu-tions.


We discuss implications for automatic labeling in Section 5.

users prefer shorter labels, labels that include topic words and phrases, and abstraction in topic labeling.


An open question that we do not address is whether this generalizes to understanding entire topic models.

simple word list visualizations are useful for quick and high-quality topic summarization, but does this mean that a collection of word listsone per topic-will also be optimal when displaying the entire model?


z noticeably differs from r only when correlations exceed .5, and r to z changes decoding behaviour in select circumstances.

r to z can influence how word labels are as- signed to similarity vectors by upweighting high value correlation coefficients at the final stage of decoding.


As an empirical demonstration, we reran all of our 21 actual (non-permuted) model-based decoding analyses, that are reported later in Section 5.2, whilst selecting voxels from all 70 words (as opposed to leave-2-out voxel-selection on 68/70 words).

decoding analyses were repeated for all 7 model combinations, and tested first on all words, then for the most concrete words only, and finally the most abstract words only.


Accuracies achieved using models to decode the group-level neural similarity matrices are displayed in the final column of the bar charts at the right of Figure 3.

decoding accuracies were: For all words combined: Img=84.8%, TXit&TXen=96.9% and Img&TXit&TXen=97.3%.


Since it is not feasible to enumerate all potential participants, for 'new' DRs we only predict their participant type ("bath means" in our example).

the number of categories in our model is equal to the number of previously introduced DRs plus the number of participant types of the script plus 1, reserved for a new DR not corresponding to any script participant (e.g., cellphone).


Now we turn to another question raised in the introduction: does incorporation of this knowledge make our predictions more human-like?

are we able to accurately estimate human expectations?


To address this limitation, we leverage a fast neural model to extract lookahead features.

we build a bidirectional LSTM model, which leverages full sentence information to predict the hierarchy of constituents that each word starts and ends.


We leverage RNNs for building our constituent hierarchy predictor.

an LSTM (Hochreiter and Schmidhuber, 1997) is used to learn global features automatically from the input words.


As shown in Figure 1, the word "The" starts a constituent hierarchy "S → NP".

it starts a constituent S in the top level, dominating a constituent NP.


The word "table" ends a constituent hierarchy "S → VP → NP → PP → NP".

it ends a constituent hierarchy, with a constituent S on the top level, dominating a VP (starting from the word "like"), and then an NP (starting from the noun phrase "this book"), and then a PP (starting from the word "in"), and finally an NP (starting from the word "the").


Each constituent hierarchy is generated bottom-up recurrently.

a sequence of state vectors is generated recurrently, with each state yielding a output constituent label.


By integrating a neural constituent hierarchy predictor, our parser is related to neural network models for parsing, which has given competitive accuracies for both constituency parsing (Dyer et al., 2016;Cross and Huang, 2016;Watanabe and Sumita, 2015) and dependency parsing (Chen and Manning, 2014;Zhou et al., 2015;.

our parser is more closely related to neural models that integrate discrete manual features (Socher et al., 2013;Durrett and Klein, 2015).


To model phonotactic learning, we make use of tools from Bayesian nonparametric statistics.

we make use of the notion of lexical memoization (?


Because all and-nodes are recursively memoized, our model is able to bind together particular non-oral choices (node B), learning for instance that the combination {NASALITY:+, VOICED:+} commonly recurs for both vowels and consonants in a language.

{NASALITY:+, VOICED:+} becomes a high-probability memo draw.


If we decide to do a base draw instead, we then repeat the procedure conditional on the previous n − 1 segments, recursively until we are conditioning on the empty context.

we do a memo draw conditional on /k/, or conditional on the empty context.


The mixture weights are drawn from a Dirichlet process.

for or-node n in the underlying graph G, the vector of probabilities over outgoing edges is distributed as follows.


In this setup, the distribution over next symbols given a context u is drawn hierarchically from a Dirichlet process whose base measure is another Dirichlet process associated with context tl(u), and so on, with all draws ultimately backing off into some unconditioned distribution over all possible next symbols.

in a hierarchical Dirichlet process N-gram model, H fn(u) is given as follows.


Our model assigns higher probabilities to short forms than BLICK.

our models have identified more redundant structure in the forms than BLICK, allowing them to compress the data more.


Context gates are non-linear gating units which can dynamically select the amount of context information in the decoding process.

at each decoding step, the context gate examines both the source and target contexts, and outputs a ratio between zero and one to determine the percentages of information to utilize from the two contexts.


The inputs to the decoder (i.e., s i , t i−1 , and y i−1 ) represent the contexts.

the source representation s i stands for source context, which embeds the information from the source sentence.


We confirm that context gate weight z i correlates well with translation performance.

translations that contain higher z i (i.e., source context contributes more than target context) at many time steps are better in translation performance.


In this paper, we adopt a simple strategy that performed quite well in preliminary experiments, and leave further exploration to future work.

we partition the document graph into two directed acyclic graphs (DAGs).


Furthermore, we avoid picking unlikely candidates where the triples are far apart in the document.

we considered entity triples within K consecutive sentences, ignoring paragraph boundaries.


The two variants of graph LSTMs perform on par with each other, though Graph LSTM-FULL has a small advantage, suggesting that further exploration of parametrization schemes could be beneficial.

the edge-type embedding might improve by pretraining on unlabeled text with syntactic parses.


While there is much room to improve in both recall and precision, our results indicate that machine reading can already be useful in precision medicine.

automatically extracted facts (Section 5.4) can serve as candidates for manual curation.


Most of these techniques represent each word of the vocabulary by a distinct vector, without parameter sharing.

they ignore the internal structure of words, which is an important limitation for morphologically rich languages, such as Turkish or Finnish.


These two vectors are sometimes referred to as input and output vectors in the literature.

we have vectors u wt and v wc , corresponding, respectively, to words w t and w c .


Fleshing this out, EC estimates directionalities aŝ where we estimate the expected Here u ranges over tag sequences (sentences) of u, and w is a window size that characterizes "nearby."

3 we ask: given that u i and u j are nearby tag tokens in the test corpus u, are they likely to be linked?


For training, we use the remaining 20 UD languages and tune the hyperparameters with 5-fold crossvalidation.

for each fold, we train the system on 16 real languages and evaluate on the remaining 4.


We also tested our directionality prediction system on noisy input (without retraining it on noisy input).


Nonetheless, they do perform better, and the EC heuristic continues to perform worse.

eC fails significantly on dobj and iobj.


Given head lexicon vectors for nodes, the Tree LSTM of  can be extended by leveraging x t in calculating the corresponding c t .

x t is used to estimate the input (i t ), output 1 In this paper, we work on binary trees only, which is a common form for CKY and shift-reduce parsing.


For sentiment classification, we use the same data settings as .

we use the Stanford Sentiment Treebank (Socher et al., 2013b).


(2017) show that such compositions are the key to the success, and further investigate several alternatives neural network structures.

they compare vanilla LSTMs to attention networks when composing child constituents.


Unlike ordinary discriminative word segmentation which relies only on labeled data, our semi-supervised model also leverages a huge amounts of unlabeled text to automatically learn new "words", and further constrains them by using a labeled data to segment non-standard texts such as those found in social networking services.

our hybrid model combines a discriminative classifier (CRF; Lafferty et al.


To alleviate this complexity, we once again approximate the large number of negative links using smaller number of subsamples.

while keeping the positive link rows Ψ a + intact, we approximate negative link rows in Ψ a using a smaller matrix Ψ a − that  Figure 3: Training time of the LTAI on CORA dataset with stochastic and batch variational inference.


For all documents in each test set, we use half of the words for training per-document topic proportion θ and predict the probability of word occurrence regarding the remaining half.

the predictive probability for a word in a test set w new with respect to the given words w obs and the training document D train is computed using the equation Figure 4 illustrates the per-word log-predictive probability in each corpus.


We evaluate model predictability regarding which publication is originally citing a certain publication.

we randomly remove one citation from each of the documents in the test set.


While there have been substantial advances in knowledge extraction techniques, the availability of high precision, general knowledge about the world, remains elusive.

our goal is a large, high precision body of (subject,predicate,object) statements relevant to elementary science, to support a downstream QA application task.


this is a generous notion of "a KB containing a fact", in order to be as fair as possible.


To this end, we developed a similar model to the one proposed in Section 3, except for using the original dense word representations for inducing features.

according to this modification, we made the following change in our feature function: instead of calculating Equation 2for some word i, the modified feature function we use for this baseline is instead of relying on the nonzero values in α i , each word is characterized by its k real-valued coordinates in the embedding space.


Thus, the exact effect onr or Z of changing an input m ik can be found from the partial derivatives with respect to it.

if we increased m ik by ∆ ∈ {−1, 1} (to flip this bit), the new value ofr would be exactlỹ It remains to compute these partial derivatives.


This paper considers the problem of cross-lingual syntactic transfer with limited resources of monolingual and translation data.

we use the Bible corpus of Christodouloupoulos and Steedman (2014) as a source of translation data, and Wikipedia as a source of monolingual data.


We use self-training (McClosky et al., 2006) to further improve parsing performance.

we first train a delexicalized model on treebanks T 1 .


A cross-lingual hierarchical clustering is a function C(w, l) where the clusters are shared across the (m + 1) languages of interest.

the word w 2) a function t(w, i, j) that translates a word w ∈ L i to w ∈ L j ; and 3) a parameter α such that 0 < α < 1.


We construct three author social networks based on the follow, mention, and retweet relations between the 7,438 authors in the training dataset, which we refer as FOLLOWER, MENTION and RETWEET.

3 we use the Twitter API to crawl the friends of the SemEval users (individuals that they follow) and the most recent 3,200 tweets in their timelines.


The goal of the social attention model is to assign similar basis weights to authors who are nearby in the social network G. We operationalize social proximity by embedding each node's social network position into a vector representation.

we employ the LINE method (Tang et al., 2015b), which estimates D (v) dimensional node embeddings v a as parameters in a probabilistic model over edges in the social network.


All distances were constrained to be larger than a lower bound of = 5 to ensure perceptible differences, and we used a threshold value of θ = 20 to create conditions.

2 we set a length criterion at 4σ of the mean number of words per message (about 14 words, in our case), excluding 627 utterances.


We therefore also evaluate models defined as a weighted average of L 0 and each of L 1 and L 2 , as well as an "ensemble" model that combines all of these agents.

we consider the following blends of neural base models and pragmatic models, with L i abbreviating L i (t | u, C; θ, φ) for convenience: The hyperparameters in the exponents allow tuning the blend of each pair of models-e.g., overriding the neural model with the pragmatic reasoning in L b .


Examining the full probability tables for various dev set examples offers insight into the value of each model in isolation and how they complement each other when blended together.

we see that the listener-based (L 2 ) and speaker-based (L 1 ) pragmatic listeners each overcome a different kind of "blind spot" in the neural base listener's understanding ability.


To visualize the model in action we began with a small corpus of 74 triples of semantically identical cross-language phrases.

each triple contained phrases in English, Japanese and Korean with the same underlying meaning.


The large red region on the left primarily contains zeroshot Portuguese→Spanish translations.

for a significant number of sentences, the zero-shot translation has a different embedding than the two trained translation directions.


Having a mechanism to translate from a random source language to a single chosen target language using an additional source token made us think about what happens when languages are mixed on the source or target side.

we were interested in the following two experiments: 1) Can a multilingual model successfully handle multi-language input (code-switching) in the middle of a sentence?


We would like to thank the entire Google Brain Team and Google Translate Team for their foundational contributions to this project.

we thank Junyoung Chung for his insights on the topic and Alex Rudnick and Otavio Good for helpful suggestions.


For this baseline, we report the results of the publicly available implementation of the technique (NBJ'15), as well as our own improved reimplementation (NBJ-Imp).

in NBJ-Imp, we expanded the original algorithm to handle compounding, along with sibling features as described in Section 3.2, making it essentially an ablation of our model without ILP and alternating training.


Evaluation We use the metrics proposed by Schone and Jurafsky (2000).

let X w  and Y w be the clusters for word w in our predictions and gold standard, respectively.


(2016) present three main arguments: character level models (1) do not suffer from out-of-vocabulary issues, (2) are able to model different, rare morphological variants of a word, and (3) do not require segmentation.

text segmentation is highly non-trivial for many languages and problematic even for English as word tokenizers are either manually designed or trained on a corpus using an objective function that is unrelated to the translation task at hand, which makes the overall system sub-optimal.


Note that the rightmost segment (above 'on') in Figure 1 may capture 'son' (the filter in green) although 's' occurs in the previous segment.

our segments are overlapping as opposed to in word-or subword-level models with hard segmentation.


The first strategy is to use a decision tree to pick out highly discriminative properties for each WordNet sense.

for each set of cohyponyms, 15 we train a decision tree using the associated templates as features.


We then collect candidates of automatically generated common-sense inferences (AGCI) against these contexts.

in the SNLI train set, there are over 150K different first sentences, involving 7,414 different arguments according to predicate-argument extraction.


we ask the annotator how plausible the hypothesis is true during or shortly after, because without this constraint, most sentences are technically plausible in some imaginary world.


In this paper, we describe a novel method to bridge these two different approaches.

we propose Neural Text-Entity Encoder (NTEE), a neural network model to jointly learn distributed representations of texts (i.e., sentences and paragraphs) and KB entities.


• We demonstrate that our proposed representations are surprisingly effective for various NLP tasks.

we apply the proposed model to three different NLP tasks, namely semantic textual similarity, entity linking, and factoid question answering, and achieve stateof-the-art results on all three tasks.


Furthermore, in order to conduct a fair comparison between the skip-gram model and our model, we also add skip-gram (plain), which is a skip-gram model trained using a different corpus.

the corpus is augmented using the texts in DBpedia abstract corpus 7 , and its entity annotations are treated as regular text phrases (not replaced to their unique identifiers).


(2016) for the sake of compatibility with their state-of-the-art results.

we use a public dataset proposed in Pershina et al.


Our method achieved strong results on both the CoNLL and the TAC 2010 datasets.

the NTEE model clearly outperformed the other proposed models.


It appeared that our method neglected to distinguish whether a location name refers to the location itself or a sports team.

our method often wrongly resolved these mentions referring to sports team entities into the corresponding location entities and vice versa.


Table 5: Accuracies of the proposed method and the state-of-the-art methods for the factoid QA task.

despite the simplicity of the neural network architecture of our method compared to the state-of-the-art methods (i.e., QANTA and FTS-BRNN), our method clearly outperformed these methods.


For the entities Europe and Golf, we observe similar trends to our word representations.

in our model, the most similar entities of Europe and Golf are Eastern Europe and Golf course, respectively, whereas those of the skip-gram model are Asia and Tennis, respectively.


We find that the prediction of these constituent types requires, explicitly, modeling of bottom-up structures.

bottom-up information is necessary for us to know if the span can be a noun phrase (NP) or sentence (S), for example.


We propose a new representation and algorithm for a class of graph structures that is flexible enough to cover almost all treebank structures, while still admitting efficient learning and inference.

we consider directed, acyclic, one-endpoint-crossing graph structures, which cover most long-distance dislocation, shared argumentation, and similar tree-violating linguistic phenomena.


Together with Lemma 3 and the final item definition, this means the final structure is an acyclic graph with all vertices reachable from vertex n. Next, we will show two properties that give intuition for the algorithm.

we will prove which rules add edges that are crossed in the final derivation.


In this paper we will refer to the targets of our lexicon creation efforts as formulaic sequences, following the terminology of Wray (2002;2008), wherein a formulaic sequence (FS) is defined as "a sequence, continuous or discontinuous, of words or other elements, which is, or appears to be, prefabricated: that is, stored and retrieved whole from memory at the time of use, rather than being subject to generation or analysis by the language grammar."

an FS shows signs of being part of a mental lexicon.


situations encountered in NLP problems, and giving specific recommendations for such situations.

we first demonstrate (Section 3) that the current prominent approach in the NLP literature, identifying the datasets for which the difference between the performance of the algorithms reaches a predefined significance level according to some statistical significance test, does not guarantee to bound the probability to make at least one erroneous claim.


The replicability analysis framework we employ (Benjamini and Heller, 2008;Benjamini et al., 2009) is based on partial conjunction testing.

these authors have shown that a lower bound on the number of false null hypotheses with a confidence level of 1 − α can be obtained by finding the largest u for which we can reject the partial conjunction null hypothesis H at a significance level α.


Benjamini and Heller (2008) proposed to estimate k to be the largest u for which is rejected.

the estimatork is defined as follows: where p and α is the desired upper bound on the probability to overestimate the true k. It is guaranteed that P(k > k) ≤ α as long as the p−value combination method used for constructing p u/N is valid for the given dependency across the test statistics.


13 Dependent Datasets In cross-domain sentiment classification (Table 4) and word similarity prediction (Table 5), the involved datasets manifest mutual dependence.

each sentiment setup shares its test dataset with 2 other setups, while in word similarity WS-353 is the union of WS-353-REL and WS-353-SIM.


An alignment type is NTR if either Chinese or English list of tokens for that alignment is empty.

the NTR alignment type is used when some words are dropped during the translation process.


On the pathology dataset, we explore cross-aspect transfer across different types of breast disease.

we test on six adaptation tasks, consistently outperforming all other baselines.


The purpose of this reconstruction step is to balance adversarial training signals propagating back from the domain classifier.

it forces the sentence encoder to keep rich word-level information in contrast to adversarial training that seeks to eliminate aspect specific features.


To help remove non-transferable information, we add a transformation layer to map the initial document vectors x doc,a l to their domain invariant (as a set) versions, as shown in Figure 2.

the transformed representation is given by x tr,a l = W tr x doc,a l .


Let y a = [y a 1 , y a 2 ] denote the one-hot domain label vector for aspect a ∈ {s, t}.


For example, the correlation between room quality and cleanliness is 0.81, much higher than aspect correlations in the pathology dataset.

the sentiment is typically consistent across all sentences in a review, so that selecting aspect-specific sentences becomes unnecessary.


This framework naturally generalizes to hierarchical and semisupervised extensions with no additional modeling assumptions.

word-level domain knowledge can be flexibly incorporated within CorEx through anchor words, allowing topic separability and representation to be promoted with minimal human intervention.


In this work we relax some of these simplifications by advocating a new task for natural language understanding which is multi-modal, exhibits spoken conversation, and is incremental, i.e., unfolds sequentially in time.

we argue that crime drama exemplified in television programs such as CSI: Crime Scene Investigation can be used to approximate real-world natural language understanding and the complex inferences associated with it.


What type of inference strategy is appropriate?

does access to past information matter for making accurate inferences?


Secondly, we collected goldstandard labels indicating whether the perpetrator is mentioned.

while a participant watches an episode, we record their guesses about who the perpetrator is (Section 3.1).


The system defaults to the perpetrator class for any sentence containing a personal (e.g., you), possessive (e.g., mine) or reflexive pronoun (e.g., ourselves).

it assumes that every pronoun refers to the perpetrator.


Overall, we observe that humans outperform all comparison models.

human precision is superior, whereas recall is comparable, with the exception of PRO which has high recall (at the expense of precision) since it assumes that all pronouns refer to perpetrators.


In this section we assess more directly how the LSTM compares against humans when asked to identify the perpetrator by the end of a CSI episode.

we measure precision in the final 10% of an episode, and compare human performance (first-pass guesses) and an LSTM model which uses all three modalities.


Under multiple instance learning (MIL), a dataset D is a collection of labeled bags, each of which is a group of unlabeled instances.

each document d is a sequence (bag) of segments (instances).


As mentioned earlier, one of the hypotheses investigated in this work regards the use of subsentential units as the basis of extraction.

our model was applied to sentences and Elementary Discourse Units (EDUs), obtained from a Rhetorical Structure Theory (RST) parser (Feng and Hirst, 2012).


In order to evaluate model performance on the segment level, we constructed a new dataset named SPOT (as a shorthand for Segment POlariTy) by annotating documents from the Yelp'13 and IMDB collections.

we sampled reviews from each collection such that all document-level classes are represented uniformly, and the document lengths are representative of the respective corpus.


In this paper, we explore the problem of spatial reasoning in the context of interactive worlds.

we assume access to a simulated environment, in which an agent can take actions to interact with the world and is rewarded for reaching the location specified by the language instruction.


In order to build a robust instruction following system, we examine several categories of spatial expressions that exhibit the wide range of natural language goal descriptions.

we consider instructions that utilize objects/entities present in the environment to describe a goal location.


Drawing inspiration from recent efforts to empower neural networks with a structural bias (Cheng et al., 2016; Kim et al., 2017), we propose a model that can encode a document while automatically inducing rich structural dependencies.

we embed a differentiable non-projective parsing algorithm into a neural model and use attention mechanisms to incorporate the structural biases.


(2017) introduce structured attention networks which are generalizations of the basic attention procedure, allowing to learn sentential representations while attending to partial segmentations or subtrees.

they take into account the dependency structure of a sentence by viewing the attention mechanism as a graphical model over latent variables.


(2017) induce richer internal structure by imposing structural constraints on the probability distribution computed by the attention mechanism.

they normalize f ij with a projective dependency tree using the inside-outside algorithm (Baker, 1979): This process is differentiable, so the model can be trained end-to-end and learn structural information without relying on a parser.


In this paper we use bidirectional LSTMs as a way of representing elements in a sequence (i.e., words or sentences) together with their contexts, capturing the element and an "infinite" window around it.

we run a bidirectional LSTM over sentence T , and take the output as the representations of words in T , where h t ∈ R k is the output vector for word u t based on its context.


To gain further insight on structured attention, we inspected the dependency trees it produces.

we used the Chu-Liu-Edmonds algo- 14.4% depth 6 10.3% 4.5% Same Edges 38.7% Table 5: Descriptive statistics for dependency trees produced by our model and the Stanford parser  on the SNLI test set.


Following the methodology proposed by Chambers and Jurafsky (2008), we built a database of event pairs from the GigaWord Fifth Edition corpus (Parker et al., 2011).

we used the Annotated Gigaword distribution (Napoles et al., 2012), which has been automatically annotated with typed dependency information (de Marneffe and Manning, 2008).


For this, we compute event chains, which are defined as sequences of events that share the same actor or object, in subject or direct object role (Chambers and Jurafsky, 2008).

we encode the following additional features in the Events feature set: (4) the length of the longest chain found in the essay (i.e., number of event pairs in the chain); (5) the score of the longest chain (computed as the sum of PMI values for all links (event pairs) of the chain); (6) the length of the second longest chain found in the essay; (7) the score of the highest scoring chain in the essay; (8) the score of the second highest scoring chain in the essay; (9) the score of the lowest scoring chain is the essay; (10) the sum of scores for all chains in the essay.


Features based on connectivity, shape and PageRank were extracted, giving a total of 19 Graph features.

the features used were: percentage of nodes with degrees one, two and three; the highest, second-highest and median degree in the graph; the highest degree divided by the total number of links; the top three PageRank values in the graph, their respective negative logarithms, and their essay length-normalized versions; the median PageRank value in the graph, its negative log and essay length-normalized version.


The node state vector h t is generated using a modification of the standard LSTM equations to include both hierarchical and timing structures for each comment.

we use two forget gates -one for the previous (or subsequent) hierarchical layer, and one for the previous (or subsequent) timing layer.


To that end, we define a backward LSTM that has a similar set of update equations except that only the first child will pass the hidden state to its parent.

the update equations are the same except that π(t) is replaced with κ(t), p(t) is replaced with s(t), and a different set of weight matrices and bias vectors are learned.


Experiments reported here use this framework.

all the comments with karma lower than 1 are assigned to level 0, and each subsequent level corresponds to karma less than or equal to the median karma in the rest of the comments based on the training data statistics.


Second, we identified words that may be indicative of comments that are over-and underpredicted by the graph-structured model without text and for which the graph-LSTM model with text reduced the error by more than 2 levels.

we choose those words w in comments having the highest ratio r = p(w|t)/p(w), where t indicates an over-or underpredicted comment, subject to minimum occurrence constraints (5 for overpredicted comments, 15 for underpredicted comments).


Some options include summarizing over the children, adding a separate forget gate for each Ex karma Comment 1 0 7 7 0 Republicans are fundamentally dishonest.


Hence, Regier's (1996) setting does not seem plausible to deal with actions given that, in contrast to the spatial prepositions that they use, which are mutually exclusive (an object cannot be "above" and simultaneously "below" another object), actions are not.

actions exhibit large spatial overlap and, therefore, attempt to predict thousands of different actions from the relative locations of the objects seems infeasible.


Here, we propose an ad hoc setting for both, learning and evaluating spatial representations.

instead of learning to predict spatial relations from visual input as in Regier's (1996) work, we learn the reverse direction, i.e., to map the relation (and two objects) to their visual spatial arrangement.


To evaluate how well our embeddings match human mental representations of spatial knowledge about objects, we collect ratings for 1,016 word pairs (w 1 , w 2 ) asking annotators to rate them by their spatial similarity.

objects that exhibit similar locations in most situations and are placed similarly relative to other objects would receive a high score, and lower otherwise.


4.1) for evaluation purposes: (i) Original set: a test split from the original data which contains instances unseen at training time.

the test combinations (S, R, O) might have been seen at training time, yet in different instances (e.g., in different images).


To validate results in our Prediction task we employ a 10-fold cross-validation (CV) scheme.

we split the data into 10 parts and employ 90% of the data for training and 10% for testing.


3.2.1), we consider the classification problem of predicting above/below relative locations.

if the predicted y-coordinate for the Object center O c y falls below the y-coordinate of the Subject center S c y and the actual Object center O c y is below the Subject center S c y , we count it as a correct prediction, and as incorrect otherwise.


4 is that both visual and linguistic embeddings show good correlations with human spatial ratings (ρ > 0.45 and ρ > 0.53 respectively), suggesting that visual and linguistic features carry significant knowledge about  spatial properties of objects.

linguistic features seem to be more spatially informative than visual features.


Experimental results show that the proposed approach significantly improves the performance in Chinese-English, German-English, and English-German translation tasks.

the proposed model outperforms the conventional coverage model in terms of both the translation quality and the alignment error rate.


Intuitively, the attention-based decoder selects source annotations that are most relevant to the decoder state, based on which the current target word is predicted.

c t is some source information for the PRESENT translation.


Note that for both GRU-o and GRU-i, we leave enough freedom for GRU to decide the extent of integrating with subtraction operations.


All the activation functions for the FUTURE layer obtain BLEU score improvements: GRU +0.52, GRU-o +1.03, and GRU-i +1.12.

gRU-o is better than  Table 2: Case-insensitive BLEU on Chinese-English Translation.


These rules provide an explanation for the operations performed.

it learns to select relevant rules without explicit annotations for them.


For the above example, the assignment of the variables are shown below: In order to determine the direction of the transfer, we require some classification of verbs.

we classify each verb into one of five classes: HAVE, GET, GIVE, CONSTRUCT and DESTROY.


We observe that our approach is competitive even when just relying on visual attention.

we consider using only the temporal features and motion features (MFATT-TM), by removing the semantic branch with other components of our model unchanged.


For each relation r, the relation schema imposes a type constraint on the entities that may appear as its source or target.


Differences like this lead to differences in what is successful in each setting and what is not.

while all our proposed schemes are embedding oblivious, for concreteness, we describe and evaluate them for the Holographic Embedding or HolE (Nickel et al., 2016b) which models the label probability as: where Intuitively, the k-th dimension of circular correlation captures how related a is to b when the dimensions of the latter are shifted (circularly, via the mod operation) by k. [a • b] 0 is simply the dot product of a and b.


This allows one to make meaningful existential statements such as: if a property holds for all or most members of even one class c i , then it holds for some (reasonable number of) members of its parent class p. We use the following rules: 6 We apply these rules to address sparsity of generics tensors, making tensor factorization more robust.

given initial triples K, we use applicable rules to derive additional triples K , perform tensor factorization on K ∪ K , and then revisit the triples in K using their predicted label probabilities.


Both vanilla TransE and RESCAL resulted in poor performance; we thus report numbers only for their extensions.

we consider 3 baselines: (1) HolE, (2) TransE+Schema, and (3) SICTF which extends  : Top 20 predictions by various methods, with invalid triples underlined and uninteresting ones , such as (X, is known as, X) or (Y, resembles, Y), shown in italics.


Figure 1 shows the resulting precision-yield curves for the predictions made by each method on the Animals dataset containing 10.6K facts.

for each method, we rank the predictions based on the method's assigned score and compute the precision of the top k predictions for varying k. As expected, we observe a generally decreasing trend as k increases.


We make two observations: First, deriving new facts for these generics tensors at a high precision is challenging!

none of the baseline methods (black and pink curves), which represent state of the art for named-entity tensors, achieve a yield of more than 10% of T (i.e., 1K predictions) even at a precision of just 60%.


Second, external information, if used appropriately, can be surprisingly powerful in this setting.

simply incorporating relation schema (ITRS, blue curve) allows HolE-based completion to double the size of the starting tensor T by producing over 10K new triples at a precision of 82%.


The word embeddings for the above two domains were trained by us on the respective corpora using word2vec (Mikolov et al., 2013) which employs a two-layer neural network and is by far the most widely used word embedding technique.

we ran word2vec with skip-gram of a fiveword window to construct word vectors of 500 dimensions, as recommended by previous studies 7 .


The qualitative analysis of the sentiment lexicons induced by different methods shows that they differ only on those borderline, ambiguous words (such as "soft") residing in the noisy overlapping region between two clusters in the vector space (see Section 3).

sentProp is based on label propagation over the lexical graph of words, so it could be easily misled by noisy borderline words when sentiment clusters have considerable overlap with each other, kind of "over-fitting" (Bishop, 2006).


What we discovered is that the simple method of identifying neutral documents from the binary sentiment classifier's decision boundary works surprisingly well, as long as the right thresholds are found.

we take the probabilistic outputs of a binary sentiment classifier trained as before, and then put all the documents whose proba-bility of being positive lies not close to 0, not close to 1, but in the middle range into the neutral class.


This paper presents our exploration towards answering the above research question.

the main contributions of this paper are as follows.


The documents visited to reach the end points will define the support document set S q .

s q comprises chains of documents leading not only from the query subject to the correct answer candidate, but also to type-consistent false answer candidates.


We restrict the set of entities in the bipartite graph to drugs in DRUGBANK and human proteins in SWISS-PROT (Bairoch et al., 2004).

the graph has drugs and proteins on one side, and MEDLINE abstracts on the other.


Predicting the answer most frequently observed at training time achieves strong results: as much as 38.8% / 44.2% and 58.4% / 67.3% on the two datasets, for the full and validated test sets respectively.

a simple frequency statistic together with answer type constraints alone is a relatively strong predictor, and the strongest overall for the "unmasked" version of MEDHOP.


The TF-IDF retrieval baseline clearly performs better than random for WIKIHOP, but is not very strong overall.

the question tokens are helpful to detect relevant documents, but exploiting only this information compares poorly to the other baselines.


All of these previous approaches center around learning how to combine facts from a KB, i.e., in a structured form with pre-defined schema.

they work as part of a pipeline, and either rely on the output of a previous IE step (Banko et al., 2007), or on direct human annotation (Bollacker et al., 2008) which tends to be costly and biased in coverage.


To test if this explains the improved performance, we compared multilingual and bilingual models under similar data size conditions, i.e., the total parallel corpora size across all language pairs used to train the multilingual model is equivalent to the size of the bilingual corpus used to train the bilingual model.

we compared the following under similar data size conditions: (a) {bn,hi,kn,ta}-en multilingual model (50% hi-en training pairs) with hi-en bilingual model (b) {cs,pl,sk,sl}-ar multilingual model (30% pl-ar training pairs) with pl-ar bilingual model.


Answering such questions requires that readers integrate information which may be distributed across several statements throughout the document, and generate a cogent answer on the basis of this integrated information.

they test that the reader comprehends language, not just that it can pattern match.


We find that the proposed dataset meets the desiderata we set out in Section 3.1.

we constructed a dataset with a number of long documents, characterized by good lexical coverage and diversity.


We found that the retrieval is challenging, even for humans not familiar with the presented narrative.

the task often requires referring to larger parts of the story, in addition to knowing at least some background about entities.


Such traces are known as transfer effects, and they can be phonological (a foreign accent), morphological, lexical, or syntactic.

psycholinguistic research has shown that the choice of lexical items is influenced by the author's L1, and that non-native speakers tend to choose words that happen to have cognates in their native language.


model to re-score the final output of the Seq2seq decoder (for each time step) by computing a linear combination of the output vocabulary distributions proposed by the Seq2seq model and polite-LM.

let p S2S t and p LM t denote the output probability distributions proposed by the Seq2seq model and the LM model at time t, respectively.


To address these two issues, we present our label-fine-tuning (LFT) model, which prepends a predicted continuous style label at the beginning of each input sentence to specify the intended politeness level.

we add to the vocabulary a single politeness label and attach with it a trainable word embedding, just like what we would do to a normal token.


Intuitively, this prepended label serves as the prior for the intended style of the generated response sequence, while the source utterance serves as the prior for the content of the generated sequence.

the label and the source sentence cooperatively determine what the overall response looks like.


The teacher forcing part minimizes the average of the maximum-likelihood loss at each decoding step.

let y * = {y * 1 , y * 2 , ..., y * n } be the ground-truth response for a given source (conversation history) utterance sequence x.


The maximum-likelihood training objective is the minimization of the loss: We use a policy gradient method (Williams, 1992;Sutton et al., 2000) to calculate the second term in the objective function.

we sample a generated response for each input sequence (conversation history) x, and assign to it a reward R, which in our case is the politeness classifier's probability of the response classified as polite.


(2015), for a [X 1 , Y, X 2 ] triple, our retrieval model treats the context (X 1 , Y ) and each response (X 2 ) as two documents and converts them to their TF-IDF based vectors (Ramos, 2003) to check for similarity.

we first obtain all candidate responses in the training set that are polite, 7 and calculate their TF-IDF vectors.


This approach is similar to the one above but uses the 10 manually-chosen most-polite generic utterances as candidate responses for each context.

we collect all ground-truth polite requests from the Stanford Politeness Corpus, split each one into sentences, and then manually pick the most frequent 10 polite sentences.


Human To evaluate our models' ability to generate polite responses without sacrificing dialogue quality, we conducted several comprehensive human evaluation studies on Amazon Mechanical Turk (MTurk).

we compare the three stylistic models w.r.t.


2013, we use accuracy (i.e., percentage of correctly labeled messages for binary polite/rude labels) to evaluate our politeness classifier's generalization ability.

we used data from the training set of WIKI, and test on both the test set of WIKI and the entire SE (Stack Exchange) corpus.


We argue that a shift to rapid discovery science should be visible in the way citations are used to frame works in the field as a whole.

we expect that as consensus is reached (1) authors are expected to have fewer comparisons to other works and instead can simply acknowledge past work as background and (2) the remaining comparisons concentrate on fewer works, reflecting those works' status as accepted benchmarks of performance.


Our work is inspired by recent successes of memory-augmented neural networks on multiple NLP tasks Sukhbaatar et al., 2015;Miller et al., 2016;Gu et al., 2018), especially the efficient cache-like memory networks for language modeling (Grave et al., 2017;Daniluk et al., 2017).

the proposed approach augments NMT models with a continuous cache (CACHE), which stores recent hidden representations as history context.


As seen, the proposed approach indeed pays more attention to the most recent history (e.g., the leftmost positions) in all domains.

the larger the cache, the more attention the model pays to the most recent history.


Our work is based on the Memory Networks, which have proven useful for question answering and document reading tasks (Weston et al., 2016;Hill et al., 2016).

we use the Key-Value Memory Network (Miller et al., 2016), which is a simplified version of Memory Networks with better interpretability and has yielded encouraging results in document reading (Miller et al., 2016), question answering (Pritzel et al., 2017) and language modeling (Tran et al., 2016;Grave et al., 2017;Daniluk et al., 2017).


Our primary goal is to learn a generative model of sentences for use as a language model.

1 we model sentence generation as a prototypethen-edit process: 1.


Since we construct our edit vectors as the sum of word vectors, and similarities between word vectors have traditionally been measured with cosine similarity, we design q to add noise to perturb the direction of the vector f .

a sample from q is simply a perturbed version of f : obtained by adding von-Mises Fisher (vMF) noise, and we perturb the magnitude of f by adding uniform noise.


To maximize our training objective, we must be able to compute To compute ∇ Θq L gen , we use a reparameterization trick.

we can rewrite z ∼ q(z | x , x) as z = h(α) where h is a deterministic function differentiable with respect to Θ q and α ∼ p(α) is an auxiliary random variable not depending on Θ q (the details of h and α are given in Appendix 6).


We now assess whether the edit vector possesses glob-ally consistent semantics.

applying the same edit vector to different sentences should result in semantically analogous edits.


In our final sample of languages, we select languages belonging to morphological types different from the fusional one, which is over-represented in the PW.

we include new isolating (Min Nan, Burmese, Khmer), agglutinative (Basque, Georgian, Kannada, Tamil, Mongolian, Javanese), and introflexive languages (Amharic).


To achieve this, we turn to another source of information: character patterns.

since M w does not have any information about character patterns from lower layers, we seek a way to: a) detect words with similar subword structures (i.e., "morpheme"level information), and b) let these words share their semantic information.


As one important finding, we observe that the gains in perplexity using our fine-tuning AP method extend also to these larger evaluation datasets.

we find improvements of the same magnitude as in the PTB-sized data sets over the strongest baseline model (Char-CNN-LSTM) for all MWC languages.


In this section, we demonstrate that our institutional act tagger provides an efficient and reliable tool for departments to detect and monitor conversational patterns during traffic stops.

we focus on conversational openings, a fundamental aspect of conversations (Schegloff and Sacks, 1973) that is also important for procedural justice (Whalen and Zimmerman, 1987;Ramsey and Robinson, 2015).


So if too many α w,j 's are allowed to be nonzero, then some must necessarily have small coefficients, which makes the corresponding components indistinguishable from noise.

raising k often picks not only atoms corresponding to additional senses, but also many that don't.


Here, we are interested in contexts described by metadata (vs. word history or related documents) and in neural network approaches due to their flexibility for representing diverse types of contexts.

we focus on recurrent neural networks (RNNs) due to their widespread use.


The goal of our experiments is to show that the FactorCell model can deliver improved performance over current approaches for multiple language model applications and a variety of types of contexts.

results are reported for contextconditioned perplexity and generative model text classification accuracy, using contexts that capture a range of phenomena and dimensionalities.


Since the model applies end-to-end learning, we can investigate what the context embeddings learn.

we looked at location (hotels are from 25 cities in the United States) and class of hotel, neither of which are input to the model.


Recurrent neural networks have emerged as the state-of-the-art approach to approximating p(X).

the LSTM cell (Hochreiter and Schmidhuber, 1997) is a specific RNN architecture which has been shown to be effective on many tasks, including language modeling (Press and Wolf, 2017;Jozefowicz et al., 2016;Merity et al., 2016;Inan et al., 2017).


1 In this work, we seek to capture all three aspects in a single, unified framework which allows for multiple rounds of communication.

we study human communication in a sequential language game in which two players, each with private knowledge, try to achieve a common goal by talking.


To focus on the strategic aspects of InfoJigsaw, we filtered the dataset to reduce the words in the tail.

we keep a game if all its messages contain at least one of the 12 most frequent words (shown in Figure 3d) or "yes" or "no".


Then, define the literal semantics of a single-word message x to be x def = {s : s ∧ v x = 0}, where ∧ denotes element-wise and and 0 denotes the zero matrix.

single-property messages can be glossed as "some goal-consistent object has property x".


tion was chosen because it results in a higher utility than the alternatives.

she infers that her partner's private state cannot be one in which a t does not lead to high utility.


In this paper, we address the problem of verb selection for data-to-text NLG through a principled data-driven approach.

we show how to employ Bayesian reasoning to train a probabilistic model for verb selection based on large-scale realworld news corpora, and demonstrate its advantages over existing verb selection methods.


Second, the verb w would be passed through a communication "channel" and be corrupted by the "noise" to produce the percentage change x according to the likelihood function (aka the channel model) P (x|w).

the percentage change x that we see is actually the distorted form of its associated verb w. An alternative, but equivalent, interpretation is that when a pair (x, w) is passed through the noisychannel, the verb w will be lost and finally only the percentage change x will be seen.


1, revealing that the empirical distribution of verbs follows the Zipf's law (Powers, 1998), which is related to the power law (Adamic, 2000;Newman, 2005).

the frequency of the i-th popular verb, f i , is proportional to 1/i s , where s is the exponent characterizing the distribution (shown as the slope of the straight line in the corresponding log-log plot).


Previous research in NLG and psychology suggests that there is wide variation in how different people interpret verbs and words in general, which is probably much larger in the general population than amongst journalists.

the MRR metric would probably underestimate the effectiveness of a verb selection method, since a verb different from the one really used by the writer is not necessarily a less appropriate choice for the corresponding percentage change from the reader's perspective.


In general, CLSC methods make use of generalpurpose bilingual resources-such as hand-crafted bilingual lexica or parallel corpora-to alleviate or eliminate the need for task-specific TARGET annotations.

the bilingual resource of choice for the majority of previous CLSC models is a full-fledged Machine Translation (MT) system (Wan, 2008(Wan, , 2009Lu et al., 2011;Zhou et al., 2016), a component that is expensive to obtain.


In this work, we propose a language-adversarial training approach that does not need a highly engineered MT system, and requires orders of magnitude less in terms of the size of a parallel corpus.

we propose an Adversarial Deep Averaging Network (ADAN) that leverages a set of bilingual word embeddings (BWEs; Zou et al., 2013) trained on bitexts, in order to eliminate the need for labeled TARGET training data.


To illustrate the limitations of ADAN-GRL and motivate the formal introduction of our ADAN model, consider the distribution of the joint hidden features F for both SOURCE and TARGET instances: In order to learn language-invariant features, ADAN trains F to make these two distributions as close as possible for better cross-lingual for qiter = 1 to k do 4: feature vectors   8: Update Q parameters to minimize loss q ClipW eights(Q, −c, c) 11: Main iteration 12: Sample unlabeled batch x tgt ∼ X tgt 14: 15: 16: Eqn 417: Update F, P parameters to minimize loss 18: until convergence generalization.


To shed more light on the surprisingly clear separation given that individual words have a mixed distribution in both languages (not shown in figure), we first try to isolate the content divergence from the language divergence.

the English and Chinese reviews are not translations of each other, and in fact may even come from different domains.


While an extensive study of alternative architectures is beyond the scope of this work, in this section we present a brief experiment illustrating that our adversarial framework works well with other F architectures.

we consider two popular choices: (i) a CNN (Kim, 2014) that has a 1d convolutional layer followed by a single fully-connected layer to extract a fixed-length vector; and (ii) a Bi-LSTM with two variants: one that takes the average of the hidden outputs of each token as the feature vector, and one with the dot attention mechanism (Luong et al., 2015) that learns a weighted linear combination of all hidden outputs.


We do this with value scenarios (Nathan et al., 2007;Czeskis et al., 2010).

we look at two kinds of value scenarios: Those concerning NLP technology that fails to take into account an appropriate match between training data and deployment context and those that envision possible positive as well as negative consequences stemming from the widespread use of the specific "technology" we are proposing in this paper (data statements).


Had Big U included data statements for all system components (including third-party components) in their documentation, then City Hospital IT staff would have been positioned to recognize the potential limitation of DiseaseAlert and to work proactively with Big U to ensure the system performed well in City Hospital's context.

in reviewing data statements for all system components, the IT staff could note that the language ID component was trained on data unlike what they were seeing in their local tweets and ask for a different language ID component or ask for the existing one to be retrained.


(2017) argue that careful attention to and publication of demographic data that may correlate with health inequities can facilitate further work through meta-analyses.

individual studies usually lack the statistical power to do the kind of sub-analyses required to check for health inequities, and failing to publish demographic information precludes its use in the kind of aggregated meta-analyses that could have sufficient statistical power.


Unfortunately, high scores on these tasks do not necessarily translate into acceptable performance for downstream applications such as machine translation (Guillou, 2012) and fact extraction (Nakayama, 2008).

high-scoring systems successfully identify coreference relationships between string-matching proper names, but fare worse on anaphoric mentions such as pronouns and common noun phrases (Stoyanov et al., 2009;Rahman and Ng, 2012;Durrett and Klein, 2013).


Although existing corpora have promoted research into coreference resolution, they suffer from gender bias.

of the over 2,000 gendered pronouns in the OntoNotes test corpus, less than 25% are feminine (Zhao et al., 2018).


We choose Wikipedia as our base dataset given its wide use in natural language understanding tools, but are mindful of its well-known gender biases.

less than 15% of biographical Wikipedia pages are about women.


Although name spans are given in the data, we urge the community not to treat this as a gold mention or Winograd-style task.

systems should detect mentions for inference automatically, and access labeled spans only to output predictions.


Table 4 shows that all systems struggle on GAP.

despite modeling improvements in recent years, ambiguous pronoun resolution remains a challenge.


Table 5 provides evidence that this low performance is not solely due to domain and task differences between GAP and OntoNotes.

with the exception of Clark and Manning (2015), the table shows that system performance on pronoun-name coreference relations in the OntoNotes test set 14 is not vastly better than GAP.


Further, the relative performance of the four systems is different on GAP than on OntoNotes.

interesting is that the current strongest system overall for OntoNotes, namely, Lee et al.


In this visualization, we see a further factor contributing to the low performance of off-the-shelf systems, namely, their low recall.

whereas personal pronouns are overwhelmingly anaphoric in both OntoNotes and Wikipedia texts, OntoNotes-trained models are conservative.


Despite not having access to syntactic structure, TRANSFORMER-SINGLE far outperforms all surface cues above.

we find evidence for the claim that Transformer models implicitly learn language understanding relevant to coreference resolution.


lighted gaps in the existing state of the art, and proposed the application of Transformer models to address these.

we show how traditional linguistic features and modern sentence encoder technology are complementary.


The objectives of this paper are to apply neural methods to DRS parsing.

we are interested in answers to the following research questions (RQs): 1.


For both the neural parsers, only about 2% of the ill-formed DRSs are ill-formed because of a syntactic error in an individual clause (e.g., b1 Agent x1, where a fourth argument is missing), whereas all the other errors are due to a violated semantic constraint (see §3.2).

the produced output is a syntactically well-formed DRS but is not interpretable.


For each noun or verb type W t appearing in the training data, as identified by the Stanford POS tagger, 2 we extract the senses associated to it in WordNet 3 (Fellbaum, 1998) using NLTK.

4 we extract the set of definitions D t = {d tj |j = 1, .


Moreover, we define the initial centroid of each cluster as the word vector corresponding either to the definition d tj of the respective sense, or alternatively to the example e tj illustrating the sense.

for each token w i and its context vector u i the algorithm decides whether the token is assigned to one of the sense clusters S j to which previous tokens have been assigned, or whether it is assigned to a new empty cluster, by selecting the option that has the highest probability, which is computed as follows: for a non-empty sense, the probability is proportional to the popularity of the sense (number of tokens it already contains, N j ) and to the weighted sum of two cosine similarities s(•, •): one between the context vector u i of the token and the definition of the sense d tj , and another one between u i and the average context vector of the tokens already assigned to the sense (µ j ).


To model word senses for NMT, we concatenate the embedding of each token with a vector representation of its sense, either obtained from one of the clustering methods presented in §2, or learned during encoding, as we will explain.

the new vector w i representing each source token w i consists of two parts: where w i is the word embedding learned by the NMT, and µ i is the sense embedding obtained from WSD or learned by the NMT.


Table 5 displays the performance of both sense-aware phrase-based and neural MT systems with the training sets of 500K lines listed in Table 1 on five language pairs.

we compare several approaches that integrate word sense information in SMT and NMT.


For NMT, the best-performing method used the output of k-means to initialize the sense embeddings that are learned by our system.

it appeared that learning sense embeddings for NMT is better than using embeddings learned separately by other methods, although such embeddings may be useful for initialization.


Given an input phoneme string x (i) , we then must find the corresponding y ∈ Σ * that minimizes where threshold is a step function that maps all non-positive reals to −1 and all positive reals to 1.

we seek the phoneme string y that shares the most features with the maximum a posteriori decoded binary vector.


Within the context of rule-based systems, several proposals focus on the question of rule generalization, rather than rule synthesis.

given a set of predefined rules, the systems implement metrics to decide whether rules should generalize to novel forms, depending on the number of exceptions in the data set.


With enough good synthetic languages to use for training, even nearest-neighbor could be an effective method.

one could obtain the parser for a test corpus simply by copying the trained parser for the most similar training corpus (under some metric).


We will choose our universal parameter values by minimizing an estimate of their expected loss, where L train is a collection of training languages (ideally drawn IID from the distribution D of possible human languages) for which some syntactic information is available.

each training language has a treebank (x ( ) , y ( ) ), where x ( ) is a collection of POS-tagged sentences whose correct dependency trees are given by y ( ) .


We selected the hyperparameters that maximized the average unlabeled attachment score (UAS) (Kübler et al., 2009), which is the evaluation metric that is reported by most previ-11 However, as we are interested in transfer to unseen languages, our Table 2 follows the principle of Eisner and Wang (n.d.) and does not test on the Finnishftb or Latin treebanks because other treebanks of those languages appeared in training data.

latinitt and latinproiel fall in the same training folds as French and Italian, respectively.


T D is the directionalities typology that was studied by Liu (2010) and used as a training target by Wang and Eisner (2017).

t D ∈ [0, 1] 57 is a vector of the directionalities of each type of dependency relation; it specifies what fraction of direct objects fall to the right of the verb, and so on.


This resembles the previous multi-language learning approaches (Naseem et al., 2012;Täckström et al., 2013b;Zhang and Barzilay, 2015;Ammar et al., 2016) that exploited the WALS features.

we use 81A, 82A, 83A, 85A, 86A, 87A, 88A and 89A-a union of WALS features used by those works.


We also notice large differences between labeled and unlabeled F1 scores for some relations, especially rarer ones.

the system mislabels the arcs that it correctly recov-ers.


Experiments on sentence modeling with zero-context (sentiment analysis), singlecontext (textual entailment) and multiplecontext (claim verification) demonstrate the effectiveness of ATTCONV in sentence representation learning with the incorporation of context.

attentive convolution outperforms attentive pooling and is a strong competitor to popular attentive RNNs.


(ii) For extra-context, word new is generated to represent word, together with its cross-text alignment att context , in the context left context and right context .

the decision for the word is made based on the connected  hidden states of cross-text aligned terms, with local context.


these two systems work on two input sentences, each with a set of matching scores Figure 3: Attentive pooling, summarized from ABCNN (Yin et al., 2016) and APCNN (dos Santos et al., 2016).


Even if matching scores could convey the phrase-level entailment degree to some extent, matching weights, in fact, are not leveraged to make the entailment decision directly; instead, they are used to weight the sum of the output hidden states of a convolution as the global sentence representation.

fine-grained entailment degrees are likely to be lost in the summation of many vectors.


The attentive context vector then accumulates all related parts together for a given position.

our intra-context attentive convolution is able to connect all related spans together to form a comprehensive decision.


It specifies the extent of compatibility for a single predicate between different typed entailment graphs, with 1 being completely compatible and 0 being irrelevant.

β p, (t 1 , t 2 ), (t 1 , t 2 ) determines how much we expect the outgoing edges of p(:t 1 , :t 2 ) and p(:t 1 , :t 2 ) to be similar.


Learner error patterns (or error statistics) are extracted from the annotated learner data.

given an error type, we collect all source/label pairs from the annotated sample, where both the source and the label belong to the confusion set, and generate a confusion matrix, where each cell represents P rob(source=s|label=l).


To leverage knowledge within an AMR graph, we adopt a graph recurrent network (GRN)  as the AMR encoder.

a full AMR graph is considered as a single state, with nodes in the graph being its substates.


A recurrent neural network is used to model the state transition process.

the transition from g t−1 to g t consists of a hidden state transition for each node (such as from a j t−1 to a j t ), as shown in Figure 3.


To incorporate additional knowledge from an AMR graph, an external attention model is adopted over the baseline model.

the attention memory from the AMR graph is the last graph state In addition, the contextual vector based on the graph state is calculated as: Finally,ζ m is incorporated into the calculation of the output probability distribution over the target vocabulary (previously defined in Equation 1): Given a set of training instances {(X (1) , Y (1) ), (X (2) , Y (2) ), .


To deal with rare and compound words, byte-pair encoding (BPE) 5 (Sennrich et al., 2016) is applied to both sides.

8,000 and 16,000 BPE merges are used on the News Commentary v11 subset and the full training set, respectively.


We also compare with Transformer (Vaswani et al., 2017) and OpenNMT (Klein et al., 2017), trained on the same dataset and with the same set of hyperparameters as our systems.

we compare with Transformer-tf, one popular implementation 7 of Transformer based on TensorFlow, and we choose OpenNMT-tf, an official release 8 of OpenNMT implemented with TensorFlow.


In terms of BLEU score, Dual2seq is significantly better than Seq2seq in both settings, which shows the effectiveness of incorporating AMR information.

the improvement is much larger under the small-scale setting (+3.2 BLEU) than that under the large-scale setting (+1.7 BLEU).


Influence of AMR Parsing Accuracy To analyze the influence of AMR parsing on our model performance, we further evaluate on a test set where the gold AMRs for the English side are available.

we choose The Little Prince corpus, which contains 1,562 sentences with gold AMR annotations.


We showed that AMRs can improve neural machine translation.

the structural semantic information from AMRs can be complementary to the source textual input by introducing a higher level of information abstraction.


In this work we empirically confirm this hypothesis for Modern Hebrew, an MRL with complex morphology and severe wordlevel ambiguity, in a novel transition-based framework.

we propose a joint morphosyntactic transition-based framework which formally unifies two distinct transition systems, morphological and syntactic, into a single transition-based system with joint training and joint inference.


NLP research in recent years has shown increasing interest in parsing typologically different languages, as evident, for instance, by the universal dependencies 1 initiative (Nivre et al., 2016).

much attention is drawn to parsing morphologically rich languages (MRLs), which differ significantly from English in their structure and characteristics (Tsarfaty et al., 2010).


In each instantiation, a transition sequence y for x is a sequence of configurations that are obtained by applying transitions t 1 ...t n ∈ T sequentially.

starting with an initial configuration c 0 = c s (x), we find y = c 0 , ..., c n such that c i+1 = t i+1 (c i ) and c n ∈ C t .


Unless specified otherwise, the set of terminal configurations is There are various options for defining transitions over such configurations in the dependency parsing literature for English.

three transition systems have been successfully applied to English as well as other languages (cf.


To facilitate a fair comparison of Arc Standard to Arc (Z)Eager, we have to adapt the feature set of Zhang and Nivre (2011) to the different arc system (to the extent that this is possible), and to the different language type.

the RNF set depends on word order, by encoding the arc direction explicitly.


In the first, infused scenario, we verify that each lattice contains the gold morphological analysis.

if the gold path is not present in L = MA(x) (hence, an OOV), we automatically infuse the gold path into L. We contrast this with uninfused scenarios, where we use a realistic morphological analyzer with its (incomplete) lexical coverage as is, compliant with Adler and Elhadad (2006).