This baseline however on average predicts 20 discriminative attributes, whereas DAN activates, only 4.

the baseline has a trivial recall advantage.


They have been used for both rule-based (Taboada et al., 2011) and unsupervised (Turney, 2002;Hu and Liu, 2004; or supervised (Mohammad et al., 2013;Tang et al., 2014b;Vo and Zhang, 2015) machine-learning-based sentiment analysis.

constructing sentiment lexicons is one important research task in sentiment analysis.


Each unit aligned at the top of Figure 1 corresponds to a BIO tag.

there are 43 units in the shogi NE case.


The difference in BIO accuracies between them is statistically significant (McNemar's test, p < 0.01).

we can say that our method successfully integrates real world information into text information to build a better solution to the NER problem.


All possible sentences are generated from the templates in the cluster, the resulting sentences are ranked and the best sentence for each relation selected to produce the final description.

the TD approach is a top-down approach, driven to generate sentences expressing the relations found in the company's RDF data using realizations that are typically used on Wikipedia.


To address redundancy, we remove those sentences which were conveyed previously in the description using exactly the same wording.

sentences which are equal to or subsets of other sentences are removed.


To avoid this problem, the exponential decay approach proposed in (Clarkson and Robinson, 1997) has been adopted to express the distance of contextual words from the current word.

the bag-of-words vector with decay weights can be defined as following: where i, k Positions of the current word and words within the BoW model respectively.


5 we can build up hypotheses from left-toright on the target side.

we can represent a partial hypothesis h = (y t 1 , h s ) by a translation prefix y t 1 and an accumulated score h s .


Second, rosé wine is made from red grapes, but the grapes are processed in a different way to red wines.

we expect to find a certain amount of overlap between red and rosé.


One could argue color prediction from wine reviews is trivial where the wine color is actually mentioned in the review.

we also performed an additional experiment with a BoW feature set (with optimized SVM parameters) where the words red, white, and rosé were removed.


Over time if Tumblr provides new signals or if we decide to incorporate new features, CCMC can easily adopt them.

we believe that CCMC is very suitable for solving the Tumblr recommendation task.


While nDCG@k uses the position of correct answer in the top-k ranked list, it does not penalize for unadopted posts or missing adopted posts in the top-k ranked list.

to judge the performance of the algorithms, one has to consider all three metrics together.


Moreover, the remaining unknown target words can now be more successfully replaced with the corresponding source-aligned words.

we obtain a significant improvement in terms of translation quality (up to 3 BLEU points).


Existing event extraction work often heavily relies on a rich set of hand-designed features and utilizes existing NLP toolkits and resources (Ji and Grishman, 2008;Patwardhan and Riloff, 2009;Liao and Grishman, 2010;McClosky et al., 2011;Huang and Riloff, 2012;Li et al., 2013a;Li et al., 2013b;Li et al., 2014).

it is often challenging to adapt prior methods to multi-lingual or non-English settings since they require extensive linguistic knowledge for feature engineering and mature NLP toolkits for extracting the features without severe error propagation.


However, since LCTM assumes that Gaussian variance σ 2 is relatively small, the chance of a word being assigned to distant concepts is negligible.

we can reasonably assume that each word is assigned to one of M ≪ S nearest concepts.


There is an accumulation of evidence that the use of dense distributional lexical representations, known as word embeddings, often supports better performance on a range of NLP tasks (Bengio et al., 2003;Turian et al., 2010;Collobert et al., 2011;Mikolov et al., 2013a;Mikolov et al., 2013b;Levy et al., 2015).

word embeddings have been commonly used in the last few years for lexical similarity tasks and as features in multiple, syntactic and semantic, NLP applications.


For example, the 8-bit precision is 2 −7 = 0.0078, so 10 −5 is not significant enough to update the vector with 8-bit values.

we consider the following two ways to improve this.


The predicted labels are then considered while predicting the labels for next tweets in the test data.

we follow a greedy sequence classification approach.


We selected 650 non-troll comments for the paid trolls, and other 578 for the mentioned trolls as follows: for each paid or mentioned troll comment, we selected a non-troll comment at random from the same thread.

we have two separate non-troll sets of 650 and of 578 comments.


The literature uses the term bi-RNN to refer to two related architectures, which we refer to here as "context bi-RNN" and "sequence bi-RNN".

in a sequence bi-RNN (bi-RNN seq ), the input is a sequence of vectors x 1:n and the output is a concatenation (•) of a forward (f ) and reverse (r) RNN each reading the sequence in a different directions: in a context bi-RNN (bi-RNN ctx ), we get an additional input i indicating a sequence position, and the resulting vectors v i result from concatenating the RNN encodings up to i: the state vector v i in this bi-RNN encodes information at position i and its entire sequential context.


While the MSR data set includes a list of specified training texts, there is no comparable material for SAT questions.

the requisite word-context matrices were constructed by computing token cooccurrence frequencies from the New York Times portion of the English Gigaword corpus (Parker et al., 2009).


However, as far as we know, syntactic complexity as a high level feature of language production has not been investigated under the theoretical lens of the Interactive Alignment Model (Pickering and Garrod, 2004).

the focus of this study is to track the syntactic complexity of different interlocutors as the conversation develops.


On the other hand, the following speaker has to accommodate this change of context, by first producing short acknowledging phrases at the early stage, and gradually increase his contribution as the topic develops.

the convergence of syntactic complexity within a topic episode is a reflection of the process in which two interlocutors contribute jointly to build up common ground (Clark and Brennan, 1991) with respect to a certain topic.


From the perspective of Interactive Alignment, interactions between interlocutors in a dialogue are accompanied by the alignment of linguistic elements at multiple levels, including syntactic rules.

the micro-level convergence of syntactic complexity is predicted by the IAM.


Thus, the micro-level convergence of syntactic complexity is predicted by the IAM.

our findings point to the possibility of a unified perspective that combines the two theories.


In contrast, in cQA, the question and the intended answers might differ significantly both in terms of length and in lexical content.

it is not clear a priori whether the MTE network can work well to address the cQA problem.


Finally, the second line shows that adapting the network to do classification (MTE-CQA classif ication ), giving it a question and a single comment as input, yields a performance drop of 0.6 MAP points absolute compared to the proposed pairwise learning model.

the pairwise training strategy is confirmed to be better for the ranking task, although not by a large margin.


This is a sideeffect of the use of co-occurrence, which relies on syntagmatic rather than paradigmatic similarities, and which is insensitive to word senses.

context-unaware distributional semantics allows too many spurious overlaps.


The idiosyncrasy of collocations makes them also language-specific.

in English, you take [a] walk, in Spanish you 'give' it (dar [un] paseo), and in German and French you 'make' it ([einen] Spaziergang machen, faire [une] promenade); in English, rain is heavy, while in Spanish and German it is 'strong' (fuerte lluvia/starker Regen).


Our goal is to capture the relation that holds between the training bases and the collocates with the same gloss, such that given a new base and a gloss, we can retrieve its corresponding collocate(s) with this gloss.

given absolute certainty, deep thought, and strong wind as training examples, storm as input base and 'intense' as gloss, we aim at retrieving the collocate heavy.


2, we now want to minimize each distance.

the objective is given by: subject to Q being an orthogonal matrix.


Thus, our hypothesis is that antonyms are usually very similar in semantics except that they differ on a single "semantic axis," the polarity axis.

1 The second operation is "neutral version of word v": our hypothesis is that neutral words are words with a value close to zero in the polarity subspace.


POSIT suffers from the same issues with locality as BOW, but its shallow positional annotations may capture additional shallow syntactic phenomena in the data.

pOSIT may be considered a link from BOW towards DEpS.


A common element in all such models is the claim (or conclusion) being forwarded by the argument.

a natural first step in synthesizing arguments in a general setting is being able to synthesize claims in such a setting.


Specifically, we investigate the topic interpretability by examining the "coherence" of a topic generated by topic modeling algorithms, i.e., how close topic words are within a topic.

this measure is a "within-topic" measure.


That is, if the Turkers are confused about which second word to choose, then their answers will be scattered amongst the remaining five words.


Conversely, if the second word is obvious, the Turkers will begin to congregate around that second choice, meaning that their answers will be focused.


Those errors are often not covered by edit trees that are observed in the training data whereas the correct form is.

substituting a form not supported by an observed edit tree with a close one that is supported promises to reduce the error rate.


Edges that go between tagged tokens of a sentence V i represent potential dependency edges.

for the sentence i, the induced subgraph G[V i ] is the (ambiguous) dependency graph.


Only long distance dependencies can help the model recognize tag correctly in this example.

long distance information is an important factor for CWS task.


In general, tree structure works better than chain structure to model long term information.

we use gated recursive neural network (GRNN) (Chen et al., 2015) which is a kind of tree structure to capture long distance dependencies.


On the contrary, small amplification gate will hardly reach the desired effect.

we introduce the threshold mechanism to guarantee the robustness of the algorithm, where gradient which is greater than threshold will not be expanded.


Any reasonable MT system will nowadays generate texts where most words are correct, so the label distribution is very skewed towards the "OK" class.

we suggest instead the multiplication of F 1 -scores for individual classes: it is equal to zero if one of the components is zero, and since both are in the [0,1] range, the overall result will not exceed the value of any of the multipliers.


If several adjacent words are tagged as "BAD", they are likely to be part of an error which spans over a phrase.

we also evaluate word-level F 1scores and alternative metrics which are based on correctly identified erroneous or error-free spans of words.


One of the purposes of system comparison is to identify the best-performing system.

we expect a good metric to be able to distinguish between systems as well as possible.


This means that the system rankings produced by other metrics may be unfairly considered inaccurate.

we suggest a more objective metric evaluation procedure which uses only synthetic datasets.


In the following, we discuss sources of these problems in the data, modeling, and research design, and suggest possible solutions to address them.

of the situatedness of language, any data set carries a demographic bias, i.e., latent information about the demographics in it.


Specific to text classification, the latent features are the hidden states of LSTM at the end of a sentence.

the sharing schemes are different in how to group the shared features.


Accordingly, datasets typically differ with respect to their annotation of (often rather complex) argument structure.

feature sets would have to be manually adapted to and designed for each new sample of data, a challenging task.


Paragraphs have an average length of 66 tokens and are therefore much shorter than essays, which have an average length of 368 tokens.

prediction on the paragraph level is easier than 2 All MCs are considered as equivalent in meaning.


As said, text sequences are much longer on essay level than on paragraph levelhence, there are on average many more entities on essay level.

there are also many more possible relations between all entities discovered in a text-namely, there are O(2 m 2 ) possible relations between m discovered components.


During training, the model learns to represent variables by backpropagating gradients from a time step where a variable is selected by the decoder, through the key-variable memory, to an earlier time step when the key embedding was computed.

the encoder/decoder learns to generate representations for variables such that they can be used at the right time to construct the correct program.


NSM executes non-differentiable operations against a KB, and thus end-to-end backpropagation is not possible.

we base our training procedure on REINFORCE (Williams, 1992;.


The reward r is non-zero only at the last step of decoding, and is the F 1 score computed comparing the gold answer and the answer generated by executing the program a 0:T .

the cumulative reward of a program a 0:T is The agent's decision making procedure at each time is defined by a policy, ⇡ ✓ (s, a) = P ✓ (a t = a|x, a 0:t 1 ), where ✓ are the model parameters.


While REINFORCE assumes a stochastic policy, we use beam search for gradient estimation.

in contrast with common practice of approximating the gradient by sampling from the model, we use the top-k action sequences (programs) in the beam with normalized probabilities.


Then, we run REINFORCE, where we normalize the probabilities of the programs in beam to sum to (1 ↵) and add ↵ to the probability of the best found program C ⇤ (x i ).

the model always puts a reasonable amount of probability on a program with high reward during training.


The output of the transition system is then grounded (e.g., to a knowledge base) with a neural mapping model under the assumption that grounded and ungrounded structures are isomorphic.

2 we obtain a neural model that jointly learns to parse natural language semantics and induce a lexicon that helps grounding.


This can be verified by calculation: The choice W =Ũ and H =Ṽ makes the first term in (9) vanish, and it also minimizes the second term in (9).

it follows that the features (Ũ,Ṽ) constitute a sufficient dimensionality reduction of G.


Our trigger model tends to assign the same subtype to event mentions triggered by the same word.

it often assigns the wrong subtype to triggers that possess different subtypes in different contexts.


That is quite similar to the zero pronoun resolution task.

the automatically generated training samples is called pseudo training data.


Proper emotion expressing can touch the hearts of the readers and improve the expressiveness of writing.

considering it as an independent mode is also reasonable.


We notice that discourse modes can mix in the same sentence.

the annotation standard allows that one sentence can have multiple modes.


Accessing information of past and future sentences provides more contextual information for current prediction.

we use a bidirectional RNN to connect sentences.


It is easy for students to use narration, but it is more difficult to manipulate description and emotion expressing well.

the ability of descriptive and emotional writ- ing should be an indicator of language proficiency and can better distinguish the quality of writing.


Thus, given x we optimize the marginal probability function: where p(y | z) is the Kronecker delta function δ e(z),y , which is 1 if the execution of z, denoted as e(z), generates y and 0 otherwise.

we can redefine p(y|x), the marginal over all programs Z, as a marginal over programs that would generate y, defined as Z(y).


For instance, during the computation of the probabilities for the input copy mechanism in Equation 1, the affinity function f between the current context q and a given input u k is generally implemented by projecting u and q into a single vector followed by a non-linearity, which is projected into a single affinity value.

for each possible input u, 3 matrix multiplications must be performed.


One important drawback is that it builds on artificial rather than "real" data i.e., data that would be extracted from an existing knowledge base.

the training corpora built using this method cannot be used to train KB verbalisers i.e., generation systems that can verbalise KB fragments.


The verbalisations collected for single triples were used to construct input with bigger size.

for input with a number of triples more than one, the crowd was asked to merge the sentences corresponding to each triple (obtained in step 2) into a natural sounding text.


As mentioned in Section 3, in the RNNLG dataset, all inputs can be viewed as trees of depth one while in the WEBNLG dataset, input may have various shapes.

rNNLG texts will be restricted to syntactic forms which permit expressing such multiple predications of the same entity e.g., subject relative clause, VP and sentence coordination etc.


In order to generate natural answers, typical products need lots of Natural Language Processing (NLP) tools and pattern engineering (McTear et al., 2016), which not only suffers from high costs of manual annotations for training data and patterns, but also have low coverage that cannot flexibly deal with variable linguistic phenomena in different domains.

this paper devotes to develop an end-to-end paradigm that generates natural answers without any NLP tools (e.g.


We assume a vocabulary where UNK indicates any out-of-vocabulary (OOV) words.

we have adopted another two set of SUs X Q and X KB which cover words/entities in the source question and the partial KB.


Furthermore, the answer often appears multiple times in the document in spurious contexts (the year '2012' can appear many times while only once in relation to the question).

we treat sentence selection as a latent variable that is trained jointly with the answer generation model from the answer only using reinforcement learning.


If the answer string is missing from the document this often implies a spurious question-answer pair, such as ('what time is half time in rugby', '80 minutes, 40 minutes').

we pruned question-answer pairs without the exact answer string.


In addition, we leverage the global knowledge inside the underlying KB, aiming at integrating the rich KB information into the representation of the answers.

it could alleviates the out-of-vocabulary (OOV) problem, which helps the crossattention model to represent the question more precisely.


However, we argue that simply selecting three independent CNNs is mechanical and inflexible.

we go one step further, and propose a crossattention based neural network to perform KB-QA.


The questions and KB triples were represented by vectors in a low dimensional space.

the cosine similarity could be used to find the most possible answer.


As opposed to mutually exclusive labels used in object recognition, object names are more flexible, subject to communicative preferences and semantically related to each other.

we investigate models of referential word meaning that link visual to lexical information which we assume to be given through distributional word embeddings.


(2014), referential meaning can be represented as a translation function that projects visual representations of objects to linguistic representations of words in a distributional vector space.

in contrast to standard object recognition systems or the other models we will use here, cross-modal mapping does not treat words as individual labels or classifiers, but learns to directly predict continuous representations of words in a vector space, such as the space defined by the word2vec embeddings that we use in this work.


2016's WAC model, building the training set for each word w as follows: all visual objects in a corpus that have been referred to as w are used as positive instances, the remaining objects as negative instances.

the classifiers learn to predict referential appropriateness for individual words based on the visual features of the objects they refer to, in isolation of other words.


When building the training set for such a word predictor w, instead of simply dividing objects into w and ¬w instances, we label each object with a real-valued similarity obtained from cosine similarity between w and v in a distributional vector space, where v is the word that was used to refer to the object.

we task the model with jointly learning similarities and referential appropriateness, by training it with Ridge regression on a continuous output space.


However, there is an interesting pattern when comparing accuracies @1 and @2 to accuracies in the top 5 predictions.

looking at accuracies for the top (two) predictions, the various models that link referential meaning to word representations in the distributional space all perform slightly worse than the plain WAC model, i.e.


From each pair, we randomly include the singular or plural noun in the set of zero-shot nouns.

we make sure that the model encounters singular and plural names during training, but it never encounters both variants of a name.


Importantly, four of five dimensions of knowledge in our study-weight, strength, rigidness, and speed-are either not visual or not easily recognizable by image recognition using currently available computer vision techniques.

our work provides unique value to complement recent attempts to acquire commonsense knowledge from web images (Izadinia et al., 2015;Bagherinezhad et al., 2016;.


This is generally not true, since a verb, especially a common action verb, can often invoke a number of different frames according to frame semantics (Fillmore, 1976).

given a number of different frame relations v 1 ...v T associated with a verb v, we define random variables F with respect to a specific frame relation v t , i.e., F a vt .


While existing frame annotations such as FrameNet (Baker et al., 1998), PropBank (Palmer et al., 2005), and VerbNet (Kipper et al., 2000) provide rich frame knowledge associated with a predicate, none of them provide the exact kind of physical implications we consider in our paper.

our work can potentially contribute to these resources by investigating new approaches to automatically recover richer frame knowledge from language.


|U(c, t G )| + n pc (A ∪ I(c, t G )), where "pc" stands for problematic cycles, is a closer upper bound to the loss (c) and the following holds: As mentioned before, unlike the monotonic approach, a node can have a different head in A than in I(c, t G ) and, as a consequence, the resulting graph A ∪ I(c, t G ) has maximum in-degree 2 rather than 1, and there can be overlapping cycles.

the computation of the non-monotonic terms n c (A ∪ I(c, t G )) and n pc (A ∪ I(c, t G )) requires an algorithm such as the one by Johnson (1975) to find all elementary cycles in a directed graph.


Regarding the upper bound |U(c, t it presents a more variable relative error, ranging from 0.1% to 4.0%.

although we do not know an algorithm to obtain the exact loss which is fast enough to be practical, any of the three studied loss bounds can be used to obtain a feasible approximate dynamic oracle with full non-monotonicity.


Since W is symmetric and positive definite W = U SU .

then U and the Equation 2 can be rewritten as Given an input example o ∈ D, a new lowdimensional representationx can be thus determined by considering the corresponding item of C asx where c is the vector whose dimensions contain the evaluations of the kernel function between o and each landmark o j ∈ L. the method produces l-dimensional vectors.


When the features show themselves finally, some major damages might have already been done.

it is important to design algorithms that can detect review spam as soon as possible, ideally, right after they are posted by the new reviewers.


(2013c) have proved that the linguistic features are not effective enough in detecting real-life fake reviews from the commercial websites, where we also obtain the same observation (the details are shown in Section 3).

the main difficulty of the cold-start spam problem is that there are no sufficient behaviors of the new reviewers for constructing effective behavioral features.


Besides, there is no explicit charac-teristic tag available in the review system, and we need to find a way to record the reviewers' latent characters information in footprints.

we encode these behavioral information into our model by utilizing an embedding learning model which is similar with TransE (Bordes et al., 2013).


The transE part of our model has effectively recorded the behavioral information of the review graph.

our model is more effective by jointly embedding the textual and behavioral informations, it helps to augment the possible behavioral information of the new reviewer.


Our objective is to make the reconstructed embedding r s similar to the target sentence embedding z s while different from those negative samples.

the unregularized objective J is formulated as a hinge loss that maximize the inner product between r s and z s and simultaneously minimize the inner product between r s and the negative samples: where D represents the training data set and θ = {E, T, M, W, b} represents the model parameters.


U reaches its minimum value when the dot product between any two different aspect embeddings is zero.

the regularization term encourages orthogonality among the rows of the aspect embedding matrix T and penalizes redundancy between different aspect vectors.


In addition, it is by nature impossible to predict whether a new (i.e., nonexistent in the data) user agrees or disagrees with given topics.

in this section, we apply matrix factorization (Koren et al., 2009) in order to predict missing values, inspired by research regarding item recommendation (Bell and Koren, 2007;Dror et al., 2011).


Only when a user u expresses a preference for topic t do we compute an element of the sparse matrix r u,t , Here, #(u, t, +1) and #(u, t, −1) represent the numbers of occurrences of instances (u, t, +1) and (u, t, −1), respectively.

an element r u,t approaches +1 as the user u favors the topic t, and −1 otherwise.


In contrast, this study is the first attempt to recognize and organize various axes of topics on social media with no prior assumptions regarding the axes.

we think our study provides a new tool for computational social science and political science that enables researchers to analyze and interpret phenomena on social media.


It is possible to generalize our work to model heterogeneous signals, such as interests and behaviors of people, for example, "those who are interested in A also support B," and "those who favor A also vote for B".

we believe that our work will bring about new applications in the field of NLP and other disciplines.


For example, in Freebase, the aforementioned marriage event instance is represented as m.02nqglv (see details in Section 2).

we cannot directly use an event instance and an argument, like m.02nqglv and Barack Obama, to label back in sentences.


Intuitively, if a verb occurs more times than other verbs in the labeled sentences of one event type , the verb tends to trigger this type of event; and if a verb occurs in sentences of every event types, like is, the verb will have a low probability to trigger events.

we propose Trigger Candidate Frequency (TCF) and Trigger Event Type Frequency (TETF) to evaluate above two aspects.


Because the number of nouns in one sentence is usually larger than that of verbs, it is hard to use TR to find nominal triggers.

we propose to use linguistic resource FrameNet to filter noisy verbal triggers and expand nominal triggers.


And the objective of multi-instance learning is to discriminate bags rather than instances.

we define the objective function on the bags.


However, these sentences miss labeling triggers.

we leverage these rough labeled data and FrameNet to find triggers and use SDS to generate labeled data.


The reason is that the heuristics for data labeling are stricter as k grows.


This suggests that time tokens account for time expressions.

to recognize time expressions, it is essential to recognize their time tokens.


The problem is at the beginning of the training, we have no prior knowledge about the noise pattern, thus, both T and p are less reliable, making the training procedure be likely to trap into some poor local optimum.

we require a technique to guide our model to gradually adapt to the noisy training data, e.g., learning something simple first, and then trying to deal with noises.


The at-least-one assumption is often too strong in practice, and there are still chances that the sentence bag may be false positive or false negative.

it is important to model the noise pattern to guide the learning procedure.


Each action step in the grammar model naturally grounds to a time step in the decoder RNN.


Both datasets provide gold predicates (their index in the sentence) as part of the input.

each provided predicate corresponds to one training/test tag sequence.


Only the BIO hard constraints significantly improve over the ensemble model.

in our final results, we only use BIO hard constraints during decoding.


A known challenge for syntactic parsers is robustness on out-of-domain data.

we provide experimental results in Table 8 for  On the CoNLL 2005 development set, the predicted syntax gives a 0.5 F1 improvement over our best model, while on in-domain test and outof-domain development sets, the improvement is much smaller.


The reason that existing models lose important information in the context is that they first represent the whole context as a vector and then match the context vector with a response vector.

responses in these models connect with the context until the final step in matching.


Finally, inference/decoding has to be performed by integrating all modules resulting in complex decoding.

it is quite difficult for non-experts to use/develop ASR systems for new applications, especially for new languages.


Also, to train the framewise posterior p(s t |x t ), we have to provide a framewise state alignment s t as a target, which is often provided by a GMM/HMM system.

conventional DNN/HMM systems make the ASR problem formulated with Eq.


Note that the CTC formulation uses L-length letter sequence C = {c l ∈ U|l = 1, • • • , L} with a set of distinct letters U.

similarly to section 2.1, by introducing framewise letter sequence with an additional "blank" and by using the probabilistic chain rule and conditional independence assumption, the posterior distribution p(C|X) is factorized as follows: CTC has three distribution components similar to the DNN/HMM case, i.e., framewise posterior distribution p(z t |X), transition probability p(z t |z t−1 , C) 1 , and prior distributions of letter and hidden-state sequences, p(C) and p(Z), respectively.


In addition, the proposed method does not require GMM/HMM construction for initial alignments, DNN pre-training, lattice generation for sequence discriminative training, complex search in decoding (e.g., FST decoder or lexical tree search based decoder).

the method greatly simplifies the ASR building process, reducing code size and complexity.


Cognitively speaking, all translations, regardless of the source and target language, are susceptible to the same constraints.

translation products are expected to share similar artifacts.


The powerful signal of interference is evident, e.g., by the finding that a classifier trained to distinguish between originals and translations from one language, exhibits lower accuracy when tested on translations from another language, and this accuracy deteriorates proportionally to the distance between the source and target languages (Koppel and Ordan, 2011).

it is possible to accurately distinguish among translations from various source languages (van Halteren, 2008).


Furthermore, the actual subgroupings of languages within this family are not clear-cut (Ringe et al., 2002).

algorithms that attempt to reconstruct the IE languages tree face a serious evaluation challenge (Ringe et al., 2002;Nakhleh et al., 2005a).


The search space is limited to the rules that include w in their support set, a fairly small search space and the corresponding computation readily tractable: s. t. r P R add score r sem prq ą t r sem score r orth prq ą t r orth score w sem ppw 1 , wq P SS r q ą t w sem score loc sem ppw 1 , wq P SS r q ą t loc sem Where t 1 = {w sem, loc sem}, t 2 = {r sem, r orth}, and t r sem , t r orth , t w sem , t loc sem are hyperparameters of the system.

now given r˚= φ suffix Ý ÝÝ Ñ suf, w 1 is defined as w 1 rÝ Ñ w. the algorithm segments w Ñ w 1 -suf.


In addition, downsampling with stride 2 essentially doubles the effective coverage (i.e., coverage in the original document) of the convolution kernel; therefore, after going through downsampling L times, associations among words within a distance in the order of 2 L can be represented.

deep pyramid CNN is computationally efficient for representing long-range associations and so more global information.


Yet different words might match phrases of different lengths.

we hope the question representations could also comprise vectors that summarize various lengths of phrase information (different levels of abstraction), in order to match relation representations of different granularity.


For the KBQA evaluation, we also start with their entity linking results 6 .

our results can be compared with their reported results on both tasks.


High-quality keyphrases can facilitate the understanding, organizing, and accessing of document content.

many studies have focused on ways of automatically extracting keyphrases from textual content (Liu et al., 2009;Medelyan et al., 2009a;.


Furthermore, we incorporate a copying mechanism (Gu et al., 2016) to allow our model to find important parts based on positional information.

our model can generate keyphrases based on an understanding of the text, regardless of the presence or absence of keyphrases in the text; at the same time, it does not lose important in-text information.


In fact, there are millions of scientific papers available online, each of which contains the keyphrases that were assigned by their authors.

we collected a large amount of high-quality scientific metadata in the computer science domain from various online digital libraries, including ACM Digital Library, Sci-enceDirect, Wiley, and Web of Science etc.


In a broader sense, this model attempts to map a long text (i.e., paper abstract) with representative short text chunks (i.e., keyphrases), which can potentially be applied to improve information retrieval performance by generating high-quality index terms, as well as assisting user browsing by summarizing long documents into short, readable phrases.

far, we have tested our model with scientific publications and news articles, and have demonstrated that our model has the ability to capture universal language patterns and extract key information from unfamiliar texts.


(2015), answering verbs and prepositions are less dependent with the presence of document.

most of the related works are focusing on solving NE and CN types.


The nature of the signal (whether it is language-related or not) should not be the first concern from the perspective of methodology.

studies outside the field of speech communication and linguistics could also be enlightening to our work.


We try several lags on each entropy series, and pick the smallest p-value.

we obtain a mean p-value of .23 on MapTask, and a mean p-value of .27 on DJD.


The new model that uses PSO to predict the Box-Cox transformed PATHDEV also yields significant coefficients: β = 3.85, SE = 1.67, F (1, 113) = 5.32, p < .05.

the correlation between PSO and PATHDEV is reliable.


The frequency components with higher energy (peaks in spectrum) are more dominant than those with lower energy (troughs) in determining the nature of the signal.

it makes sense to only include the peak frequencies into the model, because they are more "representative" of the signal, and so the "noise" from the low energy frequencies are filtered out.


On the contrary, if B rushes to speak a lot shortly after the A's input, then it will probably cause information overload and be harmful to communication.

we believe that the RP statistic captures the extent to which interlocutors manage the proper "timing" of information contribution to maintain effective communication.


During model training, the affect category is inferred from the context data itself.

we define a suitable feature extractor which can utilize an affective lexicon to infer emotion in the context.


(3) Significantly smaller than the Fisher corpus, which is 25 times the size of the other corpora combined.

we perform training in two separate stages -training of the baseline and Affect-LM models on the Fisher corpus, and subsequent adaptation and fine-tuning on each of the emotionally colored corpora.


One typical use case is that of building natural language capability through intent and slot modeling (without actually building a domain classifier) targeting a specific application.

our experimental settings are rather different from previ-ously considered settings for domain adaptation in two aspects: • Multiple source domains: In most previous works, only a pair of domains (source vs. target) have been considered, although they can be easily generalized to K > 2.


This matches our common sense that their are only a few of domains that are close enough to be of help to a test domain.

it is advisable to find the optimal k value through several rounds of experiments on a development dataset.


However, the conventional approach of designing a dialog manager (Williams and Young, 2007) does not scale well to open-domain conversation models because of the vast quantity of possible decisions.

there has been a growing interest in applying encoder-decoder models  for modeling open-domain conversation (Vinyals and Le, 2015;Serban et al., 2016a).


For example, dialog acts (Poesio and Traum, 1998) have been widely used in the dialog managers (Litman and Allen, 1987;Raux et al., 2005;Zhao and Eskenazi, 2016) to represent the propositional function of the system.

we conjecture that it will be beneficial for the model to learn meaningful latent z if it is provided with explicitly extracted discourse features during the training.


The RNN computes a hidden state (vector), which is retained for the next timestep (step 8), and passed to a dense layer with a softmax activation, with output dimension equal to the number of distinct system action templates (step 9).

1 the output of step 9 is a distribution over action templates.


By contrast, unary rules indicate deletions, in that they make "progress" in r s without adding new words to w. We can compute the minimal-cost tree of G using the Viterbi algorithm.

to summarize, we can calculate s * from the intersection of a contextfree grammar G representing the REs to o u with the automaton F (r s ) representing the edit distance to r s .


6 deleted "blue" and substituted "below" with "above".

we mark these words with focus, and obtain the contrastive RE "the BLUE button BELOW the window".


When we utter a contrastive RE, the user has previously heard the RE r s , so some of the information in r s E is redundant.

we might obtain a more succinct, and possibly better, RE by dropping such redundant information from the RE.


Moreover, it requires an additional parsing-task-specific training mechanism to recover the hidden syntax in NMT.

in the absence of explicit linguistic knowledge, the seq2seq model in NMT tends to produce translations that fail to well respect syntax.


Finally, the Mixed RNN encoder ties the structural label sequence and word sequence together in the closest way.

the degrees of coupling the word and structural label sequences in these three encoders are like this: Mixed RNN encoder > Hierarchical RNN encoder > Parallel RNN encoder.


For Y 's parse tree T , instead of directly modeling the tree itself, we predict a parsing action sequence A which can map Y to T .

at top level our SD-NMT model can be formulated as where Two recurrent neural networks, Word-RNN and Action-RNN, are used to model generation processes of translation sequence Y and parsing action sequence A respectively.


As the target dependency trees are known in the bilingual corpus during training, we pre-compute the partial tree state and syntactic context at each time stamp for each training instance.

it is easy for the model to process multiple trees in one batch.


In addition, unlike representation learning for plain text, learning for representation of structures such as networks is not well studied in general.

traditional and latest text-based models (Castillo : Fragments of the propagation for two source tweets.


However, these same dynamics make constructing such tools difficult, as the language used to discuss new events and political agendas continuously changes.

the rich social interactions on Twitter can be leveraged to help support such analysis by providing alternatives to direct supervision.


The impact in the "large changes" portion is slightly positive in precision and slightly negative in recall.

most of the benefit of the additional character-level attention stems from improvements in the "small changes" portion.


English is undoubtedly the most prominent in this regard and English resources are the most numerous.

nLP methods are mostly designed for English and tested on English data, even if there is no guarantee that they are universal.


However, a cursory check shows that these images are quite homogeneous, with a predominant number of dogs depictions.

we independently extract 1K images and split them into 46 thematic groups (e.g.


5 For each sentence pair (a, b) created according to this procedure, its reverse (b, a) is also included in our corpus.

the working set consists of 20K sentence pairs.


Hence, we conclude that Fleiss' kappa is not a reliable measure of inter-annotator agreement in relation to relatedness scores.

we decided to use Krippendorff's α instead.


However, because these metrics were proposed for human readability, they do not necessarily correlate with those used in RC systems.

in any system analysis, ideally we would have to consult a variety of features.


For example, some questions can be solved by external knowledge without even seeing their context.

we should clarify what constitutes a "solvable" or "reasonable" question for RC.


The model must therefore be constrained at training time to follow decisions that exactly reproduce the gold tree, since supervision cannot be provided otherwise.

the model is never exposed to its mistakes, which can lead to a lack of calibration and poor performance at test time.


On the other hand, their model is statistical while ours is neural.

they integrate external knowledge as features, while we integrate it by shared network parameters.


While this idea is simple and intuitive, we found that it alone was not enough to achieve competitive accuracy with real NMT models.

we make two improvements: First, we propose a hybrid model, where the high frequency words are predicted by a standard softmax, and low frequency words are predicted by the proposed binary codes separately.


However, N can be chosen as N V because the softmax prediction is only required for a few frequent words.

we can control the actual computation for the hybrid model to be much smaller than the standard softmax complexity O(HV ), The idea of separated prediction of frequent words and rare words comes from the differentiated softmax (Chen et al., 2016) approach.


The 2 methods proposed in previous sections impose constraints for all bits in q, and the value of each bit must be estimated correctly for the correct word to be chosen.

these models may generate incorrect words due to even a single bit error.


In this case, all bits can be estimated using exactly the same parameters, and we can not expect that we will benefit significantly from applying this redundant representation.

we need to choose an error correction method in which the characteristics of original bits should be distributed in various positions of the resulting bit arrays so that errors in bits are not highly correlated with each-other.


4 In contrast, BLEU scores ac- tually increase when training 2-layer vs. 1-layer models (+1.11/+0.56 BLEU for Arabic-Hebrew word/char-based models).

translation quality improves when adding layers but morphology quality degrades.


These two sources of information need to jointly point to the most relevant source word(s) and predict the next most likely word.

the decoder puts significant emphasis on mapping back to the source sentence, which may come at the expense of obtaining a meaningful representation of the current word.


The vocal modulations and facial expressions in the visual data, along with textual data, provide important cues to better identify affective states of the opinion holder.

a combination of text and video data helps to create a more robust emotion and sentiment analysis model (Poria et al., 2017a).


Every utterance in a video is spoken at a distinct time and in a particular order.

a video can be treated as a sequence of utterances.


Because negative PMI values tend to be unreliable, we use positive PMI (PPMI), which involves replacing all negative PMI values with zeros (Niwa and Nitta, 1994).

we obtain stance dimensions by applying singular value decomposition to the matrix constructed as follows: Truncated singular value decomposition performs the approximate factorization X ≈ U ΣV , where each row of the matrix U is a k-dimensional description of each stance marker, and each row of V is a k-dimensional description of each subreddit.


The memberships of multiple subreddits on the same topic (e.g., r/science and r/askscience) often do not overlap considerably.

we hypothesize that users of Reddit have preferred interactional styles, and that participation in subreddit communities is governed not only by topic interest, but also by these interactional preferences.


For example, accuracy penalizes a classifier just as much for labeling a document from 'rec.sport.baseball' with 'rec.sport.hockey' as with 'alt.atheism' despite the similarity between sports newsgroups.

after building a confusion matrix between the predicted and true classes, external clustering metrics reveal confusion between classes.


Some of the data violate assumptions of normality.

we use Wilcoxon's signed-rank test (Wilcoxon, 1945) to determine if the differences between multiword anchors and single word anchors are significant.


For example, we separately model the relation "ORG-AFF" for each token in the entity "ITV News".

we model the relations between "ITV" and "Martin Geissler", and "News" and "Martin Geissler" separately.


For each token in the sequence, we perform a softmax over all candidate tags to output the most likely tag: Our network structure as shown in Figure 2 also contains connections from the output y t−1 of the previous time step to the current top hidden layer.

our outputs are not conditionally independent from each other.


In our approach, we use pointer networks to identify the related entities.

for our approach described so far if we only compute the argmax on our objective then we limit our model to output only one relation label per token.


PHYS relation is easier identified with respect to GPE entity than PER entity.

our bi-directional encoding of relations allows us to encode these relations with respect to both entities in the relation.


Although scoping can be explicitly controlled via '[ ]', 'isolate', and '{ }', it is an unnatural concept for non-programmers.

when the choice is not explicit, the parser generates all three possible scoping interpretations, and the model learns which is intended based on the user, the rule, and potentially the context.


However, a rule that often generalizes incorrectly should be down-weighted, along with the score of its packings.

a different rule might be induced next time, even with the same definition.


The bottom plot of Figure 6 suggests that users are defining and using higher level concepts, since programs become longer relative to utterances over time.

of the automatic but implicit grammar induction, some concepts do not generalize correctly.


Though sparseness is favorable in practice, it is generally NP-hard to find the optimal solution under 0 constraints.

we resort to an approximated algorithm in this work.


However, the dramatic difference here is that with parameters in the dense partition regarded as constant, the cost function is decoupled w.r.t.

each relation r. In other words, the optimal choice of I H r , I T r is independent of I H r , I T r for any r = r. we only need to consider the optimization for a single relation r, which is essentially an assignment problem.


As we mentioned earlier, the sparse attention vectors fully capture the association between relations and concepts and hence the knowledge transfer among relations.

we visualize the attention vectors for several relations on both WN18 and FB15K in Figure 3.


As we mentioned in the introduction, acquiring labeled training data for discourse relations is a timeconsuming process since it would require human annotators to inspect the full discussions.

we further propose a variation of our model where it treats the discourse relations as latent variables, so that p(c|x, w) = d p(c, d|x, w).


Concretely, for each iteration, we first optimize on d by maximizing Message-passing (Smith and Eisner, 2008) is used to find the best d. In the second step, we search for c that maximizes We believe that candidate phrases based on the same concepts should have the same predicted label.

candidates of the same phrase type and sharing the same head word are grouped into one cluster.


Otherwise, it can be labeled as NEGATIVE.

models that better handle semantics and context need to be considered.


Their annotation covers the AMI-SUB dataset.

we run the prediction experiments on AMI-SUB by using the same annotation.


For instance, the popular Penn Discourse Treebank (PDTB) benchmark data (Prasad et al., 2008) was annotated by first inserting a connective expression (i.e., implicit connective, as shown in the above example) manually, and determining the abstract relation by combining both the implicit connective and contextual semantics.

the huge performance gap between explicit and implicit parsing (namely, 85% vs 50%), as well as the human annotation practice, strongly motivates to incorporate connective information to guide the reasoning process.


In contrast, as shown in Figures 5(b)-(c), the feature vectors are increasingly mixed as training proceeds.

our framework has suc- Relation networks a-CNN and i-CNN are measured with multi-class classification accuracy (with or without implicit connectives, respectively), while the discriminator is evaluated with binary classification accuracy.


; Yu and Joachims, are enough for optimizing MELA on the training set, at least when the data is separable.

in such conditions, we cannot expect a very large improvement from RL.


If the data is linearly separable the percep- where l(•) is an edge loss function.

the latter equation and monotonicity imply l(y i , h * i , e) = 0, ∀e ∈ĥ i , i.e., there are no edge mistakes, otherwise by fixing such edges, we would have a smaller ∆, i.e., negative, contradicting the initial positiveness hypothesis.


Additionally, for each edge mistake more, both loss functions increase, implying monotonicity.

they satisfy all the assumptions of Proposition 1.


However, the features we used for this are not factorizable over the edges of the latent trees.

we design a new LSP algorithm that can use our learned loss in an approximated max search.


2 cannot be longer applied as our new loss function is nonfactorizable.


Finally, another important issue is connected to the fact that on the way as we incrementally construct a max-violating tree according to Alg.

3, ∆ ρ decreases (and MELA grows), as we add more mentions to the output, traversing the tree nodes v. to equalize the contribution of the loss among the candidate edges of different nodes, we also scale the loss of the candidate edges of the node v having order i in the document, according to the formula ∆ ρ = i |V | ∆ ρ .


Practically, this means that lexicon entries (words or phrases) may be treated as additional observations.

these entries provide information about how names are spelled.


A simple choice for H y is a Markov process that emits characters in Σ ∪ {$}, where $ is a distinguished stop symbol that indicates the end of the string.

the probability of producing $ controls the typical string length under H y .


Most languages do not have corpora annotated for parts-of-speech, named-entities, syntactic parses, or other linguistic annotations.

rapidly deploying natural language technologies in a new language may be challenging.


The LORELEI language packs lack adequate indomain lexical resources for our purposes.

we simulate in-domain lexical resources by holding out portions of the annotated development data and deriving dictionaries and name lists from them.


Our goal is to learn a semantic parser that maps natural language utterances into executable programs when only indirect supervision is available: examples are labeled with the correct execution result, but not the program itself.

we must search the space of programs for those that output the correct result, while not being misled by spurious programs: incorrect programs that coincidentally output the correct result.


In contrast, RANDOMER spreads probability mass over multiple rewardearning programs, including the correct ones.

of overfitting, we observed at test time that BS-MML only references people by positional indices instead of by shirt or hat color, whereas RANDOMER successfully learns to use multiple reference strategies.


Model Note that we simply denote π(D, S) by π(S) as it is not ambiguous which document collection is used when S is a summary of D. In order to be able to use an exact and efficient solver like ILP, we constrain π to be a linear function.

we look for π of the following form: Two functions are jointly learned: f θ is a function scoring individual sentences, and g γ is a function scoring the intersection of sentences.


This analysis confirms the initial claim that ROUGE and AP behave quite differently and measure different aspects of summary quality.

we believe systems developed and trained for AP are worth studying because they necessarily capture different aspects of summarization.


Our goal is to maximize the output summary probability given the input sentence.

we optimize the negative log-likelihood loss function: where D denotes a set of parallel sentencesummary pairs and θ is the model parameter.


2 We also find that except for the empty titles, this test set has some invalid lines like the input sentence containing only one word.

we further sample 2000 pairs as our internal test set and release it for future works 3 .


We evaluate our model using the ROUGE recall score since the reference summaries of the DUC 2004 test set are capped at 75 bytes.

we decode the summary to a fixed length 18 to ensure that the generated summary satisfies the minimum length requirement.


While human evaluation should always be used to evaluate dialogue models, it is often too expensive and time-consuming to do this for every model specification (for example, for every combination of model hyperparameters).

having an accurate model that can evaluate dialogue response quality automatically -what could be considered an automatic Turing test -is critical in the quest for building human-like dialogue agents.


However, we found the ADEM model learned more effectively when this embedding size was reduced.

after training VHRED, we use principal component analysis (PCA) (Pearson, 1901) to reduce the dimensionality of the context, model response, and reference response embeddings to n. We found experimentally that n = 50 provided the best performance.


When ADEM is used in practice, it will take as input responses from a new model that it has not seen during training.

it is crucial that ADEM correlates with human judgements for new models.


As a result, the lack of interpretability makes it very difficult to understand translation process and debug NMT systems.

it is important to develop new methods for visualizing and understanding NMT.


For example, the context word set for As both hidden states and contextual words are represented as real-valued vectors, we need to factorize vector-level relevance at the neuron level.

the neuron-level relevance between the m-th neuron in a hidden state v m ∈ R and the n-th neuron in a contextual word vector u n ∈ R is denoted as r un←vm ∈ R, which satisfies the following constraint: the vector-level relevance between a hidden state v and one contextual word vector u ∈ C(v) is denoted as R u←v ∈ R, which quantifies the contribution of u to the generation of v. It is calculated as Definition 4 the relevance vector of a hidden state v is a sequence of vector-level relevance of its contextual words: our goal is to compute relevance vectors for hidden states in a neural network, as shown in Figure 2.


For matrix multiplication such as v = Wu, its basic form that is calculated at the neuron level is given by Figure 4: Visualizing source hidden states for a source content word "nian" (years).

for element-wise multiplication such as We use the following method to calculate its weight ratio: for maximization such as v = max{u 1 , u 2 }, we calculate its weight ratio as follows: the general local redistribution rule for LRP is given by Algorithm 1 gives the layer-wise relevance propagation algorithm for neural machine translation.


After model training and selection on the training and development sets, we use the resulting NMT model to translate the test set.

the visualization examples in the following subsections are taken from the test set.


More and more researchers are working with social media text, historical data, or spoken language transcripts, to name but a few.

the need for NLP tools that are able to process this data has become more and more apparent, and has triggered a lot of work on domain adaptation and on developing more robust preprocessing tools.


In our model we use the scores f j to compute the attention.

h j should be considered in the graph model.


Given a topic T , the topic-sensitive PageRank is similar to Eq.

3 except that y becomes: y T is always a one hot vector and only y 0 = 1, indicating the 0-th sentence is s j .


2 In this paper, we take V to be the set of all IPA symbols corresponding to vowels.

a draw from a point process is a vowel inventory V ⊆ V, and the point process itself is a distribution over such inventories.


Since a BPP predicts each vowel independently, it only models focalization.

the model provides an appropriate baseline that will let us measure the importance of the dispersion principle-how far can we get with just focalization?


, e N : are the lengths of vectors e i , e j while θ is the angle between them.

we should choose the columns of E so that focal vowels get long vectors and similar vowels get vectors of similar direction.


Rather, each language is drawn conditionally on its parent language.

one should estimate a stochastic model of the evolution of linguistic systems through time, and identify "naturalness" with BPP MPP DPP changes from n − 1 changes from n − 1 changes from n − 1 n MAP inventory additions deletions MAP inventory additions deletions MAP inventory additions deletions a a  i, u, a  a  i, u, a  a  4  i, u, a, o o  i, u, a, e  e  i, u, a, o  o  5 i, u, a, o, e e i, u, a, e, @ @ i, u, a, o, @ o Table 2: Highest-probability inventory of each size according to our three models (prototype-based embeddings and d = 3).


Inspired by the work on domain adaptation (Ajakan et al., 2014;Ganin et al., 2016;Bousmalis et al., 2016), we hope that the features extracted by shared layer is invariant across the heterogonous segmentation criteria.

we jointly optimize the shared Figure 3: Architecture of Model-III with adversarial training strategy for shared layer.


layer via adversarial training (Goodfellow et al., 2014).

besides the task loss for CWS, we additionally introduce an adversarial loss to prevent criterion-specific feature from creeping into shared space as shown in Figure 3.


An adversarial loss aims to produce shared features, such that a criterion discriminator cannot reliably predict the criterion by using these shared features.

we maximize the entropy of predicted criterion distribution when training shared parameters.


Formal documents (like the eight datasets in Table  2) and micro-blog texts are dissimilar in many aspects.

we further investigate that if the formal texts could help to improve the performance of micro-blog texts.


Once words have been wronglysegmented, word embeddings and traditional onehot word features, used in dependency parsers, will mistake the precise meanings of the original sentences.

pipeline models achieve dependency scores of around 80% for Chinese.


In this paper, we focus on the extraction of triplets that are composed of two entities and one relation between these two entities.

we can model the triplets directly, rather than extracting the entities and relations separately.


LSTM is capable of learning long-term dependencies, which is beneficial to sequence modeling tasks.

based on our tagging scheme, we investigate different kinds of LSTM-based end-to-end models to jointly extract the entities and relations.


In reality, however, the process of manually labeling a training set with a large number of entity and relation is too expensive and error-prone.

we conduct experiments on a public dataset 1 which is produced by distant supervision method (Ren et al., 2017) to validate our approach.


The methods based on neural networks can well fit the data.

they can learn the common features of the training set well and may lead to the lower expansibility.


They only obtain E1 and do not find its corresponding E2, or obtain E2 and do not find its corresponding E1.

it leads to the prediction of more single E and less (E1, E2) pairs.


Thus it leads to the prediction of more single E and less (E1, E2) pairs.

entity pair (E1, E2) has higher precision and lower recall than single E. Besides, the predicted results of (E1, E2) in Table 2 have about 3% improvement when compared predicted results in Table 1, which means that 3% of the test data is predicted to be wrong because the relation type is predicted to be wrong.


When compared with LSTM-LSTM, LSTM-LSTM-Bias uses a bias objective function which enhance the relevance between entities.

in this example, LSTM-LSTM-Bias can extract two related entities, while LSTM-LSTM can only extract one entity of "Florida" and can not detect entity "Panama City Beach".


Due to this, the different entailed hypotheses of the same premise land up in different splits of the dataset (e.g., one in train and one in test/validation) in many cases.

we regroup the premiseentailment pairs and modify the split as follows: among the 190, 113 premise-entailment pairs subset of the SNLI corpus, there are 155, 898 unique premises; out of which 145, 822 have only one hy-pothesis and we make this the training set, and the rest of them (10, 076) have more than one hypothesis, which we randomly shuffle and divide equally into test and validation sets, so that each of these two sets has approximately the same distribution of the number of reference hypotheses per premise.


NLP tools with high precision are needed to compute these metrics, which is a problem for Portuguese since no robust dependency or constituency parsers exist.

the transcriptions had to be manually revised; they were segmented into sentences, following a semantic-structural criterion and capitalization was applied.


For instance, minimizing log likelihood can lead to a model accurately predicting the opposite domain, compromising the goal of inducing domain-invariant representations.

we propose to instead optimize the shared parameters for random domain predictions.


This can limit the model's capacity to learn task-specific features.

models that can be trained in an end-toend fashion on different target tasks are desirable.


Since there are not many training instances, our neural models may not learn well for this task.

we also present versions of our model, where we use pre-trained models from discrimination task on WSJ corpus (last two rows in the table ).


In short, constructing natural questions of reasonable difficulty would seem to require an abstractive approach that can produce fluent phrasings that do not exactly match the text from which they were drawn.

and in contrast to all previous work, we propose here to frame the task of question generation as a sequence-to-sequence learning problem that directly maps a sentence from a text passage to a question.


Although most of the pairs have over 50% overlap rate, about 6.67% of the pairs have no nonstop-words in common, and this is mostly because of the answer offset error introduced during annotation.

we prune the training set based on the constraint: the sentence-question pair must have at least one non-stop-word in common.


Although these approaches achieve state-of-the-art performance, they produce only one globally optimal summary which is impractical for various users due to the subjectivity of the task.

we research interactive computer-assisted approaches in order to produce personalized summaries.


, τ , the querying algorithm selects an instance ofx t ∈ C u for which the learning algorithm is least certain.

our learning goal of active learning is to minimize the expected loss L (i.e., hinge loss) with limited querying opportunities to obtain a decision function f (1) , f (2) , .


This is primarily because DBS is a corpus of cohesive extracts, whereas DUC'04 consists of abstractive summaries.

the oracles created using abstractive reference summaries have lower overlap of concepts as compared to that of the oracles created using extractive summaries.


More importantly, besides such strict regulations, a good quatrain should also read fluently, hold a consistent theme, and express a unique affection.

poem generation is widely recognized as a very intelligent activity and can be performed only by knowledgeable people with a lot of training.


Our task is similar to these tasks in that the system takes sequential data and generates text.

it is natural to use an encoder-decoder model in modeling stock prices.


Market comments often mention numerical values that appear in the input data, but they also mention values obtained through arithmetic operations, such as differences in prices as in (3) and (6) in Figure 1, or rounded values as in (5).

another problem arises: what type of operation is suitable for text to be generated?


closing price of the previous day", 227 in this example indicates the difference between the closing price of the previous trading day x long, M−1 and the latest price x short, N −1 denoted by z in Table 1.


(2), we can see that the optimal values of F i and F j with respect to (x i , y i ) and (x j , y j ) are irrelevant if y i = y j .

the original optimization problem can be decomposed and solved separately according to different candidate hypernyms.


By utilizing the transductive learning framework, we boost the F-measure by 1.7% and 2.1%, respectively.

our method is effective to predict hypernyms of Chinese entities.


Event triggers are mostly verbs and eventive nouns but can occasionally be adjectives and other content words.

the task is typically addressed as a classification problem where the goal is to label each word in a sentence with an event type or an O type if it does not express any of the defined events.


If the number of course concepts is n, the number of all possible pairs to be checked could reach n × (n − 1)/2, which requires arduous human labeling work.

for each dataset, we randomly select 25 percent of all possible pairs for evaluation.


Instead of a simple descriptive feature, each of our proposed feature determines whether a concept pair has prerequisite relation from a specific aspect; its function is similar to an independent weak classifier.

rather than using a linear combination of features for classification (e.g., SVM and LR), a boosting model (e.g., Random Forest) is more suitable for this task.


It is difficult to transform prior knowledge represented in discrete forms to continuous representations required by neural networks.

a number of authors have endeavored to integrate prior knowledge into NMT in recent years, either by modifying model architectures Cohn et al., 2016;Tang et al., 2016;Feng et al., 2016) or by modifying training objectives (Cohn et al., 2016;Feng et al., 2016;.


It is difficult to achieve this end by directly modifying model architectures because neural networks usually impose strong independence assumptions between hidden states.

extending a neural model requires that the interdependence of information sources be modeled explicitly Tang et al., 2016), making it hard to extend.


Although modifying training objectives to include additional knowledge sources as additive terms can partially alleviate this problem, these terms have been restricted to a limited number of simple constraints Cohn et al., 2016;Feng et al., 2016) and incapable of combining arbitrary knowledge sources.

it is important to develop a new framework for integrating arbitrary prior knowledge sources into NMT.


Phrases, which are sequences of consecutive words, are capable of memorizing local context to deal with word ordering within phrases and translation of short idioms, word insertions or deletions (Koehn et al., 2003;Chiang, 2005).

phrase tables that specify phrase-level correspondences between the source and target languages also prove to be an effective knowledge source in NMT (Tang et al., 2016).


To alleviate this problem, we follow  to approximate the full search space Y(x (n) ) with a sampled sub-space S(x (n) ).

the KL divergence can be approximated as KL Q(y|x (n) ; γ) P (y|x (n)  .


For phrase table, we first remove phrase pairs that occur less than 10 times and then retain entries with probabilities higher than 0.5 in both directions.

both bilingual dictionary and phrase table contain highquality translation correspondences.


On the other hand, an exhaustive exploration of the output space would require scoring |v| T sequences, which is intractable for most real-world models.

a search or decoding algorithm is often used as a compromise between these two extremes.


The numC parameter in Algorithm 1 represents the total number of tokens in all constraints.

the hypotheses in a beam can be separated into two types (see lines Grid ⇐ initGrid(maxLen, numC, k) initialize beams in grid 4: for t = 1, t++, t < maxLen do 6: n, s, g = ∅ 8: if hyp.isOpen() then the model in Algorithm 1 implements an interface with three functions: generate, start, and continue, which build new hypotheses in each of the three ways.


Online escort advertisements are difficult to analyze, because they lack grammatical structures such as constituency.

any form of inference must rely more on context than on grammar.


Detecting whether or not an advertisement is suspicious requires years of practice and experience in working closely with law enforcement.

annotation is a highly complicated and expensive process, which cannot be scaled using crowdsourcing.


The annotator is presented with the following options: "Certainly no," "Likely no," "Weakly no," "Unsure," 2 "Weakly yes," "Likely yes," and "Certainly yes."

the degree to which advertisements are suspicious is quantized into seven levels.


While these contain different characters, semantically they are the same, and they occur in the same context.

our expectation is that both the unigrams will be mapped to similar vectors.


This is shown in Figure 6d, where the model predicts the Entity label for "the requests" instead of "All the requests".

we also consider a relaxed scoring scheme where predictions are scored in token level instead of phrase level (see Table 4).


In particular, removal of the nodes participating in many triangles tends to separate the original graph into several connected components.

given a word u, we extract a network of its nearest neighbors from the synonymy graph G. Then, we remove the original word u from this network and run a hard graph clustering algorithm that assigns one node to one and only one cluster.


However, to model such multi-predicate interactions, the joint approach in the previous studies relies heavily on syntactic information, such as part-of-speech (POS) tags and dependency relations predicted by POS taggers and syntactic parsers.

it suffers from error propagation caused by pipeline processing.


Because we gather evidence using an automated process, the documents are not guaranteed to contain all facts needed to answer the question.

they are best seen as a source of distant supervision, based on the assumption that the presence of the answer string in an evidence document implies that the document does answer the question.


For example, the mentions Independence Day in d 2 and July 4th in d 3 have a common mention sense embedding during training since they refer to the same holiday.

text and knowledge are bridged via mention sense.


Then, we introduce a well designed mention sense disambiguation method, which can also be used for entity linking in a unsupervised way.

text and knowledge are combined via the bridge of mentions.


Then, we introduce a w designed mention sense disambiguation meth which can also be used for entity linking in a supervised way.

text and knowledge are co bined via the bridge of mentions.


However, the global optimum is expensive, in which each mention gets an optimum sense, to search over the space of all mention senses of all mentions in the document.

we approximately assign each mention independently: where P (C(m l )|s l j ) is proportional to cosine similarity between context vector and mention sense cluster center µ l j to measure the mention's local similarity, namely local probability.


It controls the importance here s ⇤ j = g(< m l , e j >) is obtained from anhors in wikipedia articles.

similar words and mention senses ill be closed in text space, such as w film nd s ⇤ Independence Day (film) , or w celebrations and ⇤ Independence Day (US) because they frequently ocur in the same contexts.


When encounter a mention, we map it to a set of mention senses, and then find the nearest one according to the distance from its context vector to each mention sense cluster center, which will be discussed in Section 5.

4 Considering all the above representation learning components, we define the overall objective function as linear combinations: we approximately assign each mention independently: where P (C(m l )|s l j ) is proportional to cosine similarity between context vector and mention sense cluster center µ l j to measure the mention's local similarity, namely local probability.


Since each mention sense corresponds to an entity in the given KB, the disambiguation method is equivalent to entity linking.

text and knowledge base is bridged via the multiprototype mention embeddings.


The objective function is as follows: where s * j = g(< m l , e j >).

if two mentions refer to similar entities and share similar contexts, they tend to be close in semantic vector space.


Given the annotated corpus D , we use a word w i or a mention sense s * j to predict the context words by maximizing the following objective function: where s * j = g(< m l , e j >) is obtained from anchors in Wikipedia articles.

words and mention senses will share the same vector space, where similar words and mention senses are close to each other, such as celebrations and Independence Day (US) because they frequently occur in the same contexts.


However, the global optimum is expensive with a time complexity of O(|M||M * l |).

we approximately identify each mention sense independently: where P (C(m l )|s * j ), local context information (Section 3), denotes the probability of the local contexts of m l given its mention sense s * j .


For example, if a human asks a robot to boil the water but the robot has no knowledge what this verb phrase means and how this verb phrase relates to its own actuator, the robot will not be able to execute this command.

it is important for robots to continuously learn the meanings of new verbs and how the verbs are grounded to its underlying action representations from its human partners.


The robot's representation of the environment is often incomplete and error-prone due to its limited sensing capabilities.

it is not clear whether previous approaches can scale up to handle noisy and incomplete environment.


For robotic learning, previous work (Cakmak and Thomaz, 2012) has also demonstrated the necessity of question answering in the learning process.

in our view, interactive learning beyond demonstration of primitive actions should play a vital role in the robot's acquisition of more reliable models of grounded verb semantics.


The loss computation can be numerically unstable if elements of the diagonal covariances are very small, due to the term log(d f r + d g r ) and 1 d q +d p .

we add a small constant = 10 −4 so that d f r + d g r and d q + d p becomes d f r + d g r + and d q + d p + .


This is because the gradient computation of the model is relatively fast, so a complex gradient update algorithm such as Adam becomes the bottleneck in the optimization.

we choose to use Adagrad which allows us to better scale to large datasets.


Some authors claim the inverse (Wedding and Boyd, 1999): that movies reflect existing cultural values of the society, adding weight to their ability in influencing individual beliefs of what is accepted as the norm.

they are studied in multiple disciplines to analyze their influence.


In particular character names get changed, sometimes even gender.

some characters may not be aligned to the correct entry from IMDb.


Despite this rising interest, and despite many works that deal with sarcasm identification González-Ibánez et al., 2011;Riloff et al., 2013;Barbieri et al., 2014), to the best of our knowledge, generation of sarcasm interpretations has not been previously attempted.

the following sections are dedicated to previous work from neighboring NLP fields which are relevant to our work: sarcasm detection, MT, paraphrasing and text summarization.


(2013) observed that a certain type of sarcasm is characterized by a contrast between a positive sentiment and a negative situation.

they described a bootstrapping algorithm that learns distinctive phrases connected to negative situations along with a positive sentiment and used these phrases to train their classifier.


SIGN, in contrast, targets sentiment words and changes them frequently.

we do not observe substantial differences between the algorithms in the automatic measures that are mostly based on ngram differences between the source and the interpretation.


In contrast, the coverage of sentiment similarities extracted from co-occurrence patterns is quite wide because document is a long context, while their accuracies are not as high as the similarities based on syntactic rules.

we propose to combine these two types of sentiment similarities to obtain a balance between accuracy and coverage.


Active learning methods iteratively select the most informative instances to label and add them to the training set (Settles, 2010).

an important issue in these methods is how to measure the informativeness of unlabeled samples.


However, many researchers have found that unlabeled instances with high uncertainties can be outliers, whose labels are useless and even misleading (Settles, 2010;Zhu et al., 2010).

here we combine uncertainty with representativeness to avoid outliers.


In addition, our approach with dynamic weighting scheme in combining uncertainty and density outperforms that with constant weighting scheme, which implies that it is beneficial to emphasize representative samples at initial iterations and gradually focus on difficult samples at later iterations.

the experimental results validate the effectiveness of our active learning algorithm.


However, the networks could not provide superior results than the SVM regressors.

for this report, we only report the SVM methods.


Without loss of generality, we assume the network is directed, as an undirected edge can be considered as two directed edges with opposite directions and equal weights.

the structure-based objective aims to measure the log-likelihood of a directed edge using the structure-based embeddings as Following LINE (Tang et al., 2015), we define the conditional probability of v generated by u in Eq.


3as Vertices in real-world social networks usually accompany with associated text information.

we propose the text-based objective to take advantage of these text information, as well as learn text-based embeddings for vertices.


Note that, when we only keep 5% edges for training, most vertices are isolated, which results in the poor and meaningless performance of all the methods.

we omit the results under this training ratio.


We attempt to choose a better configuration of the number of bi-LSTM layers and the hidden dimension based on the development set performance, but the default settings turn out to perform the best.

we stick to all default hyper-parameters in Dozat and Manning (2017) for training the Singlish parsers.


Figure 4: The finite automaton whose state 0 begins non-loose chains and state 1 loose chains The automaton is symmetric: states with uppercase names are symmetrically related with corresponding lowercase states.

it suffices to define the initial and uppercase-named states: 0 the initial state for a non-loose chain; I a bidirectional chain: Z a 3-turn chain; 1 the initial (and only) state for a loose chain; Not all states make these distinctions.


The nonderivative representations for our axiomatic properties share the same Dyck language D 55 and homomorphism, but differ in terms of forbidden patterns.

also any conjunctive combination of these two properties shares these components and thus define a context-free language.


", the agent should focus on her friends who went to Columbia University.

we want this utterance to be sent to item nodes connected to columbia, and one step further to other attributes of these items because they might be mentioned next as relevant information, e.g., jessica and josh.


In evaluation on two datasets, we show that: a) NBT models match the performance of delexicalisation-based models which make use of hand-crafted semantic lexicons; and b) the NBT models significantly outperform those models when such resources are not available.

we believe this work proposes a framework better-suited to scaling belief tracking models for deployment in real-world dialogue systems operating over sophisticated application domains where the creation of such domain-specific lexicons would be infeasible.


However, training corpus contains much more annotated arguments than triggers (about 9800 arguments and 5300 triggers in ACE 2005 dataset) because each trigger may be along with multiple event arguments.

the unbalanced data may cause joint models to favor AE task.


By contrast, the component for pre-prediction of arguments always exploits the extracted trigger information.

we argue that annotated arguments are actually used for AE, not for ED in existing joint methods, which is also the reason we call it an indirect way to use arguments for ED.


We have analyzed that arguments are capable of providing significant clues to ED, which gives us an enlightenment that ar-guments should be focused on when performing this task.

we propose a neural network based approach to detect events in texts.


All events they published are also frames in FN.

we treat frame elements annotated in FN corpus as event arguments.


Indeed, the impact of the inference approach on the different usages of latent topic models for text collections remains to be better understood.

proof Any segmentation of the sentence of length M starts with either a segment of length 1, a segment of length 2, • • • , or a segment of length L. g(M ; L) can be defined through the following recurrence relation: together with the initial values g(1; L), g(2; L), • • • , g(L; L), which can be computed offline (for example, for L = 3, one has: g(1; 3) = 1, g(2; 3) = 2, g(3; 3) = 4).


Proof Any segmentation of the sentence of length M starts with either a segment of length 1, a segment of length 2, • • • , or a segment of length L. Thus, g(M ; L) can be defined through the following recurrence relation: together with the initial values g(1; L), g(2; L), • • • , g(L; L), which can be computed offline (for example, for L = 3, one has: g(1; 3) = 1, g(2; 3) = 2, g(3; 3) = 4).

note that g(1; L) = 1 for all L. : due to the recurrence relation on g. This proves the first part of the proposition.


Generally speaking, weakly supervised semantic parsers operate on one assumption -a candidate semantic parse is treated as a correct one if it results in answers that are identical to the gold answers.

a straightforward modification of existing structured learning algorithms in our setting is to use any semantic parse found to evaluate to the correct answers during beam search as a reference parse, and then update the model parameters accordingly.


A 5 -A 8 Op-GT, Op-GE, Op-LT, Op-LE The arguments of these comparison operations are extracted from question in advance.

the action modules just need to decide whether such relations are indeed used in the question.


There has been one immediate observation from these studies, D2clustering always outperforms two of its degenerated cases, namely Tf-idf and AvgDoc, and three other popular methods: LDA, NMF, and PV, on all tasks.

for document clustering, users can expect to gain performance improvements by using our approach.


Our work treats Wikipedia as a collection of articles and does not rely on its internal graph structure.

our approach is generic and could be switched to other collections of documents, books, or even daily updated newspapers.


Large-scale QA systems like IBM's DeepQA (Ferrucci et al., 2010) rely on multiple sources to answer: besides Wikipedia, it is also paired with KBs, dictionaries, and even news articles, books, etc.

such systems heavily rely on information redundancy among the sources to answer correctly.


SQuAD questions have been collected via a process involving showing a paragraph to each human annotator and asking them to write a question.


This is because the well-trained LSTM-Jump is aware of the jumping signal at the first token and hence can directly jump to the output position to make prediction, while LSTM is agnostic to the signal and has to read the whole sequence.

the reading speed of LSTM-Jump is hardly affected by the length of sequence, but that of LSTM is linear with respect to length.


On the contrary, the question can guide LSTM-Jump on how to read selectively and stop early when the answer is clear.

when it comes to the output stage, the "memory" is both fresh and uncluttered that a more accurate answer is likely to be picked.


It turns out that the last-read sentence is still talking about Christmas with a negative voice.

the model stops to take "Christmas" as the output.


The 0 feature extractor is the additive identify because it produces the zero vector for any input.


Without loss of generality, we can fix this mapping to be the identity mapping, thereby rendering the final vectorized form equal to f (x).

that is, f&1 = f. (F, &) is a commutative monoid.


This refactoring is done at the level of feature templates (i.e., feature extractors); the actual savings are realized when the feature vectors are computed by applying this feature function to an input.

the simplification, though seemingly modest at the template level, can lead to a substantial speed improvements when the features vectors are actually manifested from data.


As it is impractical to enumerate all possible pivot sentences, the two NMT models are trained separately in practice: Due to the exponential search space of pivot sentences, the decoding process of translating an unseen source sentence x has to be divided into two steps: The above two-step decoding process potentially suffers from the error propagation problem (Zhu et al., 2013): the translation errors made in the first step (i.e., source-to-pivot translation) will affect the second step (i.e., pivot-to-target translation).

it is necessary to explore methods to directly model source-to-target translation without parallel corpora available.


For the pivot and likelihood methods, the Spainish sentence segment 'sentáis al volante' is lost when translated to English.

both methods miss this information in the translated French sentence.


2016, we build a tree encoder on top of the sequential encoder (as shown in Figure 3(a)).

if node k is a leaf node, its hidden state is the annotation produced by the sequential encoder: the encoder is able to capture both sequential context and syntactic context.


constructed a crosslingual entity linking collection for 21 languages, which covers ground truth for the largest number of languages to date.

we compare our approach with theirs that uses a supervised name transliteration model .


In fact, simply using the model saved at the end of training gives poor performance.

we need a mechanism to select a good model.


We test if our methods can improve performance by generating new rescored matrices by rescoring all of the pairs of words by taking into account global constraints that apply to cognates detection.

our methods are complementary to previous methods for creating cognates detection systems.


We are not aware of others evaluating according to this much more difficult condition, but we think it is important to consider especially given the potential impacts it could have on the global structure methods we've put forward.

we run a second set of evaluations where we take the ten thousand most common words in our corpora for each of our languages, which contain many of the cognates from the standard test sets and we add in any remaining words from the standard test sets that didn't make it into the top ten thousand.


In morphologically rich languages, a lemma can have hundreds of individual forms.

both generation and analysis of such morphological inflections are active areas of research in NLP and morphological processing has been shown to be a boon to several other down-stream applications, e.g., machine translation (Dyer et al., 2008), speech recognition (Creutz et al., 2007), parsing (Seeker and Ç etinoglu, 2015), keyword spotting (Narasimhan et al., 2014) and word embeddings (Cotterell et al., 2016b), inter alia.


In addition, we preprend a symbol λ ∈ L to the input string (e.g., λ = Es, also represented by an embedding), so the RNN can handle multiple languages simultaneously and generalize over them.

our final input is of the form λS + Σ + .


Both in terms of form (new words are mainly built using a templatic system) and categories (it has tags such as construct state), Arabic is very different.

we do not expect it to support morphological knowledge transfer and use it as a baseline for all target languages.


While recent work in sequence transduction advocate jointly training the alignment and the decoding mechanisms Yu et al., 2016), we instead show that in our case it is worthwhile to decouple these stages and learn a hard alignment beforehand, using it to guide the training of the encoder-decoder network and enabling the use of correct alignments for the attention mechanism from the beginning of the network training phase.

our training procedure consists of three stages: learning hard alignments, deriving oracle actions from the alignments, and learning a neural transduction model given the oracle actions.


For example, according to Chinese verb formation theory (Sun et al., 2009), in CPB, an agent of a verb is often marked as its Arg0, but not all Arg0 are agents.

roles are defined for predicates with similar syntactic and semantic regularities, rather than single predicate.


We will consider the combination of the standard benchmark, CPB 1.0 dataset (Xue and Palmer, 2003), with the new corpus, CSB, because there are a lot of differences between them, as we discussed in Section 2.

a number of challenges arise for this task.


And our dataset has no frame files and use different role set.

it is hard to find explicit mapping or hierarchical relationships among their role sets, or decide which system is better, especially when there are more than two resources.


However, it is well known that adding data in very different genre to training data may hurt parser performance (Bikel, 2004).

we also need to deal with domain adaptation problem when using heterogeneous data.


Though they can also leverage knowledge from different domains, they have following drawbacks: finetuning cannot avoid catastrophic forgetting because learned parameters, whether embeddings or other hidden weights, will be tuned after the model has been initialized; And multi-task learning cannot ignore previously learned harmful features because some features are learned in shared layers, although it avoids forgetting by randomly selecting a task to learn at each iteration.

to solve the above-mentioned challenges, we further introduce progressive learning which we believe is more suitable for the task.


Lastly, they considered whether their hyperparameters were inadequately tuned, but extensive hyperparameter tuning did not change the story.

the reason for the performance gap, and how to correct it, was left as an open problem.


In addition to providing context-sensitive token embeddings, the proposed method implicitly regularizes the embeddings of related words by forcing related words to share similar concept embeddings.

the representation of a rare word which does not appear in the training data for a downstream task benefits from all the updates to related words which share one or more concept embeddings.


That is, context f (i, t) = lstm in(t i−1 ) for the forward RNN and context b (i, t) = lstm in(t i+1 ) for the backward RNN.

each token gets two representations, one from each RNN.


Second, when class labels are treated as though they cannot be decomposed, every class label must be modeled independently, even those containing overlapping words ("American jazz musician", "French jazz musician").

the number of meaning representations to be learned is exponential in the length of the class label, and quickly becomes intractable.


As a result, the number of meaning representations to be learned is exponential in the length of the class label, and quickly becomes intractable.

compositional models of taxonomic relations are necessary for better language understanding.


Our work addresses this limitation by modeling fine-grained class labels compositionally.

the proposed method can combine evidence from multiple sentences, and can perform IsA extraction without requiring any example instances of a given class.


In this case, we can freely add e (i,l) without violating either 1EC or P2 conditions.

case 2 does not lead to any maximal 1Ec/P2 graph.


If sr k < sl k+2 , e (sl k ,sr k ) and e (sl k+2 ,sr k+2 ) share no common endpoint, violating 1EC.

sr k = sl k+2 = s k+2 , and Condition 3 is satisfied.


The sequence labeling models are able to learn this bias in the label distribution without obtaining much additional information from words that have the majority label (O for outside of an entity; C for correct word).

we propose an additional training objective which allows the models to make more extensive use of the available data.


However, the model has access to the full context on each side of the target token, and predicting information that is already explicit in the input would not incentivise the model to learn about composition and semantics.

we must design the loss objective so that only sections of the model that have not yet observed the next word are optimised to perform the prediction.


Based on development experiments, we found that error detection was the only task that did not benefit from having a CRF module at the output layer -since the labels are very sparse and the dataset contains only 2 possible labels, explicitly modeling state transitions does not improve performance on this task.

we use a softmax output for error detection experiments and CRF on all other datasets.


), which are manually annotated to achieve high performance.

they are hard to obtain in practical application scenarios.


In this work, we make a naive assumption that two neighboring sentences share a "common root".

a cross-sentence dependency path can be represented as two shortest dependency path branches from the ends to the "common root", as shown in Figure 2.


deep-coref − reports the performance of deep-coref [lea] in which WikiCoref's words are not incorporated into the dictionary.

for deep-coref − , WikiCoref's words that do not exist in CoNLL will be initialized randomly instead of using pre-trained word2vec word embeddings.


Moreover, the commonly used evaluation metric ROUGE yields a noisy surrogate evaluation (despite its good correlation with human judgments) compared to the much more meaningful evaluation based on human judgments.

the current end-toend evaluation does not provide any insights into the task of automatic summarization.