Even though the comments sometimes signal the editors' intents, they are not infallible, editors may fail to complete the comment field, may only label one of the multiple edit intentions for a single revision, or write comments that are inaccurate, irrelevant, or incomplete.

the first author annotated the 2,200 revisions from the expanded corpus and merged it with the randomly sampled corpus.


We hypothesize that Mnemonic Reader's self-alignment layer, which helps model long-distance relationships between parts of the paragraph, makes it better at locating all pieces of evidence that support the correct answer.

it can be more confident in the correct answer, compared to the fake answer inserted by the adversary.


Compared to the overall rating, users are less motivated to give aspect ratings.

it is more practically useful to perform document-level multi-aspect sentiment classification task, predicting different ratings for each aspect rather than an overall rating.


As shown in Figure 2 (right), the hierarchical attention of keywords is applied to both sentence level (to select meaningful words) and document level (to select meaningful sentence).

our model builds aspectspecific representations in a bottom-up manner.


As we discussed in the introduction, the set of selected question keywords may not best characterize the aspect for different documents.

the IAM module introduces a backward attention to use document information (word or sentence) to select useful keywords of each aspect as the document-specific question to build attention model.


In our experiments, LSTM-based HAN obtains slightly better results.

we report the results of HAN with LSTM as the encoder.


In the second sentence, we observe that the value of word internet increases with the number of hop.

we can see that the more sensible weights are obtained for words through the proposed iterative attention mechanism.


In a similar vein, claims in persuasive students' essays (PE) heavily rely on phrases signaling beliefs ("In my opinion, although using machines have many benefits, we cannot ignore its negative effects.")

or argumentative discourse connectors whose usage is recommended in textbooks on essay writing ("it is not need for young people to possess this ability.").


3 We are not interested in optimizing the properties of a certain learner for this task, but rather want to compare the influence of different types of lexical, syntactical, and other kinds of information across datasets.

4 we used a limited set of learners for our task: a) a standard L2-regularized logistic regression approach with manually defined feature sets 5 , which is a simple yet robust and established technique for many text classification problems (Plank et al., 2014;He et al., 2015;Zhang et al., 2016a;Ferreira and Vlachos, 2016); and b) several deep learning approaches, using state-of-the-art neural network architectures.


Compared to many traditional NLP systems, which are trained on newswire or similar text, an automated system capable of explaining abstract language, or finding alternative text expressions for slang (and other unknown terms) would exhibit a deeper understanding of the nuances of language.

research in this area may open the door to a variety of interesting use cases.


That is, these annotations cannot be generated without background knowledge or added context.

in this preliminary work we focus on predicting CI lyric annotations.


tf (t) is the frequency of the concept t (unigram/bigram) in the whole topic.

s i inherits the salience of its sentence, and also considers the importance of its concepts.


The objective function and constraints are linear.

the optimization can be solved by existing ILP solvers such as the simplex algorithm (Dantzig and Thapa, 2006).


However, there are less parameters in GRU resulting in improvements for the efficiency of training.

we choose AttenC-dot-gru as the attention structure of our framework in the subsequent experiments.


Bayes rule is applied to p θ (z t |y <t , z <t ), and we can extract log p θ (z) from the expectation, transfer the expectation term E q φ (zt|y<t,z<t) back to KL-divergence, and rearrange all the terms.

the following holds: Let L(θ, φ; y) represent the last two terms from the right part of Equation 4: Since the first KL-divergence term of Equation 4 is non-negative, we have log p θ (y <t ) ≥ L(θ, φ; y) meaning that L(θ, φ; y) is a lower bound (the objective to be maximized) on the marginal likelihood.


The reason is that the recurrent variational auto-encoders used in our framework have better representation ability and can capture more effective and complicated latent structures from the sequence data.

the summaries generated by DRGD have consistent latent structures with the ground truth, leading to a better ROUGE evaluation.


Then, if we are able to estimate the subjects with small amounts of documents and the external information in them, the identification of salient sentences from a document can be supported by sentence features that have been learned from document subject estimation.

smaller amounts of actual reference summaries are only needed as mutually learning feature representations for both subject estimation and sentence identification from pseudo-rough reference summaries.


We assume that a document is classifiable into certain subjects that represent meta-information of the document, and assume that sentences for a summary are extracted in relation to the subjects.

solving document classification supports sentence extraction from a document with multi-task learning of both tasks.


All the important facts available from all sources about a news event are combined into a single article for the reader's convenience, and the information is presented in a neutral manner avoiding the bias that may be present in other news sources.

we treat a Wikinews article as an ideal overview article (i.e., reference) of the source news articles.


In addition, recognizing second order relations between event chains relies on adequate number of event mentions that are already linked.

our model conducts event coreference in two stages.


Once you find a completed hypothesis (by generating the </s> symbol), there are still other active hypotheses in the beam that can continue to grow, which might lead to better scores.


Other possibilities include training (styling) on famous personalities, or certain types of poetry, or song lyrics, or even mixing styles by providing two or more datasets for styling.

our targets are highly recognizable styles, and use of these for emphasis (or caricature) by human puppeteers who can choose from multiple options and guide neural models in a direction they like.


As regards directly integrating beam-search, we found (a) that beam-search often produces a set of similar top-N candidates, and (b) that decoding with only the objective p(Y |X) can easily lead to irrelevant candidates.

(See section 7.3) we use the selective-sampling method to generate candidates for all our experiments; this (a) samples stochastically then (b) selects using a learned objective from data.


We also found that the previous sentence in the styled corpus (i.e., previous sentence in the speech) provides helpful context for the current sentence, analogous with a question-answer link.

we use both pseudo context and the previous sentence as possible sources S to fine-tune the in-domain decoder.


However, there is no standard as to what are the test ones (it is unknown from the earliest paper using this data (Stolcke et al., 2000)).

we randomly split the set into two, 20 conversations in each, with similar amount of utterances.


Our model learns not to use background topics (η = 1) for NPS, because background topics may filter out function words and DA-related words that occur more consistently throughout a conversation than content words do.

of filtering out content words, the foreground topics reflect various acts in conversation.


Topic assignment is based on similarity of words in a sentence to other sentences in that topic, and short sentences often find similar sentences more easily than long sentences do.

learned topics tend to be characteristic of short sentences that are similar enough to form the separate topics, and as a result, long sentences may be assigned the same topic regardless of the DA actually performed.


Recurrent neural networks(RNNs) such as gated recurrent units (GRUs) have the ability to keep the information from the beginning to the end to some extent.

the cue word added on the first step of the neural networks can still influence the generation of the later steps.


However, we observe that, although the network is capable of deciding what to keep in the cell state to affect the later generation, the influence of the added information in the beginning of decoding is becoming weaker and weaker over time.

to provide the model a broader and more flexible space for learning, we propose a global information inception pattern, which fuses the cue word C w as the auxiliary information at every step of decoding.


Exhausting all the budget at early training stage, named Early teaching heuristic (Early), is simple and straightforward but wastes teaching opportunities on unnecessary cases.

it is im-perative to design some effective heuristics to instruct when the teacher should give a hand to the student.


If the success rate over a training process keeps above this threshold all the time, we will think this training process is absolutely safe.


2017is not generative, rather it is based on retrieval of system responses, based on the history of the dialogue up to that point.

for direct comparison, and for simplicity of exposition, we do the same here: we apply the method described for creating a user simulation (section 3.2.1), this time for the system side, resulting in a 'system simulation'.


With such data, it was easy for the agent to learn the locally optimal policy that "failed fast"; that is, the policy would finish the dialogue immediately, so that the agent could suffer the least amount of per-turn penalty.

we provided some rule-based examples that succeeded reasonably often, and did not flush the buffer until the performance of the RL agent reached an acceptable level.


as shown in Figure 4 and Table 1, the performance of the HRL agent on types B and C users (who may need to go back to revise some slots during the dialogue) does not drop much compared to type A users, despite the increased search space in the former.


The Stanford parser score is not designed to measure grammaticality, however, it will generally prefer a grammatical parse to a non-grammatical one.

7 lower parser scores indicate less grammaticallycorrect utterances.


In the Joint Copy model, however, the source records also compete with words that cannot be copied.

training the Conditional Copy model with the supervised loss of Gülçehre et al.


Words with medium values of log( F E F H ) (mws)we selected 27 words having not so high and not so low log( F E F H ) at uniformly at random.

full set of words (f ull) -in total we selected 57 target words for the purpose of our evaluation.


Rank ranges: We split each of the three rank lists (U U R, ground truth and baseline) into five different equal-sized ranges as follows -(i) surely borrowed (SB) containing top 20% words from each list, (ii) likely borrowed (LB) containing the next 20% words from each list, (iii) borderline (BL) constituting the subsequent 20% words from each list, (iv) likely mixed (LM) comprising the next 20% words from each list and (v) surely mixed (SM) having the last 20% words from each rank list.

we have three sets of five buckets, one set each for U U R, the ground truth and the baseline based rank list.


The number of target words falling in each bucket across the three rank lists are the same and are noted in table 2.

the precision and recall as per the definition are also the same.


In contrast, the supervised datasets which have the SUD ground truth are pretty small, ranging from 896 for the intersection of the likes, status updates and SUD (LikeStatusSUD in Table 1) to 3508, which is the intersection of the likes and SUD (LikesSUD in Table 1).

the main focuses of this research include (1) employing unsuper-vised feature learning to take advantage of a large amount of unsupervised data (2) employing multiview learning to combine heterogeneous user data for better prediction.


A similar rule-based assignment is also used for gender.

we compute the response words and their normalized frequencies based on the responses from 80% of the users chosen randomly, and accordingly predict the demographic group for the remaining 20% of the users based on a decision across the entire set of a user's responses.


In the second model, the optimization is rather symmetric, allowing tagged and generic embeddings to influence each other.

the optimization function seeks to predict both tagged and untagged words in the vicinity given a target word, instead of only focusing on predicting untagged words like EMB1.


Note that the model assumes a locally coherent mixture of global modes and updates the mixture for each observed word.

each comment receives a sequence of association probabilities over the global modes.


Our goal is to target dimensions of interpersonal relationships between any two individuals, from weak links (e.g., journalists interviewing celebrities) to strong ties (e.g., close friends).

we create a corpus 1 by first retrieving pairs of people, and then annotating dimensions for their relationships.


By varying the value of k, one can explore the main topics (if k is large) or the secondary topics, that are still recurrent (when k is low).

recurrent topics and their dynamics can be analyzed on different time scales.


Using regression to relate variables, although rarely done in Computer Science domains, is standard practice in social and political science (Fox, 1997), though typically not over thousands of potential independent variables as we do here.

we also correct for multiple hypotheses by using the Benjamini-Hochberg procedure (Benjamini and Hochberg, 1995) over our significance tests.


In other words, bursty incomplete N-grams inevitably overlap bursty phrases.

the proposed method performs the extraction of bursty phrases as the set cover problem in which all bursty N-grams are covered by a minimum set of bursty phrases.


Here, the increment of the occurrence of the phrase contributes to the increment of the occurrence of overlapping N-grams.

(incomplete) Ngrams that overlap the phrase can also burst.


Consequently, (incomplete) Ngrams that overlap the phrase can also burst.

bursty incomplete N-grams always have their original bursty phrases that overlap each other.


In word-based and noun phrase-based methods, the dictionary or vocabulary significantly affects the performance.

we also used neologd 6 (Sato et al., 2017) (version v0.0.5 updated at May 2, 2016), a neologism dictionary extracted from many language resources on the web, as an additional dictionary (+Dic).


Noun phrasebased methods tended to miss highly bursty phrases of irregular forms.

the minz-score of extracted top K bursty phrases became small.


By comparing Y (s) to the answer set A, the precision is 1 6 and the recall is 1.

the F 1 score used for the reward is 2 7 .


Note that we use the same architectures and initialization procedures for MMRN-PIPELINE/JOINT and REIN-FORCE/REINFORCE+.

the superior performance of MMRN-JOINT shows that the joint learning plays a crucial role in addition to the choices of architecture.


3 All other unit and relation types from the specific models are ignored, since there is no clear mapping between them.

texts with myside bias General Model we model the overall argumentation of an argumentative text as a forest of trees.


The first method learns a linear mapping between the word2vec embedding space V w2v and the learned word embedding space V cnn by solving a linear regression problem .

any word from V w2v can be mapped into V cnn for encoding sentences.


The second method fixes the word vectors in V cnn as the corresponding word vectors in V w2v , and we do not update the word embedding parameters during training.

any word vector from V w2v can be naturally used to encode sentences.


5 The Leitner scheduler reviews instances of q i at every 2 i iterations.

instance in lower queues (difficult/forgotten instances) are reviewed more frequently than those in higher queues (easy/recalled ones).


In principle, reviewing instances could be delayed for any number of epochs; in practice however, delay is bounded both below and above (e.g., by queues in the Leitner system).

we assume that, at each epoch e, instances could be delayed for at least one iteration and at most k − e iterations where k is the total number of training epochs.


"Clauses that are subjects or objects of a main verb are not treated as EDUs" .

existing discourse segmenters heavily rely on information derived from constituent trees usually following the Penn Treebank (PTB) (Marcus et al., 1993) guidelines.


The trained policy is not guaranteed to work well in real world scenarios.

on-line policy training has been of great interest (Gašić et al., 2011).


Although the improvement of efficiency can benefit the safety of learning process, no matter how efficient the algorithm is, an unsafe on-line learned policy can lead to bad user experience at the beginning of learning period and consequently fail to attract sufficient real users to continuously improve the policy.

it is important to address the safety issue.


In reality, we can tolerate a system to make mistakes at the beginning but it should improve at a significant speed until reaching the ideal performance S i .

later failures should weight more than early failures to evaluate efficiency.


Since the previous hidden representations are all fixed at a given timestep, and the current hidden representation is directly a function of these previous hidden representations as well as the current input, the audience's current poll depends directly on the timestep's input.

at a given timestep, our framework allows us to determine the input that would maximize the audience's favorability toward the orating team.


We view the presence of applause or laughter from the audience as a sign of endorsement during that particular turn.

at that particular timestep, the hidden state should be able to directly predict this occurrence.


Moreover, the one-hot vector will only ever have mass in the indices for the 'for' and 'against' teams, and neglecting the 'undecided' index, which is an important sector in the polling.

we create a soft vector as follows: a random number, n, is chosen in the interval ( 1 3 , 1).


On average, such reactions occur 21 times during a debate, with a standard deviation of 10.

this approach adds 2,268 more supervised signals to the dataset.


At a given timestep, the previous hidden and cell states are known.

a is directly a function of the current input x at a given timestep.


One insight here is that since the languages es and pt are highly related, we need much less data to train an accurate mapping matrix W (pt→es) than we to need to learn an accurate W (pt→en) .

we train a mapping function from Portuguese to Spanish, which makes predictions as follows.


We would like to improve predictions made by Equation 7 by improving the mapping matrix W (en→pt) .

when training using the Portuguese to English training data, we want our objective to both minimize the loss defined in Equation 3 and simultaneously to let W (en→pt) mimic predictions made through the pathŷ i (en)← (es)← (pt) as "soft targets" within a margin.


For each IL character s, it writes two RL characters t and t ′ with respective probabilities p 21 (t|s) and p 22 (t ′ |s).

we assume p 22 (t ′ |s) is independent of t. we can estimate p( as modeled in wFST2.


As expected, the results obtained with this straightforward approach show positive trends, but at the same time reveal its limitations for all the languages involved.

the focus of our work shifts to Q2 and Q3.


we have e p ="Tim Berners-Lee" and c p ="Turing Award" as an entity pair in R p .


This process iterates until no more edges in G p can be added to G * p .

the vertices in G * p form the desired clique C * p .


We select the clique with largest edge weights as the maximum edge weight clique for G p .

the seed relation instance collection is defined as the total runtime complexity is O(k|L p | 2 ).


In "[E]军事([E] military)", it indicates that the entity is related to the topic "军事(military)".

we define a new relation predicate "话题(topic-of)" and establish the relations between entities and "军事(military)".


In our work, such logical mapping rules are required for only a couple of relation types.

the proposed approach needs very minimal human work.


Furthermore, the partially explicit and implicit patterns (see (Nastase and Strube, 2008)) do not have their counterparts in Chinese.

our method is superior to existing systems.


For example, per:alternate names can be mapped to alternativeNames, birthName and nickName in DBPedia.

for each slot type, we collect many triples: query, slot, filler and use TransE (Bordes et al., 2013), which models slot types as translations operating on the embeddings of query and filler, to derive a representation for each slot type.


This task is easier than the general tagging problem: if we can effectively identify the product in, e.g., the title of a post, then we do not need to identify additional references to that product in the body of the post.

we also consider a postlevel model, which directly tries to select one token (or NP) out of a post as the most likely product.


A mention is ending at the previous word (E) 3.

a mention is continuing to the next word (C) for each token, the possible combinations of cases are as follows: ECS, EC, CS, C, ES, E, S, and X, where X means none of the three cases applies.


Entity and word embeddings are pre-trained as discussed in Section 3.

the only learnable parameters are the diagonal matrices A and B, plus the parameters of f .


Training and prediction in binary CRF models as the one above is NP-hard.

in learning one usually maximizes a likelihood approximation and during operations (i.e.


In general, such minimization is inherently limited in scope due to the absence of domain knowledge.

minIE does not and cannot correctly minimize all its extractions in all cases.


We also add y 0 and y n as the start and end tags of a sentence.

t becomes a square matrix of dimension m + 2.


The situation is even worse in domain-specific NER systems, since their data annotation requires the engage of domain experts.

in many special domains, only trained models or APIs are available, while their training data are private and inaccessible.


During training, we measure the similarity between y ft and y fs with KL-divergence, and minimize their difference.

for a given x i , we optimize: Q j is the KL-divergence.


Furthermore, it is inappropriate to evaluate the correctness of IOB label (i.e., l k ) of each word (i.e., w k ), since the IOB sequence y i of a sentence x i is generated dependently.

we choose sentence x i as our MIL instance, and the correctness of x i is evaluated by the likelihood probability of all words with correct BIO tags.


ity", "daily exercises", or "men with diabetes".


We show the following objective function of our relational similarity modeling: To model the transformation process in Equation 4, we need to know how to measure the similarity of the triplet ( A, B, R).

we develop a similarity measurement function SimiScore( A, B, R) with learnable weights (W * ), the similarity function takes an input named entity pair of ( A, B) and a relation R, returns a similarity score dist ∈ R 1 representing how semantically close ( A, B, R) are, as in Function 1.


Jansen et al., as another example, demonstrated domain independence on the task of non-factoid question answering by using both shallow and deep discourse structure, along with lexical features, to train their classifiers (Jansen et al., 2014).

domain independence is certainly possible for discourse related tasks, but there does not yet seem to be a one-size-fits-all solution.


To put this in perspective, the Corman detector (Ceran et al., 2012) uses more than 20,000 features, and achieves an F 1 of 0.66.

we were able to achieve 83% of the performance of the Corman detector with 4,000 times fewer features.


The analysis of the output of the individual modules and their contribution to the overall system uncovers an issue: those modules that produce a high number of incorrect suggestions, thus overcorrecting actually correct input tokens, are at the same time those modules that are the only ones producing correct suggestions for some of the incorrect input words.

those uniquely suggested corrections are not chosen in the decision modules due to an overall weak performance of this module.


Her speech contains a lot of words that are incorrect in standard German, however, the goal is it to preserve this "errors" in the digital version.

correction merely on the basis of the OCR text without consulting the printed version or an image-digitized facsimile, can essentially never be perfect.


The charge distribution is imbalanced, and the top 5 charges take more than 60% of the cases.

we evaluate the charge prediction task using precision, recall and F1, in both micro-and macro-level.


It can be observed that the difference between the normalized expectation E[n ϕ i ]/d i of two documents is insignificant.

the measure described in Equation 2will be unable to tell the difference between these two documents.


Since in our benchmark generation, the percentage of outliers does not exceed 1%.

the perfect results for any r ≥ 1% should be 1.0.


These questions are constructed by taking one document from the corpus as d i and another document not from the corpus as d j .

the one not from the corpus should be the answer.


Due to the lack of effective management, most of the discussion forums within MOOCs are overloaded and in chaos (Huang et al., 2014).

a key problem is how to manage the forum contents.


Moreover, within MOOC settings, the course contents can be broken down to knowledge points, and each video clip just corresponds to a knowledge point.

we propose to fulfill the application, which is to associate threads to subtitles of video clips, i.e., threadsubtitle matching.


To learn the latent similarity, we make use of the relationship between words and documents, and then similar documents can be clustered, while different documents are scattered.

we propose the third component, L 3 , shown in the middle part of Fig.


labeled data (thread-subtitle matching pairs) are only used for evaluation, and we do not utilize dev dataset.


For all the evaluated baselines, we use the same parameter setting.


For instance, change from numeric '18' to word 'eighteen' may be a minor change and less crucial for the author to review, as compared to an edit that alters the facts of the document.

in our work, we focus on automatically inferring the impact/change introduced by edits, and predict the perceived importance of such edits by authors.


Our analysis of the document corpus revealed that most text edits to sentences do not significantly change the structure of the sentence.

a simple heuristic based approach can be used to identify edit types.


We hypothesized that the importance of edits would be affected by both the nature of the edits, characterized by the aforementioned categories, as well as the relevance of the sentence to the content of the document.

we chose features that capture both these aspects and have divided them into two groups, namely, change-related features and relevance-related features.


Baselines: To the best of our knowledge, our work is the first that attempts to infer importance/impact of text edits between document versions.

we did not have established baselines to compare against.


In addition to adding elements to the end of the sequence, the stack LSTM provides a pop operation which moves the stack pointer to the previous element.

the LSTM can be understood as a stack implemented so that contents are never overwritten, When the action OUT is taken, the stack is cleared by moving the stack pointer to the initial position.


The attention context computation in the GRU can be re-factored as follows: Crucially, the hidden vector representation s j is only dependent on the source sentence, while a ij is dependent on the target hypothesis.

the original computation U c i requires total |T | × b multiplications per sentence, but the re-factored version U s j only requires total |S| multiplications.


In the GRU equation, for the first target hidden layer, x i represents the previously generated word, and h i−1 encodes the hypothesis up to two words before the current word.

if two partial hypotheses in the beam only differ by the last emitted word, their h i−1 vectors will be identical.


The dimensionality of each hidden output of LSTMs is 100, and the hidden outputs of both forward LSTM and backward LSTM are concatenated, thereby the output of each BLSTM for each time step is 200.

the input to the common BLSTM and the private BLSTM is 128+200=328 dimensional.


Compared with standard attentional NMT, the Encoder of SDRNMT-1 model consists of a convolutional architecture and an bidirectional RNN, as shown in Figure 2.

the proposed Encoder can not only learn compositional representations for dependency units but also greatly tackle the sparsity issues associated with large dependency units.


Using our classifier with all features, we carried out 10 runs of a 10-fold crossvalidation on the training data and we obtained an accuracy (standard deviation) of 84.14 (0.06), 84.30 (0.14) and 84.28 (0.11) when using 10, 20 and 30 images, respectively.

no significant differences in accuracy were observed for different numbers of images.


The best SemEval-2014 system obtained an accuracy of 84.57  and other heavily engineered, finely-tuned systems (Beltagy et al., 2016;Yin and Schütze, 2017) reported up to 3% points of accuracy improvement since then.

our results are still below the state of the art.


Following Zipf's law, words in natural language tend to sort into ranges of frequent words (the majority of tokens) and rare words (the majority of types), with a large class of intermediate-frequency terms in the middle.

the large majority of interactions are between frequent terms or between frequent and infrequent terms.


It is obvious that computing the distance between x t−i and each opinion word is expensive.

we run an off-the-shelf clustering algorithm over opinion words in the training set and then use the produced n c centroids to estimate the indicator score v I i of w t−i being an opinion word: where x t−i is the distributed representation of w t−i and c j is the centroid vector representation of j-th cluster.


The aspect decision of w t is also affected by relative position between w t−i and w t .

mIN employs the second weight v P to explicitly model their positional relevance and the initial weight for the i-th segment v P i is calculated as below: where n m is the number of hidden state in H O t .


It tries to use the current word to predict the surrounding context words, where the prediction is defined over the embeddings of these words.

it learns the word embeddings by maximizing the likelihood of predictions.


The second proposed model uses Bayes's rule to reverse the probabilities P (y|x) = P (x|y)P (y) to get argmax y P (y|x) = argmax y P (x|y)P (y).

we have a reverse model of the probability P (x|y) that the given root words were generated from the portmanteau and a character language model model P (y).


Concept maps can be used to concisely represent important information and bring structure into large document collections.

we study a variant of multidocument summarization that produces summaries in the form of concept maps.


Instead, user studies (Chin et al., 2009;Kang et al., 2011) show that they note down important keywords and phrases, try to identify relationships between them and organize them accordingly.

we believe that the study of summarization with similarly structured outputs is an important extension of the traditional task.


This could be done by manually annotating spans representing concepts and relations, however, the size of our clusters makes this a huge effort: 2288 sentences per topic (69k in total) need to be processed.


the round 2 utterance has no meaning independent from round 1).

symbols are reused across tasks to describe different attributes (i.e.


well-being of families and on societies in general.

identifying individuals at risk of selfharm and providing support to prevent it remains an important problem (Ferrari et al., 2014).


Additionally, users in acute distress need prompt attention, and any delay in responding to these users could have adverse consequences.

identifying individuals at risk of self-harm in such support forums is an important challenge.


We attempted to paraphrase this post, but found that any paraphrase containing general language related to both the type of abuse and to the user's comments was enough to identify the user.

to protect the anonymity of the users in our dataset, we do not publish posts in any form.


We assume that the evaluation set is identically distributed to the training set.

if o is positively correlated with ), we say bias has been amplified.


Solving such a giant inference problem with constraints is hard.

we present an approximate inference algorithm based on Lagrangian relaxation.


Solving a constrained optimization problem on such a scale is difficult.

we consider relaxing the constraints and solve Eq.


It is well established that textual information can be predictive of age, race, gender, and many other social factors of the author (Koppel et al., 2002;Burger et al., 2011;Nguyen et al., 2013;Weren et al., 2014;Verhoeven and Daelemans, 2014;Rangel et al., 2016;Blodgett et al., 2016), or even the audience of the text (Voigt et al., 2018).

any system that incorporates raw text into its decision process is at risk of indirectly conditioning on such signals.


Leakage via Embeddings Even though we found out the RNN is much more responsible to the leakage then the Embedding, those still contribute to the leakage and are easier to inspect.

we turn to inspect the encoders' Embedding.


As previously discussed, we have M reporting articles per claim.

once we have the per-article credibility scores from our model, we take an average of these scores to generate the overall credibility score for the claim: This aggregation is done after the model is trained.


Our primary objective is to measure the relationship between identity and support in online social systems.

we next develop a classifier to automatically label replies with support.


Wellman and Wortley (1990) note that the availability of support is related to social distance, which is in part expressed linguistically through the degree of formality (Hovy, 1987;Sigley, 1997).

we include features from Pavlick and Tetreault (2016), which examined linguistic markers of formality.


Advice giving is a core component of many theories of support (MacGeorge et al., 2011) and such advice is frequently wrapped in politeness language (Feng et al., 2013), e.g., hedging suggestions rather than imposing direction, which provides face-saving opportunities for the person receiving support (Clark and Schunk, 1980).

we include the feature set of Danescu-Niculescu-Mizil et al.


Data Individuals convey their gender in multiple ways beyond using gender-associated names.

to capture this variety, we collect usernames from two online platforms where users have self-declared their perceived gender.


(2016) 0.6520 0.5216 Table 4: Gender inference (Macro-F1) Gender can also manifest through more subtle, stylistic cues (e.g., Schnoebelen, 2012;Flekova and Gurevych, 2013;Bamman et al., 2014;Volkova et al., 2015;Garimella and Mihalcea, 2016;Carpenter et al., 2016).

even when a person chooses a neutral username, their linguistic style may reveal their gender.


Thus, even when a person chooses a neutral username, their linguistic style may reveal their gender.

we construct a regression model to infer the degree to which either gender is expressed through text.


At each stage of combination, we tried both summing and averaging.

our recent history representations were built by aggregating either word embeddings or SPLex scores, which maintained their dimensionality of 300 or 2, respectively.


Additionally, we remove the 16 examples labeled with 4 from our NOT-ENTAILED examples in this pruned test set since some of these examples are arguably entailments.

the test set contains 761 examples, out of the original 3, 670 test examples.


the author intends that we consider them in relation to each other, inviting us to note, for example, a disparity of wealth distribution between demographic groups.


6 We found after 1000 examples that the annotators tended to have high agreement, also generally choosing found endings over generations (see Table 2).

we collected the remaining 112k examples with one annotator each, periodically verifying that annotators preferred the found endings.


To start, a piece of text t (t ∈ S ∪ {x}) is represented as a sequence of l hidden states, forming a feature map T ∈ R d×l , where d is the dimensionality of hidden states.

we first stack a vanilla CNN (convolution & max-pooling) (LeCun et al., 1998) over T to get a representation for t. each evidence candidate s i has a representation s i , and the claim x has a representation x.


Formally, this module is defined by the process function f H : where at stage k, a t is a set of attention weights which are inferred based on the previously assigned attention weights a .

the highlights at a specific stage k will be dependent on previous highlights.


The output of the is softmax activated to produce attention weights a [k] t at every stage k of the multistage fusion process: is fed as input into the HIGHLIGHT LSTM at stage k + 1.

the HIGHLIGHT LSTM functions as a decoder LSTM (Sutskever et al., 2014;Cho et al., 2014) in order to capture the dependencies on previous attention assignments.


Therefore, the attention weight between the n-th word h s n and the t-th video frame h v t is determined by not only the content of the video and sentence but also their interaction status.

we design one network to compute the relevance score of one video frame with respect to each word: where vector w, matrices W * , bias vector b, and bias c are the network parameters to be learned.


Second, Inception-V4 performs generally better than VGG16 and C3D, which is consistent with the finding in (Canziani et al., 2016).

more powerful visual representations of video features will undoubtedly improve the the performance of our proposed TGN on the NSGV task.


Datasets after OntoNotes, such as WikiCoref (Ghaddar and Langlais, 2016), are seldom studied.

we mainly compare PreCo with OntoNotes in this paper.


For example, two co-referred mentions could be far apart and require careful searches, and an annotator may miss it.

we further refine annotations as shown in Figure 3.


We call an error case LFW if there are low-frequency words 3 in its related mentions 4 .

the lower LFW rate a training set contains, the more precisely it may expose the drawbacks of the algorithm.


There is an obvious difference between the F1 scores achieved with all gold mentions and non-singleton gold mentions.

the room for improvement by better mention detection is not as enormous as suggested in (Lee et al., 2017).


For example, the CWS task splits " ¯•:: (Houston Airport)" into " ¯• (Houston)" and ":: (Airport)", while the NER task takes " ¯•:: (Houston Airport)" as a whole entity.

how to exploit task-shared information and prevent the noise brought by CWS task to Chinese NER task is a challenging problem.


It is used in tasks and domains for which we do not necessarily have coreference annotated corpora.

generalization is of special importance for coreference resolution.


A decision tree does not represent all patterns of data.

it is not possible to explore all feature combinations from a decision tree.


We propose an efficient algorithm to determine informative feature-values in large datasets.

of a better generalization, we achieve state-of-the-art results in all examined outof-domain evaluations.


However, as highlighted in several prior research efforts (Alex et al., 2007;Finkel and Manning, 2009;Lu and Roth, 2015), mentions may overlap with one another in practice.

models based on such a simplified assumption may result in sub-optimal performance for a down-stream task when they are deployed in practice.


Particularly, we build a bilingual entity network from inter-language links 1 in KBs for regularizing cross-lingual entities through a variant of skipgram model (Mikolov et al., 2013c).

monolingual structured knowledge of entities are not only extended to cross-lingual settings, but also augmented from other languages.


The bilingual EN G en−zh merges entities in different languages into a unified network, resulting in the possibility of using the same objective as in mono-lingual ENs.

we naturally extend mono-lingual function to cross-lingual settings: where C ′ (e y i ) denotes cross-lingual contextsneighbor entities in different languages that linked to e y i .


Thus, we naturally extend mono-lingual function to cross-lingual settings: where C ′ (e y i ) denotes cross-lingual contextsneighbor entities in different languages that linked to e y i .

by jointly learning mono-lingual representation with cross-lingual entity regularizer, words and entities share more common contexts, and will have similar embeddings.


same entities in art belling errors incre almost irrelevant to at filtering out wro smaller weights an weights.

we ilarity between s y k,l where sim is sim use cosine simila per.


Suppose that sentences {s en k,l |l ∈ L} contain the same entities in articles of entity e y m, the wrong labelling errors increase because some of them are almost irrelevant to e y m. Knowledge attention aims at filtering out wrong labelled sentences through smaller weights and related sentences with higher weights.

we define it proportional to the similarity between s y k,l and e y m: ψ(e y m , s y k,l ) ∝ sim(e y m , where sim is similarity measurement, and we use cosine similarity in the rest of the paper.


For example (Figure 1), there will be no direct relation between Chinese entity 福 斯特 (Foust) and English entity Piston by merely adding the equivalence relation between Foust and 福斯特, which is in contradiction with the fact that Foust belongs to Piston, no matter in which language.

we build bi-lingual entity network by making cross-lingual linked entities that inherit all relations from each other.


Thus, if a sentence also mentions another entity, it implicitly expresses their relation.

we make a similar assumption as in relation extraction: If two entities participate in a relation, and both of them joint inference among knowledge base and text across languages, capturing mutually complementary knowledge.


We define it according to the maximum similarity:  while, abundant text corpus contains large amount of potential knowledge complementary to existing KBs.

researchers leverage both types of resources to improve various natural language processing (NLP) related tasks, such as relation extraction (Weston et al., 2013;Lin et al., 2017), and entity linking (Tsai and Roth, 2016;Yamada et al., 2016;Ji et al., 2016).


we make a similar assumption as in relation extraction: If two entities participate in a relation, and both of them  2016; Ji et al., 2016Ji et al., ).


we make a similar assumption as in relation extraction: If two entities participate in a relation, and both of them where x y i is either a word or an entity, and C(x y i ) denotes: (i) contextual words in a pre-defined window of x y i if x y i ∈ D y , (ii) neighbor entities that linked to x y i if x y i ∈ G y , (iii) contextual words of w y j if x y i is entity e y i in an anchor ⟨w y j , e y i ⟩ ∈ A y .


However, we argue that this may mislead to an inconsistent training ob-Foust belongs to Piston, no matter in which language.


we make a similar assumption as in relation extraction: If two entities participate in a relation, and both of them 4   324   325   326   327   328   329   330   331   332   333   334   335   336   337   338   339   340   341   342   343   344   345   346   347   348   349   3   3   3   3   3   3   3   3   3   3   3   3   3   3   3   3   3   3   3   3   3  neighbors NBA, All-star and NBA 选秀 (draft), etc.


we make a similar assumption as in relation extraction: If two entities participate in a relation, and both of them < e en,Kobe , e zh,Kobe > < e en,Joe , e zh,Joe > ing (Tsai and Roth, 2016;Yamada et al., et al., 2017;Ji et al., 2016).


For example (F will be no direct relation between Ch 斯特 (Foust) and English entity Pis adding the equivalence relation betw 福斯特, which is in contradiction w Foust belongs to Piston, no matter guage.

we build bi-lingual ent making cross-lingual linked entities relations from each other.


The intuition is that we consider in a Wikipedia article has a pseud the page entity (talking something tity) (Yamada et al., 2017).

if mentions another entity, it implic their relation.


Thus, if mentions another entity, it implic their relation.

we mak sumption as in relation extraction ties participate in a relation, and at filtering out wrong labelling sentences, and ψ ′ (w en i , w zh j ) is cross-lingual attention to deal with the unbalanced information through possible aligned words.


Recently, the surge of cross-lingual analysis pushes the entity linking task on crosslingual settings .

we comprehensively measure our joint inference ability among words and entities using the tri-lingual EL benchmark dataset KBP2015, which consists of 944 documents and 38,831 mentions, and divides them into 444 and 500 documents for training and evaluation.


This construction of w r provides an additional shared cross-lingual layer, added to the translated pivot features of the output layer.

an inherent limitation of the AEbased models when it comes to CLCD transfer, is that they cannot be employed in the lazy setup.


Each character is represented by a d-dimensional lookup embedding.

each input sequence is represented as a feature map of dimensionality d × k. We use the convolution layer to learn the representation for each sliding n-gram characters.


(2015) further show that enforcing the linear mappings to be orthogonal matrices achieves higher performance.

we let our encoders M l be orthogonal linear matrices, and the corresponding decoders can be obtained by simply taking the transpose: Thus, applying the encoder or decoder to an embedding vector is accomplished by multiplying the vector with the encoder/decoder matrix.


Computationally, when compared to the BWE-Pivot and BWE-Direct baselines, one step of MAT training costs similarly to N BWE training steps, and in practice we train MAT for the same number of iterations as training the baselines.

mAT training scales linearly with the number of languages similar to BWE-Pivot (instead of quadratically as in BWE-Direct).


1: repeat 2: Select at random lang j ∈ L 5: loss += Lr(ti, tj) refinement loss 9: Update all M parameters to minimize loss 10: orthogonalize(M) see §3.3 11: until convergence When learning MWEs, however, it is desirable to go beyond aligning each language with the target space individually, and instead utilize the relations between all languages as we did in MAT.

we in this section propose a generalization of the existing refinement methods to incorporate a multilingual objective.


As mentioned in §3, orthogonal linear mappings are the preferred choice when learning transformations between the embedding spaces of different languages (Xing et al., 2015;Smith et al., 2017).

we perform an orthogonalization update (Cisse et al., 2017) after each training step to ensure that our mappings M are (approximately) orthogonal: where β is set to 0.001.


The higher similarity between word vectors can indicate similar meanings of words.

embeddings that encode semantics have been shown to serve as the good initialization and benefit several NLP tasks.


We linearly combine the contextual information from different languages as: The likelihood of selecting each sense identity z ik for w i can be formulated in the form of Bernoulli distribution with a sigmoid function σ(•): where Q en is a 3-dimensional tensor with each dimension denotes W en , z ik for a specific word i in W en , and the corresponding latent variable, respectively.

q en ik will retrieve the latent variable of k-th sense of i-th English word.


Our preliminary experiments corroborate this intuition.

in this work we also opt for implementing G(x; θ G ) as a deep neural network.


Hypernyms, in addition, are the backbone of lexical ontologies (Yu et al., 2015), which are in turn useful for organizing, navigating and retrieving online content (Bordea et al., 2016).

we propose to evaluate the contribution of cross-lingual embeddings towards the task of hypernym discovery, i.e., given an input word (e.g., cat), retrieve or discover its most likely (set of) valid hypernyms (e.g., animal, mammal, feline, and so on).


Namely, we can assume that 1) some information was not communicated by the original sentence, 2) that information should have been communicated (according to a human editor), and 3) that information is communicated by the inserted phrase.

we believe that a large data set of such edits is inherently valuable for researchers modeling inference and discourse She died there in 1949 after a long illness.


To do this, we focus our attention specifically on the English, Spanish, and German subcorpora, as these are languages for which we could find a sufficient number of native speakers to perform the necessary annotation for our analysis.

the discussion and results in this section may not be representative of the other languages in the corpus.


However, the majority of the PW languages are similar from the perspective of genealogy (26/40 are Indo-European), geography (28/40 are Western European), and typology (26/40 are fusional).

the PW set is not a representative sample of the world's languages.


The cross-lingual links play an important role to bridge the language gap in a multilingual KG; however, not all the equivalent entities are connected by cross-lingual links in most multilingual KGs.

increasingly more research work studies the problem of cross-lingual KG alignment, aiming to match entities in different languages in a multilingual KG automatically.


But KGs are relational multi-graphs, entities are connected by typed relations.

we design a particular method for computing A of a KG; we let a ij ∈ A indicate to what extent the information of alignments propagates from the i-th entity to the j-th entity.


However, most languages still do not have sememe-based linguistic knowledge bases.

we present a task of cross-lingual lexical sememe prediction, aiming to automatically predict sememes for words in other languages.


However, most languages do not have such sememe-based linguistic KBs, which prevents us understanding and utilizing human languages to a greater extent.

it is important to build sememe-based linguistic KBs for various languages.


However, most WRL methods learn distributional information of words from large corpora while the valuable information contained in semantic lexicons are disregarded.

some works try to inject semantic information of KBs into WRL (Faruqui et al., 2015;Mrkšic et al., 2016;Bollegala et al., 2016).


Formally, we define the score function P (x j |w T t ) of sememes x j given a target word w T t as: where r s is the descending rank of word similarity cos(w S s , w T t ) for the source word w S s , and c ∈ (0, 1) is a hyperparameter.

c rs is a declined confidence factor which can eliminate the noise from irrelevant source words and concentrate on the most similar source words when predicting sememes for target words.


3, the largest predicted probabilities at each decoding step max(p N i (V T )) are respectively 0.8, 0.35 and 0.75.


we design the dynamic weight λ i according to the exception rate r(x j ): where β γ is a learnable parameter.


If the entropy of probability distribution − K k=1 p L k logp L k > E 0 ( E 0 = 1.5 in this paper), we treat this word as an ambiguous word.

the sentences containing troublesome words can be divided into four different parts: 1) sentences which contain both lowfrequency and ambiguous words (Low+Amb, Rectify Deterio Arthur(test) 51 17 Baseline+MEM 70 11 Table 5: The numbers of rectification (Rectify) and deterioration (Deterio) caused by different models.


Sentence structure can be viewed as starting with an action or state (described via verbs or other predicates) and the entities or propositions involved in that activity or state (usually described via arguments to verbs).

certain words in the output translation, like verbs, are crucial to the understanding of the target language sentence but only provide marginal value in n-gram matching evaluations like the BLEU score.


But it is more challenging to obtain the parallel utterance data u f t from u e t ).

we have to leverage external bilingual data sources to alleviate the problem.


The teacher-student optimization only adjusts the weights related to the language-specific parts in Figure 3 (i.e., utterance encoding and context gating).

the shared weight ||W y || is seen as a constant.


It is also worth mentioning that our XL-NBT algorithm only takes several hours to achieve the reported score, while the translator model takes much more time and memory to train depending on the complexity.

the simplicity and efficiency makes our model a better fit for rarelanguage and limited-budget scenarios.


The encoder-decoder abstraction makes it conceptually feasible to build a system that maps any source sentence in any language to a vector representation, and then decodes this representation into any target language.

various approaches have been proposed to extend this abstraction for multilingual MT (Luong et al., 2016;Dong et al., 2015;Johnson et al., 2017;Ha et al., 2016;Firat et al., 2016a).


It learns language embeddings as a context for translation and uses them to generate the parameters of a shared translation model for all language pairs.

it provides these models the ability to learn parameters separately for each language, but also share information between similar languages.


This is because the same per-language parameters are used to translate to and from a given language, irrespective of the language at the other end.

as long as we train our model using some language pairs that involve a given language, it is possible to learn to translate in any direction involving that language.


Since the textual styles, sentence structures and terminologies in different domains are often remarkably distinctive, whether such domainspecific translation knowledge is effectively preserved could have a direct effect on the performance of the NMT model.

how to simultaneously exploit the exclusive and shared translation knowledge of mixed-domain parallel sentences for multi-domain NMT remains a challenging task.


The Chinese words "'OE¬"(congress), "AE Y"(bills), " \"(inclusion), and "AE §"(agenda) are frequently used in Laws domain and imply the Laws style of the sentence, while other words in this sentence are common in all domains and they mainly indicate the semantic meaning of the sentence.

it is reasonable to distinguish and encode these two types of words separately to capture domain-specific and domain-shared contexts, allowing the exclusive and shared knowledge to be exploited without any interference from the other.


Third, +WDC(T) also outperforms DL4NMT, revealing that it is reasonable and effective to emphasize domain-specific words in model training.. Last, +WDC achieves the best performance when compared with both +WDC(S) and +WDC(T).

we believe that word-level domain contexts on the both sides are complementary to each other, and utilizing them simultaneously is beneficial to multi-domain NMT.


Besides, our method is also general to other NMT models.

we plan to apply our method to the NMT with complex architectures, for example, lattice-to-sequence NMT , hierarchy-to-sequence NMT (Su et al., 2018), NMT with context-aware encoder  and Transformer (Vaswani et al., 2017) and so on.


they will be the nearest neighbor to a disproportionate number of other vectors (Radovanović et al., 2010).

if we allow oneto-many alignments, we will find the embeddings of certain elements of V src acting as hubs, i.e.


Tricks from combinatorial optimization make it possible to maximize over all bipartite matchings in polynomial time.

we fall back on the Viterbi approximation for the E-step (Brown et al., 1993;Samdani et al., 2012).


However, the property also hinders its parallelizability thus RNNs are slow to execute on modern hardware optimized for parallel execution.

a number of more parallelizable sequence models were proposed such as ConvS2S (Gehring et al., 2017) and the Transformer (Vaswani et al., 2017).


The joint neighborhood discovery and word translation objective is given by: The neighborhood discovery part of the objective, R(θ), does not depend on availability of supervised data and only requires monolingual data on the source language side.

we can discover neighborhoods in an unsupervised manner on a large set of monolingual word embeddings, then initialize using this pre-trained D which is then jointly optimized with the translation part of the objective L(θ).


While a number of prior work proposed various approaches for retrieval, which have been shown to improve accuracy by a few points, we compare all methods using the same retrieval method, nearest neighbor.

for (Conneau et al., 2018), we report the results for the variant of their method called: adv -Refine -NN.


Nevertheless, a drastic decrease arises, when we perform the 4th-pass decoding.

it's necessary to introduce a flexible multi-pass decoding, which has the ability to adaptively choose the suitable decoding passes.


That is, when the RNN decoder generates the t-th word y t in decoding phase, only y <t can be utilized, while the possible words y >t are directly neglected.

it's difficult to capture global information especially the ungenerated words for the current dominant RNN decoder without new significant innovation.


These results are consistent with above conclusion drew according to the decoding depth.

it's proven the necessity of our proposed auxiliary policy network to choose the decoding depth.


Given the k-th source sentence x (k) , we use x i ∈ R D×1 to denote the vector representation of the ith source word x i , which is a sum of word embedding and positional encoding.

the initial matrix representation of where X ∈ R D×I is the concatenation of all vector representations of source words.


Increasing the number of self-attention layers does not lead to any improvements.


In order to correctly answer questions, these phrase representations (index vectors) need to correctly encode their meaning with respect to their context.

pIQA constraint enforces evaluating research in document comprehension and phrase representation learning.


However, whereas their main focus is on re-ranking retrieved sentences to maximize the rewards of correctly answering the questions, our focus is to increase the answer recall of paragraphs with less noise.

our work is complementary to the work of Wang et al.


Plot (c) depicts the average number of documents that contain the ground-truth answer.

the recall lowers with increasing corpus size, yet this not necessarily compromises a top-n system, as it often contains the correct answer more than once.


First, we establish that deep question answering is subject to a noiseinformation trade-off.

the number of selected documents in deep QA should not be treated as fixed, rather it must be carefully tailored to the QA task.


First, the nodes and edges of CG are (sequences of) linguistic expressions, such as tokens, lemmas, phrases, etc.

the nodes and edges of CG might exhibit compositionality and ambiguity, while KG does not have those properties.


Theoretical analysis has revealed that adversarial training tries to minimize the Jensen-Shannon (JS) divergence between the real and fake distribution.

one can view our model as replace KL divergence in Equation 1 with JS divergence and change the Gaussian prior to the target distribution.


The optimization problem is a linear assignment problem (LAP).

we can efficiently solve the algorithm by using the Hungarian method (Kuhn, 1955).


We note that while various methods in the literature were initially presented as fully supervised (Mikolov et al., 2013), semi-supervised (using a seed dictionary) (Haghighi et al., 2008), or unsupervised (Zhang et al., 2017b), most of them can be straightforwardly adapted to run in any of these settings.

we evaluate all methods both in the unsupervised setting in which we are primarily interested, and also the commonly evaluated semi-supervised setting with 500 seed pairs.


This is maybe because 'angry' is caused by more various events and it is more difficult to capture and utilize those cause events.

it is necessary for the emotion classification to have an encoder which can extract the eventbased information of emotion cause from texts.


When issuing an opinion, people tend to center on several keywords which convey the main idea.

in some sense, self attention is a kind of dependency parsing that drives the model to focus on salient parts of the context.


Findings indicate that our proposed approach leads to a small performance boost after using the topic embeddings.

our simand MISC) in the original publications.


It suggests that the utterance-level dependency greatly benefits the learning of wordlevel dependency.

it is expected that the AEM+Attention model achieves the best G-score.


It shows that the proposed model is sensitive to the unseen utterance representations.

we would like to explore more approaches to address this problem in the future work.


However, in our experiment, many OOD sentences were misclassified into IND because OOD sentences were not in the negative examples of the classifiers.

the confidence scores of the basic binary classifiers are not sufficiently reliable evidences of OOD.


When we train G to generate sentences similar to IND sentences and D to classify real IND sentences and fake sentences generated by G, we expect D to reject OOD sentences.

we use the low confidence score by D about an input sentence as the evidence that the sentence is OOD.


We define G that generates fake data , and D that measures the probability of either f(x) or f(G(z)) from the real data as  Because G(z) continuously changes during the training, f of traditional GAN also changes.

we define C ∈ R |D| that computes domaincategory of f(x) as where W c is a weight matrix and b c is a bias vector.


Blender combines the benchmark with U. Exp2Imp transformation is performed for the instances in U.

blender mixes the true implicitly-related argument pairs and all the counterfeits in U.


In these systems, an important aspect is the output summary length, which may vary according to user needs.

output length has been a common tunable parameter in pre-neural summarization systems and has been incorporated recently in few neural models as well (Kikuchi et al., 2016;Fan et al., 2017;Ficler and Goldberg, 2017).


On the other hand, generating words from a decoder state is based on timeconsuming calculations of a softmax distribution, given that the vocabulary size is relatively large.

we would like to try abandoning decoder word generation, while just directly copying words from the input document instead.


It is thus non-trivial to use it for policy learning in RL framework directly.

we propose to use a classification-based scorer, in which different score categories and their probabilities constitute an action space.


the rank information that is deemed to be important for essay scoring.

we enforce a penalty in addition to the cross entropy loss.


Identifying a well-formed question should be easy by parsing with a grammar, such as the English resource grammar (Copestake and Flickinger, 2000), but such grammars are highly precise and fail to parse more than half of web queries.

in this paper we present a model to predict whether a given query is a well-formed natural language question.


To the best of our knowledge, (Lee and Dernoncourt, 2016;Ortega and Vu, 2017;Tran et al., 2017) are the latest approaches in dialog act classification, which also reported on the same data splits.

we compare our research against these works.


As deep learning models usually contain a large number of parameters that need to be learned, to prevent over-fitting (Lawrence et al., 1997;Srivastava et al., 2014) we usually need a large dataset to train upon.

we curated a bigger dataset with hierarchical labels from Wikipedia meta information provider DBpedia 5 .


We noticed that using the default hyperparameters of the Structured Self-attentive classifier with high attention hops (m >= 8) performed poorly compared to use just one attention hop (m = 1).

we reported the results of using one attention hop (m = 1) as our baselines for fair comparison.


For example, there are 80 parent nodes in the taxonomy of the DBpedia dataset and HDLTex needs to build 80 RNNs, where each subclassifier contains around 67 million parameters.

we can barely fit the whole model of HDLTex on our CPU 7 because it requires 60 GB RAM to build these 80 deep neural networks.


However, Supervised Anchors requires topic assignments for each training document 3 for use as features for some downstream classifier.

the process of building a classifier scales linearly with the number of documents and can be time consuming compared to topic recovery.


Since the harmonic mean tends towards the lowest values, the resulting pseudoword anchor largely ignores superfluous cooccurrence patterns in the constituent words.

while individual words forming the anchor may be ambiguous, users can combine multiple ambiguous words to intuitively express a single coherent idea.


We record each step of the user interactions and find a Pearson correlation coefficient of .88 between development accuracy and test accuracy.

labeled Anchors allows participants to interactively see updated classification accuracy and have confidence that held-out test accuracy will also improve.


As has been noted in several publications (Chang et al., 2009), optimization for perplexity alone tends to negatively impact topic coherence.

without introducing topic coherence as a training objective, topic modeling likely produces sub-optimal results.


As one can imagine, similar words would have similar vector representations in E and would have similar weights in each column of W .

the factorization constraint encourages words with similar meaning to be selected or de-selected together thus potentially improving topic coherence.


In this neighborhood, NTM-R substantially outperforms the GSM, NVDM and NTM baseline models.

we consider NTM-R the best model overall.


Since this depends solely on the n-gram LM, the true conditional probability under Markov assumption is not modeled and, therefore, context dependency beyond the window of (n − 1) is ignored.

attempting to utilize a higher amount of context can lower the probability of some tokens resulting in poor scores.


However, crucially, since exp(•) > 0, every tree is assigned strictly nonzero probability.


A convolution filter is typically implemented as a linear affine transformation followed by a nonlinear function, which fails to account for language compositionality.

it limits the use of high-order filters that are often warranted for natural language processing tasks.


As in conventional CNNs, the computation of the convolution operation with RNFs can be easily parallelized.

rNF-based CNN models can be 3-8x faster than their rNN counterparts.


However, BASELINE separately models these features using two distinct TreeL-STMs.

the non-terminal tree nodes only retain only one type of syntactic information propagated from their descendants in the tree.


Though we could select successive nodes in the AST as retrieved pieces, such as [assign; expr * (targets); expr] from Figure 1, we would miss important structural information from the rules that are used.

we choose to exploit actions in the generation model rather than AST nodes themselves to be candidates for our retrieved pieces.


The difference is due both to the fact that ALL syn groups together the output of 4 (syntactically driven) generation models and to the input syntactic constraint, which ensures greater diversity.

in the example shown, ALL syn yields 15 paraphrases each with strong syntactic differences as summarised below.


However, we found that most boundaries are actually only influenced by nearby EDUs, thereby forcing the model to attend to the whole sequence will bring in unnecessary noises.

we propose a restricted self-attention mechanism, which only collects information from a fixed neighborhood.


Training parts of both datasets are labeled with DS, and testing parts are annotated by human.

they are suitable for evaluating how our model can narrow the gap between DS and ground-truth context-dependent labels.


This is because LME is capable of evaluating the appropriateness of each label and distinguishing context-dependent ones from global-but-context-irrelevant ones.

lME helps the system to focus on more reasonable types.


All these embeddings except pre-trained word embedding are randomly initialized and optimized during training.

the input layer is a sequence of word representations V = {ṽ 1 ,ṽ 2 , ...,ṽ n }.


From equation L dif f , we can see that the f p s , f p t would be easily optimized to zero if we did not place a constraint, in which case the model would fail to train.

we further reconstruct the input sentence from both genre-specific features and genre-agnostic features.


As technology progresses, we are increasingly collaborating with AI agents in interactive scenarios where humans and AI work together as a team, e.g., in AI-assisted diagnosis, autonomous driving, etc.

far, AI research has typically only focused on the AI in such an interaction -for it to be more accurate, be more human-like, understand our intentions, beliefs, contexts, and mental states.


As the subject is typically the theme (or topic) of a the clause (Quirk et al., 1985), the intuition here is that facts with the same subject express information about the same entity.

each cluster will contain a ranked set of facts about each entity in the document.


we adapt a simplified Attention Sum (AttSum) reader throughout all experiments.


Through careful observation (detailed in Section 5.2), we find that each model has a speciality in solving problems.

we propose an ensemble model which selects the result according to models' generation probability: where y = {y 1 , ..., y T } is the target sequence, and x = {x 1 , ..., x S } is the source sequence.


The question focus network applies convolutions that operate on windows centered on each question word.

the input and output resolutions are the same.


For a training set X = {x 1 , x 2 , ..., x m } of m samples and a test set Y = {y 1 , y 2 , ..., y n } of n samples, such that X ∩ Y = ∅, each component in the full kernel matrix is defined as follows (step S2 in Figure 2): where z i and z j are samples from the set We then normalize the kernel matrix by dividing each component by the square root of the product of the two corresponding diagonal components: We transform the normalized kernel matrix into a radial basis function (RBF) kernel matrix as follows: As the kernel matrix is already normalized, we can choose σ 2 = 0.5 for simplicity.

equation (4) becomes: each row in the RBF kernel matrixK is now interpreted as a feature vector, going from step S2 to step S3 in Figure 2.


Using this labeling scheme, in the restaurant data, 1034 out of 1117 test points have the same sentencelevel and aspect-level labels.

a sentencelevel classifier with accuracy 75% also classifies 70% aspect-labels correctly.


Intuitively, any sentiment word also indicates emotion and should not be ignored by our emotion classification task.

we propose a shared-private (SP) model as shown in Fig 1.c, where we employ a shared LSTM layer to extract shared sentiment features for both sentiment and emotion classification tasks, and a target-specific LSTM layer to extract specific emotion features that are only sensitive to our emotion classification task.


However, without any constraints, the two feature spaces may both tend to pay more attention to frequently occurring and important sentiment words like great and happy, but less to those rarely occurring but crucial emotion words like anxiety and panic.

to encourage the two feature spaces to focus on sentiment words and emotionspecific words respectively, we propose using the attention weights computed from the shared layer as extra inputs to compute the attention weights of the target-specific layer.


These words are easily kept and then the decoder barely depends on sentiment-memories.

it is difficult to handle the sentiment transformation with these examples.


Therefore the amount of available training data is limited.

a good model for ABSA has to make the best possible use of the available data.


So we do not split them into shorter units (sentences).

the second hierarchy level of (Ruder et al., 2016), that combines the output of consecutive sentences in a document, is superfluous and omitted in our experiments.


The design choices for the BiLSTM in this step remain the same as in the baseline model.

then, we transform the feature vector v extracted from the text X to a score vectorŷ (a) for each aspect a ∈ A and apply softmax normalization:ŷ (a)  where for each aspect, we predict its presence or absence as well as its polarity in one step: the loss is simply the cross entropy summed over all aspects: with   Keeping the formalization as an end-to-end task, we replace the BiLStM by a convolutional neural network (CNN) as described in (Kim, 2014).


The main assumption behind such statistical approaches is that keywords contain essential information to infer the sentiment of a whole sample.

this type of statistical approach does not readily consider complex syntactic interactions between individual words; instead, the main focus lies in the system's learning of the relevant knowledge through texts relevant to the sentiment analysis task.


Next, we note that the sentences that we tested are general sentences whose components could be found in any linguistic corpus; in fact, CoreNLP's treebank does contain annotated sentences containing the "... has never ..." or "... has no ..." constructions, and so the model should correctly analyze such sentences.

we draw the conclusion that these models lack an understanding of long-term dependencies, and based on our experiments, they specifically fail to understand the c-command relation (Radford, 2004).


Finally, we remind readers that the c-command relation is associated with Chomskyan grammars; and need not be necessary within other models of syntax.

learning the c-command relation may lead to better analyzers, but it need not be the only way to improve performance.


Moreover, reducing noise in the input of RTE module, by de-noising the DR and SR modules, appears to be crucial for improving the overall performance.

our ranker outperforms the TF-IDF method by 38.8% in evidence retrieval F1, with 65 times faster inference speed, achieving a promising performance in a large-scale fact extraction and verification dataset.


However, popularity is not solely determined by content, e.g., a text or an image, but is highly affected by contexts, e.g., timing, and authority (Cheng et al., 2017;Burghardt et al., 2017;Suh et al., 2010;Hessel et al., 2017;Jaech et al., 2015).

to utilize popularity as an indicator of informativeness, we need to exclude the effect of context.


Because informativeness of each post is annotated via crowdsourcing, the extracts can be ranked, but the appropriate number is unknown.

we employ ranking metrics in the evaluation.


Karma scores follow Zipf's law (Cheng et al., 2017).

we smooth the karma scores as follows: where k represents the karma score.


As explained in Sec 3, the informativeness of each post is annotated via crowdsourcing and it is difficult to determine the appropriate number of posts needed to create a summary.

we employ ranking metrics for evaluation.


Initial results were less satisfying and most errors occurred because mono-varietals were predicted to be blends and vice versa.

we explored two solutions: (1) we separated our data into mono-varietals (Model 1), and blends (Model 2), and trained HSLDAd separately; (2) we created a smaller yet more balanced training set regarding mono-varietals and blends (Model 3).


The utterances are corrupted by background noise, and a large portion of them are accented.


The size of WFSAs needed to recover QRNN grows exponentially in the window size.

at least for QRNNs, Proposition 11 has more conceptual value than practical.


Rather, we wish to study a more constrained class of methods to better understand the connections between WFSAs and RNNs.

in Definition 8, we restrict the semirings to be "simple," in the sense that both operations take constant time and space.


As the target of R is to make z sparse, it can be "turned-off" after achieving a satisfying sparsity.

we extend R 1 to a margin-based regularization: In addition, we also want to make up the relaxation made on z, i.e., relaxing its values from binary to [0, 1].


Typically, highway layers conduct nonlinear transformation as: where σ is the sigmoid function; ⊙ is the elementwise product operation; g is a nonlinear activation function; t is called transform gate and (1 − t) is called carry gate.

the input of the k-th GCN layers should be h (k) instead of h (k) .


Since dependency graph has 55 different edge labels, incorporating all of them overparameterizes the model significantly.

following Nguyen and Grishman, 2018;Vashishth et al., 2018a) we use only three edge labels based on the direction of the edge {forward (→), backward (←), selfloop ( )}.


However, they suffer from two problems: (1) The features they used often need to be manually designed and may involve error propagation from existing NLP tools; (2) Sentence-level and document-level information are integrated by a large number of fixed rules, which is complicated to construct and it will be far from complete.

how to use a neural-based model to automatically extract sentence-level and document-level information and dynamically integrate them is another challenging problem.


"B" and "I" represent the position of the word in a trigger to solve the problem that a trigger word contains multiple words such as "take over", "go off" and so on.

the total number of tags is N t = 2 ⇤ |N eventT ype | + 1, where |N eventT ype | is the size of the predefined event types and |N eventT ype | = 33 in this paper as stated above.


The Second Tagging Layer: TLSTM2 Though the TLSTM1 can capture the interdependency between current event candidate and the former predicted event tags, it can not capture the interdependency between the current event candidate and the later predicted event tags.

we devise a second tagging layer (TLSTM2) upon the LSTM1 to capture the interdependency between the current event candidate and both of former and later predicted event tags from TLSTM1.


hibitively slow due to the large number of valency pattern combinations.

we limit the number of combinations for each token to the highestscoring 500.


1 Data annotation is a major bottleneck for the application of supervised learning approaches to many problems.

unsupervised methods that learn directly from unlabeled data are increasingly important.


(8) we know that only f −1 φ is required for accomplishing learning and inference; we never need to explicitly construct f φ .

we directly define the architecture of f −1 φ .


(2014) show that its Jacobian matrix is tri-angular with all ones on the main diagonal.

the Jacobian determinant is always equal to one (i.e.


We found that most tagging errors happen in noun subcategories.

we do the one-to-one mapping between gold POS tags and induced clusters and plot the normalized confusion matrix of noun subcategories in Figure 4.


This Hamming loss is computed as the size of the symmetric difference between the set of constituents γ and γ G in the trees t and t G , respectively.

the loss function is defined as: and, according to the authors, it can be efficiently computed for a non-binary bottom-up transition system by counting the individually unreachable arcs from configuration c (|U(c, γ G )|) plus the erroneous constituents created so far (|γ c \ γ G |):  We adapt the latter to efficiently implement a loss function for the top-down and in-order strategies.


Since T has m constituents, by induction hypothesis, T is a reachable set from configuration c. Since (X, l, r) is individually reachable by hypothesis, it must satisfy at least one of the conditions for constituent reachability.

as these conditions are different for each particular algorithm, we continue the proof separately for each: Top-down constituent-decomposability In this case, we enumerated three constituent reachability conditions, so we divide the proof into three cases: If the first condition holds, then the constituent (X, l, r) has already been created in c. it will still be present after applying any of the possible transition sequences that build T starting from c. Hence, T = T ∪ {(X, l, r)} is reachable from c. If the second condition holds, then j ≤ l < r and the constituent (X, l, r) can be created by l−j Shift transitions, followed by one Non-Terminal transition, r − l Shift transitions and one Reduce transition.


This will leave the parser in a configuration whose value of j is r, and where stack elements with left span index ≤ l (apart from those referencing the new non-terminal and its leftmost child) have not changed.

constituents of T are still individually reachable in this configuration, as their left span index is either ≥ r (and then they meet the second reachability condition) or ≤ l (and then they meet the third), so T is reachable from c. Finally, if the third condition holds, then we can create (X, l, r) by applying r − j Shift transitions followed by a sequence of Reduce transitions stopping when we obtain (X, l, r) on the stack (this will always happen after a finite number of such transitions, as the reachability condition guarantees that l is the left span index of some constituent already on the stack, and that (X, l) is on the stack).


Any smaller value would leave this node without enough children (which cannot be fixed later due to the leftto-right order in which we build the tree), and any larger value would create a node with too many children.

we can map negative values to a single label.


This is because an opening parenthesis followed by a closing parenthesis would represent a leaf node, and there are no leaf nodes between (• i ) and (• i+1 ) in the tree.

we can write α i as α i) α i( , where α i) is a string matching the expression [)] * and α i( a string matching the expression [(X] * .


A negative δ(β i ) = −k corresponds to closing k parentheses after β i , so the number of common ancestors will conversely decrease by k. A value of zero means no opening or closing parentheses, and no change in the number of common ancestors.

different parenthesized strings β 1 β 2 .


Then, w i+1 is the leftmost word in the second child subtree, and X is the lowest common ancestor of w i and w i+1 .

c i = X, and a tree with identical structure but a different nonterminal at that position will generate a label sequence with a different value of c i .


That is, we suppose that different languages use similar underlying syntactic/semantic graphs, but differ in how they realize this graph structure on the surface.

given a gold POS corpus u of the unknown target language, we may hope to explain its distribution of surface POS bigrams as the result of applying some target-language surface realization model to the distribution of cross-linguistically "typical" unordered trees.


All that we can optimize is the parameter vector of this model.

we deny ourselves the freedom to realize each individual tree in an ad hoc way.


These features are able to examine the tree's node labels (POS tags) and edge labels (dependency relations).

when a is a verb, the model can assign a positive weight to "subject precedes verb" or "subject precedes object," thus preferring orderings with these features.


As for the dummy nodes, a n is realized by the length-1 string h where h is the head token of node a, while a 0 and a n+1 are each realized by the empty string.

c an simply assigns count 1 to the bigrams BOS h and h EOS, and c a 0 and c a n+1 each assign expected count 1 to BOS EOS.


Furthermore, the single swap of adjacent nodes only changes 3 bigrams (possibly including boundary bigrams).

it is possible to obtain the marginal probabilities with O(1) additional work per permutation.


15 According to the family (and sub-family) information at http://universaldependencies.org.

we hope that the optimizer will find a systematic permutation that reduces the divergence.


However, "water drink boy" is presumably object-verb-subject.

using crosslingual embeddings, we would try to realize the unordered source trees so that their word strings, with few edits, can achieve high probability under a neural language model of the target.


In the testing phase, instead of using region proposals like (Wu et al., 2016a), we directly feed the whole image into the word prediction CNN in order to keep simple and efficient.

each image is encoded into a fixed-length vector, where each dimension represents the probability of the corresponding word occurring in the image.


Intuitively speaking, question-relevant explanations for images should be more likely to help predict right answers than irrelevant ones.

we propose another metric to measure the relevance between the words and the question.


The agent participates in a sequence of such interactions, and the models improve from labels acquired over multiple interactions.

the agent's expected reward per interaction is expected to improve as more interactions are completed.


By asking algorithms to answer different types of questions, ranging from object identification, counting, or appearance, to more complex questions about interactions, social relationships, or inferences about why or how something is occurring, we can evaluate different aspects of a model's multimodal semantic understanding.

several popular image-based VQA datasets have been introduced, including DAQUAR (Malinowski and Fritz, 2014), COCO-QA (Ren et al., 2015a), FM-IQA (Gao et al., 2015), Visual Madlibs (Yu et al., 2015), VQA (Antol et al., 2015), Visual7W , etc.


Retrieval: Due to the size of TVQA, there may exist similar questions and answers in the dataset.

we also implement a baseline two-step retrieval approach: given a question and a set of candidate answers, we first retrieve the most relevant question in the training set, then pick the candidate answer that is closest to the retrieved question's correct answer.


Since our questions are raised by people watching the videos, it is natural for them to ask questions about specific and unique objects/locations/etc., mentioned in the subtitle.

it is not surprising that TFIDF based similarity between answer and subtitle performs so well.


Additionally, temporal reasoning is crucial for answering the TVQA questions.

future work also includes integrating better temporal cues.


We seek to extensively study this aspect, particularly with respect to temporal prepositions (Pratt-Hartmann, 2004).

we collect the TEMPOral reasoning in video and language (TEMPO) dataset which builds off the recently collected DiDeMo dataset (Hendricks et al., 2017).


We argue that to do proper temporal reasoning, pre-determined, query independent context features may not cover all possible temporal relations.

we propose to model the context as a latent variable, allowing our method to learn which context moments to consider as a function of the video and importantly, the query.


Word similarity datasets have been one of the oldest, still most prominent, benchmarks for the evaluation and comparison of semantic representation techniques.

several word similarity datasets have been constructed during the past few decades; to name a few: RG-65 (Rubenstein and Goodenough, 1965), WordSim-353 (Finkelstein et al., 2002), YP-130 (Yang and Powers, 2005), MEN-3K (Bruni et al., 2014), SimLex-999 (Hill et al., 2015), and SimVerb-3500 (Gerz et al., 2016).


Context can also help to locate the key words in the gloss of the correct word sense.

we introduce a co-attention mechanism to generate co-dependent representations for the context and gloss.


In other words, we should pay more attention to the words which can "overlap" between the context and the gloss when generating the representations of context and gloss.

we introduce a co-attention mechanism to model the mutual influence between the representations of context and gloss.


All studies listed above show that integrating lexical resources especially gloss into supervised systems of WSD can significantly improve the performance.

we follow this direction and seek a new way of better integrating gloss knowledge.


Note that each element of A in the j-th column indicates the similarity between j-th gloss word and each context word.

we can get the attention weight for each context word through a softmax function across the column of A: where A :j denotes j-th column of A and A c :j denotes j-th column of A c ∈ R n×m .


Specifically, for the word-level co-attention model CAN w : For the sentence-level co-attention model For the hierarchical co-attention model HCAN in Figure 2, the outputs of the word and sentence level layer are merged together to generate the final results.

the final similarity score between i-th gloss and context is computed as the weighted sum of word-level score β w i and sentence-level score β s i : Meanwhile, the final context embedding vector is also generated by a combination of two levels' context embedding vector:ĉ w andĉ s .


In order to transfer from word-level encoding space to sentence-level encoding space, we introduce a non-linear projection layer on top of the wordlevel context vectorĉ w .

the final context embedding vectorĉ is generated bŷ In total, for the hierarchical co-attention model HCAN: During training, all model parameters θ are jointly learned by minimizing a cross-entropy loss betweenŷ and the true label y. where M is the number of examples in the dataset, N i is the word sense number of i-th example, y ij andŷ ij are the true and predict probability of the i-th example belongs to j-th label.


(2017b) map all the sense annotations in the training and test datasets to WordNet 3.0 via a semi-automatic method.

we choose wordNet 3.0 as the sense inventory for extracting the gloss.


Meanwhile, context can also help to focus on the words in gloss of the right word sense.

we propose a co-attention mechanism to model the gloss-to-context and contextto-gloss attention.


Annotating metaphors is not an easy task, due to the inherent ambiguity and subjectivity.

we investigate approaches for annotating novelty in metaphors, before closing the resource gap for token-based annotations by creating a layer of novelty scores.


3 We use a best-worst scaling factor of 1.5 and four items per tuple.

each metaphor appears in six different best-worst scaling comparisons.


Both Pearson correlation (r = 0.04) and Spearman's rank correlation (ρ = 0.03) are close to zero and indicate no correlation (Figure 2).

while concreteness has been shown to work well as a feature to distinguish between literal and non-literal language in general (Beigman Klebanov et al., 2014;Tsvetkov et al., 2014), it does not seem useful for discerning between novel and conventionalized metaphoric usage in particular.


And indeed, if a difference of concreteness between components of an expression hints at a metaphor, as is proposed by the conceptual metaphor theory, then it is plausible that it does not hint at novelty of the metaphoric expression at the same time.

we investigate a feature with more semantic capacity in the next subsection.


Generally, very novel metaphors also display a high metaphoricity.

we expect that low-POM verbs (i.e., verbs that occur similarly often in many different contexts) exhibit a low novelty score on average and low variance, while high-POM verbs should show a higher average novelty score.


The relative performance of SOCO varies over benchmarks: in two out of five cases the performance is roughly on par with SGM, and compared to GLOVE, CBOW and FOCO, our method performs best in three out of five benchmarks.

the overall picture is that our approach indeed is able to capture meaningful similarity relations, and that it performs comparably to regular first-order word embedding methods.


Through equations 4and 5, the model will put high probability on factors z w reflecting patterns that can supplement the linguistic embeddings e w in explaining the wordcontext co-occurrences.

the extracted latent visual factors can contribute to improve the performance on predicting the occurrence of a word and its contexts in the linguistic corpus, which would encourage the model to leverage the perceptual information.


Note that our learning objective is to jointly explain visual features and word-context co-occurrences.

two visually similar words, which never occur within the same context, could be mapped into slightly different directions in the latent space.


From this imbalanced distribution of relations, it immediately follows that accuracy alone, as evaluation measure, is not sufficient to identify the best performing model.

in the following section we report, and analyze, the F 1 scores of the predicted NomBank and PCEDT relations across all the STL, TL and MTL models.


Recent work by Dima (2016) and Vered and Waterson (2018) suggest that the gains achieved in noun-noun compound interpretation using word embeddings and somewhat similar neural classification models are in fact a by-product of lexical memorization (Levy et al., 2015); in other words, the classification models learn that a specific set of nouns is a strong indicator of a specific relation.

in order to gauge the role of lexical memorization in our models also, we quantify the number of unseen compounds that the STL, TL and MTL models predict correctly.


As expected, vectors from this method had near zero accuracy in the semantic portion, but scored well on the syntactic portion, showing that most of these analogies can be answered with morphology alone.

we conclude that the word representation layer in deep biLMs is only faithfully encoding morphology with little semantics.


We notice that with indicates an emotional state in with confusion and refers to an accompanier in combine with, while it suggests the idea of a tool or means in wash with water.

preposition sense disambiguation (PSD) is vital for natural language understanding and a closer look at the function of prepositions in specific contexts is an important computational step.


We consider a linear approximation of phrasal embeddings under three settings: (1) Sense-specific embedding, approximating the representation of a VPC as the sum of the vectors of its verb and its preposition with a specific sense.

we have global prep is the global preposition embedding disregarding its sense.


These input embeddings can either be learned by the model or be fixed to a pretrained embedding.

the last hidden state computed by this LStM is then transformed linearly to yield an m-dimensional definition embedding h. the encoder whose parameters are θ = {E, Ω, W, b} computes this embedding h as the subsequent decoder can be seen as a conditional language model trained by maximum likelihood to regenerate definition D w given definition embedding h = f θ (D w ).


Due to the use of such comparators, it is much easier to locate similes compared with locating other types of metaphors.

it is possible to collect and annotate large scale of simile sentences and investigate data driven simile recognition.


On the other hand, if the component extractor tells the classifier that the tenor and the vehicle likely exist, the classifier gets additional information for decision.

we propose a multitask learning approach to combine them.


Intuitively, the words corresponding to the tenor or the vehicle or near comparators should be more important.

we introduce the attention mechanism, which was firstly proposed for neural machine translation (Bahdanau et al., 2014).


In many cases, the comparator and the vehicle are near, and similes often have some collocations involving the comparator such as "oe... 7(the same as)" or "1oe(just like)".

we attempt to emphasize such local information.


Each sentence is convolved by a dynamically generated filter by another sentence.

the convolved features of one sentence can be regarded as context-aware representations under the influence of another sentence.


But for NLI, whether a word (or feature) being important in one sentence depends on another sentence.

a better convolution operation should have the ability to extract substantial features from one sentence according to another sentence.


Since each word in a sentence is a symbol that can not be directly processed by neural networks, we need first map each word to a d dimensional embedding vector.

the two sentences are mapped to two matrix E x ∈ R de×m and E y ∈ R de×n respectively.


Let u i and v i be intra and inter attentive vector of the i-th word in sentence x, a heuristic and effective composition operator is used to combine two vectors.

we can obtain two fused representations X (c) and Y (c) for two sentences, which are further fed into the prediction layer or another stacked interaction layer.


The interaction layer connects playing in Premise to Hypothesis instrument, and connects playing in Hypothesis to Premise violin.

the correlation between instrument in Hypothesis and violin in Premise are boosted, as we know these are important to reasoning.