Our inference model converts the output hidden vectors of BiLSTMs to the fixed-length vector with pooling operations and puts it into the final classifier to determine the overall inference class.

in addition to using mean pooling and max pooling similarly to ESIM , we propose to use weighted pooling based on external knowledge to obtain a fixed-length vector as in Equation (11)(12).


4 Experiment Set-Up Lexical Semantic Relations As described in Section 3.1, to incorporate external knowledge (as a knowledge vector r ij ) to the state-of-theart neural network-based NLI models, we first explore semantic relations in WordNet (Miller, 1995), motivated by MacCartney (2009).

the relations of lexical pairs are derived as described in (1)-(4) below.


In our work here, we also capture the relation between any two words in WordNet through relation embedding.

we employed TransE (Bordes et al., 2013), a widely used graph embedding methods, to capture relation embedding between any two words.


The first directly uses 18 relation embeddings pretrained on the WN18 dataset (Bordes et al., 2013).

if a word pair has a certain type relation, we take the corresponding relation embedding.


Our new entailment system, called AdvEntuRe, demonstrates that in the moderate data regime, adversarial iterative data-augmentation via only a handful of linguistic rule templates can be surprisingly powerful.

we observe 4.7% accuracy improvement on the challenging SciTail dataset (Khot et al., 2018) and a 2.8% improvement on 10K-50K training subsets of SNLI.


We present three different techniques to create adversarial examples for textual entailment.

we show how external knowledge resources, hand-authored rules, and neural language generation models can be used to generate such examples.


For instance, if p entails h and h entails f (h), p would entail f .

Table 3: Entailment label composition functions L (left) and N (right) for creating second-order examples.


For every mini-batch of examples X + Z G , we compute the discrimina-tor loss L(C ; D(X + Z G ; Â)) and apply the negative of this loss to each word of the generated sentence in G s2s .

the discriminator loss value replaces the cross-entropy loss used to train the seq2seq model (similar to a REIN-FORCE (Williams, 1992) reward).


For further analysis of the negation-based generator in Table 1, we collect only the negation examples in test set of SNLI, henceforth referred to as nega-SNLI.

we extract examples where either the premise or the hypothesis contains "not", "no", "never", or a word that ends with "n't'.


In this paper, we look at improving distributed word representations for Korean using knowledge about the unique linguistic structure of Korean.

we decompose Korean words into the jamo level, beyond the characterlevel, allowing a systematic use of subword information.


Overall, our model outperforms comparison models, showing close distance between predicted vector a + b − c and the target vector d (a:b=c:d).

performance is improved more in syntactic analogies.


We demonstrated the effectiveness of the learned word vectors in capturing the semantic and syntactic information by evaluating these vectors with word similarity and word analogy tasks.

the vectors using both jamo and character-level information can represent syntactic features more precisely even in an agglutinative language.


CSP predicts the sememe candidates for a target word from its word embedding and the corresponding character embeddings.

we follow SPWE and SPSE as introduced by  to model external information and propose Sememe Prediction with Word-to-Character Filtering (SPWCF) and Sememe Prediction with Character and Sememe Embeddings (SPCSE) to model internal character information.


HowNet provides sememe annotations for Chinese words, where each word is represented as a hierarchical tree-like sememe structure.

a word in HowNet may have various senses, which respectively represent the semantic meanings of the word in the real world.


Although they seem straightforward, the complexities and challenges in each step can add up.

we tackle the issues of treating small corpora and selecting pole words.


we use the Google analogy test dataset (Mikolov et al., 2013a), which contains 19,544 questions -8,869 semantic and 10,675 syntactic questions -in 14 relation types, such as capital-world pairs and opposite relationships.


Consider a corpus of our interest C and a reference corpus R. The model M is pre-trained on R, and then we start training it on C. We use the superscript e to represent the e-th epoch of training.

m e C is the model after the e-th epoch trained on C. Then, we evaluate the model regarding two aspects: general semantic relations and context-dependent semantic relations.


To further refine the antonym pairs, we create a crowdsourcing task on Figure Eight, formerly known as CrowdFlower.

we ask crowdworkers: Do these two words have opposite meanings?


However, another random pair ('happy' and 'evil') results in the worst performance with the AUC of 67.2.

an axis defined by only two pole words is highly sensitive to the choice of the word pair.


We map the word on our predefined 732 axes, which are explained in Section 3.3.1, and rank the axes based on the projection values on the axes.

the top axes describe the word with the highest strength.


Secondly, some methods (Bansal et al., 2014;Zhang et al., 2016) do explore the taxonomy space by regarding the induction of taxonomy structure as inferring the conditional distribution of edges.

they use the product of edge proba- Figure 1: An illustrative example showing the process of taxonomy induction.


Since it is infeasible to estimate the quality of all possible taxonomies, we design an end-to-end reinforcement learning (RL) model to combine the two phases.

we train an RL agent that employs the term pair representations using multiple sources of information and determines which term to select and where to place it on the taxonomy via a policy network.


To address this issue, we utilize several surface features described in previous studies (Yang and Callan, 2009;Bansal et al., 2014;Zhang et al., 2016).

we employ Capitalization, Ends with, Contains, Suffix match, Longest common substring and Length difference.


We regard the process of building a taxonomy as making a sequence of actions.

we define that an action a t at time step t is to (1) select a term x 1 from the remaining vocabulary V t ; (2) remove x 1 from V t , and (3) attach x 1 as a hyponym of one term x 2 that is already on the current taxonomy T t .


We use REINFORCE (Williams, 1992), one instance of the policy gradient methods as the optimization algorithm.

for each episode, the weights of the policy network are updated as follows: where v i = ∑ T t=i γ t−i r t is the culmulative future reward at time i and γ ∈ [0, 1] is a discounting factor of future rewards.


For a fair comparison, we reuse the hypernym graph as in TAXI (Panchenko et al., 2016) and SubSeq (Gupta et al., 2017) so that the inputs of each model are the same.

we restrict the action space to be the same as the baselines by considering only term pairs in the hypernym graph, rather than all |V | × |T | possible term pairs.


• Gloss Module: Like the context module, the gloss module encodes all the glosses of the target word into a separate vector representations of the same size.

we can get |s t | word sense representations according to |s t | 3 senses of the target word, where |s t | is the sense number of the target word w t .


In the scoring module, we make final predictions based on the last pass attention of memory module and the context vector c. Note that GAS only uses the original gloss, while GAS ext uses the entended glosses through hypernymy and hyponymy relations.

the relation fusion layer (grey dotted box) only belongs to GAS ext .


Our proposed method can jointly model the sentiment semantics and domain specificity of words, expecting the learned embeddings to achieve superior performance for the task of sentiment classification.

our method can automatically determine and produce domain-common embeddings and domainspecific embeddings.


The learning of domain-common embeddings can allow the ad-vantage of data augmentation of common semantics of multiple domains, and meanwhile, domainspecific embeddings allow us to capture the varied semantics of specific words in different domains.

for each word in the vocabulary, we design a distribution to depict the probability of the word being domain-common.


The representation of w is specific to the occurrence of w in D p or D q .

each word w can be represented as follows: where ⊕ denotes the concatenation operator.


And based on this representation, it trains a classifier with the source rich labeled data.

for data of the source domain X s and data of the target domain X t , it trains a feature generator G(•) with restriction P (G(X s )) ≈ P (G(X t )).


Thus, in this work, we try to explore a path to use the target domain specific information with as few as possible target labeled data.

we first introduce a novel method to extract the domain invariant and domain specific features of target domain data.


These methods focus on extracting the domaininvariant features with the help of unlabeled data.

ganin et al., (2016) incorporated an adversarial framework to perform this task.


In contrast with most of the previous methods, which pay more attention to domain invariant information, we showed that domain specific information could also be beneficially used in the domain adaptation task with a small amount of in-domain labeled data.

we proposed a novel method, based on the CMD metric, to simultaneously extract domain invariant feature and domain specific feature for target domain data.


Instead of predicting the overall sen-timent polarity, fine-grained aspect based sentiment analysis (ABSA) (Liu and Zhang, 2012) is proposed to better understand reviews than traditional sentiment analysis.

we are interested in the sentiment polarity of aspect categories or target entities in the text.


Each kernel corresponds a linguistic feature detector which extracts a specific pattern of n-gram at various granularities (Kalchbrenner et al., 2014).

the input sentence is represented by a matrix through the embedding layer, where L is the length of the sentence with padding.


The outputs of two gates are element-wisely multiplied for the max pooling layer.

gated Tanh-ReLU Units (gTRU) with aspect embedding are connected to two convolutional neurons at each position t. we compute the features c i as where v a is the embedding vector of the given aspect category in ACSA or computed by another CNN over aspect terms in ATSA.


The two convolutions in Equation 2 and 3 are the same as the convolution in the vanilla CNN, but the convolutional features a i receives additional aspect information v a with ReLU activation function.

s i and a i are responsible for generating sentiment features and aspect features respectively.


From these, we then extract token-level scores that are tied to specific prediction outcomes.

we train n linear models f i (x) = w i x + b i for tasks i = 1, .


This dataset includes 25 categories of products and is used to generate our sentiment embeddings using linear models.

we train linear SVMs using scikit-learn to extract word coefficients in each domain and also for the union of all domains together, yielding a 26-dimensional sentiment embedding.


For a more detailed analysis, we conducted additional experiments with alternative embedding conditions.

as a simpler means of achieving gains over standard CNNs, we propose to use CNNs with word vectors augmented with sentiment cues.


This encoder contextualizes words based on local context words to capture short range relationships between words.

we do the convolution operation using filter matrices W f ∈ R h×d with filter size h to a window of h words.


Finally, we concatenate all feature vectors of different h's for each i as the new word vector: The second part of HCWE is based on an RNN model which is used when texts are longer and include word dependencies that may not be captured by the CNN model.

we use a bidirectional LSTM and concatenate the forward and backward hidden state vectors as the new word vector, as shown below: The answer to the question whether to use local or global context to encode words for sentiment classification is still unclear, and both CNN and RNN models have previous empirical evidence that they perform better than the other (Kim, 2014;McCann et al., 2017).


Therefore, we can train separate classifiers for each pooled vectors using the same parameters W and b.

for each v ∈ V, we calculate the loss L v using the above formulas.


We approach this based on different types of metadata that people use to describe their moves on Wikipedia talk pages, the richest source of deliberative discussions on the web.

we extract the entire set of about six million discussions from all English Wikipedia talk pages.


The idea of this approach is not only to model the strategies, but also to allow for an operationalization of the resulting model by providing a dataset for training classifiers.

the metadata can also be used to label discussions based on distant supervision (Mintz et al., 2009).


The result of the parsing process is a large-scale corpus of Wikipedia discussions.

the Webis-WikiDiscussions-18 corpus we created contains about six million discussions, consisting of about 20 million turns.


We found that these links are used in Wikipedia discussions to point to evidence on the linked web pages.

they often link to research, news, search engines, educational institutions, and blogs.


To create a corpus for our model, we decided to rely again on the metadata.

for each category in our model, we retrieved the metadata instances that had been used to derive the category, and then labeled any turn that included any metadata with this category.


Since this paper does not aim at proposing a novel approach for the classification tasks, but rather at showing the ability to operationalize the model, we follow existing work that has proposed methods for the tasks at hand.

we implement a rich set of features that have been used by others before.


For each foreign word w f and each English word w e , we have multiple scores for the pair p f,e = (w f , w e ), used to rank w e against all other w e ∈ E, where E is the English dictionary used in the experiment.

we have TXT(p f,e ) and IM-AGE(p f,e ) for all pairs.


Our model leverages this compositional structure to generate reports in a hierarchical way: it first generates a sequence of high-level topic vectors representing sentences, then generates a sentence from each topic vector.

the context vector is inputted into a sentence LSTM, which unrolls for a few steps and produces a topic vector at each step.


We treat the tag prediction task as a multi-label classification task.

given an image I, we first extract its features {v n } N n=1 ∈ R D from an intermediate layer of a CNN, and then feed {v n } N n=1 into a multi-label classification (MLC) network to generate a distribution over all of the L tags: where l ∈ R L is a tag vector, l i = 1/0 denote the presence and absence of the i-th tag respectively, and MLC i means the i-th output of the MLC network.


We formulate the process of crafting adversarial examples in neural image captioning systems as optimization problems with novel objective functions designed to adopt the CNN+RNN architecture.

our objective function is a linear combination of the distortion between benign and adversarial examples as well as some carefully designed loss functions.


• We propose powerful and generic loss functions that can craft adversarial examples and evaluate the robustness of the encoder-decoder pipelines in the form of a CNN+RNN architecture.

our loss designed for targeted keyword attack only requires the adversarial caption to contain a few specified keywords; and we allow the neural network to make meaningful sentences with these keywords on its own.


The vector p t represents a probability distribution on V with each coordinate p (i) t defined as: Following the definition of softmax function: Intuitively, to maximize the targeted caption's probability, we can directly use its negative log probability (5) as a loss function.

the inputs of the RNN are the first N − 1 words of the targeted Applying (5) to (2), the formulation of targeted caption method given a targeted caption S is: Alternatively, using the definition of the softmax function, (3) can be simplified as as large as possible, it is sufficient to require the target word S t to attain the largest (top-1) logit (or probability) among all the words in the vocabulary at position t. we aim to minimize the difference between the maximum logit except S t , denoted by max k∈V,k =St {z (k) t }, and the logit of S t , denoted by z (St) t .


To bridge the gap, we use the originally inferred caption ) from the benign image as the initial input to RNN.

after minimizing (8) for T iterations, we run inference on I + δ and set the RNN's input S 1 as its current top-1 prediction, and continue this process.


In this paper, we investigate how to model geometric reasoning (a form of visual reasoning) using deep neural networks (DNN).

we address the task of answering questions through geometric reasoning-both the question and answer are expressed in symbols or words, but a geometric representation is created and used as part of the reasoning process.


Synthetic datasets for QA: Recently, synthetic datasets for QA are also becoming crucial in AI.

bAbI  has driven the development of several recent DNN-based QA systems (Kumar et al., 2016;Sukhbaatar et al., 2015;Henaff et al., 2016).


There are other differences to prior works as well.

(Seo et al., 2015) combined information from textual questions and diagrams to build a model for solving SAT geometry questions.


The attention network produces a scalar attention weight a i for the i th sentence which is given by In the proposed model, we make the simplifying assumption that the visual representation of the current sentence does not depend on future sentences.

it can be completely determined from the previous sentences in the description.


Further, we compare our model to DMN+ (Xiong et al., 2016) and MemN2N (Sukhbaatar et al., 2015), which achieved state-of-the-art results on bAbI .

we compare the 3-hop versions of DSMN, DMN+, and MemN2N.


The agent's response can be modeled as a sample from a probability distribution over the possible sequences.

for one session, given the visual input v t and conversation history H the agent's response a t can be generated by sampling from a distribution of the speaking action The agent interacts with the teacher by outputting the utterance a t and receives feedback from the teacher in the next step, with w t+1 a sentence as verbal feedback and r t+1 reward feedback (with positive values as encouragement while negative values as discouragement, according to a t , as described in Section 3).


Experimental results show that the proposed model is able to directly convert an input speech mixture into multiple label sequences without requiring any explicit intermediate representations.

no frame-level training labels, such as phonetic alignments or corresponding unmixed speech, are required.


A contextual inflection model works as follows: Rather than just generating the proper inflection for a single given word form out of context (for example walking as the gerund of walk), our generative model is actually a fully-fledged language model.

it generates sequences of inflected words.


As this marginalization is intractable, we derive a variational inference procedure that allows for efficient approximate inference.


Because q φ is trained to be a variational approximation for any input f , it is called an inference network.

it will return an approximate posterior over the latent variables for any observed sequence.


As our model is fully generative and directed, we may easily take samples from the complete joint.

we will take K samples by forward sampling and define them as D sleep .


While this work only focuses on the creation of an improved morphological inflector p θ (f | , m), one could imagine a situation where the encoder was also a task of interest.

the goal would be to improve both the decoder (the generation model) and the encoder (the variational approximation).


In this paper, we investigate models that use recurrent neural networks with sentence-level context for initial character and word-based representations.

we show that optimal results are obtained by integrating these context sensitive representations through synchronized training with a meta-model that learns to combine their states.


While the literature is too large to enumerate, Giménez and Marquez (2004) is a good example of an accurate linear model that uses both word and sub-word features.

like most systems of the time, they use ngram affix features, which were made context sensitive via manually constructed conjunctions with features from other words in a fixed window.


(2016) is probably the jumping-off point for most current architectures for tagging models with recurrent neural networks.

they used a combined word embedding and recurrent character encoding as the initial input to a BiLSTM that generated context sensitive word encodings.


The feature that distinguishes our model most from previous work is that we apply a bidirectional recurrent layer (LSTM) on all characters of a sentence to induce fully context sensitive initial word encodings.

we do not restrict the context of this layer to the words themselves (as in Figure  1b).


The specific model is shown in Figure 2, consisting of a factorial conditional random field (FCRF; ) with neural network potentials calculated by long short-term memory (LSTM; (Hochreiter and Schmidhuber, 1997)) at every variable node ( §3).

learning and inference in the model is made 2 the latter is common because many UD resources were created by full or semi-automatic conversion from treebanks with less comprehensive annotation schemes than UD.


The baseline model (Cotterell and Heigold, 2017) we compare with regards the output space of the model as a subsetỸ ⊂ Y whereỸ is the set of all tag sets seen in this training data.

they solve the task as a multi-class classification problem where the classes are individual tag sets.


This item represents a set of computations (transition sequences) that start from a configuration of the form pσ, i|β, Aq (where σ is the stack and i|β is the buffer, with i being the first buffer node) and take the parser to a configuration of the form pσ|i, j|β 1 , Aq.

the computation has the net effect of placing node i on top of the previous contents of the stack, and it ends in a state where the first buffer element is j.


To verify that our lexical representations can function for morphologicallyrich languages and smaller treebanks, we restricted ourselves to running a subset of the exact models that we evaluated on English.

we evaluated the model that uses a character-level LSTM, with and without the addition of learned word embeddings.


In this paper, we show that the choice of encoder can have a substantial effect on parser performance.

we demonstrate state-of-theart parsing results with a novel encoder based on factored self-attention.


Whereas, tags heading sentences will remain unchanged.

if the amount of consecutive ECs in the current position is an odd number, we choose to attach the extra EC to the following word for consistency and clarity.


Call Σ and ∆, the input alphabet of M 1 and the output alphabet of M 2 , the outer alphabets.

the composition of M 1 and M 2 is the weighted FSt for each pair of transitions with the same inner symbol, the composed transducer has a transition transitions with the same source, target, input, and output symbols are merged, adding their weights.


The key issue for treebank conversion is that sentences in the source-side and target-side treebanks are non-overlapping.

there lacks a bi-tree aligned data in which each sentence has two syntactic trees complying with two guidelines as shown in Figure 1.


Although the overall inter-annotator dependency-wise consistency rate is 76.5%, the sentence-wise consistency rate is only 43.7%.

56.3% (100 − 43.7) sentences are further checked by a third expert annotator.


Their study found no effects of syntax-based predictors over and above sequential ones.

no effects emerged in the 500-700 ms window, where one might have expected a P600.


After extracting the positive cases, we then use the governor information of positive cases to extract negative data.

we extract sentences containing the same governors but not any of the target adverbs as negatives.


For our future work, we shall continue our investigation into approaches that can estimate the word error rate using convolutional neural networks.

we would like to explore combining the DNN numerical features with the CNN word embedding features.


This is in contrast to a naive loss-based approach, which has to query the classifier for every possible change to compute the exact loss induced by those changes.

apart from the overhead of calling the function in eq.


In Figure 1, we plot the success rate of the adversary against an acceptable confidence score for the misclassification.

we consider the adversary successful only if the classifier misclassifies the instance with a given confidence score.


We use the full test set for this experiment, and we compare HotFlip adversarial training with the white-box (supervised) adversarial training (Miyato et al., 2017) that perturbs word embeddings, which we adapt to work with character embeddings.

the adversarial noise per character is constrained by the Frobenius norm of the embedding matrix composed of the sequence of characters in the word.


As shown in Table  2, our approach decreases misclassification error and dramatically decreases the adversary's success rate.

adversarial training on real adversarial examples generated by HotFlip, is more effective than training on pseudo-adversarial examples created by adding noise to the embeddings.


These examples, albeit interesting and intuitive, are not abundant, and thus pose less threat to an NLP word-level model.

given the strict set of constraints, we were able to create only 41 examples (2% of the correctly-classified instances of the SST test set) with one or two flips.


Hence, we exploit the natural structure of text articles to obtain weakly-labeled data.

our underlying assumption is that sentences belonging to the same section are typically more thematically related than sentences appearing in different sections.


Here we investigate the performance of our model in the more classical task of semantic relatedness of sentences.

we examine the SemEval 2014 Task 1: semantic relatedness SICK dataset (Marelli et al., 2014).


Named Entity Recognition can be posited as a standard sequence classification problem where consists of example label pairs where both the examples and the labels are themselves sequences of word vectors and entity types, respectively.

an input example is a equal-length sequence of entity-type labels y i,j ∈ Y where Y is the set of all entity type labels and includes a special other 'O'-label with which all words that are not entities are labeled.


Our single model achieves 67.8% F1 and our 5-model ensemble achieves 69.2% F1.


To investigate the influence of context features on analogical reasoning, we consider not only word features, but also ngram features inspired by statistical language models, and character (Hanzi) features based on the close relationship between Chinese words and their composing characters 5 .

we use word bigrams for ngram features, character unigrams and bigrams for character features.


Many approaches for sentiment analysis of metaphorical texts have been proposed in the area of NLP (Smith et al., 2007;Veale, 2012;Reyes and Rosso, 2012;Kozareva, 2013;Strzalkowski et al., 2014).

along with the rapid explosion of social media applications such as Twitter and Weibo, emotional texts containing metaphorical expressions have increased considerably.


With the aim of constructing a corpus in the study of emotionality of metaphorical texts in real-world Chinese, data collection took place in accordance with two principles: (1) balance, and (2) relatively abundant emotional information.

to ensure the corpus is balanced in genre, theme, and style, we selected data from a wide range of sources including books, journals, movie scripts, and networks.


In this study, following Shutova (2017), we define metaphors as both novel and conventional, but we exclude "dead metaphors" (from which the literal sense has disappeared) from conventional metaphors.

conventional metaphors only include those for which the literal and metaphorical senses are clearly distinctive, and both are used contemporarily.


We annotated 4,600 sentences out of a total of 5,605 metaphorical sentences as containing emotions.

a significant proportion of Chinese metaphorical expressions in the corpus contain emotions.


We thus curate a subset of data instances for human annotation of comment quality, which is also used for enhancing automatic metrics as in the next section.

we randomly select a set of 1,610 articles such that each article has at least 30 com-ments, each of which contains more than 5 words, and has over 200 upvotes for its comments in total.


Due to space limitations, here we only present the enhanced ME-TEOR, and defer the formulations of enhancing BLEU, ROUGE, and CIDEr to the supplements.

mETEOR performs word matching through an alignment between the candidate and references.


We follow previous setting (Papineni et al., 2002;Lowe et al., 2017) to evaluate the metrics, by conducting human evaluations and calculating the correlation between the scores assigned by humans and the metrics.

for each article in the test set, we obtained six comments, five of which come from IR-T, IR-TC, Seq2seq, Att, and Att-TC, respectively, and one randomly drawn from real comments that are different from the reference comments.


We can see that attention and back-transfer loss play important roles in the model.

when both of them are removed (last row in Table 4), although the classification accuracy improves, the perplexity and the content preservation drop significantly.


(2017), is able to capture social attitudes towards gender issues, as well as the emerging trends in netspeak.

in the discussions around LGBTQ issues in 2014, the words that emerge are closer to the actual experiences of the group, with words referring to 'passing' (a reference to transsexuals) and 'coping', as well as more positive emotion words ('yayy', 'harmony').


Our analysis makes use of the following definition of matrix similarity: (2) Here tr(M) denotes the trace of the matrix M. tr(M 1 T M 1 ) computes the Frobenius norm ||M 1 || 2 , and tr(M 1 T M 2 ) is the Frobenius inner product.

cos(M 1 , M 2 ) computes the cosine similarity between the vectorized versions of matrices M 1 and M 2 .


In this paper we focus on the task of extracting the most commonly used Web pages in named entity recognition (IE) information from a large corpus of news articles.

we propose a new kernel method that can be applied to a large collection of Web pages by automatic detection of labeled and unlabeled data.


Writingediting Networks (Final Draft) In this paper we focus on the task of identifying the most commonly relevant features of Web documents.

we propose a generic, automated IE algorithm that can be applied to a large collection of Web pages containing full large documents.


As expected, Table 4 shows that people with less domain knowledge are more easily deceived.

non-CS human judges fail at more than half of the 1-to-1 sets for the same titles, which suggests that most of our system generated abstracts follow correct grammar and consistent writing style.


During training, we utilise such sequences of binary labels to supervise the memory update gate activations of each chain.

each chain is encouraged Ricky fell while hiking in the woods to open its gate only when it sees a trigger bearing information semantically sensitive to that particular aspect.


This may be due to the fact the quality of information contained in the gold labeled data deteriorates.

using the right amount of weekly-supervision is an important hyper-parameter that needs to be carefully chosen.


Therefore, some researchers have proposed to accelerate the NMT training by resampling a smaller subset of the data that makes a relatively high contribution, to improve the training efficiency of NMT.

kocmi and Bojar (2017) empirically investigated curriculum learning based on the sentence length and word rank.


Therefore, we adopt the ratio of differences (dif ) between training costs of two training iterations to be the criterion, It should be noted that some of dif x,y are negative.

the costs of some sentence pairs even increase after one epoch training.


Our work investigates how we can efficiently model speaker-related variations to improve NMT models.

we are interested in improving our NMT system given few training examples for any particular speaker.


However, BLEU is not a perfect evaluation metric.

we are interested in evaluating how much of the personal traits of each speaker our models capture.


Although the topics that a speaker talks about can be considered as a manifestation of speaker traits, we also perform a control experiment on a different dataset to verify that our model is indeed learning more than just topical information.

we train our models on a speaker annotated version of the Europarl corpus (Rabinovich et al., 2017), on the en-de language pair 6 .


(2009), CForm achieves higher precision and recall on idiomatic usages than literal ones.

the relatively low recall for the literal class indicates that many literal usages occur in a canonical form.


For this purpose, we also study low-rank embeddings of the PPMI matrix, which allow us to make predictions for unseen pairs.

let m = |{x : (x, y) ∈ P ∨ (y, x) ∈ P}| denote the number of unique terms in P. Furthermore, let X ∈ R m×m be the PPMI matrix with entries M xy = ppmi(x, y) and let M = U ΣV ⊤ be its Singular Value Decomposition (SVD).


First, we consider WeedsPrec (Weeds et al., 2004) which captures the features of x which are included in the set of a broader term's features, y: Second, we consider invCL (Lenci and Benotto, 2012) which introduces a notion of distributional exclusion by also measuring the degree to which the broader term contains contexts not used by the narrower term.

let x i denote the degree of inclusion of x in y as proposed by Clarke (2009).


Intuitively, the SLQS model presupposes that general words appear mostly in uninformative contexts, as measured by entropy.

sLQs depends on the median entropy of a term's top N contexts, defined as , where H(c i ) is the shannon entropy of context c i across all terms, and N is chosen in hyperparameter selection.


In nearly all cases, we find that patternbased approaches substantially outperform all three distributional models.

strong improvements can be observed on BLESS (0.76 average precision vs 0.19) and WBLESS (0.96 vs. 0.69) for the detection tasks and on all directionality tasks.


it returns the distribution closest to softmax(z) whose attention probabilities are bounded by u. Martins and Kreutzer (2017) have shown that this transformation can be evaluated in O(J log J) time and its gradients backpropagated in O(J) time.


It has the advantage of fading out content in favor of more shallow patterns still based on the original text, without introducing additional processing such as part-of-speech tagging.

we investigate to what extent gender prediction can rely on generic non-lexical features (RQ1), and how predictive such models are when transferred to other languages (RQ2).


We briefly describe our techniques here and direct the interested readers to the paper.

the EL model computes several similarity/coherence scores S in a "feature abstraction layer" which computes several measures of similarity between the context of the mention m in the query document and the context of the candidate link's Wikipedia page which are fed to a feed-forward neural layer which acts as a binary classifier to predict the correct link for m. the feature abstraction layer computes cosine similarities (Sil and Florian, 2016) between the representations of the source query document and the target Wikipedia pages over various granularities.


Therefore, similar to the case at word level, we construct the gold sentence-level attention signals β * for the document d, as illustrated in Figure 2b, and further take the square error as the general loss of the attention at sentence level to supervise the learning process: We employ another Bi-GRU encoder and a softmax output layer to model the ED task, which can handle event triggers with multiple words.

given a sentence s j (j = 1, 2, ..., L) in document d, for each of its word w jt (t = 1, 2, ..., T ), we concatenate its word embedding w jt and entity type embedding e jt with the corresponding document embedding d as the input r jt of the Bi-GRU and thus obtain the hidden state f jt : Finally, we get the probability vector o jt with K dimensions through a softmax layer for w jt , where the k-th element, o jt , of o jt indicates the probability of classifying w jt to the k-th event type.


DEEB-RNN1/2/3 means they uses the gold attention signals as supervision information.

dEEB-RNN1 uses only the gold word-level attention signal (λ = 1 and µ = 0), dEEB-RNN2 uses only the gold sentence-level attention signal (λ = 0 and µ = 1), whilst dEEB-RNN3 employs the gold attention signals at both word and sen-  tence levels (λ = 1 and µ = 1).


As human labeling is infeasible when training complicated neural networks, we propose a new method that can leverage unlabeled data to learn a matching model.

instead of random sampling, we construct D by retrieving (y i,2 , .


Therefore, we attempt to iteratively optimize the Seq2Seq model and the matching model and check if the matching model can be further improved.

we update the Seq2Seq model every 20 mini-batches with the policy-based reinforcement learning approach proposed in (Li et al., 2016b).


We use the term "extended vocabulary" to denote the union of the slot vocabulary and all words from the input utterances.

the probability distribution over the extended vocabulary is calculated as: the model makes the final predictions using a weighted combination of the predictions from two individual components.


We create specific datasets by re-sampling from the original corpus.

let group A denote all the training utterances that contain nonenumerable slots, and group B denote the rest of the training utterances.


Instead, the phenomenon described is more closely tied to other properties of neural networks.

continuity guarantees that topological properties of the input, such as connectedness, are preserved (Armstrong, 2013).


Concept similarity tasks provide a suitable setting to study the semantic structure of distributed representations (Pennington et al., 2014).

semantically similar concepts should ideally be close together.


That is, semantically similar concepts should ideally be close together.

our interest is in comparing X with its projection f (X) through a mapping with random parameters, to understand the extent to which the mapping may disrupt or preserve the semantic structure of X.


1 Generally, the merges are based on the common distribution of consonant phonemes in most natural languages, as well as the etymology of the characters in each abugida.

three or four graphemes are preserved for the different articulation locations (i.e., guttural, palate, dental, and labial), that two for plosives, one for nasal (NAS.


After many experimentations, a general "shallow and broad" configuration was adopted.

simplified grapheme bi-grams are first embedded into 128-dimensional vectors 3 and then encoded in one layer of a bi-directional LSTM, resulting in a final representation consisting of a 512-dimensional vector that concatenates two 256-dimensional vectors from the two directions.


We compare our model with the following baselines: • Randomly predict (RP): We randomly decide whether the source paper can be accepted.

the probability of acceptance of every source paper is always 0.5 using this strategy.


We follow a standard approach to assign tags to words.

o and I are used to indicate whether a word appears outside or inside the negation scope respectively.


Compared with the Linear model, Semi models achieve better results.

semi o model achieves the best result on F 1B at the scope level among all the models and achieves the highest result on F 1 at the token level.


1 Textual entailment (TE) is a fundamental problem in natural language understanding and has been studied intensively recently using multiple benchmarks -PASCAL RTE challenges (Dagan et al., 2006(Dagan et al., , 2013, Paragraph-Headline (Burger and Ferro, 2005), SICK (Marelli et al., 2014) and SNLI (Bowman et al., 2015).

sNLI -while much easier than earlier datasets 1 https://github.com/yinwenpeng/sciTail Premise P Pluto rotates once on its axis every 6.39 Earth days.


Our model DEISTE is implemented in convolutional neural architecture (LeCun et al., 1998).

dEISTE consists of (i) a parameterdynamic convolution for exploration strategy (a) given above; and (ii) a position-aware attentive convolution for exploration strategies (b) and (c).


The proposed sense-aware neural model differs from the baseline neural model in that it models multiple sequences of word senses corresponding to different WSD results.

the sense-aware model works on the sense level, but the baseline model works on the word level.


Then each row of the adjacency matrix A can be normalized (i.e., so that each row sums to 1), turning it into a right stochastic matrix S. Negative sampling, in the skip-gram model, uniformly draws at random for each training word one of its surrounding words as the (positive) target word.

this range is determined by the size of the training context c. a surrounding word w s of the training word w t must satisfy the following condition: For the same corpus, let us set d max equal to c in word2vec and generate a word co-occurrence network of the whole corpus.


Steady state means that no matter which vertex one starts with, the distribution of the destination vertex probabilities remains the same.

all S t columns will have the same value.


We pre-compute this distribution off-line for each token so that the added complexity is proportional to the size of the vocabulary.

precomputing the co-occurrences and graphs using corpus2graph (Zhang et al., 2018) takes about 2.5 hours on top of 8 hours for word2vec alone on the entire Wikipedia corpus using 50 logical cores on a server with 4 Intel Xeon E5-4620 processors : the extra cost is not excessive.


CBOW predicts the target word w t given nearby context words in a window with width δ: The set C near wt contains in total at most 2δ words, including δ words to the left and δ words to the right of a target word.

we train the word vectorsṽ wt and v c (c ∈ C near wt ) by maximizing the following prediction probability: The CBOW captures both semantic and syntactic word similarity through the training using nearby context words.


Therefore, as the first extension, we propose excluding the nearby context C near wt from all the context C all wt .

we use the distant context words only: We expect that training with this type of context will lead to word vectors containing the stylesensitive information only.


We asked crowdsourced workers to select style-sensitive words in utterances.

for the crowdsourced task of picking "style-sensitive" words, we provided workers with a word-segmented utterance and asked them to pick words that they expected to be altered within different situational contexts (e.g., characters, moods, purposes, and the background cultures of the speaker and listener.).


First, we tested the ability to capture syntactic similarity of each model by checking whether the POS of each word was the same as the POS of a neighboring word in the vector space.

we calculated SYNTAXACC@N defined as follows: where I[condition] = 1 if the condition is true and I[conditon] = 0 otherwise, the function POS(w) returns the actual POS tag of the word w, and N (w) denotes the set of the N top similar words {w } to w w.r.t.


In this paper, we hypothesize that aspect-level sentiment classification can be improved by employing knowledge gained from document-level sentiment classification.

we explore two transfer methods to incorporate this sort of knowledge -pretraining and multi-task learning.


As a counter-example, if the training/testing data is in the laptop domain, then embeddings from the electronics domain are considered to be out-ofdomain embeddings (e.g., the word "adapter" may represent different types of adapters in electronics rather than exactly a laptop adapter).

only laptop reviews are considered to be in-domain.


Instead of optimising individual word embeddings, our model uses general-purpose embeddings and optimises a separate neural component to adapt these to the specific task.

our neural Supervised Directional Similarity Network (SDSN) dynamically produces task-specific embeddings optimised for scoring the asymmetric lexical entailment relation between any two words, regardless of their presence in the training resource.


In this paper, we study methods for reducing the amount of human input needed to learn common sense.

we focus on learning relative comparisons of (one-dimensional) object properties, such as the fact that a cantaloupe is more round than a hammer.


In this work, we propose a simple method to learn the distributed representations for output structured semantic representations which allow us to capture cross-lingual features.

following previous work (Wong and Mooney, 2006;Jones et al., 2012; Susanto and Lu, 2017b), we adopt a commonly used tree-shaped form as the underlying meaning representation where each tree node is a semantic unit.


To incorporate neural features, HT-D (NN) defines the following scoring function: where Θ is the set of parameters of the neural networks and G is the neural scoring function over the (n,m,h) tuple (Susanto and Lu, 2017b).

the neural features are defined over a fixed-size window surrounding a word in n paired with its immediately associated semantic unit.


We follow a standard practice for evaluations which has been adopted in the literature (Lu, 2014(Lu, , 2015Susanto and Lu, 2017b).

to evaluate the proposed model, predicted outputs are transformed into Prolog queries.


In all experiments, we follow the experimental settings and procedures in (Lu, 2014  2015; Susanto and Lu, 2017b).

we use 600 instances for training and 280 for test and set the maximum optimization iteration to 150.


Note that there exist multiple versions of logical representations used in the GEO-QUERY dataset.

one version is based on lambda calculus expression, and the other is based on the variable free tree-shaped represen-tation.


We can observe that when distributed logical representations are included, both HT-D and HT-D (NN) can lead to competitive results.

when such features are included, evaluation results for 5 out of 8 languages get improved.


We now consider the non-seasonal time intervals (spans of years).

we consider the scenario when one wants to apply a classifier trained on older data to future data.


To do this, we leverage recent advances in contextadaptive neural language modeling.

we make use of the recently introduced FactorCell model that uses an embedding vector to additively transform the weights of the language model's recurrent layer with a low-rank matrix (Jaech and Ostendorf, 2017).


In this paper, we focus on designing a review generation model that is able to leverage both user and item information as well as auxiliary, textual input and aspect-aware knowledge.

we study the task of expanding short phrases into complete, coherent reviews that accurately reflect the opinions and knowledge learned from those phrases.


The hidden state at time-step t is updated via the GRU unit based on the previous hidden state and the input word.

: where h 0 ∈ R n is the decoder's initial hidden state and h t ∈ R n is the hidden state at time-step t. To fully exploit the encoder-side information, we apply an attention fusion layer to summarize the output of each encoder and jointly determine the final word distribution.


For example, a level 12 article may have simplified counterparts for levels 8 and 4.

the corpus contains instances where the same input leads to different outputs.


Our goal is to study how much each of these components contribute to the empirical performance of LSTMs.

it is worth considering the memory cell in more detail to reveal why it could serve as a standalone powerful model of long-distance context.


Our experiments investigated whether the S-RNN is a necessary component of LSTMs.

are the gates alone as powerful of a model as an LSTM?


Under these limitations, we show that different RNN variants have different computational power.

we show that the LSTM and the Elman-RNN with ReLU activation are strictly stronger than the RNN with a squashing activation and the GRU.


we consider the Elman RNN (SRNN) (Elman, 1990) with squashing and with ReLU activations, the Long Short-Term Memory (LSTM) (Hochreiter and Schmidhuber, 1997) and the Gated Recurrent Unit (GRU) Chung et al., 2014).


However, for computation bound by input length (real-time) there is a more interesting hierarchy.

real-time counting languages cut across the traditional Chomsky hierarchy: real-time k-counter machines can recognize at least one context-free language (a n b n ), and at least one context-sensitive one (a n b n c n ).


Our comatching approach explicitly treats the question and the candidate answer as two sequences and jointly matches them to the given passage.

for each position in the passage, we compute two attention-weighted vectors, where one is from the question and the other from the candidate answer.


In order to capture the sentence structure of the passage, we further modify the model presented earlier and build a hierarchical LSTM (Tang et al., 2015) on top of the co-matching states.

we first split the passage into sentences and we use P 1 , P 2 , .


Although this model seems to be using story comprehension features such as event sequencing, since the endings are included in the sequences, the biases within the endings have influenced the predictions and the weak performance of the model in SCT-v1.5 suggest that this model had picked up on the biases of SCT-v1.0 as opposed to really understanding the context.

the posterior probabilities for each ending choice using their features are quite similar on the SCT-v1.5.


In this work, we propose a Multi-sentimentresource Enhanced Attention Network (MEAN) for sentence-level sentiment classification to integrate many kinds of sentiment linguistic knowledge into deep neural networks via multi-path attention mechanism.

we first design a coupled word embedding module to model the word representation from character-level and word-level semantics.


Different from (Zhang et al., 2015), the designed Char-CNN is a fully convolutional network without max-pooling layer to capture better semantic information in character chunk.

we first input one-hot-encoding character sequences to a 1 × 1 convolution layer to enhance the semantic nonlinear representation ability of our model (Long et al., 2015), and the output is then fed into a multi-gram (i.e.


Inspired by (Lin et al., 2017), we also design a penalization term to ensure the diversity of semantics from different sentiment-resourcespecific sentence representations, which reduces information redundancy from different sentiment resources attention.

the final loss function is presented as follows: where y j i is the target sentiment distribution of the sentence,ŷ j i is the prediction probabilities, θ denotes each parameter to be regularized, Θ is parameter set, λ is the coefficient for L 2 regularization, µ is a hyper-parameter to balance the three terms, ψ is the weight parameter, I denotes the the identity matrix and ||.|| F denotes the Frobenius norm of a matrix.


All the tested factors con-tribute greatly to the improvement of the MEAN.

the accuracy decreases sharply when discarding the sentiment words.


In this paper, we take the concept a step further by using a conditional language model, instead of a language model.

we address a sentiment classification task for a tweet analysis service as a case study and propose a pretraining strategy with unlabeled dialog data (tweet-reply pairs) via an encoder-decoder model.


The other kind of unreliable data comes from the hashtags not only functioning as metadata.

a hashtag in a tweet may also function as a content word in its word form.


In the SemEval dataset, the tweets with one of the three indication hashtags #not, #sarcasm, and #irony, are collected and human-annotated as one of four types: verbal irony by means of a polarity contrast, other verbal irony, situational irony, and non-ironic.

the false-alarm tweets, i.e., the non-ironic tweets with indication hashtags, are distinguished from the real ironic tweets in this dataset.


As shown in the last row of Table  3, the irony detector trained on the small data does not compete with the models that are trained on larger amount of self-labeled data.

our data pruning strategy forms a semi-supervised learning that benefits from both self-labeled data and human annotation.


In this layer, we implement the idea of discovering domain-specific aspects for cross-target stance inference.

the key observation we make is that the domain aspects that reflect users' major concerns are usually the core of understanding their stances, and could be mentioned by multiple users in a discussion.


We observe a wide range of phenomena, extending beyond expected phenomena like negation, antonymy, and entity changes.

sQUADRUN is much more diverse than RULEBAsED, which creates unanswerable questions by applying entity, number, and antonym swaps to existing sQuAD questions.


Currently, researchers have paid great attention to retrieval-based dialogues in opendomain.

people study the problem by investigating context-response matching for multi-turn response selection based on publicly recognized benchmark data sets.


In this paper, we propose a novel Transformerbased architecture for multi-turn document grounded conversations.

we devise an Incremental Transformer to encode multi-turn utterances along with knowledge in related documents.


We integrate our utterance rewriter into the single-turn engine and compare with the original model by conducting the online A/B test.

we randomly split the users into two groups.


Following the recent line of work on generative dialog systems, we treat the problem of generating an appropriate response given a conversation history as a conditional language modeling problem.

we want to learn a conditional probability distribution P θ (y|x) where y is a reasonable response given the conversation history x.


The discriminators end up assigning extremely high probabilities to most of the human responses, and close-to-zero densities to generated responses.

the amount of negative examples is too small to train a discriminator to obtain good decision boundaries and generalization.


Evaluation on two large-scale multi-turn response selection tasks has demonstrated that our proposed model significantly outperforms the state-ofthe-art model.

visualization analysis shows that the spatio-temporal features enables matching information in segment pairs and time sequences, and have good interpretability for multi-turn text matching.


As illustrated in Figure 2, the proposed STM framework includes two parts: (i) representation module and (ii) matching block.

representations of the dialogue context and candidate answers are first learned through from dual encoders, and deep 3D ConvNets (Ji et al., 2013) are then used to match attentions between the dialogue contexts and candidate answers.


We let t = C x [C 0 [t y ]] be a minimal separation of t verifying the bound and t 0 = C 0 [t y ].

we have: e By Lemma 14, t 0 = D 0 [t y ] contains all nodes of t 0 .


Our exploration spans from simple operations, such as a random projection, to deep neural network models, such as a regularized autoencoder.

we introduce a semantic-preserving objective, which is augmented with the standard autoenoder architecture to encourage abstracting informative binary codes.


We also consider an alternative strategy to adaptively choose the dimension of the resulting binary representations.

principal Component Analysis (pCA) is utilized to reduce the dimensionality of pre-trained continuous embeddings.


To address this shortcoming, we further consider an autoencoder architecture, that leverages the reconstruction loss to hopefully endow the learned binary representations with more information.

an encoder network is utilized to transform the continuous into a binary latent vector, which is then reconstructed back with a decoder network.


This observation is consistent with the results reported in (Kiros and Chan, 2018).

a binarization layer is directly appended over the InferSent architecture (Conneau et al., 2017) during training, which gives rise to much larger drop in terms of the embeddings' quality (we have conducted empirical comparisons with (Kiros and Chan, 2018) in Table 1).


Note that all four proposed strategies can be easily extended to other pre-trained general-purpose sentence embeddings as well.

a bidirectional LSTM architecture along with a max-pooling operation over the hidden units is employed as the sentence encoder, and the model parameters are optimized on the natural language inference tasks, i.e., Standford Natural Language Inference (SNLI) (Bowman et al., 2015) and Multi-Genre Natural Language Inference (MultiNLI) datasets (Williams et al., 2017).


• Sentence matching, which comprises semantic relatedness (SICK-R, STS14, STSB) and paraphrase detection (MRPC).

each pair of sentences in STS14 dataset is associated with a similarity score from 0 to 5 (as the corresponding label).


We extend it and add information about structural context.

we incorporate information about siblings s (blue arrows in the figure).


(2018) and use derivatives to estimate how sensitive a particular part of the architecture is with respect to changes in input.

for every vector x we measure how it is influenced by every word represen- tation x i from the sentence.


The BiLSTM-based parsers can compensate for the lack of traditional structural features.

the features drawn from partial subtrees become redundant because the parsing models encode them implicitly.


In this work, we propose a conceptually simpler approach to the issue, which is agnostic on any parser architecture, namely, automatic generation of CCGbanks (i.e., CCG treebanks) 1 for new domains, by exploiting cheaper resources of dependency trees.

we train a deep conversion model to map a dependency tree to a CCG tree, on aligned annotations of the Penn Treebank (Marcus et al., 1993) and the English CCGbank (Hockenmaier and Steedman, 2007) (Figure 1a).


To overcome this limitation, we propose a model that incorporates the given tag hierarchy within the neural NER model.

this model learns to predict a tag sequence only over the fine-grained tags in the hierarchy.


To fix the semantics-equality above, we use the notion of the 'Other' tag in NER, which has the semantics of "all the rest".

for every d / ∈ T F G , a fine-grained tag 'd-Other' ∈ T F G and an edge 'd-Other'→'d' are automatically added to the graph, hence 'd-Other'∈ Fine(d).


Our proposed aggregation method in §2.1 is based on an assumption that the true annotations are independent from each other, which simplifies the model but may generate undesired results.

entities predicted by different transfer models could be mixed, resulting in labels inconsistent with the BIO scheme.


The basis of this dynamic composition mechanism is the reliability signals that inform the model of the quality of each word embedding.

we assume that if a word occurs more frequently, its word embedding will be more fully trained as it has richer contexts and its embedding is updated more often during training.


Experiments on 20 languages and 294 distant language pairs demonstrate the advantages of the unsupervised pivot translation for distant languages, as well as the effectiveness of the proposed LTR for path selection.

in the best case, LTR achieves an improvement of 5.58 BLEU points over the conventional direct unsupervised method.


Our results show that the different losses work in tandem to learn a better mapping than any one can on its own ( §4.2).

we show that two instantiations of the semi-supervised framework, corresponding to different supervised loss objectives, outperform their supervised and unsupervised counterparts on nu-merous language pairs across two datasets.


Our analysis ( §4.4) demonstrates that adding supervision to the learning objective, even in the form of a small seed dictionary, substantially improves the stability of the learning procedure.

for cases where either the embedding spaces are far apart according to GH distance or the quality of the original embeddings is poor, our framework converges where the unsupervised baselines fail to.


Unsupervised Distribution Matching: Given all word embeddings X and Y, the unsupervised loss L W |D aims to match the distribution of both embedding spaces.

for our formulation, we use an adversarial distribution matching objective, similar to the work of Lample et al.


We parameterize our discriminator with an MLP, and alternatively optimize the mapping matrix and the discriminator with the corresponding objectives: Aligning Known Word Pairs: Given aligned bilingual word embeddings S, we aim to minimize a similarity function (f s ) which maximizes the similarity between the corresponding matched pairs of words.

the loss is defined as: Weak Orthogonality Constraint: Given an embedding space X , we define a consistency loss that maximizes a similarity function f a between x and W T W x, x 2 X .


We study relation extraction for knowledge base (KB) enrichment.

we aim to extract entities and their relationships from sentences in the form of triples and map the elements of the extracted triples to an existing KB in an end-to-end manner.


In this paper, we study how to enrich a KB by relation exaction from textual sources.

we aim to extract triples in the form of h, r, t , where h is a head entity, t is a tail entity, and r is a relationship between the entities.


Table 3 shows that the end-to-end models outperform the existing model.

our proposed n-gram attention model achieves the best results in terms of precision, recall, and F1 score.


We can observe that all the C-AGGCN models with varied values of K are able to outperform the state-of-the-art C-GCN model (Zhang et al., 2018) (reported in Table 2).

with the same setting as K=1, C-AGGCN surpasses C-GCN by 1.5 points of F1 score.


When the size of training data increases, we can observe that the performance gap becomes more obvious.

using 80% of the training data, the C-AGGCN model is able to achieve a F1 score of 66.5, higher than C-GCN trained on the complete training set.


Compared to our work, their motivations and network structures are different.

each node only attends to its neighbors in GATs whereas AG-GCNs measure the relatedness among all nodes.


We will show empirical results confirming this observation both on synthetic and real-life data.

we will demonstrate this behavior in case of spatial and temporal aggregation.


For common topics, the probability or their occurrence does not depend on location of the document.

p(t k |l) = p(t k ), where l is location.


As compared to Figure  3, we can see a slight deterioration of the quality of discovered spatially distinct topics from the aggregated data.

all of the 4 discovered spatial topics are corrupted with more words from the common topics, which is particularly visible from the rightmost bin containing and an almost equal mixture of words from topics D1, C1, and C2.


For each minibatch of positive triples, a small number of negative triples are sampled by corrupting head or tail for each positive triple, then related parameters in the model are updated by minimizing the binary negative log-likelihood such that positive triples will get higher scores than negative triples.

the loss function is written as follows, where λ ∈ R is the L 2 regularization coefficient for entity embeddings only, T + and T − are the sets of positive and sampled negative triples in a minibatch, and y equals to 1 if (h, r, t) ∈ T + otherwise −1.


Another parametrization technique for D K where K ∈ {4, 6} is to parametrize each element in the matrix R (l) directly.

we have where is the reflection indicator .


To justify if non-Abelian relation compositions can be discovered by DihEdral in an ideal situation, we generate a synthetic dataset called FAMILY.

we first generated two generations of people with equal number of male and females in each generation, and randomly assigned spouse edges within each generation and child and parent edges between the two generations, after which the sibling, parent_in_law and   3 where r 3 is treated as the composition of r 1 and r 2 .


We thank the anonymous reviewers for insightful comments.

this work has been funded by the Klaus tschira Foundation, Heidelberg, Germany, and partially funded by the German Research Foundation as part of the Research training the NER tags in WikiAnn for the language in question are well-balanced for higher-ranked languages on the left and become more skewed for lower-ranked languages towards the right.


In this section, we will answer the research questions raised in §1 by focusing on the effectiveness of our augmentation framework.

we will explore three types of constraints by augmenting: 1) intermediate decisions (i.e.


The goal of sequence-to-sequence learning is to learn a function for mapping an input sequence x into an output sequences y.

for the example of machine translation, where y is a translation of x, the model, parametrized by a set of weights θ, learns to maximize p θ (y | x).


the Chart parser has highest performance in all four domains.


In our experiments, we demonstrate that the automatically adapted sentiment dictionary outperforms the previous state of the art in predicting the financial outcomes excess return and volatility.

automatic adaptation performs better than manual adaptation.


In our experiments, we demonstrated that the automatically adapted sentiment dictionary H4N RE outperforms the previous state of the art in predicting the financial outcomes excess return and volatility.


Although the reading difficulty of a text partly contributes to the overall exercise difficulty of C-tests, there are many other factors with a substantial influence (Sigott, 1995).

we can generate many different C-tests from the same text and thus reading difficulty and text simplification alone are not sufficient to determine and manipulate the difficulty of C-tests.


We use the same data (819 filled C-tests), metrics, and setup as Beinborn (2016).

we perform leave-one-out cross validation (LOOCV) and measure the Pearson correlation ρ, the rooted mean squared error RMSE, and the quadratic weighted kappa qwκ as reported in the original work.


We finally evaluate how well each strategy S reaches a given target difficulty.

we sample a random corpus text and τ , create the C-test using strategy S, predict the test difficulty d(T ) and measure its difference to τ using RMSE.


As we described previously, the proposed method stemmed from a specific need in the banking industry where a large number of incidents had to be mapped to a newly defined taxonomy of operational risks.

it was designed to avoid the tedious and time consuming effort of asking experts to manually review thousands of incidents.


Therefore, we analyze each conference call at sentence level.

we want to represent a conference call as a sequence of sentences with corresponding audio clips.


Prior research (Kogan et al., 2009;Rekabsaz et al., 2017) uses only shallow machine learning model (such as logistic regression) and bag-of-word features to represent financial documents.

the relation and dependencies among the sentences are largely ignored.


BiLSTM is the bidirectional LSTM (Hochreiter and Schmidhuber, 1997), which is an extended model of recurrent neural network (RNN).

lSTM is designed to acquire key information from time series data while overcoming the defect that traditional RNN might lose information in long time series.


The gain of MDRM over past volatility baseline v past diminishes from 54% (τ = 3) to 6% (τ = 30).

short term volatility prediction is much more difficult than long term prediction.


This can also be confirmed that when τ = 30, the MSE of past volatility method is as small as 0.231, which is even better than tf-idf bagof-words model and is only slightly worse than MDRM.

the benefit of using complex deep model for long-term volatility prediction is smaller than for short-term volatility prediction.


Prior to trying to detect concealed information, we first ask why might we be able to do so systematically, as opposed to random guessing?

what makes concealed information detectable?


We design a systematic set of auxiliary losses, enforcing the separation of style and content latent spaces.

the multi-task loss operates on a latent space to ensure that the space does contain the information we wish to encode.


Style-Transfer Accuracy (STA): We follow most previous work (Hu et al., 2017;Shen et al., 2017;Fu et al., 2018) and train a separate convolutional neural network (CNN) to predict the sentiment of a sentence (Kim, 2014), which is then used to approximate the style transfer accuracy.

we report the CNN classifier's accuracy on the style-transferred sentences, considering the target style to be the ground-truth.


We use masked language model probabilities computed by pretrained BERT model 2 as an additional feature during rescoring to further improve our sentence-level baseline and our cross-sentence GEC model.

we replace tokens in a hypothesis T with the [MASK] token, one at a time, predicting the log probability of the masked token using BERT.


Importantly, we are interested in investigating, and hopefully mitigating, possible sources of the noise that might be found in some of the models.

we hypothesize that the alignment procedure adds considerable noise to the acd, and plan to test how TR can alleviate some of that noise.


Finally, we evaluated on a small, handcrafted set of change and stable words and found that SGNS with Temporal Referencing gives the largest separation between words that undergo semantic change and those that stay stable over time.

we observe a similar behavior between this smaller testset and the synthetic sense injection, supporting our sense injection method as a good proxy for isolating and studying lexical semantic change.


Furthermore, adversarial training is employed between two attention layers to learn better word weights via a discriminator.

a shared attention layer is incorporated to learn public word weights between two emotion dimensions.


The word scared and disaster in red suggest two emotion dimensions.

scared suggests a low Valence score and a low Dominance score, while Disaster denotes a low Valence score and a high Arousal score.


In this way, better weight information can be learned to represent words' importance for rating dimensional scores.

our proposed AAN has two features: • First, AAN conducts adversarial learning between two attention layers to decide the val-ues of words for rating two emotion dimension scores.


Specifically, our proposed AAN has two features: • First, AAN conducts adversarial learning between two attention layers to decide the val-ues of words for rating two emotion dimension scores.

we propose an adversarial training algorithm to learn two sets of better word weights which contribute to two emotion dimensions in two attention layers.


It trains two regressors for two emotion dimensions in a joint learning style without any adversarial training technique.

this approach has the similar structure to AAN, except the absence of the discriminator.


In the Letters domain, the performance in all dimensions reaches a high level in r-value compared with those in other domains.

there is no extremely low r-value (<0.20) in any dimension of either Reader's emotion or Writer's emotion.


We can conclude from Table 2 that HFFN achieves consistent improvements on accuracy and F1 score in IEMOCAP 4-way and individual emotion recognition tasks compared with other methods.

hFFN outperforms other methods by a significant margin on the recognition of Angry and Neutral emotions.


This shows some potential positive link between the two levels of attention mechanism.

rIA can provide more refined information during transmission between local interactions, so that the output states to be processed by GIA are more focused on useful information and freer of noise, maximizing the effect of GIA.


IMN introduces a novel message passing mechanism that allows informative interactions between tasks.

it sends useful information from different tasks back to a shared latent representation.


This component consists of a word embedding layer followed by a few feature extraction layers.

we employ m s layers of CNNs after the word embedding layer in f θs .


In addition, we also allow messages to be passed between the components in each iteration.

for this problem, we send information from the AE task to the AS task as shown in Figure 1.


AE aims to extract all the aspect and opinion terms 5 appearing in a sentence, which is formulated as a sequence tagging problem with the BIO tagging scheme.

we use five class labels: Y ae = {BA, IA, BP, IP, O}, indicating the beginning of and inside of an aspect term, the beginning of and inside of an opinion term, and other words, respectively.


As shown in Figure 1, we makeŷ ae i , the outputs from AE available to AS in the selfattention layer, as the sentiment task could benefit from knowing the predictions of opinion terms.

the self-attention matrix A ∈ R n×n is computed as follows: where the first term in Eq.


To exploit interactions between different tasks, the message passing mechanism aggregates predictions of different tasks from the previous iteration, and uses this knowledge to update the shared latent vectors {h s 1 , ..., h s n } at the current iteration.

the message passing mechanism integrates knowledge fromŷ ae i ,ŷ as i ,ŷ ds , a ds i , and a dd i computed on an input {x 1 , ..., x n }, and the shared hidden vector h s i is updated as follows: where t > 0 and [:] denotes the concatenation operation.


IMN further boosts the performance and outperforms the best F1-I results from the baselines by 2.29%, 1.77%, and 2.61% on D1, D2, and D3.

for AE (F1-a and F1-o), IMN −d performs the best in most cases.


The first is related to propagation of the errors encountered during C and A extraction to the similarity identifier and AR identifier affecting the overall performance.

when a word is incorrectly identified as part of C or A, their similarity measure is affected and then the decision about the AR.


This is also true for their associated emotions.

the context acts as a set of parameters that may influence a person to speak an utterance while expressing a certain emotion.


To accomplish this, we crawl through the subtitles of all the episodes and heuristically extract the respective timestamps.

we enforce the following constraints: 1.


Next, we prune redundant spans in R using the non-maximum suppression algorithm (Rosenfeld and Thurston, 1971).

we remove the span r l that possesses the maximum score u l from the set R and add it to the set O (line 10-11).


Instead, we propose to summarize the target representation from contextual sentence vectors according to its span boundary, and use feed-forward neural networks to predict the sentiment polarity, as shown in Figure 3(b).

given a target span r, we calculate a summarized vector v using the attention mechanism (Bahdanau et al., 2014) over tokens in its corrsponding bound (s i , e j ), similar to  and He et al.


However, their model only shares shallow embedding and LSTM layers between ASC and DSC (document-level sentiment classification) tasks.

the document-level knowledge is merely used for improving the word representations in ASC.


This clearly demonstrates the effectiveness of these components.

transCap-A performs the worst, since it cannot generate aspect-related feature capsules after removing aspect routing from transCap.


In this paper, we propose a progressive self-supervised attention learning approach for neural ASC models, which automatically mines useful attention supervision information from a training corpus to refine attention mechanisms.

we iteratively conduct sentiment predictions on all training instances.


Specifically, we iteratively conduct sentiment predictions on all training instances.

at each iteration, the context word with the maximum attention weight is extracted as the one with active/misleading influence on the correct/incorrect prediction of every instance, and then the word itself is masked for subsequent iterations.


However, the existing attention mechanism in ASC suffers from a major drawback.

it is prone to overly focus on a few frequent words with sentiment polarities and little attention is laid upon low-frequency ones.


To this end, we iteratively conduct sentiment predictions on all training instances.

at each iteration, we extract the context word with the maximum attention weight from each training instance to form attention supervision information, which can be used to guide the training of attention mechanism: in the case of correct prediction, we will remain this word to be considered; otherwise, the attention weight of this word is expected to be decreased.


Input: D: the initial training corpus; θ init : the initial model parameters; α: the entropy threshold of attention weight distribution; K: the maximum number of training iterations; 1: sm(x) ← ∅ 5: end for 6: for k = 1, 2..., K do 7: x ← MaskWord(x, sa(x), sm(x)) 11: if yp == y then 17: end for 24: effects, whose attention weights are expected to be decreased.

at the k-th training iteration, we adopt the following steps to deal with (x, t, y): In Step 1, we first apply the model parameters θ (k−1) of the previous iteration to generate the aspect representation v(t) (Line 9).


Hence, we extract the context word x m with the maximum attention weight (Line 15-20) that will be exploited as attention supervision information to refine the model training.

we adopt two strategies to deal with x m according to different prediction results on x : if the prediction is correct, we wish to continue focusing on x m and add it into s a (x) (Lines 16-17); otherwise, we expect to decrease the attention weight of x m and thus include it into s m (x) (Lines 18-19).


Figure 3 and 4 show the experimental results of different models.

mN(+AS) with α =3.0 achieves the best performance, meanwhile, the optimal performance of TNet-ATT(+AS) is obtained when α =4.0.


Adopting our method, attention weights of "cute" in training instances with neural or negative polarity are significantly decreased.

in these instances, the average weight of "cute" is reduced to 0.07 times of the original.


In this paper, we treat these two tasks as two sequence labeling problems and propose a novel Dual crOss-sharEd RNN framework (DOER) to generate all aspect termpolarity pairs of the input sentence simultaneously.

dOER involves a dual recurrent neural network to extract the respective representation of each task, and a cross-shared unit to consider the relationship between them.


Thus, we also use a sentiment lexicon to guide ASC.

we train an auxiliary word-level classifier on the branch of ASC for discriminating positive words and negative words based on the sentiment labelsŶ S p .


We compare pragmatic and non-pragmatic models at training and at test, while varying conditions on the training data to test hypotheses regarding the utility of pragmatic inference for learning.

we show that incorporating pragmatic reasoning at training time yields improved, state-of-the-art accuracy for listener models on the color reference task from Monroe et al.


We perform this comparison repeatedly to evaluate the value of pragmatics at training and test under various data conditions.

we evaluate the hypotheses that (1) the pragmatic inferences enabled by the l 1 training will reduce sample complexity, leading to more accurate meaning functions especially under small data sizes, and (2) the effectiveness Algorithm 1 RSA pragmatic listener (l 1 ) neural network forward computation.


This might include scaling with linguistic complexity and properties of referents.

the argument in our introduction suggests that especially frequent objects and low-cost utterances are the seed from which pragmatic inference can proceed over more complex language and infrequent objects.


We extend this model to a MTL framework by training it to optimize a secondary objective at the bottom layers of the network, along with the main one (L 1 ).

the model is trained to predict a documentlevel score along with word-level labels indicating the (predicted) GR type of dependents in the document.


To evaluate model performance on this dataset, we again follow previous work (Barzilay and Lapata, 2008; Tien Nguyen and Joty, 2017) and calculate pairwise ranking accuracy (PRA) between an original text and its 20 permuted counterparts.

pRA calculates the fraction of correct pairwise rankings in the test data (i.e., a coherent/original text should be ranked higher than its permuted counterpart).


2 In addition to the WSJ, we employ this dataset in order to assess the effectiveness of our coherence model for tasks involving the prediction of varying degrees of coherence in people's everyday writing.

the dataset contains texts from four domains: Yahoo online forum posts, emails from Hillary Clinton's office, emails from Enron and Yelp business reviews.


We thus perform an additional level of percolation over the marginals to incorporate the children's children of the tree.

after equation 4, we calculate: Third, the max-pooling layer gives the model a way of aggregating over sentences while ignoring the learned structure.


More generally, however, uncertain knowledge of the world that surrounds us, including novel objects, is not only a machine learning challenge: it is simply a very common aspect of human communication, as speakers rarely have perfect representations of their environment.

the richness of verbal interaction allows us to communicate these uncertainties and to collaborate towards communicative success (Clark and Wilkes-Gibbs, 1986).


In this paper, we introduce an end-to-end reinforcement learning based coreference resolution model to directly optimize coreference evaluation metrics.

we modify the state-of-the-art higherorder mention ranking approach in Lee et al.


In this paper, we propose a goal-directed endto-end deep reinforcement learning framework to resolve coreference as shown in Figure 1.

we leverage the neural architecture in  as our policy network, which includes learning span representation, scoring potential entity mentions, and generating a probability distribution over all possible coreference linking actions from the current mention to its antecedents.


2017, we formulate the task as a sequence of linking decisions for each span i to the set of its possible antecedents, denoted as , a dummy antecedent and all preceding spans.

the use of dummy antecedent for a span is to handle two possible scenarios: (i) the span is not an entity mention or (ii) the span is an entity mention but it is not coreferent with any previous spans.


The problem of entity-level representation (also referred to as high-order coreference models) has attracted considerable interest recently, with methods ranging from imitation learning (Clark and Manning, 2015) to iterative refinement .

tackled this problem by iteratively averaging the antecedents of each mention to create mention representations that are more "global" (i.e., reflect information about the entity to which the mention refers).


One obvious disadvantage of generative models is that they maximize the likelihood of training text but never see the incoherent text.

to produce a binary classification decision about coherence, such a generative model only sees data from one class.


However, although incoherent next sentence is generally not predictable given the context, the inverse is not necessarily true.

a coherent sentence does not need to have high conditional loglikelihood, as log likelihood can also be influenced by other factors such as fluency, grammaticality, sentence length, and the frequency of words in a sentence.


Through experimental validation, we found that margin loss to be superior for this problem.

l takes on the form: where η is the margin hyperparameter.


Therefore, we only select sentences from the same document to construct negative pairs.

suppose s i comes from document d k with length n k , then p(s |s i ) is a uniform distribution over the n k − 1 sentences {s j } j = i from d k .


Therefore, we designed the much harder Wiki-C and Wiki-D to check whether the coherence models capture cross-domain transferrable features.

in the Wiki-D setting, we even test whether the models trained on Person articles from Wikipedia generalize to WSJ articles.


We would like to note that in both cross-dialect settings, we use the validation set from the same dialect as the training set, in order to prevent any use of information about the test dialect during training.

the settings are intra-dialect with re-  Table 2: Accuracy rates, weighted F 1 scores and macro-averaged F 1 -scores (in %) for the five evaluation tasks: binary classification by dialect, Moldavian intra-dialect 6-way categorization (by topic), MD→RO cross-dialect 6way categorization, Romanian (RO) intra-dialect 6-way categorization, and RO→MD cross-dialect 6-way categorization.


This limits our understanding of CLE methodology as: 1) BLI is an intrinsic task, and agreement between BLI and downstream performance has been challenged (Ammar et al., 2016;Bakarov et al., 2018); 2) BLI is not the main motivation for inducing cross-lingual embedding spacesrather, we seek to exploit CLEs to tackle multilinguality and downstream language transfer (Ruder et al., 2018b).

previous research does not evaluate the true capacity of projectionbased CLE models to support cross-lingual NLP.


By systematically evaluating CLE models for many language pairs on BLI and three downstream tasks, we shed new light on the ability of current cutting-edge CLE models to support cross-lingual NLP.

we have empirically proven that the quality of CLE models is largely task-dependent and that overfitting the models to the BLI task can result in deteriorated performance in downstream tasks.


We reckon that VI is strongly influenced by frequency.

the more frequent a word is in corpus C b , the more its vector will be updated after initialization on C a .


Furthermore, various prior analyses with particular semantic groups are available for comparison and replication (e.g., cases that involve paraphrasing or coreference (Chen et al., 2016;Weissenborn et al., 2017;Wadhwa et al., 2018)).

we analyze Bi-Directional Attention Flow (BiDAF) (Seo et al., 2016) on SQuAD v1.1 (Rajpurkar et al., 2016) in the rest of the paper.


We also asked participants to describe their impressions with free-form comments, which were very positive for all of them -all thought Errudite enhanced their error analysis experience.

four users stated that they felt it systematically scaled up the analysis, making it more precise and thus inspiring more confidence.


While often insightful, these suffer from potential pitfalls similar to labeling individual instances: they are laborious, often subjective, and hard to reproduce.

just as in manual error labeling mentioned in §1, typical automatic grouping also struggles with the trade-off between being reproducible/scalable, and being in-depth and meaningful.


To select documents for human annotation, we align Wikipedia documents with Wikidata under the distant supervision assumption (Mintz et al., 2009).

we first perform named entity recognition using spaCy 2 .


To assess the challenges of DocRED, we conduct comprehensive experiments to evaluate state-ofthe-art RE systems on the dataset.

we conduct experiments under both supervised and weakly supervised benchmark settings.


Table 6 shows that the aforementioned features all have a contribution to the performance.

entity types contribute most due to their constraint on viable relation types.


This study also implies that idiom representation is a key factor for the success of Chinese reading comprehension with idiom.

a good cloze test model should have not only a proper model structure, but also a good method to represent idioms.


Assuming that our model has K topics, and once again using z i as the topic assignment for token w i , we consider two partitions S = {S 1 , ..., S K } and T = {T 1 , ..., T K } of the set of tokens w, such that S i = {w j | z j = i} and Variation of information is where H z [•] is entropy with respect to topic distribution and MI (S, T ) is the mutual information between S and T .

we measure how much information we lose in our topic assignments if we reassign every token to the topic of the token that follows.


The threshold can be adjusted to create models for different purposes.

in this paper we experimented with two models: one optimized for high recall, the other for high F1.


These architectures are encoder-decoder architectures where the encoder extracts three different sets of features: task, task group, and task universe features, and the decoder produces desired outputs based on feature representations.

the first one ( Figure 1) is a parallel MTL architecture where task, task group, and task universe features are encoded in parallel and then concatenated to produce a composite representation.


Although we conduct experiments on multi-domain NLU systems of virtual assistants, the architectures can easily be applied to other tasks.

the encoder/decoder could be instantiated with any components or architectures, i.e., Bi-LSTM (Hochreiter and Schmidhuber, 1997) for the encoder, and classification or sequential labeling for the decoder.


SE   Overall, on both benchmark data and Alexa data, our architectures with group encoders show better results than others.

the serial architecture with highway connections achieves the best mean Slot F1 of 94.8 and 97.2 on Snipsmusic and Snips-location respectively and median Slot F1 of 81.99 on the Alexa dataset.


In addition, when we attempt to learn three sets of features -task, task universe, and task group features -the serial architecture for feature learning helps.

when we have datasets from many domains, learning task features in the first stage and common features, i.e., task universe and task group features, in the second stage yields the best results.


For this reason, a set of possible alignments is maintained.

when accepting a non-terminal, all possible nodes in the input that it may correspond to are identified and a state is maintained for each possibility.


Figure 1 illustrates a motivating dialog example between two conversation participants, which spans multiple related KG entities from a starting seed entity The Catcher in the Rye.

we observe that there exists a small subset of walkable patterns within a KG or a preferred sequence of graph traversal steps which often leads to more engaging entities or attributes than others (e.g.


To this end, we propose a new model called DialKG Walker that can learn natural knowledge paths among entities mentioned over dialog contexts, and reason grounded on a large commonsense KG.

we propose a novel graph decoder that attends on viable KG paths to predict the most relevant entities from a KG, by associating these paths with the given input contexts: dialog, sentence, and a set of starting KG entities mentioned in the previous turn.


Figure 2 illustrates the overall architecture of the DialKG Walker model which retrieves a set of entities from a provided KG given multiple modalities of dialog contexts.

for each turn the model takes as input a set of KG entities mentioned at its current turn, a full sentence at the current turn, and all sentences from previous turns of dialog, which are encoded using Bi-LSTMs with self-attention modules (Section 2.2).


We compute zeroshot relevance score in the KG embeddings space, thus allowing for robust prediction for KG entities and domains unseen during training as well.

we use the supervised hinge rank loss for KG embeddings prediction as a choice of L f , defined for each sample (Moon and Carbonell, 2017).


It can be seen that the proposed Di-alKG Walker model outperforms other state-ofthe-art baselines, especially for recalls at small ks.

when textual contexts are added as input (E+S and E+S+D), the model learns to condition its walk path output on textual contexts, thus outperforming the non-textual ablation model (E).


Transfer learning: In Figure 3, we show that cross-domain performance can greatly improve with a relatively small addition of in-domain target data, via the transfer learning approaches.

it can be seen that (TL:Adv), which simultaneously trains for both source and target data (effectively doubling the training size) with additional adversarial discriminator for source and target domains, achieves the best performance especially for domains that are semantically close (e.g.


For simplicity and generalization consideration, we use the string match in our model for knowledge extraction.

for each triplet t ∈ G where the head and tail of t are both lists of words, if its head is the same as the string of s, we consider it to be a related triplet.


Stanford parser 6 is employed to generate plurality, animacy, and gender markups for all the noun phrases, so as to automatically generate linguistic knowledge (in the form of triplets) for our data.

the plurality feature denotes each s and p to be singular or plural.


Overall, all models perform significantly worse if they are used cross domains.

if we train the End2end model on the CoNLL dataset and test it on the i2b2 dataset, it only achieves 75.2 F1.


We define this distribution using a shared representation between our new estimator p φ,θ (y | H) and p θ (y | P, H).

the two share an embedding of H from the hypothesis encoder f H,θ .


11 Many NLI models encode P and H separately (Rocktäschel et al., 2016;Mou et al., 2016;Liu et al., 2016;Cheng et al., 2016;Chen et al., 2017), although some share information between the encoders via attention Duan et al., 2018).

12 representations are concatenated, subtracted, and multiplied element-wise.


To integrate and reason over information from multiple pieces of evidence, we propose a graph-based evidence aggregating and reasoning (GEAR) framework.

we first build a fully-connected evidence graph and encourage information propagation among the evidence.


Given an input sentence, we employ BERT (Devlin et al., 2019) as our sentence encoder by extracting the final hidden state of the [CLS] token as the representation, where [CLS] is the special classification embedding in BERT.

given a claim c and N pieces of retrieved evidence {e 1 , e 2 , ..., e N }, we feed each evidence-claim pair (e i , c) into BERT to obtain the evidence representation e i .


Specifically, given a claim c and N pieces of retrieved evidence {e 1 , e 2 , ..., e N }, we feed each evidence-claim pair (e i , c) into BERT to obtain the evidence representation e i .

we also feed the claim into BERT alone to obtain the claim presentation c. Note that we concatenate the evidence and the claim to extract the evidence representation because the evidence nodes in the reasoning graph need the information from the claim to guide the message passing process among them.


Initially, our model for locating the tags is unlikely to be reliable, so we adopt a curriculum learning paradigm.

we provide the classification stage the reference location of the tag from the training data with probability p, and the inferred location of the tag with probability 1 − p. We start the joint multi-task training by setting this probability to 1 and decrease it as training progresses .


In this paper, we explore several linguistic aspects of the collaboration process occurring during counseling conversations.

we address the differences between high-quality and low-quality counseling.


Following this line of work, this paper presents the development of a counseling dataset that can be used to implement data-driven methods for the automatic evaluation of counseling quality.

we conduct several linguistically inspired analyses on high-quality and low-quality counseling interactions with the final goal of providing insights into conversational strategies used by good counselors.


Thus, we restricted our search to video titles indicating that the counseling was conducted using MI and (optionally) including information about the quality of the interaction.

we use keywords such as effective MI, using MI, good MI, MI counseling demonstration, role play MI, ineffective MI, bad MI, bad MI counseling, and how not to do MI.


In addition to our empirical assessments of counseling quality, for each conversation in the final set, we obtain standard measurements of MI adherence, which can be used as a proxy of MI quality.

we annotate two micro-skills that are frequently used in the evaluation of MI counseling: reflective listening and questions (Tollison et al., 2008).


Since the quality of counseling interventions relies highly on the active collaboration between counselors and clients, we analyze several linguistic aspects of the conversation in relation to counseling quality.

we focus on language exchange patterns over the conversation, sentiment trends, linguistic alignment, and topics discussed during the conversation.