We choose 6 representative methods with the best performance: SVM+LDA, CNN-pretrain, LSM-  pretrain, GCN-HIN, TextGCN and HGAT, to study the impact of the number of labeled documents.

we vary the number of labeled documents per class and compare their performance on the AGNews dataset.


In this paper, we propose a novel heterogeneous graph neural network based method for semisupervised short text classification, which takes full advantage of both limited labeled and large unlabeled data by information propagation.

we first present a flexible HIN framework for modeling the short texts, which can integrate any additional information and capture their rich relations to address the semantic sparsity of short texts.


With the argument that a piece of news often describes one or several events which could be represented as a collection of elements, this study extracts news elements as enti- ties, actions and nouns from news titles and texts, with respect to their tf-idf scores.

the tf-idf is a well recognized tool to measure the importance of a word/phrase inside the document and to extract the representative elements at the document level.


Four experiments are implemented and explored in this section.

we first visualize the News2vec embeddings in terms of the news vectors and feature vectors with the dimension reduction plot and the correlation heat maps.


Moreover, the distance between two clusters is in-line with the relevancy between the clustered topics in reality.

neighboring relationships are exhibited between similar topics such as sports & lottery ticket, current affairs & society, stock & finance, and fashion & entertainment, whilst the distance between less relevant clusters, such as constellation & finance, game & education, is relatively larger.


In this paper, we introduce a novel approach to LS that addresses these issues.


We also experiment with using the MedLit dataset as source, both alone and together with data from a source EHR dataset.

for each target EHR dataset we compare the following models: • ClvC: train on labeled data from ClvC EHRs In all cases we evaluate our model on our two test sets: ClvC and i2b2.


The best average F-score was 65.2% using the RNN and 71.2% for BERT on the sentence classification task.

we found that Assessment and Plan performs quite poorly.


Extensive experiments on four benchmark datasets validate the effectiveness of our approach.

learning user and item representations is critical for recommendation (Tay et al., 2018).


Thus, we aim to incorporate the external commonsense knowledge, i.e., intent and sentiment, into the learning process to generate better event representations.

we propose a simple and effective model to jointly embed events, intents and emotions into the same vector space.


However, complex deep learning models are often sensitive and vulnerable to a tiny modification.

malicious attackers can destroy the models by adding a few inconspicuous perturbations into input data, such as masking images with unrecognizable filters and making low-key modifications for texts.


Finally, we recover the input sentence based on the predicted recover embeddings from the embedding estimator.

the input text X needs to be recovered from the perturbed text X a by fixing tokenlevel perturbations based on its approximate embeddings.


However, the improvements are not consistent in IMDb.

sC performs worse with lower discrimination accuracy and over-correcting the documents.


The EoG model outperforms all baselines for all pair types.

for the inter-sentence pairs, performance significantly drops with a fully connected graph (Full) or without inference (NoInf).


To alleviate this issue, we adopt the denoising auto-encoder (DAE) following the previous work (Lample et al., 2019) that tries to reconstruct the corrupted input.

we randomly shuffle a small portion of words in the original sentence to the corrupted input which are then fed to the denoising auto-encoder.


To build the pseudo-parallel corpus of style s a and s b , we first samples 80,000 sentences in style s a as queries.

each query searches a pseudo-reference from the nonparallel corpus of style s b according to the TF-IDF based cosine similarity.


FPDG takes the entity label of each word into account, since the product attribute information is always conveyed by entity words.

we first propose a Recurrent Neural Network (RNN) decoder based on the Entity-labelguided Long Short-Term Memory (ELSTM) cell, taking both the embedding and the entity label of each word as input.


Experiments conducted a large-scale real-world product description dataset show that our model achieves the state-of-the-art performance in terms of both traditional generation metrics as well as human evaluations.

fPDG increases the fidelity of the generated descriptions by 25%.


Since product attribute information is always conveyed by entity words (up to 89.72% in input keywords), FPDG generates faithful product descriptions by taking into account the entity label of each word.

first, we propose an Entity-label-guided Long Short-Term Memory (ELSTM) as the cell in the decoder RNN, which takes the entity label of an input word, as well as the word itself as input.


Extensive experiments conducted on this dataset show that FPDG outperforms the state-of-the-art baselines in terms of traditional generation metrics and human evaluations.

fPDG greatly improves the fidelity of generated description by 24.61%.


The word hidden state w h t attends to the SAM outputs to predict the next word, while the entity label hidden state l h t is used to attend to the keyword memory and predict entity label of next word.

eLSTM is a hybrid of three LSTMs with a deeper interaction between these cells.


Performances of all ablation models are worse than that of FPDG in terms of almost all metrics, which demonstrates the necessity of each module in FPDG.

eLSTM makes the greatest contribution to FPDG, improving the BLeU-1, BLeU-2 scores by 2.71% and 1.51%.


To tackle this challenge, based on the consideration that product attribute information is typically conveyed by entity words, we incorporate the entity label information of each word to enable the model to have a better understanding of the words and better focus on key information.

we propose an Entity-label-guided Long Short-Term Memory (ELSTM) and a token memory to store and capture the entity label information of each word.


Two types shall have similar embeddings, if they are close in a graph.

types that are related by subClassOf relation shall have similar embeddings.


Other more recent approaches include two-level reinforcement learning models (Takanobu et al., 2019), two layers of attention-based capsule network models , and self-attention with transformers (Verga et al., 2018).

(Miwa and Sasaki, 2014;Katiyar and Cardie, 2016;Zhang et al., 2017;Zheng et al., 2017;Verga et al., 2018;Takanobu et al., 2019) also seek to jointly learn the entities and relations among them together.


Initially our model for locating the tags is unlikely to be reliable, and so we adopt a curriculum learning paradigm.

we provide the classification stage the reference location of the tag from the training data with probability p, and the inferred location of the tag with probability 1 − p. We start the joint multi-task training by setting this probability to 1 and decrease it as training progresses (Bengio et al., 2015).


The majority of the errors result from our model missing the property span.

we see that 35% and 81% of the errors are due to model not detecting medications and symptoms property.


The Rectified Anchor Word (RAW) algorithm enjoys its transparent and consistent behaviors, working seamlessly on various types of textual and non-textual real datasets.

our AP+ADMM algorithm outperforms the previous AP+ExpGrad (Lee et al., 2015), being more comparable to Gibbs sampling and less sensitive to parameters.


Finally we also encode the individual edit operations in both pre-edit l cooked Edit: changing the phrase "chicken wing" to "salmon".

the values -1, 1 and 0 are used to represent deletions, additions and unchanged tokens, respectively.


This is because of the imbalance between positive and negative samples in our data.

the number of negative samples is 4 times greater than the number of positive samples when the size of the candidate set is 5 -and even greater when it is 10.


The obvious way is merely removing duplicates.

on the assumption that log p(t) log p(t ) for all t ∈ T B and t / ∈ T B , Equation (1) implies that marginalization over the final beam outputs can provide better approximation: reranking after summing out the probability of duplicates can give better ordered list of candidates.


PMRR values are often omitted in the literature because of the similar tendency to MRR.

pMRR does not give much additional information about the quality of the generated results.


Experimental results show that both the global attention and symptom graph are effective to boost dialogue symptom diagnosis.

our proposed model achieves the state-of-the-art performance on the constructed dataset.


Furthermore, we propose an approach, which embeds a global attention and symptom graph to improve the performance of dialogue symptom diagnosis.

the global attention aims at incorporating more related information from the whole dialogue and corpus for better symptom entity representation, which will be used for symptom recognition and inference.


We report the results of the recent advanced baselines as well as the variants of our proposed method.

we compare the performance of the following models: • Bi-RNNs (Dyer et al., 2015): The models use LSTM or GRU for the sentence encoder, and treat symptom entity recognition as a classification problem with the softmax function.


Furthermore, by integrating either our corpus-level attention or document-level attention into the existing models, the performance can be significantly boosted.

our model with global attention achieves the best performance in terms of all metrics.


The results show that our proposed model with global attention significantly outperforms the Bi-LSTM CRFinference model for symptom inference across all the categories.

we achieve substantial improvements for inferring the False and Uncertain categories of symptoms, by utilizing the global information in the current dialogue and the whole corpus To investigate the effect of the symptom graph for symptom inference, we compare the models with and without symptom graphs.


We are further given an additional sentence s 2 , which is counterfactual to the initial context s 2 .

s 2 states something contrary to that in s 2 , which in turn can make the original ending s 3:5 no longer valid.


Motivated by this requirement and following the goal of developing unsupervised methods for counterfactual rewriting, we design a reconstruction objective for learning a noisy reproduction of the original ending.

we provide the model with both the original story and a masked context as input (S, [s], s 1 , [mask]) and train the model to reconstruct the original ending s 3:5 : where [s] denotes a separator token and [mask] is a special mask token.


Setup We conduct a pairwise comparison between the best model (GPT2-M + Sup) with other models along the same three dimensions as in the first evaluation setting (section 5.1).

crowdworkers were presented outputs of a pair of systems, and asked to choose which one is better, or "equally good" or "equally bad", in terms of each of the three criteria.


To better model the dependencies between the output tag labels, we propose a more powerful autoregressive decoder.

we run a singlelayer Transformer decoder on top of the BERT encoder (see Fig.


Green dashed arrow refer to policy gradient optimization process with reward r. To explicitly promote the semantic coherence between questions and answers, we propose a RL-CVAE model, as shown in Figure 2.

rL-CVAE is initialized by the pretrained CVAE model that acts as an agent in reinforcement learning.


When training GRU-MatchPyramid model, we use negative sampling method to construct the training data, which has been successfully applied in retrieval-based chatbot  and text matching task (Liu et al., 2016).

for each question (groundtruth) and answer pair (q, a) in the conversation generation training data, we randomly sample five negative questions q − , chosen from other posts' ground-truth responses.


Similar with GRU-MatchPyramid's training process, we create a coherence score set R − with negative examples, and use min-max scaling to normalize the coherence score r 0 + .

we randomly select n negative questions q 1 − , q 2 − , ... q n − from other posts's ground-truth responses, and then calculate their coherence scores with a via Equation 5, i.e.


We are aware that numerous uses of these techniques can pose ethical issues and that best practices will be necessary for guiding applications.

we note that people expect comments on news to be made by people.


The generation network then simulates how people comment the article after reading it, and acts as a decoder of the comment.

from the bottom to the top, the reading network consists of a representation layer, a fusion layer, and a prediction layer.


, (a w , e w )) denote the salient spans, where a i and e i refer to the start position and the end position of the i-th span respectively, we propose detecting the spans with a multi-label classifier and a pointer network.

we formalize the recognition of (a 1 , .


To speed up convergence, we initialize our model through pre-training the reading network and the generation network.

∀(T i , B i , C i ) ∈ D, we construct an artificial span setS i , and learn the parameters of the two networks by maximizing the following objective: S i is established in two steps: first, we collect all associated comments for (T i , B i ), extract n-grams (1 ≤ n ≤ 6) from the comments, and recognize an n-gram in B i as a salient span if it exactly matches with one of the n-grams of the comments.


As far as the standard VAE is concerned, it assumes that the latent code is Gaussian distributed so that we cannot distinguish which part of code controls the structure and which part controls the semantics.

it is difficult to generate effective texts by controlling text features directly.


Moreover, the encoder of the topic modeling component is served as a discriminator to force the decoder of the sequence modeling component to generate texts having the semantics as close to the original texts as possible.

no extra training supervised by labeled data is needed for this discriminator.


Different from existing work, our model combines a sequence modeling component and a topic modeling component so that the final latent code of our model contains both topic semantics and sequential structure of texts.

our model can learn these two features simultaneously, since AutoEncoding Variation Bayes (AEVB) are employed to train these two components.


(a) Draw topic assignment According to the generative process above and the marginalization of t i , the likelihood p(x) of the document x can be derived as where Further, we adopt AEVB and the reparameterization trick to achieve posterior inference and parameter learning.

similar to (Srivastava and Sutton, 2017), we first draw a Gaussian random vector by a reparameterization trick and then pass it through a softmax function to parameterize the multinomial document topic distributions.


Besides the reconstruction loss which drives the generator to produce realistic sentences, we introduce a discriminator which enforces the generator to produce texts in a coherent topic with z t .

we let the encoder in our topic modeling component act as the discriminator.


It is because that the role of the discriminator is to take the topic distributions as the supervised signals to improve the generation so that the sequence decoder can generate more topic relevant texts.

the discriminator does not improve the topic modeling component itself.


In our model, each dimension of the latent variables in the topic modeling component corresponds to a topic, therefore we can manipulate the variables manually to verify whether the generation can express a given topic while remaining the same structure.

given a document and its latent code, we change the topic part of the code and keep the structural part of the code unchanged.


The rest rows show the generated question-answering pairs when we change the original topic to a target topic while keeping the structure latent variable unmodified.

we modify the original topic distribution to a new distribution whose dimension of the target topic is 1 and others are 0s.


Experimental results show that our model has a better text modeling ability than several state-of-the-art methods and learns disentangled latent representations for texts which shows the superiority in a classification task.

our model can generate meaningful question-answering pairs, which provides an alternative transfer learning way and helps to broaden the knowledge in other fields.


(2018) in taking the features of detected objects as the embeddings of images.

the object detector detects m objects {o 1 , .


the softmaxnormalized score: An attention layer is called self-attention when the query vector x is in the set of context vectors {y j }.

we use the multi-head attention following Transformer (Vaswani et al., 2017).


Once the dataset is created, we formally define a new task, visual pronoun coreference resolution (Visual PCR), and design a novel visual-aware PCR model VisCoref, which can effectively extract information from images and leverage them to help better resolve pronouns in dialogues.

we align mentions in the dialogue with objects in the image and then jointly use the contextual and visual information for the final prediction.


Currently, the overwhelming majority of NLVL models are top-down approaches: they first cut a video into a set of segment candidates, then they do classification and regression for each candidate.

they can be further grouped into two sub-types: 1) sliding-window-based (Gao et al., 2017;Hendricks et al., 2017;Liu et al., 2018b,a;Ge et al., 2019;Chen and Jiang, 2019;Xu et al., 2019;: the video is explicitly segmented by multiple predefined temporal sliding-window scales.


In this paper, we propose a dense bottom-up framework for NLVL: DEnse Bottom-Up Grounding (DEBUG), to mitigate the problems in exist-ing NLVL frameworks.

we regard all frames falling in the ground truth segment as positive samples (i.e., foreground).


It calculates two different attention weights between the two modal features.

it first computes a similarity matrix S ∈ R T ×N , where S ij denotes the similarity between frame featuref i and word featurew j .


For each sample, there are three branch subnets, which aim to predict its classification score, boundary distances, and confidence score respectively.

the whole architecture of DEBUG is shown in Figure 4, and the details about the three branch subnets are: Classification Subnet.


We only assign boundary regression targets for positive frames.

for a positive frame at i-th position, if the ground truth segment range is (t s , t e ) (i.e., t s ≤ i ≤ t e ), the regression target is t where l * i and r * i represents the distance from i-th frame to the left and right boundaries, respectively.


More importantly, we propose a simple yet effective framework, CrossWeigh, to handle label mistakes during NER model training.

it partitions the training data into several folds and train independent NER models to identify potential mistakes in each fold.


We only consider the surface name in this entity set.

no matter "Chicago" is LOC or ORG, it only counts as its surface name "Chicago".


All training sentences that have entities included in test_entities i will be excluded in training process of the model M i .

(3) We call this step as entity disjoint filtering.


Here, we assign a weight w i to all sentences marked, while the weights of other sentences remain 1.

we set ∀i(1 ≤ i ≤ n), where is a parameter.


In practice, it can be chosen according to the quality of mistake estimation module.

we first estimate the precision of the detected mistakes of a single iteration.


This could   be also helpful to active learning.

in this experiment, we apply our noise estimation module to the concatenation of training and testing data.


In our experiments (see Table 6), we find that t = 3 provides a good enough result.

during mistake estimation, we have to choose the number of folds to partition the data.


In this work, we ask the question "how can we efficiently bootstrap a high-quality named entity recognizer for a low-resource language with only a small amount of human effort?"

we leverage recent advances in data-efficient learning for low-resource languages, proposing the following "recipe" for bootstrapping low-resource entity recognizers: First, we use cross-lingual transfer learning (Yarowsky et al., 2001;Ammar et al., 2016), which applies a model trained on another language to low-resource languages, to provide a good preliminary model to start the bootstrapping process.


Specifically, we leverage recent advances in data-efficient learning for low-resource languages, proposing the following "recipe" for bootstrapping low-resource entity recognizers: First, we use cross-lingual transfer learning (Yarowsky et al., 2001;Ammar et al., 2016), which applies a model trained on another language to low-resource languages, to provide a good preliminary model to start the bootstrapping process.


Inspired by this finding and considering the fact that named entities are both important and sparse, we propose an entity-targeted strategy to save annotator effort.

we select uncertain subspans of tokens within a sequence that are most likely named entities.


To compute the likelihood of a sequence where some labels are unknown, we use a constrained CRF which marginalizes out the un-annotated tokens.

let Y L denote the set of all possible sequences that include the partial annotations (for un-annotated tokens, all labels are possible), and we compute the likelihood as: To improve our model with the newly labeled data, we directly fine-tune the initial model, trained on the transferred data, on the data acquired through active learning, referred as FINETUNE.


In this paper, we focus on human-grounded evaluations of local explanation methods for text classification.

we propose three evaluation tasks which target different purposes of explanations for text classification -(1) revealing model behavior to human users, (2) justifying the predictions, and (3) helping humans investigate uncertain predictions.


(2) ArXiv Abstract is a text classification dataset we created by collecting abstracts of scientific articles publicly available on ArXiv 2 .

we collected abstracts from the "Computer Science (CS)", "Mathematics (MA)", and "Physics (PH)" categories, which are the three main categories on ArXiv.


As for the classifiers, we used 1D CNNs with the same structures for all the tasks and datasets.

we used 200-dim GloVe vectors as non-trainable weights in the embedding layer (Pennington et al., 2014).


To explain, we randomly selected only examples from the subclass 'Computation and Language', 'Dynamical Systems', and 'Quantum Physics' as training and validation examples for the class 'Computer Science', 'Mathematics', and 'Physics', respectively.

the training and testing data of the worse CNN came from different distributions.


In this paper, two mixture-prior generative models are proposed, under the objective to produce high-quality hashing codes for documents.

a Gaussian mixture prior is first imposed onto the variational auto-encoder (VAE), followed by a separate step to cast the continuous latent representation of VAE into binary code.


Motivated by this observation, in this paper, we propose to produce high-quality hashing codes by imposing appropriate mixture priors on generative models.

we first propose to model documents by a VAE with a Gaussian mixture prior.


To avoid the separate casting step used in GMSH, inspired by NASH , we further propose a Semantic Hashing model with a Bernoulli Mixture prior (BMSH).

we replace the Gaussian mixture prior in GMSH with the following Bernoulli mixture prior where γ k ∈ [0, 1] m represents the probabilities of z being 1.


When label information is available, it can be leveraged to yield more effective hashing codes since labels provide extra information about the similarities of documents.

a mapping from the latent representation z to the cor-responding label y is learned for each document.


The same network architectures as VDSH and NASH are used in our experiments to admit a fair comparison.

a two-layer feed-forward neural network with 500 hidden units and ReLU activation function is employed as the encoder and the extra classifier in the supervised case, while the decoder is the same as that stated in (2).


Comparing to strong baselines on three public datasets, the experimental results indicate that the proposed methods using mixture priors outperform existing models by a substantial margin.

the semantic hashing model with Bernoulli mixture prior (BMSH) achieves state-of-the-art results on all the three datasets considered in this paper.


To balance the effectiveness (retrieval of the nearest neighbor) and the efficiency (complete the process within limit time) of the retrieval, some empirical tricks are usually applied in previous search on graph methods: a) use directed edges instead of undirected edges; b) restrict the degree of outgoing edges for each node; and c) select more diverse outgoing edges (Malkov and Yashunin, 2018; Morozov and Babenko, 2018).

for the inner product case, ip-NSW proposed in (Morozov and Babenko, 2018) applies all tricks listed above (although the authors did not mention it in the paper, the implementation did inherit all features from HNSW).


Both evaluations confirm the superiority of our model, where the SP information is effectively preserved and the words' overall semantics are enhanced.

further analysis also indicates that our MWE embeddings are more powerful than all existing embedding methods in SP acquisition with around 1/20 in their size comparing to previous embeddings (Levy and Goldberg, 2014).


Formally, let W be the vocabulary set containing n words w 1 , w 2 , ..., w n and R the relations set containing m relations r 1 , r 2 , ..., r m , the proposed model is expected to produce m + 1 embeddings for each word, where there are m relational embeddings representing relations and a center em-bedding c for the general semantics.

each relational embedding has an overall and a local version, denoted as v and u, where u records the difference between v and c. For both the relational embeddings (v and u) and the center embedding, we use a similar way as that in word2vec (Mikolov et al., 2013) that each one has two variants to represent the semantics of a word w being the head or tail in different relations.


Moreover, considering there are various relations among words, we use a lower dimension s for storage compression, with s < d. Thus, a transformation matrix X is introduced to transform u into the same vector space of c. As a result, the final embeddings v for head and tail of w under relation r are formulated as: (1) To train v r h,w and v r t,w for each w under r, we adopt the negative sampling strategy to conduct the learning process.

for each r, we use a relation tuple set T r , with each tuple t = (w h , r, w t ) ∈ T r , where w h is the head word and w t the tail word.


end end end end drifting range a, whose value controls the distance between the local relational embeddings and the center embedding.

if X T u equals to a , which is greater than a, we scale down X T and u with a • 1 ka , where k is a scaling parameter set to 0.8 throughout this work.


There exist several studies for compressing neural language models.

quantization (Lin et al., 2016;Hubara et al., 2016) and groupwise low rank approximation (Chen et al., 2018a) achieve the state-of-the-art performance on tasks like language modeling and machine translation.


for each of the N way codebook, we use only one d m -dimensional vector to rescale the base codebook C. The number of parameters in S can thus be reduced and the computation of U becomes Now, with the N -way codebook defined by C and S, given an embedding matrix E with V vocabularies (i.e.


MulCode could be combined with quantization to achieve better performance.

the M × K base codebooks as well as the rescaling codebook could be quantized to further reduce the memory usage.


In addition, on machine translation task, it achieves 41.38X with BLEU score drops around only 1% after retraining.

we observe that the effect of retraining is more prominent for MulCode and simple quantization compared to GroupReduce.


Experimental results show that our method outperforms the state-of-the-art compression methods by a large margin.

on the IWSLT-14 data set, our method combined with quantization achieves 41.38 times compression rate for both the embedding and Softmax matrices.


English) aligned in the same space are also provided.

for the bilingual setting, we can use English word vectors to facilitate the bias mitigation.


For nouns with both gender forms, we aim at removing the gender inequality on the semantic gender direction as shown in Figure 1 and Figure 2.

we minimize the difference between the distance of feminine form of the word w f and the masculine form of the word w m to an anchor point (e.g., the origin point) on the semantic gender direciton d s : (3) Note that for most cases, w m and w f lie on the opposite sides of the anchor point on the semantic gender axis.


Mitigating Before Alignment (De-Align) As gender bias can be observed when aligning gendered languages with language without grammatical gender, we consider using English to facilitate mitigating bias.

we first apply the "hard-debasing" approach Bolukbasi et al.


Hybrid Method (Hybrid) We also consider a hybrid method that integrates the aforementioned two approaches.

we first mitigate English embeddings, align English embeddings with the embeddings of gendered languages, and then shift words in languages with grammatical gender along the semantic gender direction.


Following the notion of analogy, we design an evaluation task to test how bilingual word embeddings translate an English occupation with the presence of another word.

given an English word E i (a noun, an adjective or a verb), an English occupation word E o , and the corresponding Spanish/French translation S i of E i , we adopt the analogy test "E i :E o = S i :?"


Similar to the nearest neighbor matching model, we can feed the hidden vector of BERT in the last layer, h L i , into an affine transformation followed by softmax.

h i in Equation 7 is instan-tiated by h L i .


To take into account all layers, we compute the weighted sum of all hidden layers, h l i , where 1 ≤ l ≤ L, corresponding to a word position i, through attention mechanism.

h i in Equation 7 is replaced by the following equation: where m ∈ R d model is a projection vector to obtain scalar values with the key vectors.


P2F attention represents y j using the similarities between y j and every word in x.

let f j = softmax(A :,j ), where f j ∈ R n denotes the attention weights on x according to y j .


The nodes involved in the math expression trees can be classified into two categories, namely, operator and quantity nodes.

operator nodes are the tree nodes that define the types of operations involved in expressions.


In this work, we adopt an approach that is inspired by (Lu et al., 2008;Lu, 2014), which learns a latent joint representation for words and semantics in the form of hybrid trees where word-semantics correspondence information is captured.

we introduce a text-math tree representation that jointly encodes both text and the math expression tree.


We will then use these three symbols to define the set of word association patterns, which are used to specify how the words interact with the sub-trees of the current node.

for expression tree nodes with arity 0 (i.e., quantity nodes), only one pattern w is allowed to be attached to them, indicating that a contiguous word sequence from a given problem text is associated with such expression tree nodes.


Instead of directly computing the summation over all possible latent text-math structures, we essentially replace the by the max operation inside the arg max.

we first find the latent text-math tree t * which yields the highest score and contains the input text x.


Despite such a diversity, in this paper, we manage to develop a simple and effective neural SRL model to integrate syntactic information by applying argument pruning method in a uniform way.

we introduce new pruning rule based on syntactic parse tree unlike the k-order pruning of , which is only simply determined by the relative distance of predicate and argument.


Therefore, we introduce language-specific rule based on syntactic dependency parses to prune some unlikely arguments, henceforth syntactic rule.

given a predicate p and its argument a, we define d p and d a to be the distance from p and a to their nearest common ancestor node (namely, the root of the minimal subtree which includes p and a) respectively.


In the argument pruning layer, our model drops these candidate arguments (more exactly, BiLSTM representations) which do not comply with the syntactic rule.

only the predicates and arguments that satisfy the syntactic rule will be output to next layer.


Results indicate that high-quality syntax can further improve model performance, showing syntactic information is generally effective for multilingual SRL.

based on gold syntax, the top-1 argument pruning for Catalan and Spanish has reached 100 percent coverage (namely, for Catalan and Spanish, all arguments are the children of predicates in gold dependency syntax tree), and hence our syntax-aware model obtains significant gains of 1.43% and 1.35%, respectively.


BERT is also a transformer encoder that has access to the entire input but this choice requires a special training regime.

they multi-task between predicting a subset of masked input tokens, similar to a denoising autoencoder, and a next sentence prediction task.


In future work, we will investigate variations of our architecture.

we had initial success sharing the parameters of the two towers which allows training much deeper models without increasing the parameter count.


(2016) point out three distinguishing characteristics of relevance matching: exact match signals, query term importance, and diverse matching requirements.

exact match signals play a critical role in relevance matching, more so than the role of term matching in, for example, paraphrase detection.


For all the candidate arguments, we simplify the representations by employing the mean operation over the BiLSTM outputs within the corresponding argument spans, which achieves similar results compared with the attention-based span representations (He et al., 2018a) on English SRL in our preliminary experiments.

formally, for word-based SRL, we only need to set the length of candidate arguments to be 1.


Unlike them, we inject semantic relations of finer granularity using phrasal paraphrase alignments automatically iden-tified by Arase and Tsujii (2017) to improve semantic equivalent assessment tasks.

our method learns to discriminate phrasal and sentential paraphrases on top of the representations generated by BERT.


In this paper, we use structural relations in sentence pairs for sentence representations.

we employ phrasal paraphrase relations that introduce the notion of a phrase to the model.


The second task predicts whether a pair of sentences are consecutive in a document to learn the relation between the sentences.

as illustrated in   pre-trained BERT model 1: Obtain a set of phrase alignments A as pairs of spans for each s, t ∈ P 2: WordPiece tokenization of P 3: Accommodate phrase spans in A to BERT's token indexing: Encode b t by the BERT model 7: Compute loss: L(Θ) 8: For phrasal paraphrase task: For sentential paraphrase task: Compute gradient: ∇(Θ) 12: Update the model parameters 13: until convergence token of every input is always the special token of [CLS].


We assume that this method transmits learning signals to the underlying BERT in a more effective manner.

we use mean-pooling to generate representations of source and target phrases in Eq.


The typical neural semantic parser aims to estimate p (y|x).

in our work, we decompose p (y|x) into two processes (as shown in Figure 2), i.e., one process is to learn the anonymous utterancex given the input utterance x for the purpose of reducing lexical problem, i.e., while the other process is to learn the neural semantic parser with anonymous utterancex as the additional input, i.e., in the following, we will discuss how to learn the anonymous utterance (Eqn (2)) and how to learn the neural semantic parser with anonymous utterance as additional input (Eqn (1)) in detail.


We implement a linear gate with encodings of the input utterance and aggregated encodings of column names as input: where W gate stands for learnable parameters, C Q t is the aggregated encoding of column names, and is the concatenation of Q t and C Q t .

to obtain the most relevant column names for the t-th token in the input utterance, we leverage attention mechanism (Bahdanau et al., 2015) towards column names {C 1 , .


For cells, considering that table content is not available due to the privacy problem, we simply predict a substring from the input utterance.

we follow the longest match principle to merge consecutive tokens in the input utterance, which are labeled as CELL by the channel selection stage, to one cell.


Then, maximizing the log likelihood of p(y|x) is equivalent to maximizing its Evidence Lower BOund (ELBO) (Kim et al., 2018), i.e., One popular strategy to maximizing ELBO is the coordinate ascent.

it iteratively executes following two steps (Neal and Hinton, 1998): 1) variational E-step, which maximizes ELBO w.r.t.


Thus, the actual objective is, where J MLE (ϕ) is the log likelihood of the generated SQL query given the input utterance and the anonymous utterance, and J ER (θ) is the expected reward of the anonymous utterance given the input utterance (see Eqn (8) for details.)

when optimizing J MLE (ϕ), we use the concatenation of the encoding of the input utterance (denoted as e(x t )) and the encoding of the anonymous utterance (denoted as h(x t )) as the input for the neural semantic parser, i.e., g t = [e(x t ); h(x t )], t ∈ {1, .


In this work, we propose a learning-based approach to reduce the lexical problem before the neural semantic parser on text-to-SQL generation.

we propose a two-stage anonymization model and leverage implicit supervision from SQL queries by policy gradient to guide its training.


We propose a new approach to modeling these interactions while maintaining efficient inference.

we use Capsule Networks (Sabour et al., 2017): each proposition is encoded as a tuple of capsules, one capsule per argument type (i.e.


Our intuition is that modeling interactions through the output spaces rather than hoping that the encoder somehow learns to capture them, provides a useful inductive bias to the model.

capturing this interaction explicitly should mitigate overfitting.


For our dataset, we take advantage of programming assignments written in Jupyter notebooks to build a high quality dev/test set.

we collect 1,510 code context examples from assignments created with nbgrader (Hamrick, 2016), and mine an additional 2,215 examples from in-class exercise notebooks.


We now discuss the MISP framework ( Figure 1) in more detail.

we highlight the function of each major building block and the relationships among them, and leave the description of a concrete embodiment to Section 4.


Intuitively if the base semantic parser gives a low probability to the top prediction at a step, it is likely uncertain about the prediction.

we say a prediction o t needs user clarification if its probability is lower than a threshold p * , i.e., This strategy is shown to be strong in detecting misclassified and out-of-distribution examples (Hendrycks and Gimpel, 2017).


standard deviation) across different perturbations under dropout, the model is likely uncertain about it.

the uncertainty on prediction o t is calculated as: where W i is the parameters of the base semantic parser under the i-th dropout perturbation, and the uncertainty score is the standard deviation of the prediction probabilities over N random passes.


We tune the threshold p * within 0.5 ∼ 0.95 and s * within 0.01 ∼ 0.2.

for uncertaintybased detection measured by Bayesian dropout, the number of passes N is set to 10, with a dropout rate 0.1.


For any base parser, MISP-SQL improves their performance by interacting with users.

for SQLNet, MISP-SQL outperforms the DialSQL system with only half the number of questions (1.104 vs. 2.4), and has a much simpler design without the need of training an extra model (besides training the base parser, which DialSQL needs to do as well).


In this paper we eliminate such a strong limitation and propose a novel structure-aware selfattention approach to better modeling the relations between indirectly connected concepts in the state-of-the-art seq2seq model, i.e., the Transformer.

a few different methods are explored to learn structural representations between two concepts.


Although such graph-to-sequence models can achieve the state-of-the-art results, they focus on modeling one-hop relations only.

they only model concept pairs connected directly by an edge (Song et al., 2018;Beck et al., 2018), and as a result, ignore explicit structural information of indirectly connected concepts in AMR graphs, e.g.


To this end, we extend the encoder in the state-of-the-art seq2seq model, i.e., the Transformer (Vaswani et al., 2017) and propose structure-aware self-attention encoding approach.

several distinct methods are proposed to learn structure representations for the new self-attention mechanism.


Each output element z i is a weighted sum of a linear transformation of input elements: where in Equation 2 are obtained by the self-attention model, which captures the correspondences between x i and others.

the attention weight α ij of each element x j is computed using a softmax function: where is an alignment function which measures how well the input elements x i and x j match.


1 In order to distinguish the edge direction, we add a direction symbol to each label with ↑ for climbing up along the path, and ↓ for going down.

for the special case of i == j, we use None as the path.


With an L-layer GAT network, features from L hops away can be propagated to the aspect target node.

given a node i with a hidden state h i l at layer l and the node's neighbours n[i] as well as their hidden states, a GAT updates the node's hidden state at layer l + 1 using multi-head attentions (Vaswani et al., 2017).


As we mentioned before, we believe that both explicit and implicit structures are crucial for building a successful model for TSA.

we first present an approach to learn flexible explicit structures based on latent CRF, and next present an approach to efficiently learn the rich implicit structures for exponentially many possible combinations of targets.


We use a log-linear formulation to parameterize our model.

the probability of predicting a possible output y, which is a list of targets and their associated sentiment information, given an input sentence x, is defined as: where s(x, y, h) is a score function defined over the sentence x and the output structure y, together with the latent variable h that provides all the possible combinations of sentiment spans for the (x, y) tuple.


In general, our model EI outperforms all the baselines.

it outperforms the strongest baseline EIsignificantly with p < 0.01 on the English and Spanish datasets in terms of F 1 scores 5 .


In this work, we argue that properly modeling both explicit structures in the output space and the implicit structures in the input space are crucial for building a successful targeted sentiment analysis system.

we propose a new model that captures explicit structures with latent CRF, and uses LSTM and self-attention to capture rich implicit structures in the input space efficiently.


So, apart from the assignment probabilities, we further introduce an interactive attention weight α which is learned interactively between the context and the aspect term.

we implement scaled dot-product attention which can be described as mapping a query and a set of key-value pairs to a weight on the word-level token.


In this section, we investigate the effects of different routing iteration numbers.

we conduct experiments on all the three datasets and vary routing iteration numbers r from 1 to 4.


Accordingly, we propose a Neural Personal Discrimination (NPD) approach to address above challenges by determining personal attributes from posts, and connecting relevant posts with similar attributes to jointly learn their emotions.

we employ adversarial discriminators to determine the personal attributes, with attention mechanisms to aggregate attributes-aware words.


Figure 1 illustrates our proposed neural personal discrimination model for emotion detection.

we first learn the representation of each post using a LSTM model, same as the vanilla joint model.


To generate diverse examples, we propose to use a large lexical knowledge base, WordNet, to add perturbations on training examples by replacing some words with their synonyms (e.g., sad and unhappy), neighbor words (e.g., fox and wolf), or super-superior words (e.g., chair and armchair) 3 , as shown in Figure 1.

the output of the generator is a sequence of actions deciding which words should be replaced and their replacements.


It takes a text sequence as input and an action sequence as output.

we define five actions representing the replacement decision, whether a word   should be replaced or its replacement type, as shown in Table 2.


Then, we feed the real example (x, y) and the generated example (x , y) into the classifier to get the replacement reward r(â).

we define the reward as the absolute difference of the probability of y between the real example and the generated example: where φ is the parameter of the classifier.


Second, the handling of attributes means that Trait avoids overfitting to the more prevalent attributes in a dataset.

trait can learn a more refined conditional probability distribution that incorporates specific attributes than otherwise possible.


While some state of the art DA algorithms focus on improving on a downstream task, such as classification in bilingual or crosslingual framework (Hangya et al., 2018)  , others (Zeng et al., 2018), (Shang et al., 2018) focus on tackling DA at the word level for various downstream applications such as classification, tagging etc.

work by  and (K Sarma et al., 2018) address the issue of context-based domain adaptation within a single language.


Typical downstream applications such as cross lingual and/or multi-domain sentiment classification, using algorithms proposed by (Hangya et al., 2018), , make use of DNNs 1 https://github.com/naacl18sublong/ Friedland with RNN blocks such as BiLSTMs to learn both generic and domain specific representations.

work focused on multi-domain sentiment classification as in (Liu et al., 2016), (Nam and Han, 2016), proposes building neural architectures for each domain of interest, in addition to shared representation layers across all domains.


While both methods have achieved tremendous success in producing word (ELMo) and sentence (BERT) level encodings that perform well in several disparate NLP tasks such as question-answer solving, paraphrasing, POS tagging, sentiment analysis etc, these models are computationally expensive and require large amounts of training data.

when used in a transfer learning setting, both algorithms assume that a large amount of data is present in the source as well as the target domains.


The proposed method domain-invariant feature distillation (DIFD) does not need any sentiment polarity labels in the target domain, which is more consistent with the practical settings.

we focus on the aspect-level cross-domain sentiment classification, and train a shared sentiment classifier and two respective aspect detectors in the source and target domains.


Finally, the two orthogonal features are fed into their corresponding downstream tasks.

we input the domain-invariant feature into the sentiment classifier to predict the sentiment polarity, and input the domain-dependent feature into the aspect detector of the specific domain to identify the aspect terms.


For a specific aspect term, the domain-invariant feature based on the aspect-opinion attention contains more information of its corresponding opinion words, which is beneficial to the final aspectlevel sentiment classification.

we first calculate the position representation of a specific aspect term with its position x a and the sentimentdominant context H c : where x a = {0 1 , ..., 1 i+1 , ..., 1 i+m , ...0 n } represents the word positions of an aspect subsequence in the input sentence x with non-zero values and m is the length of the aspect.


To further accelerate the distillation process of the domaininvariant features, we also introduce an adversarial loss of the domain classifier for the feature extractor.

we calculate the loss in Equation 10 with the flipped domain labels inspired by the work (Shu et al., 2018): We model the AD task as a sequence labeling problem, and each word in the sentence is marked as a tag in {B, I, O}, which means the word is at the beginning (B) or the inside (I) of an aspect term or other word (O).


all the aspect terms of an input sentence in one forward pass.

we first linearly transform the aspect-dominant hidden state h d into a threedimensional vector.


The statistics of these three datasets are shown in Table 1.

we collect the aspect-term level sentences and corresponding labels from these datasets.


Based on the deep transition Gated Recurrent Unit (GRU) Pascanu et al., 2014;Miceli Barone et al., 2017;Meng and Zhang, 2019), an aspect-guided GRU encoder is thus proposed, which utilizes the given aspect to guide the sentence encoding procedure at the very beginning stage.

we specially design an aspect-gate for the deep transition GRU to control the information flow of each token input, with the aim of guiding feature selection and extraction from scratch, i.e.


A-GRU: Aspect-guided GRU (A-GRU) is a specially-designed unit for the ABSA tasks, which is an extension of the L-GRU proposed by Meng and Zhang (2019).

we design an aspect-gate to select aspect-specific representations through controlling the transformation scale of token embeddings at each time step.


We propose an aspect-reconstruction approach to guarantee the aspect-specific information has been fully embedded in the sentence representation.

we devise two objectives for two subtasks in ABSA respectively.


And we investigate the impact of λ for aspect-oriented objectives.


In summary, we formulate the task of DASC as a semi-Markov Decision process (Sutton et al., 1999b), i.e., hierarchical reinforcement learning with a high-level policy and a low-level policy.

our HRL approach for DASC works as follows.


On the other hand, when model training is finished, i.e., both high-level and low-level policy finish all their selections, the goal of sentiment rating predictor is to perform DASC.

we first regard last statev n of LSTM h as the representation of all selected clauses while last state w n,kn of LSTM l as the representation of all selected words.


The parameters in HRL are learned according to Algorithm 1.

these parameters can be divided into two groups: 1) θ h and θ l of high-level policy π h and low-level policy π l respectively.


The other hyperparameters are tuned according to the performance in the development set.

we adopt Adam optimizer (Kingma and Ba, 2014) with an initial learning rate of 0.012 for cross-entropy training and adopt SGD optimizer with a learning rate of 0.008 for all policy gradients training.


The main idea of the proposed approach is to perform sentiment classification like human beings.

our approach employs a high-level policy and a lowlevel policy to perform clause selection and word selection in DASC respectively.


Here we suggest the task of producing a rebuttal in response to a long argumentative text.

we consider spoken speeches around four minutes long.


To assess this, annotators were shown speeches from iDebate18, alongside a matching list of GPclaims determined to be relevant in the previous stage.

claims annotated as supporting a motion were shown for speeches in which the speaker is arguing in favor of that motion, and vice versa.


Finally, asking again about a claim's mention enables result validation, when the answer is known a-priori with high confidence.

claims for which the annotation is unanimous were used for this purpose.


Bert Recently, the Bert architecture (Devlin et al., 2018) has proven successful on similar tasks, and we provide its results as an additional baseline.

we select at random 80% of the motions as an ad-hoc train set, and the remainder as a test set (bert-test).


Prior One important difference between GPclaims and iDebate claims is that the same GPclaim can (and does) appear across different motions and speeches.

given a training set, the a-priori probability that a GP-claim will be mentioned in a speech can be computed.


Recent work has used them to improve the overall performance (Chen et al., 2016;Dong et al., 2017), interpretability (Amplayo et al., 2018a;Angelidis and Lapata, 2018), and personalization (Ficler and Goldberg, 2017) of neural network models in different tasks such as sentiment classification (Tang et al., 2015), review summarization (Yang et al., 2018a), and text generation (Dong et al., 2017).

user and product information have been widely incorporated in sentiment classification models, especially since they are important metadata attributes found in review websites.


Matrix-based A more intuitive way of representing attributes is through the weight matrix W .

given the attribute embeddings u and p, we linearly transform their concatenation into a vector w of size D 1 * D 2 where D 1 and D 2 are the dimensions of W .


To conduct a fair comparison among the different methods described in Section 2, we compare these methods when applied to our base model using the development set of the datasets.

we use a smaller version of our base model (with dimensions set to 64) and incorporate the user and product attributes using nine different approaches: (1) bias-attention: the bias-based method injected to the attention mechanism, (2-5) the matrix-based method injected to four different locations (matrix-embedding, matrix-encoder, matrix-attention, matrix-classifier), and (6-9) the CHIM-based method injected to four different locations (CHIM-embedding, CHIM-encoder, CHIM-attention, CHIM-classifier).


To create the task-specific datasets, we split the dataset again such that no users and no products are seen in at least two different splits.

if user u is found in the train set, then it should not be found in the dev and the test sets.


remains unanswered, since different tasks and settings produce different best models.

cHIM-embedding achieves the best accuracy while cHIM-classifier achieves the best RMSE on sentiment classification.


Finally, we also investigate whether injecting in to more than one location would result to better performance.

we jointly inject in two different locations at once using CHIM, and do this for all possible pairs of locations.


We introduce an attention mechanism to extract such words that are important to the meaning of the clause and aggregate the representation of these informative words to construct the clause vector.

where w m is the parameter for computing attention signals and ⊕ is concatenate operation.


Even though sentiment words are key to clause representation generation, most existing models do not focus on sentiment words or place less emphasis on them when producing clause representation.

attention is distracted by irrelevant words with less sentiment polarity.


The improvement is significant with p-value less than 0.001 in one sample t-test.

rHNN manages to boost the performance by 3.06% in F-measure compared to Lamb-daMArT, which exhibits that by restraining the parameters with knowledge-based regularizations, rHNN is better to identify the emotion cause cues than feature engineering.


Thus, we focused our attention on pairs that are presumably most valuable to train a learning algorithm.

we annotated 14k randomly selected pairs, that satisfy the following criteria: 1.


Following Habernal and Gurevych (2016b), we further examined to what extent our labeled pairs satisfy transitivity.

a triplet of arguments (A, B, C) in which A is preferred over B, and B is preferred over C, is considered transitive if and only if A is also preferred over C. We examined all 892 argument triplets for which all pair-wise combinations were labeled, and found that transitivity holds in 96.2% of the triplets, further strengthening the validity of our data.


To overcome these limitations, we propose a novel task: performing fine-grained analysis of texts by detecting all fragments that contain propaganda techniques as well as their type.

we create a corpus of news articles manually annotated at the fragment level with eighteen propaganda techniques and we propose a suitable evaluation measure.


Ours is the first work that goes at a fine-grained level: identifying specific instances of propaganda techniques used within an article.


Going at this fine-grained level can yield more reliable systems and it also makes it possible to explain to the user why an article was judged as propagandistic by an automatic system.

we designed an annotation schema of 18 propaganda techniques, and we annotated a sizable dataset of documents with instances of these techniques in use.


For each dataset, we report three best systems for the comparisons 2 .

we compare with the following systems: Bag of Feature -Multimodal Sentiment Analysis (BoF-MSA) (Blanchard et al., 2018), Memory Fusion Network (MFN) (Zadeh et al., 2018b), Deep Fusion -Deep Neural Network (DF-DNN) (Nojavanasghari et al., 2016), Multi View -LSTM (MV-LSTM) (Rajagopalan et al., 2016), Early Fusion -LSTM (EF-LSTM) (Zadeh et al., 2018c), Tensor Fusion Network (TFN) , Random Forest (RF) (Breiman, 2001), Support Vector Machine (Zadeh et al., 2016), Multi-Attention Recurrent Network (MARN) (Zadeh et al., 2018a), Dynamic Fusion Graph (DFG) (Zadeh et al., 2018c), Multi Modal Multi Utterance-Bimodal Attention (MMMU-BA) (Ghosal et al., 2018), Bi-directional Contextual LSTM (BC-LSTM) (Poria et al., 2017b) and Multimodal Factorization Model (MFM) (Tsai et al., 2018).


The layer has hence transformed its position nearly equidistant to both the negative and positive clusters.

convolution layer is able to robustly create a transformation which can handle any task specific noise.


In this setting, we consider incorporating a larger context from the discourse, in order to assess the impact of a claim.

we consider up  Table 5: Results for the baselines and the BERT models with and without the context.


The current explosion in digital technology in recent years has led to a vast amount of opinionated materials on the internet.

individuals have expressed opinions on several aspects of products, services, blogs, and comments which are deemed to be influential, especially when making purchase decisions based on product reviews (Schouten and Frasincar, 2015).


These observations motivate us to develop a neural model which can operate on the dependency tree of a sentence, with the aim to make accurate sentiment predictions with respect to specific aspects.

we propose a convolution over a dependency tree (CDT) model which exploits a GCN to model the structure of a sentence through its dependency tree, where node (word) embeddings of the tree are initialized by means of a Bi-directional Long Short Term Memory (Bi-LSTM) network.


Finally, an aggregator is applied over the enhanced aspect embeddings to distill a dense vector embedding for the classification task.

we aim to extract embeddings which encode both contextual and dependency information between a specific aspect expression and opinion words, providing supervisory signals for the aspect-based classification task.


The GCN approach ensures that the sentence structure represented by the dependency tree is encoded efficiently, whereby the representations for nodes encode the local position of opinion words and the target words in the dependency tree.

the dependency tree G for any arbitrary sentence can be represented as an n × n adjacency matrix A, with entries A ij signaling if node i is connected to node j by a single dependency path in G. A ij = 1 if node i is connected to node j, and A ij = 0 otherwise.


The BiLSTM and the GCN can be interpreted as message passing networks.

the BiLSTM allow aspect words of an arbitrary sentence to be contextualize, while the GCN finds the local position of aspect words in the syntactic dependency tree.


To distinguish CDT as the new state-ofthe-art in aspect-based sentiment classification, we compare CDT with several well established models, showing that CDT outperforms the very recent models in the classification task.

we perform case studies with visualizations to verify our approach of aggregating only aspect vectors for the final embedding.


We summarize the statistics of the datasets in Table 1  For fairness in model comparation, we use similar parameters in compared models.

we exploit 300-dimensional Glove vectors (Pennington et al., 2014) for the word embeddings, as well as a 30-dimensional part-of-speech (POS) embeddings, 30-dimensional position embeddings, which is used to identify the relative position of each word with respect to the aspect in the sentence.


These models extract a final embedding by aggregating all learned embeddings using an average pool.

we compare our proposed model with very recent models on the benchmark datasets.


Even with simple architectures, we find that ASP-BiLSTM and ASP-GCN have competitive performance with the recent models on benchmark datasets.

aSP-GCN outperforms the models on the Rest14 dataset.


In our experimentation, we find that as we increase the number of layers the accuracy performance increase to an extent.

aSP-GCN increase in model performance over 6 layers of the GCN.


We perform a subsequent run of ASP-GCN on the same sentence s to extract a final embedding, but in this instance we conceal a specific input word w. We conceal w by mapping it to the zero vector before applying ASP-GCN on s. As a result a final embedding h (s\w) is generated for s. If h s = h (s\w) , the word w has no impact on the representation h s .

h s does not capture w or no information flows from w to h s .


One reason might be: despite the measures are empirically proved to reflect generalization, they are only one view towards generalization respectively.

in recent generalization theory (Novak et al., 2018;Bartlett et al., 2017), the measures are evaluated between models with extremely evident difference in generalization ability (measured by test performance difference), for example, between models trained with random labels and true labels.


The steps of the method can be seen in Figure  2.

we first fine-tune a linear classification layer on top of pretrained cross-lingual representations on source language training data.


Therefore, we design a more implicit loss to help the student refrain from the incoherent translation results by acting towards the teacher in the hidden-state level: where , and φ is a penalty function.

we let where −1 ≤ γ st , γ tr ≤ 1 are two thresholds controlling whether to penalize or not.


Therefore, we use the word alignment information from the ART model to help the training of the NART model.

we minimize KL-divergence between the per-head encoder-to-decoder attention distributions of the teacher and the student to encourage the student to have similar word alignments to the teacher model, i.e.


Therefore, we propose to project the embeddings of target languages to the space of the source language, instead of the opposite direction.

we aim at finding an appropriate linear transformation W, such that Wx i approximates y i .


We took contextualized representations of the English word nature and its Spanish transla- tion naturaleza in different contexts from pretrained English and multilingual BERT respectively and visualize their distributions in Figure 2(a), where we can observe obvious clustering of word senses.

words with sense nature-1 and naturaleza-1 mean the physical world, whereas nature-2 and naturaleza-2 mean inherent features.


We select several languages from each language family, and investigate the effect of the amount of training data on the performances of zero-shot cross-lingual dependency parsing.

we take the SVD-based approach, since it is faster than the GD-based one, and trained different transformation models with different amount of parallel sentences from Europarl dataset on each of the 13 selected languages.


In our model, we add a continuous vector representation of the language identity l (i.e., a language embedding) as an additional input to the neural networks in NDMV.

to predict an ATTACH rule probability P ATTACH (c|h, dir, val, l), we use a multilayer neural network that takes the embeddings of the head token h, valence val and language identity l as input, uses a weight matrix W dir specific to the direction dir in the first layer, and uses a weight matrix W c consisting of all the child POS tag vectors in the softmax output layer.


Note that our approach performs worse than the monolingual baseline on some languages, and we speculate that it is partly caused by data imbalance.

the worst-performing Hindi language has only 4997 training sentences, much smaller than the average 7532.


In order to compare the coverage of incidents of hate with coverage of homicides and kidnappings, we contrast the overlap of the extracted incidents with those reported by the FBI.

for 159 cities that have representation of the three crimes both in Patch and FBI crime reports, we calculate the ratio of Patch-based predictions to FBI reports for each crime.


The contributions of this paper are two-fold: First, we have shown that event detection can be applied to the study of hate crimes.

we demonstrated that using MIL for event detection, in conjunction with local news articles, can provide conservative estimates of the occurrence of hate crimes in cities with no official representation.


They may also be of high-order (i.e., the paths more than one hop in the dependency tree).

according to our statistics on the benchmark ACE-2005 dataset, about 51% (4977/9793) of event related entities need more than one hop to get to the corresponding trigger words and related entities in their dependency trees.


In this paper, we propose a Hierarchical Modular Event Argument Extraction (HMEAE) model, to provide effective inductive bias from the concept hierarchy of event argument roles.

we design a neural module network for each basic unit of the concept hierarchy, and then hierarchically compose relevant unit modules with logical operations into a role-oriented modular network to classify a specific argument role.


Event Extraction involves predicting named entities, event triggers, event arguments, and argument roles.

each token d i is predicted as an event trigger by assigning it a label t i .


Most of the previous RST parsers have been based on supervised learning approaches.

they require an annotated corpus of sufficient size and quality, and heavily rely on the language and domain dependent corpus.


Comparing our methods with the baseline, right-branching (RB), we could find larger differences without considering the granularity levels of a document, whereas the differences became smaller by considering three granularity levels.

right-branching Method Span F 1 score .808 HHN16 .826 WLW17 .856 Table 3: Micro Span F scores on the test set of RST-DT.


BERT-large improves over BERT-base in a variety of ways including pronoun resolution and lexical matching (e.g., race track and track).

the BERT-large variant is better at distinguishing related, but distinct, entities.


We view discourse relations as additional labels to predict at the same time we predict next words in language modeling.

we propose to jointly train a model that predicts a sequence of words and a sequence of RST labels by taking advantage of shared representations, following previous sequence labeling problems such as named entity recognition (Collobert et al., 2011) and partof-speech tagging (Huang et al., 2015).


In this model, we aim to utilize latent representations to characterize the flow between sentences.

we define delta, subtractions of hidden represenations of adjacent sentences as such latent information.


In this work, we employ Representational Similarity Analysis (RSA) as a simple method of interpreting neural models' representational spaces as they relate to other models and modalities.

we conduct an experiment wherein we investigate the correspondence between human processing difficulty (as reflected by gaze fixation measurements) and the representations induced by popular pretrained language models.


human readers (Raney et al., 2014;Ashby et al., 2005) and to be dependent upon a number of linguistic factors (Van Gompel, 2007).

it has been demonstrated that word frequency, syntactic complexity, and lexical ambiguity play a strong part in determining which sentences are difficult for humans to process (Rayner and Duffy, 1986;Duffy et al., 1988;Levy, 2008).


2013proposed learning word embeddings using a predictive neural-network approach.

the skip-gram implementation with negative sampling mines word cooccurrences within a text window, and the network must learn to predict the surrounding context from the target word.


reveals a critical limitation of the state-of-the-art models in maintaining consistency.

we motivate our definition of consistency based on classical deductive logic (Tarski and Tarski, 1994) that defines a consistent theory as one that does not entail a contradiction.


We train a question generation model that given representations of the image and the source QA pair, generates a new question.

our question generator concatenates the deep features of an image (extracted using a ResNet152 (He et al., 2016) network) and a QA pair (extracted using a 1-layer LSTM (Hochreiter and Schmidhuber, 1997)) to represent a visual fact.


In this paper, we introduce GeoSQA-an SQA dataset in the geography domain consisting of 1,981 scenarios and 4,110 multiple-choice questions at high school level.

each diagram has been manually annotated with a highquality natural language description of its content, as illustrated in Figure 1.


For this reason, we propose to systematically ask all question types for each generated story.

for a single story involving agents A, B, and object O, we ask all following questions: Reality: Where is O?


Experiments on four standard benchmarks showed that our multi-passage BERT outperforms all state-of-the-art models on all benchmarks.

on the OpenSQuAD dataset, our model gains 21.4% EM and 21.5% F 1 over all non-BERT models, and 5.8% EM and 6.5% F 1 over BERT-based models.


From Table 2, we can see that our multi-passage BERT model outperforms all state-of-the-art models across all benchmarks, and it works consistently better than our BERT-RC model which has the same settings except the global normalization.

on the OpenSQuAD dataset, our model improves by 21.4% EM and 21.5% F 1 over all non-BERT models, and 5.8% EM and 6.5% F 1 over BERT-based models.


2 The Proposed Dataset Generally, the reading comprehension task can be described as a triple P, Q, A , where P represents Passage, Q represents Question and the A represents Answer.

for spanextraction reading comprehension task, the question is annotated by the human, which is much more natural than the cloze-style MRC datasets (Hill et al., 2015;Cui et al., 2016).


(2019), we adopt BERT for our baseline system.

we slightly modify the run squad.py script 5 for adjusting our dataset, while keeping the most of the original implementation.


We combine their complementary strength, by a hybrid approach allowing multi-granular interactions, but represented at word level, enabling an easy integration with strong word-level signals.

we propose MICRON: Multigranular Interaction for Contextualizing RepresentatiON, a novel approach which derives contextualized uni-gram representation from n-grams.


applying dot product between γ and τ, we can contrast matching scores by word importance.

a widely adopted example of τ is IDF, computed either globally (treating all passages in the dataset as a corpus) or locally (treating only candidate passages of given question as a corpus) (Blair-Goldensohn et al., 2003).


In this paper, we study non-factoid question answering.

our approach is inspired by the complementary strength of representationand interaction-focused approaches.


We align question tokens with column names and cell text using the Levenshtein edit distance between n-grams in the question and the table text, similar to previous work (Shaw et al., 2019).

we score every question n-gram with the normalized edit distance 2 and connect the cell to the token span if the score is > 0.5.


In this work, we investigate the expressive power of siamese architectures (Bromley et al., 1994) coupled with hard negatives (i.e., difficult negative samples for the model being learnt), against more sophisticated models.

all our models ignore any interaction between question and answer representations, and use a shared encoder to independently encode questions and answers.


We note that the performance gains demonstrated in this paper are obtained with a restricted class of models.

we restrict to siamese networks, and don't model any interaction between question and answer representations.


Motivated by the performance of bag-of-words like sentence representations (Arora et al., 2016), we also experiment with distributional bag-ofwords models where sentence embeddings are obtained by pooling across the dimensions of corresponding word embeddings.

we used Max-Pooling and Max-Min-Pooling, the latter being obtained by concatenating the outputs of Max and Min Pooling.


To make sure the languages have never been seen before, we artificially make unseen languages by permuting the whole vocabulary of existing languages.

all the words in the sentences of a specific language are replaced by other words in the same language to form the sentences in the created unseen language.


QUARTZ aims to fill this gap by providing the first open-domain dataset of qualitative relationship questions, along with the requisite qualitative knowledge and a rich set of annotations.

quaRTz removes the requirement, present in all previous qualitative reasoning work, that a fixed set of qualitative relationships be formally pre-specified.


Other recent work has shown that even state-of-the-art neural models struggle with numerical operations and quantitative reasoning when trained in an end-to-end manner (Saxton et al., 2019;Ravichander et al., 2019).

even BERT (Devlin et al., 2019) is not very good at doing simple calculations.


While we could recursively search for compositions with deep derivations, here we are guided by what is required in the DROP data and simplify inference by heavily restricting multi-step composition.

spans can be composed into a pair of merged spans (Merge), and the sum of two numbers (Sum) can subsequently be summed with a third (Sum3).


The ablations indicate that the improvements due to the addition of extra ops (Diff100, Sum3, Merge) are roughly consistent with their proportion in the data.

the Diff100 and Sum3 derivations increase the oracle performance by 8.7% and 1.4% respectively, corresponding to model improvements of roughly 9% and 1.1%, respectively.


Very recently, neural networks have been used for QG.

the encoder-decoder seq2seq model (Du et al., 2017;Song et al., 2018) is used to encode a passage and generate a question corresponding to the answer.


However, one issue with the current neural models is that the generated questions are not quite relevant to the corresponding passages and target answers.

the neural QG models tend to generate generic questions (e.g., "what is the name of...?").


Second, we propose a QA-based reranker to rerank QG results.

we use a neural QA model to evaluate the quality of generated questions, and rerank them according to the QA model scores.


It cannot copy a part of a word to reproduce an appropriate form of the word in the generated question.

it cannot copy words with morphological changes.


Although various methods have been developed (Zhao et al., 2009;Quirk et al., 2004;Barzilay and Lee, 2003), the recent progress on paraphrase generation is mainly from neural network modeling (Prakash et al., 2016).

the encoder-decoder framework is widely adopted (Cho et al., 2015), where the encoder takes source sentences as inputs and the decoder generates the corresponding paraphrase for each input sentence.


Algorithm 1 Online learning 1: Input: A training example (x (i) , y (i) ), current schedule rates The REINFORCE Algorithm.

when ↵ = 0, = 0, and Decode(⇡(y | h t )) is defined as as: when ↵ = = 0, bothỹ t 1 andỹ t will choose the sampled values from the Decode function with policy ⇡ ✓ .


The discriminator's objective will train it to distinguish between a sample x from training data X , and a generated samplex, in a binary classification setup.

we define the discriminator loss for x,x as follows: 2.3 Learning Generator parameters θ and discriminator parameters φ are trained together under following objective: Note, in addition to using a traditional adversarial objective, we also include a likelihood term to help stabilize the generator.


We also compare with RHYME-GAN-NS which uses a simpler non-structured discriminator.

it uses a discriminator which first runs a characterlevel encoder for each ending word -similar to RHYME-GAN -but then instead of computing pairwise similarity matrix, it utilizes a LSTM on the sequence of the computed representations.


Then, during RL stage, the conventional way is to adopt self-critical strategy to fine-tune based on the target evaluation metric, Distributional Semantic Reward During evaluating the quality of the generated sentences, ROUGE looks for exact matches between references and generations, which naturally overlooks the expression diversity of the natural language.

it fails to capture the semantic relation between similar words.


Both are state-of-the-art on NLI and commonsense tasks.

we test OpenAI GPT (Radford et al., 2018), and BERT (Devlin et al., 2018).


To analyze if the embeddings are senseinformative, our work focuses on learning a mapping between the semantic space of contextualized word embeddings and the space of definition embeddings.

a better mapping indicates richer sense-specific cues in the contextualized word embedding.


The goal of this paper is to analyze whether we can distill sense-specific information from the pretrained contextualized word embeddings such as ELMo and BERT for better interpretation.

given the embedding of a (word, context) pair, our model learns a non-linear mapping network f : X → Y to project it into the desired  Pre-Trained BERT Figure 1: Illustration of the proposed model using contextualized embeddings from BERT as the context-dependent component.


Moreover, unlike the prior work that required discrete token generation to interpret the pretrained word embeddings (Noraset et al., 2017;Gadetsky et al., 2018;, we reformulate the definition modeling task from an NLG problem to a classification problem via learning a mapping between two semantically continuous spaces, which greatly simplifies the hardness, making significant improvement as shown in Table 2.

as the input representations of Noraset et al.


Our first approach relies on the usage of textbooks from specific domains of short answer grading.

we collect textbooks corresponding to the domains and chunk them into paragraphs and feed each paragraph as a document for pretraining.


In this paper, we explore how to incorporate structured domain knowledge, available in the form of a knowledge graph (UMLS), for the Medical NLI task.

we experiment with fusing embeddings obtained from knowledge graph with the state-of-theart approaches for NLI task, which mainly rely on contextual word embeddings.


We deviate from the standard decoder by removing the selfattention mask that prevents left-to-right decoders from attending on future tokens.

our decoder is bi-directional, in the sense that it can use both left and right contexts to predict each token.


Our training scheme builds on their work, with the following differences: we use separate model parameters for source and target texts (encoder and decoder), and we also use a different masking scheme.

we mask a varying percentage of tokens, only from the target, and do not replace input tokens with noise.


In this work, we utilize word association test, which contains rich types of word connections annotated by human participants, to explore how gender stereotypes spread within our minds.

we use random walk on word association graph to derive bias scores for a large amount of words.


We build a word association graph G = (V, E) whose vertices V are cues and responses, edges E encode associations in the test, and weights of edges represent the strength of associations.


he / she brother / sister father / mother man / woman son / daughter boy / girl husband / wife uncle / aunt grandfather / grandmother gentleman / lady L. In order to balance influence from different genders and avoid their heterogeneity, we collect them pair-by-pair.

, where w k m (a masculine word) and w k f (a feminine word) are semantically related but different in gender.


To test the robustness of the proposed scores, we are interested in how L influences the results.

we apply a bootstrap sampling strategy on the entire L. we sample 8 gender word pairs each time from L. For all 45 possible combinations of the 8 gender words, we compute their bias scores to get confidence intervals w.r.t.


For training the neural network, the crossentropy loss is back propagated.

the gradient of the negative cross-entropy loss is considered to update the model parameters.


Reference: you have to sacrifice your first born child for swimming lessons in nyc.

sarcasmBot: a very useful piece of information!


To alleviate under-translation, we propose phrase segmentation-based padding.

we first segment each line in a classical poem into several sub-sequences, we then join these subsequences with the special padding tokens <p>.


Among transformer-based models, both +Anti OT and +Anti UT outperforms the naive transformer, while Anti OT&UT shows the best performance, this demonstrates alleviating undertranslation and over-translation both helps generate better poems.


We expect the generated poems to not only keep the semantic of the original text, but also demonstrate terseness, rhythm and other characteristics of ancient poems.

we chose 20 famous fragments from four types of Chinese literature (5 fragments for each of modern prose, modern poems, pop song lyrics and Song lyrics).


This paper proposes a novel framework to jointly model Chinese variational, semantic, and contextualized representations for Chinese text spam detection task.

a Variation Family-enhanced Graph Embedding (VFGE) algorithm is designed based on a Chinese character variation graph.


baselines on constructed Chinese character variation graph to get graph based character embeddings.

metapath2vec++ required a human-defined metapath scheme to guide the random walks.


In order to capture inter-dependencies between types, we propose a hybrid model that incorporates latent type representation in addition to binary relevance.

the model learns to predict a low-dimensional vector that encodes latent type features obtained through Principle Label Space Transformation (Tai and Lin, 2012) and reconstruct the sparse and high-dimensional type vector from this latent representation.


We further propose a two-step attention mechanism to actively extract the most relevant information from the sentence.

we calculate the attention for context words in a mention-aware manner, allowing the model to focus on different parts of the sentence for different mentions.


Formally, our model aims at maximizing the conditional probability p(y|x), which is decomposed as: where s = {b k i:j } denotes the set of segments that leads to y through a specific combination.

4 we divide the problem of extracting discontiguous entities into two subtasks, namely segment extraction and segment merging.


To make our model capable of extracting such overlapping segment combinations, we employ the model of neural segmental hypergraphs from , which uses a hypergraph-based representation to encode all the possible combinations of segments without ambiguity.

the segmental hypergraphs adopt a log-linear approach to model the conditional probability of each segment combination for a given sentence: where f (x, s) is the score function for any pair of input sentence x and output segment combination s. In segmental hypergraphs, each segment combination s corresponds to a hyperpath.


To this end, this paper proposes Gazetteer-Enhanced Attentive Neural Networks (GEANN), whose architecture is shown in Figure 1.

to better decouple name and context knowledge for incorporating gazetteer information, we first design a new region-based attentive neural network (ANN), which introduces attention mechanism (Bahdanau et al., 2014;Vaswani et al., 2017;Su et al., 2018) to explicitly model the association between mentions and contexts.


The resulting corpus contains 6,072 VA candidates, 3,023 of them were classified to be true VAs by the human annotators.

the baseline approach has a precision of 49.8% (see Table 1).


2 If these expressions are dealt with differently, the statistics of the same compound can be dispersed, especially for low frequency compounds.

the more distinct these representations are, the more difficult identifying chemical compound entities becomes.


Note that P i, j is the score of the j-th tag for the i-th word.

the probability of the output tag sequence where Y w is the set of all possible tag sequences, A is a matrix of transition scores, A i, j represents a score that transits from the i-th tag to the j-th tag, and y −1 and y N+1 are the special tag for the start of the sentence and the end of sentence, respectively.


Both the training and test sets of the original FewRel dataset are constructed by manually annotating the distantly supervised (Bunescu and Mooney, 2007;Mintz et al., 2009) results on Wikipedia corpus and Wikidata (Vrandečić and Krötzsch, 2014) knowledge bases.

they are from the same domain, yet in a real-world scenario, we might train models on one domain and perform few-shot learning on a different one.


Instead of computing coupling coefficients between primary capsules and category capsules during routing process, we design a capsule-guided routing mechanism, which leverages prior knowledge about the sentiment categories to guide the routing process effectively and efficiently.

we use a set of sentiment capsules to store prior knowledge about the sentiment categories.


This proposition shows that if we know the noise transition matrix, we can use it to recover the clean labels.

we can put noise transition matrix on clean labels to deal with noisy labels.


Then we use the scores predicted from A-network to select some possibly clean sentences from this batch and train A-network on the selected sentences.

speaking, we use the predicted scores to compute sentiment labels by arg max i {ÿ = i|ÿ ∼ P (ÿ|x, θ)}.


We used all sentences with only one polarity (positive or negative) for their aspects.

we only used sentences with aspects having the same sentiment label in each sentence.


Motivated by recent advances in multi-task learning (Liu et al., , 2017Balikas et al., 2017;Yu et al., 2018;Cohan et al., 2019) and the effectiveness of sentiment information in stance detection (Sobhani et al., 2016;Sun et al., 2018Sun et al., , 2019, we propose an attention-based multi-task framework that takes sentiment classification as an auxiliary task for stance detection.

we first encode words of input sentences in fixed-length dense vectors (Mikolov et al., 2013;Bojanowski et al., 2017) and then feed them as input to Bidirectional Long Short-Term Memory networks (BiLSTM) (Hochreiter and Schmidhuber, 1997;Schuster and Paliwal, 1997).


We use the sentiment lexicon 1 by Hu and Liu (2004) and construct a stance lexicon 2 (of almost 2,000 words) with words related to the five targets in the SemEval-2016 dataset.

we construct a stance lexicon for each target from the training data available from the Se-mEval 2016 dataset and from an extra 1,000 tweets for each target that we collected from Twitter using specific hashtags.


The intuition of using sentiment and stance lexicons is to provide guidance to the attention layer.

given an input sentence, we mark sentiment and stance lexicon words as 1 and mark as 0 the remaining words that are not present in any of the two lexicons in order to obtain a lexicon vector lex.


We can observe that AT-JSS-Lex model outperforms all baseline models except on "Atheism."

the proposed model outperforms JOINT model by 5.17% and 3.11% in M acF avg and M icF avg , demonstrating the effectiveness of attention mechanism and lexicon information.


Our network configurations are identical to those of .

we use 512 hidden states and 128-dimensional word vectors.


The central idea of this paper is to use informative priors to restrict the degree to which different words can inhabit different dimensions, such that one or more dimensions become interpretable and connected to one's research interest.

we place informative priors on word types that we expect to discriminate on a particular dimension, e.g.


To measure the extent to which the inferred dimension reflects the semantic concept of interest, we consider how well-placed a number of prespecified hold-out word types (not a part of V ± ) are in this particular dimension.

accuracy is computed as the fraction of hold-out words that are placed on the correct side of 0 on the dimension of interest in the embedding and context vectors.


We further show that a diagnostic classifier trained on the biased baseline neural network also does not generalize to new samples.

the biases detected in Elazar and Goldberg (2018) seem restricted to their particular data sample, and would therefore not bias the decisions of the model on new samples, whether in-domain or out-of-domain.


To address the above problems, we propose the Deep Gating Network (DGN) for gradually filtering and aggregating charge-specified information at different levels of granularity.

we stack multiple blocks of an LSTM layer and a charge-specific gating layer for generating a focused charge-based representation of the case description.


Each block transforms its input semantic vectors into more sophisticated and focused representations based on gated feature selection and combination.

each gating block consists of a bi-LSTM layer for context aggregation and a gating layer for charge-specific feature filtering.


The adaption from classification to regression can be easily accomplished by replacing the original final layer with that of Eq 1.

we adapted the following models:  • ATAE-LSTM (Wang et al., 2016): it concatenates aspect embedding and the output of LSTM, and uses self-attention to obtain aspect-based representation.


To better aid the epigraphist's task, PYTHIA returns multiple predictions as well as the level of confidence for each result, rather than a single prediction per text restoration.

we provide a set of the Top 20 predictions decoded using beam search.


As can be seen in Figure 3, the correlation between the "context length" and the predictive performance of our model is positive.

the performance peaks around 500 characters of context length.


We propose a novel tensor embedding method that can effectively extract lexical features for humor recognition.

we use wordword co-occurrence to encode the contextual content of documents, and then decompose the tensor to get corresponding vector representations.


We show that this simple method can capture features of lexical humor effectively for continuous humor recognition.

we achieve a distance of 0.887 on a global humor ranking task, comparable to the top performing systems from SemEval 2017 Task 6B (Potash et al., 2017) but without the need for any external training corpus.


For each sentence s in D, we count the wordword co-occurrence in a small window H, and build a frequency matrix W s ∈ Z V ×V , where Z denotes the set of integers.

w s (i, j) indicates the frequency that word w i and w j cooccur in s within the window H. In this way, we can capture the lexical patterns of s in w s .


The objective of tensor decomposition is to find an approximationŴ of W so    where is the predefined rank parameter, and ⊗ is the outer product, namely, v r ⊗ v r ⊗ d r being a three-dimensional tensor, and With the tensor decomposition, we can find low-rank embeddings of sentences that capture the similarity of contextual patterns (Hosseinimotlagh and Papalexakis, 2018).

c = where the s-row of c is the embedding vector of sentence s. The Euclidean distance of embeddings is used to measure the similarity of two sentences.


Given the decomposed C T , we compute a centroid as the average m of all sentence vectors of a corpus: The Euclidean distance to the center is then taken as an indicator of the degree of humor.

given two sentences s 1 and s 2 and their embeddings x 1 and With the lexical similarity captured by tensor embeddings, we can build a similarity graph, and use a label propagation algorithm (Zhou et al., 2003) for semi-supervised humor recognition.


In this way, we can use only a small portion of labeled data to predict the remaining unlabeled data effectively (Zhou et al., 2003).

with the tensor embeddings, we first find the K nearest neighbors of each data point, and build a similarity graph G. We then form an affinity matrix W , where W ij = 1 if i and j are connected, otherwise, W ij = 0.


News recommendation can help users find interested news and alleviate information overload.

modeling news and users is critical for news recommendation, and capturing the contexts of words and news is important to learn news and user representations.


We use the 300dimensional bilingual word embeddings pretrained by MUSE (Conneau et al., 2018) and 300dimensional randomly initialized character embeddings.

we first collect monolingual pre-trained word embeddings from fast-Text (Bojanowski et al., 2017).


We follow the general translation step in Section 2.

we replace English words with target language words if the translation pairs exist in the bilingual dictionaries (Rolston and Kirchhoff, 2016).


While we can observe how models perform overall on the named entities, we can not know how models differ from short to long entities.

probing if models hold strong biases towards shorter entities can help interpret the process of cross-lingual transfer learning.


We propose a novel annotation scheme to give scenarios of how humor arises in text.

our annotations of linguistic humor not only contain the degree of funniness, like previous work, but they also contain key words that trigger humor as well as character relationship, scene, and humor categories.


Humor plays important role in human communication, which not only serves to exchange ideas or convey messages, but also involves emotion regulation such as provoking laughter, generating amusement, and reducing stress (Wooten, 1996;Morse, 2007).

with the rapid growth of social media applications such as Facebook and Twitter, a significantly increasing number of individuals are using these social media public platforms to release humorous texts.


(2018b) pointed out some problems in the NL-RX datasets.

there are some ambiguities since Locascio et al.


(The student may even succeed on some pairs that it has never been shown, thanks to n-gram clues.)

we measure the "goodness" of the updated L2 word embedding matrix F. For each pair p = (e,f ) 2 P, sort all the words in the entire L1 vocabulary according to their cosine similarity to the L2 word f , and let r p denote the rank of e. For example, if the student had managed to learn a matrix F whose embedding of f exactly equalled E's embedding of e, then r p would be 1.


In order to dynamically point to more related messages in the thread history, we use an one-way attention, which has been successfully adopted in many NLP tasks (Tan et al., 2016;Hermann et al., 2015).

given the sequence outputs of each thread's LSTM, {h l (l,k) } K k=1 , we perform a weighted mean pooling to get a context embedding e l cxt , attended by the one-step LSTMencoded m i , denoted asĥ i .


CATD models are significantly superior to all baselines and CATD-COMBINE generally performs best.

all baselines failed on Politics, probably because there are more threads in Politics than the other two datasets (see Appendix B), making disentanglement more difficult.


If the sentiment classifying knowledge about how to comment on these concepts can be acquired, it will be helpful for sentiment classification when meeting these concepts in free texts again.

the task of sentiment classification of entire text has changed into recognizing the semantic orientation of the context of all Concerned Concepts.


In contrast, S i neg is acquired from the training texts with negative view.

s i pos and s i neg respectively reserve the features for positive and negative classification to CC i in corpus.


We evaluate these methods in Japanese-to-English retrieval using the NTCIR-3 topics and data sets.

we explore the following questions: • Can translations obtained from context vectors help CLIR performance?