Given the target domain, we first train LAED model(s) on the MetaLWOz data -here we exclude from training every domain that might overlap with the target one.

for the Navigation domain in SMD, it's Store Details, for Weather it's Weather Check, and for Schedule it's Update Calendar and Appointment Reminder.


By transferring latent dialogue knowledge from multiple sources of varying generality, we obtained a model with superior generalization to an underrepresented domain.

we showed that our few-shot approach achieves state-of-the art results on the Stanford Multi-Domain dataset while being more dataefficient than the previous best model, by requiring significantly less data none of which has to be annotated.


In ensembling, multiple models are trained and their predictions are averaged during inference.

if α l denotes the output of model l ∈ [1, L], the output probability is defined as: Since ensembling reduces the model variance while maintaining low bias, it is most effective when the models are diverse and each model excels at a particular type of input.


Then, based on the KG, we present structured dialogue management (Section 3) to explore the optimal dialogue strategy with reinforcement learning.

our dialogue management consists of (1) the KG-based dialogue state tracker (KG-DST) that treats embeddings of nodes in the KG as dialogue states and (2) the hierarchical dialogue policy (HDP) where high-level and low-level agents unfold the dialogue together.


The green edge represents the triplet crawled from the existing KG and the blue edges represent the triplets generated based on a navigation electronic map.

for each personal information entity, we first crawl its points of interest (POI 6 ) within one kilometer and the POI types in the Amap.


The dialogue features record the dynamic information of v ∈ V p ∪V a during the dialogue.

dialogue features include whether the node has been explored by the manager or workers and whether the node appeared in the system action of the last turn.


Thus, we want the known information of "Gulou Subway Station" to be sent to its successors.

for v ∈ V p ∪ V a , we compute its embedding recursively as follows: where E k t (v) is the depth-k node embedding in the t-th turn, N (v) denotes the set of nodes adjacent to v, W k is the parameter in the k-th iteration and the aggregate function is the element-wise max operation.


After obtaining the dialogue states and node embeddings, our system will unfold the dialogue according to a hierarchical policy.

the manager first selects v p ∈ V p as a worker to verify the identity state of v p according to a high-level policy π m .


Besides, to prevent the two agents from making decisions in haste, domain rules are applied to their dialogue policies by "Action Mask" (Williams et al., 2017).


Besides, we provide an internal reward to optimize the low-level policy.

if the worker gives a correct decision about the corresponding personal information, it will receive a positive reward r w crt .


To generate simulated loan applicants, we first estimate the function relations between p(k i = 1) and Freq(e h i , e t i ) via human experiments.

we ask 31 volunteers to answer derived questions 10 about their own and others' personal information.


Then, we get simulated loan applicants 11 following a "sampling and calibration" manner.

given an applicant's personal information, we first sample the identity state randomly.


As shown in Section 4.3, we use this system to pre-train Full-S. • MP-S: A neural dialogue system which uses message passing to infer dialogue states but uses a flat policy to unfold dialogues.

the manager selects answer nodes directly to generate derived questions.


To test whether the manager follows the two rules, we calculate the probabilities of RS1 and RS2 under Cond1 and Cond2 respectively.

in the test data, p(RS1|Cond1) = 0.95 and p(RS2|Cond2) = 0.96.


Meanwhile, we study cases where the manager does not follow the two rules and find some interesting phenomena.

if only one worker's decision is "Fraud" and the applicant can answer a few questions given by this worker, the manager's decision may be "Non-Fraud".


This is because the message passing mechanism models the relation between "HaoDe" and "Xiao Liu Fruit".

because the two entities are closely related to each other, if applicants know "Xiao Liu Fruit", they may well know "HaoDe".


Unlike past work that has focused on diversifying the output at word-level or discourse-level with a flat model to alleviate this problem, we propose a hierarchical generation model to capture the different levels of diversity using the conditional variational autoencoders.

a hierarchical response generation (HRG) framework is proposed to capture the conversation intention in a natural and coherent way.


Intuitively, another latent level can be introduced to generate different expressions from the same dialog act, and a hierarchical structure can be used to model the response generation process.

the knowledge base is first constructed over the pairs of expressions and dialog acts to capture the latent correlation between them.


To tackle the challenges, we propose to take advantage of the hierarchical nature of response generation.

we investigate: (1) how to automatically learn a hierarchical model to naturally capture the response generation process; (2) how to adaptively learn and adjust the influence ratio between expression and content.


Our solutions to these questions result in a new architecture for neural response generation.

a novel hierarchical response generation (HRG) framework is proposed to effectively capture the process of response generation.


1This objective can be transformed as the variational lower bound of CVAE (Doersch, 2016), and thus can be optimized efficiently.

the variational approximation q φ (e|y) is constructed to approximate the intractable posterior p θ (e|a).


Therefore, the text convolutional network proposed in (Kim, 2014) is leveraged to mine the relationship between expression patterns and responses.

the network q φ (e|y) consists of several convolutional filtering, local contrast normalization, and max-pooling layers, followed by several connected linear layers.


A balance between content and expression is effectively kept by feeding [w, α • e] into decoder.

before decoding, an expression state is initialized as q 0 = e to record the current expression representation.


To overcome the latent variable vanishing problem in CVAE, we use the heuristic method (Bowman et al., 2016) to encode the meaningful information in z.

we multiply the first KL term in Eq.12 by a scalar, which starts at 0 and linearly increases to 1 over the first 10,000 batches.


In this paper, we investigate the problem of generating meaningful responses by imitating the hierarchical process of human response.

a hierarchical response generation model is proposed to hierarchically generate the expressions and fill them with appropriate content naturally and coherently.


For each conversation, the model generates parameters of the encoder-decoder by referring to the input context.

we propose two adaptive parameterization mechanisms: a context-aware and a topic-aware parameterization mechanism.


To gather the benefits of both approaches, in this paper, we propose an adaptive neural dialogue generation model which utilizes a single encoderdecoder for diverse conversations, meanwhile, the encoder-decoder is specifically parameterized according to each conversation.

we propose two adaptive parameterization mechanisms: 1) context-aware parameterization directly generates parameters of the encoder-decoder model by capturing local semantics of the given context; 2) topic-aware parameterization enables parameter sharing among conversations with similar topic distributions by first inferring the latent topics of the input context, and then generating the parameters with respect to the inferred latent topics.


To benefit the model parameterization with both the local and global information, we further adapt parameters of the encoder-decoder by utilizing both the context representations ξ t and the topic distribution θ.

the LSTM weight W at time step t is adapted as follows: where Ψ t is the gating function deciding whether the parameterization relies more on the context or the topics.


We name it KBRD, standing for Knowledge-Based Recommender Dialog System.

the dialog generation system provides contextual information about items to the recommender system.


In order to perform end-to-end training, we demonstrate the combination of the recommender system and conversation system.

the input of the recommender system is constructed based on the dialog history, which is a representation of mentioned items in the dialog.


The knowledge can build a connection between dialog contents and items.

incorporating Dialog Contents we have a knowledge graph G consisting of triples (h, r, t) where h, t ∈ E and r ∈ R. E and R denote the sets of entities and relations in the knowledge graph.


Instead of simply averaging these vectors, we choose a linear combination of the |T u | vectors.

we apply self-attention mechanism (Lin et al., 2017) that takes H u as input and outputs a distribution α u over |T u | vectors: where W a1 ∈ R da×d is a weight matrix and w a2 is a vector of parameters with size d a .


By further introducing the recommender system's knowledge of the items that have appeared in dialog, we guide the dialog system to generate responses that are more consistent with the user's interests.

we add a vocabulary bias b u to the top layer of the decoder inspired by Michel and Neubig (2018).


Distinct n-gram is a measurement for the diversity of natural language.

we use distinct 3-gram and 4-gram at the sentence level to evaluate the diversity.


The evaluation for recommendation is Recall@K. We evaluate that whether the top-k items selected by the recommender system contain the ground truth recommendation provided by human recommenders.

we use Recall@1, Recall@10, and Recall@50 for the evaluation.


This represents the efficiency of the recommender system, which can save users' time and efforts.

we demonstrate the performances of four systems in Figure 2.


Retrieval Even though this issue can be analyzed in more detail, we shall content ourselves with some simple statements.

rand it involves the combination equation human To the cloud !


Retrieval Is there really any difference in the low energy physics?


However, most existing multi-turn dialogue datasets only provide sets of utterances, without any information about relations between utterances.

they lack the necessary supervisions to restore the incomplete utterance.


A small commonly used word list is provided, and annotators could use those words to ensure the fluency of rewritten sentences.

all words in the restored utterance come from previous and original utterances or the extra word list.


Such unbalanced probability makes models mentioned above tend to simply regenerate the original utterance to maximize the conditional probability of the generated sentence during the beam-search process.

the Seq2Seq model and pointer generative network tend to regenerate the original utterance and cannot effectively restore the original utterance.


Pick: inspired by recent advances in transfer learning for language representation, we fine-tune the pre-trained deep bidirectional transformers for language understanding model (BERT) (Devlin et al., 2019) as a classifier to select omitted words from previous utterances.

instead of discrete classifications, we first formulate this word selection problem as a sequence tagging problem.


This metric focuses on n-grams that contain at least one restored word, excluding other n-grams.

the n-gram restoration precision, recall, and F-score can be calculated as:  Table 3: p, r, f here represent the restoration precision, recall and F-score that we propose in Section 3.3.


We compare the performance of the following methods on the collected dataset: • Syntactic: model proposed by Kumar (Kumar and Joshi, 2016).

they replace each out-of-vocabulary word in the corpus with a numbered unknown token.


Then human evaluation is further conducted.

human annotators are instructed to evaluate the quality and fluency of restored utterances.


Since it is hard to evaluate the retrieval-stage, we evaluate the end-to-end response selection performance.

we first rewrite the contexts in the test set with CRN, and then retrieve 10 candidates with the rewritten context from the index 6 .


Given a dialogue dataset D with personas, an example of the dataset can be represented as (c, p, r, y).

c = {u 1 , u 2 , ..., u nc } represents a context with {u m } nc m=1 as its utterances and n c as the utterance number.


During training, the skeleton-guided response generator is trained in a similar manner as the denoising autoencoder (Vincent et al., 2008), where the model learns to recover an input pattern that is partially corrupted.

we employ a random mechanism for generating the skeletons used for training.


In this way, we learn a robust response generator that is compatible with different types of skeletons.

for any golden query-response pair (q, r), we randomly generate a training skeleton through the following procedures.


Previous studies (Shang et al., 2018;Tao et al., 2018;Mou et al., 2016) formulated the training of matching models as binary classification learning, where negative sampling is used to free human annotation.

for query q and golden response r + , a negative response r − can be randomly sampled from other responses in the training set.


Thus, the main evaluation is done by human annotators.

we evaluate the quality of a response on three criteria: informativeness, relevance, and fluency.


• Keywords We generate a skeleton by preserving the most informative words in the retrieved response.

the words with the highest TF-IDF values are preserved and the others are removed.


We propose the COnditional MEmory Relation Network (COMER), a scalable and accurate dialogue state tracker that has a constant inference time complexity.

1 our model consists of an encoderdecoder network with a hierarchically stacked decoder to first generate the slot sequences in the belief state and then for each slot generate the corresponding value sequences.


, w T , [SEP ]} into a pretrained BERT model and obtains its contextual embeddings E X .

we leverage the output of all layers of BERT and take the average to obtain the contextual embeddings.


Different from traditional binary relevance methods, they proposed a sequence generation model for MLC tasks which takes into consideration the correlations between labels.

the model follows the encoder-decoder structure with an attention mechanism (Cho et al., 2014), where the decoder generates a sequence of labels.


The difference is that the hidden variables in our model are structured and learned from extra resources, and thus encode more semantic and syntactic information.

we segment responses in D P with Viterbi algorithm (Zucchini et al., 2016), collect all chains of states as a pool and sample a chain from the pool uniformly.


To address this problem, we follow the idea in , which introduces an auxiliary loss that requires the decoder in the generation network to predict the bag-of-words in the response y.

the response y is now represented by two sequences simultaneously: y o with word order and y bow without order.


We use the K-means clustering algorithm to group z's using a pre-trained embedding corpus (Song et al., 2018).

then we revise the posterior network to perform a two-stage cluster sampling by decomposing q(z|y, x) as : we first compute q(c kz |y, x), which is the probability of the cluster that z belongs to conditioned on both x and y.


Besides, to validate the effectiveness of clustering, we implemented a modified DCVAE (DCVAE-CD) that uses a pure categorical distribution in which each variable has no exact meaning.

the embedding of each latent variable does not correspond to any word embedding.


However, the improvement of our W2W model is rather more obvious with longer context length.

for the dataset Len-15, W2W improves 5% on p@1 and 10% on session accuracy over SIRNN as shown in Figure 4, which indicate the robustness of W2W model in complex scenarios.


Human Evaluation In order to assess how well the models can maintain the replier's consistency, we conduct a human evaluation.

we randomly sample 300 context from the test set and apply 5 models to generate responses for each context.


Error Analysis To improve the performance of SSVN in the future, we take the worse cases (i.e., the grade '1' and '2') in human judgement as an example to analyze our errors.

we divide the errors with the grade '1' into grammatical error, replier-nonconsistency and logic contradiction, which occupy 19.79%, 31.49% and 48.72%, respectively.


In addition to the utterance-level information, we also model user-level information in latent continue space.

we embed user-level and utterance-level information into two multimodal distributions, and combine these two multimodal distributions into a mixed distribution.


Based on the user embeddings as U, we utilize learned user personalizations in the latent space.

the conditional prior distribution of WAE part is a Gaussian mixture distribution (GMD) conditioned on the learned user embeddings, namely personalization GMD.


In consistent with previous study (Gu et al., 2019), we compute the similarity between the bag-of-words (BOW) embeddings representations of generated results and reference.

we calculate three metrics:1) Greedy (BOW-Greedy), i.e., greedily matching words in two utterances based on the cosine similarities, and the total scores is then averaged across all words (Rus and Lintean, 2012); 2) Average (BOW-Average), cosine similarity between the averaged word embeddings in the two utterances (Mitchell and Lapata, 2008); 3) Extrema (BOW-Extrema),  cosine similarity between the largest extreme values among the word embeddings in the two utterances (Forgues et al., 2014).


We utilize BLEU score (Papineni et al., 2002) to measure n-grams overlaps between ground-truth and generated response.

we follow the conventional setting in previous work (Gu et al., 2019) to compute BLEU scores using smoothing techniques (smoothing 7) 4 .


Compared to the supervised model, the self-supervised model fine-tuned by seeker models shows significant improvements in the gamerelated measures.

the BERT-R model shows a +27.7% improvement in goal success ratio.


In this paper, we propose a novel model that bridges this gap between theory and practice.

our proposed model leverages the dialogue acts for response selection, as well as is practical.


Our simple yet novel way of combining the dialogue act representations of the context and response with the utterance representations of the context and response promotes cross similarities, and thereby bring in ensemble characteristics in the model.

the ensemble model outperforms all other non-ensemble models, and is robust to the errors made by any underlying components of the ensemble.


In addition, we propose a low-rank version of HOCA which significantly reduces the com-putational complexity of HOCA.

the attention weights of HOCA are computed based on the similarity tensor between modalities, fully exploiting the correlation information of different modalities at each time step.


7 and 8, we rewrite (C I l n ) r l as 1 : We assume a low-rank factorization of the tensor W I l n−1 .

w I l n−1 is decomposed into a sum of k rank 1 tensors, where the space of w I i j is R 1×t i .


On MSVD, L-HOCA-UBT has outperformed SCN, TDDF, Rec-Net, LSTM-TSA, MM-TGM, on all the metrics.

l-HOCA-UBT achieves 86.1% on CIDEr, making an improvement of 5.7% over MM-TGM.


majority of works are trained via supervised learning that requires a large corpus of caption labeled images such as the MS COCO caption dataset [Lin et al., 2014].

the dataset is constructed with 1.2M images that were asked the annotators to provide five grammatically plausible sentences for each image.


To develop the mechanism of pseudo-labeling, we use generative adversarial network (GAN) [Goodfellow et al., 2014], for searching pseudo-labels for unpaired data.

in order to find the appropriate pseudo-labels for unpaired samples, we utilize an adversarial training method for training a discriminator model.


Motivated by the nearest neighbor aware semi-supervised framework [Shi et al., 2018], we artificially generate pseudo-labels for respective unpaired datasets.

we retrieve a best match captioñ y i in D y u given a query image x i , and assign it as a pseudo-label, and vice versa (x i for y i ).


From 100% to 10%, as the amount of paired samples decreases, the fluency and the accuracy of the descriptions get worse.

we observed that most of the captions generated from the Paired Only baseline trained with 10% of paired data (11,329 pairs) show erroneous grammatical structure.


DAN consists of two kinds of attention modules, REFER and FIND.

rEFEr module learns latent relationships between a given question and a dialog history by employing a multi-head attention mechanism.


However, the agent performing these tasks still has a long way to go to be used in real-world applications (e.g., aiding visually impaired users, interacting with humanoid robots) in that it does not consider the continuous interaction over time.

the interaction in image captioning is that the agent simply talks to human about visual content, without any input from human.


Given the question and dialog history features, REFER module aims to attend to the most relevant elements of dialog history with respect to the given question.

we first compute scaled dot product attention (Vaswani et al., 2017) in multi-head settings Figure 2: Illustration of the single-layer REFER module.


We evaluate individual responses at each question in a retrieval setting according to (Das et al., 2017).

the dialog agent is given a list of 100 candidate answers of each question and asked to rank the list.


As shown in Table 1, DAN significantly outperforms all other approaches on NDCG, MRR, and R@1, including the previous state-of-the-art method, Synergistic (Guo et al., 2019).

dAN improves approximately 1.0% on MRR, 1.7% on R@1 and 0.3% on NdCG in Visdial v1.0 dataset.


Then, we have checked the contribution of the reference-aware representations for the SC and SI questions, respectively.

we make a comparison between DAN, which utilizes reference-aware representations (i.e., e ref t ), and No REFER, which exploits question representations (i.e., q t ) only.


FIND denotes the use of FIND module only, and RE-FER denotes the use of single-layer REFER module only.

rEFEr uses the output of rEFEr module as the encoder outputs.


better performance than with VGG-16 features.

the use of object-level features boosts the MRR performance of DAN.


As shown in Table 4, the use of the residual connection (i.e., Res) boosts the MRR score of DAN.

dAN utilizes the excellence of deep residual learning as in Rocktäschel et al., 2015;Yang et al., 2016;Kim et al., 2016;Vaswani et al., 2017).


We stack the REFER modules up to four layers with each different number of attention heads, h ∈ {1, 2, 4, 8, 16, 32, 64}.

we conduct the ablation experiments with twenty-eight models to set the hyperparameters of our model.


For n ≥ 2, REFER (n) indicates that DAN uses a stack of n identical REFER modules.

for each pair of successive modules, the output of the previous REFER module is fed into the next REFER module as a query (i.e., q t ).


Transcriptions also include reliably annotated markers for audience behavior.

the "laughter" marker has been used in NLP studies as an indicator of humor (Chen and Lee, 2017).


We evaluate our architecture on three tasks pertaining to movie understanding.

we use the recently introduced dataset of Frermann et al.


In cases where the dataset contains, apart from snippetlevel, also token-level annotations, the attention module can be directly supervised: we add a term ... to the model's loss that minimizes the error of the attention scores, with respect to token-level annotations.

we consider a case where the token-level annotations are of three different types and we create a supervised attention head for each type, averaging their outputs.


casting the task of perpetrator identification as a binary classification task, is based on the premise that there is, at most, one perpetrator.


According to the properties of POS tags, we propose a more credible adaptive spatial attention model to predict next word.

the related video features of noun and verb are applied as the supervision to get a sentinel gate.


we create a new dataset for this problem, and we explore a variety of features modeling the claim, the image, and the relationship between the claim and the image.


We have presented our efforts towards fighting Fauxtography, namely detecting fake claims about images, which is an under-explored research direction.

we created a new dataset for this problem, and we explored a variety of features modeling the claim, the image, and the relationship between the two.


And the query x is also encoded by a similar self-attention mechanism.

the attention processing units (APU) we employ is based on the Transformer model (Vaswani et al., 2017), which has achieved great success in natural language processing.


And instead of encoding these two channels individually, we design a cross connection between the APU-2 to learn the interactions both inside and between different modalities.

we use the updated query O f q to guide the middle attention layer of the video channel APU-2, in order to learn the query-aware video information v f q .


After the submission of this paper, several new works were published with excellent results on VCR, in some cases exceeding the performance of our system.

we mention ViLBERT , VL-BERT (Su et al., 2019), Unicoder-VL (Li et al., 2019a), and VisualBERT (Li et al., 2019b).


We create triggers that are prepended before the user input t to cause the model to generate similar content to a set of targets Y.

5 our trigger causes the GPT-2 language model (Radford et al., 2019) to output racist content.


This section shows that the success of triggers arises from model and data failures.

we confirm that models exploit biases in the SNLI dataset (Section 6.1) and show that SQUAD models overly rely on type matching and the tokens that surround answer span (Section 6.2).


Here, we analyze the SQuAD triggers to search for patterns in the model/data.

we investigate the triggers' alignment with high PMI tokens, the impact of answer types, and the models' sensitivity to the placement of the triggers.


In this section we introduce several metrics of different natures that correlate with the drop in a model performance without the need of any annotated examples from the target domain.

we will consider three families of measures: • H-divergence based metrics: based on the capacity of another classification model to distinguish between samples from D s and D t .


For a given task, this representation is supposed to provide a view of the input that highlights the relevant part that could be helpful for a correct classification in this task.

those intermediate representations should not be sensitive to task-irrelevant features that provide nevertheless strong signals to distinguish between the source and the target domains (yielding high PAD values).


However, they are prone to fail when there is a severe change in the marginal distribution that is task irrelevant.

the well established PAD measure is rendered useless in a setting where we artificially exaggerate that phenomena.


For all α > 1 it permits sparse solutions, in stark contrast to softmax.

for α = 2, it recovers the sparsemax mapping (Martins and Astudillo, 2016), which is piecewise linear.


This shows that the initialization of α does not predetermine its sparsity level or the role the head will have throughout.

head 8 in the encoder self-attention layer 2 first drops to around α = 1.3 before becoming one of the sparsest heads, with α ≈ 2.


We present the first empirical analysis of Transformers with sparse attention mappings (i.e., entmax), showing potential in both translation accuracy as well as in model interpretability.

we analyzed how the attention heads in the proposed adaptively sparse Transformer can specialize more and with higher confidence.


Our primary technical contribution is the introduction of a tool for reporting validation results in an easily interpretable way: expected validation performance of the best model under a given computational budget.

3 given a budget sufficient for training and evaluating n models, we calculate the expected performance of the best of these models on validation data.


We observe that as the time budget increases, the expected best-performing model changes.

we find that our experimental setup leads to the same conclusion as Peters et al.


In contrast, the stylistic content that specifies attributes of the specific font (such as serifs, drop shadow, texture) is represented as finer-grained information that enters into the decoder at a later stage, by parameterizing filters, as shown in the right half of Figure 2.

we form our decoder as follows: first 1 sometimes erroneously referred to as a "deconvolution" the character embedding is projected to a low resolution matrix with a large number of channels.


One approach for achieving robustness to the number of observations is through the training procedure.

when computing the approximate posterior for a particular font in our training corpus, we mask out a randomly selected subset of the characters before passing them to the encoder.


Encoder Architecture: Another way to encourage this robustness is through inductive bias in the encoder architecture.

we use a convolutional neural network which takes in a batch of characters from a single font, concatenated with their respective character type embedding.


Because these datapoints are less informative with respect to a model's ability to generalize to previously unseen styles, we additionally evaluate on a second test set designed to avoid this redundancy.

we choose the 10% of test fonts that have maximal L 2 distance from their closest equivalent in the training set, which we call "Test Hard".


For each batch B A and each batch B R , we construct batches of corresponding negative pairs T A (B A ) and T R (B R ), containing new pairs of words sampled among those present in the batch of positive pairs.

half of the negative examples t l and t r for ATTRACT (or RE-PEL) pairs are chosen by retrieving the nearest (or farthest) neighbours to x l and x r , respectively, in terms of cosine similarity.


The intrinsic evaluation is based on a set of (true) word similarity benchmarks manually translated from (subsets of) the English SimLex  and re-scored in the target languages.

9 the benchmarks 8 For both models, the hyper-parameters are chosen with a grid search over the intervals h={1024, 2048, 4096}, l={2, 3}, lr={0.1, 0.01, 0.001}, and optimizers in {Adam, SGD}, using a held-out dev set (10% of the constraints).


The intuition is that the meaning of a word is not intrinsically metaphorical or emotional, but both of these phenomena may manifest when the word is used in a particular context.

we train deep learning architectures on metaphor identification and emotion prediction tasks jointly.


These signatures can indicate that a verb licenses positive (+), negative (−), or neutral (•) inferences.

karttunen (2012) defines these as two-bit signatures, to reflect that verbs 2 may behave differently in positive vs. negative environments.


For example, a factive verb construction like "know that" has a +/+ signature, indicating that the complement projects positively in both positive and negative environments.

both "He knows that the answer is 5" and "He does not know that the answer is 5" imply that "The answer is 5".


We generate NLI-style premise/hypothesis pairs based on sentences drawn from the train+dev splits of the MultiNLI (Williams et al., 2018) corpus.

we collect all sentences appearing in MNLI 4 which contain any verbcomplement construction, e.g.


We then represent each verb v (in the context of a given sentence S) using a continuous analog of the projectivity signatures discussed in §3.

we take the mean score for S, C as a measure of the veridicality of v (in the context of S) in a positive environment, and the mean score for ¬S, C as a measure of the veridicality in a negative environment.


While embedding under such verbs should not license any inferences about the truth of the complement, we observe that, in practice, these verbs tend to behave like +/− verbs.

the complement is taken as true in positive environments and as false in negative environments.


Second, we observe that, while individual verb types tend to behave in line with their expected signatures on average, signatures provide a weak signal for predicting the inferences licensed by the verb in any sentence individually.

within each signature, we see high variance across contexts, in all cases span- The GAO has indicated that it is unwilling to compromise.


We now turn to our primary question: do current NLI models capture the veridicality of verbs?

we are interested in the behavior of a distributional model that is not specifically endowed with lexical semantic information related worthy effects.


We use the state-of-the-art BERT-based NLI model.

we use the original Tensor-Flow implementation 11 of the NLI model built on top of the pertained BERT language model (Devlin et al., 2018).


Broadly speaking, the model's predictions appear to follow the same trends as the humans' ratings (Figure 1).

averaged across contexts, the model's treatment of verbs is the same as the humans' treatment: largely in line with the signatures, but with a bias against assuming neutral (non-veridical) behavior.


Moreover, we see that this effect is not just driven by sensitivity to the specific vt bigram.

simply swapping "to" with "that" (or vice-versa) in a naturally-occurring context leads to a small  shift in the distribution of the model's predictions away from the target D vt distribution, but not to the same degree as replacing a "that"-taking verb with a "to"-taking verb in a naturally-occurring "that" context (or vice-versa).


We compute the percentage of the time that a color word's translation occurs with a suffix that is strongly associated with one of the top 10-highestranking colors on the basicness scale, according to the aggregation we mention in §4 and detail in §8.

words that associate with a typical color affix in that language tend to be colors.


To well address this, we come up with an attention-based neural network to model contextual information.

we introduce a framework which consists of a Bidirectional Long Short-Term Memory (BiLSTM) neural network and a Conditional Random Fields (CRF) layer to effectively encode the order information and the long-range context dependency in a sentence.


In this paper, we come up with two types of attention mechanisms to enforce the model to take full advantage of the contextual information for negative focus detection.

we introduce a framework which consists of a bidirectional long short-term memory neural network and a conditional random fields layer (BiLSTM-CRF) to effectively encode the order information and the long-range context dependency in sentence.


However, we show that most of these models often fail on harder tasks with more realistic application scenarios.

the existing models underperform on tasks that require the model to be sensitive to local contexts such as candidate ranking in conversational dialogue and in machine translation.


We observed that the naive pairwise ranking loss that uses a fixed margin unfairly penalizes the locally positive sentences during training.

the loss should be active only for local windows that differ in D pos and D neg .


Additionally, we exclude the cases of the overlap between windows.

the sentences are allowed to be permuted only inside their respective window.


To search for optimal parameters, we conducted various experiments while varying the hyper-parameters.

minibatch size in {5, 10, 20, 25}, sentence embedding size in {128, 256}, lightweight convolution kernel size in {3, 5, 7, 9}, bilinear output dimension size in {32, 64} are investigated.


To investigate the impact of different components in our proposed model, we conducted two sets of ablation study on the local and global discrimination tasks.

we want to see: (i) the impact of our global model component, and (ii) the impact of the language model (LM) loss.


To address the above limitations, this paper proposes a novel Topic-Guided Coherence Modeling (TGCM) for sentence ordering by capturing local and global dependencies among sentences.

tGCM is composed of two major components: topic-sensitive sentence encoder and attentive pointer decoder.


The primary goal of sentence set ordering is to put an unordered set of sentences into a coherent paragraph in the correct order.

given a paragraph p, the correct sentence sequence and its order are denoted by , respectively, where |p| denotes the number of sentences in p. Given a shuffled sentence sequence, our TGCM outputs a sentence sequenceŜ p whose order is denoted byÔ p .


We leverage an extended version of LSTMs (Hochreiter and Schmidhuber, 1997) as our base model in Figure  3, which overcomes the gradient vanishing of existing RNNs.

attention-based bidirectional LSTMs (Att-BLSTMs) (Zhou et al., 2016) are mostly used in sentence modeling.


Since the two models do not consider all sentences at a time, they cannot leverage paragraph-level information.

they cannot capture the global dependencies that help sentence ordering.


We apply blank noise variational smoothing (Kong et al., 2019) with a dropout rate of 0.25 to the tied embeddings to regularize the model.

for each document we sample a set of word types to drop and replace their word embeddings with the < UNK > token's word embedding.


(5) Generally, performance is highly dependent on the dataset domain and structure.

the performance on the Instr-DT is generally lower and saturates earlier than on RST-DT.


We study this question on the ACL and EMNLP paper collections and present an analysis on how well deep learning techniques can infer the authors of a paper.

we explore Convolutional Neural Networks trained on various aspects of a paper, e.g., content, style features, and references, to understand the extent to which we can infer the authors of a paper and what aspects contribute the most.


Notably, Hill and Provost (2003) showed that the authors of a scientific paper can be inferred with fairly high accuracy using only the papers it references.

using the vectorspace representations of the list of references (or citations) in a paper and measuring similarity between these representations and the pattern of citations of an author, they are able to infer the authors of a paper with accuracy up to 60% for the top-10% most prolific authors, and show that selfcitations are an important predictive factor.


As Hill and Provost (2003) have shown before, citations alone can be a strong indicator of the authors of an article, with self- citations being especially telling.

our baseline algorithm consists of simply ranking the authors cited in an article in reverse order of how frequently they were cited overall in the article's references list, and outputs as predicted authors the top (10) most cited authors in this ranking.


The attention layer aggregates the sequence of feature vectors (C) by measuring the contribution of each vector to form the high level representation of the harassment story.

that is, a fully connected layer with non-linear activation was applied to each vector c i to get its hidden representation u i .


Only a small percentage of the words appearing in news can be used for meaningful information extraction and analysis (Manning et al., 1999;Hovold, 2005).

we eliminated too frequent (at least once in more than 50% of the days) or too rare (appearing in less than 100 articles) (Manning et al., 2008).


ADR detection from social media is, however challenging, as online users report ADRs using a different language style and terminology that largely depend on the user's medical proficiency, as well as the type of online medium (e.g., health forums vs micro-post social networks).

laymen often use diverse dialects (Karisani and Agichtein, 2018) when describing medical concepts, and make abundant use of informal terminology.


with different subsets of training data and different size of newly generated content for each labeled sample) and compare their performance.

the core advantage of our approach is that we are able to generate any number of additional training data samples.


• Reward: Reward in our problem is defined based on how successfully the agent obfuscated the private-attribute information against the attacker so far.

we defined the reward function at state s t+1 according to the confidence of private-attribute inference attacker C p k for private-attribute p k given the resultant text embedding at state s t+1 , i.e., v t+1 .


Sentiment analysis has many applications in user-behavioral modeling and Web (Zafarani et al., 2014).

we predict sentiment of the given textual embedding.


Second, selecting the correct answer requires reading the context paragraphs.

if we were not provided with the context paragraph for the second problem, for example, the plausible correct answer could have been B or C instead.


We use Amazon Mechanical Turk (AMT) to collect questions and answers.

for each paragraph, we ask a worker to craft at most two questions that are related to the context and require commonsense knowledge.


To take advantage of such questions and encourage AI systems to be more consistent with human understanding, we create unanswerable questions for COSMOS QA.

from validation outputs, we collect 2, 369 questions for which at least two workers correctly picked the answer and at least on worker determined that it is answerable without looking at the context or requires no com-mon sense.


To create false negative training instances, we randomly sample 70% of questions from the 33, 219 good question sets and replace their least challenging negative answer with "None of the above".

we fine-tune three BERT 2 next sentence prediction models on COSMOS: BERT(A|P, Q), BERT(A|P ), BERT(A|Q), where P , Q, A denotes the paragraph, question, and answer.


Considering the unique challenge of COSMOS, we explore two related multiple-choice datasets for knowledge transfer: RACE (Lai et al., 2017), a large-scale reading comprehension dataset, and SWAG (Zellers et al., 2018), a large-scale commonsense inference dataset.

we first fine-tune BERT on RACE or SWAG or both, and directly test on COS-MOS to show the impact of knowledge transfer.


To test machine for human-level intelligence, we leverage a state-of-the-art natural language generator GPT2 (Radford et al., 2019) to automatically generate an answer by reading the given paragraph and question.

we fine-tune a pre-trained GPT2 language model on all the ⇥ Paragraph, Question, Correct Answer ⇤ of COSMOS training set, then given each ⇥ Paragraph, Question ⇤ from test set, we use GPT2-FT to generate a plausible answer.


Path query answering (path QA) is the task of predicting the truth value of path queries with respect to the unobserved set F * of ground truth facts, when its incomplete subset F ⊂ F * is only available.


/r k , o), a candidate set was first computed.

the candidates are the entities for which r k (i.e., the last relation in the test query) takes as its object at least once in F * .


In this paper, we strive toward the above two issues via incorporating diversified contexts and answer-aware loss.

we propose a neural encoder-decoder model with multi-level copy mechanisms to generate such questions.


Therefore, this task has attracted more attention in recent years (Serban et al., 2016;Elsahar et al., 2018).

kBQG is the task of generating natural language questions according to the input facts from a knowledge base with triplet form, like <subject, predicate, object>.


To address the aforementioned two issues, we exploit more diversified contexts for the given facts as textual contexts in an encoder-decoder model.

besides using predicate contexts from the distant supervision utilized by Elsahar et al.


Then we use a multi-level copy mechanism (K-B copy and context copy), which allows copying from KBs and textual contexts.

we first map the input of the decoder into an embedding representation by looking up word embedding matrix, then we use position embedding (Vaswani et al., 2017) to enhance sequential information.


Inspired by the CopyNet (Gu et al., 2016), we directly copy words in the textual contexts C, and it does not rely on any POS tagging.

the input sequence χ for the context copy is the concatenation of all words in the textual contexts C. Unfortunately, χ is prone to contain repeated words because it consists of rich contexts for subject, predicate and object.


(2) Answer coverage: We define a novel metric called answer coverage to identify whether the generated question refers to a definitive answer.

answer coverage is obtained by automatically calculating the percentage of questions that contain answer type words, and answer type words are object contexts (entity types for the object are regarded as answer type words).


It demonstrates that all these components are useful.

the last line in Table 4, replacing diversified contexts with contexts used in Elsahar et al.


To further prove the effectiveness of our model, we will see how useful the generated questions are for training a question answering system over knowledge bases.

we combine humanlabeled data with the same amount of modelgenerated data to a typical QA system (Mohammed et al., 2018).


We employ a neural semantic parsing based approach to tackle the problem.

given a question, a semantic parsing model is used to produce a logical form which is then executed on the KB to retrieve an answer.


To handle ellipsis or coreference in conversations, our model takes current user question combined with dialog history as the input question U .

all those sentences are concatenated with a [SEP] separated, and then a special token [CTX] is appended.


Therefore, we design a novel type-aware entity detection module in which the prediction is fulfilled in a joint space of IOB tagging and corresponding entity type for disambiguation.

the prediction space is defined as E = {O, {I, B} × {ET k } N (t) k=1 } where ET k stands for the k-th entity type label, N (t) denotes number of distinct entity types in KB, and |E| = 2×N (t) +1.


An operator is legitimate if its left-hand semantic category in the definition is identical to the leftmost nonterminal (i.e., un-instantiated semantic category) in the incomplete logical form parsed so far.

the decoding of a logical form begins with the semantic category start.


HRED+KVmem (Saha et al., 2018) and D2A (Guo et al., 2018) are two typical approaches, and we compared them with our proposed approach.

hRED+KVmem is a memory network (Sukhbaatar et al., 2015;Li et al., 2017) based seq2seq model, which combines hRED model (Serban et al., 2016) with keyvalue memory network (Miller et al., 2016).


They will create bilingual parallel questions and find corresponding answers to the questions after reading the bilingual parallel passages.

we encourage workers to create questions according to the following rules: • For each parallel passage, at least three bilin- gual question-answer pairs are to be created.


In order to guarantee the quality of created question-answer pairs, we use a strict quality control procedure during data annotation.

30% annotated data from each group are randomly sampled and passed to the three reviewers  who will review all answers created by the workers and correct answers if they consider they are wrong.


Unlike SQuAD (Rajpurkar et al., 2018) where nearly half of questions are dominated by what questions, the question type distribution in BiPaR is of better dispersion over multiple question types.

biPaR has 15.6% why and 9.5% how questions.


Like evaluations on other extraction-based datasets, we used EM and F1 to evaluate model accuracy on BiPaR.

we used the evaluation program of SQuAD1.1 for the English dataset in BiPaR, and the evaluation program 5 of CMRC2018 for the Chinese dataset in BiPaR.


We consider two versions of this procedure that differ in how the entity linking is implemented: RE n makes use of a naïve entity linking solution based on exact string matching, while RE o uses an oracle for entity linking in addition to string matching.

assume we query for the object o of a test subjectrelation fact (s, r, o) expressed in a sentence x.


The size of the vocabulary can influence the performance of a model for the LAMA probe.

the larger the vocabulary the harder it would be to rank the gold token at the top.


The experimental results on a public numerical MRC dataset DROP (Dua et al., 2019) show that our NumNet model achieves significant and consistent improvement as compared to all baseline methods by explicitly performing numerical reasoning over numbers in the question and passage.

we show that our model could effectively deal with questions requiring sorting with multi-layer NumGNN.


But we found that a large proportion of aligned sentence pairs in machine translation are extracted from parallel documents, such as MultiUN corpus and OpenSubtitles corpus.

these MT corpus are document-level corpus in which each sentence and its translation is well aligned.


Motivated by this, we propose Adversarial Domain Adaptation for MRC (AdaMRC), a new approach that utilizes adversarial learning to learn domain-invariant transferable representations for better MRC model adaptation across domains (see Figure 1(b), where the two feature distributions learned by AdaMRC are indistinguishable through adversarial learning).

our proposed method first generates synthetic question-answer pairs given pas-sages in the target domain.


4.3), where a domain label is predicted to distinguish a feature vector from either the source domain or the target domain.

the MRC module is composed of an encoder and a decoder.


All the parameters (θ e , θ d , θ c ) are jointly optimized, with the objective of training the encoder to correctly predict the answer span, but also simultaneously fool the domain classifier.

the encoder learns to map text input into a feature space that is invariant to the switch of domains.


the QG model consists of a lexicon encoding layer, a BiLSTM contextual encoding layer, and an LSTM decoder.


It also includes a cross-attention layer for fusion.

the output of the lexicon encoder is appended with the CoVe vector and passed to the contextual encoding layer, which is a 2-layer Bidirectional LSTM that produces hidden states of the passage H p ∈ R T ×2m and the question H q ∈ R T ×2m , where m is the hidden size of the BiLSTM.


KEAG is capable of leveraging symbolic knowledge from a knowledge base as it generates each word in an answer.

we assume that each word is generated from one of the four information sources: 1. question, 2. passage, 3. vocabulary and 4. knowledge.


To address the answer generation problem, we propose a novel KEAG model which is able to compose a natural answer by recurrently selecting words at the decoding stage.

each of the words comes from one of the four sources: question q, passage p, global vocabulary V, and knowledge K. at every generation step, KeAG first determines which of the four sources to inspect based on the current state, and then generates a new word from the chosen source to make up a final answer.


The presence of discrete latent variables z, however, presents a challenge to training the neural KEAG model, since the backpropagation algorithm, while enabling efficient computation of parameter gradients, does not apply to the nondifferentiable layer introduced by the discrete variables.

gradients cannot propagate through discrete samples from the categorical distribution P (z t |F, s r t ).


To address this problem, we create a differentiable estimator for discrete random variables with the Gumbel-Softmax trick (Jang et al., 2017).

we first compute the discrete distribution P (z t |F, s r t ) with class probabilities π 1 , π 2 , .


To estimate the expectation in Equation 11, we use Monte Carlo sampling on the source selector variables y in the gradient computation.

the Gumbel-Softmax trick is applied to generate discrete samplesŷ from the probability given by: where x r t is the embedding of the answer word in timestep t, W y and b y are learnable parameters.


N = 100 or 500) returned documents instead of sorting them by a lexical metric produces better results and delivers more meaningful documents for answering the question at hand.

some better documents can be found in the first N (where N is a small number), but the best is not always the one with the highest lexical (e.g.


We build on the QANet (Yu et al., 2018) model by modifying the encoder of its self-attention modeling layer.

we specialize self-attention heads to focus on specific discourse-semantic annotations, such as, e.g., an ARG1 relation in SRL, a CAUSA-TION relation holding between clauses in shallow discourse parsing, or coreference relations holding between entity mentions.


Our aim is to extract structured knowledge about these phenomena from long texts and to integrate this information in a neural self-attention model, in order to examine to what extent such knowledge can enhance the efficiency of a strong reading comprehension model applied to NarrativeQA.

we enhance self-attention with knowledge about entity coreference (Coref), their participation in events (SRL) and the relation between events in narrative discourse (Shallow Discourse Parsing (Xue et al., 2016), DR).


We then provide empirical studies to validate design decisions.

we prove the necessity of both paragraph-level retrieval and sentencelevel retrieval for maintaining good performance, and further illustrate that a better semantic retrieval module not only is beneficial to achieving high recall and keeping high upper bound for downstream task, but also plays an important role in shaping the downstream data distribution and providing more relevant and high-quality data for downstream sub-module training and inference.


Our system obtained substantially higher scores than all previously published results with a ∼4 and ∼3 points absolute improvement on Label Accuracy and FEVER Score.

the system gains 74.62 on the evidence F1, 22 points greater that of the second system, demonstrating its ability on semantic retrieval.


We demonstrate that our approach has a bigger impact on downstream QA approaches that use these justification sentences as additional signal than a strong baseline that relies on information retrieval (IR).

the contributions of this work are: (1) We propose an unsupervised, non-parametric strategy for the selection of justification sentences for multi-hop question answering (QA) that (a) maximizes the Relevance of the selected sentences; (b) minimizes the lexical Overlap between the selected facts; and (c) maximizes the lexical Coverage of both question and answer.


We use the Lucene implementation 4 of the BM25 IR model (Robertson et al., 2009) to estimate the relevance of each justification sentence to a given question and candidate answer.

we form a query that concatenates the question and candidate answer, and use as underlying document collection (necessary to compute document statistics such as inverse document frequencies (IDF)) either: sentences in the entire KB (for ARC), or all sentences in the corresponding passage in the case of reading comprehension (MultiRC).


In all situations we used BERT (Devlin et al., 2018) for answer classification.

we employed BERT as a binary classifier operating over two texts.


Oracle Query vs Single-hop Query We evaluate the oracle query against the single-hop query, i.e., querying with the original question, on the HOTPOTQA dev set.

we compare the recall of gold paragraphs, because the greater the recall, the fewer documents we need to pass into the expensive neural multi-hop QA component.


We would first retrieve at least 50 candidate documents for each query for consideration, and boost the query scores of documents whose title exactly matches the search query, or is a substring of the search query.

we IR System R@10 for d1 R@10 for d2  multiply the document score by a heuristic constant between 1.05 and 1.5, depending on how well the document title matches the query, before reranking all search results.


We mainly employ three heuristics to find the semantic overlap between the retrieval context and the desired documents: longest common subsequence (LCS), longest common substring (LC-SubStr), and overlap merging which generalizes the two.

the overlap merging heuristic looks for contiguous spans in the retrieval context that have high rates of overlapping tokens with the desired document, determined by the total number of overlapping tokens divided by the total number of tokens considered in the span.


Our model performs a two-step translation by cascading two sequence-to-sequence models, one for generating initial SQL statements from natural language input, and the other to fix ill-formed SQL statements.

• Evaluation metric: While existing NL2pSQL metrics penalize the mismatch of any terms, we propose novel metrics allowing the mismatches for non-essential terms, yet requiring matches for important SQL keyword: Canonical-BLEU abstracts the identifier names of the SQL statements before calculating the BLEU scores while SQL-BLEU calculates the BLEU scores using only the SQL keywords.


Our goal is to curate question and code pairs: For example, in the SO Question 6308594, 4 the natural language input is "how can I copy data from one column to another in the same table", and the code is "update table set columnB = columnA".

we collect all the answers where they have a single code block and their corresponding questions are labeled with an SQL tag from 'Posts.xml'.


Next, we generate its ill-formed counterpart, by perturbing each of the collected well-formed SQL statements.

we randomly decide to inject into or remove from the well-formed SQL statement a token for k iterations to generate multiple ill-formed statements for a given well-formed SQL statement.


For overall performance, GUpdater significantly outperforms all baseline models, which coincides to our intuition.

in the 1hopsubgraphs, GUpdater beats IE-gold + R-GCN by 0.42 in Added Acc, 0.65 in Deleted Acc, indicating that GUpdater can well capture both explicit and implicit information in the news snippets.


However, such reward engineering depends on meticulous artificial design with domain expertise, which can be significantly challenging in practice (Ng et al., 2000).

these RL-based methods are extremely sensitive to their reward functions, where a little variation may lead to a significant fluctuation of the reasoning performance.


In this way, for different datasets, the reward functions can be automatically tuned to approximate the optimal performance, eliminating extra reward engineering and manual interventions.

to enable the policy-based agent to find more diverse evidential paths for predictions, we propose a path-based GAIL method, which can learn the reasoning policy by imitating the path-level semantic features of the demonstrations.


Recently, DeepPath (Xiong et al., 2017) and MINERVA (Das et al., 2018) were proposed to address the problem above by using reinforcement learning, where they are committed to learn a policy which guides the agent to find more superior evidential paths to maximize the expected reward.

deepPath parameterizes its policy with a fully-connected neural network and uses manual reward criteria, including global accuracy, efficiency and diversity, to evaluate the path quality.


In general, RL-based reasoning methods strive to find relational paths to tune their reasoning policies for predictions and formulate the path-finding problem as a Markov decision process (MDP).

in such a process, the policy-based agent decides to take an action a ∈ A from the current state (i.e., the current entity and its context information) s ∈ S to reach the next one according to its reasoning policy π, where the action space is defined as all the relations in G. each relational chain in the relational paths can be considered as a reasoning chain.


As shown in Figure 1, our framework DIVINE consists of two modules, namely a generative adversarial reasoner and a demonstration sampler.

the reasoner is composed of a generator and a discriminator.


A straightforward approach is to directly apply GAIL to train the reasoner.

the policy-based agent in the reasoner is trained to find evidential paths by imitating the state-action pairs in each expert trajectory (i.e., demonstration).


In addition, in the scenario of knowledge graph reasoning, since the reasoning chains consist of only relations, the demonstrations do not necessarily contain the state information.

the demonstrations can be composed of only relational paths.


For each package P = {x 1 , x 2 , ..., x N } containing N relational paths, we encode the package to a realvalued matrix as: where x n ∈ R k is the k-dimensional path embedding and ⊕ denotes the concatenation operator for the package representation p ∈ R N ×k .

given a relational path x = {r 1 , r 2 , ...r t , ...}, the path embedding x is encoded as: where each relation r t is mapped into a real-valued embedding r t ∈ R k pre-trained by TransE (Bordes et al., 2013).


After packing, we feed the package representation p into our discriminator D to parameterize its semantic features D(p).

a convolutional layer activated by ReLU nonlinearity is first used to extract local features by sliding a kernel ω ∈ R h×l for a new feature map: where b c denotes the bias term.


For each query relation, we use all positive entity pairs to sample demonstration candidates from the given KG.

for each positive entity pair, we use bi-directional breadth-first search to explore the shortest path between two entities.


Specifically, for each positive entity pair, we use bi-directional breadth-first search to explore the shortest path between two entities.

since the shorter paths incline to characterize more direct correlations between two entities, we prefer to use them for initialization to ensure the quality of demonstration candidates.


In the training phase, all the positive entity pairs are used to generate demonstration candidates Ω E for the imitation learning process.

for each positive entity pair, the demonstration sampler is required first to choose the corresponding demonstrations, while the generator is conducted to generate some relational paths.


The details of the two datasets are described in Table 1.

nELL-995, which is known to be a simple dataset for reasoning tasks, is generated from the 995th iteration of the nELL system (Carlson et al., 2010) by selecting the triples with Top-200 frequently occurring relations.


against the negative triples used in DeepPath and MINERVA.

there are approximately 10 corresponding negative triples for each positive ones.


To investigate the effectiveness of the path-based GAIL method, we train the policy-based agent in DeepPath on NELL-995 by our path-based GAIL method and the original GAIL method, respectively.

in the process of training the a-  gent by the original GAIL method, the demonstrations are composed of state-action trajectories.


We conduct ablation studies by embedding Deep-Path into our framework to quantify the role of components.

we re-train our framework by ablating certain components: • W/O Semantic Matching, where no semantic matching is performed on long-tail entities.


• W/O Dynamic Sampling, where no dynamic demo sampling is performed to incorporate the local environment of entities.

we only adopt the static demo sampling method to obtain demonstrations.


For the second type of framework, attention mechanism is introduced to model word-level interaction between two sentences and a higher accuracy is achieved (Rocktäschel et al., 2015;Parikh et al., 2016;Wang et al., 2017).

multi-layered deep matching network with attention shows that deeper models outperform shallower models (Duan et al., 2018).


Thus, KG completion with the goal of filling in missing parts of the KG has become an urgent issue.

kG completion aims to predict whether a relationship between two entities is likely to be true, which is defined as the link prediction in kGs.


For each triple (h, r, t) in KG, we employ vectors to represent the entity pair and the relation.

h∈R d denotes the head entity h, t∈R d denotes the tail entity t and r∈R d indicates the relation r. We assume the paths connecting two entities contain indicative information for the direct relation between these two entities.


At inference time, the fact extraction agent dynamically augments the graph with only the most informative edges, and thus enables the reasoning agent to identify positive paths effectively and efficiently.

during policy learning, the reasoning agent will be rewarded when reaching the targets, while this positive feedback will also be transferred back to the fact extracting agent if its edge suggestions are adopted by the reasoning agent, i.e.


The goal of the reasoner is learning to reason via paths finding on KGs.

given e s , r q , the reasoner aims at inferring a path from e s to some entity e o regarding r q , and specifying how likely the relationship r q holds between e s and e o .


Similar to MINERVA (Das et al., 2017), we want the state to encode all observed information, i.e., we define s t R = (e s , r q , h t ) ∈ S R , where h t encodes the path history, and (e s , r q ) is the context shared among all states.

we use a LSTM module to encode the history, h t = LSTM(h t−1 , [r t , e t ]) (see Fig.


The policies of both agents are updated via RE-INFORCE algorithm (Williams, 1992) (details are in later of this section).

we introduce the details on agent collaboration as follows.


Therefore we define that the fact extractor receives a step-wise delayed reward from the reasoner.

when the reasoner finishes exploration (at time T ) and arrives at the correct target, we consider the path is effective and positive for reasoning.


The MDPs of both agents are explicitly defined above now, and we can use the typical REINFORCE algorithm (Williams, 1992) to train the two agents.

their goals are maximizing the reward expectation, defined as where R(s, a) is the reward of selecting a given s, and π θ (a|s) is the policy learned by the agents and will be defined formally in Section.


(2), we can see that REIN-FORCE will update the parameters only when G t is non-zero.

the value of γ determines how the parameters are updated and to what extent the the internal states will be influenced by the future.


This is because zero-rewarded experiences are mostly negative examples.

if a state of the extractor is zero-rewarded, we can infer that either the suggested edge is not selected by the reasoner, or the selected edge does not contribute to reaching the target.


We use model pre-training and adaptive sampling to increase training efficiency.

we first train the reasoner on the original KG to get a better initialization.


To show the impact of graph size, we sub-sample the KG into different subgraph.

for the two datasets, we first partition the whole KG into three parts according to the proportion 8:1:1 (The training set, validation set, and testing set).


Traditional CVAE can model the event-target relation.

given an observed event, CVAE can generate its corresponding targets.


Inspired by the coattention mechanism , we propose an attention-based inferer (ABI) to estimate the mean and standard deviation of a distribution belongs to p θ (•) or q φ (•) by capturing semantic interactions of input sequences.

given two input sequences (e.g., representations of contexts and events) a = {a 1 , .


One-class models assume the availability of data for only one (positive) class, rather than positive and negative data.

one-class clustering assumes that the data consists of few positive cases among a large group of outliers (Bekkerman and Crammer, 2008;Banu Topic question: Why did MH-17 crash?


The process breaks down to three steps: first, we pair c j with each of the sentences t k ∈ T and compute a sentence-sentence relational vector v j,k which is the sum over all the word-word bi-linear contraction (the bi-linear contraction of two word embeddings Then, we average over the sentencesentence relation vectors to obtain a summary vector p j = average  Since our models' supervision comes in the form of whole scenarios, supervising our iterative insertion clustering model is not completely straightforward.

there may be multiple correct sentences that can be added to the current scenario.


We thus optimize the marginal log-likelihood of making any correct decision.

9 we minimize: L = −log j p(c j ), which maximizes the sum of the probabilities of the correct candidates.


It can make better use of seed alignments to propagate over the entire graphs with KG-based constraints.

as for the knowledge embedding model, we utilize TransE to implicitly complete two KGs towards consistency and learn relational constraints between entities.


We raise a potential problem with this approach: it fails to take into account differences in ease of memorization across layers.

we find that while linear and MLP probes on the first layer of ELMo (ELMo1) achieve slightly higher part-of-speech accuracy than those on the second layer (ELMo2), (97.2 compared to 96.6, for a loss of 0.6 ), the same probes achieve much greater selectivity on ELMo2 (31.4 compared to 26.0, for a gain of 5.4).


The possible behaviors C(v) are as follows: attach to self: Always attach tokens of this type to themselves.

y i = i. attach to first: Always attach tokens of this type to the first token.


That is, y i = i. attach to first: Always attach tokens of this type to the first token.

y i = 1. attach to last: Always attach tokens of this type to the last word in the sentence.


The MLP-1 probe is defined as follows: Note here that h i broadcasts to R T ×d ,while W 1 ∈ R ×d , and W 2 ∈ R 1× broadcast as well.

each [h j ; h i ] pair is mapped to a single scalar independently of all others, leading to T logits used as input to the softmax.


In all cases, we see that the right choice of probe leads to considerably higher selectivity than dropout or no regularization.

for part-of-speech tagging, our chosen MLP-1 probe achieves 16.6 selectivity, up from 4.5, and on dependency head prediction, 6.2 selectivity, up from -0.7.


As we note, these results depend on the probe as well as the representation; given what we know about probes' capacity for memorizing at the type level, we explore an alternative to the hypothesis that ELMo1 has higher-quality part-of-speech representations than ELMo2.

word identities are strong features in part-of-speech tagging when used in combination with other indicators; since ELMo1 is closer to the word representations than ELMo2, it may be easier to identify word identities from it, meaning the probe may utilize word identities more readily, as opposed to picking up on a representation of part-of-speech.


We find probes on ELMo2 to be strikingly more selective than those on ELMo1, consistent across all probes, both for part-of-speech tagging and dependency head prediction.

the linear probe on ELMo2 achieves selectivity of 31.4, compared to selectivity of 26.0 for ELMo1, for a gain of 5.4.


Although we do want tag t i to consider context-e.g., to distinguish between noun and verb uses of word i-we want "most" of t i 's information to come from word i itself.

it should come from ELMo's level-0 embedding of word i, denoted byx i -a word type embedding that does not depend on context.


In practice, we found that I(T i ; X |X i ) was small even when = 0, on the order of 3.5 nats whereas I(T i ; X) was 50 nats.

the tags extracted by the classical method were already fairly local, so increasing above 0 had little qualitative effect.


The graph in Figure 4 shows a "U-shaped" curve, with the best overall error rate at = 0.01.

moderate compression of ELMo embeddings helps for predicting POS tags.


The increasing use of neural networks has also led to a convergence in feature representations and learning algorithms for transition-based and graph-based parsers.

most recent systems rely on an encoder, typically in the form of a BiLSTM, that provides contextualized representations of the input words as input to the scoring of transitions -in transition-based parsersor of dependency arcs -in graph-based parsers.


It is important to note that while the ELMo models we work with are monolingual, the BERT model is multilingual.

while the standalone ELMo models were trained on the tokenized WikiDump and CommonCrawl for each language respectively, the BERT model was trained only on the former, albeit simultaneously for 104 languages.


Furthermore, a linear mixed effect model analysis reveals that, when accounting for language as a random effect, there are no significant interactions between the improvement of each model (over its respective baseline) and factors such as language family (IE vs. non-IE), dominant word order, or number of training sentences.

the improvements for both parsers seem to be largely independent of treebank-specific factors.


We see that there is a clear tendency for the transition-based parser to have better precision and the graph-based parser better recall.

9 non-projective dependencies are more likely to be correct when they are predicted by the transition-based parser using the swap transition, but real non-projective dependencies are more likely to be found by the graphbased parser using a spanning tree algorithm.


First of all, although the baseline parsers exhibit the familiar pattern of accuracy decreasing with sentence length, it is not the transition-based but the graph-based parser that is more accurate on short sentences and degrades faster.

although the transition-based parser still seems to suffer from search errors, as shown by the results on dependency length and distance to the root, it no longer seems to suffer from error propagation in the sense that earlier errors make later errors more probable.


We model graph parsing with an encoder-decoder architecture that takes as input a sentence w and outputs a directed acyclic graph G derived using the rewriting system of Section 2.

we model its derivation tree in top-down, left-toright order as a sequence of actions a = a 1 .


In addition to word embeddings 7 , we also report on experiments which make use of additional features.

for each word we add information about its universal PoS tag, lemma, universal semantic tag, and dependency label.


We tuned the hyper-parameters and performed early stopping based on question answering accuracy on the validation set.

we considered {50, 100, 200} dimensional representations, λ ∈ {0.1, 1, 10}, retrieving {10, 20} knowledge tuples and {[x -y; x*y], [x, y]} combination functions for during the development of the model.


Therefore, we propose a straightforward yet effective graph construction algorithm via path finding among mentioned concepts (C q ∪ C a ).

for each question concept c i ∈ C q and answer concept c j ∈ C a , we can efficiently find paths between them that are shorter than k concepts 4 .


We thus apply GCNs on the plain version (unlabeled, nondirectional) of schema graphs, ignoring relation types on the edges.

the vector for concept c i ∈ V g in the schema graph g is initialized by their pre-trained embeddings at first (h Then, we update them at the (l + 1)th layer by pooling features of their neighboring nodes (N i ) and their own at the l-th layer with an non-linear activation function σ: In order to capture the relational information in schema graphs, we propose an LSTM-based path encoder on top of the outputs of GCNs.


If changing the target boundary a t = {i , ..., j } improves this score, we will make a modification.

we consider updating the end position to j in the range of j ± N , within a pre-defined window size N , and enqueue a possible update j if the average score of S for a s and a t is higher than that of a s and a t .


For negative examples, we generate synthetic noises, similarly to translation/generation errors, by adopting a naive but effective method to change words or sentences in a positive example (Levy et al., 2017;Zhang et al., 2019;Yang et al., 2019).

we modify pseudo-positive labels to generate synthetic negative examples of unanswerable or wrong answer.


To deal with pseudo-positive examples unequally, we set the margin δ dynamically, which is derived from F1 score of QA model.

we relatively assign a larger margin to a higher confidence instance, and smaller otherwise.


We set the initial function f to be uniform constant, and then gradually increase the contribution of instance weighting as learning steps.

we design the function f as follows: where t is the number of current iteration step, c and λ are hyper-parameters.


Supposing zero-annotation at training, we use their training set (2K) in Wiki Kr as our dev set, and evaluate our model on the test set.

third dataset is a history RC dataset (7K), which we collect on Koreanspecific topics to show the robustness of our model with respect to domain gap: dealing with Korean history documents, as denoted as History Kr.


In this paper, we highlight the learning challenge and show that our learning method, independent of the model architecture, can give a significant gain.

we assume that one of mentions are related to the question and others are false positives because (i) this happens for most cases, as the first example in Table 2, and (ii) even in the case where multiple mentions contribute to the answer, there is often a single span which fits the question the best.


, z n } by finding exact match or similar mentions of y, where is a span of text with start and end token indices s i and e i .


To evaluate if our training strategy generalizes to other weak supervision problems, we also study a semantic parsing task where a question and an answer are given but the logical form to execute the answer is not.

we consider a task of answering questions about a given table by generating SQL queries.


Since the set of potential queries is infinite, we approximate Z tot as a set of non-nested SQL queries with at most three conditions.

4 given A as a set of aggregating operators {sum, mean, max, min, count} and C as a set of where f is an SQL executor.


Our training strategy outperforms the First-Only baseline and MML by a large margin, consistently across two base models.

with BERT, we achieve an absolute gain of 10%.


We describe the detailed model architecture used as a base model.


Evidence shows that children learn to use GAs compositionally early on: when asked to assess whether an object is 'tall' or 'short' in a visual context, 4-year-old children are able to (a) restrict the reference set by means of linguistic cues and (b) derive a tall/short threshold relative to that set.

4-year-olds do not interpret GAs categorically but compositionally.


(2009), who explore a number of statistical functions to describe speakers' use of the adjective 'tall' (following the authors, we assume these functions to be valid for any GAs, including 'big').

3 we use their Relative Height by Range (RH-R) function, according to which any item within the top k% of the range of sizes in the reference set is 'tall' ('big').


Datasets The four final datasets, including 20K datapoints each, are perfectly balanced with respect to the number of cases for each combination of variables used.

250 <scene, sentence> datapoints are included for each of the 80 classes (4 shapes * 5 colors * 2 sizes * 2 ground truths), where one class is e.g.


Unless otherwise specified, visual features representing the scenes are extracted via a fixed Convolutional Neural Network (CNN) fed with images resized to 224 x 224 pixels.

features from the conv4 layer of a ResNet-101  pre-trained on Im-ageNet (Russakovsky et al., 2015) are used.


The gradient minimal pair evaluation is a more lenient version of absolute minimal pair evaluation: Here, we count a pair as correctly classified as long as the Boolean classifier's output for the acceptable class is higher for the acceptable sentence than for the unacceptable sentence.

the classifier need only predict the acceptable sentence has the higher likelihood of being acceptable, but need not correctly predict that it is acceptable.


However, these results vary widely across the different methods we compare.

bERT performs nearly perfectly on the gradient minimal pairs task across all of minimal pair configurations and nearly all licensing environments.


Future studies would benefit from employing a variety of different methodologies for assessing model performance withing a specified domain.

a result showing generally good performance for a model should be regarded as possibly hiding actual differences in performance that a different task would reveal.


Hence, we also explore a regime where we condition the universal prior over the weights on typological side information.

we consider concatenating typological features to hidden states (Östling and Tiedemann, 2017) and generating the network parameters based on the typological features (Platanios et al., 2018).


In the few-shot setting instead, we assume that some data for the target language D e is available, so we treat the posterior as a prior over the weights and update the weights accordingly.

we maximize the log-likelihood of the target language data and the regularizer derived through the Laplace Approximation scaled by a factor λ: As a baseline for the UNIV prior, we perform Maximum A Posteriori inference with an uninformative prior N (0, 1).


A first variant, labeled OEST, is inspired byÖstling and Tiedemann (2017).

assuming the standard LSTM architecture where o t is the output gate and c t is the memory cell, we modify the equation for the hidden state h t as follows: we concatenate the typological features to all the hidden states.


Moreover, the ZERO-SHOT UNIV models are on a par or better than even the FEW-SHOT NINF models.

the most helpful supervision comes from a universal prior rather than from a small in-language sample of sentences.


We develop features to capture the properties of a word in the explanandum, and show that our proposed features not only have relatively strong predictive power on the echoing of a word in an explanation, but also enhance neural methods of generating explanations.

while the non-contextual properties of a word itself are more valuable for stopwords, the interaction between the constituent parts of an explanandum is crucial in predicting the echoing of content words.


To address these challenges, we leverage /r/ChangeMyView, a community dedicated to sharing counterarguments to controversial views on Reddit, to build a sizable dataset of naturally-occurring explanations.

in /r/ChangeMyView, an original poster (OP) first delineates the rationales for a (controversial) opinion (e.g., in Table 1, "most hit music artists today are bad musicians").


Here we provide a data-driven study of everyday explanations in the context of persuasion.

we investigate the "pointers" in explanations, inspired by recent work on pointer networks (Vinyals et al., 2015).


The delta mechanism in /r/ChangeMyView allows members to acknowledge opinion changes and enables us to identify explanations for opinion changes (CMV moderators, 2019b).

it requires "Any user, whether they're the OP or not, should reply to a comment that changed their view with a delta symbol and an explanation of the change."


Although the average likelihood declines as the document frequency gets lower, we observe an intriguing U-shape in the scatter plot.

3 the words that are most likely to be echoed are either unusually frequent or unusually rare, while most words in the middle show a moderate likelihood of being echoed.


Table 3 further shows the intuition for including each feature, and condensed t-test results after Bonferroni correction.

we test whether the words that were echoed in explanations have different feature values from those that were not echoed.


To examine the utility of our features in a neural framework, we further adapt our word-level task as a tagging task, and use LSTM as a baseline.

we concatenate an OP and PC with a special token as the separator so that an LSTM model can potentially distinguish the OP from PC, and then tag each word based on the label of its stemmed version.


Consistent with results in sequence tagging for word-level echoing prediction, our proposed features can enhance a neural model with copying mechanisms (see Table 6).

their use leads to statistically significant improvement in ROUGE-1 and ROUGE-L, while slightly hurting the performance in ROUGE-2 (the difference is not statistically significant).


Next, we analyzed the extracted labels and found that some labels have a similar meaning but are worded differently.

we noticed the presence of the following cases: 1.


An external measure is then used to evaluate the output of the clustering algorithm for each semantic space.

we use Bcubed F 1 -score (Bagga and Baldwin, 1998) to evaluate the effectiveness of our approach in modeling topics in the dataset.


Our experiments clearly show the benefit of removing topic's features for identifying an argument's frame.

we find that identifying generic frames benefits from removing topic features, which are actually the hardest case.


We take advantage of a feature of Reddit: when responding to a post, users can easily quote another user's response, which shows up in the metadata.

in CMV, this feature is used to highlight exactly what part of someone's argument a particular user is targeting.


Second, in addition to summarization, we take advantage of a dataset-specific constraint: a target cannot also be a source unless it is related to the main claim.

if B is a predicted target in A → B, we predict that there is no relation for B → A except when A is a main claim.


For a worker, we compute his or her unreliability score as follows: for x in X(w) where w is a worker, U (w) is the unreliability score of w, X(w) is the set of annotations annotated by w (possibly over multiple news editorials), and Average(u(x)) for x in X(w) indicates the average of the unreliability scores of all the annotations conducted by the worker w. In this study, the annotators were asked to proceed with the annotation in one direction.

they were asked to read a sentence, to annotate the sentence, then to read the next sentence, and so on.


To make sure that the IAA of good squared and good annotations is not the result of a different sample size, we randomly sampled 10%, 20%, ..., and 90% of all the annotations.

we adopted random sampling as a baseline method to compare with our proposed method of the twostep reason selection, and checked the effect of the sample size on IAA.


It has been argued that pairwise evaluation is useful and sufficient for machine translation evaluation (Guzmán et al., 2015(Guzmán et al., , 2017.

duh (2008) has shown that ranking-based evaluation measures can achieve higher correlations with human judgments, as ranking judgments are easier to obtain from human judges and are also easy to use in training, while also directly achieving the purpose of comparing two systems.


Generated noisy example 2: He was creative, generous, funny, loving and talented, and we will miss him dearly.

we aligned the system outputs with the reference translation using an automatic alignment tool (Dyer et al., 2013), and we found examples in which the pronouns did not match the reference translation.


We use zero-padding (shown as shaded boxes) to make P r and P s fixed-length.

we then use scaled multiplicative attention (Vaswani et al., 2017) to compute a contextual representation for the pronouns in r and s. we consider the rows of P r (resp.


Therefore, we propose to tightly integrate these constraints into the discourse relation inference process by manipulating hidden word representations to reflect relations between words, and meanwhile balance attentions to contexts and constraints through adding a knowledge regularization term in the final objective function.

we choose the paragraph-level model we proposed (Dai and Huang, 2018) as the base model, which exploits wider paragraph-level contexts and has been shown effective for PDTBstyle discourse parsing.


Similar as above, we use one feedforward neural network f coref () to update the hidden word representation h x i for words within each coreference cluster.

the output word vector has the following form: where W coref and b coref are the weights and bias, h C k is a coreference vector calculated by applying max-pooling to all word representations in one cluster: The role of coreference vector is similar to "context vector" utilized in soft attention mechanism (Bahdanau et al., 2015), but we use simple max-pooling instead of computing weights 6 for different word vectors.


The resulting regularization term is also minimized as a part of the objective function during model training.

the knowledge regularization will smoothly penalize constraint depending on whether this constraint can be applied for interpreting relevant relation in a particular context.


In other words, the knowledge regularization will smoothly penalize constraint depending on whether this constraint can be applied for interpreting relevant relation in a particular context.

we use cosine similarity 8 to measure the similarity of two vectors, so the score function for triplet (h, r, t) has the following form: where h r is the relation-specific vector which will be updated as parameters during model training.


Knowledge Preprocessing: Table 1 gives an overview of the relation types used in our experiments and the number of triplets (clusters) identified in the PDTB dataset.

for coreference relations, we utilized the Stanford CoreNLP coreference resolver to identify coreference clusters in each paragraph.


Experiments showed that our method achieved precision and recall above 98% and 64%, respectively.

it could extract causalities whose cause and effect were written distantly in a Wikipedia article.


In this study, we define a causality A → B according to Wikipedia and Wikidata.

a → B if Wikipedia describes a as causing B either explicitly or implicitly, or if a (B) has the has effect (has cause) relation to B (a) in Wikidata.


This was the only supervision of our method.

we chose Cause, Causes, Effect, and Effects as such keywords for en and translated them to the other languages by reference to Wik-tionary.


Because we aimed to extract causalities that were easy to verify by reading individual Wikipedia articles, we restricted the triples to those whose component entities co-occurred in an article.

we used only triples such that the article corresponding to one entity had a link to the article corresponding to the other entity in at least one language.


RELATED This method regarded an entity pair whose entities were semantically related to each other as a causality, because such relatedness has been shown to imply causality (Do et al., 2011;Riaz and Girju, 2010).

this method used a semantic relatedness defined as 1−sr(a, b), where sr(a, b) is a Wikipedia-link-based distance measure that was proposed by Milne and Witten (2008) and has been widely used (Lee et al., 2015).


We examined 100 samples from the causalities in the data ( §3.1) that PROP correctly classified as causalities (i.e., true positives) to measure their verifiability.

we examined how many of them consisted of cause and effect entities between which we could easily identify causality by reading their individual Wikipedia articles.


Inspired by the above observations, we propose a model called Attribute-aware Sequence Network (ASN) to consider attribute information into review summarization.

aSN is based on sequence to sequence models (S2S), which are popular methods in text summarization (Rush et al., 2015;See et al., 2017;Li et al., 2018a) and review summarization (Wang and Ling, 2016;Ma et al., 2018).


(ii) We test our method on the Pubmed and arXiv datasets and results appear to support our goal of effectively summarizing long documents.

while overall we outperform the baseline and previous approaches only by a narrow margin on both datasets, the benefit of our method become much stronger as we apply it to longer documents.


Benefiting from the success of neural sequence models in other NLP tasks, Cheng and Lapata (2016) propose a novel approach to extractive summarization based on neural networks and continuous sentence features, which outperforms traditional methods on the Dai-lyMail dataset.

they develop a general encoder-decoder architecture, where a CNN is used as sentence encoder, a uni-directional LSTM as document encoder, with another uni-directional LSTM as decoder.


Compared with other neural extractive models, our models (both with attentive context and concatenation decoder) have better performances on all three ROUGE scores, as well as METEOR.

the improvements over the Baseline model show that a combination of local and global contextual information does help to identify the most important sentences (more on this in the next section).


Besides learning the association between data and text from parallel data-text pairs as in previous work, our model attends to relevant external knowledge, encoded as a temporary memory, and combines this knowledge with the context representation of data before generating words.

both infobox and background knowledge facts are encoded and a dual-attention mechanism is proposed to guide the decoder to generate text.


To tackle the rare and unknown words problem, we adopt the copy mechanism (See et al., 2017).

a gate p gen ∈ [0, 1] is introduced to switch between copy mode and generation mode.


KBGain is defined as the average ratio between count of the tokens falling into category C and the length of their corresponding reference on the test set.

for all tokens in the reference except stop words and punctuation, we select those matched 4 on string with the object tokens of KB but not with the infobox.


Manually evaluating the generated results of all datasets is labour intensive, so we choose to evaluate two sample datasets for case study purposes.

we sample 50 instances each from the Wik-iBio and the Album datasets, and ask two annotators to extract facts tuples (subject, relation, object) from the references and the generated sentences 7 .


We show that it can be formulated as a contextual bandit problem if we select the sentences sequentially.

for each document, the context includes its documentary representation learned through rough reading.


In rough reading, we aim to form a general cognition on a given document which encodes the document into sentence embeddings as well as produces the feature set F including global and local features.

bidirectional LSTMs (BiL-STMs) on word-and sentence-level are first used to encode a document with N sentences into Second, a global featureS ∈ R ds is computed as an average of all the sentence vectors.


In detail, a stacking of N sentence vectors is represented as, We apply 1-D convolutional neural networks on S 1:N followed with a max-over-time pooling so that a final document-level representation can be extracted.

we altogether used K convolutional nets with K different window sizes to summarize different gists of paragraphs.


As we need to learn the relations between each sentence and the rest of the document, we update the sentence representations by, Then, we utilize a decoder Dec 1 to score the Salience and Coverage for each sentences, and a secondary score function Dec 2 to screen the sentences that might have Redundancy.

we implement Dec 1 and Dec 2 as multi-layered perceptrons.


In our work, we find the use of -greedy with stochastic policy works well enough to encourage exploration.

with a probability of 1 − , the agent chooses the sentence following the current policy, i.e., to sample an index a t ∈ [1, N ] from the multinomial distribution with sentence affinities {µ 1 , µ 2 , .


We consider improving abstractive summarization quality by enhancing target-to-source attention.

a contrastive mechanism is taken, by encouraging the contribution from the conventional attention that attends to relevant parts of the source sentence, while at the same time penalizing the contribution from an opponent attention that attends to irrelevant or less relevant parts.


To our knowledge, we are the first to use contrastive attention for NLP and deploy it in the sequence-to-sequence framework.

we take Transformer (Vaswani et al., 2017) as the baseline summarization model, and enhance it with a proponent attention module and an opponent attention module.


We focus on the encoder-decoder attention, which builds the connection between source and target by informing the decoder which area of the source text should be attended to.

in the encoder-decoder attention, Q is the single vector coming from the current position of the decoder, K and V are the same sequence of vectors that are the outcomes of the encoder at all source positions.


Based on depicting heatmaps of all heads for a few sentences, we choose the head that corresponds well to the relevant relation between source and target to derive the opponent attention 1 .

let α c denote the conventional encoder-decoder attention weights of the head which is used for deriving the opponent attention: where q and k are from the head same to that of α c .


The opponent probability P o (y i |y i−1 1 , X) is computed by stacking several layers on top of attention o , and a softmin layer in the end as shown in the right part of Figure (1).

where W is the matrix of the linear projection sublayer.


Due to the contrary properties of softmax and softmin, jointly maximizing P c and P o actually maximizes the contribution from the conventional attention for summary word generation, while at the same time minimizes the contribution from the opponent attention 3 .

the training objective is to let the relevant part attended by attention c contribute more to the summarization, while let the irrelevant or less relevant parts attended by attention o contribute less.