Finally, the resulting packed label vector is used together with input word vectors as the hidden state vector for the current layer.

our model is named label attention network (LAN).


We instead propose a tractable upper bound by constructing a set O(z, θ) ⊇ F(z, θ).

note that whenever max O 0-1 (z, θ) = 0, this fact is sufficient to certify robustness to all perturbed examples B perturb (z).


Cross entropy: , where the sigmoid function σ is monotonic.

we can compute bounds on the loss O(z, θ) = [ final , u final ] from bounds on the logits.


As described above, we compute u final in a modular fashion: each layer has an accompanying function that computes bounds on its outputs given bounds on its inputs.

we can easily apply IBP to new architectures.


In both cases, we could not train a CNN to achieve more than 52.2% certified accuracy on the development set.

transforming pre-trained word vectors and applying interval bounds after is crucial for robust training.


In contrast, many realistic scenarios of learning from language would involve not learning from language alone, but learning from a mix of supervision, including both traditional labeled data, and natural language advice.

automated learners should be capable of learning from a blend of observations, explanations and clarification.


3 To explain, while useful from an information theoretic sense, a teacher's explanations may be too complex to handle for the learning agent's parser, in which case it might be preferable to stick to asking about the teacher about instance labels (which would require minimal parsing).

question-asking strategies need to be sensitive to the learner's own semantic parsing ability, which may also change during the course of interactions with users.


This could include factors such as the pedagogical phase in the learning process (exploratory vs confirmatory), previous questions asked, etc.

defining a rich enough state space is an important consideration for a formulation of conversational learning.


At each step in a simulated interaction, either (1) the label for a new example from this set is revealed by the teacher, (2) a new data measurement is provided as a natural language explanation by the teacher, or (3) the learner seeks a clarification about the parse of the previous explanation.

from the learner's perspective, over time the size of the unlabeled data decreases, and the labeled examples increase (see Equation 2).


Limitations Our evaluation setup in the discriminative training case is not ideal: the negative samples in both the train and test sets are artificially created by the same mechanism.

high performance on the test set in the discriminative case may result from "leakage" in which the model learns to rely on idiosyncrasies and artifcats of the data generation mechanism.


BasicKD provides individual learning goals for every sample and element of the target domain, using a scalar α to weight.

the source model has the same influence on all target samples, in which the diversity of domain relevance is neglected.


The intuition behind using different generation models is that each optimization stage can benefit from the new/unseen generated synthetic training data.


The relation-specific meta information is helpful in the following two perspectives: 1) transferring common relation information from observed triples to incomplete triples, 2) accelerating the learning process within one task by observing only a few instances.

we propose two kinds of relation-specific meta information: relation meta and gradient meta corresponding to afore mentioned two perspectives respectively.


In task T r , we firstly calculate the score for each entity pairs (h i , t i ) in support set S r as follows: where x represents the L2 norm of vector x.

we design the score function inspired by TransE (Bordes et al., 2013) which assumes the head entity embedding h, relation embedding r and tail entity embedding t for a true triple (h, r, t) satisfying h+r = t. the score function is defined according to the distance between h + r and t. Transferring to our few-show link prediction task, we replace the relation embedding r with relation meta R Tr as there is no direct general relation embeddings in our task and R Tr can be regarded as the relation embedding for current task T r .


is the score for negative sample (h i , t ′ i ) corresponding to current positive entity pair should be small for task T r which represents the model can properly encode truth values of triples.

gradients of parameters indicate how should the parameters be updated.


Thus gradients of parameters indicate how should the parameters be updated.

we regard the gradient of R Tr based on L(S r ) as gradient meta G Tr : Following the gradient update rule, we make a rapid update on relation meta as follows: where β indicates the step size of gradient meta when operating on relation meta.


For 5-shot, MetaR increases by 29.9%, 40.5%, 32.6% and 17.5% on MRR, Hits@10, Hits@5 and Hits@1 on NELL-One with average improvement of 30.13%.

for the first question we want to ex-  Table 5: Results of ablation study on Hits@10 of 1-shot link prediction in NELL-One.


As there are two kinds of relation-specific meta information in this paper, relation meta and gradient meta, we want to figure out how these two kinds of meta information contribute to the performance.

we conduct an ablation study with three settings.


Table 5 shows that removing gradient meta decreases 29.3% and 15% on two dataset settings, and further removing relation meta continuous decreases the performance with 55% and 72% compared to the standard results.

both relation meta and gradient meta contribute significantly and relation meta contributes more than gradient meta.


Even NELL-One has 37.1% one-shot entities, adding background knowledge into dataset increases the number of training tasks significantly, which complements the sparsity problem and contributes more to the task.

we conclude that both the sparsity of entities and number of tasks will affect performance of MetaR.


IRT models are usually fitted to RPs of hundreds or thousands of human subjects, who usually answer at most 100 questions.

the methods for fitting these models have not been scaled to huge data sets and large numbers of subjects (e.g.


Updating the generator with policy gradient always leads to an unstable training process because it's difficult for the generator to derive positive and stable reward signals from the discriminator even with careful pretraining (Che et al., 2017).

the generator gets lost due to the high variance of reward signals and the training process may finally collapse .


As mentioned above, MLE suffers from the exposure bias problem (Bengio et al., 2015;Ranzato et al., 2016).

reinforcement learning has been introduced to text generation tasks such as policy gradient (Ranzato et al., 2016) and actorcritic (Bahdanau et al., 2017).


L RAML,θ can be trained in a MLE-like fashion but sampling from the distribution Q φ (X) is intractable in the adversarial setting, because Q φ (X) varies with the discriminator D φ .

we introduce importance sampling to separate sampling process from D φ and obtain the final loss function: where P s (X) denotes a stationary distribution and To optimize this loss function, we first construct the fixed distribution P s (X) to get samples, and devise the proper reward function r φ (X) to train the generator in a stable and effective way.


We devise the reward function r φ (X) according to the discriminator's output D φ (X) and the stationary distribution P s (X): Intuitively, this reward function encourages the generator to generate sentences with large sampling probability and high rewards from the discriminator.

the weight of samples W φ (X) can be calculated as follows: So far, we can successfully optimize the generator's loss L G θ via Equation 6.


The motivation of introducing the latent variable z into the model is to model the uncertainty in the generative process.

it is preferable that z capture contextual interdependence between tokens in y.


In training, We expect this is possible when we have prior knowledge that Y 1 depends only on X 1 and Y 2 depends only on X 2 .

with chain rule, Since P (Y 1 |X 1 ) and P (Y 2 |X 2 ) are both high in training, if they generalize well, they should also be high in test, so that their product should be high in test.


Noise is helpful in training, but is not necessary in inference.

we set α as a hyper-parameter for training and α = 0 for inference.


While DBMs are able to correctly identify a small set of triples with relative features such as (skyscraper, apartment, tall) as discriminative, interpretation of relative relations requires two types of features which are not targeted by the models, namely: (i) the extraction of precise numerical reference points at scale (dealing with variations of dimensional units) and (ii) the ability to extrapolate the relations for unobserved lexemes by an explicit mechanism of comparative/transitive reasoning.

the current model is able to identify "skyscraper" as being taller than an "apartment", but fails to identify neither as taller than a "giraffe".


Even fine-tuning the pre-trained model with taskspecific dataset may take several hours to finish one epoch.

reducing computational costs for such models is crucial for their application in practice, where computational resources are limited.


Similarly, let θ s denote parameters to be learned for the student model, and denote the corresponding probability output from the student model.

the distance between the teacher's prediction and the student's prediction can be defined as: where c is a class label and C denotes the set of class labels.


Thus, the distance between the teacher's prediction and the student's prediction can be defined as: where c is a class label and C denotes the set of class labels.

besides encouraging the student model to imitate the teacher's behavior, we can also fine-tune the student model on target tasks, where taskspecific cross-entropy loss is included for model training: the final objective function for knowledge distillation can be formulated as: where α is the hyper-parameter that balances the importance of the cross-entropy loss and the distillation loss.


However, this approach is often hand-designed which requires human experts to define the form of each bivariate copula, and hence often results in overfitting.

gaussian copula (Xue- Kun Song, 2000;Frey et al., 2001) is often used since its multivariate expression has a simple form and hence does not suffer from the curse of dimensionality.


If the posterior collapses to the prior and has a factorized form, then the joint posterior likelihood will not be maximized since the joint posterior is unlikely factorized.

maximizing the joint posterior log-likelihood along with ELBO forces the model to generate readable texts while also considering the dependency structure of the true posterior distribution, which is never factorized.


In this case, the decoder is forced to generate texts from non-standard Gaussian latent codes.

the validation loss also decreases monotonically in general.


Algorithm Robustness Previous researches (Morimoto and Doya, 2005;Henderson et al., 2018) show that reinforcement learning-based methods usually lack robustness and are sensitive to the initialization, seeding datasets and pre-trained steps.

we design an experiment to detect whether the trained meta-leaner is sensitive to the initialization.


We did detect heads that paid increased (compared to the pre-trained BERT) attention to nouns and direct objects of the main predicates (on the MRPC, RTE and QQP tasks), and negation tokens (on the QNLI task), but the attention weights of such tokens were negligible compared to [CLS] and [SEP].

we believe that the striped attention maps generally come from BERT pretraining tasks rather than from task-specific linguistic reasoning.


Since BERT relies heavily on the learned attention weights, we define disabling a head as modifying the attention values of a head to be constant a = 1 L for every token in the input sentence, where L is the length of the sentence.

every token receives the same attention, effectively disabling the learned attention patterns while maintaining the information flow of the original model.


In addition to the above two ways of using Wikipedia data, the sentences in the weakly paired documents can be used as monolingual data to optimize the losses of the unsupervised machine translation.

our proposed losses can be combined with the loss functions of unsupervised machine translation.


In our unsupervised scenario, the most related initial model for selecting pairs is the unsupervised NMT model.

we first train an unsupervised NMT model followed Lample et al.


It receives reward R k at the end of the sequence, from which we subtract a state-dependent baseline R t k to reduce variance.

we maximize (R k − R t k ) log p(En t k |En <t k , Fr k ).


However, the first attempts at fully unsupervised CLWE induction failed exactly for these use cases, as shown by .

the follow-up work aimed to improve the robustness of unsupervised CLWE induction by introducing more robust self-learning procedures (Artetxe et al., 2018b;Kementchedjhieva et al., 2018).


However, this approach may lose information about relation types.

we re-gard relations and attributes of entities as bag-ofwords features to explicitly model these two aspects.


Note that we only consider the top-F most frequent relations and attributes to avoid data sparsity issues.

for each entity, both of its relation and attribute features are Fdimensional vectors.


This aligns well with our objective-bridging gaps between descriptions written in different languages.

we propose two methods for applying multilingual BERT, POINTWISEBERT and PAIRWISE-BERT, to help make alignment decisions.


In fact, the feature sets of the ILL pair already convey information that captures their similarity (e.g., the "starring" marked in blue are shared twice).

by directly using feedforward networks, HMAN is able to effectively capture such knowledge.


While the DPR dataset is split into train and test sets (Rahman and Ng, 2012), the WSC dataset contains a single (test) set of only 273 instances for evaluation purposes only.

we use the DPR dataset as training set when evaluating on the WSC dataset.


Natural logic is a flexible approach to doing logical inference directly on natural language expressions.

in this setting, we can work directly with natural language sentences while retaining complete control over all aspects of the generated dataset.


A fair training dataset exposes each local function to all possible inputs.

a fair training dataset for NLI will have the following properties.


However, these systems do not consider the purpose of those effects in the overall action sequence.

they are unaware if they predict effects that have no apparent purpose in the process, possibly indicating a prediction error (e.g., the erroneous predictions in red in Figure 1).


For example, if π t j = Create entity e and e is mentioned in a later step s k , then an edge will be added between s t and s k to G t .

score p = c > 0, resulting in a positive boost in the score for π t j .


This information (especially π ik ) is not present in any existing KB.

we train a model using a large collection of positive and negative examples of valid/invalid edges.


This is because the g edge score encourages the model to predict state changes that result in more action dependency links, often at the cost of a drop in precision.

the g edge and g kb scores together help XPAD discover meaningful action dependency links.


Ellipsis (15%): (Sometimes ungrammatical) elided references to earlier entities will hide the purpose of those entity's state changes, e.g., Water flows downwards... Enters the dam...

this analysis suggests several ways that the dependency graph computation could be improved.


In particular, we collected paraphrases for stories containing k = 1, 2, 3 supporting facts and then replaced the entities from these collected stories with placeholders in order to re-use them to generate longer semi-synthetic stories.

an example of a story generated by stitching together two shorter paraphrases is provided below: instead of simply collecting paraphrases for a fixed number of stories, we instead obtain a diverse at the time of submission, we have collected 6,016 unique paraphrases with an average of 19 paraphrases for every possible logical clause of length k = 1, 2, 3.


When generating stories, entity names are randomly drawn from a set of 300 common gendered English names.

depending on each run, the entities are never the same.


3 In other words, during training, the model sees all logical rules but does not see all combinations of these logical rules.

in addition to linguistic generalization, success on this task also requires logical generalization.


When we use segments to separate the context from the utterance we are trying to classify, we observe an average of a 7.4 point increase in OFFENSIVE-class F1.

it appears that the use of contextual information to identify OFFENSIVE language is critical to making these systems robust, and improving the model architecture to take account of this has large impact.


or sequence-based metrics (i.e., BLEU and EM).

we only integrate the GECOR model with the copy mechanism into the dialogue system.


sequential dialogue history and structure knowledge bases.

two weaknesses arise in such methods: (1) It is difficult to model different types of structured information in only one memory network.


The gating mechanism applied is adopted from Bidirectional GRU (Cho et al., 2014a) in our case.

the context-dependent representation of in-puts, denoted as where − → n k and ← − n k are the forward and backward representation of inputs, respectively.


While attention-based models are promising, they are insufficient to capture syntactical dependencies between context words and the aspect within a sentence.

the current attention mechanism may lead to a given aspect mistakenly attending to syntactically unrelated context words as descriptors (Limitation 1).


It is possibly because aspect words tend to co-occur with sentiment ones in both global and local context.

without supervision, it is likely our models discover them together.


Actually, not all the words contribute equally to the domaininvariant feature space though fine-grained adaptation is required.

we propose a novel Selective Adversarial Learning (SAL) method to dynamically learn an alignment weight for each word, where more important words can possess higher alignment weights to achieve a local semantic alignment based on adversarial training.


The correlation vector r L a,i will be fed to the GRL before the domain discriminator, which is used to predict the probability scores And meanwhile, the aspect attention weight α L a,i at the final hop serves as a selector to be a learnable alignment weight for each word.

the selective domain adversarial loss is a weighted crossentropy loss for all the words from the labeled source data D s and unlabeled target data D t : Existing studies (Yosinski et al., 2014;Mou et al., 2016) have already shown some evidence that lowlevel neural layer features (i.e., low-level task) are more easily transferred to different tasks or domains.


Thus, the selective domain adversarial loss is a weighted crossentropy loss for all the words from the labeled source data D s and unlabeled target data D t : Existing studies (Yosinski et al., 2014;Mou et al., 2016) have already shown some evidence that lowlevel neural layer features (i.e., low-level task) are more easily transferred to different tasks or domains.

we choose the r L a,i from the lowlevel AD task to be aligned instead of the feature h U i from the high-level ADS task to transfer.


However, we found the optimization process tends to be unstable since it may be hard to jointly optimize many objectives.

we propose an empirically alternating strategy to train the L M +ρL O and L D iteratively, which separates the whole word representation learning into a discriminative stage and a domain-invariant stage.


Note that for the unsupervised domain adaptation setting, no labels are available for the target domain.

for each transfer pair, its training dataset is the combination of the labeled training data of the source domain and the unlabeled training data of the target domain.


Absent of the proposed stacking architecture, all the baselines fail to accomplish the auxiliary AD task meantime.


that the original word hidden representations (LSTM B hidden states) are not suitable for transfer.

we need to resort to the correlation vectors inferred by the DMI that models the transferable latent relations between aspect and opinion words.


Instead, it is trained by optimizing the distillation objective, i.e., the cross entropy between the teacher's (soft) predictions and the student's predictions: While the teacher only uses the seed words in s i to form its prediction q i , the student uses all the words in s i .

using the distillation loss for training, the student learns to use both seed words and non-seed words to predict aspects.


By leveraging the seed words in a more direct way, Teacher is able to outperform the MATE-* models.

we can use Teacher's predictions as supervision for the student, as we describe next.


It is because negative feedback may happen for many unpredictable reasons, such as users being too busy to go online.

for conversation recommendation, we rely more on the positive feedback and design the weighted binary cross-entropy loss below: where T is a set of training instances.


It is possibly because Twitter exhibits more informal language style.

cR JTD, taken bag-of-words input, can better fit the data than cONVMF, taking word orders into account.


Other examples, an analysis of the US Presidential Election in 2016 (Allcott and Gentzkow, 2017) revealed that fake news was widely shared during the three months prior to the election with 30 million total Facebook shares of 115 known pro-Trump fake stories and 7.6 million of 41 known pro-Clinton fake stories.

automatically detecting fake news has attracted sig- Most existing methods devise deep neural networks to capture credibility features for fake news detection.


Though such difficulty is reflected in a number of bestpractice guides for science writers (Zweig and De-Voto, 2018), suggestions in those guides are generally based on the judgment of only a few individuals (Adams et al., 2017), and there lacks consistent teaching of strategies for thinking about probability and causality (Stanovich, 1992).

authors of observational studies sometimes wrongly extrapolate their results for clinical practice recommendations rather than suggest-ing follow-up randomized controlled trials (RCTs) (Prasad et al., 2013).


So, if (Ps i + b g ) t i is multiplied to the gate g i , only the information that is not contained in both s i and t i is allowed to pass through.

((Ps i + b g ) t i ) g i can be un-derstood as the information in s i that is complementary to t i .


To address this issue, for each edge e i,j ∈ E, we introduce negative sampling (Mikolov et al., 2013b) to simplify the optimization process.

the conditional distribution p(h i |h j ) into the following form: (10) where K is the number of negative samples, and is the distribution of vertices with d v representing the out-degree of vertex v. Adam (Kingma and Ba, 2014) is employed to optimize the entire model based on randomly mini-batch of edges in each step.


However, surveys often do not pose wellformed questions, reflect true voter opinion, or capture mass public opinion (Zaller et al., 1992;Bishop, 2004).

researchers often seek to compare survey findings with those of mass media as the relationship between public opinion and the media has been widely established (Baum and Potter, 2008;McCombs, 2018).


(Bielefeldt et al., 2011) They encourage to change what people think instead of merely changing what they do, so they advocate more speech that educates about cultural differences, diversity, and minorities as a better strategy to counter hate speech.

in order to encourage strategies of countering online hate speech, we propose a novel task of generative hate speech intervention and introduce two new datasets for this task.


Despite the large size of this dataset, the ratio of the hateful tweets are relatively low (5%).

the size of the hateful tweets is around 5k in this dataset, which is not significantly larger than that of the previous datasets.


Another result due to this difference is that the rewards with regard to ease of answering or information flow do not apply to this case, but the reward for semantic coherence does.

the reward of the agent is: where rw(c, r) is the reward with regard to the conversation c and its reference response r in the dataset.


As mentioned in Section 4, annotators tend to have strategies for intervention.

generating the common parts of the most popular strategies for all the testing input can lead to high scores of these automatic evaluation metrics.


The language in the dataset is hyper-local to the community we are studying and differs significantly from standard American English.


Domain Experts: In our domain, stakes are high and user behavior is frequently subject to biases and misinterpretations.

we ask domain experts rather than crowd workers to extend the annotation, and ask them to provide further insights about a domain that computer scientists are usually not at all familiar with.


Previous studies have observed that public stances towards rumorous messages are crucial signals to detect trending rumors (Qazvinian et al., 2011;Zhao et al., 2015) and indicate the veracity of them (Mendoza et al., 2010;Procter et al., 2013;Liu et al., 2015;Jin et al., 2016;Glenski et al., 2018).

stance classification towards rumors is viewed as an important preceding step of rumor veracity prediction, especially in the context of Twitter conversations (Zubiaga et al., 2016a).


As shown previously, they are two closely related tasks in which stance classification can provide indicative clues to facilitate veracity prediction.

these two tasks can be jointly learned to make better use of their interrelation.


Note that only a subset of this dataset has stance labels, and all conversations in this subset are already contained in SemEval-2017 task 8 dataset.

pHEME dataset is used to evaluate veracity prediction task.


As λ increases from 0.0 to 1.0, the performance of identifying f alse and unverif ied rumors generally gains.

when the supervision signal of stance classification becomes strong, the learned stance features can produce more accurate clues for predicting rumor veracity.


When labeled data is insufficient, existing methods may not work well as expected.

the deficiency in labeled data has raised a significant challenge to the person-job fit algorithms.


We assign each document to the top P topics with the largest probabilities.

the edge between a document and a topic is built if the document is assigned to the topic.


For example, as shown in Figure 1, the short text "the seed of Apple's Innovation: In an era when most technology..." is semantically enriched by the relations with the entities "Apple Inc." and "company", as well as the topic "technology".

it can be correctly classified into the category of "business" with high confidence.


The proposed Subnode model expresses each of the news node as a bag of subnode information of its associated features including semantic features such as event elements with high tf-idf scores, text structure features such as words count and paragraphs count, and side information such as published date, news type and emotion.

each news vertex e ∈ V e is represented as: where word count, paragraph count and sentiment are ordinal data rather than real values.


However, a system that is only aimed at selecting the simplest candidate may return a substitution that does not fit the surrounding context, is ungrammatical or not semantically equivalent to the original.

we consider all three aspects of high-quality substitution selection: contextual simplicity, semantic equivalence and grammaticality, which are outlined below.


ELMo provides deep contextualized word representations, which are learned from the internal states of a deep bidirectional language model.

these embeddings are able to model complex syntactic and Oak is strong and also gives shade g.c.


For example, a user may never or rarely cl news, and his/her news click data may be uninformative for representing this user.

modeling the informativeness of different views may benefit user representation learning.


This is because our approaches incorporate both word-and newslevel attention networks to simultaneously select important words and news for learning more informative news and user representations.

our approaches can outperform baseline methods.


This may be because users usually use post search queries to search engines to seek for interested content, and may browse several related webpages for detailed information.

search queries and browsed webpage titles contain rich information of user preferences, which is useful for learning accurate user representations.


In addition, search queries are usually very short with a few words, while webpage titles are often like complete sentences with much more words.

webpage titles are more informative for learning user representations.


This is because different words in the same news, query or webpage title may have different importance for learning user representations.

selecting important words can help learn more informative news, query and webpage representations.


This may be because search queries and webpage titles are important clues of user interests, which are very useful for news recommendation.

our approach can recommend news effectively to the users with very limited or even no news browsing histories, which can effectively improve their experiences.


For example, we can infer that this user is interested in the health topic from her queries and browsed webpages, which is not revealed by her clicked news.

incorporating heterogeneous behaviors of users are useful for learning more accurate user representations.


I will recommend it to others", the two neighboring sentences have close relatedness and they both express positive opinion towards the book.

we use a sentence-level CNN network to learn contextual sentence representations by capturing sentence-level local contexts.


For example, the sentence "The book tells a good story" is more informative than the sentence "I read this book today" in representing this book.

we use a sentence-level attention network to select important sentences to learn informative representations of reviews for recommendation.


1, the first review of Item-1 is more informative than its second review.

we use a review-level attention network to recognize informative reviews and attend to them.


Luckily, users bought the same items may have some relatedness, as well as the items bought by the same users.

we propose to indirectly mine the user-user and item-item relatedness by modeling the second-order interactions in the user-item graph, as shown in Fig.


This is because the first-order information of graph contains the interactions between users and items, and the second-order information can reveal the useruser and item-item relatedness.

more information can be incorporated as the depth increases, which can benefit user and item representation learning.


This is because different words in the same review have different importance in representing this review.

selecting important words via a word-level attention network can learn more informative sentence representations.


This is probably because our approach can mine the user-user and item-item relatedness by modeling the first-order and second-order interactions between users and items.

our RMG approach can learn better user and item representations.


Such commonsense knowledge is not explicitly expressed but can be found in a knowledge base such as Event2Mind (Rashkin et al., 2018) and ATOMIC (Sap et al., 2019).

we aim to incorporate the external commonsense knowledge, i.e., intent and sentiment, into the learning process to generate better event representations.


In other words, malicious attackers can destroy the models by adding a few inconspicuous perturbations into input data, such as masking images with unrecognizable filters and making low-key modifications for texts.

developing techniques to equip models against adversarial attacks becomes a prominent research problem.


First, images consist of continuous pixels while texts are discrete tokens.

a token can be replaced with another semantically similar token that drops the performance, so perturbations with natural looks cannot be easily recognized compared to previous approaches that capture unusual differences between the intensities of neighboring pixels.


However, removing words from sentences results in fractured sentences, causing the performance of NLP models to degrade.

it is essential to recover the removed tokens.


According to (Hirschauer, 2010), coverage and divergence should be considered for the acceptance decision of a paper.

we propose to utilize cross-review match to learn a comprehensive semantic representation from different reviewers.


The latent vector produced by Enc (a) is the representation on the latent space of style s a while the Dec (b) relies on the information and features from the latent space of style s b .

we introduce a projection function to project the latent vector from the latent space of s a to s b .


Our statistics show that the average length of ancient poems is 9 while 17 for modern Chinese sentences.

we set the minimum length of the ancient poem as 3, and the maximum length of the modern Chinese (news, open-domain conversations).


This result may be explained by the fact that the edit distance between formal and informal texts are smaller than between ancient poems and modern Chinese texts.

it is more challenging for model to preserve the content meaning when transferring between ancient poems and modern Chinese text.


The three GCNs encode different kinds of semantic correlations between entities, so that HMGCN could have a trade-off in decisionmaking.

the trained parameters W (i) have considered opinions from all three GCNs.


Online or stochastic algorithms (Hoffman et al., 2010(Hoffman et al., , 2013 achieve some scalability but at the cost of sacrificing the quality of topics.

the basic Latent Dirichlet Allocation (Blei et al., 2003) is still prevalent for practitioners despite various recent advances (Srivastava and Sutton, 2017;Xun et al., 2017;Xu et al., 2018).


For example, if f = Dir(α) It makes the marginal expectations Similarly we can represent up to the third moments: By assuming w ∼ Dir(α), we can fortunately attain closed form expressions of all three population moments only in terms of B and α, allowing the non-central second and third moments to be further represented by lower-order moments and α 0 (Anandkumar et al., 2012a).

once users construct the empirical moments given the training data and choose α 0 , tensor decomposition allows us to recover B and α up to some precision by matching the empirical moments to these population moments.


Instead of learning W , Anchor Word algorithms learn the correlations A by again leveraging the separability assumption: , which can be simplified to A = B −1 S * C SS B −1 S * .

the co-occurrence of the anchor words s k and s l transparently captures the correlation between the pair of topics k and l. Note that the anchor words are generally rare words -in order to be the vertices of an underlying convex hull of the word co-occurrence space -whose co-occurrences are even rarer and noisier.


10 While we report Coherence, the metric could be deceptive if a model learns many duplicated topics whose top words are mostly frequent words (Huang et al., 2016).

following the trends of Gibbs generally implies better performance.


To the best of our knowledge, no prior work focuses on jointly modeling the relationship between comments and edits.

in this paper we tackle two novel tasks.


Specifically, the number of negative samples is 4 times greater than the number of positive samples when the size of the candidate set is 5 -and even greater when it is 10.

the baseline classifiers tend to naively predict a negative label, which artificially boosts precision to the detriment of recall.


Also, characterlevel models are prone to errors due to long-range dependency (Sennrich, 2016).

these limitations arouse to consider alternatives to represent a query in a shorter sequence.


More recent works used LSTM as encoder which performed better in sequential problems, (Huang et al., 2015) used bidirectional LSTM as encoder, and the BiLSTM-CRF model achieved state-of-the-art on many datasets.

many researchers chose BiLSTM-CRF model as a baseline model when solving sequential problems.


Document-level Attention In a dialogue, the information provided by a single sentence is very limited and the same word may indicate different meanings due to the ambiguity.

we apply the document-level attention mechanism to make full use of the information in the whole dialogue to alleviate the ambiguity problem.


Both Document and Corpus-level Attention In order to integrate the information obtained based on document-level attention and corpuslevel attention, we concatenate h pi , H C pi and H D pi , and feed it into another Bi-LSTM model.

the final hidden state of each word contains the complementary information from both the dialogue and corpus.


The co-occurrence number of two entities is normalized by min-max normalization to obtain the edge weight.


Intuitively, the associations between symptom entities can help enhance symptom inference.

we first define a smoothness loss that quantitatively measures the entity associations in the constructed symptom graph: where y i is 0 or 1 that depends whether the entity has been recognized by the symptom recognition module.


This is necessary to accurately reason about the new consequences with minimal edits to the original sequence of events, instead of being confounded by spurious correlations (Woodward, 2002;Bottou, 2019).

a key measure of the task besides consistency is that the rewriting must perform minimal edits to the original story.


That is, s 2 states something contrary to that in s 2 , which in turn can make the original ending s 3:5 no longer valid.

the goal of the task is to rewrite the ending, such that the edited ending s 3:5 minimally modifies the original one and regains narrative coherency to the new counterfactual context.


We observe that the answer has strong semantic coherence to its question and post, which can be used to guide question generation.

we devise two methods to further enhance semantic coherence between post and question under the guidance of answer.


In particular, we note that people expect comments on news to be made by people.

there is a risk that people and organizations could use these techniques at scale to feign comments coming from people for purposes of political manipulation or persuasion.


In training, since salient spans are not explicitly available, we treat them as a latent variable, and jointly learn the two networks from article-comment pairs by optimizing a lower bound of the true objective through a Monte Carlo sampling method.

training errors in comment prediction can be back-propagated to span selection and used to supervise news reading comprehension.


Hence, we turn to maximizing a lower bound of Objective J which is defined as: Let Θ denote all parameters of our model and To calculate the gradient, we have to enumerate all possible S i s for (T i , B i ), which is intractable.

we employ a Monte Carlo sampling method to approximate ∂L i ∂Θ .


We can see that all variants suffer from performance drop and No Reading is the worst among the three variants.

we can conclude that (1) span prediction cannot be simply replaced by TF-IDF based keyword extraction, as the former is based on a deep comprehension of news articles and calibrated in the end-to-end learning process; (2) even with sophisticated representations, one cannot directly feed the entire article to the generation network, as comment generation is vulnerable to the noise in the article; and (3) pre-training is useful, but optimizing the lower bound of the true objective is still beneficial.


In the test set of Tencent data, the average lengths of salient spans with and without sampling are 11.6 and 2.6 respectively, and the two numbers in Yahoo data are 14.7 and 2.3 respectively.

deepCom can leverage discourse-level informa-  Table 6: Human score distributions tion rather than a few single words or bi-grams in comment generation, which demonstrates the advantage of our learning method.


Specifically, similar to (Srivastava and Sutton, 2017), we first draw a Gaussian random vector by a reparameterization trick and then pass it through a softmax function to parameterize the multinomial document topic distributions.

we can replace α with the parameters of Gaussian prior µ 0 , σ 2 0 .


As training proceeds, τ gets close to 0, yielding the increasingly peaked distribution that finally emulate the discrete cases.

the i-th word generated by the decoder is where W v ∈ R |V |×e denotes word embedding.


Inferring the objects from the vision side helps learn the object relationships, and inferring from the language side helps learn the crossmodality alignments.

we perform two sub-tasks: RoI-Feature Regression regresses the object RoI feature f j with L2 loss, and Detected-Label Classification learns the labels of masked objects with cross-entropy loss.


This is as if each label is predicted independently using a soft-label training regime, which is exactly the KL divergence loss proposed by .

our Soft-Label Chain CRF can be viewed as an extension of this soft-label discriminative model.


It also happens in the annotation of other coreference datasets (Chen et al., 2018).

we generate clusters from different pronouns in the same dialogue rather than merging antecedents for each pronoun separately.


In this work, we focus on jointly leveraging the contextual and visual information to resolve pronouns.

we formally define the visual-aware pronoun coreference resolution as follows: Given an image I, a dialogue record D which discusses the content in I, and an external mention set M, for any pronoun p that appears in D, the goal is to optimize the following objective: where F (•) is the overall scoring function of p refers to c given I and D. c and s denote the correct mention and the candidate mention, and C and S denote the correct mention set and the candidate mention set, respectively.


Similarly, given two mentions n i and n j , if they refer to the same detected object c k , then both b n i ,c k and b n j ,c k should be large.

we can use the maximum of their product among all K classes except "null" as the probability of n i and n j related to the same object in I.


There are mainly two reasons behind: (1) The visual information in our model is first converted into textual labels and then transformed into vector representation in the same way as the dialogue context.

the vector space of contextual and visual information is perfectly aligned.


Along with the increase of λ vis , our model puts more weight on the visual information.


-Cooking YouTube EPIC-KITCHENs (Damen et al., 2018)   tain at least one subtle but critical difference in action, tool or facial area.

it requires finegrained discrimination within temporal and spatial context.


For example, brow and brow bone are closely adjacent to each other and the under-eye area overlaps with cheeks.

fine-grained understanding is required to distinguish these areas.


As shown in Figure 6(b), the themes involve diverse aspects, such as "smokey" and "natural" for makeup style, "night" and "halloween" for occasions, and "pink" and "red" for color tone.

the theme inference task requires a fine-grained comprehensive understanding of the video content in various aspects.


Recently, a model combining both topdown and bottom-up attention reaches the winner of multiple challenges (Anderson et al., 2018).

how to combine the top-down and bottomup attentions effectively is still an unexplored problem in the vision-and-language tasks.


The motivation of this subnet design comes from that the prediction confidences from different frames should be different, e.g., it is more difficult for a frame near the start point to detect the end point than a frame near the end point.

we set the ground truth confidence of each frame based on its "centerness" in the segment.


We are able to identify label mistakes in about 5.38% test sentences, which is a significant ratio considering that the state-of-the-art test F 1 score is already around 93%.

we manually correct these label mistakes and form a cleaner test set.


These label mistakes bring up two challenges to NER: (1) mistakes in the test set can interfere the evaluation results and even lead to an inaccurate assessment of model performance; and (2) mistakes in the training set can hurt NER model training.

in this paper, we conduct empirical studies to understand these mistakes, correct the mistakes in the test set to form a cleaner benchmark, and develop a novel framework to handle the mistakes in the training set.


Compared to the results on the original test set, the re-evaluation results are more accurate and stable.

we believe this new test set can better reflect the performance of NER models.


This indicates that the corrected test set may be more discriminative.

we believe this corrected test set can better reflect the accuracy of NER algorithms in a stable way.


Then, we choose = 1 − p, because 1 − p represents the fraction of these detected label mistakes that might be still useful during the model training.

for the sentences that are marked as potentially mistake in that iteration, of them are actually correct.


As shown in Table 3, compared with the three algorithms, applying CrossWeigh always leads to a higher F 1 score and a comparable, sometimes even smaller, standard deviation.

it is clear that CrossWeigh can improve the performance of NER models.


In boosting algorithms, the training data is assumed to be perfect.

it trains models using the full training set and then increases the weights of training instances that fails the current model in the next round of learning.


Since we use the marginal probability for computing p entity , it already factors in the transition probability between tags.

any invalid sequences such as BP ERIORG have low scores.


For this ablation, we train a ETAL+PARTIAL-CRF where no transferred data is used.

to create the seed data, we randomly annotate 200 tokens in the target language and thereafter use ETAL.


Further, we set the ijk element of the tensor W ∈ R ne×nr×ne to 1 if the fact (e s , r, e o ) holds and -1 otherwise.

the product of the entity embeddings and the relation embedding with the core tensor, after applying the logistic sigmoid, accurately represents the original tensor.


In other words, the training and testing data of the worse CNN came from different distributions.

the macro-average F1 of the worse CNNs are 0.81 and 0.85 for the Amazon and the ArXiv datasets, respectively.


For this reason, the relative positions of the words in the sequence is very significant information to be considered in our task.

furthermore, due to the limited labeled data most neural-based models with a large number of parameters can not be trained well.effective methods for solv-ing these particular challenges need to be developed for this new scientific resource classification task.


To address this problem, the straight-through(ST) estimator (Bengio et al., 2013) is employed to estimate the gradient for the binary random variables, where the derivative of z i w.r.t φ is simply approximated .

the gradients can then be backpropagated through discrete variables.


However, inner product is a typical non-metric measure, which distinguishes MIPS from traditional ANNS problems.

methods designed for ANNS may have performance limitations in MIPS.


Therefore, in this paper, we propose a multiplex word embedding model, which can be easily extended according to various relations among words.

each word has a center embedding to represent its overall semantics, and several relational embeddings to represent its relational dependencies.


Second, the resulting embedding size is too large 1 , which is not appropriate either for storage or usage.

in this paper, we propose a multiplex word embedding (MWE) model, which can be easily extended to various relations between two words.


To deal with it, we propose to use a small dimension for relation-dependent embeddings and use a transformation matrix for each relation to project them into the same space of the center embeddings.

the two types of embeddings can be jointly trained and the quality of the relation-dependent ones are guaranteed.


Instead, we use its center embedding to transfer information among different relations and the sum of the center embedding and a local embedding to represent the final embedding for the corresponding relation 2 .

moreover, considering there are various relations among words, we use a lower dimension s for storage compression, with s < d. a transformation matrix X is introduced to transform u into the same vector space of c. As a result, the final embeddings v for head and tail of w under relation r are formulated as: (1) To train v r h,w and v r t,w for each w under r, we adopt the negative sampling strategy to conduct the learning process.


To balance the learning process between the overall semantics and the relation-specific information, we adopt an alternating optimization strategy 3 (Bezdek and Hathaway, 2003) to adjust λ based on different stage of the training instead of using a fixed weight for λ.

specifically, considering c is more reliable and possesses more information while u is learned with shared c, we alter-natively update c and u upon the convergence of c. we set λ to 1 in the first half of the training process and 0 afterwards.


As a result, pseudo-disambiguation only evaluates how good the model fits the corpus rather than evaluating the SP acquisition models based on ground truth.

in this section, we evaluate different SP acquisition methods with ground truth (human labeled datasets).


As a comparison, D-embeddings are trained on multiple prototypes for each word to record relation information, thus their size dramatically explodes with the increasing of the vocabulary.

it is easily computed for D-embeddings that 200,000 words will result in a 10GB model, which is unfeasible to be used in downstream tasks.


As a result of this bias, the embeddings may cause undesired consequences in the resulting models (Zhao et al., 2018a;Font and Costa-jussà, 2019).

extensive effort has been put toward analyzing and mitigating gender bias in word embeddings (Bolukbasi et al., 2016;Zhao et al., 2018b;Dev and Phillips, 2019;Ethayarajh et al., 2019).


x f and y f , since different languages can encode the same information differently due to structural divergences (Dorr, 1994).

if the parallel corpus contains m sentence pairs where x e occurs on the source side and y f on the target side, we extract a total of 2m paths.


The output of this affine transformation is normalized by softmax such that all its values sum to 1.

the predicted sense s i of word x i is formulated as where s i is a vector of predicted sense distribution for word x i , while W lexelt(x i ) and b lexelt(x i ) are respectively the projection matrix and bias vector specific to the lexical element (lexelt) of word x i , which consists of its lemma and optionally its partof-speech tag.


We do not test word2vec and GloVe on floats/negatives because they are out-of-vocabulary.

far, our synthetic experiments evaluate on held-out values within the same range as the training data (i.e., numerical interpolation).


Although AnsAcc is most convincing, it cannot cover the entire test set.

we apply two query level metrics: SymAcc detects whether all the SQL-related words are correctly involved in the predicted queries, for example column names, cell values and so on.


Unlike the second example, instead of using the pattern AwB, the first joint representation adopts the pattern BwA for the SUB expression node.

our model is able to work without the underlying knowledge on whether an operator is commutative or not.


In our setup, we did not make hard constraint that each prediction must contain one or two variables.

missing or redundant variables appearing in the predicted equations are one of the major error sources.


As the interaction proceeds, the user often makes reference to the relevant mentions in the history or omits previously conveyed information assuming it is known to the system.

in the context-dependent scenario, the contextual history is crucial to understand the follow-up questions from users, and the system often needs to reproduce partial sequences generated in previous turns.


In contextdependent semantic parsing, we consider an interaction I consisting of n utterance-query pairs in a sequence: At each turn t, the goal is to generate Y t given the current utterance X t and the interaction history Furthermore, in the cross-domain setting, each interaction is grounded to a different database.

the model is also given the schema of the current database as an input.


We even observe that to date the best reported results of some languages (Catalan and Japanese) are still from the initial CoNLL-2009 shared task (Hajič et al., 2009).

we launch this multilingual SRL study to fill the obvious gap ignored since a long time ago.


Considering that all arguments are predicate-specific instances, it has been generally observed that the distances between predicate and its arguments on syntactic tree are within a certain range for most languages.

we introduce language-specific rule based on syntactic dependency parses to prune some unlikely arguments, henceforth syntactic rule.


For example (in Figure 1), given the sentence Keep your heart and mind open, the predicate Keep and the corresponding syntactic dependency (bottom), by definition, (0, 1) is inside the syntactic rule on this occasion.

these candidate arguments (i.e., your, and, mind) will be pruned by the argument pruning layer.


It should be noted that the predicate sense annotation of Czech and Japanese is simply the lemmatized token of the predicate, a one-to-one predicate-sense mapping.

we ignore them and conduct experiments on other five languages.


As  point out, POS tag information is highly beneficial for English.

we conduct an ablation study on indomain test set to explore how the language features impact our model.


Nevertheless, syntactic contribution is overall limited for multilingual SRL in this work, due to strong syntax-agnostic baseline.

more effective methods to incorporate syntax into neural SRL model are worth exploring.


(2018) built a polyglot model (training one model on multiple languages) for multilingual SRL, but their results were far from satisfactory.

this work aims to complete the overall upgrade since CoNLL-2009 shared task and leaves polyglot training as our future work.


Due to the privacy problem, the table content is not allowed to be used on most text-to-SQL tasks (Zhong et al., 2017;Yu et al., 2018c).

we use the concatenation of the embedding of the tokens in the input utterance, the embedding of the separator and the embedding of the column names in the table as the input for BiLSTM or BERT.


However, directly computing expected reward requires traversing all the possible anonymous utterancex, which is unrealistic.

we leverage Monte Carlo estimate as the approximation of the expected reward.


As discussed in Section 4.2, the true posterior we can obtain is in the form of the unordered set.

we actually minimize the the expected reward, i.e., J ER (θ), instead of the KL divergence (see Section 4.3).


To encode this intuition, we propose another loss term L u (θ): where b (T ) j|• are the semantic role logits in T -th iteration.

the final loss L * (θ) is the combination of the two losses: where η is a discount coefficient.


It implies that adding uniqueness constraint on loss actually hurts the performance.

we use the loss L * (θ) with η equals to 0 in the rest of the experiments, which is equivalent to the loss L(θ) in Eq (16).


If the instructor provided boilerplate within the target cell, it is extracted and placed into the context above.

each prompt is paired with one solution, which forms our target cell.


Note that, both the lexicon and the grammar in our system are domain-agnostic in the sense that it is not specific to any database.

it can be reused for new domains in the future.


As introduced in Section 4.3, the agent raises a binary question to the user about a predicted SQL component o t .

the received user feed- back either confirms the prediction or negates it.


They have to convert the original dependency tree into a binary tree first and place the aspect target at the root node.

sentiment features can be propagated recursively from the leaf nodes to the root node.


For an aspect target with more than one word, we first replace the whole target word sequence with a special symbol " target ", then pass the modified sentence into the dependency parser.

there is a meta-node representing the target sequence in the dependency graph and its local feature vector is the average of embedding vectors of words in the target.


The model becomes more sensitive, which fluctuates greatly in loss and accuracy during training.

it is appropriate to limit the routing iteration number r and set it to be 2 or 3 depending on the performance.


In this study, we think that the posts in social media can be connected through the authors' backgrounds.

we propose a Neural Personal Discrimination (NPD) model to connect people and learn their emotions collectively.


This specific information is important because it is often the subject or object of the sentence and the bearer of sentiment.

it needs to be fully preserved in the process of sentiment modification.


Then, for the well-trained sentiment classifier, the average attention value of sentence x is calculated as:ᾱ Words with attention weight larger thanᾱ can be regarded as sentiment words, and the remaining words can be regarded as content words.

2 each sentence x can be separated into content words c and sentiment words s. 3 In previous work, there is an information gap at both ends of the decoder, making the decoder to bear both the target sentiment addition and content reconstruction simultaneously.


Another Seq2Seq model S2S θne is trained on the new negative corpus N − to generate negative sentiment words s − based on content words c − .

for the well-trained model S2S θpe (or S2S θne ), given content words c, S2S θpe (or S2S θne ) is able to generate corresponding positive (or negative) sentiment words s + (or s − ).


The reason is that there is no information gap at both ends of our decoder when generating sentiment-transferred sen- tences, which relieves the burden on the decoder.


Furthermore, the semantic similarity can reduce the risk of changing the label when modifying the input text.

it is a good choice to include this relation in models unless the replacement exactly impacts the original label in specific tasks.


As shown in Table 1, sentiment polarity may be different when different aspects are considered.

the given aspect (term) is crucial to ABSA tasks (Jiang et al., 2011;Ma et al., 2017;Xing et al., 2019;Liang et al., 2019).


For instance, this soft-attention mechanism tends to regard the name of a hotel "Hilton" with a good reputation in Clause3 as a positive word which could mislead the model into assigning a higher rating to aspect room.

a well-behaved approach should highlight sentiment-relevant words and discard noisy words for a specific aspect during model training.


The state s h i at i-th time-step should provide adequate information for deciding to se-lect a clause or not for ) which could be cast as a binary classification problem.

we adopt logistic function to define where h ∈ R} is the trainable parameter; ∼ denotes sampling operation; σ denotes sigmod function.


A careful selection of pairs to annotate is required since there are too many pairs for a comprehensive labeling, and sampling at random would rarely yield a pair such that the claim is mentioned in the sentence.

we limited annotation to claims which were labeled as mentioned explicitly (and assumed all iDebate claims to be so), and paired them only with sentences which are somewhat similar to them (based on word2vec, Mikolov et al., 2013).


Such a response makes several assumptions which are clearly violated here, such as the involvement of government.

further validation of rebuttals may benefit from first verifying their relevancy to the topic.


Different from previous work where models use complex architectures such as hierarchical LSTMs (Chen et al., 2016;Zhu and Yang, 2017) and external memory networks (Dou, 2017;Long et al., 2018), we aim to achieve improvements by only modifying how we represent and inject attributes.

we use a simple classifier as our base model, which consists of four parts explained briefly as follows.


This subsequently makes the weights difficult to optimize during training.

directly incorporating attributes into the weight matrix may cause harm in the performance of the model.


For example, in the emotion cause of Ex.1, the sentiment words cheered and happily express a positive polarity, also play a crucial role to provoke the emotion happiness.

capturing these sentiment words can enhance the causal connection between learned features and predictions.


Correspondingly, for non-sentiment words: Our training objective is to lead the model to pay more attention to sentiment words.

the regularization term is formally expressed as: where m is a hyper-parameter for margin.


As mentioned, annotating all pairs in a large collection of arguments is often not feasible.

we focused our attention on pairs that are presumably most valuable to train a learning algorithm.


When annotators fail a test question, they are alerted.

beyond monitoring annotator's quality, test questions also provide annotators feedback on task expectations.


As pairs of arguments with a high difference in individual quality scores appear to agree with argument-pair annotations, one may deduce the latter from the former.

it may be beneficial to dedicate the more expensive pair-wise annotation efforts to pairs in which the difference in individual quality scores is small, reminiscent of active learning (Settles, 2009).


In many cases, there are no labeled data for individual articles, but there are such labels for entire news outlets.

often all articles from the same news outlet get labeled the way that this outlet is labeled.


: "You must be a Republican or Democrat; you are not a Democrat.

you must be a Republican"; "There is no alternative to war."


: "It is a good idea to listen to victims of theft.

if the victims say to have the thief shot, then you should do it."


Table 5), and instances span from single tokens to full sentences or even longer pieces of text.

in our precision and recall versions, we give partial credit to imperfect matches at the character level, as in PD.


the model would not get any information from that example.

only examples strongly classified as negative in a lowergranularity task would be ignored in the highgranularity task.


Unlike softmax, which normalizes over all dimensions, the sigmoid allows each output component of layer L g k to be independent from the rest.

the output of the sigmoid for the positive class increases the degree of freedom by not affecting the negative class, and vice versa.


All-propaganda is a baseline which always output the propaganda class.

since the performance range of the tokenlevel classification is low, we think it is more effective to get additional information after aggressively removing negative samples by using ReLU as a gate in the model.


As discussed earlier, one of the main challenges for multi-modal information analysis is to exploit the interaction among the input modalities.

we introduce an Inter-modal Interactive Module (IIM) that aims to learn the interaction between any two modalities through an auto-encoder like structure.


We aggregate the errors (l 1 + l 2 ) at the encoded vector and backpropagate it to the input (text).

the weights in the encoder part will adjust according to the desired task-specific label as well.


To show that our proposed IIM module, indeed, learns the interaction among the distinct modalities, we also perform an ablation study of the proposed CIA architecture.

we omit the IIM module from our architecture and compute the self-attention on the pair-wise fully-connected representations for the prediction.


For each filter l ∈ R 1×d when applied on w i , outputs a new feature C il .

a new feature vector C i is obtained for each word w i after convolving with K filters.


However, in social media not all posts have to express arguments or stay on topic (Rakshit et al., 2017), and qualitative evaluation of the posts can be influenced by many other factors such as interactions between the individuals .

it is difficult to measure the effect of argumentative pragmatic context alone in argument quality prediction without the effect of these confounding factors using the datasets and models currently available in this line of research.


However, as stated in (Rakshit et al., 2017) many posts on social media platforms either do not express an argument, or diverge from the main topic of conversation.

it is difficult to measure the effect of pragmatic context in argument impact and persuasion, without confounding factors from using noisy social media data.


There are cases in the dataset where the same claim has different impact labels, when presented within a different context.

we claim that it is not sufficient to only study the linguistic characteristic of a claim to determine its impact, but it is also necessary to consider its context in determining the impact.


The local position within the dependency tree encodes dependency information of a word with respect to its neighbors.

the BiLSTM and the GCN allow embeddings for aspect words to have discriminative features, providing supervisory signals for the classification task.


First, we follow through the conventional procedure to extract a final embedding h s for a given sentence s using ASP-GCN.

we perform a subsequent run of ASP-GCN on the same sentence s to extract a final embedding, but in this instance we conceal a specific input word w. we conceal w by mapping it to the zero vector before applying ASP-GCN on s. a final embedding h (s\w) is generated for s. If h s = h (s\w) , the word w has no impact on the representation h s .


BLEU (Papineni et al., 2001), are extrinsic and heavily relies on the reference provided.

we ask a fundamental question: what benefits, which are more consistent across different DA methods and translation tasks, can DA in general obtain?


Hints from word alignments We observe that meaningful words in the source sentence are sometimes untranslated by the NART model, and the corresponding positions often suffer from ambiguous attention distributions.

we use the word alignment information from the ART model to help the training of the NART model.


In our experiments, a parser is trained on source language data and applied directly to all the target languages.

we propose to project the embeddings of target languages to the space of the source language, instead of the opposite direction.


Although this can be computed in CPUs within several minutes, more memories will be required with the growth of the amount of training data.

we present an approximate solution, where W is optimized with gradient decent (GD) and is not constrained to be orthogonal.


However, at the level of hundreds of thousands of data points, these patterns, if they do exist, become undeniable.

in this work, we introduce new datasets and methods so that we can study subtle gender associations in language at the large-scale.


Much research simply considers the top-n words from each cluster, but this method can be subjective and time-consuming to interpret.

there are efforts to design methods of automatic cluster labeling (Manning et al., 2008).


We know that these words have been clustered in domain-specific embedding space, which means that in the context of the domain, these words are very close semantically.

we choose S * that minimizes the total distance between its synsets.


Likewise, in the professor domain, we had guessed that women are associated with communication and men with knowledge, and there is a 100% female cluster labeled communication and a 89% male cluster labeled cognition.

cluster labeling proves to be very effective at  pulling out the patterns that we believed we saw at the word-level, but could not formally validate.


However, evaluating its coverage is inherently hard, as this would require exhaustively annotated corpora as gold standard.

assessing and optimizing coverage has typically been an afterthought at best, and is usually completely disregarded.


In most cases, RST parsers have been developed on the basis of supervised learning algorithms, which require a high quality annotated corpus of sufficient size.

research on RST parsing has focused on English, with the largest annotated corpus being the RST Discourse Treebank (RST-DT) (Carlson et al., 2001).


When we use the split score, the span consisting of only two atomic units can be divided into two uniquely, without calculating the split score for the span.

we compute V [b][e] = 1 when e − b < 2, instead of when b = e, in Equation (4).


We think the result was related to the right-heavy binary conversion of RST trees: RST trees with multi-nuclear relations were transformed into binary trees beforehand by applying the right-heavy branching procedure.

rB obtains better scores when the number of documents whose rST trees contain the multi-nuclear relations is larger.


We can obtain the information of sentences boundaries in most cases, however, sometimes we cannot obtain the information of paragraph boundaries.

it is significant that our method outperformed baselines on D2S2E settings.


However, the goal of this paper is not to achieve state-of-the-art performance on this task, but rather we are trying to understand the mechanisms of LSTMs.

we choose a relatively simple model (see Figure 1) !


For example, existing property norm datasets can be extended through human verification of features predicted with high confidence by Feature2Vec, with these features being added to the norms and subsequently incorporated into Feature2Vec in an iterative, semi-supervised manner (Kelly et al., 2012).

feature2Vec provides a useful heuristic to add interpretable feature-based information to these datasets for new words in a practical and efficient way.


For some domains, reasoning with domain knowledge is essential to SQA.

such questions often appear in exams like China's version of the SAT called Gaokao.


On real-world data, this ability would be much weaker as it would require additional competencies such as co-reference resolution, handling polysemy and ambiguities, and common-sense reasoning.

solving the tasks we propose can only be considered a prerequisite for a fully functional theory of mind which will ultimately have to be evaluated in real-world scenarios.


First, viewing passages of the same question as independent training instances may result in incomparable answer scores across passages.

globally normalizing scores over all passages of the same question (Clark and Gardner, 2018) may be helpful.


These instructions ensure that the rewrites only resolve conversation-dependent ambiguities.

we encourage workers to create minimal edits; in Section 5.2, we take advantage of this to use BLEU for evaluating model-generated rewrites.


However, 75.0% of the aggregate history has pronouns and the percentage of mixed category pronouns increase to 27.8% of our data.

pronoun disambiguation potentially becomes a problem for a quarter of the original data.


Although some efforts have been made to create RC datasets for Chinese (He et al., 2018;Shao et al., 2018) and Korean (Seungyoung Lim, 2018), it is not feasible to collect RC datasets for every language since annotation efforts to collect a new RC dataset are often far from * Equal contribution 0 All the modifications of existing corpora used in this paper would be released in https://github.com/ntu-spmllab/artificial-reading-comprehension-datasets trivial.

the setup of transfer learning, especially zero-shot learning, is of extraordinary importance.


Then We carried out one-way ANOVA test and found out the significance of the three factors are ranked as below: (1) > (2) >> (3  that the characteristics of training data is more important than translated into target language or not.

although translation degrades the performance, whether translating the corpus into the target language is not critical.


We want to condition the gate vector not only on a context vector and a single input vector but also on the entire sequence of inputs.

we design function f to be dependent on all the inputs in the sequence and the context vector.


partially parroting the input), state-of-the-art metric scores can still be surpassed.

for future paraphrase generation research which achieve good evaluation scores, we suggest investigating whether their methods or models act differently from simple parroting behavior.


Furthermore, the score deviation between different samples is small.

although the exact test sets used by (Gupta et al., 2018) and (Li et al., 2018) are not available, it is logical to assume that parroting performance would still exceed or be on par with the state-of-the-art on those test sets.


For "what church", the QA model detected "reformed church" as the answer while for "what was forbidden", the QA model correctly detected the target answer "catholic services".

the QAbased reranker is able to find the answer-relevant questions.


Formally, we define the probability of the model choosing the correct sequence S 1 over spurious sequences as the softmax over the lengthnormalized log probabilities of all plausible sequences: where T i refers to the length of the text sequence S i .

we can practically minimize the negative log-likelihood of Eq.


In this setup, after training a system with (real+synthetic) data, it is used to regenerate synthetic data for the final system.

the input data to the final system is a combination of real and regenerated synthetic data as shown in Figure 2.


Many forms of poetry make use of rhyming patterns on line-ending words (Oliveira, 2017).

to characterize a rhyming poem, a model needs (1) to know what it means to rhyme (2) to identify the specific permissible rhyming patterns for a particular poem type.


The discriminator's output is determined by a two layer 2D convolutional neural network applied to S. Convolutional neural networks have been shown to capture local as well as global patterns in 2D data -for example, images.

our discriminator is composed of two main components: computation of a ma-trix S, and a convolutional neural network to classify the computed matrix S. The pair-wise computation provides a useful inductive bias to identify the notion of rhyming, whereas the convolutional network is a suitable choice to capture overall rhyming patterns.


However, our hierarchical language model (RHYME-LM) performs worse than DEEP-SPEARE as per sampling efficiency.

structured discriminator is the driving factor behind the observed improvement with RHYME-GAN.


Since the beginning words which are not question words compose a large vocabulary while the number of each word is few, it is irrational to train a classifier to predict all types of these beginning words.

we reduce the question type vocabulary to only question words.


• Unlike ROUGE, our DSR does not rely on crossentropy loss (XENT) to produce readable phrases.


Background While sequence models are usually trained using XENT, they are typically evaluated at test time using discrete NLP metrics such as BLEU (Papineni et al., 2002), ROUGE (Lin, 2004), METEOR (Banerjee and Lavie, 2005).

they suffer from both the exposure bias and non-differentiable task metric issues.


However, RL models have poor sample efficiency and lead to very slow convergence rate.

rL methods usually start from a pretrained policy, which is established by optimizing XENT at each word generation step.


It also requires a model to generalize to unseen database schema as different databases are used for training and testing.

a model should understand not only the natural language question but also the schema of the corresponding database to predict the correct SQL query.


Moreover, the labeled data for short answer grading have question and reference answer pairs available but contextual information between the pairs and even just questions are easily ignored.

in this research, we aim to explore and evaluate various methods of updating the pre-trained BERT language model (LM) * on such domain-specific available resources in the context of ASAG to answer the following research questions.


We concatenate this single dimension with our concatenated resultant embeddings.

e w = e BioELM o,w ⊕ e DistM ult,w ⊕ sent w .


Accordingly, the attention weights corresponding to these words are decreased since the columns corresponding to these words have lighter color.

all words in mt other than "mit" and "dem" are copied to pe.


What's more insightful is that the revealed bias correlates well with census data, and its trends on historical texts matches women's movements in 1960-1970s (Garg et al., 2017.

word embeddings might be a reasonable tool in the study of gender stereotypes.


One advantage of graph-based models is that the derived bias scores are characterized by the intrinsic structure of the word association graph.

if we roughly assume the word association graph is an externalization of concept networks in human brains, as properties of the graph, the bias scores can be seen as a benchmark to help examining validities of other models.


Those clusters are connected by some hubs (i.e., high degree words), through which gender information is able to spread quickly.

words in a cluster usually have a same gender polarity.


In Section 5.4, we will further show that G enjoys some key small-word properties.

inherently, people may be "lazy" at carefully bounding gender biases.


In this method, we consider only local consistency (without the second term of Equation 1).

we find high consistency between their bias scores (Pearson's r = 0.789, p = 0.0).


(2016) has adopted these kind of approaches to study the propagation of words' sentiment.

it is natural to compare the graphs built from word embeddings with the word association graph here.


We pick top 2% words with largest node degrees as the representatives of hub words, and count the average occurrence of them in Wikipedia.

the hubs of word association graph (297868) appears much more frequently than hubs in word embedding graphs (12931 of G e local , 3373 of G e local ).


It turns out that most genderspecific words (except for he, she, daughter, aunt) in L are contained in the hub words of the word association graph, while none of them lie within those of the embedding-based graphs.

there is no wonder that L in the word association graph could distribute its gender information to other words in the graph more efficiently, leading to better performance in detecting gender bias of words.


From this set appropriate situation phrases can be "retrieved" depending on the given input.

finding out appropriate negative situations boils down to two sub-problems of (a) preparing a finite set of negative situations, and (b) setting up the negative situation retrieval process.


Our objective is not only to make the model generate aesthetic and terse poems, but also keep rich semantic of the original vernacular paragraph.

our model gives users more control power over the semantic of generated poems by carefully writing the vernacular paragraph.


By nature, classical poems are more concise and abstract while vernaculars are more detailed and lengthy, to express the same meaning, a vernacular paragraph usually contains more characters than a classical poem.

when summarizing a vernacular paragraph S to a poem T S , T S may not cover all information in S due to its length limit.


As demonstrated in Table 6, the last two sentences in both human poems (marked as red) echo each other well, while the sentences in machine- 3 We did not require the expert group's participation as many of them have known the gold poems already.


While the variation graph can provide comprehensive knowledge of Chinese character variations, efforts need to be made to address these two problems: (1) the variation patterns can be very flexible, and the compounded (long-range) variation information transfer may exist.

short-range (local) graph information, e.g., character vertex's neighbors, may be insufficient for spam detection.


A possible explanation was that, the review spam text usually had richer content and more complex variation patterns than SMS spam text.

a good variation representation model may have certain advantages.


Determining temporal relations (e.g., before or after) between events has been a challenging natural language understanding task, partly due to the difficulty to generate large amounts of high-quality training data.

neural approaches have not been widely used on it, or showed only moderate improvements.


In the beginning, we tried to directly use the output (i.e., a scalar) and the influence on performance was negligible.

here we discretize the CSE output, change it to categorical embeddings, concatenate them with the LSTM output, and then produce the confidence scores (Fig.


The discoveries that LSTMs readily improve the feature-based stateof-the-art CogCompTime on the MATRES and TCR datasets by a large margin not only give the community a strong baseline, but also indicate that the learning problem is probably better defined by MATRES and TCR.

we should move along that direction to collect more high-quality data, which can facilitate more advanced learning algorithms in the future.


Moreover, since manual annotation is very labor-intensive, existing approaches have to rely on distant supervision to train models (Ling and Weld, 2012;Ghaddar and Langlais, 2018).

the use of extra information to help with the classification process becomes very important.


Given a mention m and its context, the task is to predict a set of types τ ⊂ T suitable for this mention.

this is a multi-class, multi-label classification problem (Ling and Weld, 2012).


After mapping the Freebase types of Donald Trump to the target tag set, this sample will be weakly annotated as /person/politician, /person/tv personality, and /person/business, which is exactly the same as the type information (the "Types From KB" in Figure 1) obtained through EL.

during training, when the EL system links the mention to the correct entity, the model only needs to output the types in the KB type representation.


Due to our focus on individuals, it is no VA that should be part of our data set.

we can only determine precision and recall of our new approaches based on the baseline data set described in Section 3.


In addition, we leveraged crowdsourcing to check whether we could use (untrained) people to evaluate VA candidates.

workers on the crowdsourcing platform Figure Eight had to check whether a sentence contained a VA. 10 Crowdsourcing resulted in 600 judgments for 200 randomly selected candidates of our baseline data set with an inter-annotator agreement calculated by Cohen's Kappa of 0.72 between expert and workers.


This point is important: if a dataset contains idiosyncratic casing, then it is likely that NER or POS models have fit to that casing (especially with these two wildly popular datasets).

truecasing, since it can't recover these idiosyncrasies, is not likely to be the best plan.


We also observe that the scorer compares words all-to-all allowing multiple same words in an extraction to match a corresponding one in the gold.

simply repeating a word in the extraction will give it a high precision score.


However, existing research lacks empirical investigations on highly semantic aspects of elementary units (EUs), such as propositions for a persuasive online argument.

this paper focuses on a pilot study, revealing a persuasion strategy using EUs.


3 Each thread contains a triple of OP, positive, and negative.


We consider the token-level annotation (Stab and Gurevych, 2017), rather than the clause level one, to extract accurate minimal EU boundary and to remove irrelevant boundaries such as for example and therefore.


Meanwhile, the CNN performing P (ÿ = j|x, θ) estimates the noise transition matrix Q to deal with noisy labels.

we add a transition layer into this CNN.


Observe that even though target Hillary Clinton does not appear in this tweet, we can still infer that opinion holder is less likely to be in favor of Hillary Clinton.

identifying target related information is of vital importance to stance detection.


There are several variations of such selection mechanisms (Abney, 2007), and we rely on a balancing approach that considers the same number of instances for each class.

we select a subset {x s | s = 1, ..., K t } of the unlabeled data for each class, containing the top K t highest confidence items based on the current trained model.


Both sentences are neutral, which is not useful for learning sentiment flipping.

on Amazon, we further filter out the sentence pairs that have Jaccard similarities exceeding 0.8.


The mean of sentences extracted from the model trained with adversarial training is closest to the neutral value 5, but independent ttests show that the differences between all classes are insignificant (p > 0.05).

the results show that humans had difficulties determining the gender of the author.


a previous tweet in a thread (Zubiaga et al., 2016).

stance detection is an important task for analyzing discourse in online forums and social media platforms and is a component in assessing the veracity of claims (Kochkina et al., 2018).


To further improve performance, we designed PYTHIA's encoder to take an additional input stream of word embeddings, as it is difficult to model the word-level context using only character-level information.

we generated a list of the 100k most frequent words appearing in PHI-ML, and using a separate lookup table we concatenated at each time-step the embedding of each character with the embedding of the word it belongs to.


The ground-truths for incomplete epigraphic texts were lost over millennia.

in order to generate a ground-truth sequence, we artificially removed part of the input text and treated this as the ground-truth sequence.


First, the humor judgments differ from individual to individual.

collecting perceptually consistent human labels is expensive and time-consuming.


Second, fine-grained degrees of humor add a further challenge.

methods on small sample learning or even unsupervised rule-based methods merit investigation.


Massive news articles are generated everyday and it is impossible for users to read all news to find their interested content (Phelan et al., 2011).

personalized news recommendation is very important for online news platforms to target user interests and alleviate information overload (IJntema et al., 2010).


In addition, a news article may interact with multiple news articles browsed by the same user.

we propose to apply multi-head self-attention to enhance the representations of news by capturing their interactions.


Combining both additive and self-attention can further improve our approach.

these results validate the effectiveness of the attention mechanism in our approach.


Language variations exist among source and target languages and training on monolingual corpus might limit learning the variations.

we translate the source language, English, to the target language for the training corpus to reconstruct source language sentences and learn source and target languages jointly.


Although anagram generation is NP-hard, the basic algorithm that can output a set of words as anagrams is relatively simple (Jordan and Monteiro, 2003).

many anagram generation software and web services exist including the Internet Anagram Server 2 and the Anagram Artist 3 .


Given input string s, we start the search with the initial string t =" S " and then try to append a word w ∈ V to the tail of t for each step to search for anagrams of s. If we find that t yields no solution by appending any words to it, we pop back the last word from t and continue the search by appending a different word to the tail of t. Since there are |V | N possible sentences consisting of N words, naive DFS is intractable.


We use the fact that the sentence probability p(t) defined in (1) is monotone decreasing when we append new word w to the sentence tail.

once we have found K anagrams and their probabilities are larger than θ, we can omit unnecessary subsequent search steps if the current partial sentence t satisfies θ > p(t).


Due to the massive structure of BERT with a complex attention mechanism, it is difficult to interpret the significance of different lexical units in the text.

we propose a simple technique to interpret the contribution of each token in the text in determining the stance.


For the application described in this paper, false positive identifications of a class are worse than false negatives.

the classifiers are calibrated on the development set to a precision close to 0.90 and maximum recall.


(2016) tried to obtain data from machine-generated sentences.

there are situations that even expert humans cannot accurately classify.


We employ a reasonable model that is not trained on any human students at all, but only on text that a generic student is presumed to have read.

our model is not personalized, although it may be specialized to the domain of L1 text that it was initially trained on.


This threshold has the effect of only giving credit to an embedding of f such that the correct e is in the simulated student's top 4 guesses.

§3.5's machine teacher focuses on introducing L2 tokens whose meaning can be deduced rather accurately from their single context (together with any prior exposure to that L2 type).


When judging a free text, the context of CC i met in a sentence is respectively compared with the positive and negative sentiment classifying knowledge of the same CC i trained from corpus.

an evaluating function E (see section 3.5) is defined to evaluate the semantic orientation of the free text.


Generally speaking, empty words only have structural function without sentiment meaning.

we just consider substantives in context, which mainly include nouns/noun phrases, verbs, adjectives and adverbs.


our algorithm involves iterating through the data that the system is permitted to classify during each iteration.

there are a few negative training data in the bottom cluster, and the test stories were judged as NO but were evaluated as YES.


It is difficult to understand their meaning without any modification.

we calculate the percentage to what degree nouns are modified by any modifier in large corpora.


But that doesn't matter, since semantic dependency between the spurious head and modifier will be used to measure the meaning similarity of coordinated structures.

our selection preference model still works in coordination constructions.


By defining a bunsetsu like that, we can analyze a sentence in a similar way that is used when analyzing the grammatical role of words in inflecting languages like German.

strictly speaking, bunsetsu order rather than word order is free except the bunsetsu that contains the main verb of a sentence.


Furthermore, in the case of a Japanese corpus, we will need more time because we have to deal with word boundaries as well as POS tags.

it would be desirable to avoid or reduce POS annotations while minimizing the loss of performance of the parser.


In addition, we cannot identify easily content words and function words in a bunsetsu.

we here chose the simplest form of feature sets.


For example, some recent studies report that the statistical, innate structure of language plays an important role in children's language acquisition [11].

it is important to understand the innate structure of language, in order to shed light on how people actually acquire it.


Therefore, it is important to understand the innate structure of language, in order to shed light on how people actually acquire it.

this paper verifies assumption (A) in a fundamental manner.


The decrease in H(X|X n ) globally indicates that given an n-length sequence x n and another (n + 1)-length sequence y n+1 , the following inequality holds on average: One reason why inequality (2) holds for language data is that there is context in language, and y n+1 carries a longer context as compared with x n .

if we suppose that x n is the prefix of x n+1 , then it is very likely that holds, because the longer the preceding n-gram, the longer the same context.


Without any grounding on this assumption, Kempe [8] simply calculates the entropy of branching for a fixed length of 4-grams.

the length of n is set to 3, h(x i−3,i ) is calculated for all i, and the maximum values are claimed to indicate the word boundary.


We use the subscripts suc and prev to indicate the regular and reverse orders, respectively.

the regular order is denoted as h suc (x n ), while the reverse order is denoted by h prev (x n ).


As the process is slow and heavy, the experiment could not be run through millions of words.

we took out portions of the corpora used for each language, which consisted of around 2000 words (Chinese 2039, Japanese 2254).


More analytically, we tend to regard A as an attribute for objects of class C when A works as if function v = A(o), o ∈ C where v is necessary to us to identify o (especially to distinguish o from o ( = o) ∈ C).

obvious applications of attributes are ones such as summarization [1,2] and question-answering [3].


By definitions we used, attributes are what we want to know about the object.

if A is an attribute of objects of class C, we can arrange questions (consisting of A and C) that require the values for A as the answer.


These attributes whose parts are elided (e.g., seito representing seito suu) are also useful since they are actually used in sentences as in the above example.

they should be assessed as correct attributes in some way.


2, it is unfortunately ungrammatical since ikutu cannot be used for the number of persons.

seito representing seito suu will fail the QA test.


However, we think this attribute is also useful in practice for people who have a keen interest in foreign films.

the evaluators were asked whether the attribute was common for most instances of the class when the attribute was judged to be correct in the QA test.


It will be important to estimate how many attributes are needed for practical applications, e.g., by manually analyzing the use of pattern "C no A" exhaustively for a certain class, C. In addition, since we selected classes that were successfully obtained with a hyponymy acquisition method, we cannot deny the possibility that the proposed method has been evaluated for the classes for which reliable statistics can easily be obtained.

the evaluation of more difficult (e.g., more infrequent) classes will be an important future work.


First, personal and possessive pronouns are often references to other entities.

clauses containing such pronouns are discarded as ambiguous.


Wrong information is a central problem of developing a knowledge base by using natural language documents.

it is important to investigate a method of detecting and correcting wrong information in natural language documents.


Also, the answerer often asks the questioner back when the question is imperfect.

significant sentences in DA mails can be classified into two types: answer type and question type sentence.


A user can access the document from which (DA4-1) was extracted and obtain more detailed information.

the set of (Q4) and (DA4-1) was determined as correct.


In these cases, significant sentences in the question mails were successfully extracted and there is likelihood of matching user's question and the significant sentence extracted from question mails.

the precision of the significant sentence extraction was emphasized in this task.


When multiple rules match an input string at a given position, the longest-matching rule is selected.

these parsers always produce a single best analysis and operate very fast.


The parameters k λ and k µ play similar roles to the transition and emission probabilities in HMMs [12].

the most probable label sequence for input sequence X is Y* which maximizes a posterior probability.


We use a similar set of features to Toutanova and Manning [20], but our procedure differs from theirs in that it applies the stochastic model before unpacking, in a guided search through the generation forest.

we avoid enumerating all candidate realizations.


The second-best larger instantiation, in turn, will be obtained from moving to the second-best hypothesis for one of the elements in the (right-hand side of the) decomposition.

nested local optimizations result in a top-down, exact n-best search through the generation forest, and matching the 'depth' of local decompositions to the maximum required ME feature context effectively prevents exhaustive cross-multiplication of packed nodes.


Selective unpacking, in contrast, appears only mildly sensitive to input complexity and even on complex inputs adds no more than a minor cost to total generation time.

we obtain an overall observed run-time performance of our wide-coverage generator that is bounded (at least) polynomially.


It may not improve parsing performance [7].

a new approach of grammar binarization was proposed in this paper.