In the present work, we focus on incorporating a decision-making module akin to an action selection policy into a visually-grounded encoder-decoder architecture and leave the integration of other more advanced dialogue management aspects for future work.

work on incremental dialogue processing, where a system needs to decide not only what to respond but also when to act (Rieser and Schlangen, 2011), has some similarities with the problem we address in the present paper, namely, when to stop asking questions to guess a target.


As stated in the introduction, we believe that a good visual dialogue model should not only be measured in terms of its task success but also with respect to the quality of its dialogues.

unnatural repetitions and unnecessary questions should be avoided.


The utility of additional semantic information for the task of next utterance selection in an automated dialogue system is the focus of study in this paper.

we show that additional information available in the form of dialogue acts-when used along with context given in the form of dialogue history-improves the performance irrespective of the underlying model being generative or discriminative.


While MADP is proposed under the view of multi-agent reinforcement learning, it's can be interpreted as the special instance of our proposed graph-based policy with a fully connected graph structure.


This approach has the benefit that the new network can be easily combined with existing and future algorithms for RL.

this paper advances a new network, but uses already published algorithms.


A general unified framework is proposed to integrate multi-modalities seamlessly.

we begin by encoding single modal parts into dense vectors.


We notice that in the basic encoder-decoder model, what the decoder needs to generate a response is only a hidden vector.

for the same decoder, latent vectors directly decide what will be generated.


Note that the major-encoder produces only one latent vector when fed one post, and the major-decoder receives only one latent vector per decoding procedure.

the major-encoder is shared between path 1 and path 2, and the major-decoder is shared between the path 2 and path 3.


For each baseline, there is a corresponding version of MEMD, whose major-encoder and major-decoder are the same to the baseline's encoder and decoder respectively.

the baseline and its corresponding MEMD are structurally identical in test.


In the RNs, the representations of the source words produced by RNNs are taken as objects and the relationships between them are reasoned.

our method introduces a RN component between the encoder and the attention layer in the RNN encoder-decoder framework (Sutskever et al., 2014;Bahdanau et al., 2015).


RNNsearch This system is an improved version of RNNsearch where the decoder employs a conditional GRU layer with attention module, consisting of two GRUs and an attention module for each step 4 .

equation 4 is substituted with the following two equations: Besides, for the calculation of attention in equation 2, s j−1 is replaced withs j−1 .


However, as can be seen from the boldfaced sections marked in results generated with RNMT, proposed model with CNN could captures more source information successfully.

rNNs are skilled in modeling the order information of a sequence, while CNNs mainly focus on local features around some specific word.


(2017) further compare different methods for training a multi-domain system.

they compare concatenation that simply concatenates the multi-domain corpora, staking that iteratively trains the NMT system on each domain corpus, selection that selects a set of out-of-domain data which is close to the in-domain data, and ensemble that ensembles the multiple NMT models trained independently.


(2016) propose to append word-level features to the embedding layer of NMT to control the domains.


The results in Table 7 show that this hybrid method improves the accuracy further.

the improvements on Icelandic are around 5%.


For the incorrectly normalized changed spellings, the average edit distance is smaller than 2.

we just need less than two edits to translate an incorrectly normalized spelling into the correct one.


In addition, the size of the training set is crucial to NMT models.

transformer models are superior to RNN-based models only when provided with more training data.


To that end, we analyze the potential interactions between predictions with gold labels of all other predictions.

when training our models, we provide the classifier with the information which sequence of tokens is an experiencer (in the case of emotion phrase prediction) and which sequence of tokens is an emotion (in case of experiencer, cause, and target detection).


(2014) was seminal in the sense that it is arguably the first one employing prediction-based word embedding models to trace diachronic semantic shifts.

they used incremental updates (see below) and Continuous Skipgram with negative sampling (SGNS) (Mikolov et al., 2013a).


In a series of replication and control experiments, they demonstrated that some of the regularities observed in previous studies are largely artifacts of the models used and frequency effects.

they considered 10-year bins comprising equally sized yearly samples from Google Books 5-grams of English fiction for the period 1990-1999.


From these tables, we have the following overall observations: (1) What call for special attention is that, SATM exhibits the unstable performance on various datasets.

sATM performs poorly on May, the worst on July and the best on June.


(2015b) as our basic user embedding learning approach, of which we make use of the second-order proximity.

this approach is summarized as following.


In this paper, we propose a user embedding learning method to address cross-media user profiling.

we first propose a heterogeneous user network composed of two cross-media networks for learning user embedding and then employ the user embedding to train the model for classification (or regression).


In this work, we first use reward values defined by negative Hamming distance and then re-weight the reward based on entailment reward s(x, y, y * ).

given a sentence y * of length , we count the number of sentences within an edit distance d, where d ∈ {0, .


We observe that translation systems trained on 1,000,000 sentence pairs have better BLEU scores than the reference systems trained on 2,000,000 parallel sentences from Europarl, showing that parallel sentence extraction allows translation systems to 9 https://dumps.wikimedia.org/ 10 http://www.statmt.org/moses/giza/GIZA++.html 11 https://github.com/kpu/kenlm 12 https://github.com/OpenNMT/OpenNMT-py 13 We could have trained NMT systems for more epochs to obtain better BLEU scores, but our objective is not to compare SMT with NMT systems.

14 the two first systems for each approach and not from the total of 28 machine translation systems.


As a result, the burden of modeling reordering is shifted from the attention network to a dedicated reordering model.

we find that combining an Inversion Transduction Grammar based transition system with a RNN results a reordering model that is both fast and accurate.


In this work, we choose another bidirectional approach to process the sequence in order to learn more temporal dependencies.

an RNN layer processes the input sequence in a forward direction.


We extend the previous efforts on addressing the lack of large-scale mental health-related language data.

we propose improved data collection methods through which we can obtain highquality large-scale datasets of labeled diagnosed conditions paired with appropriate control users.


Our steps mirror the ones of Soldaini and Yom-Tov (2017), who were also interested in identifying self-diagnosed users, albeit on query logs.

we leveraged two synonym mappings to generate alternative formulations of the disorders of interest: • MedSyn (Yates and Goharian, 2013) is a laypeople-oriented synonym mapping ontology.


The SMHD dataset is available through a Data Usage Agreement (DUA) 7 protecting the users' privacy.

the DUA specifies that no attempt should be made to publish portions of the dataset (which could result in users being identified), contact users, identify them, or link them with other user information.


Recall that, as discussed in 3.3, users can be diagnosed with one or more conditions; as such, we also evaluate the baseline classifiers in a multi-class, multi-label setting.

given a user, classifiers are trained to predict if and which conditions the user was diagnosed with.


Our work raises the difficulty in building strong models.

we show that Quality Estimation models often behave differently in Quality Estimation feature space, depending on whether the scale of feature space is small, medium or large.


Given different training/test settings, it is also not clear which models produce more robust QE performance than others.

1 we are interested in both Standard and Domain Adaptation (DA) settings (i.e.


Meanwhile, our experiments show that the QE feature space is highly complex.

we show that while the feature space could be large, most of the features in the space are usually useful so that applying sparse QE models is often less effective than non-sparse QE models.


Increasing the number of units in the hidden layer larger could degrade the performance.

we show that training a large NN could be overfitting, and available QE datasets are not large enough to reduce the risk.


we believe utilizing a dataset from one specific distribution/domain to use for other distribution/domain is very useful.


Giving an infinite number of units plus a prior over weights and a Bayesian interpretation altogether turns Neural Networks to Gaussian Processes (GPs).

hTER scores Y are represented as a sample from a multivariate Gaussian distribution from input X as Y ∼ N (0, K(X, X).


Inference with a GP model is thus expensive, but this can be addressed by learning a small number M of pseudo data points and using them to train the model instead (Snelson and Ghahramani, 2006).

2 we can approximate the inverse of the covariance matrix with the inverse of the M ×M "pseudo matrix" instead.


One additional contribution of this work is to provide an understanding of this issue.

we introduce another kind of deeper NNs for QE, namely Deep Gaussian Processes (Damianou and Lawrence, 2013).


For this reason, the model can be trained in an end-to-end fashion with back-propagation.

the model can be trained end-to-end using stochastic variational approach (Wilson et al., 2016a) or semi-stochastic block-gradient approach (Al-Shedivat et al., 2017).


We introduce the model to QE because of their robustness.

we put network weights w with normal priors p(w) = N (0, I) in our experiments.


For large-scale QE feature space, we enrich the feature  set using unsupervised learning.

we trained a multiplicative LSTM (Krause et al., 2017) with 4, 096 units on a very large public English corpus (see McAuley et al.


Meanwhile, Table 1 reveals that the relationship between the space of input features and corresponding output is fully non-linear.

having a linear QE model provides a far suboptimal performance (e.g.


Table 2: Detailed analyses with different models in the Medium-scale setting.

using deep NNs (DNNs) does not help much.


However, we notice that having a larger NN may not improve performance, as in Table 2.

it helps only the case when the model is robust in training, as in GPs.


It is therefore unlikely to need a QE model that is capable of approximating non-stationary functions.

the relationship between the space of input features and corresponding output is unlikely to have jump discontinuities or isolated tall peaks.


As shown in Table 3, we found that DKL seems to be a right approach to address the difficulty.

replacing the simple SE kernel function with a NN improves the performance of GPs significantly.


7 We study different DA and KT settings.

we use EN-DE training/validating data come from WMT 2016, and evaluate on WMT 2017 EN-DE test data.


Second, we raise our concern of applying shallow NNs to these settings.

shallow NNs often predict an HTER score that is outside the range of [0:1] given an input that is not close to the input space of training data.


In this paper, we study fake news detection with different degrees of fakeness by integrating multiple sources.

we introduce approaches to combine information from multiple sources and to discriminate between different degrees of fakeness, and propose a Multi-source Multi-class Fake news Detection framework MMFD, which combines automated feature extraction, multi-source fusion and automated degrees of fakeness detection into a coherent and interpretable model.


In this paper, we study the problem of multi-class fake news detection with multiple sources.

we aim to answer two major research questions -(1) how to effectively combine information from multiple sources for fake news detection and (2) how to discriminate between degrees of fakeness mathematically.


We demonstrate the effectiveness of the approach by conducting a set of experiments.

we seek to answer the following research questions.


This function penalizes cross entropy by an intra-class sparsity.

it just tries to move samples toward their class centers.


Hyper-parameter value range notes number of BiLSTM layers {2, 3} layer connections {stitch, skip} stitch denotes learnable β parameters, while skip only uses the last BiLSTM layer as classification input subspace constraint weights {0.1, 0.2}  In our experiments, we consider four different non-literal language tasks (Table 3).

we include two metaphor detection tasks, with data being annotated on a token and on a construction level (i.e., grammatical constructions that share one label).


Comparing MERGED with MTL-all we see that the performance difference is mainly due to MTL-all introducing fewer new classification errors over the STL baseline.

: both metaphor tasks, where training (and evaluation) data for a particular token or construction is sparse, profit heavily from inclusion of the idiom datasets.


To capture this, we define the tie strength of each individual i as the highest value of her incident edges.

for all individuals j directly connected to i: Taking the maximum captures what we are after: A community member who only has weak connections will have low tie-strength and be considered a weak-tie user, while a member with either strong connections only or with both strong and weak connections will have high tie-strength and be considered a strong-tie user, who will be part of a local clique (Luce and Perry, 1949).


For each subreddit, we only consider terms that: • are used at least 10 times in the subreddit; • are not present during the first 3 months of the community's existence; and • are introduced within the initial quarter of the community lifespan.

we restrict our analysis to newly introduced Internet slang terms that are not present from the very beginning of the community's activity, but are not introduced too late so as to be able to observe their trajectory for a substantial period of time.


Our work opens a range of interesting questions.

we are looking forward to investigating how the patterns we have observed are affected by users' membership in multiple forums and how these multi-community users transfer new terms between communities.


We favor words that occur on the first position in multiple n-best lists.

for a given n-best list, we associate weights in a Borda sense (Borda, 1781) to all the words from the list: we give weight n to the word produced with the highest confidence, n -1 to the second one, and so on, until weight 1 to the n-th produced word.


We give more importance to the vote of the languages that obtained a better performance, multiplying the weight by the training accuracy.

for each n-best list L i , let π(i) be the training accuracy for language i (COV 10 ).


By contrast, the verb entry for colar ('to strain') contains no table at all, but simply provides a pointer to the inflection table for tostar.

it is presupposed that, for example, given the analogy tostar:tuesto::colar:x, the astute reader can solve for x without undue problems, though no explicit procedure for doing so is given.


common forms, in majority voting, we now exclude such paradigms from voting.

only include candidates which have some common MSDs with the target paradigm P .


The density f (t−s|θ st ) is estimated similarly to f (x s i |δ s ), which corresponds to an assumption that the translations t−s also follow a Gaussian distribution.

we estimate , where x s i is again the i th coordinate of s and similar for x t i .


Note that if the assumption that the considered relation corresponds to a translation is wrong, we can expect the variance of f to be similar, in which case the last factor in the evaluation of P (θ st |s, t) will be approximately 1.

the model implicitly takes into account how much the translation assumption appears to be satisfied.


The first baseline we consider is the 3CosAvg (or 3CA) method proposed in , which essentially treats the relation induction problem like an analogy completion problem, where we use the average translation vector across all pairs (s i , t i ) from the training data.

this method assigns the following score to the test pair (s, t): Another method proposed in , called LRCos (or LRC), is based on the assumption that (s, t) is likely correct if cos(s, t) is high and t is of the correct type, where a logistic regression classifier was trained on the target words {t 1 , ..., t n } to predict the probability that t is a valid 'target word'.


To allow for a more direct comparison with our methods, instead of using a logistic regression classifier, we will use our Bayesian estimation for the probability that s and t are of the correct type.

we use the score score LRC (t, s) defined as follows: As our final baseline, we train a linear SVM classifier using the training pairs (s 1 , t 1 ), ..., (s n , t n ) as positive examples.


We conduct a comparative evaluation against previous embeddings and find that our embeddings are highly useful for downstream tasks: across four classic sequence labeling tasks we consistently outperform the previous state-of-the-art.

we significantly outperform previous work on English and German named entity recognition (NER), allowing us to report new state-of-the-art F1-scores on the CONLL03 shared task.


We experimentally verify our approach in the classic sequence labeling tasks of named entity recognition for English and German, phrase chunking and part-of-speech tagging, and find that our approach reliably achieves state-of-the-art results.

for both German and English NER, our approach significantly improves the state-of-the-art.


To summarize, this paper proposes contextual string embeddings, a novel type of word embeddings based on character-level language modeling, and their use in a state-of-the-art sequence labeling architecture.

we • illustrate how we extract such representations from a character-level neural language model, and  integrate them into a simplified sequence labeling architecture; • present experiments in which we quantiatively evaluate the usefulness and inherent semantics of the proposed embeddings against previous embeddings and their stacked combinations in downstream tasks; • report a new state-of-the-art on the CONLL03 NER task for English (93.09 F1, ↑0.87 pp vs. previous best) and German (88.33 F1, ↑9.56 pp vs. previous best), and state-of-the-art scores for chunking and PoS; • release all code and pre-trained language models in a simple-to-use framework to the research community, to enable reproduction of these experiments and application of our proposed embeddings to other tasks.


This indicates that the inherent semantics of the proposed embeddings are meaningful enough as to require much less powerful learning architectures on top to perform downstream sequence labeling tasks.

for PoS tagging, the simple feedforward map is competitive to BiLSTM and much more effective to train.


We then decode the meta-embedding to reconstruct the original source embeddings asŝ 1 (w) andŝ 2 (w) using two decoders.

we reconstruct the two components of the meta-embedding m(w) in (1) independently to the corresponding source embeddings.


Yin and Schütze (2016) proposed the use of SVD to reduce the dimensionality of the meta-embeddings created from concatenation.

let N represents the number of words common to the vocabularies of CBOW and GloVe embeddings.


Different from the semantic relatedness datasets, the words in the SCWS have their contexts and partof-speech tags.

the human subjects can know the usage of the word when they rate the similarity.


Sometimes, the retro model increases the closeness to MEN's score.

genSense-all can handle all the possible situations well and outperform retro.


In variational Seq2Seq, however, the attention mechanism unfortunately serves as a "bypassing" mechanism.

the variational latent space does not need to learn much, as long as the attention mechanism itself is powerful enough to capture source information.


However, it relies on a reparametrization trick to propagate reconstruction error's gradient back to the recognition neural network (Kingma and Welling, 2014).

the sampling of latent variables should be drawn from a fixed distribution (without parameters) and then transformed to a desired sample using the distribution's parameters.


We use mean-field variational inference as an efficient approximation to full likelihood maximization by assuming a fully factorized approximate model: and then minimizing Kullback-Leibler divergence KL(q p) via coordinate ascent by iteratively updating the variational parameters ν.

q(θ | ν (δ) ) ∼ Dir(ν (δ) ) where where S(j) is the set of measurements provided by annotator j.


around 3 times more parameters are zeroed out given the same hidden state size d h .

the number of recurrent weights is 3 times larger in LSTM than in SCRN.


One issue is that it is unclear how human judgments could be used to rate the diversity of the generated descriptions, because diversity is a global property of the data.

: you cannot judge the diversity of a single description, because that is not what diversity is about.


Let D be a dataset that consists of triples including a posted question q, a headline candidate x, and a voted score y, i.e., (q, x, y) ∈ D. In the pairwise ranking method, we train a ranking model as a binary classifier that determines whether the condition y i > y j is true or false for two candidates x i and x j in the same question (q i = q j ).

we first define the index pairs of positive examples by P Note that we do not need to consider the negative examples N = {(j, i) | (i, j) ∈ P } since they yield the same formula as P in the optimization process.


The position feature is a binary vector of ten dimensions, where each dimension represents the coarse position (or coverage) of a headline candidate for a question.

we equally split a question (character sequence) into ten parts and set one to each dimension if and only if the corresponding part overlaps a candidate.


This means that our method detected 9.9% of the uninformative headlines and improved them with the high accuracy of 94.9%.

our method could successfully improve the overall performance while simultaneously avoiding any negative side effects.


Although the change rates are higher than our method, the winning rates are lower.


There have also been several classical studies based on nonneural approaches to headline generation (Woodsend et al., 2010;Alfonseca et al., 2013;Colmenares et al., 2015), but they basically addressed sentence compression after extracting important linguistic units such as phrases.

their methods can still yield erroneous output, although they would be more controllable than neural models.


We take two attributes of natural questions into consideration while training the generator.

two discriminators are proposed to distinguish samples from two pairs of counter-question-distributions, namely human written vs machine generated and natural vs descriptive.


Hence, the uncertainty in the dot product can be increased either by moving the density towards the origin or by increasing the variance.

the model has too many degrees of freedom.


In these experiments, given a pair of words, the system needs to predict if the first word entails the second one or the other way around.

it is known that entailment holds for the pair but its directionality needs to be predicted.


We argued that the posterior densities in our model encode semantic properties of a word given its context.

the posteriors can be used to disambiguate the word.


Our approach is informed by methods in cognitive neuroscience and psycholinguistics, where such controls are standard practice for studying the brain.

to ensure validity of our tests we introduce three mechanisms of control.


However, how to represent word in an effective way remains an open problem for diverse natural language processing tasks, including machine reading comprehension for different languages.

for a language like Chinese with a large set of characters (typically, thousands of), lots of which are semantically ambiguous, using either word-level or character-level embedding alone to build the word representations would not be accurate enough.


In this paper, we present various simple yet accurate subword-augmented embedding (SAW) strategies and propose SAW Reader as an instance.

we adopt subword information to enrich word embedding and survey different SAW operations to integrate word-level and subword-level embedding for a fine-grained representation.


In this paper we explore generalized pooling methods to enhance sentence embedding.

by extending scalar self-attention models such as those proposed in , we propose vectorbased multi-head attention, which includes the widely used max pooling, mean pooling, and scalar selfattention itself as special cases.


To help replicate our results, we publish our code 3 , which is developed from our codebase for multiple tasks (Chen et al., 2018;Chen et al., 2017a;Chen et al., 2016;.

we use Adam (Kingma and Ba, 2014) for optimization.


In this paper, we propose a generalized pooling method for sentence embedding through vector-based multi-head attention, which includes the widely used max pooling, mean pooling, and scalar selfattention as its special cases.

the proposed model aims to use vectors to enrich the expressiveness of attention mechanism and leverage proper penalty terms to reduce redundancy in multi-head attention.


The main aim of this work is sarcasm detection in online discussion forums.

we propose a hybrid network, named CASCADE, that leverages both the content and the context required for sarcasm detection.


For each S j u i , we provide the comment as an input to a pre-trained CNN which has been trained on a multi-label personality detection task.

the CNN is pre-trained on a benchmark corpus developed by Matthews and Gilliland (1999) which contains 2400 essays and is labeled with the Big-Five personality traits, i.e., Openness, Conscientiousness, Extraversion, Agreeableness, and Neuroticism (OCEAN).


• Main imbalanced: To emulate real-world scenarios where the sarcastic comments are typically fewer than non-sarcastic ones, we use an imbalanced version of the Main dataset.


We investigate the learnt user embeddings in more detail.

we plot random samples of users on a 2D-plane using t-SNE (Maaten and Hinton, 2008).


This paper attempts to marry the interpretability of statistical machine learning approaches with the more robust models of joke structure and joke semantics capable of being learned by neural models.

we explore the use of semantic relatedness features based on word associations, rather than the more common Word2Vec similarity, on a binary humour identification task and identify several factors that make word associations a better fit for humour.


This paper aims to marry the interpretability of statistical machine learning approaches with the more nuanced models of joke structure and joke semantics of neural approaches.

we explore the effectiveness modelling joke semantics using semantic relatedness features based on word associations, rather than the more common semantic similarity features based on word embeddings.


Furthermore, we introduce a novel method for targetting our semantic features using joke structure to help reduce noise and increase reliability.

we experiment with integrating the extraction of humour anchors, the "meaningful, complete, minimal set of word spans" (Yang et al., 2015) that allow humour to occur, into the humour classification process itself, the first work to do so.


In this paper, we propose a novel deep learning architecture for incorporating entities and their inherent attributes from various domains into a unified framework.

our model not only encodes the entities of reviewer, item, and review, but also their attributes such as location, date, price ranges.


In order to capture the relation between these attributes, we take their difference as a new feature.

we subtract Date (review) from YelpJoinDate (reviewer) as the new dateDif feature, and subtract Rating (review) from Average Rating (item) as the new ratingDif feature, and treat the new locDif feature as 1 when the reviewer and item have same location otherwise 0.


This is reasonable because they are not only the components of the entity itself, but also they are shared by two entities.

we find that the date (dateDif) feature plays a critical role in spam review detection.


To the best of our knowledge, we surpass previously reported F1 scores on this dataset.

our system significantly outperforms the Bi-LSTM-CNN-CRF models of (Chiu and Nichols, 2016) and (Strubell et al., 2017) by an absolute gain of 1.68 and 0.96 points respectively.


Chiu and Nichols (2016) note that the 4-class gazetteer they used yielded marginal improvements on ONTONOTES, contrarily to CONLL.

they observe that mentions that match LOC entries in their gazetteer often match GPE, NORP and FAC lists.


Additionally, we aim to further avoid noisy pseudo-labels by preserving local neighborhood structure of documents.

for each document we update its pseudo-label by also considering the information of its nearest neighboring documents.


Compared with these previous algorithms, our PL-DNB creates pseudo-labels by further considering the information of nearest neighboring documents.This can lead to more accurate pseudo-labels than only using seed words.

other dataless text classifiers aims to accurately categorize text documents by understanding the labels, i.e., embedding documents and category labels into a same semantic space (Chang et al., 2008;Palatucci et al., 2009;Elhoseiny et al., 2013;Song and Roth, 2014).


For example, the Micro-F1 and Macro-F1 scores of PL-DNB+S D are only about 0.01 and 0.025 lower than those of NB on Newsgroup.

pL-DNB may approach the supervised version of NB given sufficient seed words.


The main reason of this trend is that the number of documents with pseudo-labels decreases as δ becomes larger.

less labeled documents can be used for training naive Bayes (ref.


In contrast, music genre classification frequently relies on both audio and lyrics.

close to our work is Tsaptsinos (2017) who uses a hierarchical attention network (Yang et al., 2016), which, however, is using just the lyrics.


Attention models can provide insight into the decision making of the model.

they allow to analyze the weighing of parts of the input in the decision making process.


However, we also find that the mappings to lower dimensions do not fully support the claim of one single dimension of fluency.

in 3D-mapping, a more complex picture evolves.


A solution to this problem would be beneficial to all organizations which supply literary services, such as libraries, online bookstores and search engines.

approaches to solve the Genre Identification problem would provide decision making aids to library scientists, but could also help with recommendation systems and information retrieval.


Following the voting scheme method employed in the "All Chapters" technique, which yielded a high increase in performance, we digitized each chapter within their respective books into one of ten bins based on the chapter index relative to the number of chapters.

chapter c ∈ C was placed in bin b = c |C| * 10 .


In this reproduction paper, we replicate and extend several past studies on transfer learning for entity recognition.

we are interested in entity recognition problems where the class labels in the source and target domains are different.


Our results empirically demonstrate when each of the published approaches tends to do well.

simpler approaches often work best when there is very little labeled target data, while neural transfer approaches tend to do better when there is more labeled target data.


For example, we hope to run similar comparisons with multi-task and few-shot learning approaches.

we believe methods for few-shot learning in image classification could potentially be adapted to sequential classification problems such as entity recognition.


Similarly, "texas ave" is a valid and preferred bigram (over two unigrams).

as shown in Algorithm 1, we first tokenize all location names in the gazetteer to construct the n-gram model and then save the resulting lists of unigrams, bigrams, and trigrams (Lines 2-5).


The second line of solutions centres around learning with neural network models, which follows what we call an APA architecture or its variants.

the APA architecture includes three components: an entity alignment component, which identifies the subject entity implied in the question, a path label prediction component, which predicts a path label, and an object answering component, which finds the object entitie(s).


Additionally the KB K usually also prescribes a set T of entity types.

for each entity e ∈ E, its type t(e) is a token in T .


We will denote by q the question presented in natural language.


For each of the components, we can derive an appropriate loss function: the (entity) alignment loss al , the (path label) prediction loss pr , the (subject-path) verification loss ve , and the (object) answering loss an .

it is worth noting that to properly define the verification loss ve , we need to introduce "negative examples" for CLS.To that end, denote D qsr is the set of all training examples for path label prediction (ENC-DEC).


Each candidate entity is used as a hypothetic subject entity s, and passed to ENC-DEC together with q and K. In this step, a list of M r candidate path labels are generated.

these candidate path labels are the highest ranked M r path labels among all path labels r for which there is a path with label r in the KB that leaves from s. The ranking of the path labels is according to the learned prediction loss pr .


Since FB2M does not contain the entity type information, we retrieve this information from Freebase to annotate the type for each entity.

freebase contains two kinds of types, "common.topic.notable types" (or "notable type") and "type.object.type".


APVA-BASE is the APVA-TURBO model without full turbo training.

after training in the Base Phase, it only enters the Turbo Phase once and stops right after the training of CLS.


The above analysis demonstrates that our model is interpretable.

iRN has merits at: •Providing a traceable reasoning path for question answering.


Based on the experiment results, we make several observations: (1) Our method performs better than the other methods, and are proved to be highly reliable for the text classification task.

the ToWE-SG method significantly outperforms the other baselines on the 20NewsGroup, 5AbstractsGroup, and MR.


Experimental results demonstrate that our approach achieves competitive performance on several text classification tasks, including sentiment analysis, question type classification and subjectivity classification.

the accuracies are MR (82.1%), SST-5 (50.4%), TREC (96%) and SUBJ (93.9%).


In recent years, increasing works transfer CNNs to natural language processing tasks, such as representation learning (Liu et al., 2015), information retrieval (Shen et al., 2014), text classification (Kalchbrenner et al., 2014), etc.

feed-forward CNNs with word embedding have been proven to be a relatively simple yet powerful kind of models for text classification (Kim, 2014).


Competitors slightly outperform our models in IMDB, since its unique character that the sequences are much longer than other datasets.

lK-MTl surpasses the PA-MTl, which result demonstrates that the memory and forgetting mechanisms in leaky unit indeed have advantages over other sharing approaches, which help distinguish the helpfulness of the features and making a balance between features in current task and borrowed features.


Our results clearly indicate that both DNNs were able to generalize and obtain promising precision results, that are further improved when their scores are averaged.

combining the predictions of a DNN trained over prefixes of sentences enriched with claims, with those by a DNN trained over suffixes of such sentences, results in a pincer-movement like approach, that successfully pinpoints a wide range of CS in a massive unstructured corpus, while using only weak supervision for training.


Fermi did not believe that atomic bombs would deter nations from starting wars, nor did he think that the time was ripe for world government.

fission products do not themselves undergo fission, and therefore cannot be used for nuclear weapons.


In addition, the precision on sentences containing one or both of the weak signals is even higher.

the precision on the subset of sentences matching q strict is 0.42, a factor of two compared to the precision of (Levy et al., 2017) on this set of sentences.


Next, we consider the more complex case where the topic entity and the answer entity are not directly connected by a single predicate in DBpedia.

the results of our SPARQL query over DBpedia to enumerate the paths connecting the mapped entities contain multiple hops.


For simple baseline comparisons, we conduct experiments with methods which take bag-of-words as input feature representations.

we compute tf-idf weighted n-gram features for each document to train multinomial Naïve Bayes and Logistic Regression classifiers.


where rank i refers to the rank position of the first correct candidate answer for the i th question.

mRR is the average of the reciprocal ranks of results for the questions in Q.


For example in Figure 3, except that "皮鞋 (leather shoes)" have definite labels, all of the remaining characters can be labeled as "B-PDT", "I-PDT" and so on.

we denote a set of label sequences z for every distantly supervised sentence, whose probability is naturally the sum of probability of each possible label sequenceỹ in z.


where Θ is the set of all NE tagger parameters.

if the sentence is annotated by hand and each character has definite label, the set z includes only one label sequence.


We then use an attention model introduced by (Ganea and Hofmann, 2017) to compress the context into a single embedding.

we define the contextual representation The weight vector α is computed using the following attention considering all k entity candidates: where A is a learned matrix that scores the relatedness between word and entity.


We train W and c in a mention-wise procedure using cross entropy loss.

for each mention x i we use ground truth entities for the other mentions in E i and update weights of C and w such that the true candidate of x i gets higher score than its false candidates.


We train the pruning function by reducing it to a rank learning problem.

in training, we collect all the complete solutions that are considered in each pruning step, and compute their hamming losses from the ground truth.


We use different alias mappings for TAC and CoNLL.

for TAC we only use anchor-title alias mappings constructed from hyper-links in the Wikipedia.


In the last two rows of Table 3, we list variants of our model where LDS search is removed.

given the initial local solution, our discrepancy propagator (equation 5) can be applied without any discrepancy to re-evaluate the candidates for all mentions and obtain a more globally compatible solution.


For heuristic h 1 , we use a fixed depth strategy.

given a document with n mentions, we consider two different depth limits: τ = 25%n and τ = 50%n, which lead to an average depth of 5 and 10 per document respectively.


We also analyze the behavior of the coherent global models based on the rarity of the entities for the given mention.

we measure the rarity of e for given query mention m by p(e|m), as defined in 2.2.3, scaled to (0, 100).


In our proposed global model we address the intractable global optimization in a search framework.

we use Limited Discrepancy Search (Doppa et al., 2014).


Since the current state-of-the-art of NER is provided by neural models (Lample et al., 2016;Chiu and Nichols, 2016), we focus on them.

we implement a baseline model with a bidirectional LSTM (BLSTM) as it is effective and does not require a heavy feature engineering for the task.


The validation and the test sets are the same in both steps in terms of the sentence instances, but different in label annotation.

we create four datasets by (i) selecting one of the four categories as the new NE category to be recognized in the subsequent step; and (ii) substituting the labels of such category with O (not an entity) in D train S , D valid S and D test S .


This suggests that our progressive learning technique enables the models to more accurately predict the new categories and in a shorter time.

the transferred parameters not only help the model to reach a superior performance, but also provide a better initialization for the model to converge faster.


", a semantically similar question can be found in the existing QA collection of this product.

the first question is a good match with the new question.


Given a question q 0 associated with this product newly posed by a user, we retrieve semantically similar questions from the QA collection Q t via searching the question-answer pairs.

we utilize the Positional Language Model (PLM) (Lv and Zhai, 2009) for question retrieval and obtain a ranked list of retrieved existing questions denoted by L q * ,a * |q .


We expand the existing collection of question-answer pairs by including some relevant reviews to form the positive instance training set.

given a question q in the training set with its answer set A q , we add similar reviews as candidate answers into A q .


The snippets in the response are compposed of review sentences and an existing QA pair.

the first ranked snippet is a question-answer pair talking about transferring photos from a mobile phone to a Mac.


We hypothesise that for MRLs, languages where multiple meaning-bearing elements may be carried by a single space-delimited token, these choices will have measurable effects on task perfromance, and that these effects may vary for different architectural designs: fully-connected, convolutional or recurrent.

we hypothesize that morpheme-based representations will have advantages in terms of their generalization capacity and task accuracy, due to their better OOV coverage.


Deep learning (Goodfellow et al., 2016) has seen a surge of interest in recent years, transforming all application domains of machine learning.

neural network (NN) architectures currently dominate the development of models and applications in NLP, including syntactic parsing (Chen and Manning, 2014;Durrett and Klein, 2015), sequence labeling (Grave, 2008), information extraction dos Santos et al., 2015), text classification (Lai et al., 2015;Zhang et al., 2015) and sentiment analysis (Dos Santos and Gatti, 2014;Dong et al., 2014).


Here, we consider Modern Hebrew, a Semitic language with very different characteristics than English.

hebrew is a morphologically rich language (MRL), of which word structure is internally complex and word order is quite flexible.


As in English, CNNs show excellent performance for this task, providing state of the art results for Hebrew sentiment analysis, above 89% accuracy.

this is obtained in the token-based settings, where tokens are compatible with the "n-gram" views filtered through the convolution.


We argue that the scores assigned to each positive interpretation in the dataset reflect the confidence of the annotators rather than what we are interested in predicting.

in the end a binary classification system that can predict whether some positive meaning is implied in a negated statement or not would suffice in most real-world applications.


From these results, we learn that the challenge of this task is to make a system perform well for the minority classes of each role.

we need to know when the verb or ARGM-ADV results to TRUE and when the other roles result to FALSE.


We argue that this is because the sentence contains contrastive focus.

create contrasts with another verb mentioned in the previous context: enrich.


This is partially because, as Kawamura (2007) explains, negation does not require a focused element (in contrast to focus-sensitive elements such as only, even and either) and at the same time induces ambiguity in focus constructions.

negation without focus can still be grammatical; its absence simply produces a different interpretation.


In addition, in sentences without focus-sensitive elements, focus-shift does not affect the truth conditions (even though intuitively the interpretations are different), whereas the truth conditions of sentences with a focus-sensitive element such as only do change.

if John reads something other than books in the bathroom, Sentence 8a and 8b can both be TRUE, while the focus-shift causes Sentence 9a to be FALSE.


In order to solve this problem, we propose to integrate the supervised attention mechanisms into the hierarchical LSTM model that help to bias the attention scores toward the heuristically important words in the sentences (supervised attention) (Mi et al., 2016).

we rely on linguistic intuitions to heuristically select the informative context words for the problem of police killing detection.


Distant supervision is another element of IE that is employed in this research to generate training data for police killing detection.

distant supervision has been used to produce training data for relation extraction (Craven et al., 1999;Bunescu and Mooney, 2007;Mintz et al., 2009;Riedel et al., 2010;Surdeanu et al., 2012) and event extraction (Reschke et al., 2014).


In order to combine the hidden vectors h i,j,k , the attention mechanism computes the weighted sum of the hidden vectors to obtain a single representation vector for the input sentence s i,j (Bahdanau et al., 2015).

each hidden vector h i,j,k is given a weight α i,j,k to estimate its contribution for the final representation of the container c i for the problem of police killing recognition.


, x i,j,K ) at the word level to produce the sentence representation vector u i,j .


In order to further demonstrate the benefits of supervised attention for police killing detection, we consider two more baseline models when the model attentions at the word level are excluded from "H-LSTM+SemAtt" and "H-LSTM+SynAtt", resulting in the models "Mean+SemAtt" and "Mean+SynAtt".

in such models, the sentence representation vector u i,j for the sentence s i,j would not be obtained via the attention-based weighted sum in Equation 3.


We use the same procedure to split the original training data for development as those employed by (Keith et al., 2017) to generate the original dataset.

we first sort all the positive entities in the original training data using the descending order of their death times.


In this work, we answer the above questions by adapting all these methods to the same IE task, and performing a rigorous comparative analysis.

the contributions of this paper are the following: • We are the first to adapt the mean teacher (MT) framework (Tarvainen and Valpola, 2017) to a semi-supervised IE task, in particular named entity classification (NEC).


Lastly, the teacher/student approaches perform comparatively -each outperforms the other on one dataset -despite the fact that MT is a simpler framework than LN.

in the mean teacher framework both student and teacher are decoupled whereas in ladder networks there are connections between the two, which makes porting LNs to new tasks more difficult.


(2016) analyze the different architectural aspects of LN and note that the lateral connections and corresponding reconstruction costs (second term in the above cost function) are critical for semi-supervised learning.

it is important for unlabeled data to be used for regularization to be able to learn good abstractions in the different layers.


We explored two different architectures for the student model: Simple model (MT simple): In this model, we used bag-of-word averaging to obtain embeddings for entity mentions and the patterns matching them.

we: (a) average word embeddings to obtain a single embedding for the entity mention and each of its patterns; and (b) average the resulting pattern embeddings to produce the embedding of the corresponding context.


To create an automatically calculable proxy, we focus on automatically constructing large-scale datasets (stress tests), which test NLI models on phenomena that account for most errors in our analysis.

we generate adversarial examples which test "word overlap", "negation", "length mismatch", "antonyms", "spelling error" and "numerical reasoning".


Propositional Logic Framework: Assume a premise p and a hypothesis h. and if p and h are neutral, they remain neutral.

if the premise or hypothesis is in conjunction with a statement that is independently true in all worlds, the entailment relation is preserved.


Statistical models view this relationship probabilistically, addressing it in terms of whether a human would likely infer H from P. In this paper, we wish to bridge these two perspectives, by arguing for a visually-grounded version of the Textual Entailment task.

we ask whether models can perform better if, in addition to P and H, there is also an image (corresponding to the relevant "world" or "situation").


All our ROP systems are superior to the state-of-the-art.

they are superior to RNN-pathtype (Das et al., 2017), the state-of-the-art, even though we do not need and do not use information about the types of entities.


The results show that the proposed network improves performance on noisy data.

this paper addresses the following issues: 1.


As shown in Figure 2, we fed the words into the bidirectional LSTM in sequence.

where E i is the embedded word vector of the ith word, bi LST M is the bidirectional LSTM, and t → i and t ← i denote respectively the forward and backward contextual states of the given input word vector.


We used an 80-20 training-testing split that considers speaker independency.

there are 1755 utterances for training and 444 utterances for testing.


The result shows that our system achieves state-of-the-art on all three published datasets.

we achieved 76.2% accuracy and 74.8 weighted F1 score on CMU-MOSI, outperforming the previous methods by a margin   of 2.3% to 7.8%, which demonstrates the effectiveness of the proposed architecture.


This paper examines the hypothesis that lexical variation measures become unstable in learner English because of spelling errors.

it tests the hypothesis on English learner corpora of three groups (middle school, high school, and college students).


Apart from TTR, as far as we know, no one has yet revealed their influence on other lexical variation measures such as K. It would be especially interesting to see how they affect K which is stable with respect to text length.

in view of this background, in this paper, we explore the influence of spelling errors on TTR and K. we explore the following hypothesis: Hypothesis: Lexical variation measures become unstable in learner English because of spelling errors.


At the same time, the values of r are expected to be small for most words.

it is expected that a word is spelt correctly most of the time and only a few are misspelled 6 .


In this paper, we have addressed the influence of spelling errors on lexical variation measures.

we have explored the following hypothesis: Hypothesis: Lexical variation measures become unstable in learner English because of spelling errors.


In contrast, this is not the case with K, which shows no more than 1% difference throughout the three groups.

it is highly stable with respect to spelling errors.


While our findings suggest that it will likely hold for other learner Englishes and also for other languages, mother tongue interference or the nature of other languages might affect the results.

in certain languages (e.g., Japanese), it becomes much harder to identify spelling errors.


In this study, we do not integrate sentiment information into document representation directly, but use attention mechanism to measure the correlation between document representation and sentiment information.

we simply extract the sentimental word sequence We then learn the representation of sentiment information through this sentimental word sequence using LSTM model, and use H (sent) = LST M (x s m , h m−1 ) as sentiment representation.


We consider the impacts of different linguistic features when representing the document, and propose a novel hierarchical attention model for stance detection.

we employ a neural model to represent the documents and their linguistic features with attention mechanism.


A candidate to fill the blank can be selected according to how similar its vector is to the context vector.

given a context where the p-th position is the blank, the best candidate d * would be: where MLP is a non-linear projection that maps the context representation to a vector with dimensionality same as that of the target embeddings.


For each graph, we successively evaluate link prediction accuracy after retrofitting to links of other relation types.

for each relation type r 2 R, we retrofit to G \r = (V, E \r ) where E \r = {(i, j, r 0 ) : (i, j, r 0 ) 2 E, r 0 6 = r} is the set of edges with all relations of type r removed.


As shown in   the top retargets predicted by the linear retrofitting model are all medically plausible.

the model confidently predicts that Kenalog would treat contact dermatitis, an effect also found in a clinical trial (Usatine and Riojas, 2010).The second most confident prediction of drug retargets was that Kenalog can treat pemphigus, which is indicated on Kenalog's drug label, 6 but was not previously included in the knowledge graph.


The three gates are designed to capture long-range contextual information by using nonlinear summation units.

in this study, we follow the LSTM with forget gates and peephole connections to predict Mongolian phrase break.


The labels correspond to the actual stops in the utterance.

each word is labeled as "B" (means break) if there is a short break after the word.


For computing the deviation on a particular datapoint, we use the same mapping.

the deviation can only be 0 (agreement with Y or N) or 3 (disagreement).


deviation 0.32 0.32 0.32 0.32 0.45 0.36 0.37 0.41 0.36 Table 7: Deviation from average assigned value, all 5 values used (0-4) These deviations are (somewhat surprisingly) larger than the deviations computed after mapping the labels to the binary (Y/N) values only.

the deviation is smaller when the "unsure" labels (R Y , R N ) are mapped to Y and N, and the D value also to N, disregarding the difference between these two.


For the transfer leaning setting, we first train our model on noisy phrase pairs from the Paraphrase Database (PPDB) (Ganitkevitch et al., 2013), then evaluate it on every SemEval STS task from 2012 through 2015 and the 2014 SemEval SICK dataset (Marelli et al., 2014).

we follow the experiment setting of Wieting et al.


For simplicity, we use the minibatch for this set, but it could be a different set.

we choose t 1 for a given s 1 , s 2 as follows: where S b ∈ S is the current mini-batch.


That is, we choose t 1 for a given s 1 , s 2 as follows: where S b ∈ S is the current mini-batch.

we want to choose a negative example t i that is similar to s i according to the current model.


When continuing to train on this dataset, we initialize the model with the parameters learned from PPDB, use the hyperparameters that maximize Pearson's r on the 2016 STS tasks and try dropout (Srivastava et al., 2014)  Results: In Table 1, we show the results of our model when trained on PPDB XL only.

this set of results is the median of five runs with random initialisations of the learnable parameters.


Despite the auxiliary break level attribute embedding, both stages of the model have a tendency to learn ambiguous morpheme transformations for adjacent Eojeol.

even though the morpheme transformation and tags are correct, the Eojeol boundaries were incorrectly identified.


A popular operator for composing a relational embedding from two word embeddings is PairDiff, which is the vector difference (offset) of the word embeddings (Mikolov et al., 2013b;Levy and Goldberg, 2014;Vylomova et al., 2016;Bollegala et al., 2015b;Blacoe and Lapata, 2012).

given two words a and b represented by their word embeddings respectively a and b, the relation between a and b is given by a − b under the PairDiff operator.


A space of particular interest from a computational point-of-view is the bilinear operators that can be parametrised using tensors and matrices.

we consider operators that consider pairwise interactions between two word embeddings (second-order terms) and contributions from individual word embeddings towards their relational embedding (first-order terms).


The first term in (2) corresponds to the pairwise interactions between h and t. P, Q ∈ R d×d are the nonsingular 2 projection matrices involving first-order contributions respectively of h and t towards r. Let us consider the problem of learning the simplest bilinear functional form according to (2) from a given dataset of analogous word-pairs D + = {((h, t), (h , t ))}.

we would like to learn the parameters A, P and Q such that some distance (loss) between analogous word-pairs is minimised.


This is a linear transformation in the word embedding space and does not affect the topology of the embedding space.

translating word embeddings such that they have a zero mean has shown to improve performance in similarity tasks Mu et al.


For example, methods relying on topic models detect latent topics from tweets and use that semantic structure to guide the event detection task.

latent Dirichlet Allocation, aka lDA (Blei et al., 2003), and extensions have been widely used to model topics from large corpora.


In addition, it is clear that using either counts or oLDA ∞ t topic scores, bcp generates many more false positives than the other three methods.

on oLDA ∞ t none of the other three method generate a full positive that is not also produced by bcp.


Many Matching, to learn the matching measurement with one question sentence and multiple answer sentences.

we first adopt the BIMPM (Wang et al., 2017) to implement the matching measurement with one question sentence and one answer sentence, namely One vs. One Matching.


Furthermore, to deal with the second challenge, we introduce both the word-level and sentencelevel attention mechanisms.

the word-level attention is applied in the One vs. One Matching learning process while the sentence-level attention is applied in the One vs.


In previous studies for researching QA matching, there are mainly two corpora, namely TREC-QA and WikiQA.


The resultant vector s has the same length as the input word because of the zero-paddings.

to extract salient features, we apply a max-pooling operation on the vector s. we use max pooling with stride to derive locally salient features that can contain suffixes, roots, and prefixes.


We hypothesize that distinguishing such spurious candidates from STVs should be possible by relying on the morpho-syntactic variability profile.

e has good chances to be a morpho-syntactic variant of e if both are morpho-syntactically similar.


Therefore, we define the values of δ based on the similarity of document pairs using a bilingual dictionary.

for a document d 2 ∈ D ( 2 ) , the transfer distribution of d 2 , denoted as δ d 2 , is a normalized vector of size |D ( 1 ) |, i.e., the size of corpus D ( 1 ) .


Each cell in δ d 2 corresponds to a document d 1 ∈ D ( 1 ) , defined as: where {w } contains all the word types that appear in document d , and {w 1 } ∩ {w 2 } indicates all word pairs (w 1 , w 2 ) that can be found in a dictionary as translations.

(δ d 2 ) d 1 is the proportion of words in the document pair 2 ) that are translations of each other.


Therefore, we propose dynamic focusing, where we avoid choosing a specific focal threshold and selection scope.

we adjust the transfer distribution during inference dynamically, beginning with a dense transfer distribution and iteratively sharpening the distribution using deterministic annealing (Ueda and Nakano, 1994;Smith and Eisner, 2006;Paul and Dredze, 2015).


A robust multilingual topic model should produce similar distributions over topics for a pair of word translations c = (w 1 , w 2 ), where we call c a concept.

given a topic k, the probability of expressing a concept i in language 1 should be similar to language 2 .


We anneal all transfer distributions at iteration t only if LIS t−I:t > LIS t−2I:t−I .

if the average LIS score during iteration t − I and t has been increasing since iteration t − 2I to t − I, we treat this as a warning sign of increased LIS and thus anneal the transfer distributions.


In this article, we discuss which text, speech, and image technologies have been developed, and would be feasible to develop, for the approximately 60 Indigenous languages spoken in Canada.

we concentrate on technologies that may be feasible to develop for most or all of these languages, not just those that may be feasible for the few most-resourced of these.


To flesh out the ENDURANT portion of REO, we opted to integrate all daughters of Object and the classes just mentioned as daughters of our ENDURANT class.

this was done in a linking model, similar to the linking models that serve to map the generic event concepts in REO's 'Reference Model' to particular groups of events in the SRL resource models (see Brown et al.


In addition, we plan to augment ongoing research evaluating the utility of REO in scene understanding in images and video (Tahmoush and Bonial, 2016).

we will be exploring how the information relating ENDURANTS and PERDURANTS for a given event type might be leveraged in putting together the pieces relating objects recognized in an image to higher-order scene understanding and perhaps even activity recognition.


We opted to keep preprocessing at a minimum to ensure that any emergent linguistic or relational regularity is captured during the network construction stages.

we applied sentence segmentation and word tokenization using nltk 1 .


in factorization based methods for learning word embeddings (Turney and Pantel, 2010).

we express the strength of association between words w i and w j as follows: In our case, x ij is the number of times word w i appears near word w j , weighted by the nearness of their co-occurrences, and let I w i be the word positions in the corpus at which w i occurs, then we define: where n(p, q) = 0 if the word positions p and q belong to a different sentence, or if |p − q| > 10, i.e.


Our general strategy for learning relation vectors is based on averaging word vectors.

for each sentence s in which w i occurs before w j (within a distance of at most 10), we construct three vectors, based on the words a 1 , ..., a k which appear before w i , the words b 1 , ..., b l which appear in between w i and w j and the words c 1 , ..., c q which appear after w j : where we write v w for the vector representation of the word w. These vectors are then averaged over all sentences S ij where w i occurs before w j : Since we can similarly obtain such vectors from sentences where w j appears before w i , we end up with a relation vector whose dimensionality is six times higher than the dimensionality of the word embedding, which would be impractical in the kind of applications we envisage (see Section 4).


Because of this latter part, we can think of the norm of the relation vectors r w i w j as a measure of how strongly the words w i and w j are related.

if sentences mentioning w i and w j contain few or no words that describe their relationship, we might expect r w i w j to be close to the 0 vector.


Finally, the 'heavy-metal' case is a paramount example of how the the relation vectors may capture information which is fundamentally different than what is encoded by word vectors.

the diffvec vectors all express relationships from the metalwork domain (e.g., 'heavymetals' or 'annihilator-metal'), which reflects the fact that the music-related interpretation of the word 'metal' is not its dominant sense.


The main intuition is that when w 1 and w 2 are similar, they should also be related to similar words.

we first determine the closest match between the neighbors of w 1 and the neighbors of w 2 , as follows: Note that to identify these neighbors, we compare both their word vectors v n 1 and v n 2 , and their relationships to the target words, r w 1 n 1 and r w 2 n 2 .


The proposed searching system is based on the way people manually use FE dictionaries.

a user chooses a category that expresses the user's intent.


In our research, we use categories based on move-step structures for writing assistance systems.

we use the categorisation system that is adopted in Academic Phrasebank made by Morley (2018) because the categorisation of this resource is similar to move-step structures.


The other is a sentence with half or two thirds of the composing words removed.

we simply select words from every two or three words in a sentence.


For example, 'it' should be appended to be clear from the figure that and be important to investigate if and 'there' to be significant difference between and be some limitation to this study.

examining FEs extracted using the existing methods, it is apparent that there are many overly general expressions.


Despite the competitiveness of our system, the greedy algorithmic characteristic of the iterative subgraph-based algorithm has a negative effect on its performance.

some previously analyzed words can affect other words and it makes them mis-disambiguated.


For the future work, we plan to extend our WSD system to the multi-lingual system.

we are going to research to generate of multi-lingual word vector representation using the Babel synset information.


Inspired by their idea, we introduce a text sequence encoding model with dynamic routing mechanism.

we propose two kinds of dynamic routing policies.


The goal of dynamic routing is to encode the meaning of X into M fix-length vectors To transfer information from a variable number of representation H to a fixed number of vectors V , a key problem we need to slove is to properly design a routing policy of information transfer.

what and how much information is to be transferred from h i to v j .


Using a biaffine scorer, our model directly predicts all semantic role labels for all given word pairs in the sentence without relying on any syntactic parse information.

we augment the BiLSTM encoder with a non-linear transformation to further distinguish the predicate and the argument in a given sentence, and model the semantic role labeling process as a word pair classification task by employing the biaffine attentional mechanism.


In this section, we get insight into the proposed model, exploring how the deep BiLSTM encoder and the biaffine attention affect the labeling results respectively.

we present two groups of results on the CoNLL-2009 English test set.


Noting that the work (Zhao and Kit, 2008) and (Zhao et al., 2013) are similar to ours in modeling the dependency-based SRL tasks as word pair classification, and that they successfully incorporate the syntactic information by applying argument candidate pruning, we further perform empirical study to explore whether employing such pruning method enhance or hinder our model.

we use the automatically predicted parse with moderate performance provided by CoNLL-2009 shared task, with the LAS score about 86%.


Thus, we further augment our model with the predicate identification ability.

we first attach all the words in the sentence to the virtual root <VR> and label the word which is not a predicate with the None role label.


Instead of including thousands of feature values into an author template, we find the most frequent and then within them the most informative feature values of each feature type.

we may include some character trigrams by first selecting the most frequent ones and then within those the most informative ones as measured by their informational entropy across all the authors.


We can observe the similar behavior as in the closed author set case.

grammar based features gave better performance in news article and blogs datasets and character trigrams gave higher performance in Reuter 50 50 dataset.


We show that the model outperforms the existing RNN models with respect to discovering 'open-vocabulary' slots without any external information, such as a named entity database or knowledge base.

we confirm that it performs better with a greater number of slots in a dataset, including unknown words, by evaluating the models on a dataset of several domains.


As a result, a multitude of studies regarding recurrent neural network (RNN) models for slot filling and intent detection has been completed and has resulted in outcomes that generally outperform most other past approaches.

there have been various attempts to improve the performance of RNNs for spoken dialogue systems such as label sampling (Liu and Lane, 2015), the hybrid type of Elman and Jordan (Mesnil et al,.


The proposed model also outperformed the existing RNN models for both tasks on the dataset of five domains.

we demonstrate that our model has outstanding performance on 'openvocabulary' slots and its efficacy is increased as more data in a domain contain 'open-vocabulary' slots including unknown words.


The intuition behind equation 8is that each pass in an upper bGRU layer provides more global information than a lower one and updates the states of each word time step to redeem correlations between distant words with the help of sentence-level information in the opposite direction of lower bGRU layer.

we stack two layers of the bGRU and treat them as a pair of short-term and long-term meanings.


For the ATIS corpus, Table 3 shows that our models for both 'Joint' and 'Slot only' achieved comparable results to the state-of-the-art result of slot filling.

our 'Slot only' model provided improvement from 95.78 to 95.93 in comparison to the previous best 'Slot only' model.


Not to our surprise, adding ideas improved the performance and our model demonstrated the best performance for both slot filling and intent detection.

as the difficulty of domains associated with 'open-vocabulary' slots increased, like calendar and message, a larger gap in performance emerged to the amount of about a 5% absolute gain for slot filling and about a 1% error rate for intent detection.


While the majority of such techniques require the allowed process behavior to be specified in formal models, such as process models or business rules, recent advances of NLP in the context of BPM overcome this restriction.

a technique (van der Aa et al., 2018) has been developed that can perform compliance checks on the basis of textual process documentation.


This paper analyzes different linguistic aspects that may help represent style.

we study the role of syntax and lexical words (nouns, verbs, adjectives and adverbs) in representing style.


Natural language processing (NLP) techniques have been applied to extract information from company filings (Lee et al., 2014), financial news articles (Xie et al., 2013) and social media texts (Bollen et al., 2011) in order to gain understandings of financial markets.

traditional methods exploited statistical signals, such as lexical and syntactic features (Wang and Hua, 2014;Schumaker and Chen, 2009), sentiment (Bollen et al., 2011) and event structures (Ding et al., 2014;Ding et al., 2015;Ding et al., 2016) from these text sources, which suffers from feature sparsity problems.


To address the above challenges, we build a neural model that selects and represents relevant sentences from a full news document with respect to a specific firm of interest.

we leverage conditional encoding (Rocktäschel et al., 2015) to encode information of a given stock into the dense representation of the news abstract.


For example, we can easily calculate the semantic relatedness between two words 'dog' and 'cat' using cosine distance of their continuous representations.

a representation learning method aims at mapping a linguistic unit with k words S = [w 1 , w 2 , ..., w k ] into a continuous vector in a low dimensional feature space.


To effectively learn the representations of large linguistic units, a promising approach is word composition, which is based on the Frege's principle of composition: (Pelletier, 2001) -the meaning of a complex expression is determined by the meanings of their constituent expressions and the rules used to combine them.

given a linguistic unit S with [w 1 , w 2 , ..., w k ] words, a composition method first infers the vectors of all words in S based on embedding matrix.


The proposed model attempts to address the word sense disambiguation and word composition in a unified architecture.

inspired by TWE , this paper utilizes topic distribution as the global context of a linguistic unit.


Our method employs topic distribution of a linguistic unit as its contextual information.

given a linguistic unit S = [w 1 , w 2 , ..., w n ], its topic distribution C = {p(topic 1 |S), p(topic 2 |S), ...} and the topic-specific embeddings of all words, we construct the representation of S as follows: 1) We calculate the topic-biased representation of S under topic t by composing the topic-specific word embeddings as follows: where f is a base composition model, which composes a sequence of vectors into a single representation.


We train the LDA model and the topic-specific word embeddings on the British National Corpus (BNC) (Consortium and others, 2007), which contains more than 93 million terms, and a bigger corpus -a snapshot of the English Wikipedia corpus 1 with about 990 million tokens.

we use GibbsLDA++ (Phan and Nguyen, 2007) to estimate the topic distribution and infer the topic assignment for each word in the corpus.


In some tasks that investigate the meaning of words independently of contexts, e.g., building WordNet (Miller, 1995) or calculating word-word similarity, the same word X can only be assigned to an unique label 0 or 1 for each Y.

y is function of X, that is, for each X, P (y |X) equals 0 or 1.


Despite the simplicity of the proposed model and the very small number of example words used to bootstrap it, the neural activation semantic model performs surprisingly well compared to state-of-the-art word embeddings.

the neural activation semantic model performs better than the state-of-the-art for the task of semantic similarity estimation between very similar or very dissimilar words, while performing well on other tasks such as entailment and word categorization.


This mental process, which is not currently completely understood, has captivated the interest of both neuroscientists and computational linguists (Haxby et al., 2001;Ishai et al., 1999;Cree and McRae, 2003;G. Kanwisher et al., 1997;Just et al., 2010).

when a person experiences a visual stimulus of a concept, reads, speaks or writes a word, particular neuronal regions in the brain are activated (Pulvermller, 2001).


Thus, employing ISR comparison with human performance is only possible on few data sets.

we are not aware of any pair of data sets with significantly overlapping BE5 ratings.


Comparing our system performance against human SHR (based on 20 participants per study; see Section 3.4), again our approach seems to be highly reliable (color coding of Table 4b).

eRM using the FFNN model outperforms SHR in over half of the applicable cases (25 of 38).


We also show that pairwise MTL can inform us what tasks can benefit others or what tasks can be benefited if they are learned jointly.

we identify tasks that can always benefit others as well as tasks that can always be harmed by others.


We note that our definition of "task" is not the same as "domain" or "dataset."

we differentiate between tasks based on whether or not they share the label space of tags.


They have an encoder e with parameters θ that encodes a sequence of word tokens into a sequence of vectors and a decoder d with parameters φ that decodes the sequence of vectors into a sequence of predicted tagsŷ 1:L .

c 1: The model parameters are learned by minimizing some loss function L(ŷ 1:L , y 1:L ) over θ and φ.


To equip our model with the ability to perform one-to-many mapping (i.e., multiple tasks), we augment the model with "task embeddings."

we additionally learn  a function f (t) that maps a task ID t to a vector.


When pairwise MTL does not improve upon STL, most of the time All MTL will not improve upon STL, either -with one exception: COM.

the pairwise MTL performances of UPOS, XPOS, NER and FRAME (TE⊕DEC) are mostly negative and so are their All  Table 5: F1 scores for TE⊕ENC.


However, the performance of All MTL when tested on SEM almost achieves the lowest.

sEM is harmed more than it is benefited but pairwise MTL performances cannot tell.


Zhang and Yang (2015) argue that evaluating performance, model selection and parameter tuning have distinct requirements in terms of the bias and variance of our estimators.

as long as bias is approximately constant across different models/parameters, it will have little effect on the selected model.


', and the next line of dialogue is 'on the floor', this corpus is annotated in such a way that could support recovery of semantic roles across sentences in a dialogue.

the first sentence is annotated with 'fall' as the main verb.


In this paper, we propose a multi-layer representation fusion (MLRF) approach to fusing stacked layers.

we design three fusion functions to learn a better representation from the stack.


• We investigate methods to learn functions that fuse different levels of sentence representations.

we propose to learn a 2D representation by the self-attention mechanism.


We propose a position-related edit distance (PRED) method , which ignores deletions at the end of word b.

here, a is a Uyghur word and b is a donor word, ED is the traditional edit distance algorithm.


In this paper, we propose adaptive weighting for WSMs to control the contribution of each input.

we apply adaptive weighting for both GRU and the output state in NMT.


Unlike the conventional weights that are independent of each other, adaptive weights are learned to be dependent on each other and more importantly be able to dynamically select the amount of input information.

we use gate mechanism to incorporate adaptive weights in GRU and in computing the output states.


Our goal is to enable a WSM dynamically selects the amount of input information when there exist two or more inputs.

we apply the above idea into the attention-based seq2seq model from two perspectives: (1) adaptive weighting for GRU; and (2) adaptive weighting for the output state o t as in Figure 1.


The TiGer corpus is preprocessed before grammar induction.

punctuation tokens are attached to lower nodes in order to reduce the number of discontinuous brackets using disco-dop.


In this work, we investigate the possibility to combine the two models into a unique framework and develop efficient architecture to leverage both advantages.

we propose Double Path Networks for Sequence to Sequence learning (DPN-S2S), which contain a convolutional path and a self-attention path with attention information fusion between the encoder and the decoder.


We then explore several remedies and modifications to propose tractable and efficient NCE training strategies.

we propose to make the scaling factor a trainable parameter of the model, and to use the noise distribution to initialize the output bias.


Let us explore the repartition of the partition function values during the last epoch of training.


Still, as we will show in this paper, the actual evaluation settings used by various research groups exhibit differences which can in some cases have dramatic consequences for the reported results.

all available research on frame semantic parsing relies on some form of pre-and post-processing of FN XML data.


For each parser, we re-implement lexicon creation methods in order to include only data seen during training (see §3.1).

we reproduce the predicate-frame map of SIMPLEFRAMEID, the frame-to-frame-element map of SEMAFOR and OPEN-SESAME, and the hierarchy feature-related relation maps of SEMAFOR.


Overall, KL-BEAM achieves 66.1 UAS on average, which corresponds to what could have been achieved by BEAM with 32 sentences.

if more than 32 target sentences are available, on average one should prefer monolingual parsing (at least when using BEAM) over KL-BEAM transfer: 15 the amount of knowledge transferred across languages seems very limited.


Recurrent neural networks (RNNs) (Elman, 1990) is commonly used to build block for seq2seq model.

our model employs the Long Short-Term Memory (LSTM) (Hochreiter and Schmidhuber, 1997) variant of RNN, which uses numbers of gates to address the problem of vanishing or exploding gradient when trained with back-propagation through time.


To prevent deterioration of the translation performance for long source-language sentences, the attention-based NMT tries to capture the relation between the states in the encoder and the decoder, and the decoder generates the translations by referring to the history of the hidden states in the encoder.

the alignment score a i is calculated on the basis of the similarity between the decoder's hidden stateh i and each of the encoder's hidden states c = (h 1 , h 2 , .


Sennrich and Haddow (2016) have improved the attention-based NMT model by using the linguistic features of source-language sentences.

the encoder receives the lemma, subword tags, POS tags, and dependency labels of source-language sentences in addition to source-language words.


To quantitatively confirm the pure effectiveness of chunk information of NE tags, we evaluated the performance of the proposed model that uses only chunk information of NE tags (Proposed IO).

proposed IO receives either of 'I' tag or 'O' tag as the NE tag.


Results on Figure 3(a) show that our model generates less repetitive translation.

the proportion of duplicates of our model is less than half of that of the conventional Seq2Seq model.


In this paper, we propose Knowledge-aware Attentive Network (KAN), a transfer learning framework for crossdomain answer selection, which uses the knowledge base as a bridge to enable knowledge transfer from the source domain to the target domains.

we design a knowledge module to integrate the knowledge-based representational learning into answer selection models.


Thus, we proposed Knowledge-aware Attentive Network (KAN) for transfer learning on answer selection task.

we design a context-guided attentive convolutional neural network, which incorporates knowledge embeddings into sentence representations, to strengthen the representation learning of documents.


In this paper, we describe a semantic parsing approach to the problem of KB QA.

for each input question, we construct an explicit structural semantic parse (semantic graph), as in Figure 1.


In this paper, we address this gap and investigate ways to encode the structure of a semantic parse and to improve the performance for more complex questions.

we adapt Gated Graph Neural Networks (GGNNs), described in Li et al.


The q-node is always present and denotes the answer to the question.

all entities from the KB that can take its place so that all relations and constraints hold, constitute the answer to the question.


For this reason, we believe that annotation schemes for natural language generation tasks that are aimed at evaluating language quality need to be treated with great care.

an unchecked focus on reduction of disagreement among annotators runs the danger of creating generation goals that reward output that is more distant from, rather than closer to, natural human-like language.


Indeed, it can happen that a question, although ungrammatical, is perfectly understandable 12 .

it can be possible to work out what the question is asking, even if it is ungrammatical.


To generate the final document encodingd S , an additional modification of u S−1 is introduced before applying Equation 8.

an additional Question Evidence Common Word Feature (qe-comm) (Li et al., 2016) is introduced for each document token, indicating whether the token is present in the query.


Overall Figure 2 suggests that DGR achieves highly competitive performance across different document and query lengths in comparison to the other variations including the GA reader.

dGR perform better or similarly to the GA reader ("dGR -(a) & (b) & (c)") in all categories except when query length is between 30 and 40 where GA reader wins with a small margin.


A generic pattern observed in our study is that GA reader tends to generate more uniform attention distributions while DGR produces more focused attention.

each layer of DGR tends to focus on different sub-parts and examine different hypotheses, illustrating the significant impact of the proposed strategies on the attention mechanism.


In this process, we summarize key results, highlight limitations and propose some open research challenges.

we detail the different types of evidence used by different approaches and how they affect the NLP requirements on the systems developed.


While fact checking and verification are often used interchangeably, recent work has sought to define them as two distinct but complementary processes (Silverman, 2016).

verification is defined as "scientific-like approach of getting the fact and also the right facts" (Kovach and Rosenstiel, 2014), which often involves verifying the source, date and the location of materials.


This is probably because the proposed method encourage the model to utilize feature vectors corresponding to users' posts.

stances are represented not only by vectors of topics but also by vectors of other variables.


Our experiments suggest important differences in fake news content as compared to legitimate news content.

we observe that classifiers relying on the semantic information encoded in the LIWC lexicon show consistently good performance across domains.


Likewise, the results in Figure 2b show noticeable differences among legitimate and fake content on the celebrity domain.

legitimate news in tabloid and entertainment magazines seem to use more first person pronouns, talk about time (Relativity,Time, FocusPast), and use positive emotion words (posemo), which interestingly were also found as markers of truth-tellers in previous work on deception detection (Pérez-Rosas and Mihalcea, 2014).


Thus, the goal of this paper is to evaluate the performance of the state of the art in OIE systems on scientific text.


To test our hypothesis, we investigate if one can predict the antonym of a word.

given a word v whose word embedding is w v we determine the antonym to be that word a with word embedding w a such that the cosine similarity between w a , w v is minimized over all possible choices of a.


As a byproduct of our optimization formulation we show that the word embeddings learned by SWESA are polarity aware and perform very well on antonym tasks, even without being explicitly trained for such tasks.

strongly polarized word embeddings are easily distinguished.


Further, we apply CWI to the text retrieval task, and measure its effect on the vocabulary complexity of retrieved texts from the perspective of a language learner.

the algorithm attempts to find texts with a minimum proportion of non-complex words and a maximum number of challenging words for the learner.


In the annotation of discourse nuclearity, we are consistent with RST, which is divided into hypotactic ("Mononuclear") and paratactic ("Multi-Nuclear").

it is divided into three categories: Nucleus Ahead, Nucleus Behind and Multi-Nucleus.


Elaboration  1208  36  9  Causality  365  29  20  Coordination  453 15 731 Table 9: Nuclearity distribution of discourse relationships.

we analysed the distribution of pragmatic functions.


On the calculation of semantic similarity, we used the word2vec model to train on CTB8.0 and calculate the semantic similarity between the two discourse units according to the semantic similarity algorithm proposed by Xu (2009).

we discretized the final semantic similarity into 10 levels.


The problem of corpus based content construction could be perceived as a summarization task once we have identified the required candidate content fragments from the corpus.

the problem could be cast as a multi-document summarization or query-focused summarization.


To utilize the unlabelled data, we extend the traditional encoder-decoder model by leveraging two independent encoders but with some layers shared.

we share the weights of the last few layers of two encoders that are responsible for extracting high-level representations of input sentences.


Each encoder consists of three identical layers, where each layer consists of a multi-head self-attention and a simple position-wise fully connected feed-forward network, which follows the powerful self-attention based model Transformer (Vaswani et al., 2017).

the encoder is composed of a stack of three identical layers.


Nevertheless, it may cause overfitting during training since the fluent label 'O' and non-fluent label 'E' dominate in the training example.

the word sequence needs a simple classification as the label sequence does, and the label sequence can benefit from the semantic information of the word sequence.


In this algorithm, we need to judge the class of the label softmax to determine the decode sequence of the model.

if the label softmax outputs 'O', we copy the word from the source side and feed it into the model for the next time step.


In order to learn some knowledge from the unlabelled corpus to make up the scarcity of labelled data, an AE is utilized to train the unlabelled data via reconstructing their inputs.

the encoder is responsible for composing the high-level latent representation of the input and the decoder endeavors to decompose this representation into its corresponding input sequence.


As such, DAs have been investigated by the research community for many years (Stolcke et al., 2000) and have been applied successfully to many tasks.

their aspiration to generality makes them an appealing option for non task-based application (e.g.


The automatic recognition of Dialogue Acts has been addressed by the literature using various machine learning techniques.

dA classification has been modeled both as a sequence labeling problem, using techniques such as HMM (Stolcke et al., 2000), neural networks (Ji et al., 2016;Kumar et al., 2017) or CRF (Quarteroni et al., 2011), and as a multi-class classification problem, using for example SVM (Quarteroni and Riccardi, 2010).


The results are again consistent with our meaning frames.

the 72 diagrams tagged as having just direction interpretations in Experiment 2 include 68 of the 70 diagrams tagged as having just empty arrows in Experiment 1.


2012in one key way: whereas they look only at events of interest (which they assume to be already identified in the data), we look at the entire text of a given pathology report, without relying on sentence-level annotation of any kind.

we hypothesize that the usefulness of incorporating precise grammatical information about negation scope generalizes to the document level.


Overall, our proposed LSTM-CSS outperforms the other methods.

it reaches 42.3% BLEU@4, which makes the relative improvement over the two state-of-the-art methods S2VT by 20.1% and MTLM by 3.7%, respectively.


It is to use a class symbol to replace both the slot label and its slot value in the source sentence, for example, "Play $song in $album".

we generalize the source sentence into a pattern sentence.


The performance of class-based model relies on whether parallel annotated SLU data is enough or not to translate class symbols correctly.

slots will be missed or mistranslated without enough parallel data.


In this work, we choose greedy decoding at test time.

the most likely word from the decoder output p(θ) at each time stamp is selected as the baseline output, as shown on the right-hand side of Figure 1.


The estimated future rewards is then incorporated in the small-width beam search to simulate the performance of a large-width beam search.

given a K l -width beam search we want to simulate, at time step t it generates a set of partial responses For one partial response y t , its future reward with regard to beam width of if a partial response presents in the top-K responses from the given K l -width beam search (K < K l ) at future time steps, we assign it with a positive future reward.


In this work, we aim to develop multilingual conversational systems that can return appropriate responses in many languages.

we assume the two types of systems: (i) language-specific systems and (ii) language-invariant systems.


The results demonstrate that our methods allow models to effectively adapt to low-resource target languages.

our method using Wasserstein GAN (Arjovsky et al., 2017) achieves high-performance for simultaneously dealing with multiple languages with a single unified model.


In both two-and five-language adaptation, WGAN achieved the best scores.

in five language adaptation, regardless of using a single, unified model, JOiNT and WGAN achieved high-performance.


Furthermore, we re-train the pre-trained models by using the three methods, FINETUNE, JOINT and WGAN.

each method uses a pre-trained model as the initial model and re-trains the parameters.


Experimental results have demonstrated that our methods allow models to adapt multiple target languages.

methods for language-invariant models can simultaneously deal with multiple languages with a single model.


This means that, to some extent, image specificity is indeed language-independent.

the data suggests that some images just elicit more varied responses than others, and it does not matter whether you speak Dutch, English, or German.


Assume that all of the schemas in two sets of schemas have a J e value as described above, at 71%.

these are fundamentally two very similar sets of schemas-each schema in one set has a counterpart in the other set sharing five out of six events.


There are two possible explanations for this: (1) the number of documents withheld in cross-validation decreases with the number of documents, and stability depends more on the number of documents that differ between cross-validations than on the fraction of documents that differ, or (2) a fixed number of schemas can better capture the information contained in a smaller set of documents because new documents add more novel and unique narratives, thereby making the content of the narratives more difficult to capture in a finite set.

provided with more and more documents, the germinators do not converge to some finite set of schemas, but instead are presented with an increasingly difficult problem to solve.


The dataset released after this round provided insights on how to improve our conversational agent and helped to detect undesirable behavior.

we decided to add a chit-chat Alice, paragraph's topic mention to the greeting and text summarization skills.


In order to capture the interactions between the original post and the reply, we propose a novel co-attention network that includes three kinds of attentions.

post argument to reply argument attention computes the relevance of each post argument with every reply argument and helps learn a series of new reply representations.


To reveal this, we begin with constructing textual adversarial samples to attack the state-of-the-art system VSE++ (Faghri et al., 2017).

we study the composition of sentences in two aspects: (1) content words including nouns and numerals and (2) prepositions indicating spatial relations (e.g., in,  Figure 1: An overview of our textual contrastive adversarial samples.


To utilize these candidate caption sets, we employ an intra-pair hard negative mining strategy.

during the optimization, we add an extra loss term: In our implementation, the candidate set C has approximately 1, 000 samples.


Given an image-caption pair and its corresponding textual adversarial samples, we are interested in the following question: what is the semantic distance between the image an adversarial caption?

which part in the image or caption, in particular, makes them semantically different?


learned word embeddings, we conduct experiments on word-level image-to-word retrieval.

we first examine how each noun is linked with a visual object.


We can also model the relationships between input objects; the similarity between the representations of two posts would entail agreement between the labels associated with them, thus allowing us to perform collective classification over all the input instances.

we define the factor graph corresponding to the dependencies between stance predictions in a debate thread, and use the similarity between the embedded representation of objects as a scoring function for the factors.


Note that this also allows our model to generate embedding for unseen text in the new space.

for a text input x, we can compute its embedding e using M hidden layers The first hidden layer l 0 is computed from the input x: Subsequent layers are computed recursively: Then the output from the final layer produces the embedding: f is the non-linear activation function.


Then DUA conducts semantic matching between each utterance and the response candidate to obtain a matching score.