Note that setting n=3-6 and adding the jongsung symbol makes this model as a specific case of our model, containing jamo-level ngrams (n=3-6) and character-level n-grams (n=1-2) as well.

jamo-level n-grams help representing overall semantic features and our model show higher performance compared to baseline models.


HowNet provides sememe annotations for Chinese words, where each word is represented as a hierarchical tree-like sememe structure.

the ensemble weight of SP-WCF and SPCSE λ SPWCF λ SPCSE = 4.0.


We will give detailed analysis in our case study.

the meaning of each character in "奥斯卡" (Oscar) is unrelated to the meaning of the word.


5 We find that 'Mario' and 'Zelda' are perceived more casual in /r/PS4, and 'Uncharted' and 'Killzone' are more hardcore in /r/PS4 than /r/NintendoSwitch.

we focus on a few pairs of subreddits that are known to express different views.


sE-MAXIs characterizes word semantics with respect to many semantic perspectives in a domainspecific word-vector space.


tive reward from current time step to the end of an episode would cancel the intermediate rewards and thus reflect whether current action improves the overall performance or not.

research was sponsored in part by U.S. Army research Lab.


• Gloss Module: Like the context module, the gloss module encodes all the glosses of the target word into a separate vector representations of the same size.

these two models have used the external knowledge, for the LEX is based on lexicographer files in WordNet.


This indicates that we are able to incorporate sentiment information into our target language without any labeled data in the target language.

we compare BLSE with ARTETXE and BARISTA by replacing the Linear SVM classifiers with the same multi-layer classifier used in BLSE and observing the distances in the hidden layer.


ARTETXE (67-46 on binary, 35-21 on 4-class) and BARISTA (61-55 on binary, 40-34 on 4-class) are significantly worse than BLSE on all experiments except Catalan and Basque 4-class.

the downside BLSE Original Figure 6: t-SNE-based visualization of the Spanish vector space before and after projection with BLSE.


Otherwise, w is repre-sented by U p w and U q w for p and q respectively.

for the other combinations, the probability of "lightweight" to be domain-common is much smaller, which indicates that the meaning of "lightweight" varies.


We use the implementation from https: //github.com/attardi/deepnl/wiki/ Sentiment-Specific-Word-Embeddings.

each word w can be represented as follows: where ⊕ denotes the concatenation operator.


The exponential increase in the availability of online reviews and recommendations makes it an interesting topic in research and industrial areas.

the co-training algorithm assumes that the data set is presented in two separate views, and two classifiers are trained for each view.


In this work, we propose a method to simultaneously extract domain specific and invariant representations and train a classifier on each of the representation, respectively.

in the space of the target domain specific representation, the distributions of source data H s spec and target data H t spec are forced to be different by minimizing the distance L dif f .


Each domain adaptation task consists of 2,000 labeled source examples,  Table 1: Average prediction accuracy with 5 runs on target domain testing data set.

we proposed a novel method, based on the CMD metric, to simultaneously extract domain invariant feature and domain specific feature for target domain data.


a large body of literature tries to utilize the relation or position between the target words and the surrounding context words either by using the tree structure of dependency or by simply counting the number of words between them as a relevance information (Chen et al., 2017).


We consider a recent sentiment lexicon called VADER (Hutto and Gilbert, 2014).

deep neural networks are widely used in sentiment polarity classification, but suffer from their dependence on very large annotated training corpora.


Firstly, lexicon-driven methods have often been used for domain-independent sentiment analysis.

the co-occurrence-based objectives of word2vec and GloVe do not consider sentiment specifically.


Baselines: Models with user and product information User and product information are helpful to improve the performance of a sentiment classifier.

the base model is a word encoder that transforms vectors of words {w i } in the text to new word vectors.


This happened with some shortcuts in particular.

most tags indicate the main function of the contribution, such as 'proposal' and 'question'.


1), which has an order of magnitude more images than the COCO dataset.

conceptual-trained models do not seem to have this problem.


Although the concatenated image and word embedding features performed the best, we do not expect to have high-quality word embeddings for words in low-resource languages.

the number of candidate English words is the number of entries in the bilingual dictionary after filtering out dictionary entries where the English word and foreign word are identical.


Previous works have shown that visual attention alone can perform fairly well for localizing objects  and aiding caption generation (Xu et al., 2015).

most recent image captioning models are based on a CNN-RNN framework (Vinyals et al., 2015;Fang et al., 2015;Karpathy and Fei-Fei, 2015;Xu et al., 2015;You et al., 2016;Krause et al., 2017).


In addition to generating an exact targeted caption by perturbing the input image, we offer an intermediate option that aims at generating captions with specific keywords, denoted by K : Intuitively, finding an adversarial image generating a caption with specific keywords might be easier than generating an exact caption, as we allow more degree of freedom in caption generation.

to the best of our knowledge, this is the very first work on crafting adversarial examples for neural image captioning systems.


Interestingly and perhaps surprisingly, our results pinpoint the Achilles heel of the language and vision models used in the tested image captioning systems.

as we need to ensure a valid and meaningful inferred caption, finding an adversarial example with specific keywords in its caption is difficult in an optimization perspective.


As described in previous sections, a DSMN forms an intermediate visual representation of the input.

room {1, 2, 3} is located in the {n, s, e, w, c, n-e, s-e, n-w, s-w} of the house and is small in size.


In this paper, we leverage conversation as both an interactive environment for language learning (Skinner, 1957) and a natural interface for acquiring new knowledge (Baker et al., 2002).

humans learn in a way very different from the supervised setting (Skinner, 1957;Kuhl, 2004).


As this marginalization is intractable, we derive a variational inference procedure that allows for efficient approximate inference.

the second difference is that our model is non-Markovian: we sample the i th morphological tag m i from a distribution that depends on all previous tags, using an LStM language model ( §4.1).


Closest to our work is Zhou and Neubig (2017), who describe an unstructured variational autoencoder.

what is our interest in a generative model of inflected forms?


(1) and overlayed with example values of the random variables in the sequence.

we fall back on a variational approximation (Jordan et al., 1999).


To keep our results comparable with the Shared Task, we use the provided precomputed word embeddings.

word-model(train-corpus, paw) paw = word-opt.update(char-preds, train-data) / * Update Meta-BiLSTM model.


(2016) is probably the jumping-off point for most current architectures for tagging models with recurrent neural networks.

critically, however, all the information fed to this representation comes from the word itself, and not a wider sentence-level context.


's (2017a) approach to mildly nonprojective parsing in what, to our knowledge, is the first implementation of exact decoding for a non-projective transition-based parser.

to resolve the conflict between including s 1 for richer representations and the unavailability of s 1 in scoring the sh transitions in the COMBINE steps, we propose a hybrid scoring approach -we use features ts 0 , b 0 u when scoring a sh transition, and features ts 1 , s 0 , b 0 u for consideration of reduce transitions.


We call this method MH 4 -hybrid, in contrast to MH 4 -two, where we simply take ts 0 , b 0 u for scoring all transitions.

exact inference has been shown to be intractable for models that support arbitrary nonprojectivity, except under strong independence assumptions (McDonald and Satta, 2007).


This can be done independently at different layers; the results of this experiment are shown in Table 1.

the query q t is compared with all keys to form a probability distribution p(t !


To control for this confound we also evaluated a version of our model that enforces block-sparsity throughout, but retains the use of componentwise addition at the inputs.

its performance is ultimately not quite as good as using a character LSTM.


For example, if there are two persons named 'Avery', it is vital to know their genders or last names to avoid confusion.

linguistic hints that suggest whether a new object is introduced.


While these methods are all tailored to their specific tasks, they all inspire our choice of a self-attending mechanism.

in all of our models, we limit the maximum length of samples and POS tags to 60 tokens.


The weights update rules of theses models use as inputs one or several of (i) a previous hidden state of a RNN network or higher level network and/or (ii) the current or previous inputs to the network.

the top curve compares a one-level model (language model) with a two-levels model (language model + meta-learner).


Targeting K = 100, we set H = 244 for the r-GRU and compare with a GRU with H = 650 which has the same number of parameters.

increasing capacity (number of tunable parameters) by augmenting the size H of the hidden (or recurrent) layer -to model more complex distributions -results in a significant increase in computational cost, which is O(H 2 ).


Table 2: Results of the dependency arc prediction task.

these results may be surprising given the findings of Linzen et al.


Table 4 reports the overall performance on the dev and on the test set.

furthermore, the performance   of the estimated WER per sentence in the glassbox is substantially better than the black-box for both development and test sets.


The ADV trained models all maintain the task performance of the BASELINE method, however they clearly have a substantial effect on the discrimination accuracy.

the authorship of training and evaluation corpora can have unforeseen impacts, including differing model performance for different user groups, as well as privacy implications.


We investigate the robustness of a classifier trained with adversarial examples, by studying its resilience to attacks and its accuracy on clean test data.

we consider the adversary successful only if the classifier misclassifies the instance with a given confidence score.


From our hypothesis, KCCA embeddings are expected to perform better than the others because CCA/KCCA provides an intuitively better technique to preserve information from both the generic and DS embeddings.

these forums are part of a mobile health intervention treatment that encourages participants to engage in sobriety-related discussions.


We also compare against a basic DA embedding baseline in both the standard framework and while initializing the neural network baseline.

the concSVD based embeddings do not exploit information in both the generic and DS embeddings.


Classically, an LSTM cell is defined as follows (biases excluded for brevity): One way of measuring the complexity of a model is through its total number of parameters.

we take the cell update recurrence parameters w i across LSTMs (we omit the c in the subscript for brevity; the index i runs across the smaller LSTMs) and for any pair we wish the following to be true: To achieve this we pack the vectorized parameters into a matrix: and apply the following regularization term to our final loss: The concatenated output h t is passed through a fully connected layer with bias before being passed through a final softmax layer: To extract a predicted entity typeŷ t at time t, we select the entity type corresponding to the most probable output:ŷ The loss is defined as the sum of the softmax cross-entropy losses along the words in the input sequence.


First, as opposed to many boosting algorithms (Freund et al., 1996;Schapire and Singer, 1999;Dietterich et al., 2000) we do not "reweigh" training instances based on the loss incurred on them by a previous iteration.

applying such techniques in new domains requires making a heavy engineering investment.


(2017), phenomena such as morphology make DST a substantially harder problem in linguistically richer languages such as Italian and German.

their downside is shifting the problem of dealing with linguistic variation back to the system designers, who have to craft semantic lexicons to specify rephrasings for ontology values.


As the number of inner span annotations is marginal and hard to detect, we additionally present scores considering only outer span annotations in Table 2.

† marks results statistically significantly better than the ones reported in Table 4. with different labels and show that only 60% of the data of the target domain is required to achieve good results.


Most related to our paper is the work by Ghaddar and Langlais (2017) which demonstrates the impact of transfer learning of the English CoNLL 2003 dataset with Wikipedia annotations.

bidirectional LSTMSs (BiL-STMs, e.g., Reimers and Gurevych, 2017) identify informative features directly from the data, presented as word and/or character embeddings (e.g., Mikolov et al., 2013;Bojanowski et al., 2017).


While these studies offer insights into the language used by developers, they are limited to sentiment and basic lexical attributes.

we reduce the 7point rating scale to a binary (formal vs. informal) scale.


(2001) points out that although affixes are rare in Chinese, there are some components behaving like affixes and can also be used as independent lexemes.

in Chinese analogical reasoning task, we aim at investigating to what extent word vectors capture the linguistic relations, and how it is affected by three important factors: vector representations (sparse and dense), context features (character, word, and ngram), and training corpora (size and domain).


The resulting corpus is split into train/dev/test sets.

our dataset consists of around 200K news articles and 4.5M human comments along with rich meta data for article categories and user votes of comments.


To be specific, we refer Q and V to the representation matrix generated by the CNN module, while K = W att V where W att is a learnable matrix.

the seq2seq model generates a summary which only contains the information of the brand and the country.


Our second goal is to determine if these trends are maintained for various types of news sources on Twitter and Reddit.

in the current work, we have presented a contentbased model that classifies user reactions into one of nine types, such as answer, elaboration, and question, etc., and a large-scale analysis of Twitter posts and Reddit comments in response to content from news sources of varying credibility.


For Reddit, there appears to be a very similar distribution across reaction types for trusted and deceptive sources.

(2017), which relies on content as well as additional metadata like the author, thread (e.g., the size of the the thread, the number of branches), structure (e.g., the position within the thread), and community (i.e., the subreddit in which the comment is posted).


The columns depict the birth year for users in the test set.

we represent the language of each user as a probabilistic distribution of 2000 topics derived using Latent Dirichlet Allocation (LDA) with α set to 0.30 to favor fewer topics per document.


For instance, a sentiment model trained on Victorian-era language would label 'aweful' as positive sentiment; however, in contemporary usage, 'awful' is considered a negative word (Wijaya and Yeniterzi, 2011).

we first establish the diachronic validity of language-based models through predictive evaluations.


And it is collected accumulatively following a diagnostic procedure in clinic, which involves interactions between patients and doctors and some complicated medical tests.

when training the policy, we use a technique known as experience replay.


For every parallel sentence pair, we randomly sample k negative sentences {N i |i = 1 .

we later realized that the cross lingual sentence similarity objective is at odds with what we want the attention layer to learn.


The max-margin loss aims to rank correct training data pairs (x i , y i ) higher than incorrect pairs (x i , y j ) with a margin of at least γ.

if the underlying map is non-linear, the locally linear approximations will depend on the neighborhood.


Typical examples are Wikipedia or news collections which report on the same facts in different languages.

as in section 4, we embed all sentences into the joint space.


The average sentence length for all the WMT training corpora is 24, while it is only 8 words for our mined texts.

we observe no improvement when adding the mined data to our best system which uses the filtered Common Crawl data (5th column of Table 6).


In recent years, neural networks with distributed word representations (i.e., word embeddings) (Mikolov et al., 2013;Pennington et al., 2014) have been introduced to calculate word scores automatically for CRFs (Chiu and Nichols, 2016;.

for example, if a sentence x in NER task is "Barack Hussein Obama and Natasha Obama", we have the corresponding s = ((1, 3, P ER), (4, 4, O), (5, 6, P ER)) and y = (B-PER, I-PER, E-PER, O, B-PER, E-PER).


Table 2: Hyper-parameters of the models built in our experiments, where † indicates the ones when using LM-BLSTM for deriving word representations and ‡ indicates the ones when using CNN-BLSTM.

semi-Markov conditional random fields (SCRFs) (Sarawagi and Cohen, 2005) have been proposed for the tasks of assigning labels to the segments of input sequences, e.g., NER.


In Section 4 we provide results for several tested values of α parameter and discuss our findings.

in this paper we use named entities as domain-specific terms for newscentric content and present a new weighting model for Latent Dirichlet Allocation.


in this section, we describe our extension to original definition model.


For SciQ dataset, we used the associated passage provided with the question.

let q, o i denote the word embeddings of words present in the question and the i th option respectively.


For fair comparison, we evaluated the best performed (on dev data) model during 500K training batches on the test data.

the updated objective function using weighted sampling J ws can be formulated as follows: only 80% of the entire corpus is used to perform the NMt training during each iteration (for the first two iteration, all of the sentences should be sampled).


Our particular setting deals with |S| ≈ 1800 but our approaches should be able to accommodate orders of magnitude more speakers.

of 1, we can assume that many speakers share similar characteristics such as gender, social status, and as such may have similar associated domains.


It is also possible to constrain decoding of linearized trees and derivations to wellformed outputs.

we suggest a training strategy for the Transformer model (Vaswani et al., 2017) which gives improved performance for long sequences, like syntax representations, without requiring additional GPU memory.


We train separate Japanese (lowercased) and English (cased) BPE vocabularies on the plain-text, with 30K merges each.

ensembling often requires component models to make predictions relating to the same output sequence position at each time step.


(2016) extended this idea to also include word deletions and substitutions with the goal of reducing human effort in translation.

for En-Es we have 2.7M out-of-domain and 1.5M in-domain sentence pairs.


Finally, similarly to the second variant, the last variant, Pseudofit fus-*, adds a supplementary representation of the target word.

in Pseudofit max, all {t j } are Variant R prec.


The second variant, Pseudofit 3 pseudo-senses, aims at determining if increasing the number of pseudo-senses, from two to three at first, can have a positive impact on results.

from the more general viewpoint of its meaning, they should be identical, or at least very close, and their differences be considered as incidental.


(2004) without explicitly modeling syntax.

systems are evaluated on the micro-averaged F1 for correctly predicting (predicate, argument span, label) tuples.


This means that a source position can be revisited many times, whereby creating one-to-many alignment cases.

the search space in attentionbased decoding consists only of translation decisions.


We experimented with three different conditions, one within language and two across language.

results and Analysis Inter-annotator agreement for the tasks was measured via Fleiss kappa (n = 3, N = 200), and was higher for the in-language experiment (K = 0.40) than for the cross-language tasks (NL →PT: K = 0.25; Fr →NL: K = 0.28).


It has the advantage of fading out content in favor of more shallow patterns still based on the original text, without introducing additional processing such as part-of-speech tagging.

• AllAbs A combination (concatenation) of all previously described features.


Note that mentions like "U.S." and "Barack Obama" do not align with any noun phrase.

the En model performs 0.5 point below the Es model on the Es test set.


Firstly, we propose an entity-centric neural crosslingual coreference model.

the recent SOTA was obtained using pipeline models of coreference and EL (Sil et al., 2018).


All the three generations only consider binary extractions from the text, while binary extractions are not always enough for their semantics representations.

for future research, we will further investigate how to generate more complex tuples such as nary extractions and nested extractions with the neural approach.


To this end, we propose a neural Open IE approach with an encoder-decoder framework.

sRLIE (Christensen et al., 2010) was developed to include an attribute context with a tuple when it is available.


That means the events detected by DEEB-RNN2 are basically the ones with explicit event triggers.

the ED Oriented Document Embedding Learning (EDODEL) module, which learns the distributed representations of documents from both word and sentence levels via the well-designed hierarchical and supervised attention mechanism.


In addition, we utilize the pre-trained word embeddings with 300 dimensions from (Mikolov et al., 2013) for initialization.

dEEB-RNN1 uses only the gold word-level attention signal (λ = 1 and µ = 0), dEEB-RNN2 uses only the gold sentence-level attention signal (λ = 0 and µ = 1), whilst dEEB-RNN3 employs the gold attention signals at both word and sen-  tence levels (λ = 1 and µ = 1).


In practice, because human labeling is expensive and exhausting, one cannot have large scale labeled data for model training.

precision at position 1 (p@1) is employed as an evaluation metric.


The tasks that dialogue systems are trying to solve are becoming increasingly complex, requiring scalability to multi-domain, semantically rich dialogues.

the Bt then passes these states to the Policy Optimization unit as full probability distributions to take actions.


These two variants of RNN are a combination of Henderson et al.

it only applies to a single domain and does not share parameters across slots.


Figure 1: An illustration for modelling discourse cohesion with memory network.

we follow the conventional data structures in transition-based dependency parsing, i.e., a queue (B) of EDUs to be processed, a stack (S) to store the partially constructed discourse trees, and a stack (A) to represent the history of transitions (actions combined with discourse relations).


We find that the Basic(word+pos+position) method significantly outperforms (Jia et al., 2018), and our Refined model performs significantly better than Basic(word+pos+position) (with p < 0.1).

and, the neural network design is flexible enough to incorporate various clues into a uniform framework, just like how we introduce the memory networks as a proxy to capture discourse cohesion.


In this section, we describe two key aspects of our annotation framework, including elementary discourse units (EDU) and discourse relations.

furthermore, our treebank SciDTB can be made as a benchmark for evaluating discourse parsers.


There is a large body of research on the relationship between training data size and system performance.

assuming that the underlying system has adjustable hyperparameters, the question arises: how should the hyperparameters be set?


We show below that the more demanding approach of optimising on each subset is superior, especially when extrapolating from small pilot datasets.

if one is not working with computationally efficient algorithms like fastText, one might be tempted to only optimise the hyperparameters once on all the pilot data, and use the hyperparameters optimised on all the pilot data when calculating the error rate on subsets of that pilot data.


In our case, we also have the document context, information which both models can leverage.

in the research presented here we are interested in predicting acceptability judgements.


Table 1: Test mean nearest neighbor overlap.

whether this is achieved has not been investigated yet.


Boldface indicates the largest score at each mNNO 10 (X, f (X)) and mNNO 10 (Y, f (X)) pair, which are abbreviated by X, f (X) and Y, f (X).

the success of the whole system relies on the ability of the mapping to make the neighborhood structure (i.e., the pairwise similarities) of the predicted vectors akin to that of the target vectors.


This can be viewed as a "soft" version of the maxmargin objective used by  for training without exploration, but retains a locallynormalized model that we can use for samplingbased exploration.

given an arbitrary cost function ∆ comparing structured outputs (e.g.


We show that a beam size of 20 can be fast while still achieving state-of-the-art performances.

we want the model to score the gold tree t * higher than any other tree t by at least a margin ∆(t, t * ): Note that ∆(t, t) = 0 for any t and ∆(t, t * ) > 0 for any t = t * .


(2017a) counts the incorrect labels (i, j, X) in the predicted tree: Note that X can be null ∅, and t * (i,j) denotes the gold label for span (i, j), which could also be ∅.


These traditional methods can work almost as well as human raters.

for instance, the proposed MHCNN achieves an improvement of 6.4% accuracy over the traditional CNN.


Contrary to our hypothesis, MWE achieves higher accuracy than Meta-Only.

(2016) represented the text using a mean word embedding (MWE) representation, we replace it with a Convolutional Neural Network (CNN) representation (Kim, 2014), which has achieved superior performance on recent text classification tasks (Dauphin et al., 2016;Wen et al., 2016;Yang et al., 2016).


To address this, we limit the length of each full-text and summary to N words, where N is empirically set to the 80 th percentile of the collection.

it remains unclear whether this signal is related to ideology or other contextual information.


By a reasonable threshold τ , we can add the semantically coherent link for w In practice, we set τ to 0.4.

it's nontrivial to apply CTM directly on phrases.


However, these benchmarks were mostly derived independently of any NLP problems.

(2018)), we put a vanilla CNN (convolution&max-pooling) over merely hypothesis or premise to derive the pair label.


Compared to Euclidean distances, vector cosine scores are normalized on each dimension and hence are robust to the scaling effect.

as it can be easily noticed by looking at the bold line, aP SynP outperforms the baselines in all settings for both Skip-Gram and GloVe, obtaining higher accuracies and OPPs.


4): While for the similarity task scores are always calculated pairwise, for spotting the outlier two different methods were tested: the pairwise comparisons and the cluster prototype.

workshops such as RepEval have been organized with the explicit purpose of finding alternative evaluation tasks for DSMs.


4 More precisely, for each deterministically identified n-gram phrase, we progress from the first to the (n − 1) th gram, randomly splitting the phrase at that point with probability e n R r=1 e r .

these character ngram parameters outperformed the range of 3 to 6 proposed by Bojanowski et al.


We hypothesize they constitute spurious events and therefore test the t-step random walk both on matrix S and its smoothed version R in which the self-loops are removed.

this range is determined by the size of the training context c. a surrounding word w s of the training word w t must satisfy the following condition: For the same corpus, let us set d max equal to c in word2vec and generate a word co-occurrence network of the whole corpus.


However, this means that using the nearby context can lead the word vectors to capture some aspects other than style.

in addition, there is another trend of research forward controlling style-sensitive utterance generation without defining the style dimensions (Li et al., 2016;Akama et al., 2017); however, this line of research considers style to be something associated with a given specific character, i.e., a persona, and does not aim to capture the stylistic variation space.


Knowledge from document-level examples with balanced labels compensates for this disadvantage.

both accuracy and macro-F1 are used for evaluation as label distribution is unbalanced.


Attention-based long short-term memory (LSTM) networks have proven to be useful in aspect-level sentiment classification.

(2016), the authors investigated the transferability of neural models in NLP applications with extensive experiments, showing that transferability largely depends on the semantic relatedness of the source and target tasks.


The first clause expresses a positive sentiment, while the second clause reveals a negative sentiment.

this may be explained with the incongruity theory, where the  setup of the text prepares an expectation for the readers, while the punchline breaks the expectation.


Again, when removing content features (Base2), the improvements are large.

for each instance, we recognize EDUs and the relations connecting them.


In this set-up, after training the model on concatenated data, it is refined for each treebank by training only on its own training set for a few additional epochs.

in this model, treebank-specific BiL-STMs are constructed for all treebanks in addition to a shared BiLSTM which is used both for parsing and for the adversarial task.


It is clear that some treebanks are bad fits to PUD, most notably Finnish FTB and Russian SynTagRus.

some treebanks are thoroughly checked by human annotators, whereas others are based entirely on automatic conversions.


(2011) extended with a SWAP transition and a static-dynamic oracle, as described in de Lhoneux et al.

even when a treebank is a bad fit, TB-EMB and C+FT can still improve substantially over using only the single model for the treebank with the best fit, as for Russian where there is a gain of nearly 8 LAS points for TB-EMB over SINGLE, when using GSD as a proxy.


Chart con-straints are trained to identify constituent boundaries in the original treebank, where nodes may have more than two children.

the effect of using chart constraints is that the parser considers fewer substructures of the input object -potentially to the point that the asymptotic parsing complexity is reduced below that of the underlying grammar formalism (Roark et al., 2012).


investigated to what extent different embedding methods are predictive of i) the occurrence of words in the original sentence, ii) the order of words in the original sentence, and iii) the length of the original sentence (Adi et al., 2016(Adi et al., , 2017.  inspected neural machine translation systems with regard to their ability to acquire morphology, while Shi et al.

even on SICK, oftentimes very shallow methods prove effective at obtaining fairly competitive results (Wieting et al., 2015).


QuEst++ offers a range of learning algorithms but defaults to Support Vector Regression for sentence-level QE.

our proposed, interpreter-focused model out-performs in all language settings with notable gains in particular for EN-JA(A-Rank) (+0.104), achieving its highest accuracy on the EN-FR dataset.


Prior work (Chan et al., 2007) shows that semantically equivalent words coming from different languages may contain shared semantic level information, which will be helpful for certain semantic processing tasks.

the authors first map first-order logical statements produced by a semantic parser or an information extraction system into expressions in tensor calculus.


guage remains a research question that is largely under-explored.

in this paper, we empirically show that the distributed representations of logical expressions learned from multilingual datasets for semantic parsing can be exploited to improve the performance of a monolingual semantic parser.


In all experiments, we follow the experimental settings and procedures in (Lu, 2014  2015; Susanto and Lu, 2017b).

german: welche flüsse fliessen nicht durch texas ?


We note that, unlike text-based interactions, DrugBank only contains the information of interacting pairs; there are no detailed labels and no information for non-interacting (negative) pairs.

92.15% and 93.09% of drug mentions in train and test data set matched the DrugBank entries.


We show that in most cases this approach can lead to improvements in classification performance, even on future time intervals.

characterizing the ways in which content distributions vary over time, and why this affects performance, is still an open question.


To do this, we leverage recent advances in contextadaptive neural language modeling.

park and Chiba (2017) find that the neural LM outperforms MpC even when MpC has been augmented with the approach from Mitra and Craswell (2015) for handling rare FactorCell  ConcatCell  1  neiman marcus  craigslist nyc  2  pottery barn  myspace layouts  3  jc penney  verizon wireless  4 verizon wireless jensen ackles 5 bed bath and beyond webster dictionary Table 4: The five queries that have the greatest adapted vs. unadapted likelihood ratio after searching for "prada handbags" and "versace eyewear".


In this paper, we focus on designing a review generation model that is able to leverage both user and item information as well as auxiliary, textual input and aspect-aware knowledge.

we also add dropout layers before and after the GRUs.


(2017) and approaching their best RDF-aware method.

note that the number of simple sentences to generate is not given as part of the input.


2 We split each prediction to  sentences 3 and report the average number of simple sentences in each prediction, and the average number of tokens for each simple sentence.

it is not required to generalize to unseen relations.


Our goal is to study how much each of these components contribute to the empirical performance of LSTMs.

in the LSTM -S-RNN and LSTM -S-RNN -OUT experiments, we ablate the S-RNN.


Indeed, Appendix A in (Chen et al., 2017) provides concrete IRNNs for recognizing the languages a n b n and a n b n c n .

the SRNN update equation is: By properly setting U and W, one can get certain dimensions of h to update according to the value of x, by h this counting behavior is within a tanh activation.


We can see that the performance of our model on different types of questions in the RACE dataset is quite similar.

still, there is a huge gap between the best machine reading performance and the human performance, showing the great potential for further research.


Finally, each word is represented as a concatenation of the characterlevel embedding and word-level embedding.

the accuracy decreases sharply when discarding the sentiment words.


• Lang, SeqAE: Pretrained with the language model and autoencoder model proposed in (Dai and Le, 2015).

looking at the results of Dial, our method improved Default even for these cases (5K to 20K), and Dial clearly outperformed the linear models.


Self-labeled data is an accessible and economical resource for a variety of learning-based applications.

figure 3 shows the performances of the irony detector trained on the data filtered with different threshold values.


The best model is the one integrating the attentive GRU encoder, which is significantly superior to all baseline models (p < 0.05), achieves an F-score of 88.49%, To confirm the effectiveness of the language modeling of POS sequence, we also try to exclude the GRU language model from our best model.

directly using the labels made by the crowd as ground-truth for training and testing may lead to inaccurate performance due to the reliability issue.


For example, "feminist", "rights", and "equality" are commonly used when talking about women's rights, and "president" and "dreams" of-   ten appear in text about politics.

for some targets that can be recognized as being related to the same and more general domains, it could be possible to generalize through certain aspects of the domains that reflect users' major concerns.


We observe a wide range of phenomena, extending beyond expected phenomena like negation, antonymy, and entity changes.

workers were asked to spend 7 minutes per paragraph, and were paid $10.50 per hour.


On average, we collected 4.8 answers per question.

sQUADRUN is much more diverse than RULEBAsED, which creates unanswerable questions by applying entity, number, and antonym swaps to existing sQuAD questions.


Figure 4 shows two example pairs of colors which are spaced equally in terms of distance in RGB, but in terms of the Delta-E metric the green colors are closer together.

(2017) study is important -comparatives describe a change in a feature with respect to a reference point.


The data consists of multi-turn realworld conversations between customers and customer service staff in Taobao 2 , which is the largest e-commerce platform in China.

note that we define g(c, r) with all blocks rather than only with the last block.


We use In (k+1) r to denote the matrix representation of u (k+1) <i as following: where is the vector representation of sentence-start token.

for all models, the hidden size is set to 512.


BLEU measures n-gram overlap between a generated response and a gold response.

gru, with the assistance of his colleague dr. Nefario and his minions, resolves to one-up this mystery rival by shrinking and stealing the moon.


For Case 2, the secondpass decoder uses more detailed knowledge than the first-pass one.

to the original Deliberation Network (Xia et al., 2017), where they propose a complex joint learning framework using Monte Carlo Method, we minimize the following loss as Xiong et al.


The recently proposed Transformer adopted the self-attention mechanism which could implicitly capture inter-word dependencies in an unsupervised way (Vaswani et al., 2017).

separately processing H and U n then combine their attention with a learned λ performs better than treating the whole dialogue tokens as s single input, though the improvement is less significant compared with previous two mentions.


The encoder and decoder LSTMs have 2 layers with 128 dimensional hidden states with a dropout rate of 0.1.

mutualFriends is a multi-turn goal-oriented dataset (He et al., 2017) where two agents must discover which friend of theirs is mutual based on the friends' attributes.


Transformer Our transformer (Vaswani et al., 2017) model uses 300 dimensional embeddings and hidden states, 2 layers and 2 attention heads with no dropout.

we want to learn a conditional probability distribution P θ (y|x) where y is a reasonable response given the conversation history x.


Neural models have become one of the most important approaches to dialog response generation.

we chose to be aggressive and set c = 0, so responses that are similar to those generated by previous models are excluded.


Furthermore, we find in our experiment that third layer tends to focus on sentence topic and more abstract meaning of the segments, which achieve better performance.

the StM block makes a 10.54% improvement at R 100 @1.


While the PSEUDO greatly depends on the quality of pseudo-samples even if a discount factor is considered.

it can leverage unlabeled data (queries or synthesized logical forms) in a more effective way which helps alleviate the problem of lack of annotated data.


We can map t to this graph by applying the following linear tree homomorphism h into H 2 : A derivation of the form * 0 (t 1 , t 2 ) evaluates to the same graph as t 1 ; the graph value of t 2 is ignored.

then by lemma 11 again, n x , hence n t 0 x = n x and e t 0 x ≤ e t 0 x + n which yields Case 2 In this case Lemma 12 applies and at least one of C 2 -C 4 generate some token z ∈ {a, b, c, d}.


On one hand, the highest frequency of source-side copy helps address sparsity and results in the highest precision and recall.

grammar-based approaches are represented by Artzi et al.


Grammar-based approaches are represented by Artzi et al.

we see space for improvement, especially on the relatively low recall of target-side copy, which is probably due to its low frequency.


Encoder Layers Our encoder layers are essentially identical to the Transformer, except with the proposed extension to self-attention to incorporate edge labels.

there was 1 example where an entity in the logical form did not appear in the input, leading to the example being dropped from the training set.


These strategies are also quite promising given that no training is required given the pre-trained continuous sentence representations.

for the continuous generic sentence embeddings, we make comparisons with fastText-BoV (Joulin et al., 2016), Skip-Thought Vectors (Kiros et al., 2015) and InferSent (Conneau et al., 2017).


These two disadvantages hinder the applicability of generic sentence representations to mobile devices, where a relatively tiny memory footprint and low computational capacity are typically available (Ravi and Kozareva, 2018).

there are no classifier parameters for both the binary and continuous representations.


Table 3: Ablation study for the AE-binary-SP model with different choices of λ sp (evaluated with test accuracy on the MR dataset).

in the case where less memory footprint or little training is preferred, Rand-binary and PCA-binary could be more judicious choices.


A similar trend can be observed for graph-based dependency parsers.

the features drawn from partial subtrees become redundant because the parsing models encode them implicitly.


Contrary to our expectation, the plain depccg retrained on Questions data performs the best, with neither ELMo nor the proposed method taking any effect.

finally, by fine-tuning on the Genia CCGbank, we observe another improvement, resulting in the highest 86.52 f1 score.


In the case of integrating datasets from the news and medical domains we found the blending task to be difficult.

for token recoding we used a two-level stacked bidirectional LSTM (Graves et al., 2013) with both output and hidden state of size 100.


3.1) for mapping their predictions onto the test tag-set.

given this, we do not provide further details of the results of M Concat in this experiment.


In all experiments we assess model performance via micro-averaged tag F1, in accordance with CoNLL evaluation (Tjong Kim Sang and De Meulder, 2003).

this model learns to predict a tag sequence only over the fine-grained tags in the hierarchy.


Experimenting on two NER corpora, one with as many as 41 languages, we show that single model transfer has highly variable performance, and uniform ensembling often substantially underperforms the single best model.

spanish for Galician) (Cotterell and Duh, 2017)  the source and target languages through phonemic transcription (Bharadwaj et al., 2016) or Wikification (Tsai et al., 2016).


We call this method BEA uns×2 and its results are reported in §4.

when some labelled text is available, how might this best be used?


Similarly, when informed that an embed-ding is noisy or uninformative, the model should rely more on other features.

contextual features should be more robust when we apply the model to new data.


As Table 3 shows, when tested on another genre, the model encounters a high percentage of names that are unseen in the training genre.

current models typically trust all word embeddings equally regardless of their reliability and thus may introduce noise and hurt the performance.


In this circumstance, they can simply share the vocabulary and learn joint BPE for source and target languages, and share the encoder and decoder, which is extremely helpful for word embedding and latent representation alignment.

for example, the distant language pair Danish→Galician can be translated by three easier hops: Danish→English, English→Spanish and Spanish→Galician.


All the sentences in the bilingual and monolingual data are first tokenized with moses tokenizer 11 and then segmented into subword symbols using Byte Pair Encoding (BPE) (Sennrich et al., 2016).

unsupervised pivot translation may result in lower accuracy than direct unsupervised translation if a poor path is chosen.


Generally speaking, routing can be performed in many types of networks, including circuit switching network (Girard, 1990), computer networks (e.g., Internet) (Huitema, 2000), transportation networks (Raff, 1983) and social networks (Liben-Nowell et al., 2005).

how to choose a good translation path is critical to ensure the accuracy of unsupervised pivot translation.


We provide two instantiations of our framework, and show that both outperform their supervised and unsupervised counterparts.

another avenue of research has been to try and modify the underlying embedding generation algorithms.


NMT 3.2M (optimized) In a bleed continent, these people were the only ones who had never been conquered by the Spanians.

the core idea is to train a simple feed-forward network, the lexical model, jointly with the original attentional NMt model.


We train neural systems with Nematus (Sennrich et al., 2017b).

both N do continue to have diffi infrequent words, particula highly-inflected categories For the neural machine use a publicly available m settings of Edinburgh's W nrich et al., 2016a).


In NMT fine-tuning, a model is first trained on a task A, typically translating a large generaldomain corpus (Luong and Manning, 2015).

eWC outperforms no-reg and L2 on News, not only reducing forgetting but giving 0.5 BLeU improvement over the baseline News model.


Table 3 shows that the end-to-end models outperform the existing model.

we collect high-quality training data by distant supervision with co-reference resolution and paraphrase detection.


These methods integrate entity linking (i.e., NED) into their models.

table 1 illustrates an example of triples extracted from a sentence.


This feature-based method tends to predict a highly frequent label as the relation (e.g., "per:title").

this fine-grained classification task is much harder than coarse-grained classification task.


Ideally, the model should be able to learn how to maintain a balance between including and excluding information in the full tree.

the performance of all models degrades a lot.


As a result, the rank of matrix W a • H a is K d + 1 or less, where the K c common topics increase the rank by only one.

the cylindrical window is moved in space and time to search for the statistically strongest clusters (Kulldorff, 2010).


Furthermore, a smaller vocabulary size is more convenient for model training since training time increases with vocabulary size (Morin and Bengio, 2005) and hence a model with a smaller vocabulary trains faster.

in this work, we answer this question for multilingual named entity recognition and part-of-speech tagging and make the following contributions: • We present a large-scale evaluation of multilingual subword representations on two sequence tagging tasks; • We find that subword vocabulary size matters and give recommendations for choosing it; • We find that different methods have different strengths: Monolingual BPEmb works best in medium-and high-resource settings, multilingual non-contextual subword embeddings are best in low-resource languages, while multilingual BERT gives good or best results across languages.


We show that our framework can successfully improve an existing neural design, especially when the number of training examples is limited.

also related to our attention augmentation is using word relatedness as extra input feature to attention neurons (e.g.


Training ends when the validation score does not increase any further (scoring 29.08 BLEU on the WMT test).

the decision whether to label an instance in a batch is made based on the average token entropy of the model's current hypothesis.


Figure 2 visualizes how our CTT model puts attention on different phrases in a tree to compute the correct sentiment.

all of these works follow a common paradigm: use an LSTM/GRU over the word sequence, extract contextual features at each time step, and apply some kind of pooling on top of that.


For SICK-R, we are getting .2774 and .3012 MSE whereas the reported MSE for DT-LSTM and CT-LSTM are .2532 and .2734, respectively.

we then project each of the resultant tensors into different semantic sub-spaces and employ a residual connection Srivastava et al., 2015) around them.


We analyze whether pre-trained representations provide a greater benefit in-domain or out-of-domain, by comparing relative performance improvements on in-domain and out-ofdomain treebanks when augmenting the neural parsers with pre-trained language representations.

these aggregate F1 scores decompose along the structure of the tree, and are dominated by the short spans which make up the bulk of any treebank.


Large models are trained with a dropout rate of 0.4 until the validation performance stopped improving (250K steps for text8 and 150K steps for enwik8), and then further trained for 20K steps with a learning rate divided by 10.

in small models, we apply dropout with a rate of 0.3 to the attention and the feedforward ReLU activations.


The memory usage is also dominated by the attention layer as the attention span increase.

the span increases at the beginning of words and in the middle of composed words, e.g., to predict the "l" in "overlook".


From the results, we find our approach can effectively recognize important words to learn more informative news representations.

the features for news consists of tF-IDF features from its title, and one-hot vectors of its topic and subtopic.


We call this kind of user interests as shortterm interest.

since not all users can be incorporated in news recommendation model training (e.g., the new coming users), it is not appropriate to assume all users have long-term representations in our models in the prediction stage.


In this paper, we automatically create sentiment dictionaries for predicting financial outcomes.

14 The main finding supported by Table 7 is that 13 Experiments with multiple text variables in one regression (manual plus automatic experiments) are presented in the appendix.


In empirical finance, a correct sentiment classification decision is not sufficient -the decision must also be interpretable and statistically sound.

(i) Firm size: the log of the book value of total assets.


The same is true for C-tests that have been manipulated to a higher difficulty level, which the participant judge harder (z = −4.57, p < 10 −5 ) and rank higher (z = −3.86, p < 6 • 10 −5 ).

we conduct our experiments on English, but our strategies can be adapted to many related languages.


In a more detailed analysis, we find two main sources of problems demanding further investigation: First, the difficulty prediction quality when deviating from DEF and second, the increasing ambiguity in harder C-tests.

they find that cloze tests replacing entire words with a gap are more difficult than C-tests or multiplechoice tests.


The text similarity metric will be details in section 3.4.

none of the configurations discussed in this section include this step.


A beam size of 1 would indicate greedy search.

the clinical/medical domain recieves much attention for NLP (Shardlow et al., 2018;Yunus et al., 2019;Jahangir et al., 2017;Nawaz et al., 2012) and is well suited to the task of text simplification as there is a need for experts (i.e., clinicians) to communicate with non-experts (i.e., patients) in a language commonly understood by both.


Where one sentence crossed multiple categories, the highest (i.e., most severe) category was chosen.

in the simplification paradigm, the source language is difficult to understand language and the target language is an easier version of that language (in our case both English, although other languages can be simplified using the same architecture).


Data Statistics We build our dataset by acquiring all S&P 500 companies' quarterly earnings conference calls in 2017.

we find that training a deep LSTM network with audio data only can result in overfitting very quickly.


For example, comparing with tf-idf bag-of-words model at τ = 3, our MDRM reduces prediction error by 19.1% (1.371 vs. 1.695).

firms in the S&P 500 index encompass roughly three-quarters of the total U.S. market capitalization.


Therefore, the result that audio-only MDRM performs better than textonly MDRM (1.412 vs. 1.431) may need careful interpretation as we have to stop audio-only model training early to prevent overfitting.

they present a bidirectional contextual LStM (bc-LStM) framework for fusing multimodal features including audio, video and text.


However, every sentence in a conference call is spoken at a distinct time and in a particular order.

we use pre-trained word embeddings and calculate the arithmetic mean of word vector in each sentence as the sentence representation.


Experiments results are presented in Section 6.

we consider using the volatility of previous τ -days before conference call to predict the τ -days volatility following the conference call.


Therefore, observing that all sources propose their evidence and make their assertions independently, and taking θ = {H s , R s , Q s , W }, we can write the full joint as: Rs+Qs , we model it by minimizing their KL divergence.

; Nakashole and Mitchell, 2014), it was assumed that clean evidence and clear connections between evidence and conflicting claims are provided, disregarding the fact that NLP systems attempting to support these tasks are noisy.


Rao and Tetreault (2018) treat the formality of writing as a style, and create a parallel corpus for style transfer with sequence-to-sequence models.

in the image domain, for example, Chen et al.


To ensure that the style space does not contain content information, we design our final auxiliary loss, the BoW adversarial loss for content, denoted as "adv(c)."

while they do not directly work with "style," they regularize the content information, so that the style and content can be better disentangled.


However, they did not use cross-sentence context.

we also analyze our proposed approach by comparing against two other ways for modeling cross-sentence context (Section 5.1).


'avg' row reports average precision (P), recall (R), and F 0.5 score of 4 independently trained single models.

there are a number of studies recently on integrating cross-sentence context for neural machine translation (NMT).


In Table 3b, all systems use two human annotations as the reference to have a fair comparison to the human-to-human agreement in the last row.


This is because the former involves a single source of synthetic lexical semantic change, while the latter involves two sources (the removal of contexts associated with different senses of a recipient word, as well as the added contexts associated with the senses of a donor word).

discussion Table 2 shows that SGNS TR gains its performance advantage over SGNS AL mainly from a better classification of the stable words (0.37 vs. 0.57).


For this, we iterated over corpus C t such that for each token w and for each of its context words c within a symmetric window we extracted the word-context pair: (w t ,c) if w is a target word and (w,c) otherwise.

if a model captures stochastic fluctuations in the words' vectors instead of true semantic change, then such a shift in the distribution will be less prominent.


makes positive publicity for the tourist attraction in order to attract tourists, which contains a distinct positive emotion.

the reason behind lies in that most texts in the Essays domain only objectively state realities.


The word scared and disaster in red suggest two emotion dimensions.

we modified this approach by changing its activation function in order to return the dimensional emotion scores.


In this way, better weight information can be learned to represent words' importance for rating dimensional scores.

there is no extremely low r-value (<0.20) in any dimension of either Reader's emotion or Writer's emotion.


For feature pre-extraction, our setting on CMU-MOSI and IEMOCAP datasets are identical to that in (Poria et al., 2017b) 1 .

despite the effectiveness this type of methods have achieved, they give little consideration to acknowledging the variations across different portions of a feature vector which may contain disparate aspects of information and thus fail to render the fusion procedure more specialized.


• Instead of conducting fusion on a holistic level, we innovate to leverage a sliding window to explore inter-modality dynamics locally.

3D-CNN (Ji et al., 2013) is applied for visual feature pre-extraction.


This shows some potential positive link between the two levels of attention mechanism.

for CMU-MOSEI dataset, as shown in Table 3, the accuracy of HffN is lower than that of BC-LSTM and CAT-LSTM, but it achieves the highest f1 s-   core with slight margin.


Moreover, BC-LSTM is over 6 times faster than TFN in time complexity measured by FLOPs and the number of parameters is over 3 times smaller.

we incorporate two levels of attention mechanism into ABS-LSTM, i.e., Regional Interdependence Attention and Global Interaction Attention.


Actually, HFFN can adopt any fusion strategy in local fusion stage other than only outer product, showing high flexibility and applicability.

in our model, when conducting outer product, all three modalities are treated equally, which is probably not the optimal choice.


(2015) find over half of the 365 analysts impossible to exactly reconstruct their decision making process.

and then I think last quarter you had mentioned the top 100 advertiser spending was up 60% year-on-year on YouTube, wondering, if you could update us on that?


CNN on the other hand can encode long distance relations existing between concepts.

the embedding is extended by introducing an additional output layer (called the constrained context, CC, output layer) to update the embedding based on the fine-grained contexts.


The development of conversational AI thus depends on the use of both contextual and multimodal information.

we observe that the number of such emotion shifts in successive utterances of a speaker in a dialogue is very frequent: 4003, 427, and 1003 in train/dev/test splits, respectively.


Multimodal DialogueRNN also outperforms its unimodal counterparts.

figure 1 shows an example where speaker's emotion changes with time in the dialogue.


According to , ERC presents several challenges such as conversational context modeling, emotion shift of the interlocutors, and others, which make the task more difficult to address.

we enforce the following constraints: 1.


Similarly, we can get the probability of the end position along with its confidence score by: During training, since each sentence may contain multiple targets, we label the span boundaries for all target entities in the list T. As a result, we can obtain a vector y s 2 R (n+2) , where each element y s i indicates whether the i-th token starts a target, and also get another vector y e 2 R (n+2) for labeling the end positions.

to address these problems, we propose a span-based extract-then-classify framework, where multiple opinion targets are directly extracted from the sentence under the supervision of target span boundaries, and corresponding polarities are then classified using their span representations.


Definition 1 (T ransCap) Given a source document-level corpus C D and the learning task T D , a target aspect-level corpus C A and the learning task T A , TransCap aims to help improve the learning of the target predictive function f A (•) in T A using the knowledge transferred from T D .

aspect-level Sentiment Classification Traditional methods for sentiment classification (Nakagawa et al., 2010;Jiang et al., 2011;Taboada et al., 2011;Mohammad et al., 2013) mostly use machine learning algorithms to build sentiment classifiers with carefully extracted features, which take massive time and resources to collect.


Compared with CNN, it replaces the scalar-output feature detectors with vector-output capsules and has the ability to preserve additional information such as position and thickness.

formally, we apply a fusing convolution operation to the sentence embedding X with a kernel f a ∈ R d h ×K , and we get the aspect routing weight a i : where e a is the aspect embedding (or average embedding in the case of multi-word aspect), T a ∈ R 1×dw is a transfer matrix to map e a to a scalar value, and b a is bias.


Humans can easily identify the positive polarity towards aspect [screen].

(ii)"-S": We remove semantic capsules, and pass weighted feature capsules directly to class capsules.


Finally, we want the length of each semantic capsule u i to represent the probability that u i 's semantic meaning is present in the current input, so we use a nonlinear "squash" function (Sabour et al., 2017) to limit its length in [0,1] as In the original capsule network, there is only one classification task and it uses class capsules to denote classes and their lengths as classification probabilities.

both datasets have a fixed training/test split.


Besides, we removed a few instances with conflict sentiment labels as implemented in .

to enhance the persuasiveness of our experimental results, we also displayed the previously reported scores of MN (Wang et al., 2018) and tNet  on the same data set.


To enhance the persuasiveness of our experimental results, we also displayed the previously reported scores of MN (Wang et al., 2018) and TNet  on the same data set.

the performance of attentional neural ASC models is still far from satisfaction.


These mined information can be further used to refine the model training via a regularization term.

the existing attention mechanism in ASC suffers from a major drawback.


When implementing our approach, we empirically set the maximum iteration number K as 5, γ in Equation 3 as 0.1 on LAPTOP data set, 0.5 on REST data set and 0.1 on TWITTER data set, respectively.

at the k-th training iteration, we adopt the following steps to deal with (x, t, y): In Step 1, we first apply the model parameters θ (k−1) of the previous iteration to generate the aspect representation v(t) (Line 9).


Contextualized word embeddings compute a representation for a target word based on the specific context the word is used within a sentence.

even more so, clustering similar arguments is a demanding task, as fine-grained semantic nuances may determine whether two arguments (talking about the same topic) are similar.


We evaluated our models on the targeted sentiment dataset released by Mitchell et al.

instead of simply aggregating the objective functions of several different tasks, we suggest to decompose a single task into multiple inter-connected sub-tasks and then integrate the representation learned into a single module for the final decision.


We solve it as two sequence labeling tasks.

(2013) and  are also about performing two sequence labeling tasks, but they extract named entities and their sentiment classes jointly.


According to the comparison in (Reimers and Gurevych, 2017), using a CRF instead of a softmax classifier as the last layer can obtain a performance increase for tasks with a high dependency between tags.

the former task is to extract aspects of a product or service from an opinion document, and the latter is to identify the polarity expressed in the document about these extracted aspects.


This may be because debates in social media are much more informal compare to Oxford style debates and therefore, in the first round, the debaters may not necessarily present an overview of their arguments (talking points) they make through the debate.


Data was collected on Amazon Mechanical Turk using an open source framework for collaborative games (Hawkins, 2015).

furthermore, relative to state-of-the-art in Monroe et al.


The aforementioned Egrid CNN model is agnostic to entities' lexical properties, which are useful features for the task.

the model is trained to predict a documentlevel score along with word-level labels indicating the (predicted) GR type of dependents in the document.


This set of probabilities replaces the ground-truth labels in a standard discriminative model outfitted with a noise-aware loss function and trained on a sufficiently large data set.

the precision potentially lost in the generalization is offset by a larger increase in recall.


Inspired by models of "rational speech acts", we extend a neural generator to become a pragmatic speaker reasoning about uncertain object categories.

we test this automatically using a model of a "competent" listener, that knows the respective object categories.


This model, and its pragmatic decoding component in particular, is designed to avoid words that are specific to categories when there is uncertainty about the category of a target object, in favour of words that are not specific to categories like, e.g., colour or location attributes.

we assume that our listener has hidden beliefs about the category of the referent, that we can marginalize over as follows: As a simplification, we condition u only on c i , instead of P (u|c i , r).


(2018) to a reinforced policy gradient model by incorporating the reward associated with a sequence of coreference linking actions.

the final regularized policy gradient estimate is as follows: where λ expr ≥ 0 is the regularization parameter that controls how diverse our model can explore.


Although recent end-to-end neural coreference models have advanced the state-of-the-art performance for coreference resolution, they are still trained with heuristic loss functions and make a sequence of local decisions for each pair of mentions.

overall, our full model achieves the state-ofthe-art performance of 73.8% F1-score when using ELMo and entropy regularization (compared to models marked with * in Table 1), and our approach simultaneously obtains the best F1-score of 70.5% when using fixed word embedding only.


Results on the test set are shown in Table 1.

the refined representation g i is an elementwise weighted average of the current span representation and its direct antecedents.


The generative models perform quite well under this setting and applying them on top of our proposed architecture as the sentence encoder further enhances their performances, as illustrated in Table 3.

we select articles marked with an articlelevel rewrite flag from wikipedia, and we sample the non-rewrite articles randomly.


Previous work raised concerns that negative sampling cannot effectively cover the space of negatives for discriminative learning (Li and Jurafsky, 2017).

we designed the much harder Wiki-C and Wiki-D to check whether the coherence models capture cross-domain transferrable features.


Although Romanian and Moldavian are supposed to be hard to discriminate, since Romania and the Republic of Moldova share the same literary standard (Minahan, 2013), the empirical results seem to point in the other direction, to our surprise.

we describe the MOROCO data set in Section 3.


Furthermore, MOROCO provides a benchmark for the evaluation of dialect identification methods.

the results of CNN+SE are still much under those of the presence bits string kernel.


The first step aims at representing each lexemecategory pair (l, C) with a Bag Of Words (BOW).

once we have determined the number of examples to draw from each category, we sample the sentences according to their perplexity, which we compute with a Neural Language Model trained on WikiText103 (Howard and Ruder, 2018) 7 .


• Sense Assignment, which assigns a sense s of the lemma l to each lexeme-category pair (l, C) (Section 2.2).

in the last few years new approaches have been developed to mitigate the burden of knowledge acquisition by providing automatically or semi-automatically tagged corpora.


At first glance, BLI performance shows a weak and inconsistent correlation with results in downstream tasks.

furthermore, there are initiatives in typological linguistics research such as the ASJP database (Wichmann et al., 2018), which offers 40-item word lists denoting the same set of concepts in all the world's languages: https://asjp.clld.org/.


The mismatches in evaluation yield partial conclusions and inconsistencies: on the one hand, some unsupervised models (Artetxe et al., 2018b;Hoshen and Wolf, 2018) reportedly outperform competitive supervised CLE models (Artetxe et al., 2017;Smith et al., 2017).

bLI is generally cast as a ranking task.


(2) As reported by (de Cruys, 2014), the NNbased method performs very well on the pseudodisambiguation task.

(3) New York Times (NYT): The NYT (Sandhaus and Evan, 2008) dataset contains over 1.8 million news articles from the NYT throughout 20 years (1987 -2007).


Specially, for each entity pair (e i , e j ), we first concatenate the entity representations with relative distance embeddings, and then use a bilinear function to compute the probability for each relation type:  Evaluation Metrics.

intra-sentence relation instance) in Stage 4 of human-annotated data collection (Sec.


2.1), 41.4% are labeled as incorrect, while 61.8% inter-sentence relation instances are labeled as incorrect, indicating the wrong labeling problem is more challenging for weakly supervised document-level RE.

the second challenge lies in the high computational cost of modeling long documents and the massive amount of potential entity pairs in a document, which is quadratic with regard to entity number (19.5 entities on average) in a document.


We normalized synonyms with only slight morphological variation.

finally, the idiom vocabulary has 3,848 entries in total, and their frequency statistics on the whole corpus is shown in Table 5.


Finally, Anchor Words algorithm (Arora et al., 2013) is a fast and scalable alternative to traditional inference techniques based on non-negative matrix factorization.

this likely reflects the underlying data: Amazon product reviews are highly focused on specific products and features, and the generated topics naturally reflect these.


Like Chang (2010), we show the annotator a topics.


While human evaluation of topic coherence is useful, automated evaluations are easier to deploy.

following the pattern used by Newman et al.


We apply this human evaluation to a wide variety of models, and find that topic switch percent (or SWITCHP) correlates well with this human evaluation.

the local topic assignments can be bewildering.


But the CRF, effective when handling sequence labelling tasks such as NER, is not suitable for predicting mentions, as mentions can be nested.

as a rule of thumb, of the two best-known forms of crowdsourcing, microtask crowdsourcing us-ing platforms such as amazon Mechanical Turk is best to label small to medium size amounts of data in a short time, and for labelling data of no intrinsic interest.


Having a predefined ontology can simplify DST into a classification problem and improve performance (Henderson et al., 2014b;Zhong et al., 2018).

for example, parking, stars and internet slots are only appeared in hotel domain, and the food slot is unique to the restaurant domain.


For benchmark data, the models are trained using an early-stop strategy with maximum epoch set to 50 and patience (i.e., number of epochs with no improvement on the dev set for both SF and IC) to 6.

we proposed a series of end-to-end multi-task learning architectures, in which task, task group and task universe features are learned nonredundantly.


That output potentially underrepresents some task-specific aspects of the input.

the features learned in one task could be transferred to reinforce the feature learning of other tasks, thereby boosting the performance of all tasks via mutual feedback within a unified MtL architecture.


As described above, the output structure predicted by the model forms a tree that should correspond neatly to the input MR, barring some instances of ellipses (as with the date time argument in  Table 6).

this yields a tree accuracy of 97.6% and 95.4% on E2E and weather.


For example, relations 'scenario by' and 'author' are close neighbors in the KG embeddings space, thus allowing for zeroshot prediction in cross-domain tests, although their training examples usually appear in two separate domains: movie and book.

we report the % of cases when a top-k chosen fact was generated by each of the models ( Table 5).


Similarly, each candidate span is denoted as s ∈ S, where S is the set of all candidate spans.

oMCS contains 600K crowdsourced commonsense triplets such as (food, UsedFor, eat) and (wind, Capableof, blow to east).


Considering that neural models are intensive datadriven and normally restricted by data nature, they are not easily applied in a cross-domain setting.

coreference, we select different thresholds for different datasets to ensure the balance between precision and recall.


In this paper, we explore how to leverage different types of knowledge to better resolve pronoun coreference with a neural model.

even though our model outperforms the End2end model by 22.7 F1 from i2b2 to CoNLL, its overall performance is still poor, which might be explained by that the i2b2 is an in-domain dataset and the knowledge contained in its training data is rarely useful for the general (news) domain dataset.


14 A reviewer noted that this method may miss similar bias "types" that are achieved through different lexical items.

in summary, in this paper we make the follow-ing contributions: • Two novel methods to train NLi models that are more robust to dataset-specific artifacts.


We use SICK because our methods resulted in good gains on it (Table 2).

8 A similar situation arises in neural cryptography (Abadi To evaluate how well our methods can overcome hypothesis-only biases, we test our methods on a synthetic dataset as well as on a wide range of existing NLI datasets.


talman & Chatzikyriakidis (2018) recently reported similar findings using many advanced deep-learning models.


In a sense this is counter-intuitive, since p θ is being trained to unlearn bias, while p φ,θ is being trained to learn it.

does the same apply to natural NLI datasets?


We utilize BERT BASE (Devlin et al., 2019) in all of the BERT fine-tuning baselines and our GEAR framework.

we also feed the claim into BERT alone to obtain the claim presentation c. Note that we concatenate the evidence and the claim to extract the evidence representation because the evidence nodes in the reasoning graph need the information from the claim to guide the message passing process among them.


Moreover, relations between real-world entities are more often events than relations between common nouns.

a more flexible approach is to represent relations as lexicalized paths in dependency graphs (Lin and Pantel, 2001;Szpektor et al., 2004), sometimes with semantic postprocessing (Shwartz et al., 2017b) and/or retransforming into textual patterns (Nakashole et al., 2012).


The "voted" version corrects the reference, when one of the three scribes misses the annotation.

to reflect this application, we define multiple metrics.


As a baseline, we trained an LSTM tagger with a CRF output layer to predict targets consisting of the simple Cartesian product of symptom body systems and their status.

when two scribes pick different valid labels and the third misses them, the "voted" reference is not better than the single reference.


To evaluate the use of MI (or the lack of it) we follow general guidelines based on the MI literature (Miller and Rollnick, 2013).

conversations where the counselor mainly provides instruction and advice, and the client complies are regarded as low-quality interactions.


A core component of such studies of linguistic socialization has been the presence of external social influences that may impose linguistic norms-for instance, individuals are often able to observe the language used in posts by other members of an online community.

for instance, as counselors take more conversations, they are exposed to a broader range of texter situations beyond what the curriculum can account for; these experiences may further reinforce or alter their linguistic practices and fill inevitable gaps in the training.


For example, prior literature suggests that experience might not translate into consequential changes in counselor behavior.

far, we have tracked the evolution of counselors according to high-level characterizations of their linguistic diversification.


such social pressures are minimized in the counseling domain we consider.


We thank the CSIR Centre for Artificial Intelligence Research for financial assistance.

full (   The task of querying one of the trained models for an answer (or top five answers) to a question takes a second or two on an ordinary desktop computer.


Such questions correspond to examinations to access specialized positions in the Spanish healthcare system, and require specialized knowledge and reasoning to be answered.

some authors (Kaushik and Lipton, 2018; have discussed the tendency of research to develop datasets and methods that accomodate the data-intensiveness and strengths of current neural methods.


In contrast, our data set contains both same stance pairs, as well as cross stance pairs (i.e., one is supporting and the other is contesting the topic).

as described in §4 we build our data from pairs with the same-stance towards the topic, as well as cross-stance pairs, in which one argument supports the topic while the other opposes it.


Finally, z is multiplied by the hidden acoustic vector h a , and (1 − z) by the hidden lexical vector h l .

we first experiment with multimodal models and two attention mechanisms to assess the extent of the benefits that lexical information can provide.


Among others, emotions are encoded in both lexical and acoustic information where each modality contributes to the overall emotional state of a given speaker.

the attention mechanism not only emphasizes semantics but it also takes into account the acoustic features.


This observation, along with the model improvements, suggests that is possible to fuse information more efficiently.

recognizing emotions is not a trivial task because an emotional state can be easily shaped by many factors: context, word content, spectral and prosodic information, among others (Barbulescu et al., 2017).


Till now, two component Bi-LSTM at the upper layer are independent to each other.

with/without pair filtering indicates whether we adopt a pair filter after applying a Cartesian product in the second step.


Attention mechanism is then adopt to get a clause representation s i .

the mistakes made in the first step will affect the results of the second step.


This often happens because we were initially arguing some principle, and now we need to apply it to an unfamiliar example.

all in all this defines 12 similarity features.


For 24 CoPAs at least 50% were so labeled.

rhetoricians have characterized the art of convincing as starting from general and basic views, facts and opinions accepted by everyone (Perelman, 1971;Kock, 2009).


In the complete dataset, 87% of the motions belong to at least one CoPA, and on average each motion belongs to 1.95 CoPAs.

importantly, the CoPAs capture different facets 9 For technical reasons we trained and evaluated the RNN method using 3-fold cross-validation.


Moreover, since we showed both CoPA claims to the annotators we checked whether claims whose stance opposed that of the speaker were ever marked positive.

in the vast majority of cases the claim was marked as implicit in the speech -according to the annotation only 10% of the speeches contain a CoPA-claim explicitly, and only 5% of the pairs are labeled as an explicit mention.


These cases are usually not seen in summarization datasets containing clusters of documents discussing single topics.

in this section we discuss results that we obtained for multi-document summarization and determining redundancy between sentences.


We also used ROUGE recall on Mainichi with a length constraint, which is the length of a reference summary.

in MRT, unlike MLE, the probability of a word at each step is calculated using previously generated words as in the test phase.


In contrast, each Q θ (y + |x) for overlength summaries decreases be-cause∆(y, y + ) > 0.

the meaning of each item in the first column is the same as table 2.


There are mainly two research directions: model architectures and optimization methods.

our research focuses on a global optimization method.


Furthermore, the summary should be non-redundant (i.e., high entropy).

we formulate simple requirements that f should satisfy: • Additivity: (I is the information measure from Shannon's theory (Shannon, 1948)) • Normalization: The first requirement states that, for two semantic units equally represented in the sources, we prefer the more informative one.


Formally, the amount of new information contained in a summary S is given by the crossentropy Inf (S, K) = CE(S, K): For Relevance the cross-entropy between S and D should be low.

if c i is the i-th bigram selected in the summary and w i is its weight computed from D, then: LexRank: (Erkan and Radev, 2004) This is a well-known graph-based approach.


This fits within the general optimization framework for summarization (McDonald, 2007;Peyrard and Eckle-Kohler, 2017b;Peyrard and Gurevych, 2018) The background knowledge and the choice of semantic units are free parameters of the theory.

they might serve as convenient approximations.


The above described greedy obfuscation effectively hindered verification in our pilot experiments.

(2012) found that machine translation is ineffective and due to its blackbox character also rather uncontrollable.


The OpenReview links in the table above give the full review texts.

we also extract features from the review sentiment representation x rs via another MLP (MLP Senti).


https://openreview.net/forum?id=S1dIzvclg&noteId= H1LYxY84l The author propose to use a off-policy actor-critic algorithm in a batch-setting to improve chatbots.The approach is well motivated and the paper is well written, except for some intuitions for why the batch version outperforms the on-line version (see comments on "clarification regarding batch vs. online setting").The artificial experiments are instructive, and the real-world experiments were performed very thoroughly although the results show only modest improvement.

the major hurdle lies in to decide the acceptance and rejection of the manuscripts based on the reviews received from the reviewers.


Several recent studies have considered to exploit a longer context information that spans multiple sentences (Mikolov and Zweig, 2012;Wang and Cho, 2016;Ji et al., 2016;Liu and Lane, 2017;Xiong et al., 2018).

we also investigate the effect of the number of utterance history being encoded.


As the first step, we ascertain if the volume of figurative usage of symptom words warrants such attention.

this set is called the 'literal usage representation'.


Table 1: SEQ vs. CAMB system results on words only and on words and phrases systems achieve similar results: the SEQ model beats the CAMB model only marginally, with the largest difference of +1.05% on the WIKINEWS data.

for phrase-level binary prediction, we consider the phrases contained within the dataset.


Thus, exploiting the news topics has the potential to improve the representations of news and users.

the topic information of news is critical for learning accurate news and user representations for news recommendation.


The word arrangement in a verse should result in a sequence of syllables which adhere to one of the prescribed meters in Sanskrit Prosody (Scharf et al., 2015).

kāvya guru performs the best when it uses syllable level encoding of input, as compared to character level transliteration schemes such as IAST 8 or SLP1 9 .


Keyword detection mainly focuses on finding important nouns or noun phrases.

we model emphasis using plain text with no additional context from the user or the rest of the design.


There are four stance categories: supporting(S), denying(D), querying(Q) and commenting(C), i.e.

many previous studies give "fake news" a stricter definition: fake news is a news article published by a news outlet that is intentionally and verifiably false (Shu et al., 2017;Zubiaga et al., 2018).


The translation model evidently goes with a linguistically plausible guess, ignoring the source side.

it is therefore instructive to consider this hypothetical situation.


q, k, v denote parametrized transformations (e.g.

position information is crucial in natural language, so a mechanism to represent such information in the model is needed.


We argue that encoder and decoder can share the same parameters if necessary.

in this work, we propose a method for Multi-NMT to boost the accuracy of the multilingual translation, which better fits on both lowresource scenario and zero-shot scenario.


To tie source and target representations more strongly backtranslation is also performed at each iteration (Sennrich et al., 2016), and synthetic parallel data is generated, by translating sentences to the other language using the system's current parameters, and then running a training step using the backtranslation as input to predict the original sentence.

we experienced a slight drop of recall due to our stricter approach for the mining process.


UBWE is a fundamental component of UNMT.

3) Regarding the two proposed methods, UBWE adversarial training performed slightly better than UBWE agreement regularization by BLEU score, although UBWE agreement regularization was better at maintaining UBWE accuracy.


This pretrained UBWE is then used to initialize the word embedding in the encoder and decoder of UNMT.

the selection of λ influences the role of the L BW E across the entire UNMt training process.


As a result, the performance of Ja-En UNMT is too poor to further empirical study if only pure monolingual data are used.

finally, the back-translation process is optimized by minimizing the following objective function: where P L 1 →L 2 (P L 2 →L 1 ) denotes the translation probability across two languages.


The empirical findings show that the performance of UNMT is significantly affected by the performance of UBWE.

this work was partially conducted under the program "Promotion of Global Communications Plan: Research, Development, and Social Demonstration of Multilingual Speech translation technology" of the Ministry of Internal Affairs and Communications (MIC), Japan.


Only with LM pretraining, the zero-shot NMT has already closed the gap between the performance and that of the strong pivot-based baseline for datasets.

then, En tokens starts to appear in the output sentence of ZS, and it totally shifts to En eventually.


The best practise for zero-shot NMT is to make z and s conditionally independent given the source sentence.

note that z is not necessarily equal to the language id j.


Our key insight is that we can use syntactic information as a proxy to the learned discrete latent space of the LT.

to retain information about the order of the input words, the transformer also includes positional encodings, which are added to the source word embeddings.


One hypothesis is that it does so to correct mistakes made by the parse decoder.

instead of producing a sequence of latent discrete variables, our model produces a sequence of phrasal chunks derived from a constituency parser.


The possible functionality of lower layers might be constructing gradually better contextual representation of the word at each position as suggested in recent contextualized embedding works (Peters et al., 2018;Devlin et al., 2018;Radford et al., 2019 the AERs become better while more unambiguous representations of the corresponding word are formed.

one possible reason is that using zero embedding in inference is exactly the same way as dropout in training, making the trained parameters perform well in inference.


In this way, the computation in equation 9 is much faster than the Monte Carlo sampling approach involving multiple samples.

as our goal is not to achieve the best word alignment but to focus on that captured by an NMT model, we only consider these features completely learned in NMT.


In NAT, the decoder generates all words simultaneously instead of sequentially.

we introduce a knowledgeable AT demonstrator to supervise each decoding state of NAT model.


This model utilizes attention mechanism to encode the context relations for predicting target relations.

we need to modify their dataset: (1) We added reversed edges if they are missing from a given triple, e.g.


Pretraining methods like BERT (Devlin et al., 2018) or Elmo (Peters et al., 2018) have proved to be extremely helpful in MRC tasks.

additionally, we construct a newly developed dataset RESUME in Chinese, which requires multi-step reasoning to construct entity dependencies, as opposed to the single-step dependency extraction in the triplet exaction in previous datasets.


These datasets do not involve hierarchical entity relations as in our previous Musk example, which are prevalent in real life applications.

as the multi-turn Qa proceeds, we progressively obtain the entities we need for the next turn.


Another facet that most existing studies fail to consider is that there may be multiple relation mentions in a given sentence if it contains multiple entities.

maxpooling is applied to each row in each layer of the stack, and the dimension of the output is equal to the total number of filters.


To see how our approach would perform as the degree of data imbalance varies, we used the same random seed to sample negative instances extracted from the Chinese corpus of ACE 2005 to add to the positive instances with different negative/positive instance ratios of 1:0.5, 1:1, 1:5, 1:10 and 1:15.

we further incorporate the embeddings of character-wise/word-wise BIO tag from the named entity recognition task into character/word embeddings to enrich the input representation.


the [british] GPE:♥♣♠ GEN-AFF-2:♥♣♠ [arm] ORG:♥♣♠ PART-WHOLE-1:♥♠|GEN-AFF-1:♥♣♠ of french distributors [pathe] ORG:♥♣♠ PART-WHOLE-2:♥♠ to show four releases .

for S2, the "NN" does not detect PART-WHOLE relation while the "GCN" correctly find it.


In general, the performances on four tasks are insensitive to the number of GCN layers 7 .

in this paper, we define joint entity relation extraction into two sub-tasks: entity span detection and entity relation type deduction.


The state-of-the-art solutions for extracting multiple entity-relations from an input paragraph always require a multiple-pass encoding on the input.

summing up the vectors confuses this information.


Because in real-world applications, whose input paragraphs dominantly contain multiple pairs of entities, an efficient and effective solution for MRE has more important and more practical implications.


Then, a maxpooling along the text length combines all features into a fixed-size representation.

our model for p(r | s) follows current state-ofthe-art practices for supervised relation extraction by using a piecewise convolutional neural network (PCNN, Zeng et al., 2015).


In a supervised setting, discriminative approaches, such as deep neural network classifiers, have demonstrated substantial improvement.

the second type of mistake is when a predicted cluster corresponds to two gold ones (low precision).


Then, the algorithm iteratively merges the most frequently co-occurring tokens into a new token until a predefined vocabulary size is reached.

the distribution over the target tokens is modeled using the previously defined transformer model as follows: where h L is the sequence of states after the final layer L, W e is the embedding matrix, and θ are the model parameters that are optimized by stochastic gradient descent.


data programming (Ratner et al., 2016(Ratner et al., , 2017 fuses dS-based labels and manual relation patterns for reducing noise.


In a bag of sentences that mention the same entity pair, it assumes that at least one sentence expresses the relation.

all the baselines for noise reduction were implemented with the source codes released by their authors.


Instead, we use a recently released sentence-level test set (Ren et al., 2017) for evaluation.

it is based on a BiLSTM with attention mechanism and utilizes attention regularization to force the model to attend the corresponding relation patterns.


(2014);  mixed a small set of crowd-annotated labels with purely DS-generated noise labels.

given the raw input representation x and the corresponding action vector a, we define the reward as follows: where the total reward is composed of two parts: one is the log-likelihood term to pursue the high prediction confidence and the other is the sparse ratio term to induce sparsity in terms of retained tokens.


For some tasks with fewer noises, including R 0 , R 7 , R u 7 and R u 9 , our method can obtain small improvements.

we define our objective as: where x denotes the input representation of the instance s. By taking the derivative of J(Θ) with respect to Θ, we can obtain the gradient Besides, we utilize the -greedy trick to balance exploration and exploitation.


The strategy of finding entities is to first generate all the word segments as entity proposals and then estimate the probability of each proposal as being an entity or not.

in this work, we propose a novel neural framework named MGNER for Multi-Grained Named Entity Recognition where multiple entities or entity mentions in a sentence could be non-overlapping or totally nested.


The hidden dimensions of the Word LSTM layer D wl , the Sentence LSTM layer D sl , the Entity LSTM layer D el and the Attention LSTM layer D ml are all set to 300.

these models are designed explicitly for recognizing nested named entities.


An easy way to model context words is to concatenate all the word representations or average them.

the goal of the Classification Network is to classify entity candidates into different categories, and thus it requires a fixed embedding size.


The assumption is that entities and their counterparts in different KGs should have similar structures and thus similar embeddings.

each channel encodes KGs via different relation weighting schemes with respect to self-attention towards KG completion and cross-KG attention for pruning exclusive entities respectively, which are further combined via pooling techniques.


It outperforms the model without gazetteers by 1.1 points in terms of F1 score.

massive wrongly matched entities can be generated with the use of gazetteers.


Note that, for both cache pointer and dynamic evaluation, we coarsely tune the associated hyperparameters on the validation set.

with negligible overhead in the number of parameters and training time, our Past Decode Regularization (PDR) method improves perplexity on the Penn Treebank dataset by up to 1.8 points and by up to 2.3 points on the wikiText-2 dataset, over strong regularized baselines using a single softmax.


For each time step t, a neural network builds a representation h t from the history that is used to predict the upcoming character.

this technique was previously considered in the context of automatic speech recognition (Wang et al., 2017) or to copy tokens from the input for code generation (Ling et al., 2016).


Neural sequence-tosequence models with soft attention, which are non-monotonic, often outperform popular monotonic models.

if we look at the data in grapheme-to-phoneme conversion, named-entity transliteration, and morphological inflection-examples are shown in Fig.


Answers (Yahoo, 10 labels, 1.4M/60K for training/testing) and Yelp Review Polarity (YelpPolar, 2 labels, 0.56M/38K for training/testing) from Zhang et al.

we employ the public available rnet model (wang et al., 2017) 6 in Tensorflow.


All our implementation is available at github.com/haejupark/soft.

inspired by the limitation of hard-coding approaches, our proposed model, TARS, begins with FS model but progressively adapts to task characteristics, as shown in Figure 1c.


As human annotation for such perturbations is expensive, an automated approach (Alzantot et al., 2018) was studied for English, to generate semanticpreserving adversaries that fool well-trained sentiment analysis and NLI models with success rates of 97% and 70%, respectively.

compared to the best performing MTL baseline, cASE achieves an improvement of 1.7% and 9.5% in Bulgarian (bg) and Urdu (ur) languages respectively.


The parameters of the network are trained to minimize the loss L task of the predicted and true distribu-tions on all the tasks as follows: where L(ŷ k , y k ) denote a typical cross-entropy loss for each task k. As overviewed in Section 1, the success of transfer learning depends on the sharing scheme in latent feature space.

we also report the results of XNLI baselines (Conneau et al., 2018), a supervised cross-lingual MTL model that combines the L adv loss using pseudoparallel data (Liu et al., 2017), the multilingual BERT (Devlin et al., 2018), and the recent work of (Artetxe and Schwenk, 2018).