The sampled method, similar to HTLE, uses the HDP model to assign topics to word occurrences during testing.

many tasks can benefit from using multiple representations per word to capture polysemy (Reisinger and Mooney, 2010).


The analogies evaluated here have the benefits of being easy to compile and evaluate, but they represent only one specific subset of TWAs.

as time depth increases, its accuracy drops sharply, while the embedding-based method remains effective, as illustrated in Figure 1.


For each relation, we generate a question per word pair consisting of the word pair plus 10 random word pairs from the same relation.

in this study, we evaluate these various techniques when used to generate Arabic word embeddings.


Since connotation of a word arises from the context in which the word is used, we want to ensure the translated connotation frames are used in similar contexts.

for the most frequently mentioned named entities, we compute heatmaps of the expected perspective being expressed towards that entity.


It shows for each annotation set the split-half reliability using the full set of annotations (10N per half-set) as well as partial sets obtained by choosing k rs annotations per term for rating scale (where k rs ranges from 1 to 10) or k bws annotations per 4-tuple for BWS (where k bws ranges from 1 to 5).

• Fixed granularity: in some cases, annotators might feel too restricted with a given rating scale and may want to place an item inbetween the two points on the scale.


Best-worst scaling (BWS) is an alternative method of annotation that is claimed to produce high-quality annotations while keeping the required number of annotations similar to that of rating scales.

observe that positive phrases that include a negator (the class that diverged most across BWS and rating scale), is also the class that has an extremely low SHR when annotated by rating scale.


The sparse prior work in natural language annotations that uses BWS involves the creation of datasets for relational similarity (Jurgens et al., 2012), word-sense disambiguation (Jurgens, 2013), and word-sentiment intensity (Kiritchenko and Mohammad, 2016a).

we set up an experiment that compares BWS and RS in terms of how similar the results are on repeated independent annotations.


are the true label and prediction probabilities for task A, N denotes the total number of training samples and C (A) is the total number of classes associated with the task A.

in this paper, we present a demographic classifier for gender, age, political orientation and location on Twitter.


We select three simulation types from Profile Injection Attack (Williams and Mobasher, 2006), i.e., random, average, and love/hate.

our model performs better than baselines in aggregating noisy labels, especially when the noise scale becomes large.


Our rerank model uses the same attentive Seq2Seq model to score candidate answers with regarding to an input question.

we show the P top 1 result in Table 1, which suggests that the attentive Seq2Seq model with s Mean-Prob has the best performance.


(c) Pretraining on SQuAD and SQuAD-T with finetuning (fourth and fifth row) significantly outperforms (by more than 5%) the highest-rank systems on WikiQA.

topWe see high correspondence between same word from question and context such as senator and john, in SQuAD-pretrained model, but the SQuAD-tpretrained model fails to learn such correspondence.


To evaluate our model, we randomly selected 200 sentences as test set and had Pr.

this cycle gradually improves both system and human annotators.


We believe that the difference in performance is not due to the attention mechanism being ineffective, but because Shen and Lee (2016) treat the classification of each utterance independently.

the Switchboard Dialog Act corpus (Jurafsky et al., 1997)  Baselines.


Word n-gram features have as value their weighted frequencies using the Log-Entropy weighting technique.

we aim to analyze the impact of spelling errors on identifying the native language of the author.


We would like to thank EST for making the TOEFL11 dataset public and Fondazione Bruno Kessler for providing the facilities to conduct the related experiments.

we use linear SVM implemented by scikit-learn package (Pedregosa et al., 2011) to perform the multi-class classification.


Table 1: The features used in the reranker.

that the disfluency is observed "after" the fluent repair in a backward language model is helpful for recognizing disfluencies.


The frequencydomain equivalents for these operations are summarized in Table 1.

moreover, ξ 0 (and ξ n/2 if n is even) are real values.


Several approaches have proposed to improve decoding speed: 1.

we can overlap the computation with source side forward propagation.


We find that our proposed chunk-based NMT model also obtains considerable accuracy improvements on German-English.

in this way, our model only extracts source context once for a chunk, and the words in one chunk will share the same context for word generation.


For example, the CRF-BiLSTM POS tagger obtained the state-of-theart performance on Penn Treebank WSJ corpus (Huang et al., 2015).


Finally, note the difference between CCA and Cluster methods for learning word embeddings which arise from the differing quality of distant supervision between the languages.

in low-resource languages, these models are seldom used because of limited labelled data.


Actually, greedy search achieves very high accuracy.

in the context of neural word segmentation, it will generalize poorly due to the severe word sparsity in Chinese.


we consider automatic features that can be computed on the endings and evaluate the accuracy of relying solely upon each feature as a classification rule.


Learning a semantic parser involves mapping language phrases to KB constants, as well as learning how language composition corresponds to logical form composition.


Such an unsupervised representation is empirically validated via semantic textual similarity tasks on 19 different datasets, where it outperforms the sophisticated neural network models, including skip-thought vectors, by 15% on average.

the principle of sentence representations is that similar sentences should have similar representations.


A sentence contains rich syntactic information and can be modeled through sophisticated neural networks (e.g., convolutional neural networks (Kim, 2014;Kalchbrenner et al., 2014), recurrent neural networks (Sutskever et al., 2014;Le and Mikolov, 2014;Hill et al., 2016) and recursive neural networks (Socher et al., 2013)).

given a longer sentence, the effect of useful word vectors become smaller and thus the average vector is less reliable at capturing the semantics.


Following the previous work, we use PART I as training set, PART II as development set, and PART III as test set.

it does not apply to Chinese social media text summarization, where texts are comparatively short and often full of noise.


Future work may focus on utilizing more sophisticated discourse and pragmatics features, and inferring degrees of credibility.

the social graph sub-network is a simple feed-forward network that takes one-hot vectors of user interactions, e.g.


On the other extreme is satire, e.g., @TheOnion, where the writer's primary purpose is not to mislead the reader, but rather entertain or criticize .

bias cues Inspired by earlier work on identifying biased language on Wikipedia (Recasens et al., 2013) we extract hedges (expressions of tentativeness and possibility) (Hyland, 2005), assertive verbs (the level of certainty in the complement  Figure 2: Neural network architecture for news classification fused with social network and linguistic cues.


Many of them used dependency parsing or constituency parsing for argument detection or elementary discourse unit (EDU) segmentation to infer the relation between them.

using only n-grams as features for the statistical model without PTB-style Tweet POS tags gives a relatively large drop (0.02) from F1.


When this model incorporates immediate morphological context, it (BiLSTM*) performs at 90.92% and beats the HMMCG baseline.


(2015) by extracting the children of the third token in the stack.

we use the fine-grained predicted POS tags provided by the organizers, and evaluate the labeled attachment scores (LAS) including punctuation.


We therefore experiment with four combined models in two groups: (1) randomly initialized word embeddings (LSTM+WORD vs. CNN+WORD), and (2) pre-trained word embeddings (LSTM+W2V vs. CNN+W2V).

both LSTM and B15-LSTM perform higher than baseline only on the three agglutinative languages (Basque, Hungarian, and Korean), and lower than baseline on the other six.


In monolingual parsing over PRED tags (Table 1), we achieve an identical average UAS with lin- Figure 3: Parsing accuracy (UAS) in relation to linear jackknifing over 26 languages, with two sources of test set POS tags.

we do not experiment with gold POS tags in the test sets.


In fold terminology, this corresponds to dividing the training set into equal parts of size 1 n , training on n−1 n ths of the training set and testing on the remaining 1 n th.

this result in itself justifies the usage of jackknifing as adaptation for the monolingual setting, provided that it is not indiscriminate.


The hyperpameter α affects the scale of the loss function; therefore, we adjust the learning rate appropriately for each α.

we use the maximum similarity score (Equation 6), denoted as MAXSIM.


From Table 1 we see that, while our method is consistently better than both the additive baseline and nonce2vec, removing stop-words from the additive baseline leads to stronger performance for more sentences.

following the success of recent neural (Mikolov et al., 2013) and matrixfactorization (Pennington et al., 2014) methods, researchers have sought to extend the approach to other text features, from subword elements to n-grams to sentences (Bojanowski et al., 2016;Poliak et al., 2017;Kiros et al., 2015).


We report the Spearman ρ (as described above) at each sample size, averaged over 100 trials obtained by shuffling each rare word's 255 contexts.

none of these datasets can be used directly to measure the effect of word frequency on embedding quality, which would help us understand the data requirements of our approach.


To efficiently compute the triples, we have used the Zettair 3 retrieval engine.

traditionally, relation extraction systems have relied on a variety of linguistic features, such as lexical patterns, part-of-speech tags and dependency parsers.


For efficiency reasons, in practice, we only consider a small subset of all context words j for which x ij = 0, which is similar in spirit to the use of negative sampling in Skip-gram (Mikolov et al., 2013b).

for efficiency reasons, the counts of the form y ij * , y i * k and y * jk are pre-computed for all word pairs, which can be done efficiently due to the sparsity of co-occurrence counts (i.e.


To analyze the benefit of our proposed word embedding variant, Table 2 shows the results that were obtained when we use standard word embedding models.

while this strategy is often surprisingly effective (Hill et al., 2016), it is sub-optimal for two reasons.


for each constraint (w i , w j ), we create K = 4 corresponding negative examples for both w i and w j , resulting in micro-batches with 2K + 1 = 9 training instances.


We decompose X ⇒ Y into two phases for training two translation models.

(1.00) Reference en conclusión , la erradicación de la pobreza necesita la voluntad y compromiso políticos .


In this triangular architecture, Z is taken as the intermediate latent variable, and translation models of Z are jointly optimized with a unified bidirectional EM algorithm under the goal of maximizing the translation likelihood of (X, Y).

t-S cannot improve the performance obviously (only about 0.2 BLEU) on RNNSearch because it fails to essentially double training data via the teacher model.


As the final vocabulary V contains all individual characters in the corpus, character-based segmentation is also included in the set of segmentation candidates S(X).

while these sequences encode the same input "Hello world", NMT handles them as completely different inputs.


l = 1 is aimed at a pure comparison between BPE and the unigram language model.

we should note that the subword regularization is mandatory for n-best decoding and the BLEU score is degraded in some language pairs without subword regularization (l = 1).


Most recently, the Transformer model (Vaswani et al., 2017), which is based solely on a selfattention mechanism (Parikh et al., 2016) and feed-forward connections, has further advanced the field of NMT, both in terms of translation quality and speed of convergence.

we compensate for the resulting longer per-step time with increased data parallelism (more model replicas), so that the overall time to reach convergence of the RNMT+ model is still comparable to that of GNMT.


This learning rate schedule is motivated by a similar schedule that was successfully applied in training the Resnet-50 model with a very large batch size (Goyal et al., 2017).

our focus on a standard single-language-pair translation task leaves important open questions to be answered: How do our new architectures compare in multilingual settings, i.e., modeling an interlingua?


We also computed the number of floating point operations (FLOPs) in the model's forward path as well as the number of total parameters for all architectures (cf.

we first increase the learning rate linearly during the number of warmup steps, keep it a constant until the decay start step s, then exponentially decay until the decay end step e, and keep it at 5 • 10 −5 after the decay ends.


The final representation is the concatenation of the context and mention representations: Label Prediction We learn a type label embedding matrix W t ∈ R n×d where n is the number of labels in the prediction space and d is the dimension of r. This matrix can be seen as a combination of three sub matrices, W general , W f ine , W ultra , each of which contains the representations of the general, fine, and ultra-fine types respectively.

despite our efforts to annotate a comprehensive type set, the gold labels still miss many potentially correct labels (example (a): "man" is reasonable but counted as incorrect).


Inferring such types for each mention (John, he) is not possible given current typing models that only predict relatively coarse types and only consider named entities.

they focus on named entities, and data has been challenging to gather, often approximating gold annotations with distant supervision.


We further evaluate our approach in two different settings: (i) ComplEx-NNE that imposes only the Non-Negativity constraints on Entity representations, i.e., optimization Eq.

we examine two types of constraints: (i) non-negativity constraints on entity representations and (ii) approximate entailment constraints over relation representations.


The compatibility of entity pairs is measured using entry-wise product between head and complex conjugate of tail entity vectors.

in this section, we explore whether the type of the score function optimized during the training has any effect on the geometry of the resulting embedding.


Entity Embeddings: As seen in Figure 4 (left), the conicity of entity vectors increases as the number of negative samples is increased for multiplicative models.

to the best of our knowledge, this is the first study of its kind.


Modeling this interaction, we believe, is the reason for the superior performance of SWAP-NET in our experiments.

let α s kj denote the probability of selecting the k th input sentence at the j th decoding step of sentence decoder: and let α w ij denote the probability of selecting the i th input word at the j th decoding step of word decoder: both conditional on the corresponding switch selection.


SWAP-NET predicts both key words and salient sentences, that are subsequently used for extractive summary generation.

sentences with words selected by the decoder are given higher importance.


Therefore, we leverage the strong rewriting ability of the seq2seq model to generate more faithful and informative summaries.

since a soft template r can also be regarded as a readable concise sentence, we use the same BiRNN encoder to convert it into hidden states In Retrieve, the template candidates are ranked according to the text similarity between the corresponding indexed sentences and the input sentence.


To sum up, our model demonstrates promising prospect in generation diversity.

a Recurrent Neural Network (RNN) encoder is applied to convert the input sentence and each candidate template into hidden states.


Most previous seq2seq models purely depend on the source text to generate summaries.

we employ a bidirectional Recurrent Neural Network (BiRNN) encoder  to read x and r. Take the sentence x as an example.


Several influential factor analysis studies have shown that the three most important, largely independent, dimensions of word meaning are valence (positiveness-negativeness/pleasuredispleasure), arousal (active-passive), and dominance (dominant-submissive) (Osgood et al., 1957;Russell, 1980Russell, , 2003.

these include: inconsistencies in annotations by different annotators, inconsistencies in annotations by the same annotator, scale region bias (annotators often have a bias towards a portion of the scale), and problems associated with a fixed granularity.


Given the semantic overlap between prepositions and possessives (the hood of the car vs. the car's hood or its hood), we analyze them using the same inventory of semantic labels.

function prediction is consistently more accurate than role prediction, with roughly a 10-point gap across all systems.


The framework was realized in version 3.0 of our comprehensively annotated corpus, STREUSLE 3 (Schneider et al., 2016).

only targets with a semantic supersense analysis involving labels from figure 2 were included in training and evaluation-i.e., tokens with special labels (see §4) were excluded.


Annotators had the option to mark units as false positives using special labels (see §4) in addition to expressing uncertainty about the unit.

(5a) would be annotated ORGROLE;LOCUS and (5b) as ORGROLE;BENEFICIARY to expose their common truth-semantic meaning but slightly different portrayals owing to the different prepositions.


As seen in the previous two experiments, EASL achieves higher Spearmans correlation on ranking MT systems with smaller number of annotations than the baseline method (DA), 21 This is the same setting as WMT14, WMT15, and WMT16 (Bojar et al., 2014(Bojar et al., , 2015, although they used TrueSkill (Gaussian) instead of EASL to rank systems.

direct assessment or direct annotation (dA) is a straightforward method for collecting graded response from annotators.


Overall, all the three methods achieve strong correlation above 19 We stress that the oracle here derives from subjective annotations: it does not necessarily reflect the true latent scalar values for each instance.

since boundedness is essential for the scalar annotation we aim to model, we propose a bounded variant which parameterizes each instance by a beta distribution as illustrated in Figure 1(d).


The views and conclusions contained in this publication are those of the authors and should not be interpreted as representing official policies of the U.S. Government.

the amount of update by a pairwise judgment depends only on the distance between instances, not on the distance from the bounds (i.e., 0 and 1).


This allows us to choose instances to annotate by order of uncertainty for each instance, and as in RA, the match quality (Eqn.

we propose a variant of TrueSkill by changing the latent distribution from a Gaussian to a beta, using a heuristic algorithm based on TrueSkill for inference.


After filtering, the resultant dataset contains 26,525 OPs along with 305,475 relatively high quality root replies.

here we describe our decoding strategy on the argument decoder.


Table 5 shows that our model with separate decoder and attention over keyphrases produce significantly more informative and relevant arguments than seq2seq trained without evidence.

this may lead to suboptimal output for text generation (Wiseman and Rush, 2016), e.g., one beam often dominates and thus inhibits hypothesis diversity.


Besides sports, in some tasks the best results were obtained for religion and science, both of which share the characteristic of dealing with very specific topics.

based on the new corpus, we define the following eight counterargument retrieval tasks of different complexity.


Such considerations make arguments more balanced (see above).

to avoid exploiting this corpus bias, no approach in our experiments captures length differences.


For a given attribute a represented as key-value pair, we use paragraph2vec (Le and Mikolov, 2014) type of embedding network to learn attribute embedding.

experiments on link prediction and entity linkage demonstrate substantial improvements over the state-ofthe-art relational learning approaches.


Improved inference of the latent boxes, either through better optimization or through Bayesian approaches is another natural extension.

it is not generally tractable to integrate high-dimensional cones under arbitrary non-product measures.


It is important to note that this procedure is orthogonal to the individual models used for generation and parsing.

overall, because our architecture can work with general graphs, it is straightforward to add linguistic biases in the form of extra node and/or edge information.


Interestingly, we found different trends when analysing the CHRF++ numbers.

we leave further investigation of this phenomena for future work.


This suggests that these C near tokens are more often copied from nearby context, than inferred from information found in the rough semantic representation of long-range context.


Many notable approaches have explored incorporation of background knowledge into the training of learning algorithms.

for our running example, y corresponds to the positive class of important emails.


(2015) in introducing a neural representation to the NLP literature.

we find that the performances of both S-LSTM and BiLSTM decrease as the sentence length increases.


Adding one additional sentence-level node as described in Section 3.2 does not lead to accuracy improvements, although the number of parameters and decoding time increase accordingly.

as can be seen from the figure, when the number of time steps increases from 1 to 11, the accuracies generally increase, before reaching a maximum value.


these comparisons, we mix all training instances, order them by the size, and put them into 10 equal groups, the medium sentence lengths of which are shown.

i t i controls information from the input x i ; l t i , r t i , f t i and s t i control information from the left context cell c t−1 i−1 , the right context cell c t−1 i+1 , c t−1 i and the sentence context cell c t−1 g , respectively.


The step number for S-LSTMs is set to 9.

these semi-supervised learning techniques are orthogonal to our work, and can potentially be used for S-LSTM also.


Due to the complexity of the DNNs they explain, these methods are necessarily approximations and come with their own sources of error.

the black box approach is maximally general, but insensitive to the magnitude of evidence found in Z n .


Across all datasets, we see only a small number of entity-problem-only examples.

since even the oracle baseline did not have much success on these question splits, and since the baseline was able to handle the small Restaurants dataset.


These poor results reflect the fact that model selection was based on the score on the UCCA development set, and that the model parameters dedicated to auxiliary tasks were very limited (to encourage using the shared parameters).

in case of reentrancy, an arbitrary parent is marked as primary, and the rest as remote (denoted as dashed edges in Figure 3).


In order to reach comparable analyses cross-linguistically, UD often ends up in annotation that is similar to the common practice in semantic treebanks, such as linking content words to content words wherever possible.

preliminary experiments using our approach produced promising results on each of the tasks' respective English development sets, when treated as a single task: 67.1% labeled Smatch F 1 on AMR (adding a transition for implicit nodes and classifier for node labels), 79.1% labeled F 1 on DM, and 80.1% LAS F 1 on UD.


Encoding of words is perhaps the most important step towards a successful end-to-end natural language processing application.

analyzing such issues is a key to fully understanding the character-level models.


To achieve this, we perform a case study on semantic role labeling (SRL), a sentencelevel semantic analysis task that aims to identify predicate-argument structures and assign meaningful labels to them as follows: We use a simple method based on bidirectional LSTMs to train three types of base semantic role labelers that employ (1) words (2) characters and character sequences and (3) gold morphological analysis.

the ability to handle it is a strong performance indicator.


The dramatic drop in German oracle model is due to the high lemma OOV rate which is a consequence of keeping compounds as a single lemma.

given more training data their performance will improve faster than morphology-level models, • They perform substantially well on out of domain data, surpassing all morphology-level models.


The sentence encoder takes pre-trained fixed word embeddings, randomly initialized lemma embeddings, part-ofspeech and named-entity tag embeddings.

their model is very data demanding as they needed to train it on additional 100K sentences parsed by other parsers.


Results are copied from Peng et al., 2017a;Buys and Blunsom, 2017).

the parsing results are quite different, as shown  Table 8: Accuracy on the test set.


To solve math word problems, previous statistical approaches attempt at learning a direct mapping from a problem description to its corresponding equation system.

an equation system does not encode these information explicitly.


We compare CNN, LSTM and SWEM wrt their parameters and computational speed.

notably, on SnLI dataset, we observe that SWEM-max performs the best among all SWEM variants, consistent with the findings in nie and Bansal (2017); Conneau et al.


Our code will be released to encourage future research.

the GloVe embeddings are employed in two ways to learn refined word embeddings: (i) directly updating each word embedding during training; and (ii) training a 300dimensional Multilayer Perceptron (MLP) layer with ReLU activation, with GloVe embeddings as input to the MLP and with output defining the refined word embeddings.


So motivated, after training the SWEM-max model on the Yahoo dataset, we selected five words with the largest values, among the entire vocabulary, for each word embedding dimension (these words are selected preferentially in the corresponding dimension, by the max operation).

sWEM exhibits comparable performance on the other two tasks, again with much less parameters and faster training.


We experiment on 17 datasets concerning natural language understanding, with corresponding data statistics summarized in the Supplementary Material.

it is of interest to study the practical value of the additional expressiveness, on a wide variety of NLP problems.


Also, pragmatic inference is a necessary step toward automatic narrative understanding and generation (Tomai and Forbus, 2010;Ding and Riloff, 2016;Ding et al., 2017).

we randomly divided our set of 24,716 unique events (57,094 annotations) into a training/dev./test set using an 80/10/10% split.


It is hidden by the topic case marker は(wa).

a promising way to solve this data scarcity problem is enhancing models with a large amount of raw corpora.


To develop models for Japanese PAS analysis, supervised learning methods using annotated corpora have been applied on the basis of morpho-syntactic clues.

we also compare our GAN-based approach with data augmentation techniques.


This suggests that the generator gradually becomes stronger than the validator.

omitted pronouns have few clues and thus these models try to learn relations between a predicate and its (omitted) argument from the annotated corpora.


shows that instead of human annotated event coreference relations, using system predicted relations resulted in a significant performance reduction in identifying the central event of a document.

we add the following objective function (equation 3) to the basic objective function and add the new constraint 4 in order to encourage coreferent event mentions to occur in topic transition sentences.


In addition, to avoid fragmented partial event chains and recover complete chains for the main events, we also encourage associating more coreferent event mentions to a chain that has a large stretch (the number of sentences between the first and the last event mention based on their textual positions).

coreferential event mentions are rarely from the same sentence ( 10%) and are often sentences apart.


Clustering performs spectral graph clustering (Pedregosa et al., 2011), which represents commonly used clustering algorithms for event coreference resolution.

we train our event extraction system and local coreference resolution classifier on 310 documents from the KBP 2015 corpus that consists of both discussion forum documents and news articles, tune the hyper-parameters corresponding to ILP using 50 news articles 6 from the KBP 2015 corpus and evaluate our system on 4 The ECB+ (Cybulska and Vossen, 2014) corpus is another commonly used dataset for evaluating event coreference resolution performance.


Then, along with our adversarial learning, the generator learns to gradually decrease the probabilities of the false positive samples.

(2017) bring the useful entity information into the calculation of the attention weights.


When the classification capacity of discriminator declines, the accuracy of being predicted as negative sample on N D gradually drops; thus,p increases.

because adversarial learning is widely regarded as an effective but unstable technique, here we illustrate some property changes during the training process, in which way to indicate the learning trend of our proposed approach.


In other words, the generator becomes better.

relation extraction is a crucial task in the field of natural language processing (NLP).


(2017) bring the useful entity information into the calculation of the attention weights.

the more robust generator can be obtained via competing with the more robust discriminator.


But considerable expertise is required for feature engineering.

there are two resource-poor conditions discussed in this section, including lack of in-domain training data and that of out-domain.


In GCL, the keys are simply the entity representations [e 1 , e 2 ] from input, in either order.

the max pooling layers on top of each Bi-LStM layers are omitted here.


The idea here is to train the model on short paragraphs to avoid overfitting.

we use a shift function similar to the canonical NTM.


BookCorpus (Zhu et al., 2015) is a large collection of free novel books written by unpublished authors, which contains 11,038 books of 16 different sub-genres (e.g., Romance, Historical, Adventure, etc.).

we calculated a PMI score between a candidate event and each context event e 1 , e 2 , ..., e n−1 based on event sequences extracted from our learned 287k narratives and we chose the event that have the highest sum score of all individual PMI scores.


Narratives we obtained from the first phase may describe specific stories and contain uncommon events or event transitions.

to be consistent with Choubey and Huang (2017), we used the same train/test splitting, the same parameters for the neural network and only considered intra-sentence event pairs.


Furthermore, the subject of the sentence is meant to represent a character.

the same as in (Hu and Walker, 2017), we calculate separate CP scores based on event co-occurrences with zero (consecutive), one or two events in between, and use the weighted average CP score for ranking an event pair, formally, CP (e i , e j ) = 3 .


Visualization models in computer vision mainly consist in inverting hidden layers in order to spot active regions or features that are relevant to the classification decision.

we call this sum Text Deconvolution Saliency (TDS).


While those methods may hold accurate information in their input recovery, they have two main drawbacks: (i) they are computationally expensive: the first method requires training a model for each latent representation, and the second relies on backpropagation for each submitted sentence; (ii) they are highly hyperparameter dependent and may require some finetuning depending on the task at hand.

this is a simple but strong method for analyzing features of text.


we then adapt the model to conversational discourse by incorporating the thread structure in its grid representation and feature computation.


However, our goal in this paper is not to build a state-of-the-art system for thread reconstruction, rather to evaluate coherence models by showing its effectiveness in scoring candidate tree hypotheses.

as we repeatedly apply the filter to each possible window with stride size 1, we get a 2D feature map Z i of dimensions (I.J +m−1)×(I.P +n−1).


These results demonstrate the efficacy of lexicalization in capturing fine-grained entity information without loosing generalizability, thanks to distributed representation and pre-trained embeddings.

glove did not perform well giving F 1 score of 86% in the standard discrimination task.


In this example, the second zero pronoun " 2 " is anaphoric and corefers to the mention "S ã∫N ö /Litigant Li Yading" while the zero pronoun " 1 " is unanaphoric.

experiments on the benchmark dataset show that our reinforcement learning model achieves an F-score of 67.2% on the test dataset, surpassing all the existed models by a considerable margin.


v featuret represents the feature vector.

unsupervised techniques have been proposed and shown their efficiency.


Our method adopts the greedy search so that it cannot exploit this handy information in the analysis of the first sentence.

the dimensions of the hidden layer in all the neural networks were set to 100.


Only when C power * C pLen is removed, the coefficients of C count * C power become significantly positive (the last two rows in Table 3).

is it significant enough to be captured by computational measures of alignment?


In our first experiment, we replicated the effect of prime speakers' power status on the linguistic alignment from target speakers, from the significant interaction term C count * C power .

our findings suggest that the probability change of LIWC categories is more likely to be a case of automatic alignment, rather than an intentional accommodation, because it is better explained by lowerlevel linguistic features (utterance length).


Although the ideal argument should not violate the canonical structure, students attempting to improve their persuasive writing skills may not understand this, and mistakenly support evidence with their own opinions.

the relationship is thoroughly explained in the text because the two components contain the same words or exhibit coreference.


Note that ranking is not an acceptable evaluation setting for studying argument persuasiveness in the essay domain, as feedback for an essay has to be provided independently of other essays.

a premise has type warrant when it does not fit any other type, but serves a functional purpose to explain the relationship between two entities or clarify/quantify another statement.


Its Strength is 4: though eloquent and relevant, P2 is lacking substance in order to be considered for a score of 5 or 6.

the arguments of an essay are typically structured as an argument tree, where the major claim, which is situated at the root of the tree, is supported by one or more claims (the children of the root node), each of which is in turn supported by one or more premises.


Similar trends are found when conducting such experiments to Text Simplification (TS) ( §4).

evaluation in monolingual translation (Xu et al., 2015;Mani, 2009) and in particular in GeC (Tetreault and Chodorow, 2008;Madnani et al., 2011;Felice and Briscoe, 2015;Bryant and Ng, 2015; has gained notoriety for its difficulty, due in part to the heterogeneity and size of the space of valid corrections (Chodorow et al., 2012;Dreyer and Marcu, 2012).


The FSA said it would probe both companies over claims they had breached the rules on the sale of private data.


As the gold standard of performing human evaluation is often too expensive, there has been a large effort develop-ing automatic metrics such as BLEU (Papineni et al., 2002), ROUGE (Lin and Rey, 2004), ME-TEOR (Lavie and Denkowski, 2009;Denkowski and Lavie, 2014) and CiDER (Vedantam et al., 2015).

next, we look at evaluating the correctness of system outputs in question answering using the MS MARCO question answering dataset (nguyen et al., 2016).


Prior work has shown that existing automatic metrics have poor instance-level correlation with mean human judgment and that they score many good quality responses poorly.

(2017) and the snet and snet.ens(emble) models from Tan et al.


Our method reaches state-ofthe-art performance and can be applied to languages other than English in an almost out-of-thebox fashion.

the lack of large training sets for the meeting domain is an obstacle to the use of neural approaches.


Our abstractor is a simple encoder-aligner-decoder model (with copying) and is trained on pseudo document-summary sentence pairs obtained via simple automatic matching criteria.

our extractive experiment serves as a complementary analysis of the effect of RL with extractive systems.


11 The reinforced extractor performs the best, because of the ability to get the summary-level reward and the reduced train-test mismatch of feeding the previous extraction decision.

we do get a small further boost to the summary quality by removing a few 'across-sentence' repetitions, via a simple reranking strategy: At sentence-level, we apply the same beam-search tri-gram avoidance (Paulus et al., 2018).


Soft-sharing: In our soft-sharing approach, we encourage shared parameters to be close in representation space by penalizing their l 2 distances.

we keep the lower-level layer (i.e., first layer closer to input words) of the 2layer encoder of all three tasks unshared, while we share the higher layer (second layer in our model as shown in Fig.


Our method (SV) outperforms baselines by directly using the distances in the SocVec space, which proves that the SocVec well captures the cross-cultural similarities between terms.

the challenge is to devise a way to compute similarities across two different vector spaces while retaining their respective cultural characteristics.


The above example reflects the moral foundations that shape the author's perspective on the issue: Harm and Cheating.

it is necessary to correctly model the relevant contextualizing information.


Notice that we use different parameters for the scoring networks σ(•).

the sketch constrains the generation process and is encoded into vectors to guide decoding.


As shown in Algorithm 1, we perform F forward passes through the network, and collect the results whereM i represents the perturbed parameters.

the coverage of training data also affects the uncertainty of predictions.


6 The promise of zero-supervision models is that we can learn cross-lingual embeddings even for low-resource languages.

we take the smallest k of the two, and use the sum of the squared differences between the largest k Laplacian eigenvalues ∆ as our similarity metric.


A related research line is to adapt these methods to the semi-supervised scenario, where the training dictionary is much smaller and used as part of a bootstrapping process.

compute the optimal orthogonal mapping maximizing the similarities for the current dictionary D: An optimal solution is given by W X = U and W Z = V , where U SV T = X T DZ is the singular value decomposition of X T DZ.


This typically uses nearest neighbor retrieval from the source language into the target language, so The underlying optimization objective is independent from the initial dictionary, and the algorithm is guaranteed to converge to a local optimum of it.

the underlying idea is to independently train the embeddings in different languages using monolingual corpora, and then map them to a shared space through a linear transformation.


The lower dimensionality and comparable corpora make an easier scenario, although it also contains a challenging pair of distant languages (Turkish-English).

the proposed unsupervised initialization is able to overcome this issue without the need of any additional information, performing at par with other character-level heuristics that we tested (e.g.


Highway networks is a type of neural network that can smoothly switch its behavior between transforming and carrying information.

in our model, each task has a separate object function.


Due to space limitations, we only show curves for Dutch Name Tagging, while we observe similar results on other tasks.

to jointly train multiple models, we perform multi-task learning using parameter sharing.


For example, our model corrects [S-PER Charles] [S-PER Picqué] to [B-PER Charles] [E-PER Picqué] because the CRFs layer fully trained on other languages assigns a low score to the rare transition S-PER→S-PER and promotes B-PER→E-PER.

with a large number of parameters, it cannot be fully trained only using the low-resource task data.


we experiment with two monolingual twitter data sets: and Spanish (4.8M) tweets using the public Twitter Streaming API 3 with language filters en and es (ii) a BACKGROUND corpus of 296K English and 150K Spanish (non-annotated) tweets released with the test data of the RepLab task (Amigó et al., 2013) described below All twitter data was tokenized using Bird et al.


For the baseline system, when using only in-domain text for creating BWEs, only the medical unlabeled set was effective, general domain word pairs could not be exploited due to the lack of general semantic knowledge in the BWE model.

(2017) tackled BLI of words in the medical domain.


We can see that BiDAF-M and DrQA-M perform surprisingly well in the within-world evaluation even though they do not use any entity span information.

we aim to generate natural-sounding first-person narratives from five diverse worlds, covering a range of different events, entities and relations.


(2017) significantly, likely because we are not subsampling the data.

we evaluate our work on TriviaQA (Joshi et al., 2017) in the wiki, web, and unfiltered setting.


For example, the model might become too reliant on selecting answers that match semantic type the question is asking about, causing it be easily distracted by other entities of that type when they appear in irrelevant text.

historians have suggested that Gordon...


Unfortunately, given source sentences x and z, P (x |x) is not comparable to P (z |z), as each has a different normalization constant, and heavily depends on the shape of the distribution around x or z.

we thus obtain 300 comparisons between human and SEA, and 300 between human and HSEA.


For example "My girlfriend loved the desserts" vs "My partner liked the desserts".

in transfer of political slant and sentiment we outperform an off-the-shelf state-of-the-art baseline using a cross-aligned autoencoder.


Intuitively, such a memory should be able to update itself dynamically by accounting not only for the factual embeddings but also for the current context of the generated sequence of words.

for example, American musician and pianist is generalized as American musician, since musician is a hypernym of pianist.


Since most answers in the bAbI dataset are unigrams, such an approach suffices.

additionally, our goal is to make our method applicable to any entity with factual information that may not have a corresponding Wikipedia-like article available.


At the core of the Dynamic Memory Networks (DMN) architecture (Kumar et al., 2016) is an episodic memory module, which is updated at each episode with new information that is required to answer a predefined question.

this task requires a textual verbalization of every triple.


To transform prompts into stories, we instead build on the convolutional seq2seq model of Gehring et al.

we are not aware of previous work that has used hierarchical generation from a textual premise to improve the coherence and structure of stories.


Besides, the Turing test of our AREL model reveals that nearly half of the workers are fooled by our machine generation, indicating a preliminary success toward generating human-like stories.

though automatic evaluation indicates slight performance boost over state-of-the-art (SOtA) methods in cloning expert behaviors, human evaluation shows that our approach achieves significant improvement in generating more human-like stories than SOtA systems.


This head-tohead compete is designed to help us understand in what aspect our model outperforms the competing algorithms, which is displayed in Table 4.

it empirically confirms that our generated stories are more relevant to the image sequences, more coherent and concrete than the other algorithms, which however is not explicitly reflected by the automatic metric evaluation.


They use orthogonal iterations to estimate a moving average of the covariance matrices, which improves memory consumption.

in what follows, we first review the CCA model and its deep variant: DCCA.


German results and Table 8 summarizes our English !

this is useful for generative modelling tasks.


For these tasks, we found that models that incorporate Picturebook led to faster convergence.

constructing grounded representations of natural language is a promising step towards achieving human-like language learning.


This connection is important for physical agents that not only can perceive and act, but also can communicate with humans in language.

the second option is to use the key phrases extracted from language effect descriptions to search the web.


Then, each student was also asked to provide positive or negative labels for the images collected by the other two students.

as part of the annotated images are used as test data to evaluate the models, it is important to remove duplicates.


To capture the most informative features, we apply max pooling (Kim, 2014) and obtain the sentence representation z ∈ R n k by employing n k kernels: Finally, we pass z to a fully connected layer for sentiment prediction: where W f and b f are learnable parameters.

the target, is capable of handling such case.


Evaluation metrics are Accuracy and Macro-Averaged F1 where the latter is more appropriate for datasets with unbalanced classes.

the non-recursive form of this equation is as follows (for clarity, we ignore the subscripts): (1 − t (k) )] h (0) the context information is integrated in each upper layer and the proportions of the contextualized representations and the transformed representations are controlled by the computed gates in different transformation layers.


A key factor for their success is the attention mechanism.

this section analyzes the problem of targetsensitive sentiment in the above model.


This also led to existing and potential research in improving attention modeling (discussed in Section 5).


In other words, addition of the iterative process with the shared representation given by SCL overcomes the errors introduced by SCL.

then the words which are significant in both the domains with the consistent polarity orientation are used to initiate the crossdomain adaptation algorithm.


6 Polarity of Words in the Unlabeled Target Domain: Generally, in a polar corpus, a positive word occurs more frequently in context of other positive words, while a negative word occurs in context of other negative words .

domain adaptation techniques work in contrast to traditional supervised techniques on the principle of transferring learned knowledge across domains (Blitzer et al., 2007;Pan et al., 2010;Bhatt et al., 2015).


Table 8 shows the percent (%) of common words among the 4 domains.

one cannot use transfer learning as the proverbial hammer, rather one needs to gauge what to transfer from the source domain to the target domain.


It can be clearly seen that emotional words are removed accurately almost without loss of non-emotional information.

they have the poor preservation of non-emotional semantic content.


Recently, several related studies for language style transfer (Hu et al., 2017;Shen et al., 2017) have been proposed.

following previous work (Shen et al., 2017;Hu et al., 2017), we instead use a stateof-the-art sentiment classifier (Vieira and Moura, 2017), called TextCNN, to automatically evaluate the transferred sentiment accuracy.


The neutralization module first neutralizes an emotional input to semantic content and then the emotionalization module is forced to reconstruct the original sentence based on the source sentiment and the semantic content.

the two baselines change not only the polarity of examples, but also the semantic content.


Our architecture combines perceptual input processing, short-term memory storage, an attention mechanism, and a reasoning module.

then, the similarity between the projected memory and the question is computed using the Scaled Dot-Product attention: Next, the memories are combined using the attention weights α i , obtaining an output vector h = j α j m j .


Furthermore, our network can be considered as an 'inner' adaptation of neural attention, modeling intra-sentence relationships between the raw word representations instead of representations that have been compositionally manipulated.

where W f ∈ R d×2 , b f ∈ R 2 are the parameters of this layer.ŷ ∈ R 2 is the output layer of our proposed model.


We propose a multi-dimensional intra-attention recurrent network that models intricate similarities between each word pair in the sentence.

in general, we found that MiARN is able to identify contrast and incongruity in sentences, allowing our model to better detect sarcasm.


Deep learning based methods have recently garnered considerable interest in many areas of NLP research.

attention helps for long text (such as debates), resulting in Attention LSTMs becoming the strongest baseline on the Debates datasets.


6 relies on g θ being able to explore other candidates during sampling.

this negative sampling by corruption approach is also used in learning knowledge graph embeddings (Bordes et al., 2013;Lin et al., 2015;Ji et al., 2015;Wang et al., 2014;trouillon et al., 2016;Yang et al., 2014;Dettmers et al., 2017), order embeddings (Vendrov et al., 2016), caption generation (Dai and Lin, 2017), etc.


For our experiments, we train all our models on a single pass of the May 2017 dump of the English Wikipedia with lowercased unigrams.

this differentiability comes at the cost of drawing N Gumbel samples per each categorical sample, where N is the number of categories.


5 We also reported the variance (Var) of the performance to evaluate the stabilities of different methods.

2) Is the im-portance of positive/negative instances remaining unchanged during the entire training process?


Then w β (θ) can be computed as the ratio of marginal negative utility M U F β (T N (θ)) to the marginal positive utility M U F β (T P (θ)), where T P (θ) and T N (θ) are T P and T N on training data with respect to θ-parameterized model: Then at each iteration of the model optimization (i.e., each step of gradient descending), we want the model to take next update step proportional to the gradient of the w β -scaled cross-entropy loss function L AS (θ) at the current point: Consequently, based on the contributions that correctly predicting one more instances of each class bringing to F-measure, the training procedure dynamically adjusts its attention between positive and negative instances.

(2015), a one-layer CNN model with a dynamic multi-pooling operation over convolutional feature maps.


In the low-data setup, tri-training with disagreement works best, reaching an overall average accuracy of 89.70, closely followed by classic tritraining, and significantly outperforming the baseline on 4/5 domains.

overall we emphasize the importance of comparing neural approaches to strong baselines and reporting results across several runs.


From this point, we can leverage the pair-wise agreement of two output layers to add pseudo-labeled examples as training data to the third model.

mT-Tri only necessitates one forward pass as well as the evaluation of the two additional output layers (which takes a negligible amount of time) and requires about as many epochs as tri-training until convergence (see Table 3, second column) while adding fewer unlabeled examples per epoch (see Section 3.4).


However, such errorcorrected sentence pairs are not sufficiently available.

instead of treating the error predictions useless, self-boost learning fully exploits them.


Limited by the size of training data, the models with millions of parameters may not be well generalized.

low that of their correct sentences, as Figure 2(a) shows.


In addition, the huge volume of native data benefits the decoder to learn better to generate a fluent and error-free sentence.

we first train a seq2seq error generation model Θ gen with S * which is identical to S * except that the source sentence and the target sentence are interchanged.


Section 6 presents conclusions and discussions.

sensitivity (true positive rate) and specificity (true negative rate) were used to evaluate the code assignment performance.


With AL, our model successfully matches this DD to a ground-truth code: 414.01 (coronary atherosclerosis of native coronary artery).

first, our model is based on deep neural network, which has arguably better modeling power than linear methods used in the baselines.


Next, we incorporate the importance order among ICD codes.

if the distance be-tween A and B in the code tree is smaller than that between A and C and we know A is the correct code, then B is more likely to be a correct code than C, since codes with smaller distance are more clinically relevant.


We use the predicted probability p i (1 ≤ i ≤ D (l) ) defined in Section 4.3 to characterize the importance of code i.


Once learned, we use θ for prediction on test tweets.

while training CBOw, we filtered out words with a frequency less than or equal to 5, and we used a context window size of 5 and k = 5 negative samples.


For our work, we trained the CBOW model from word2vec.

note that for our domain adaptation methods, we only use unlabeled data from the target domain.


The supervised component induces label information (e.g., relevant vs. non-relevant) directly in the network through the classification loss L C (Λ, Φ), which is computed on the labeled instances in the source event, D l S .

several models have been proposed for semi-supervised learning.


However the success of DNNs on a task depends heavily on the availability of a large labeled dataset, which is not a feasible option in our setting (i.e., classifying tweets at the onset of an Earthquake).

data preprocessing was performed by following the same steps used to train the word2vec model (Subsection 2.5).


Second, instead of using count-based features, we use a convolutional neural network (CNN) to compose high-level features from the distributed representation of the words in a tweet.

the updates of the shared parameters Λ = {U, V } for the discriminator work adversarially to the rest of the network, and vice versa.


Also, since there is no regularization term in the latent space, q(z) is very different from the p(z) which prohibits us from generating sentences from the latent space.

the reason is related to our previous discussion about the granularity of the latent actions.


First L CVAE encourages I(x, z|c) (Agakov, 2005), which learns z that capture context-dependent semantics.

when N approaches infinity, q (z) approaches the true marginal distribution of q(z).


SW has 2,400 human-human telephone conversations that are annotated with topics and dialog acts.

3 is to maximize both the data likeli-hood lowerbound and the mutual information between z and the input data: jointly optimizing ELBO and mutual information simply cancels out the informationdiscouraging term.


In the usage space, words like "脂 肪 肝(fatty liver)" and "久 坐(outsit)" lie closely which are both specific words, and both are far from the general words like "胖(fat)".

(4) Human evaluation: Three labelers with rich Weibo experience were recruited to conduct evaluation.


Most methods in this line are constructed within the statistical machine translation (SMT) framework, where a sequence-to-sequence (Seq2Seq) model is learned to "translate" an input utterance into a response.

s=1: 我超喜欢她的眼妆，她的气质很美！I really like her eye make-up, and her temperament is so beautiful!


This shows how such models are limited in producing consistent co-occurrence among words, as compared with a real text.

the reason is that words occur in a bursty, clustered manner at all length scales: no matter how large the segment size becomes, a segment will include either many or few instances of a given word, leading to larger variance growth.


The majority of texts were in English, followed by French and then other languages, as listed in Table 1.

taylor exponent can reasonably serve for evaluating machinegenerated text.


However, implementations usually proceed by some kind of iterative probabilistic node-pair selection process, and in this way suffer from the same statistical pitfalls as swarm frameworks.

we introduce a new framework for modeling language change in populations.


In order to confirm that this choice is not affecting the results and to test a purer implementation of the framework presented here, we remove that constraint and update every node at each iteration.

g − listeners are sensitive to the phonemic distinction, so they misinterpret M − input at the rate of mishearing one vowel for the other (Peterson and Barney, 1952) (second half of Eqn.


The height of the lowest common ancestor for two leaves (w i , w j ) is noted asd i j .

let T be a parse tree that contains a set of leaves (w 0 , ..., w n ).


We apply GM-LVeGs to part-of-speech tagging and constituency parsing and show that GM-LVeGs can achieve competitive accuracies.

gM-LVeg-D is slightly better than gM-LVeg-S in sentence accuracy, producing the best sentence accuracy on 5 of the 9 testing datasets.


Subjective Corpus By extension, one may then deem a corpus subjective if its content does not at least meet Wikipedia's neutral point of view requirement.

as the first contribution, we outline an experimental scheme to explore this hypothesis in Section 2, and conduct a series of controlled experiments in Section 3 establishing that there exists a meaningful difference between word embeddings derived from objective vs. subjective corpora.


• A thorough discussions of issues encountered when training this model.

the agreement requirements are fixed and independent of word order.


The ELBO is maximised with respect to both the model parameters θ and the variational parameters λ.

the Republican complainant in the House wanted to summon 15 people including Lewinsky to testify in court.


The attributive phrase of "Czech border region" is a complete sentence.

we will also apply the proposed linearization method to other tasks.


The analysis of the attention model indicates that the model attends heavily to the contextual history for the translation of some pronouns.

this distribution is then used to compute a new representation of that word: this new representation is set to an expectation (under the attention distribution specific to the word) of word representations from the layer below.


Among the recent machine learning methods, bag-of-words representations combined with a Support Vector Machine (Melo and Martins, 2015) or Logistic Regression (Wing and Baldridge, 2014) have also achieved good results.

to assign a vector, places must first be disambiguated (catch-22).


The first two columns of Table 3 show the MRR results under Exact Match and Partial Match conditions.

figure 1 depicts the intended syntactic structure, which we will informally call the "go to" pattern.


Let sim E (a i , a k ) be the cosine value between the embedding vectors of a i and a k : Finally, we can plug these similarity functions into Eq (6).

the gold goal-acts may not match the activity phrases found in the corpus (see discussion in Section 4.3), so we smooth the vector created with the gold goal-acts by averaging it with the normalized co-occurrence frequency vector f i extracted from the corpus.


We prepared four datasets 3 for evaluation purposes.

other variants in the group will be deleted from the candidate list and their popularity scores will be aggregated to the canonical candidate.


To alleviate such troubles, we study how to automatically resolve the true meanings of acronyms in a given context.

if we see "SP" followed by "2003", then probably its genuine meaning is "SharePoint".


We proposed a novel, end-to-end framework to solve this problem.

it is particularly useful to develop a system that can automatically resolve the true meanings of acronyms in enterprise documents.


Later, an alternative hierarchical model called VHRED (Serban et al., 2017b) was proposed, where generated responses were conditioned on latent variables.

we present detailed experiments on the publicly benchmarked Ubuntu dialog corpus data set (Lowe et al., 2015) as well a large collection of more than 127,000 technical support conversations.


There has been further work where the responses are further ranked using a deep learning based model (Yan et al., 2016a,b;Qiu et al., 2017).

table 4 compares our model with other recent generative models (Serban et al., 2017a) -LStM (Shang et al., 2015), HRED ) & VHRED (Serban et al., 2017b).We do not compare our model with Multi-Resolution RNN (MRNN) (Serban et al., 2017a), as MRNN explicitly utilizes the activities and entities during the generation process.


These model parameters alone may not be sufficient for generating natural conversations.

let p dec (r|e (k) ) be the distribution of generating the ground truth response given the exemplar embedding.


Negative politeness strategies are salient in on-track conversations as well, reflected by the use of hedges (p < 0.01) and opinion prompts (p < 0.05), which may serve to soften impositions or factual contentions (Hübler, 1983).

annotators Conversations Snippets agreement 247 1,252 2,181 87.5% Table 1: Descriptions of crowdsourcing jobs, with relevant statistics.


semantic similarity, paraphrasing, entailment).

for de-ATTN-CTX models, we observed a much  flatter distribution of attention weights for each head and, unlike in the other models, we were often able to identify a head focusing on the main verb.


our NMT system has both the capacity to attend to various parts of the input and to produce static representations of input sentences.


we use the set of 27 edit types given in the NUCLE corpus.


CLT establishes that, in most situations, when independent random variables are added, their properly normalized sum tends toward a normal distribution even if the original variables themselves are not normally distributed.

the authors prove that for large enough data, the sampling distribution of a certain function of the Pearson's correlation coefficient follows the Student's t-distribution with n − 2 degrees of freedom.


In what follows we check whether recent ACL and TACL papers follow these guidelines.

a large portion of the surveyed papers do not test the significance of their results, or use incorrect tests for this purpose.


In Section 4.2, improvements from distilling the ensemble have been witnessed in both the transition-based dependency parsing and neural machine translation experiments.

it has drawn a lot of research attention in recent years thanks to its competitive performance on both accuracy and running time.


(2016) proposed a globally normalized transition model to replace the locally normalized classifier.

to enrich word-level information, we also use POS embeddings.


"Best Published" includes the most accurate parsers in term of UAS among , Martins et al.

when humans consciously annotate a sentence with syntactic structure, they rarely ever process in fixed left-to-right order.


Surprisingly, adding synthetic data decreased the model's ability to parse the variants.

@mentions were inserted using the ATMENTION token and emojis using the EMOJI token.


We present experiments affirming that this gain is due to an explicit composition operator rather than the presence of predicted syntactic annotations.

training was done using a language modeling objective that predicts the next word given the prefix; at test time we compute agreement error rates by comparing the probability of the correct verb form with the incorrect one.


As of late, the recurrent neural network with attention (Att-RNN) is now considered a baseline encoder-decoder architecture.

copyNet (Gu et al., 2016) is a natural fit here, as it enlarges the decoding output space from V to V ∪ X.


To sum up, Sequicity, as a framework, is able to handle various inconsistent user input despite its simple design.

then, all the slot activated before should be invalid now.


(3)) to decode the current token: where v ∈ R 1×l ; W 1 , W 2 ∈ R l×d and O ∈ R |V |×d .

to recent work (Eric and Manning, 2017a) that also employs a copy-attention mechanism to generate a knowledge-base search API and machine responses, our proposed method advances in two aspects: on one hand, bspans reduce the search space from U 1 R 1 ...U t R t to B t−1 R t−1 U t by compressing key points for the task completion given past dialogues; on the other hand, because bspans revisit context by only handling the B t with a fixed length, the time complexity of TSCP is only O(T ), comparing O(T 2 ) in (Eric and Manning, 2017a).


In this work, we propose the Global-Locally Self-Attentive Dialogue State Tracker (GLAD), a new state-of-the-art model for dialogue state tracking.

(3) Here, the scalar β s is a learned parameter between 0 and 1 that is specific to the slot s. Next, we compute a global-local self-attention context c over H. Self-attention, or intra-attention, is a very effective method of computing summary context over variable-length sequences for natural language processing tasks (Cheng et al., 2016;.


Furthermore, we observe an interesting phenomenon that humans can easily achieve a high entity F1 score with a low BLEU score.

less related to dialog systems, but related to our work, are the memory based decoders and the nonrecurrent generative models: 1) Mem2Seq query generation phase used to access our memories can be seen as the memory controller used in Memory Augmented Neural Networks (MANN) (Graves et al., 2014(Graves et al., , 2016.


From the results, we can see that MGL significantly outperforms baseline methods.

', even though the latter can also be viewed as relevant for the proposed question.


The encoder transforms the post sequence where GRU is gated recurrent unit (Cho et al., 2014), and e(x t ) denotes the embedding of the word x t .

the task needs to generate responses with both informative content and controllable sentence functions.


ations, so the model learned these subtle differences on its own.

there are similar datasets with emotion annotations  but are not labeled under dialog contexts.


Training corpus size is approximately 50 GB for all experiments.

después dijo : Hacedlo por segunda vez ; y lo hicieron por segunda vez .


Still, we will see below that defining concepts as target neighborhoods works well.

low-frequency units are selected first and high-frequency units last; this prevents errors due to spurious association of highfrequency units with low-frequency units.


5 The following equation summarises the tree edit distance measure: The possible values of this metric are non-negative real numbers.

the common criteria are based on the typology of word order (Naseem et al., 2012;täckström et al., 2013;Zhang and Barzilay, 2015) or part-of-speech n-grams (Rosa and Zabokrtsky, 2015;Agić, 2017).


The authors would like to thank the anonymous reviewers.

the effectiveness of cross-lingual transfer can be challenged by variation in syntactic structures.


Even for the best model, which in this case is 5(a)-χ, PPL is four times higher than the overall PPL on Test-17.

even with high rCM data, gCM does help in improving the LM until we have 50k rCM data (comparable to monolingual, and an unrealistic scenario in practice), where the returns of adding gCM starts diminishing.


Results on the resume NER test data are shown in Table 8.

we augment the character representation with segmentation information by concatenating segmentation label embeddings to character embeddings: where e s represents a segmentation label embedding lookup table.


(2017a) to automatically segment the development and test sets for word-based NER.

word embeddings are fine-tuned during NER training.


The embedding lexicon is released alongside our code and models as a resource for further research.

word sequences such as "长江大桥 (Yangtze River Bridge)", "长江 (Yangtze River)" and "大 桥 (Bridge)" can be used to disambiguate potential relevant named entities in a context, such as the person name "江大桥 (Daqiao Jiang)".


Similar to (Nimishakavi et al., 2016), schemata induced will be of the form relation A i , B j , C k .

each event template provides a distribution over slots, where slots are clusters of NPs.


Each cell in a slice corresponds to an induced schema in terms of the latent factors from matrices A and B.

please refer to Section 3.2.2 for details.


From an entity context set, each sentence is projected into a vector space using a piecewise convolutional neural network (Zeng et al., 2015).

the state-of-the-art systems are affected by very low performance, close to 16.6% F1, as shown in the latest tAC-KBP evaluation campaigns and in the open KBP evaluation benchmark 1 .


At k th layer, we compute gating value for a particular edge (u, v, l(u, v)) as: where, σ(•) is the sigmoid function,ŵ k l(u,v) and b k l(u,v) are label specific gating parameters.

it takes in a document as input and returns its predicted date as output by exploiting the syntactic and temporal structure of document.


It takes in a document as input and returns its predicted date as output by exploiting the syntactic and temporal structure of document.

we validate the importance of using syntactic and temporal GCNs and the effect of eliminating BiLSTM from the model.


Copying vs anonymization Both copying and anonymization alleviate the data sparsity problem by handling the open-class tokens.

the graph encoder As can be seen from table 1, the performance of Graph2seq is 1.6 BLEU points higher than Seq2seq, which shows that our graph encoder is effective when applied alone.


Nevertheless, S2S still failed to capture this structural relation, which reflects the fact that a sequence encoder is not designed to explicitly model hierarchical information encoded in the serialized graph.

the copy mechanism has the following advantages over anonymization: (1) anonymization requires significant manual work to define the placeholders and heuristic rules both from subgraphs to placeholders and from placeholders to the surface tokens, (2) the copy mechanism automatically learns what to copy, while anonymization relies on hard rules to cover all types of the open-class tokens, and (3) the copy mechanism is easier to adapt to new domains and languages than anonymization.


We observe BLEU scores of 11.8 and 12.7 using only forward or backward LSTM, respectively.

we make the following changes to the decoder.


We optimize the ranking log likelihood where y g is the gold ending and y r is a randomly sampled ending.

such hard constraints do not stop RNNs from repeating through paraphrasing while preventing occasional intentional repetition.


It is not an overstatement to say that pun generation has significance in human society.

automatic and human evaluations show that our models are able to generate homographic puns of good readability and quality.


Both descriptions are true, but the latter is most salient given the player's goal.

we use sklearn's (Pedregosa et al., 2011b) NearestNeighbor module to find the closest matching game move.


This way, RAML not only can incorporate the task metric information into training, but it can also alleviate the exposure bias by exposing imperfect outputs to the model.

with the particular choice of target distribution, RAML not only makes sure the ground-truth reference remains the mode, but also allows the model to explore sequences that are not exactly the same as the reference but have relatively high rewards.


Despite the distinct evaluation metrics for the aforementioned tasks, the standard training algorithm has been the same for all of them.

empirically, it seems using a slower (β = 0.001) update rate yields the best result.


Despite the distinct training procedures, both algorithms combine the idea of fine-grained credit assignment and the entropy regularization, leading to positive empirical results.

it is not true because Q R (i.e., P Q R ) can condition on the correct reference y * .


The results of our experiments are summarized in Tables 2 to 4 which we discuss in the following sub-sections.

manual inspection of the generated answers shows that in many cases the generator ends up generating either more or fewer words compared the true answer.


Scientific progress on this endeavor is measured through multiple tasks such as machine translation, reading comprehension, questionanswering, and others, each of which requires the machine to demonstrate the ability to "comprehend" the given textual input (apart from other aspects) and achieve their task-specific goals.

while several NER systems (Passos et al., 2014) gave close to human performance on this dataset, NER on general web text, domain specific text, noisy social media text is still an unsolved problem (mainly due to the lack of representative datasets which cover the real-world challenges of NER).


Since this step was rule-based we could only employ such qualitative inspections to set this parameter.

in query based abstractive summarization the training data is of the form {query, document, generated summary} and in our case the training data is of the form {query, mini-document, generated answer}.


This option allowed them to be creative and ask hard questions where possible.

in most cloze-style datasets (Mostafazadeh et al., 2016;Onishi et al., 2016) the questions are created automatically by deleting a word/entity from a sentence.


We propose a novel answer selection model, which combines the information from all the extracted candidates using an attention-based correlation matrix.

the similar training strategy is commonly employed when RL process is involved (Ranzato et al., 2015;Li et al., 2016a;Hu et al., 2018).


(2018b) proposed a re-ranking method to resolve the above issue.

the combination of the extraction and selection models exceeds our extraction baseline by a great margin, and also results in performance surpassing the best single reranker in (Wang et al., 2018b).


After that, we apply a self attention operation on the hidden representations to obtain the final representation q of the question q: where α j encodes the importance of each question word and is calculated as: where w is a learned weight vector.

quasar-T 1 (Dhingra et al., 2017b) consists of 43, 000 open-domain trivia question, and their answers are extracted from ClueWeb09 data source, and the paragraphs are obtained by retrieving 50 sentences for each question from the ClueWeb09 data source using LUCENE.


propose a DS-QA system, which retrieves relevant texts of the question from a large-scale corpus and then extracts answers from these texts using reading comprehension models.

in the experiments, we show that our models significantly and consistently outperforms state-of-the-art DS-QA models.


(3) On TriviaQA dataset, our+AVG model has worse performance as compared to R 3 model.

the retrieved texts in DS-QA are always noisy which may hurt the performance of DS-QA.


We use the Adam Optimizer (Kingma and Ba, 2014) for optimization with a first momentum coefficient of 0.9 and a second momentum coefficient of 0.999.

in a real-world CQA forum, a question is often represented as two parts: a subject that summarizes the main points of the question, and a body that elaborates on the subject in detail.


Compared with our baseline system, AST lexical achieves +1.75 BLEU improvement on average.

when L noisy is excluded from the training objective function, it results in a significant drop of 1.66 BLEU point.


We require that x is a subtle variation from x and they have similar semantics.

we propose a more general approach to modifying the sentence at the feature level.


In the top example, the NMT seq2seq model incorrectly aligns the target side end of sentence mark eos to 下旬/late with a high attention weight (0.80 in this example) due to the failure of appropriately capturing the similarity, or the lack of it, between the source word 下旬/late and the target eos.

the most appropriate source words are selected according to their attention weights and they are made to more closely interact with target hidden states.


quality ratings, is related to the task of sentence-level quality estimation (sQE).

reinforcement Learning from Estimated or Simulated Direct rewards.


Table 5: Average seconds required for decoding one source sentence on WMT17 translation tasks.

table 1 summarizes the comparison  results.


(2018) change the auto-regressive architecture to speed up translation by directly generating target words without relying on any previous predictions.

results are reported with multi-bleudetok.perl.


Note that we use a slightly updated residual structure as implemented by tensor2tensor 1 than proposed originally.

with recent advances in network architectures for Neural Machine Translation (NMT) recurrent models have effectively been replaced by either convolutional or self-attentional approaches, such as in the Transformer.


T do // Decode with cache 7: Bt ← construct beam from Bt−1 8: At = truncate(A, t) 9: Bt.add(de-abstract(At)) 10: for z ∈ BT do //Update cache 11: Update rewards in C[x] using (z, R(z, y)) 12: return BT ∪ de-abstract(A).

in CNLVR, y is binary, and so spuriousness is a central problem.


(2013)) have become a popular target representation for semantic parsing.

looking at its analyses, we find that it frequently manages to find a projective tree which yields an (almost) correct AMR, by choosing supertags with unusual types, and by using modify rather than apply (or vice versa).


We will also look into more principled methods for splitting the AMRs into elementary as-graphs to replace our hand-crafted heuristics.

we use pretrained GloVe embeddings (Pennington et al., 2014) concatenated with learned embeddings for w i , and learned embeddings for p i .


Unlike the JAMR aligner, our heuristics ensure that exactly one node in each elementary graph is marked as the root, i.e.

it is technically much more challenging to parse a string into a graph than into a tree.


(2017) use self-training on a huge amount of unlabeled text to lower the out-of-vocabulary rate.

aMR has been used in various applications such as text summarization (Liu et al., 2015), sentence compression (Takase et al., 2016), and event extraction (Huang et al., 2016).


(2017) propose a linearization approach that encodes labeled graphs as sequences.

the final performance still falls behind the best-performing models.


Finally, we can derive the batch gradients for where I + w denotes the set of positive contexts for w, I + c denotes the set of positive words for c and  In the following, we show that AllVec can achieve the same time complexity with negative sampling based SGD methods.

another line of research is the count-based embedding, such as GloVe (Pennington et al., 2014).


As can be seen, the major computational cost in Eq.

similar observations were also found in other fields like collaborative filtering (Yuan et al., 2016).


In this paper, we presented AllVec, an efficient batch learning based word embedding model that is capable to leverage all positive and negative training examples without any sampling and approximation.

besides, both theoretical analysis and experiments demonstrate that AllVec achieves the same time complexity with the classic SGD models.


Each dependency is represented by a labeled directed arc from a head token to a modifier token, where the arc label encodes broadly applicable semantic relations.

joint inference can sometimes be prohibitively expensive.


pool of unlabelled data with size K, in order to make the policy training efficient.

at each state, we provide the aL agent with a correct action which is computed by an algorithmic expert.


By changing K to values between these two extremes, we can analyse the effect of the quality of the algorithmic expert on the trained policy; Figure 4 shows the results.

our trained policy can make better rankings of unlabelled datapoints in the pool, leading to more effective AL strategies.


A key contribution of this work is an overstability test for question answering networks.

the network then uses the type to choose between a few answers it can give for the image.


Our attribution analysis suggests that we should find more failed examples in the first group.

in this work, we find that such overstability also applies to questions.


Thus, our research studies methods for incorporating coreference information into the training of a question generation system.

the crowdworkers were users of Amazon's Mechanical turk located in the US or Canada.


As is shown by Jia and Liang (2017), these confusing answer candidates could be quite difficult for MRC models to distinguish.

the task weights β 1 , β 2 are both set to be 0.5.


For example, Russian and Arabic are morphologically-rich languages and heavily utilize grammatical markers to indicate grammatical as well as semantic functions.

this work was supported by the National Natural Science Foundation of China (61772036, 61331011) and the Key Laboratory of Science, technology and Standard in Press Industry (Key Laboratory of Intelligent Press Media technology).


Quernheim and Knight (2012) focused on feature structures and introduced a DAG-to-Tree transducer to perform graph-to-tree transformation.

chinese, as an analytic language, encodes grammatical and semantic information in a highly configurational rather than either inflectional or derivational way.


First, we see that restricting the decoder to only generate known words is extremely useful, with SEQ+DICT improving over SEQ by 6.2 points.

we also improve the accuracy of seq2seq-only derived word systems by adding external information through constrained decoding and hypothesis rescoring.


To train this model, we use a held-out portion of the training set, and filter it to only observations for which exactly one of the two models produces the correct derived form.

given the word "run," we attempt to generate the word "runner" for "someone who runs."


We observe the following factorization, Neural approximation aims at minimizing the Kullback-Leibler divergence between the q φ (Z|X, y) and p θ (Z|X, y).

(2016), differently from the posterior, we set the prior z t = µ t during decoding.


Finally, we integrate deterministic features and the final prediction hypothesis is given as where W g , W y are weight matrices and b g , b y are biases.

the layer consists of a forward GRU and a backward GRU for the preceding and following contexts of a stock symbol, s, respectively.


This under utilizes the propagation information due to such oversimplified treatment of tree structure.

rumor is commonly defined as information that emerge and spread among people whose truth value is unverified or intentionally false (Di-Fonzo and Bordia, 2007;Qazvinian et al., 2011).


For experimental evaluation, we use two publicly available Twitter datasets released by Ma et al.

the nature of top-down tree lends this model different from the bottom-up one.


In this context, here we study the task of Name Tagging for social media containing both image and textual contents.

we use Convolutional Neural Networks (CNNs) (LeCun et al., 1989) to obtain the representations of images.


For example, we can easily find matched visual patterns from associated images for verbs such as 'sing', 'run', and 'play'.

multimodality is not the only challenge to perform name tagging on such data.


In addition, social media posts often include mentions of newly emerging entities unseen in training sets, making traditional context-based entity linking often not viable.

our SnapCaptionsKB dataset is composed of 12K user-generated image and textual caption pairs where named entities in captions and their links to KB entities are manually labeled by expert human annotators.


Named entities annotated in the dataset include many of new and emerging entities found in various surface forms.

as popular social media platforms are increasingly incorporating a mix of text and images (e.g.


There is a tendency for users to be wrongly predicted to be in CA, NY, TX, and surpris-ingly OH.

deep CCA takes the two text-based and networkbased views, and finds deep non-linear transformations that result in maximum correlation between the two views (Andrew et al., 2013).


We exploit the simplicity of our model to further assimilate external features relevant for answer selection: the inverse sentence frequency (ISF, (Trischler et al., 2016)), the inverse document frequency (IDF) and a modified version of the ISF score which we call local ISF.

we wanted to explore the idea that the first sentence of the document plays a crucial part in generating summaries (Rush et al., 2015;Nallapati et al., 2016).


In this paper, we propose a simple and generic neural architecture for text hashing that learns binary latent codes for documents in an end-toend manner.

table 6: Ablation study with different encoder/decoder networks.


We experimented with four variants for our NASH model: (i) NASH: with deterministic decoder; (ii) NASH-N: with fixed random noise injected to decoder; (iii) NASH-DN: with data-dependent noise injected to decoder; (iv) NASH-DN-S: NASH-DN with supervised information during training.

the inference (encoder) and generative (decoder) networks are optimized jointly by maximizing a variational lower bound to the marginal distribution of input documents (corpus).


For the first term in (3), we should in principle estimate the influence of µ i in (2) on q θ (x|z) by averaging over the entire (uniform) noise distribution.

our work focuses on learning discrete (binary) representations for text documents.


In this paper, we present an end-to-end Neural Architecture for Semantic Hashing (NASH), where the binary hashing codes are treated as Bernoulli latent variables.

we have: Although this is clearly a biased estimator, it has been shown to be a fast and efficient method relative to other gradient estimators for discrete variables, especially for the Bernoulli case (Bengio et al., 2013;Hubara et al., 2016;Theis et al., 2017).


We would like to thank the ACL reviewers for their insightful suggestions.

discrete representations that are learned in an endto-end manner at the sentence or document level have been rarely explored.


The inference (encoder) and generative (decoder) networks are optimized jointly by maximizing a variational lower bound to the marginal distribution of input documents (corpus).

if we allow the decoder to be too expressive (e.g., a one-layer MLP), it is likely that we will end up with a very flexible similarity measure but relatively less meaningful binary representations.


Autocomplete is implemented as a Non-deterministic Finite Automaton (NFA) whose states correspond to the 7 QA-SRL slots paired with a partial representation of the question's syntax.

we report the rate at which the predicted question exactly matches the gold question, as well as a relaxed match where we only count the question word (WH), subject (SBJ), object (OBJ) and Miscellaneous (Misc) slots (see Table 1).


(2013) 88.39 82.5 (80.91) 89.28 82.4 (80.88) Ours (syntax-agnostic) − 82.9 Ours (syntax-aware) 86.0 83.3 parser with even higher parsing accuracy.

the score is influenced by the quality of syntactic input to some extent, leading to unfaithfully reflecting the competence of syntax-based SRL system.


To answer Q1 , we first explore the full few-shot learning scenario.

different from intent detection, feat only outperforms logit by a margin.


UNK, and does not reflect the smoothness of the underlying continuous distribution of certain attributes.

much work has been done in solving arithmetic (mitra and Baral, 2016;Hosseini et al., 2014;Roy and Roth, 2016), geometric (Seo et al., 2015), and algebraic problems (Zhou et al., 2015;Koncel-Kedziorski et al., 2015;Shi et al., 2015; expressed in natural language.


), and r = 0.5×10 −r is the number's precision.

it had the worst MdAPE, which means that MoG mainly reduced larger percentage errors.


25 -28 to compute the predicted distributionp θ .

• c t ∈ R d is the new memory cell vector at time step t. As can be seen in Eq.


Suggestively, the one task where NLI clearly outperforms NMT is WC.

1 In constructing our probing benchmarks, we adopted the following criteria.


We evaluate the proposed method on a commonlyused dataset 2 , which is first presented in Riedel et al.


However, the issue is that these distantly-supervised sentences that mention this entity pair may not express the desired relation type.

distant supervision relation extraction is to predict the relation type of entity pair under the automatically-generated training set.


We address these questions by analyses showing that (i) the autoencoder learns sparse and interpretable codings of relations, (ii) the joint training drives relation matrices toward a low dimension manifold, and (iii) it helps discovering compositional constraints.

on WN18RR and FB15k-237, Dettmers et al.


However, modeling relations as mappings naturally requires more parameters -a general linear map between d-dimension vectors is represented by a matrix of d 2 parameters -which are less likely to be shared, impeding transfers of facts between similar relations.

we evaluate our system using standard KBC datasets, achieving state-of-the-art on several of them (Sec.6.1), with strongly improved Mean Rank.


Then, we maximize as our KB-learning objective.

we define the score function as for the basic model.


film and language), which probably constitute the skeleton of a KB.

we have u h M r v t > 0 if and only if h, r, t ∈ T .


• The DDQ(K) agents are learned by DDQ of Algorithm 1, with an initial world model pretrained on human conversational data, as described in Section 3.1.

except for very simple tasks (Singh et al., 2002;Gašić et al., 2010Gašić et al., , 2011Pietquin et al., 2011;Li et al., 2016a;Su et al., 2016b), RL is too expensive to be applied to real users to train dialogue agents from scratch.


The former is referred to as world model learning, and the latter direct reinforcement learning.

user simulators usually lack the conversational complexity of human interlocutors, and the trained agent is inevitably affected by biases in the design of the simulator.


The judgment score of 3 or above was taken as SUCCESS and 2 or below as DEFECT.

unlike traditional systems that have on the order of 10-20 built-in domains, largescale IPDAs can have up to 40,000 skills.


The paper is organized as follows: We list related work in section 2.

more recently, deep learning was used to extract higher-level multimodal features.


Synchronized attentive information across text and audio would then intuitively help recognize the sentiments and emotions.

decision-level Fusion (dL-Fusion) Inspired by (Wöllmer et al., 2013), we extract textual and acoustic sentence representations individually and infer the results via two softmax classifiers, respectively.


3) The acoustic modality is mostly present in fusion with the language modality.

an interesting observation is that in almost all cases the efficacies of unimodal connections to terminal T is low, implying that T prefers to not rely on just one modality.


Any hypotheses, however, would require further experimentation to test.

passages like (10) yielded pairings of OR and BUT, but not BECAUSE.


Also provided are about 20 generated permutations for each document (resulting in 1986 test pairs for accidents, and 1955 test pairs for earthquakes).

from a large scale document-mining perspective, the question of extracting intra-document structure remains largely underexplored.


Cells in the grid show words with highest probabilities (empty cells may indicate that no word has a substantially higher probability than others).

table 1 shows the performance of variants of our approach for the task.


Compared with the first line, we can see that gold causal relations led to approximately 2% improvement in precision and recall in temporal performance, which is a reasonable margin given the fact that C-Links are often much sparser than T-Links in practice.

existing approaches all followed the interval representation of events (Allen, 1984), which yields 13 temporal relations (denoted byR T here) as shown in Fig.


For notation convenience, we use EE to represent the set of all event-event pairs; then ET and T T have obvious definitions.

limited attention has been paid to studying these two relations jointly.


An effective way to measure the relatedness for two words is the inner product or cosine similarity between two corresponding word embeddings.

since these two kinds of interactions represent the relative information encoded in hidden embedding space, we expect that the relevance signal extraction process could generalize well to unseen categories.


Here, we compare the proposed DAZER against the following alternative solutions.

in DAZER, we use the word embeddings learnt from an external large text corpus to represent each word.


We carry out the experiments with different window sizes to eliminate the impact of window sizes.

cNN is able to extract local and position-invariant features well (Scherer et al., 2010;collobert et al., 2011).


• f 2 : z → y, a classifier f 2 annotates the text representation z with a label.

our method maintains the interpretability of word embeddings, and enjoys a built-in ability to leverage alternative sources of information, in addition to input text sequences.


A simple way to measure the compatibility of label-word pairs is via the cosine similarity whereĜ is the normalization matrix of size K×L, with each element obtained as the multiplication of 2 norms of the c-th label embedding and l-th word embedding: To further capture the relative spatial information among consecutive words (i.e., phrases 1 ) and introduce non-linearity in the compatibility measure, we consider a generalization of (2).

a key intermediate step is the text representation.


More recently, neural network-based methods have been used for document representation and these include average word embeddings (Mikolov et al., 2013), Doc2Vec (Le and Mikolov, 2014), Skip-Thought vectors (Kiros et al., 2015), recursive neural networkbased methods (Socher et al., 2014), LSTM architectures (Tai et al., 2015), and convolutional neural networks (Blunsom et al., 2014).

it is computationally expensive for long documents (Kusner et al., 2015).


We also ablated each of the individual gaze features.

after each break, we recalibrate the camera and validate the calibration again.


We model the text quality rating on a scale of 1 to 10, using the three scores as input.

we did not wish to overburden the readers, so we kept the size of texts to Property Bad.


we call the correction model trained in this setting Seq2Seq-Super.


Previous work on combining multiple inputs in neural translation deal with data from different domains, e.g., multilingual (Zoph and Knight, 2016) or multimodal (Libovický and Helcl, 2017) data.

in the absence of ground truth transcriptions, we can use different methods to generate a noisy corrected version for each OCR'd line.


Our second baseline is the same language model (AWD LSTM) with the type information added as an additional feature (i.e., same as entity composite model).

our model aims to addresses this property by leveraging variable types and scope.


Note that in our approach, the type model only replaces named entities with entity type when it generates next word.

our approach is general and can be applied with other types of language models.


Systematic comparisons are conducted between hyperdoc2vec and several competitors on two tasks, i.e., paper classification and citation recommendation, in the academic paper domain.


This implies that structure-based methods favor networks with better connectivity.

unlike h-d2v, they do not explicitly model citation contexts.


Given a relation r(x; y), the (partial) transformation function f is the POS-tag matched replacement of x in s with y, and requires the POS tag to be noun or verb.

we show how external knowledge resources, hand-authored rules, and neural language generation models can be used to generate such examples.


To speed up the training, we train the vectors in parallel with shared parameters, and they are updated asynchronously.

top 1 accuracy or error rate for each category is frequently used metric for this task, however, in this case these rank-based measures may not be an appropriate measure since the total number of unique n-grams (e.g., SISG) or unique words (e.g., SG) over the same corpus largely differ from each other.


The corpus was developed from 1998 to 2007, and contains formal text (newpapers, dictionaries, novels, etc) and informal text (transcriptions of TV shows and radio programs, etc).

in addition, if a character lacks jongsung, the symbol e is added.


These variations are often determined by jamo-level information.

we decompose Korean words into the jamo level, beyond the characterlevel, allowing a systematic use of subword information.