In the simple questions, the query graph's structure is determinate (one-hop edge).

we use S-MART (Yang and Chang, 2015) for Freebase dataset and DBpeida Lookup 4 for DBpedia dataset.


Despite using multiple answers in training, our system only predicts a single answer in decoding time.

(2018) proposed to use convolution with self-attention instead of recurrent models in reading comprehension.


Minimum Risk Training (MRT) has been widely used in various tasks in NLP.

the questions are usually well-formed and about simple facts, and the answers are guaranteed to exist as short spans in the given candidate passages.


In the experiments, the SAOKE dataset is split into the training set, validating set and testing set with ratios of 80%, 10%, 10%, respectively.

the SAOKE dataset is a valid opendomain sentence-facts parallel dataset for both OIE and OIN tasks.


Inspired by these approaches, we develop a customized memory controller along with an external memory augmentation (Graves et al., 2016) for complicated RC tasks.

in this paper, we propose a novel deep neural network architecture to handle a long-range dependency in RC tasks.


Table 1 reports our results on the RACE benchmark dataset.

note that an optional and additional projection may be applied to w t but we found that it did not yield much empirical benefit.


Dropout rate is tuned amongst {0.1, 0.2, 0.3} on all layers including the embedding layer.

our results show that it is not necessary to construct gates in a word-by-word fashion.


Our end-to-end differentiable model jointly learns structures and modules by gradient descent.

semantic parsing models have learned structures over pre-defined discrete operators, to produce logical forms that can be executed to answer the question.


10 For a more 7 After submitting an answer, they were shown the original teacher's answer so that they could understand the context of the subsequent questions.

we divide the supporting text into 12 chunks (with a special chunk for no answer) and use the transition matrix (computed from the training set) in Figure 5b to select an answer given the position of the previous answer.


Recently, reinforcement learning and adversarial learning are widely used to select the valid instances for relation extraction (Feng et al., 2018;Qin et al., 2018b,a).

the PR curve areas of BGRU+SDP and BGRU are about 0.332 and 0.337 respectively, while BGRU+StP increases it to 0.366.


Miwa and Bansal (2016) reported that a Tree-LSTM achieves similar performance when the dependency path and the LCA subtree are used respectively.

we observe that our GCN model System with-m mask-m 83.7 -SPTree † (Miwa+2016) 84.4 -PA-LSTM ‡ (Zhang+2017) 82.7 75.3 Our Model (C-GCN) 84.8 ⇤ 76.5 ⇤ outperforms all dependency-based models by at least 1.6 F 1 .


Finally, the hidden states of entity mentions from both LSTMs are taken as inputs to a logistic regression classifier to make a prediction: where h ✏ j is the hidden state of entity ✏ j .

we train our models with a cross-entropy loss over a set of gold standard data: where X i is an input graph, y i is the gold class label of X i , and ✓ is the model parameters.


to calculate a hidden state encoding for each word, Peng et al.


On the other hand, EM-BED assigns each edge label to an embedding vector, but complicates the gated operations by changing the U s to be 3D tensors.

as the input to our DaG LSTM, we first calculate the representation for each edge (i, j, l) by: where W 1 and b 1 are model parameters, e i is the embedding of the source word indexed by i, and e l is the embedding of the edge label l. The baseline LSTM model learns DaG representations sequentially, following word orders.


Our dataset contains five classes as mentioned in Section 6.1.

(2017) by studying ternary cross-sentence relations.


(2017) and binarize multi-class labels by grouping all relation classes as "Yes" and treat "None" as "No".

the bidirectional DAG LSTM model suffers from several limitations.


(2017), but different on how to utilize edge labels in the gated network.

information flow in our graph state LSTM is relatively more concentrated at individual words, with each word exchanging information with all its graph neighbors simultaneously at each sate transition.


(2) Our HATT method has higher precision values in the ONE test set.

this automatic mechanism is inevitably accompanied by the wrong labeling problem, because not all sentences containing two entities can exactly express their relations in KGs, e.g., we may mistakenly label "Bill Gates retired from Microsoft" with the relation business/company/founders.


We also plan to apply this method to other datasets.

to further prove the effectiveness of our model, especially in distinguishing noisy labels, we select some specific relationships for detail analysis.


Both of them got a better performance by introducing more information from KG.

added the confidence of (h, r, t) in the KG as extra supervision signal.


We have tried to add syntactic features in (Zhang et al., 2017), but didn't observe improvements.

it optimizes a global loss function and bridges the discrepancy between training and testing (Section 3.3).


Such systems are often flexible to incorporate different data sources and different learning algorithms.

to further explore interactions between the entity decoder and the relation decoder, some joint decoding algorithms were studied.


Such relations require the model to predict both relation types and the entity order correctly.

various auxiliary information has been reported to improve the performance of CNNs, such as the document graph (verga et al., 2018) and position embeddings (Shen and Huang, 2016;Lee et al., 2017;verga et al., 2018).


Interestingly, the undirected softmax always benefits the result although it was only used to calculate the penalty in the training step.

the low-dimensional forward and backward representation vectors of DU j are created by concatenating the corresponding final embeddings of tokens w j , w j+1 and the LSTM hidden state of the dependency d ii+1 .


• Second, the LSTM that takes as its input the article title is beneficial for before (0.37 vs. 0.40, +8.1%) and after (0.46 vs 0.53, +15.2%), and barely for during (0.31 vs. 0.32, +3.1%).

for example, Stassen (2009) considers two parameters (permanent contact and control) and Heine (1997) defines five parameters (human possessor, concrete possessee, spatial proximity, temporal permanence, and control).


The motivation for this is that a general out-of-vocabulary model is not expected to increase the performance, since it basically learns to ignore entity properties, similar to the models for common words such as "the".

a sequence of tokens T was transcribed from the speech of both participants using Penn Treebank tokenization rules (Marcus et al., 1993).


To do this, we define the RA of a word t as the mean difference between the probability of the actual referentr being TRUE p t (r) and the mean probability for all other entities R \r for every occurrence of the word in the training data: One alternative to this metric that we considered was the area under the receiver operating characteristic (ROC) curve (AUC).

we have shown that it is possible to improve reference resolution for situated dialog by incrementally adapting word semantic model parameters to a given dialog in order to accommodate idiosyncratic language use by dyad partners, and the effect of the partners' own alignment makes this method even more beneficial over time.


Assuming that there are three state trajectories Then red states s 4 , s 9 , s 13 could be good candidates for "subgoals".

in our experiment, p is set to be 0.2, which was manually picked according to the termination probability during SDN training.


For example, for the set of questions, Why is Saltwater Taffy candy imported in LOCATION?, most LO-CATIONs are covered by a generic answer, and all the corresponding questions are judged duplicates.

: (12) q1: Non ricordo piú la password per accedere all'App (I don't remember the password for the App) q2: mi sono dimenticato la password (I forgot the password) q3: reimpostare la password (reset the password) q4: cambio password (change the password) Here, LSSVM predicts the correct cluster exactly.


This confirms our belief that the natural language generation (NLG) capabilities of current generation based models are far from being acceptable even in case of generateor-copy modes.

we did try a few pilot experiments where we setup a server to connect two AMT workers but we found that the probability of two workers simultaneously logging in was very low.


For each group of RL models, the column of the optimization objective is highlighted.

we show that reinforcement learning on coarse dialogue acts avoids degenerate solutions, which was a problem in Li et al.


Large-scale automatically-generated cloze tests (Hermann et al., 2015;Hill et al., 2016;Onishi et al., 2016) lead to significant research advancements.

we can also further verify the strengths and weaknesses of the 1B-LM by studying the performance of models and human on different question categories.


We tried to increase the dimensionality of the model but do not observe performance improvement.

of the small size, human-created questions are only used to compose development sets and test sets.


The resulting corpus (emrQA) has 1 million questions-logical form and 400,000+ question-answer evidence pairs.

building a credible QA system for patient-specific EMR QA requires largescale question and answer annotations that sufficiently capture the challenging nature of clinical narratives in the EMR.


7 We choose the number of final candidates as 10 to stay consistent with the distractor setting where candidates are 2 Table 5.

to this end, a few large-scale QA datasets have been proposed, which sparked significant progress in this direction.


Thus we constrain our dev and test sets to be hard examples.

in our pilot studies, we found that simply giving an arbitrary set of paragraphs to crowd workers is counterproductive, because for most paragraph sets, it is difficult to ask a meaningful multi-hop question.


In the second-order false-belief task, Sally observes the new location of the milk; thus, she has a true belief about the milk's location.

to perform well on the false-belief and second-order falsebelief conditions, a model needs to identify that agents have experienced different events, and, as a result, have differing knowledge about the state of the world.


At each timestep, it composes an input vector and the hidden states from arbitrarily many child units.

most traditional SRL methods rely heavily on feature templates that struggle to capture sufficient discriminative information, while neural models are capable of extracting features automatically.


In our experiments, we simplify and reformulate the RCNN model.

each word embedding representation e i of input sentence is the concatenation of several features, a randomly initialized word embedding e r i , a pretrained word embedding e p i , a randomly initialized lemma embedding e l i , a randomly initialized POS tag embedding e pos i , and a predicate-specific feature e f i , which is a binary flag set 0 or 1 indicating whether the current word is the given predicate.


In this section, we describe our linearization traversal order, which generalizes the DFS traversal applied previously only for trees.

in Section 4, we adapt the MTL strategy to train a single model for all SDP formalisms.


In addition, an ablation study shows that multi-tasking the PRIMARY tasks is beneficial over a single task setting, which in turn is outperformed by the inclusion of the AUXILIARY tasks.

we report the performance of all source-target combinations in Table 6.


Our aligner is also enhanced by rich semantic resource and recalls more alignments.

since the new concept is pushed onto the buffer, NEW action can be repeatedly applied and used to generate arbitrary concepts that aligned to the same 12 e.g., three concepts in the fragment (person :source (country :name (name :op1 "North" :op2 "Korea"))) are aligned to "North Koreans".


Automatic POS tags are assigned with Stanford POS tagger .

achieving good alignments with the JAMR aligner still faces some difficult challenges.


Such a representation is able to capture alignment between the natural language words and semantics at the span level.

we marginalize t in the above equation and the resulting model is a latent-variable CRF (Quattoni et al., 2005): Given a dataset D of (n, m) pairs, our objective is to minimize the negative log-likelihood: 7 The gradient for model parameter w k is: where f k (n, m, t) represents the number of occurrences of the k-th feature.


A valid dependency-based hybrid tree (with respect to a given semantic representation) allows one to recover the correct semantics from it.

we further investigated the outputs from both systems on Indonesian by doing error analysis.


With recursive function composition, we can obtain a complete MR as shown in Figure 1.

in our dependencybased hybrid tree, the span covered by m 2 is from "What" to "not", which only consists of the span level information associated with its first child semantic units.


• We show that exact dynamic programming algorithms for inference can be designed on top of our new representation.

in their representation, the span level information assigned to the parent semantic unit always contains the span level information assigned to all its child semantic units.


We assume that the executor and all tables are deterministic and the executor can be called as many times as possible.

given that MAVER effectively considers more negative examples during each update, we expect that it is more stable compared to the MMR algorithm.


Because of the unique indirect supervision setting of SpFD, the quality of the learned semantic parser is dictated by the choice of how to update the model parameters, often determined empirically.

for example, the token most is highly likely to co-occur with a correct program containing the keyword Max.


Policy Shaping vs Model Shaping Critique policy contains useful information that can bias the search away from spurious programs.

this supports our hypothesis that aggressive updates of margin based methods is beneficial when the search method is more accurate as compared to maximum marginal likelihood which hedges its bet between all programs that evaluate to the right answer.


(2017) studied the problem of spurious programs and considered adding noise to diversify the search procedure and introduced meritocratic updates.

rEINFOrCE and maximum margin reward both have a non-zero intensity only on a single program in K. This value is 1.0 for maximum margin reward, while for reinforcement learning, this value is the reward.


We assessed model performance using three automatic metrics which represent different aspects of the compression task and have been found to correlate well with human judgments (Toutanova et al., 2016;Clarke and Lapata, 2006).

at test time, the target length generally varies according to the domain, genre, and language at hand.


Earlier work simply relied on word occurrence information only (Rapp, 1995;Fung, 1996) while later efforts have considered more sophisticated statistics in addition (Haghighi et al., 2008).

• Unlike previous models which only consider cross-lingual transformation in a single direction, our model jointly learns the word embedding transfer in both directions for each language pair.


A similar property holds for the other direction of the loop (from B to A and then from A back to B).

our system takes two sets of monolingual word embeddings of dimension d as input, which are trained separately on two languages.


In Section 3.1.1, we used the Euclidean distance of vector pairs to define M (G) and Sinkhorn distance d sh (G).

(2017) tried to impose an orthogonal constraint to their linear transformation model and minimize the distance between the transferred source-word embedding and its nearest neighbor in the target embedding space.


For a similarity measure, we use the L2 norm with sim(x, y) = − x−y 2 .

in our experiments, we kept the 7% additional ones, but we mark these ones with a special label '-'.


We provide experimental analyses in two such settings.

for zero-shot XEL (without prior) performance drops by more than 20% for TAC15-Test, 2.4% for TH-Test and by 2.1% for McN-Test.


In contrast to the word-level alignment methods, we attempt to mine burst-level alignment to largely narrow down candidates, and introduce powerful clues for improving accuracy and discovering various language knowledge.

formally, for a node c ∈ V c in the Chinese BINet, its candidate nodes in the English BINet can be derived as: where e ∈ V e , and P(c) and P(e) are the burst periods of c and e respectively.


Moreover, our BINets are cheap to construct, which can be easily extended to other languages.

it is easy to control the quality of mined pairs, which is important for a text mining algorithm.


Therefore, the semantic meanings of words can reflect in the contexts according to these distributed representation models.

n-Hance is the second place in Se-mEval2017 task7.


The best perform of SemEval2017 task7 is Fermi.

section 3 presents our proposed word embedding and collocation attention-based network model.


words sharing the same lemmas are likely to obtain the similar representations.


Human evaluation confirmed that our proposed method can produce common errors that are likely to be made by human and such errors can serve as effective annotated spelling errors for CSC.

to evaluate the quality of the generated corpora, we adopt three benchmark datasets from 2013-2015 shared tasks  tseng et al., 2015) 14 on CSC.


Following previous studies (Chang, 1995;Huang et al., 2007;Wu et al., 2010;Dong et al., 2016), we adopt confusion sets 21 and a language model to handle the taks for Chinese spelling error correction.

empirically, given two Chinese characters, c 1 and c 2 , we set 0.25 * (len(c 1 ) + len(c 2 )) as the threshold, η, where len(c) denotes the number of strokes for the Chinese character c. If the edit distance of two characters is more than a threshold η, we consider them not to be similar in shape.


These sentences are then handled by the OCR-based method as we describe before, resulting in an annotated corpus containing about 40, 000 annotated sentences with 56, 857 spelling errors.

the development of CSC techniques are restricted and thus CSC performance is not satisfied so far (Fung et al., 2017).


In addition, to illustrate its effectiveness, we compare it with two publicly available confusion sets (Liu et al., 2009), Con1 and Con2.

since 戒 and 禁 are neither visually nor phonologically similar, the proposed corpus generation approach is unable to construct such spelling errors, so it is understandable that the trained model can not detect such spelling errors.


M 2 is the most widely used GEC evaluation metric that computes the F 0.5score of phrase-level edits made by a system.

the remaining data consists of 2.15M sentence pairs (25.47M source tokens and 28.94M target tokens).


They use a dataset of learner sentences manually annotated with subjective scores of grammaticality.

the statistics of the datasets are given in table 1.


The DAILY547 consists of one random English tweet from every day between January 1, 2011 and June 30, 2012.

hence, we predict that labeled data from the newswire should selectively be used to help tag contextual segments of tweets.


To achieve this goal, in this work we adopt bidirectional long short-term memory (bi-LSTM) networks (Schuster and Paliwal, 1997), which have been successfully used for various sequence tagging problems.

our DCNN could make better use of the context information to generate more appropriate parameters for POS tagging.


DCS currently has split the corpus into more than 560,000 text lines, all of which need not be following explicit sentence boundaries.

in this model, we augment the loss L tree such that the Steiner tree with the least energy that spans the nodes in ground truth is chosen.


Our extraction methodology is 5 Nouns and verbs were identified by mapping the fine-grained part-of-speech tag to its coarse-grained category (Petrov et al., 2012): https://github.

we further contribute a series of modeling experiments on this data.


Figure 2 shows the differences in the confusion matrices of the char-lstm and oracle for those words on which both models correctly predict the head.

our results highlight many strengths of character-level models, but also show that they are poor at disambiguating some words, particularly in the face of case syncretism.


To make the comparison as fair as possible, we consider only words that were observed in the training data.

we also notice that the model sometimes attends to features like Aspect, Polarity, and Verb-Form-since these features are present only on verbs, we suspect that the model may simply use them as convenient signals that a word is verb, and thus a likely head for a given noun.


For each λ ∈ {a, b}, GRU s λ attempts to model the emotional inertia of speaker P λ which represents the emotional dependency of a speaker with their own previous states.

emotion recognition is an interdisciplinary field of research with contributions from psychology, cognitive science, machine learning, natural language processing, and others (Picard, 2010).


The open-vocabulary object detection and retrieval task was evaluated in the same way as the standard object detection task.

finally, a confidence and a regressed bounding box are predicted for each region by applying the classifier and regressor to the region features.


We first describe Query-Adaptive R-CNN as an extension of the Faster R- CNN (Ren et al., 2015) object detection framework to open-vocabulary object detection simply by adding a component called a detector generator.

instead, we generate a regressor as well as a classifier, which enables more accurate localization with fewer proposals.


Instead of learning the subspace between the image and sentence as in standard crossmodal searches, they learn the subspace between a region and a phrase.

all of its components can be trained in an end-to-end manner.


To create the set of training instances, we run PathLSTM (Roth, 2016;Roth and Lapata, 2016) on all captions, and extract all semantic frame annotations whose roles are filled by a grounded entity.

sRL systems in NLP research use training data which have been carefully created by linguistic experts (e.g., Ruppenhofer et al.


The average number of relations and attributes in each image for different actions varies slightly.

the ratios of transitive-Relation are similar across different actions.


For both routes, the proposed model was able to predict the correct sequence of navigation behaviors.

we conduct extensive experiments to study the generalization capabilities of the proposed model for following natural language instructions.


This cell takes two inputs: a hidden state vector h t−1 from the prior cell, and a one-hot embedding of the previous behavior b t−1 that was predicted by the model.

translation approaches have to cope with the semantics and syntax of the language by discovering corresponding patterns in the data.


Thus, p-values (and, consequently, hypothesis tests) based on direct comparison of DTSR models should be considered approximate.

the public release of the corpus contains data collected from 181 subjects.


From a modeling perspective, the primary results of interest in Experiment 2 are the IRFs themselves and the insights they provide into human sentence processing.

as shown, DTSR provides comparable or improved prediction performance to the baselines, even against the -S models which are more heavily parameterized.


For expository purposes, in Section 3 the DTSR model was defined only for fixed effects.

since DTSR is domain-general, it can be a valuable component in any analysis toolchain for time series.


We consider two models: one is independent of the parser while the other is added as a component to the parser.

while the UAS of the P E baseline increases to 80.2 (P E+F i with independent training), its segmentation F1-score does not cross the 66.9 bound (P E+Hi with joint training).


Where speed is more important, it may be worth considering a system that employs a smaller character embedding and does without POS tags, using just pre-trained embeddings.

in the simplest case, we set x i equal to the word embedding e r (w i ): The superscript r refers to the fact that the word embeddings are initialised randomly at training time.


we in fact observe a much greater gain when increasing from 100 to 500 (0.9 for Finnish, 1.2 for Korean, 1.0 for Russian; 0.4 on average across the nine treebanks), suggesting that very large character embeddings are effective, and particularly useful for morphologically rich languages.


Induction of PCFGs has long been considered a difficult problem (Carroll and Charniak, 1992;Johnson et al., 2007b;Liang et al., 2009;Tu, 2012).

because of the cognitively motivated left-corner HMM sampler used in the model, its state space grows exponentially with the maximum depth and polynomially with the number of categories.


But they show a difference in terms of syntax information contained.

examples of such approaches include Tree-LSTM (Tai et al., 2015;Zhu et al., 2015b) and Recursive Neural Networks (Socher et al., 2011), in which the networks are structured according to the syntax tree of the input sentence.


During the training for such tasks, the information irrelevant to the training objective is necessarily "squeezed out", and the representations of those syntax paths that provide no distinguishing features are "pulled closer".

if there is no such token, the distribution of b is concentrated at 0, and with probability close to 0 our scheme will make a false detection.


saralāṅga "name of a pine resin", for example, is oversegmented into sarala-aṅga "pinelimb", which is the etymological derivation of this compound.

we refer to this task as Sanskrit word splitting (SwS), and subsume Sandhi and compounding phenomena under the common term splits.


5): (2) rājā+uvāca 'the king said'¯a In addition, Sandhi occurs between independent inflected words (Ex.

crNN creates oversegmentations such asśr .ṅ gavanti-ah .


The preceding utterances are serialized in the order of their onset times, so that the flow of words within an utterance is not disrupted.

all LSTM-LMs are pretrained for one or two epochs on a large corpus of "conversational Web" data (Bulyko et al., 2003), followed by normal training to convergence on the in-domain data.


Perhaps the most widely studied effect is topical coherence, or the tendency of words that are semantically related to one or more underlying topics to appear together in the conversation.

other phenomena operate more locally, but across speaker turn boundaries.


A response with a higher frequency will be assigned with a smaller E(y).

relevance Acceptance Seq2Seq 1.96±3.8e-5 1.31±5.3e-3 0.42±4.7e-4 Seq2Seq-rS 1.97±8.1e-5 1.30±2.1e-3 0.42±9.9e-4 Seq2Seq-MMI 1.94±7.2e-5 1.19±4.0e-3 0.41±1.9e-4 Seq2Seq-DD 1.86±2.8e-3 1.40±1.5e-2 0.49±2.4e-3 Ours-rWE 1.95±1.9e-4 1.30±5.1e-3 0.42±4.2e-4 Ours-rWF 1.97±1.5e-4 1.47±2.1e-3 0.51±6.3e-4 Ours-rWEF 1.96±8.3e-5 1.59±1.9e-2 0.55±4.4e-3 the average percentage of the generated responses that are assigned to relevance rating 2 (relevant, including the generic responses) and 3 (relevant as well as interesting) are presented in Table 4.


(2015) used it to assess chit-chat qualities of generic dialogue models.

we create a dictionary containing the 250k most frequent tokens.


However, the PERSONA-CHAT dataset was created using an artificial data collection mechanism based on Mechanical Turk.

current dialogue systems are not very engaging for users, especially when trained end-toend without relying on proactive reengaging scripted strategies.


Second, in many approaches the models for every slot are different.

the user utterance may also have its corresponding mbest ASR hypotheses with the normalized confidence scores (Chen et al., 2017) In this case, we can calculate the weighted word vectors, u i , where u i,j represents the word vector u i presented at the j-th ASR hypothesis, and the zero vectors are padded at the end of all the hypotheses that are shorter than the longest one to have a same length of the utterance.


StateNet shares parameters among all slots, through which we can not only transfer knowledge among slots but also reduce the number of parameters.

this is not flexible in practice (Xu and Hu, 2018).


Removing the actions LSTM dropped Exact match score slightly to 78.08.

we introduce a third metric, Tree-Labeled (TL) F 1 , which compares the sub-tree structure for a non-terminal, instead of just the token span.


(2015) extend Barabási and Albert (1999)'s preferential attachment model that was originally based on the "rich-get-richer" feedback mechanism to a biased preferential attachment model of mentor-mentee dynamics, G(n, f, p), where there further is (1) a minority-majority partition (the proportion of female nodes is less than half, f < 1 2 ) and (2) homophily.

in what remains of our study, we discard these latter authors.


In Table 1 we observe a similar trend for in-gender mentorship.


In our network, the degree of a node captures its level of social wealth: people may try to connect more often to people who already have many connections, either in order to profit from their social wealth or because they are more visible in the network.

if genders differ, then the relation (edge) is established according to the probability of homophily (p); otherwise (with proba-bility (1 − p)) it is rejected and the mentee must restart the process of finding a mentor.


We created templates (Table 2) Table 3) to retain balance in neutral and abusive samples.

(2017), we generated 1,152 samples (576 pairs) by filling the templates with common gender identity pairs (ex.


K-means clustering was performed to group activation clusters and find common themes in these reports.

each of these techniques employs a different approach and offers new information or supports previous findings.


Our model identifies synonyms and hyponyms like the word "language" in relation to the category of commenting.

while LIME analysis and first derivative saliency are both used to find word-level contributions, first derivative saliency is model-dependent and gives reasoning behind classification based on the whole model, in contrast to the locally-faithful, model-agnostic LIME analysis technique.


The classification for each category (C) are seen as positive (1) if the output is above threshold t and negative (0) if the output is below threshold t, a hyperparameter, giving the equation: 4 Experimental Setup SafeCity 2 is, to the best of our knowledge, the largest publicly-available online forum for reporting sexual harassment.

recent works in a similar vein include detecting the presence of domestic abuse stories on social media sites (Schrading et al., 2015a;Schrading, 2015;Schrading et al., 2015b).


On the same dataset,  extracted an extensive list of lexicosyntactic features from the transcripts and identified participants with AD with an accuracy of 81%.

x denotes the data points for which all views are present (i.e., the data used to learn multiview embeddings), and the subscripts P D, CAT , LET , DEM are used to represent the following views: picture description text features, category fluency text features, letter fluency text features, and demographic information.


In classification, we observe improvements, as shown in Figure 3, and obtain an F1 of 0.793 ± 0.052.

here, j ∈ {P D, CAT, LET, DEM }, which correspond to the picture description, category fluency, and letter fluency linguistic features, and demographic information, respectively.


(2018) train a model very similar to our multi-CRF model on multiple biological NER datasets with non-fully overlapping labels.

this feature representation is then projected to the label dimension L using a linear layer, giving a matrix of scores [f il ] where f il is the score for pre- BiLStM-CRF models used for named entity recognition add a CRF layer (Lafferty et al., 2001) on the output representations from the BiLStM model described.


While one may train a single model across the union of all the datasets available, this training procedure assumes that all labels (from the union of the tag set) are correctly annotated in every training instance -which is incorrect.

until recently, feature engineered machine learning models were the highest performing approaches to NER (Ratinov and Roth, 2009;Passos et al., 2014).


We also show that jointly training on multiple datasets improves performance further and achieves state-of-the-art performance on the Biocreative V CDR dataset.

this assumes that each text snippet is completely annotated across the label sets, which is not true.


Two sets of experiments are provided to show the effectiveness of pre-trained word ordering model.

connections between words in the sentence cannot be directly captured by word embeddings or language model, which are crucial for dependency parsing given its goal is to establish dependency relations between words.


This paper presents a simple framework for characterizing morphological complexity and how it encodes syntactic information.

correspondingly, let c be the set of contexts defined over elements of V. lemmas appear in the context of other lemmas, forms in the context of forms, and attributes in the context of attributes.


Aharoni and Goldberg train the model by maximizing the conditional log-likelihood (MLE) of gold edit actions derived by an independent character-pair aligner.

(2018), additionally addresses the problem of spurious ambiguity, which is not present in seq2seq models.


One problem with the MLE approach is that the aligner is trained in a disconnect from the end task.

the fully end-to-end approach addresses various shortcomings of previous training regimes (the need for an external character aligner, warmstart initialization, and MLE training biases), and leads to strong empirical results.


Another shortcoming of the RL-style methods is delayed punishment: For many NLP problems, including morphological string transduction, one can pinpoint actions that adversely affect the global score.

models did not learn much, which we attribute to sparse training signal as all action sequences producing the same y would incur the same sequence-level loss, including intuitively very wasteful ones, e.g.


This setting has been extensively studied in the image domain (Papernot et al., 2017;Chen et al., 2017a;Alzantot et al., 2018), but has yet to be explored in the context of natural language.

adversarial Text Prediction: Contradiction (Confidence = 43%) Premise: a runner wearing purple strives for the finish line.


(2017) found it is beneficial to capture different context with multiple individual attention functions.

as seen, the disagreement score on the individual component indeed increases with the corresponding regularization term.


The disagreement regularization serves as an auxiliary objective to guide the training of the related attention component.

in terms of training speed, we do not observe obvious decrease, which in turn demonstrates the advantage of our disagreement regularizations.


Strategically selecting points to annotate over alternating rounds of labeling and learning, an active learner is hoped to outperform budget-matched i.i.d.

for classification tasks, the we measure the budget in sentences while for sequence labeling, we measure the budget by the number of words because the annotator must provide one tag per word.


That's because papers often address just a single task, just a single model, and sometimes just one or two datasets.

they obtain significant improvement over classic uncertaintybased acquisition functions on the MNISt dataset and for diagnosing skin cancer from lesion images (ISIC2016 task).


All weights of the RNN except biases are denoted by ω, while a single weight (an element of any weight matrix) is denoted by w ij .

for both datasets, we use the vocabulary of 20,000 most frequent words.


framework in (Luong et al., 2015) for the P2C task.

figure 2 demonstrates the effect of the gated attention mechanism on candidates offering and unknown word replacement.


we find that the KySS scores are much more close even with different pinyin integrity, which indicates that user experience in terms of KySS are more hard improved.


This initialization approach works fine for long sequences, because it is only utilized once and its effect is quickly swamped by the recurrent steps of the network.

this is helpful when we want to efficiently incorporate data from a much larger corpus without training the RNNLM on it directly (Chelba et al., 2017;Noraset et al., 2018).


We employed the pre-trained LSA model Wiki-6 from (Stefanescu et al., 2014).

our main contributions are the following: (i) We propose a HAN framework for translation to capture context and inter-sentence connections in a structured and dynamic manner.


Similarity for pronouns, the joint model has the best result for TED talks and news.

but combining HAN decoder with HAN encoder further improves translation performance, showing that they contribute complementary information.


The proposed model improves over both seq2seq and parsed baselines by over 1 BLEU on the WMT17 English→German task.

choe and charniak (2016) adopted a similar strategy, using linearized parses in an RNN language modeling framework.


The parse2seq system fails when given only sequential source data.

we tokenize the opening parentheses with the node label (so each node label begins with a parenthesis) but keep the closing parentheses separate from the words they follow.


Our experiments on two machine translation tasks show significant improvements in comparison to training with the entire data set.

neural MTs seem to be more sensitive to noise (Chen et al., 2016), as they tend to assign high probabilities to rare events (Hassan et al., 2018).


(2013b) has been subsequently improved by changing the problem parametrization.

under these assumptions, we have finding the k nearest neighbors of Wx i among the elements of Y is equivalent to finding the k elements of Y which have the largest dot product with Wx i .


In our assumption, the temperature parameter is learned based on the information of the decoding at the current time step as well as the attention in the previous time steps, referring to the information about what has been translated and what is going to be translated.

nIST 2003nIST , 2004nIST , 2005nIST , 2006 tasks.


This as- Time (h) BLEU CE Baseline 51.3 25.1 47.31 Async (4) + τ = 4 39.7 25.07 46.59   pect of our work is similar to the work of Aji and Heafield (2017) where they drop the lower 99% of the gradient updates based on absolute value thus reducing the memory traffic.

larger batches have two well-known impacts on speed: making more use of GPU parallelism and communicating less often.


The reader may notice that if we apply delayed gradient updates, the effective batch size that the global optimizer deals with is τ times larger than the mini-batch size on which the local optimizers runs.

adam (Kingma and Ba, 2015) is used to perform asynchronous SGD with learning rate of 0.0001.


The auxiliary reconstruction objective R(•) guides the related part of the parameter matrix θ to learn better latent representations, which are used to reconstruct the DPP-annotated source sentence.

since the DPP-annotated source sentence serves as the reconstructed input, we introduce an additional DP-generation loss, which measures how well the DP is generated from the corresponding hidden state in the reconstructor.


2 We followed the approach described by Rabinovich et al.

we augmented every sentence with a tag on the English source side, identifying the gender of the speaker, as illustrated in (1).


2017published an annotated parallel dataset for EN-FR and EN-DE.

(2015) motivated the need for more personalized MT.


Finally, the decoder states are labeled either with an output token or a null symbol.

our models achieve a significant speedup over the autoregressive models, keeping the translation quality comparable to other non-autoregressive models.


the Transformer (Vaswani et al., 2017) replace RNNs with parallelizable components in order to reduce the time complexity of the training.

the drawback is that the model needs to explicitly determine the target sentence length and reorder the state sequence before it starts generating the output.


Within the other category, perhaps the most interesting cases are those where words appear to be substituted with a fluent but not adequate alternative.

working from the intuition that certain contexts indicate that copying should occur -for example, a name following a title like "Ms" or "Frau" S: Mrs Ashton, your role in this is invaluable.


Second, we encode the entire TM matched translation rather than segments into NMT with coupled encoders and a gating network.

in this paper, we propose a novel and effective method to address this issue in the combination of TM and NMT.


Baidu NMT  borrows the Word Reward method from pre-neural MT, which gives a reward r to every word generated, where r is a hyperparameter tuned on the dev set: 3 Murray and Chiang (2018) attribute the fact that beam search prefers shorter candidates to the label bias problem (Lafferty et al., 2001) due to NMT's local normalization.

by default, OpenNMT-py (Klein et al., 2017) stops when the topmost beam candidate stops, because there will not be any future candidates with higher model scores.


3 We first review existing methods to counter the length problem and then propose new ones to address their limitations.

in recent years, neural machine translation (NMT) has surpassed traditional phrase-based or syntaxbased machine translation, becoming the new state of the art in MT (Kalchbrenner and Blunsom, 2013;Sutskever et al., 2014;Bahdanau et al., 2014).


At inference time, we will encounter only Latin script words, but we will not know the language of any token.

sentences in both languages are taken from news text and similarly formal sources, and the Hindi is written entirely in Devanagari.


Other approaches have trained supervised models on POS-annotated, code-switched data (Jamatia et al., 2015;Ghosh et al., 2016;Gupta et al., 2017;Barman et al., 2016;Sequiera et al., 2015, inter alia), resources which are expensive to create and unavailable for most language pairs.

we use our jointrepresentation model to represent the token as both languages simultaneously.


More importantly, we observe appealing extraction behaviors of SemanticCaps on utterances of emerging intents as well, even if they are not trained to perform semantic extraction on utterances of emerging intents.

for example, in the intent of PlayMusic one self-attention head always focuses on the "play" action while another attention focuses on musician names.


Current intent detection methods train classifiers in a supervised fashion and they are good at discriminating existing intents such as Get Weather and Play Music whose labeled utterances are already available.

the intent labels also contain knowledge of how two intents are similar with each other.


It could be as simple as a keyword/phrase matching problem, but it could also be a nontrivial problem if compositions, hierarchies, and structures of texts are considered.

from the results, we observe that capsule networks can correctly recognize and cluster the important phrases with respect to the text categories.


Assuming U as the embedded x Seq (word sequence form of x), in source memory, we compute the match between the k-th topic and the embedding of the l-th word in x Seq by where [x; y] denotes the merge of x and y, and we use concatenation operation here (Dou, 2017;Chen et al., 2017).

such errors can be reduced by enhancing our NTM to phrase discovery topic models (Lindsey et al., 2012;He, 2016), which is worthy exploring in future work.


These three components can be updated simultaneously via a joint learning process, which is introduced in Section 2.3.

then, in target memory, via weighting target memory matrix t with ξ, we obtain the output representation R of the topic memory mechanism: the concatenation of R and U (embedded x Seq ) further serves as the features for classification.


Multi-label few-and zero-shot label prediction is mostly unexplored on datasets with large label spaces, especially for text classification.

we find as k increases, the differences between ZAGCNN and ACNN become more evident.


Also, frequent labels are generally easier to predict using machine-learning based methods.

this is intuitive for applications such as hashtag prediction on twitter.


Nevertheless, as discussed in Section 1, word-level likelihood or n-gram matching will greatly diverge from human evaluation manner.

• We propose a novel mutual reinforcement learning schema to further improve performance, which is transparent to model architectures.


Besides the long-term goal of building artificial intelligence, research on this task could become the auxiliary tool to better analyse poetry and understand the internal mechanism of human writing.


It is observed that both CVAE-based models and our proposed models outperform Seq2Seq by a large margin, showing the effectiveness of adding variational latent variable for response generation.

the size of the latent variables is set to 400.


The right part of Figure 1 denotes the decoder for generating the simplified sentence.

sentence simplification aims to reduce the complexity of a sentence while retaining its original meaning.


Then we calculate the precision, recall, and F1 score for using the correct rules.

we believe the knowledge base Simple PPDB will reduce uncertainty in the word selection.


Table 5 indicates that discrete states learned on the E2E data are quite pure.

moreover, because the discrete states align with particular fields (see below), it is generally simple to automatically infer to which fields particular latent states correspond, allowing users to choose which template best meets their requirements.


More importantly for us, however, they can also be used to guide generation.


Following standard notation (Liang et al., 2009;Wiseman et al., 2017), let x = {r 1 .

due to the black-box nature of generic encoderdecoder models, these systems have also largely sacrificed two important desiderata that are often found in more traditional systems, namely (a) interpretable outputs that (b) can be easily controlled in terms of form and content.


Finally, we highlight one significant possible issue with this model -the assumption that segments are independent of each other given the corresponding latent variable and x.

these approaches have always been conjoined with discriminative classifiers or rerankers in order to actually accomplish the generation (Angeli et al., 2010;Konstas and Lapata, 2013).


5 shows the optimal path according to the semantic substitution matrix in Fig.

two elephants in an enclosure next to fit through a brick building we need to investigate a better method to compress the lattice.


We provide 30 examples for each model asking each question from 3 participants, and use the majority choice.

these decoders are trained with embeddings from Section 2.4, with appropriate losses (RMSE for numerical and cross-entropy for categories).


Images Similar to text recovery, to find the missing images we use conditional GAN structure.

instead of learning a distinct vector for each entity and using enumeration to predict links, MKBE includes the following extensions: (1) introduce additional neural encoders to embed multimodal evidence types that the relational model uses to predict links, and (2) introduce neural decoders that use an entity's embedding to generate its multimodal attributes (like image and text).


besides, we also use the replay memory to store both old and recent experiences, and then train the network by sampling mini-batches from it.


The prior knowledge based s 0 helps the agent narrow down the candidate space more quickly when the target object is a popular object.

given the object set O and the question set A, we collect the normalized probabilities of the answer over "Yes", "No" and "Unknown" for each object-question pair.


Because the prior frequency distribution for each person-question pair is collected from thousands of users with the assumption that most of them do not lie when answering questions in the Q20 game.

it is necessary to design a better reward function to estimate a non-zero immediate reward r t , and make the long-time return G t = T k=0 γ k r t+k+1 more informative.


For example, if the input subset sources start with "p", we can learn to generate output subset targets that start with "p".

we, thus, want to emphasize that all settings-SET1, SET2, and SET3-can be considered low-resource.


, n are all bounded between −∆ and +∆ with probability at least 1/2 and since the ranking values are spaced apart by more than 2∆, the constraint ranking θ+ corresponding to the stochastic ranking vector θ + coincides with the original ranking with probability at least 1/2.

how can we investigate and understand the typological structure encoded by a probabilistic phonological framework?


This section illustrates a general method to leverage a given typology T of categorical grammars into a typology of stochastic grammars.

t/d deletion before a vowel entails deletion before a consonant.


Max-pooling over time When applying a fixedsize convolution over a variable-length sequence, the output is again of variable size.

similar to Finnish, multiple suffixes are merged.


(2016) used insightful syntactic-semantic features extracted from the target clause itself for SE type classification, which has achieved good performance across several genres when evaluated on the newly created large dataset MASC+Wiki.

our assumption is that discourse connectives are important to glue clauses together and removing them affects text coherence and information flow between clauses.


Intuitively, the connective "and" may occur between two clauses with the same SE type; "for example" may indicate that the following clause is not GENERIC.

after using the CRF layer to fine-tune the predicted SE label sequence, slight performance improvements were observed on the four small classes.


In the latter case we obtain a sentence representation by averaging the pre-trained embeddings of sentence words.

sification tasks (DRC, SAC, and SRC), we evaluate as baselines (1) the linear Support Vector Machines (SVM) with TF-IDF feature vectors and (2) SVM with RBF kernel and embedding features.


Despite (1) scientific texts being inherently argumentative (Gilbert, 1976), (2) the existence of theoretical argumentative frameworks (Toulmin, 2003;Kirschner et al., 2015), and (3) a wide range of argument extraction models in other domains (e.g., debates or essays, see Palau and Moens (2009); , inter alia), there is still very little work on automatic argumentation mining from scientific literature.


We train the MTL model jointly on different tasks by defining and minimizing the joint loss function L that combines task-specific losses L t .

there has been no work analyzing associations between argumentation and other rhetorical constructs in scientific writing, although such dependencies exist.


In this section we give a brief description of the temporal dependency parsing task (more details in Zhang and Xue (2018)).

our ranking model for temporal dependency parsing needs to rank not only the candidate antecedents but also the temporal relations between the antecedent and the anaphor.


Explanations of happenings in one's life, causal explanations, are an important topic of study in social, psychological, economic, and behavioral sciences.

finally, according to visual inspection, emojis ('E' tag) are crucial for discourse relation in social media so we take them as separate discourse arguments (e.g.,in "My test result... :(" the sad feeling is caused by the test result, but it cannot be captured by plain word tokens).


Then, we conducted univariate feature selection to restrict all remaining features to those showing at least a small relationship with the outcome.

top ngrams for causal explanations included more content words (e.g.


Note that NMF and SVD++ fail to train models when there is no ratings data available.

deepCoNN) to learn user or item embeddings.


For  instance, USA is more likely to be followed by Research than the other way around.

xML-CNN is in all cases better than the Youtube model, which shows how utilizing more linguistic structure than simply bag of entities is helpful in the NN framework.


Evaluating state of the art approaches to extreme multi-label classification.

the bigger challenge is often identifying topics for further research.


We also used different visualizations to support our findings and improve interpretability of our model.

the difficulty in predicting genre from book covers decreased the performance in multi-task settings with additional visual features.


The Q-value prediction of the permutation (1, 3, 2) almost triples that of the permutation (2, 3, 1).

our work addresses these two limitations as follows.


Since some of the posts and comments were deleted by Reddit, we remove all the deleted posts and comments from the datasets.

in addition, we compare the greedy procedure with two baseline approximation procedures.


On one hand, the agent can choose the action with the highest estimated Q-value to exploit its current knowledge of the Q-function.

the speed of content generation in a well-known discussion forum is breakneck.


• Apply softmax function to normalize the attention coefficients, • Use the normalized attention coefficients to compute a weighted sum of the comment embedding vectors, as the readout in this computation step, • The LSTM takes q l and r l as input and computes the next hidden state, q l+1 , Note that swapping any two comment embedding vectors, m c i t and m c j t , will not affect the query vector q l as well as the attention readout r l .

we need a procedure to approximate the predicted optimal action from a combinatorial action space.


Hence, where GRU m has the following parameters: The response vector o is obtained by summing output vectors in Q ′ , weighted by the relatedness measures in β: where o ∈ R Do .

unlike our proposed method, none of these methods have tried to model the inter-aspect relations.


Qualitatively, Limbic discovers aspect-sentiment pairs with higher lexical diversity.

topics with higher scores of NPMI and W2V are semantically more coherent.


We extract phrases from snippets by using two syntactic patterns commonly observed in opinionated texts including (1) existential (EX) with verb (VB) and adjective (JJ) and (2) noun (NN) with verb (VB) and adjective (JJ).

sEUs in an expansion relation have a high probability to be associated with the same sentiment.


rAM performs better than other baselines.


In addition, existing works train each instance separately.

(1) F-Aspect2Context estimates which context word has the closest similarity to one of the aspect word and are hence critical for determining the sentiment.


Traditional approaches (Jiang et al., 2011;Kiritchenko et al., 2014;Vo and Zhang, 2015) regard this task as the text classification problem and design effective features, which are utilized in statistical learning algorithms for training a classifier.

the simple average pooling in generating the guide vector might bring some information loss, especially for the aspect with multiple words or larger context.


For MOSEI dataset, word-level features were provided where text features were extracted by GloVe embeddings, visual features by Facets 2 & acoustic features by CovaRep (Degottex et al., 2014).

although it is often argued that incorporation of all the available modalities is always beneficial for enhanced performance, it must be noted that not all the modalities play equal role.


This model wrongly predicts the label of the utterance u 5 .

we use the CMU-Multi-modal Data SDK 1 (Zadeh et al., 2018a) for feature extraction.


In our proposed framework, we aim to leverage the multi-modal and contextual information for predicting the sentiment of an utterance.

the MU-SA attention (A v ) on V (visual) will be computed as follows, Mv(i,j) u k=1 e Mv(i,k) for i, j = 1, .., u The attention matrix A p ∈ R 3×d is computed for p = 1 st , 2 nd , ...u th utterances.


Finally, the outputs of pairwise attentions along with the representations are concatenated and passed to the softmax layer for classification.

for the MOSI dataset -which is relatively balancedwe obtain quite similar performance for both the classes i.e.


We have demonstrated through multiple experiments that DAS can better leverage unlabeled data, and achieve substantial improvements over baseline methods.

for DAS, we observe a nonlinear increasing trend where the accuracy quickly improves at the beginning, and then gradually stabilizes.


Thus, adaptation performance will decline with an increase in distribution difference.

n = n s + n t denotes the total number of training documents including both labeled and unlabeled 1 .


RMSProp optimizer with learning rate set to 0.0005 is used for all experiments.


The accuracy of synset matching is very important for our aspect taxonomy construction.

the reason is that it takes in-depth knowledge about the product to produce a set of words that best characterize the product, both in terms of the coverage and user interests.


Since Combo would contain noise introduced from combining webpages indexed by different search engines, it is not surprising that Combo performs slightly worse than TFG extracted from Google webpages which already cover a wide range of information solely.

experiment results using the task setting are shown in Table 2 and the detailed per-event results are listed in Table 3.


Similarly, we searched Baidu to get Chinese webpages and created CCMR Baidu.

even with such noise, our proposed methods can still outperform current state-of-the-art methods.


For the original+bias model variant, we find that performing this reduction improves the value of the objective function 65% of the time.

post-hoc methods seek to retroactively probe the behavior of an existing non-explanatory model.


Its purpose is to encourage high-recall explanations by trying to make accurate predictions from the antirationale, while the generator tries to prevent it from doing so.

this type of local explanation may not completely elucidate why a given example is assigned a given outcome, but it does simplify the relationship by identifying what attributes were considered in the decision.


They are jointly trained using a heuristic, non-saturating game loss.

we select 100 website privacy policies from the collection gathered by .


Other studies include crowdsourcing privacy policy annotations and categorizing data practices (Ammar et al., 2012;Massey et al., 2013;Wilson et al., 2016b,a), grouping text segments related to certain policy issues , summarizing terms of services (Braun et al., 2017), identifying user optout choices (Sathyendra et al., 2017), and many others.

for AC-GAN, the CNN discriminator uses convolutional filters of size {3, 4, 5} and 128 filters for each size.


For example, prior research suggests that engagement in particular sports is correlated with the political leanings (Hoberman, 1977) which suggest that improved models might need to capture deeper linguistic and contextual cues.

in case of the CNN's, we consider three convolutional filters of window sizes 3, 4, 5 each yielding a 100 dimensional feature map followed by max-over time pooling which is then passed through a fully connected layer to yield the output.


In particular, we compute z network as: To model the content of an article, we use a hierarchical approach with attention.

we track thousands of US based news outlets including country wide popular news sources as well as many local/state news based outlets like the (a) A sample news article.


• Orthographic: These features are very effective for detecting phishing websites, as malicious URLs tend to make excessive use of special characters and sections, and ultimately end up being longer.

the remainder of this paper is organized as follows: Section 2 provides an overview of related work.


They found that "fake news" pack a lot of information in the title (as the focus is on users who do not read beyond the title), and use shorter, simpler, and repetitive content in the body (as writing fake information takes a lot of effort).

the best features from this family are those about the page content, which includes a general description of the medium, its history, ideology and other information that can be potentially helpful.


In order to compute the article-related features, we did the following: (i) we crawled 10-100 articles per website (a total of 94,814), (ii) we computed a feature vector for each article, and (iii) we averaged the feature vectors for the articles from the same website to obtain the final vector of articlerelated features.

if a website is known to have published non-factual information in the past, it is likely to do so in the future.


Accordingly, we introduce topological learning for LJP and propose a unified framework, named as TOPJUDGE.

we conduct experiments on several realworld large-scale datasets of criminal cases in the civil law system.


In order to alleviate this problem, we use a weighted random sampler to sample instances from the training data.

with this basic distribution, the two posterior networks only need to capture the additional signals learned from x and labels.


During testing, the prior network will substitute for the posterior network.

the detailed computation process during training is shown as the following equations: where f is the Bi-LStM function and e is the output of the Bi-LStM layer at the last time step.


age, education rates, race) of the community from which the language originates.

without special treatment, the signal in extra-linguistic variables can be overwhelmed when combined with a large number of language-based features.


Instead, we use the MFC annotations to derive lexicons for each frame, which we then translate into Russian.

throughout this section, we refer to topics using their most representative words as determined by the "Lift" metric (Roberts et al., 2013).


(2016) 56.8 Our model 57.3 Table 5: Accuracy of primary frame classification.

these evaluation metrics determine how well our method captures which frames are present in a text.


Agenda-setting and framing can have a significant influence on public opinion by attending to particular issues at the exclusion of others (McCombs, 2002;Boydstun et al., 2013).

our main interest is in measuring the salience of frames in general, not merely focusing on the primary frame.


We can then calculate the product of all vector elements and retrieve the weighted sentiment of that cluster (e.g., 0.75 for "not a bad day in").

standardized residuals (z-scores) of the observed relative to the expected frequencies can be used to examine which observations deviated in which direction.


The intra-textual analysis approach helped us to uncover dynamics in the sentiment of vlogs that would otherwise have been blurred by an overall sentiment score.

we primarily aim to identify distinct trajectories of continuous sentiment.


The most common deletion among nonnatives was omission of the first 'r' in 'surprise', followed by omitting the first 'n' in 'government'.

content-independent features are expected to be more robust when they are used out-of-domain.


We therefore used LanguageTool, a rule-based grammar checker, to identify grammatical errors in the text.

to these two studies, we work with many more L1s (23); we explore various types of features, including features based on social network structures and content-independent features; and we evaluate our classifiers both in and outside of the domain of training.


To assess the robustness of our results, especially in the context of dialect identification, we repeated the experiments in a special scenario: we trained classifiers on all the data, but removed from the English training set users from Ireland.

there is a significant presence in this list to (subreddits focusing on) specific countries.


As mentioned before, the REINFORCE algorithm suffers from high variance in gradient estimation, mainly caused by using single sampleŷ to estimate the expectation.

in addition, we explored the source/target monolingual data for RL training.


The former one sequentially leverages the source-side and target-side monolingual data for RL training.

proposes a similar semi-supervised approach.


Sample Translations We present some sample translations from the tested models in Table 3.

we start by arguing that the recently proposed model-agnostic meta-learning algorithm (MAML, Finn et al., 2017) could be applied to low-resource machine translation by viewing language pairs as separate tasks.


The data for this experiment consisted of 200 sentences randomly selected from each of our eight topics.

we then calculated κ between the remaining instances in the expert and crowd gold standards.


In this paper, we propose a new sentential annotation scheme that is reliably applicable by crowd workers to arbitrary Web texts.

in a separate, simultaneously written paper (Stab et al., 2018) we evaluate our models in real-world application scenarios by applying them to a large document collection and comparing the results to a manually produced gold standard.


The multi-task learning (mtl) and transfer learning (trl) models are able to make use of auxiliary data that can potentially improve the results on the main task.

but despite its obvious usefulness, this sort of argument search has attracted little attention in the research community.


Also you can change the display and sound settings on each port.

we accomplish this by reducing each seed matrix to a single aspect embedding with the help of seed weight vectors z i ∈ R l ( j z ij = 1), and concatenating the results, illus-trated by the [4 × 2] aspect matrix in Figure 2: The segment is reconstructed as in Equation 5.


We also 12 Model obtained from github.com/bfelbo/deepmoji 13 The proposed model trained on six emotions dataset.

by adjusting graph G s with G o , we obtain a graph G e , referred to as the emotion graph, which preserves emotionrelevant tokens and is obtained in two steps: (1).


In contrast, word2vec contains over 3 million unique word embeddings and has been proven effective for text classification.

rare words like "whaaaaaat" and "thee" help to implicitly convey intense emotional expressions, which are also captured and considered important by our enriched patterns.


We also apply input reduction to single-input tasks including sentiment analysis (Maas et al., 2011) and Quizbowl (Boyd-Graber et al., 2012), achieving similar results.

most reduced inputs are nonsensical to humans ( Figure 2) as they lack information for any reasonable human prediction.


In this example, the leave-one-out method will not highlight "Broncos".

the previous section explains the observed pathologies from the perspective of overconfidence: models are too certain on rubbish examples when they should not make any prediction.


Recently, self-attention mechanism has been successfully applied to the field of sentence encoding, it utilizes the attention mechanism to relate elements at different positions from a single sentence.

at the same time, syntactic information can be easily integrated in the model.


This normalizeand-sample step is repeated M times, yielding M unique sentences to include in the summary.

bANDIT-SUM is shown to be better than Refresh and Sum-maRuNNer in terms of overall quality and nonredundancy.


(2018) recently proposed an extractive summarization approach based on deep Q learning, a type of reinforcement learning.

empirical results show that our method performs better than or comparable to state-of-the-art extractive summarization models which must be pre-trained on extractive labels, and converges using significantly fewer update steps than competing approaches.


(2016) used a list of 3,000 most common English words; Lee and Yeung (2018) used an ensemble of vocabulary lists of different complexity levels.

we use all the features (NRR all ) in §3.2 except for the context features as we are classifying paraphrase rules in PPDB that come with no context.


We consider two state-of-the-art CWI systems: (i) the nearest centroid classifier proposed in (Yimam et al., 2017), which uses phrase length, number of senses, POS tags, word2vec cosine similarities, ngram frequency in Simple Wikipedia corpus and Google 1T corpus as features; and (ii) SV000gg (Paetzold and Specia, 2016b) which is an ensemble of binary classifiers trained with a combination of lexical, morphological, collocational, and semantic features.

these assumptions are not always accurate and are often the major source of errors in the simplification pipeline (Shardlow, 2014).


In this work, we use the most straightforward heuristics to perform template induction, utilizing the established correspondences and inferred parameters.

(2) String-to-string: Similarly for strings, since simple matching could fail if the text contains Bob to refer to Bob Smith.


This indicates that a syntactic inductive bias is beneficial even when using sophisticated neural architectures.

targets are disambiguated to lexically specific senses rather than shared frames, and a set of generic roles is used for all targets, reducing the argument label space by a factor of 17.


To investigate the performance of the syntactic scaffold, we focus on the frame SRL results, where we observed the greatest improvement with respect to a non-syntactic baseline.

2 pipelines introduce the problem of cascading errors (T 2 's mistakes affect the performance, and perhaps the training, of T 1 ; He et al., 2013).


The HAQAE model performs comparable or better than the NOHIER model despite (in this case at least) having fewer parameters 4 .

similar to VQ-VAEs, we parameterize the individual factors of the posterior, q i (z i = k|pr(z i ), x), as: is an encoding function specific to latent z i and e ij is the jth value em-beddings for z i .


Note that, since the Berkeley parser is based on PCFGLA grammar, it may fail to get the syntactic outputs for some sentences, while the other parser does not have that problem.

early work showed that syntactic information is crucial for learning longrange dependencies, syntactic constituency structure and global constraints (Punyakanok et al., 2008;Täckström et al., 2015), while initial studies on neural methods achieved state-of-the-art results with little to no syntactic input (Zhou and Xu, 2015;Wang et al., 2015;Marcheggiani et al., 2017;He et al., 2017).


The constituent "常常" (often) are adjuncts attached to VP structure governed by the verb "练习"(practice), which will not be labeled as AM in terms of the verb "做饭"(cook-meal).

what types of error are more problematic for the neural syntax-agnostic one over the L2 data but can be solved by the syntax-based one to some extent?


This demonstrates again the importance of syntax in semantic constructions, especially for learner texts.

retraining the SRL classifier is much more effective.


Furthermore, the neural-parser-based system achieves the best overall performance on the L2 data.

one is a traditional syntaxbased SRL system that leverages a syntactic parser and manually crafted features to obtain explicit information to find semantic roles (Gildea and Jurafsky, 2000;Xue, 2008) we employ the system introduced in Feng et al.


o u t and o s t−1 are the one-hot representations of such semantic frames for the current user input and the last system output respectively.


(2017) proposed to use BBQ-Networks to extend the domain.

indicates the rate at which the system takes reasonable actions in unsupported dialog situations.


(2016) proposed to integrate turnlevel feedback with a task-level reward signal to learn how to handle new user intents.

extending RL-based systems to handle new intents and slots requires a system redesign.


In the human evaluation experiments, real users interact with different models without knowing which agent is behind the system.

the optimizer for all the neural networks is RMSProp (Hinton et al., 2012).


At a given point in the dialog, the dialog system has multiple correct next utterances (in orange).

the reverse happens when the dialog system encounters a batch with dialog B.


Here an invalid status indicates that agents have chosen a status that they are not supposed to reach according to Figure 2.

our self-play model is initialized using a model trained from the supervised learning.


Then, we define the loss as the square error between R and F (Y ): Although Mueller et al.

the second term in Equation 1 models the loglikelihood of the outcomes.


We can observe that the MAE values of both our model and S2BS are smaller than Original.

we first calculate the Jaccard Index (JI) for each pair of sentences, and keep those with JI values no less then 0.5 as the initial pairs.


Moreover, the dual reconstruction is more effective than the others.

s2Bs only disentangles one factor for capturing both content and outcome properties, and thus it cannot perform the same operations on sentence pairs.


Continuing the running example, given the expected outcome 3.1, "The food is OK" is an appropriate output, but "The food is extremely delicious" and "The service is OK" are not.

the revised sentence X * is generated from X 0 and Y * via the decoder D: the content of X 0 is preserved with Z 0 , and the expected outcome is achieved with Y * .


Table 2 shows that 11.7% of facts in the game summaries can be inferred based on the input data.

for the ESPN dataset, we select 50 games and each with one generated sentence.


Note that for classical Chinese poem, the dominant genres are quatrain and eight-line regulated verse with either 5 or 7 characters in each line.

each criterion is rated from 1 to 3, representing bad, normal, good, respectively.


Otherwise, if the encoder states are passed to the discriminator, the loss would be back-propagated to the encoder and disturb CVAE training accordingly.

µ and σ are the key parameters 6 to be learned, and they are computed by where W q and b q are trainable parameters.


In addition, we observe that all models perform poorly on column selection.

spider is the only one text-to-sQL dataset that contains both databases with multiple tables in different domains and complex sQL queries It tests the ability of a system to generalize to not only new sQL queries and database schemas but also new domains.


To preprocess the structured data, we convert the boolean feature family friendly into a meaningful phrase ("family friendly") by using the slot name.

all in all the output of the setup model BLEU ROUGE-L NIST supervised baseline E2E challenge (Dušek and Jurcıcek, 2016) Table 4: Results on the E2E dataset.


They showed that using noise in the encoder step is helpful to learn a better sentence representation.

the output still sounds very machine generated, but stops hallucinating additional information.


To tackle the two issues, we propose an answer-focused and position-aware model.

the baseline copies "leading theory" that is far away from and unrelated to the answer "homologous recombination", but neglects the phrase "second theory" that is close and relevant to the answer.


The standard training method is based on Maximum Likelihood Estimation (MLE).

the main problem is that the reward given by the classifier cannot reflect the novelty of text accurately.


Currently, a popular model for text generation is the sequence-to-sequence model (Sutskever et al., 2014;.

input: i brought friends -seven of them -who'd never been to a Bar Louie before.


Our work is a first step in this direction, and we hope it will lead to more research in designing the right metrics for AQG.

", "What kind of vehicle is parked the sidewalk?"


We repeated this process for k = 20 times and computed the optimal w i 's and δ each time.

we take questions from existing datasets for document QA, knowledge base QA and visual QA and add systematic noise to these questions.


), Meaningfulness (does the poem convey some certain messages?)

hence it's impossible to enumerate all possible sequence Y for computing the integration.


We computed p-values for each pair of models: MMI200 vs DC-MMI200 and TA vs DC-MMI200.

3 As illustrated in the figure, response generation that is based on maximum likelihood is biased towards stop-words and therefore results in responses that are safe (likely to be plausible in the context of the input), but also bland (don't contribute any new information to the conversation).


These are typically added to the learning objective for semi-supervised scenarios where available labeled data is limited.

both of these proceedures are difficult to integrate into decoding, because they are computationally expensive and would need to be called repeatedly within the inner loop of the decoder.


End-to-end neural response generation methods are promising for developing open domain dialogue systems as they allow to learn from very large unlabeled datasets (Shang et al., 2015;Sordoni et al., 2015;Vinyals and Le, 2015).

the semantic representation of a dialogue context x is then ; for a response y, it is Here, w j and v i are importance weights for each word in the sentence.


For the example in Figure 4, the vocabulary distribution, P v , has the word "from" as the most probable word and the entity distribution, P e , has the entity "S. B. Moyo" as the most probable entity.

we use a LSTM decoder, which applies a temporal attention (Bahdanau et al., 2015) to the frame representations at each step.


We believe that this is because, the questions generated by bots are not natural (human-like) and therefore does not translate to improved human dialog.

figure 7: The mean rank of all the models on the basis of METEOR score are plotted on the x-axis.


CIDEr metric downweights the n-grams common in all image captions, which are similar to tf-idf.

we use three standard image captioning evaluation metrics, BLEU (Papineni et al., 2002) and METEOR (Denkowski and Lavie, 2014), RE-OUGE (Lin, 2004) and CIDEr (Vedantam et al., 2015), to evaluate the quality of both the generated templates and generated captions.


We then retrieve topically-related images from the Flickr database, which have the same tags as the input image.

we adapt the encoder-decoder architecture which is successful in the image captioning task (Vinyals et al., 2015;Xu et al., 2015).


We evaluate in this setting by treating the sentences in the ground truth description as multiple reference captions.

we used crowd-sourcing to collect text descriptions of differences between pairs of image frames from video-surveillance footage (Oh et al., 2011), asking annotators to succinctly describe all salient differences.


In another related task -referring expression comprehension (Kazemzadeh et al., 2014;Mao et al., 2016;Hu et al., 2017) -the model has to identify which object in the image is being referred to by the given sentence.

similar to prior work (Xu et al., 2015), we extract features using a lower level convolutional layer instead of fully connected layer.


We report quantitative metrics like CIDEr (Vedantam et al., 2015), BLEU (Papineni et al., 2002), METEOR (Denkowski and Lavie, 2014), and ROUGE-L, as is often reported by works in image captioning.

we consider the following four features: the length, width, and area of the smallest rectangular region enclosing cluster z i , and the number of active elements in mask C z i .


This is especially apparent when standard metrics assign a low score to a generated sentence.

finally, we measure image and language consistency during training for the fC model and note that at the beginning of training errors are more consistent with the language model, whereas towards the end of training, errors are more consistent with the image model.


All sentences are generated using beam search and a beam size of 5.

the comparison between NBT and other models is not completely fair, but we include the number for completeness.


methods that incorporate bounding box attention (as opposed to relying on coarse feature maps), consistently have lower hallucination as measured by our CHAIR metric.


At the encoding stage, we use the hierarchical bi-directional RNN to encode the sentences and the text document, use the RNN and the CNN to encode the image set.

a logistic classifier is then applied using features computed based on the RNN model.


For each sentence beam, we try M sentences or captions and keep the one achieving best word-level scores.

we propose three multi-modal attention mechanisms which take images and captions into consideration.


To enable copying out-of-vocabulary words from source text, a pointer similar to  is built upon the attention distribution and controlled by the generation probability p gen : where U c , U s , U x , and b ptr are learnable parameters.

the infant was found about 20-30 metres from the water 's edge .


4), we observe non-significant improvements in ROUGE compared to the baseline.

the attention mechanism  Figure 2: Our 2-decoder summarization model with a pointer decoder and a closed-book decoder, both sharing a single encoder (this is during training; next, at inference time, we only employ the memory-enhanced encoder and the pointer decoder).


The higher structuralcompression and structural-coverage scores will lead to higher ROUGE scores.

the reason is that the K-best sentences generated by a word decoder will mostly be similar to each other tan et al., 2017a).


At every time-step, a model is aware of the previously generated words.

this technique could be used for low-resource domains and for problems with limited data availability.


Existing preference-based interactive learning methods suffer from high sample complexity, i.e.

1b), responsible for querying preferences from the oracle and approximating the ranking of summaries, and an RL phase (the left cycle in Fig.


We adopt a simple alternating training strategy to switch training between the main task and the auxiliary task.

concretely, we first build a baseline model by training the seq2seq model on the labeled corpus D p .


Meanwhile, the coverage mechanism sums up all the historical attention distributions.

all annotators will give a score of 1 to 5.


in our case, we are only interested in calculating precision metric.


By deriving extractive references, we can easily construct a pyramid.

although, various evaluation measures have been proposed, ROUGE-N (Lin, 2004), Basic Elements (BE) (Hovy et al., 2006) remain the de facto standard measures since they strongly correlate with various manual evaluations and are easy to use.


The pyramid method has two advantages over conventional manual evaluations: (1) the score is not intuitive but is systematically computed, i.e., the score can be explained as the sum of weights of SCUs in the pyramid, (2) the correspondences between the SCUs in a summary and the pyramid tell us whether the summary contains important SCUs or not.

φ(e j , m k ) returns the similarity score between the j-th EDU in the source documents and the k-th EDU in the reference summary as follows: LCS(•, •) returns the Longest Common Subsequence between e j and m k .


The weight of an SCU in the pyramid is defined as the number of reference summaries that contain the SCU.

we assume that a bipartite graph in which the vertices represent EDUs in the reference summary and source documents, and the edges represent similarity scores between the EDUs.


EDU weights are defined as the number of reference summaries that contain each EDU (see Figure 1).


We use two different adversarial training methods to train D and G; as shown in Fig.

the generator network and reconstructor network are a seq2seq hybrid pointer-generator network, but for simplicity, we omit the pointer and the attention parts.


A human-readable representation should comply the rule of human grammar and can be comprehended by human.

it is also possible to directly optimize evaluation metrics such as ROUGE (Lin, Figure 1: Proposed model.


The experi-mental results have proven the suitability of combining the two approaches.

looking at the results for subtask C, we can see that sizeable gains are possible when using gold labels for subtasks A and B as features to DNN C , e.g., adding gold A labels yields +6.90 MAP points.


(2016) combined recurrent and CNN models for question representation.

300-dimensional embedding vectors, trained on 100 billion words from Google News (Mikolov et al., 2013).


We examined the influence of the two filtering heuristics: (i) entity type recognition (Section 3.1) and (ii) attention (Section 3.2).

we investigated whether these questions merely require word matching (see Section 4).


this kind of question does not require a complex understanding of language-e.g., multiple-sentence reasoning, which is known as a more challenging task (Richardson et al., 2013).


We want to use these commonsense paths to fill in the gaps of reasoning between hops of inference.

we do this by computing a cumulative node score from the bottom up, where at the leaf nodes, we have c-score = n-score, and for c l not a leaf node, we have c-score(c l ) = n-score(c l ) + f (c l ) where f of a node is the average of the c-scores of its top 2 highest scoring children.


where r L denotes a special "linking" relation.

models which operate on these patterns (e.g.


Inspired by these observations, we propose to expose all of these representations to better fuse information across layers.

representation Interpretation Several researchers have tried to visualize the representation of each layer to help better understand what information each layer captures (Zeiler and Fergus, 2014;.


Second, using the diversity regularization in Figure 5(b) can encourage each layer to contribute more equally to the aggregation.

for the En⇒De task, we trained on the widely-used WMT14 dataset consisting of about 4.56 million sentence pairs.


In this work, we take the advanced Transformer as an example, which will be used in experiments later.

accordingly, the training speed decreases due to more efforts to train the new parameters.


The curves of the AMRs during the decoding on the MT03 test dataset under the proposed methods are shown in Figure 3.

dropout was applied to the output layer with dropout rate of 0.5.


Furthermore, we can find that for longer sentences that contain too detailed information, it only extracts the key information.


Output: A relevant narrative story following the input.

since the style of the sentence compression dataset is very different from that of the narrative story dataset, it is difficult for the pre-trained module to give narrative text accurate compression results.


(2017) adopt the variational inference strategy to reduce the training variance by optimizing over latent continuous variables.

the last problem is the sampling mechanism of c in Eq.


Early readability formulas were based on superficial text features such as average word lengths (Kincaid et al., 1975).

in such models, long-distance dependencies between words cannot be captured effectively due to the limits of the memorization capability of recurrent networks.


ATTOrderNet becomes more competitive through combining both advantages of LSTMs and self-attention mechanism.

3.1 Datasets Accident, Earthquake: two datasets obtained from (Barzilay and Lapata, 2008).


using position in the lead paragraph as a proxy for salience, we only include entities mentioned at least once in the first three sentences of each article.


if multiple traits are combined, they provide considerable information on a branch length, or the time elapsing from a parent to a child, because the elapsed time is roughly inversely proportional to the similarity between the two nodes.


The pulse in each box shows the corresponding probability estimated by directly fitting the 7 × 7 TRM of the surface feature.


One of the main developments in diachronic typology in the last decade has been the application of powerful statistical tools borrowed from the field of evolutionary biology (Dediu, 2010;Greenhill et al., 2010;Dunn et al., 2011;Maurits and Griffiths, 2014;Greenhill et al., 2017).

we used linear regression with L1 regularization (i.e., lasso).


Within Switzerland, we see a distinction between the German (lighter green) and the French-speaking area around Lausanne and Geneva (darker tones).

if our assumption holds, we expect to see gradual continuous change between cities and regions.


types of usage for vulgar words (Andersson and Trudgill, 1990;Pinker, 2007;Wang, 2013).

to avoid uncertainly associated computing statistics over distributions with low counts, we keep only words that appear more than 10 times in our data set (43 words) after collapsing variants of the same word (e.g.


7 is a more difficult task because the answer word "Nora" appears only once in the entire article.

as shown in Table 4, LSTM-Shuttle still yields improvement at the character level.


While, early stopping is not fast enough for classification, we can do better with text shuttling.

we cannot train it as with a normal classification problem.


Moreover, the anchor-box representation allows us to detect boxes of different classes.

when processing structured and formatted documents in which the relation between words is impacted not only by the sequential order, but also by the document layout, NLP can result in significant shortcomings.


Textual information is often represented through structured documents which have an inherent 2D structure.

we observe errors in both, the segmentation mask and the bounding box predictions.


Balduzzi and Ghifary (2016) applies type-preserving transformations to discuss the capacity of various simplified RNN architectures.

typically, c t 1 is multiplied with a parameter matrix to compute f t , e.g., the inclusion of V f c t 1 makes it difficult to parallelize the state computation: each dimension of c t and f t depends on all entries of c t 1 , and the computation has to wait until c t 1 is fully computed.


Akin to Xiong et al., 2017;, the NIRM baselines and the proposed NPRF are employed to re-rank the search results from BM25.

robust04 contains 528,155 documents and 249 queries used in the TrEC 2004 robust track (Voorhees, 2004).


propose the PACRR model based on the idea that an appropriate combination of convolutional kernels and pooling operations can be used to successfully identify both unigram and n-gram query matches.

each document's ultimate relevance score depends on both its interactions with the original query and its interactions with these feedback documents.


A major advantage of the proposed NPRF over existing neural IR models is that it allows for soft-matching query-related terms that are missing from both the query and the target document.

figure 2 shows the performance of NPRf ds as the number of feedback documents m varies (top), and as the number of top terms k varies (bottom).


To verify this, we investigate whether the BoW representation and the learned latent code can be reconstructed from each other.

trade-off Comparison Besides the overall perplexity, we are also interested in the trade-off be-tween reconstruction loss and KL, and the contribution of KL to the whole objective.


Each year STS usually has 4 to 6 different tasks and we only report the averaged Pearson's scores for clarity.

we note that our wME embedding allows potentially other random distributions, and other types of word embeddings, making it a flexible and powerful feature learning framework to utilize state-of-the-art techniques.


For additional results on all KNN-based methods, please refer to Appendix B.3.

the hyperparameters of WME are obtained through a 10-fold cross validation only on training set.


This is done independently for each language, such that the space of indices we use for each language is separate.

we apply this technique only to update G (and not H) because reversing cluster assignments in G can be done much more efficiently than in H -the total number of monolingual clusters (the clustered elements in G) is significantly smaller than the number of documents (the clustered elements in H).


As more tasks are added step by step, MTLE produces increasing performance gains for each task and achieves an average improvement of 5.5% when all the four tasks are trained together.

models of these papers ignore essential information of labels and mostly can only address pairwise interactions between two tasks.


It is obvious that although the performances of both models decrease with the decrease of the label frequency, our model continues to perform better than the baseline on all levels of label frequency.

hier refers to hierarchical model, and the subsequent number refers to the length of sentence (word) for sentence-level representations (p < 0.05).


We demonstrate the performance of five models with different attention implementation for comparison, which are model without attention, one with only attention to the source annotations from LSTM, one with only attention to the semantic unit representations from the MDC, one with the attention to both the source annotations and semantic unit representations (additive) and hybrid attention, respectively.

a special design for the CNN is the implementation of dilated convolution.


As the frequency of our label data is similar to a longtailed distribution and the data are organized by descending order of label frequency, the Seq2Seq model is inclined to model the distribution.

our model can generate semantic unit representations from the information at phrase level with small dilation rates and those at sentence level with large dilation rates.


(2017) add the adversarial losses in multi-task RNNs, which can alleviate the interference among tasks by finding a common feature space for tasks.

more concretely, we introduce a coupling coefficient c (k) ij between capsule i in l-th layer and capsule j in class capsule (l + 1)-th layer for task k, which is the result of a softmax function on b Then, instantiated parameter v ij is restricted in range [0, 1], which represents the probability that capsule i belongs to class capsule j in task k. And it is update by the algorithm is described in Algorithm 3.5.


In addition to these regularization techniques, Merity et al.

finally, our proposed method, DOC, achieved the state-of-the-art score on the standard benchmark datasets.


For this experiment, we modified a publicly available encode-decoder implementation 11 .

we conclude that DOC positively affects the training of language modeling.


(1) In disfluent structures, the repair (e.g., to Denver) frequently seems to be a "rough copy" of the reparandum (e.g., to Boston).

each non-input representation X (k) , k > 0 is formed by column-wise stacking the output of one or more convolution or auto-correlation operations applied to the preceding representation, and then applying an element-wise non-linear function.


We provide an illustration of the auto-correlation operation in Figure 2.

models which are able to detect "rough copies" are likely to perform well on this task.


In this section, we introduce our proposed auto-correlation operator and the resulting autocorrelational neural network (ACNN) which is the focus of this work.

to many other sequence tagging tasks (Plank et al., 2016;Yu et al., 2017), "vanilla" convolutional neural networks (CNNs) and long shortterm memory (LSTM) models operating only on words or characters are surprisingly poor at disfluency detection (Zayats et al., 2016).


Each sample in this dataset is an article, associated with a label indicating its topic: world, sports, business or sci/tech.

for each testing sentence, we compute the saliency scores of words.


Despiting the linear structured left-branching and rightbranching tree encoders, we find that, tree-based encoders generally perform better than Bi-LSTMs on tasks of sentence relation and sentence generation, which may require relatively more long term context features for obtaining better performances.

as desired, on shortsentence groups, the performance gap between Bi-LSTM and binary balanced tree LSTM is not obvious, while it grows with the test sentences turning longer.


Shallowness may lead to shorter path for information propagation from leafs to the root representation, and makes the representation learning more easy due to the reduction of errors in the propagation process.

leftbranching tree LSTM has a quite similar structure with linear LSTM, and it can be viewed as a linear LSTM-on-LSTM structure.


The target word here should be the name of the Albanian prime minister, which is out of vocabulary.

words with the same sememes have a high probability to appear in the articles intuitively.


• BP-sLDA (Chen et al., 2015): it applies back propagation over a deep architecture together with stochastic gradient/mirror descent for  The third kind is a supervised n-gram model named sNTM, which is applicable to predict both categorical and real-valued labels for new documents (Cao et al., 2015).

the results indicate that although all models can learn meaningful topics, SLtM performs better than baseline models in label-specific topic discovery.


When we do not use our sample strategy (mini-corpus is 1), the performance drops by a large margin.

in this way, the gradient can be backpropagated through the variational parameters.


From Table 1, we see that the average size of documents size in 20 Newsgroups is relatively short (88 compared with 302 in All News).

, m K ) denotes the weight for all words under K topics.


Moreover, analysis of social data is often principally concerned with understanding relationships between documents (such as question-asking and -answering), so a model able to capture such features is of direct scientific relevance.

to model topic transitions, we score the embeddings of parent-child pairs with a separate coupling potential as shown in Figure 1 (comments with no parents or children receive additional start/stop biases respectively).