Therefore, we introduce a parameterized idiom detector, which is used for detecting the boundary between literal and idiomatic meanings.

if a compositional interpretation is nonsensical in the context of a sentence, then the detector is supposed to check whether an idiomatic sense should be taken and whether it makes sense.


Different from this work, we focus on how to integrate detection and understanding of idioms into a unified end-to-end model, in which an idiomatic detector is introduced to adaptively control the semantic compositionality.

in the whole process no extra information is given to tell which phrases should be regarded as noncompositional.


Orthogonal Matrix Factorization (OrMF) (Guo et al., 2014) is the extension of WMF, with an additional objective to obtain nearly orthogonal dimensions in matrix factorization to discount redundant information.

for the (vec) version, vectors of a pair of sentences v 1 and v 2 are converted into one feature vector, , by concatenating the element-wise sum v 1 + v 2 and absolute differ- We also provide the (sim) variation, which directly uses the single cosine similarity score between two sentence vectors.


Both the edit distance and the LR models exploit surface word features.

the LR model that uses lemmatization and ngram overlap features achieves very competitive performance on all datasets.


We assume a one-to-one mapping g : Z → C, where C ⊂ U is the set of canonical utterances.

every logical form will be converted into a unique canonical utterance deterministically (Figure 1).


ble 4 show a consistent, moderate to strong negative correlation between the two variables.

cross-domain training is more beneficial when in-domain training data is less abundant, which is reasonable because in that case the model can learn more from the source domain data that is missing in the training data of the target domain.


We train different neural network models for these two problems, and interpret their outputs as factors in a graphical model for performing joint inference over the distribution of frames and semantic roles.

our frame identification model is a simple multi-layer neural network that learns ap-propriate feature representations for frame disambiguation.


To leverage the power of these two networks, we transfer the knowledge in the sequential model, encoded as its predictive distributions.

we do this by training a single relational model with an objective that measures both its prediction accuracy with respect to the true semantic role labels, and its match to the probability distributions provided by the sequential model.


We then map these features into a low dimensional space.

we compute an embed-ding of the argument features: , wherev a w ∈ R k is the average of argument word embeddings,v a d ∈ R k is the average embedding of the argument's dependents, andv a l ∈ R k is the average embedding of the corresponding dependency labels.


To make use of such knowledge in the relational model, we first transform the sequence distributions into span-based distributions.

we derive the marginal distribution for any given span a = (w s , ..., w t ), 1 ≤ s ≤ t < n, and a non-empty semantic role label r as: Here we drop the dependency on p, and f for brevity.


Instead, we extract candidate arguments using a pretrained sequential SRL model (described in § 4.1).

we extract the argument spans from the K-best semantic role label sequences output by the sequential model.


All of our models significantly outperform the baselines in F1.

our integrated model achieves the best F1 score of 80.5%.


We propose a novel objective function to optimize the entire framework end-to-end, where we focus more on the group-level prediction and take into account multiple important factors.

the objective function explicitly penalizes three potential errors in answer triggering: (1) false-positive and (2) false-negative predictions of the existence of a correct answer, as well as (3) ranking incorrect answers higher than correct ones.


The CRFs employ feature representations which we obtain using neural networks.

discourse constituents at all levels are modeled with recurrent neural networks (see Section 4.2 for details).


Our model differs from standard linear-chain CRFs in that the score between adjacent hidden nodes is not calculated based on a transition matrix, but is learned from observations instead.

given a constituent sequence C, the probability of forming the structure sequence Y str is written as: where the u str t represents the unary potential score of structure node l t = y t , depending on constituents c t and c t+1 .


We therefore modify the two CRFs into slidingwindow versions.

at each level, when decoding a constituent sequence, for each hidden structure node l i or relation node r i , we find all windows of constituents that contain the hidden node, and set the hidden node's label according to the window with the maximum joint probability.


In the second row, CIDER uses flat representations without LST-M minus features.

each constituent is represented as the average of the LSTM outputs within it.


Therefore, at each position i of the sequence, we obtain two s- Then we concatenate them to get the intermediate state, After that, we sum up the sequence states to get the representations of Arg-1 and Arg-2 respectively as follows: Finally we concatenate the two argument representations R Arg 1 and R Arg 2 as the argument pair representation, i.e., Clearly, in this way, there is no any correlation and interaction between the two arguments.

whatever the types of discourse relationship they hold, the argument pair representation R pair is independent from R Arg 1 or R Arg 2 .


We also perform multi-class classification on PDTB and CoNLL-2016.

a four-way classification on the four top-level classes in PDTB and a 15-way classification on the 15 sense labels in CoNLL dataset.


Suppose k candidate antecedents are extracted, our task is to determine the correct antecedent of zp from its candidate antecedent set A(zp) = {c 1 , c 2 , ..., c k }.

these candidate antecedents are represented in form of vectors which are stacked and regarded as the external memory mem ∈ R l×k , where l is the dimension of v c .


A graphical representation of this model is displayed in Figure 2.

the ZPcentered LSTM contains two standard LSTM neural networks, i.e., the LSTM p that encodes the preceding context of the AZP in a left-to-right manner, and the LSTM f that models the following context in the reverse direction.


For a given AZP, a set of noun phrases (NPs) is extracted as its candidate antecedents.

we generate the external memory by utilizing these candidate antecedents.


Inspired by this, we propose a novel method to produce representations of the candidate antecedents by utilizing both their contexts and content words.

we use the subtraction between LSTM hidden vectors to encode the candi-date antecedents, as illustrated in Figure 3.


For each multi-valued feature, we convert it into a corresponding set of binary-valued features 2 .

for the t-th candidate antecedent in the memory, v ct , taking the vector representation of the AZP v zp and the corresponding feature vector v (f eature) t as inputs, we compute the attention score as α ).


Feature ablation tests revealed that while combining all features is beneficial for the prediction of the anaphoric and pleonastic classes, the same is not true for the event class.

the inclusion of semantic features, which we designed as indicators of eventness, appears to be harmful (Figure 1).


To demonstrate the robustness of the this approach, in all experiments we report the results of a single configuration of GrASP parameters, selected based on a quick analysis over a small portion of the claim-sentence detection data (task (a) below).

4 we used minimal frequency threshold t 1 = 0.005, correlation threshold t 2 = 0.5, size of the alphabet k 1 = 100, number of patterns in the output k 2 = 100, maximal pattern length maxLen = 5, and window size w = 10.


Thus, we exploited these tags to develop a standard supervised classifier for the topic categorization of editorials.

we trained the classifier on all 1.29 million non-editorial articles that are assigned a topic, and then used it to classify editorials with unknown topic.


Based on the mean of lower bound and upper bound, we perform a significance test among the evidence type distribution across topics.

we use the chi-square statistical method with a significance level of 0.001.


While the authors target the same evidence types that we consider in our work, they approach a different task.

they classify only evidence that is related to given claims.


Instead of reading a long list of reviews, customers usually are only willing to view a handful of helpful reviews to make their purchase decisions.

helpful reviews have even greater influences on the potential customers' decision-making processes and thus on the sales; as a result, the automatic identification of helpful reviews has received considerable research attentions in recent years (Kim et al., 2006;Liu et al., 2008;Mudambi, 2010;Xiong and Litman, 2014;Martin and Pu, 2014;Yang et al., 2015.


As there are 7 types of labels, the number of possible combinations is 2 7 −1 = 127, and thus the possible number of combination ratio pairs is 127 × 126 = 16002.

the component-level feature is a 16002-dimensional real vector.


This work presents the first neural network-based approach to link extraction in argument mining.

we propose a novel architecture that applies Pointer Network sequence-tosequence attention modeling to structural prediction in discourse parsing tasks.


Second, we follow previous work that assumes a tree structure for the linking of ACs (Palau and Moens, 2009;Cohen, 1987;Peldszus and Stede, 2015;Stab and Gurevych, 2016).

a given AC can only have a single outgoing link, but can have numerous incoming links.


We implement and compare four types of neural models: 1) The previously described joint model from Section 3.4 (called Joint Model in the tables); 2) The same as 1), but without the fullyconnected input layers (called Joint Model No FC Input in the table); 3) The same as 1), but the model only predicts the link task, and is therefore not optimized for type prediction (called Single-Task Model in the table); 4) A non-sequence-tosequence model that uses the hidden layers produced by the BLSTM encoder with the same type of attention as the joint model (called Joint Model No Seq2Seq in the table).


in Argument Mining (AM) (see (Lippi and Torroni, 2016) for a recent overview).

relation-based AM (Carstens and Toni, 2015) aims to automatically identify argumentative relations to create Bipolar Argumentation Frameworks (BAFs) (Cayrol and Lagasquie-Schiex, 2005).


However, this model does not address one aspect of lattices that we argue is critical to obtaining good translation results: their ability to encode the certainty or uncertainty of the paths through the use of posterior scores.

we postulate that these scores are essential for tasks that require handling lattices with a considerable amount of erroneous content, such as those produced by ASR systems.


In this paper, we propose a lattice-tosequence model that accounts for this uncertainty.

our contributions are as follows: • We employ the popular child-sum TreeLSTM (Tai et al., 2015) to derive a lattice encoder that replaces the sequential encoder in an attentional encoder-decoder model.


Our lat2seq framework uses this network as encoder, computing the attention over all lattice nodes.

3 we replace (1) by the following: Similarly, we encode the lattice in backward direction and replace (2) accordingly.


when generating the i-th target word, the attention factor of the j-th source word (and its neighbors, precisely) is measured by the relevance between the current hidden state of the decoder, denoted by s t−1 , and the hidden state of the encoder at the j-th word h j .


Efficient model-memory joint training is beyond the scope of this paper and can be investigated in future work.

the parameter β could be optimized to balance the contribution from the model part and the memory part, but constrains have to be carefully settled to avoid overfitting to the training data.


This dictionary will be used as the knowledge to construct the local memory at run-time.

if an OOV word is encountered on either the source or target side during local memory construction, the vector of a similar word is borrowed to represent the OOV word.


However, state-of-the-art NMT approaches based on an encoder-decoder architecture with an attention mechanism as introduced by  exhibit weaknesses that sometimes lead to MT errors which a phrase-based MT system does not make.

pBMT usually can better translate rare words (e.g.


Significantly different from conventional statistical machine translation (SMT) (Brown et al., 1993;Koehn et al., 2003;Chiang, 2005), NMT adopts a big neural network to perform the entire translation process in one shot, for which an encoderdecoder architecture is widely used.

the encoder encodes a source sentence into a continuous vector representation, then the decoder uses the continuous vector representation to generate the corresponding target translation word by word.


To explore the phrase generation in NMT beyond the word-byword generation framework, we propose a novel architecture that integrates a phrase-based SMT model into NMT.

we add an auxiliary phrase memory to store target phrases in symbolic form.


Furthermore, in order to enhance the ability of the NMT decoder to effectively select appropriate target phrases, we modify the encoder of NMT to make it fit for exploring structural information of source sentences.

we integrate syntactic chunk information into the NMT encoder, to enrich the source-side representation.


Among them, 44.5% percent of them are fully correct 8 .

nPs have relative higher generation accuracy (i.e., 47.8% = 38.7%/81.0%) while VPs have lower accuracy (i.e., 21.2% = 1.7%/8.0%).


We argue that the incorporation of more hierarchical information into the representations may contribute to an improvement in the translation.

the use of global information can help in distinguishing the differences between word meanings.


In contrast with α, which denotes the correspondence between each source annotation and the current target hidden state, β is dominated by the target composite hidden state alone.

β is a time-dependent scalar in relation to the current target word, and therefore enables the attention model to explicitly quantify how far the leaf and no-leaf states contribute to the word prediction at each time step.


MF is also shown to result in good crosslingual representations for tasks such as alignment (Goutte et al., 2004), QA (Zhou et al., 2013), and cross-lingual word embeddings (Shi et al., 2015).

we represent translation as a matrix with source words in the columns and target words in the rows, and model the task of learning translations as a matrix completion problem.


Evaluating on CROWDTEST, we observe a similar trend over all 27 languages ( Figure 5).

we see that BPR W and BPR W+C suffer from the cold start issue where there are too few or no observed translations in the matrix to make accurate predictions.


Even with substantially restricted granularity, with high variance feedback, or with skewed rewards, this combination improves pre-trained models ( §6).

under realistic settings of our noise parameters, the algorithm's online reward and final heldout accuracies do not significantly degrade from a noise-free setting.


2017, which employs the algorithm in a supervised learning setting.

while an encoder-decoder critic model Q ω as a substitute for the true Q function in Eq 3 enables taking the full expectation (because the critic model can be queried with any stateaction pair), we are unable to obtain reasonable results with this approach.


However, in any realistic scenario, human feedback will vary from its average, and so the reward that our algorithm receives will be a perturbed variant of sentence-BLEU.

if the sentence-BLEU score is s ∈ [0, 1], the algorithm will only observe s ∼ pert(s), where pert is a perturbation distribution.


We investigate the model compression and decoding speedup for NMT from the views of network architecture, sparsification, computation and search strategy, and test the performance of their combination.

we present a four stage pipeline for model compression and decoding speedup.


We bridge the gap between S and Y by interpolating inner sequence between them.

we edit S toward Y , which can be seen as interpolation.


The updated objective function by domain weighting (J dw ) can be estimated as, 3.2.1 Batch weighting A straightforward domain weighting implementation is to modify the ratio between in-domain and out-of-domain data in each NMT mini-batch.

we can increase the in-domain weight by increasing the number of in-domain sentences included in a mini-batch.


Take the IWSLT EN-DE corpus in Table 1 as example, the original ratio R in between indomain data and all of the data is around 1:20.

for a 80-sized mini-batch, it would include around four sentence from in-domain data and 76 from out-of-domain data on average.


For batch weighting, we can set the ratio R in as 1:2 manually.

we load 40 in-domain and 40 out-of-domain sentences into each mini-batch.


Besides this, we also tried to learn the batch weighting dynamically.

the initial indomain data ration in mini-batch is set as 0%.


Further training  can be viewed as a special case of the proposed batch weighting method.

it trained NMT model by using 0% in-domain data at first and then using 100% in-domain data.


In addition, further training can work together with batch weighting.

nMT was trained with 0% in-domain data at first and then with batch weighing method for further training (Luong + bw in Table 4).


Dropout (Srivastava et al., 2014) is a stochastic regularization technique for neural networks.

we consider "Bayesian" dropout for recurrent neural networks (Gal and Ghahramani, 2016).


In this paper we propose one way how to address this limitation.

we rely on graph convolutional networks (GCNs) (Duvenaud et al., 2015;Kipf and Welling, 2017;Kearnes et al., 2016), a recent class of multilayer neural networks operating on graphs.


(2017) and in most previous work, we process individual predicates in isolation, so for each predicate, our task reduces to a sequence labeling problem.

given a predicate (e.g., disputed in Figure 3) one needs to identify and label all its arguments (e.g., label estimates as A1 and label those as 'NULL', indicating that those is not an argument of disputed).


One of the most popular and effective ways to represent sequences, such as sentences (Mikolov et al., 2010), is to use recurrent neural networks (RNN) (Elman, 1990).

their gated versions, Long Short-Term Memory (LSTM) networks (Hochreiter and Schmidhuber, 1997) and Gated Recurrent Units (GRU) (Cho et al., 2014), have proven effective in modeling long sequences (Chiu and Nichols, 2016;Sutskever et al., 2014).


Let W (e) denote the set of words in the name of en-tity e and N (e) the neighbors of entity e in the knowledge graph.

the neighbors of a column are the cells it contains, and the neighbors of a cell are the columns it belongs to.


We implement our parser by defining P ( |q, T ; θ), from which the library automatically implements both inference and training.

the beam search and the corresponding backpropagation bookkeeping to implement the objective in Section 3.4 are both automatically handled by the library.


In Equation 1, we observe that the data likelihood decouples into the log probability of observing the concept labels p θc (y i | z, x) conditioned on the statement evaluations and the log probability of the latent statement evaluations p θp (z | x, S).

the first term can be naturally parametrized by a discriminative classifier such as a loglinear model (with associated parameters θ c ).


The entropy term H q is constant from the perspective of model parameters, and is not relevant for the optimization.

the semantic parser is updated to agree with evaluations of natural language statements that are discriminative.


We observe that while our approach consistently outperforms the bag-of-words model (BoW), LNL also requires fewer examples to reach near optimal performance, before it plateaus.

the generalization performance for LNL is more robust than BoW for n < 10.


Consequently, scholars have studied parliamentary questions in detail, in terms of their origins (Chester and Bowring, 1962), their institutionalization (Eggers and Spirling, 2014) and their importance for oversight (Proksch and Slapin, 2011).

the United Kingdom's House of Commons, renowned for theatrical questions periods, has been studied in some depth.


We validate our framework by aligning these types with prior understandings of parliamentary proceedings from the political science literature (Section 6).

previous work (Bates et al., 2014) has categorized questions asked in Parliament according to the intentions of the asker (e.g., to help the answerer, or to adversarially put them on the spot); we find a clear, predictive mapping between these expert-coded categories and the induced typology.


Finally, our work contributes to a rapidly growing area of NLP applications to political domains (Monroe et al., 2008;Card et al., 2016;Gonzalez-Bailon et al., 2010;Niculae et al., 2015;Grimmer and Stewart, 2013;Grimmer et al., 2012;Iyyer et al., 2014b, inter alia).

relevant are applications to discourse in congressional and parliamentary settings (Thomas et al., 2006;Boydstun et al., 2014;Rheault et al., 2016).


Even though our clustering of questions is generated in an unsupervised fashion without any guidance from the coded rhetorical roles, we see that several of the types we discover closely align with these annotations.

helpful questions are highly associated with the agreement type (constituting 28% of questions of that type compared to 14% over the entire dataset; binomial test p < 0.01), reinforcing our interpretation that this type captures MPs cheerleading their own government.


We additionally provide a qualitative validation of our framework by comparing the questionasking activity of government and oppositionaffiliated MPs-as viewed through the extracted question types-to well-established characterizations of these affiliations in the political science literature.

prior work has examined the bifurcation in behavior between government and opposition members, in their differing focus on various issues (Louwerse, 2012), and in settings such as roll call votes (Cowley, 2002;Spirling and McLean, 2007;Eggers and Spirling, 2014).


Nodes which are lower in the graph constitute more specific representations, disambiguating between phrasings.

sink motifs serve as the most specific representation of a question, delineating the region of the latent space (and thus the question type) that best captures its phrasing.


As both the POS tag information and the subjectivity lexicon are only incorporated in the initialisation step, LAM LEX sits in-between LAM and LAM POS that it performs soft clustering of topic words and argument words.

during the initialisation, nouns are more likely to be topic words, but there is still a small probability that they could be either argument or background words; and similarly for words tagged as adjectives, adverts and verbs.


b) Will the Minister discuss that issue with the Israeli Government, urge them to reconsider the upcoming evictions and demolitions due for next month, and instead consider villages co-existing side by side in the spirit of peace?

c) a slightly separate debate or concern if I can put it that way to the illegal settlements that have been put forward, but nevertheless we are concerned and are having a dialogue with Israel about that.


c) That is a slightly separate debate or concern if I can put it that way to the illegal settlements that have been put forward, but nevertheless we are concerned and are having a dialogue with Israel about that.

c) caused partly by the security situation in Sinai and the Egyptian response to that, and partly by the situation between Israel and the Palestinians in Gaza.


Considering the text length is usually short in the dataset used here for emotion cause extraction, we set the size of the convolutional kernel to 3.

the weight of word w i in the i-th position considers both the previous word w i−1 and the following word w i+1 by a convolutional operation: For the first and the last word in a clause, we use zero padding, w 0 = w k+1 = 0, where k is the length of a clause.


We judge the model's understanding of real-world stories by inquiring if, like humans, it can develop an expectation of what will happen next in a given story.

we use it to predict the correct ending of a given short story from possible alternatives.


2.1 (ignoring the comparative binary features, and normalizing by the number of features).

for an aspect k, lets represent the feature extracted for the two options by f k 1 and f k 2 (each of length n) then the corresponding 'importance feature' for this aspect = | f k 1 − f k 2 |/n.


For instance, our language model of overall narrative sentiments indicates that while happy stories mostly have happy endings (with a conditional probability of 74%), the reverse is not true.

sad stories (with overall negative sentiments) end with a negative sentiment in only 52% of the cases.


Knowledge can be transferred from the emoji dataset to the target task in many different ways.

multitask learning with simultaneous  Figure 1: Illustration of the DeepMoji model with S being text length and C the number of classes.


We leverage this advantage in building our model.

we use a sub RNN to model the semantic content of each review.


A review is the foundation of our model, based on which we derive representations of both a user and a target product.

a user profile can be achieved by modeling all the reviews R U of the user, and a target product profile can be obtained by using all existing reviews R T of the product.


In this study, we use matrix factorization (Koren, 2008) to detect neighbors, which is a standard approach for recommendation (Ding et al., 2006;Li et al., 2009;.

users' rating scores of products are used to build a productusers matrix M ∈ R nt×nu with n t products and n u users.


From this vector we are able to synthesis both a customised review and a customised rating score.

we use the user representation v U and the neighbour representation v N to transform the target product representation h T = {h T 1 , h T 2 , ..., h Tn t } into a customised product representation v C , which is tailored to the taste of the user.


From these examples, we can find that, both rating score and review which generated by the proposed Joint model is closer to the real user.

in the first example, the baseline system correctly identifies the main both price and quality information, which the target user wrote in the review, yet the baseline model did not yield comments about the price based on reviews of other users.


We proposed a novel task called opinion recommendation, which is to generate the review and rating score that a certain user would give to an unreviewed product or service.

a The corresponding author is Yue Zhang.


The reason is that, for the parser to know to which constituents it should attach the current constituent, it needs to know which arguments have already been attached and which ones are missing.

even if the head of two different constituents is the same, their representation should be different because they have different requirements.


This can be explained by Lazy being very conservative about swaps: since their number is signifi-  cantly reduced, the transition becomes difficult to predict, and thus the model gives up on predicting swaps and concentrates on the statistics for the projective operations.

lazy predicts swaps only if the statistical evidence for swaps is high.


Here we employ a neural approach to smoothing.

we propose a lexicalized extension of neural DMV (Jiang et al., 2016) and we call the resulting approach L-NDMV.


We introduce globally normalized convolutional neural networks for joint entity classification and relation extraction.

we propose a way to utilize a linear-chain conditional random field output layer for predicting entity types and relations between entities at the same time.


In contrast to joint training and multitask learning, which calculate taskwise costs, we propose to learn a joint classification layer which is globally normalized on the outputs of both tasks.

we train the NN parameters based on the loss of a linear-chain  (Lafferty et al., 2001).


Instead of labeling each input token as in previous work, we model the joint entity and relation classification problem as a sequence of length three for the CRF layer.

we identify the types of two candidate entities (words or short phrases) given a sentence (we call this entity classification to distinguish it from the token-labeling task NER) as well as the relation between them.


To sum up, our contributions are as follows: We introduce globally normalized convolutional neural networks for a sentence classification task.

we present an architecture which allows us to model joint entity and relation classification with a single neural network and classify entities and relations at the same time, normalizing their scores globally.


Miwa and Bansal (2016) were among the first to use neural networks for end-to-end relation extraction, showing highly promising results.

they used bidirectional LSTM (Graves et al., 2013) to learn hidden word representations under a sentential context, and further leveraged treestructured LSTM (Tai et al., 2015) to encode syntactic information, given the output of a parser.


We address this potential issue by building a structural neural model for end-to-end relation extraction, following a recent line of efforts on globally optimized models for neural structured prediction (Zhou et al., 2015;Watanabe and Sumita, 2015;Andor et al., 2016;Wiseman and Rush, 2016).

we follow Miwa and Sasaki (2014), casting the task as an end-to-end tablefilling problem.


We follow Miwa and Bansal 2016  Different from Miwa and Bansal (2016), who use the output hidden vectors {h i } of LSTMs to represent words, we exploit segment representations as well.

for a segment of text [i, j], the representation is computed by using LSTM-Minus (Wang and Chang, 2016), shown by Figure 4, where h j − h i−1 in a left-to-right LSTM and h i − h j+1 in a right-to-left LSTM are used to represent the segment [i, j].


We use separate feature representations for entity detection and relation classification, both of which are extracted from the above three LSTM structures.

we first extract a set of base neural features, and then concatenate them and feed them into a non-linear neural layer for entity detection and relation classification, respectively.


Here we propose a novel method to integrate syntax, without need for prior knowledge on concrete syntactic structures.

we take state-of-the-art syntactic parsers that use encoder-decoder neural models (Buys and Blunsom, 2015;Kiperwasser and Goldberg, 2016), where the encoder represents the syntactic features of the input sentences.


We select the best iteration number according to the development results.

we exploit pre-training techniques (Wiseman and Rush, 2016) to learn better model parameters.


This model is the simple bilinear product between the unspecialized concept embeddings, parametrized only by the relation matrix W B .

the prediction score for a pair of concepts is given as s(c 1 , c 2 ) = e 1 W B e T 2 .


This is the reduction of the Dual Tensor model in which we use only one specialization tensor, i.e., W R .

sINGLE-T model always specializes the unspecialized embedding of a concept the same way, regardless of the concept's position in a candidate pair.


An instance-level inspection reveals that meronymy detection is more sensitive to the number of senses of the part candidate concept than of the whole concept.

if we partition the test set only according to the number of senses of the part concept, then the trends are similar to those observed for hypernymy.


The models that take the context into account perform similar to the baselines at the smallest recall numbers, but start to positively deviate from them at higher recall rates.

the ContextAtt model performs better than any other system in our study over the entire recall range.


The key observation is that the overall syntactic structure and compositional meanings of the multi-word context between events are important for distinguishing among fine-grained temporal relations.

our approach first extracts a sequence of context words that indicates the temporal relation between two events, which well align with the dependency path between two event mentions.


However, these features are often derived from local contexts of two events and are only capable of capturing direct evidences indicating the temporal relation.

when two events are distantly located or are separated by other events in between, feature based approaches often fail to utilize compositional evidences, which are hard to encode using discrete features.


We use neural net sequence models to capture structural and semantic compositionality in describing temporal relations between events.

we generate three sequences for each dependency path, the word sequence, the POS tag sequence and the dependency relation sequence.


The following works mostly expanded the feature sets (Cheng et al., 2007;Bethard and Martin, 2007;UzZaman et al., 2012;Bethard, 2013;Kolomiyets et al., 2012;Chambers, 2013;Laokulrat et al., 2013).

chambers (2013) used direct dependency path between event pairs to capture syntactic context.


To determine the appropriate rank automatically, we adopt the Automatic Relevance Determination (ARD) method (Babacan et al., 2012) by imposing a prior on each dimmension (column) of U and V .

we impose the Gaussian priors with variance λ r on the r-th columns of U and V , i.e., u .r and v .r : where IG is an Inverse Gamma distribution for modeling the variance λ r .


We thus use importance sampling to approximate the marginal distribution of X.

with the samples from a proposal distribution Q(R, E, L|X), the approximated marginal probability is defined as A similar idea of using importance sampling for language modeling evaluation has been used by .


For implementation of Q, we use a discriminative variant of ENTI-TYNLM by taking the current word x t for predicting the entity-related variables in the same time step.

in the generative story described in §2.2, we delete step 3 (words are not generated, but rather conditioned upon), move step 4 before step 1, and replace h t−1 with h t in the steps for predicting entity type R t , entity E t and mention length L t .


We focus on evaluating the accuracy of predicting the reference tokens, which is the goal of our model.

we report the perplexity of all words, words that can be generated from reference and non-reference words.


In this study, we introduce a new approach for learning language models by training them to estimate word-context pointwise mutual information (PMI), and then deriving the desired conditional probabilities from PMI at test time.

we show that with minor modifications to word2vec's algorithm, we get principled language models that are closely related to the well-established Noise Contrastive Estimation (NCE) based language models.


However, they were later adopted in many other NLP tasks, and the most popular recent word embedding learning models are no longer proper language models.

the skip-gram with negative sampling (NEG) embedding algorithm  as implemented in the word2vec toolkit, has become one of the most popular such models today.


Second, while the above analysis had been originally applied to the case of word-context joint distributions p(w, c), it is easy to see that the PMI matrix approximation analysis also holds for every Euclidean embedding of a joint distribution p(x, y) of any two given random variables X and Y .

we note that it holds for wordcontext joint distributions p(w, c), where w is a single word, but c represents its entire preceding context, rather than just a single context word, and c is a vector representation of this entire context.


(2014), who achieved excellent results on this dataset.

we used a 2-layer 300-hidden-units LSTM with a 50% dropout ratio to represent the preceding (left-side) context of a predicted word.


We break this independence by adding a second term to our loss function, which ensures that view-specific parameters encode only relevant information.

we want v 2 -specific parameters to only encode v 2 -specific information, and vice versa.


"not bad" which is more useful than "watch a movie" for determining reviews' polarities.

we select n-grams from training data via a novel Naive Bayes (NB) weighting technique, and then cluster the n-gram embeddings with K-means algorithm.


a path is a walk without repetition of nodes.


Our proposed approach measures the similarity between two textual documents by representing them as graphs-of-words, transforming these graphs into other graphs, and using graph kernels to calculate the similarity of the new graphs.

we capitalize on the shortest-path graph kernel  and we modify it to compare the graph representations of pairs of documents.


The normalization factor is introduced because the nominator of the proposed kernel depends on the length of the compared documents.

given the adjacency matrices of the transformed graph representations of two documents A 1 , A 2 where the value of each entry in the adjacency matrix is set equal to the label of the corresponding edge, and the diagonal matrices D 1 , D 2 with diagonal entries set to 1 if the corresponding term exists in the corresponding document, we first compute the matrices M 1 , M 2 as shown below: and we then compute the normalization factor using the following formula: where • F is the Frobenius norm for matrices.


However, due to the label enrichment that has been applied to the vertices of the transformed graphs, the number of matching nodes in the two graphs has been radically reduced and the number of pairs of edges that have to be considered is also reduced.

we have to consider n 2 pairs of edges as only paths between vertices whose label is the same in the two graphs are considered.


Generally, we note that majority answers tend to be more often correct if more workers agree.

as  tors agreed in 1,801 out of all 2,549 tasks (71%).


We thus propose a fuzzy classification model with two layers (Ishibuchi et al., 1995) of SVM classifiers (Wang et al., 2016), which introduces the context of the task using fuzzy indicators to model the interplay between the two groups of features.

we train a local-layer SVM classifier L l using the sentence-level features X l (computed from sentences).


This considerably reduces the need for expert involvement compared to expert labeling, while still maintaining relatively high annotation quality.

our approach compares favorably to other hybrid setups in which a similar partition of tasks is completed by experts.


The proposed approach far outperforms crowdsourcing and hybrid approaches in terms of annotation quality.

even at similar levels of expert involvement, it outperforms the HYBRID all5 approach.


These results clearly show that the importance of the shortcut connections, and in particular, the semantic tasks in the higher layers strongly rely on the shortcut connections.

simply stacking the LSTM layers is not sufficient to handle a variety of NLP tasks in a single model.


It is also a promising direction to train each task for multiple domains by focusing on domain adaptation (Søgaard and Goldberg, 2016).

incorporating language modeling tasks provides an opportunity to use large text data.


In our first and simplest model, we propose a bag-of-words encoder (with position embeddings, see §2.1.1), with a GCN on top.

inputs h (0) are a sum of embeddings of a word and its position in a sentence.


A typical scenario of neural machine translation starts with training a model to maximize its log-likelihood.

we often train a model to maximize the conditional probability of a reference translation given a source sentence over a large parallel corpus.


We train a neural machine translation model, or equivalently estimate the parameters θ g , θ f and θ e , by maximizing the log-probability of a reference translation Y = {ŷ 1 , ...,ŷ T } given a source sentence.

we maximize the log-likelihood function: given a training set consisting of N source-target sentence pairs.


The main idea behind NPAD algorithm is that a better translation with a higher log-probability may be found by injecting unstructured noise in the transition function of a recurrent network.


Instead of using the point estimate ∂R c ∂φ of the gradient of the predicted objective with respect to the actor's parameters φ, we propose to use the expected gradient of the predicted objective with respect to the critic-aware distribution Q.

where we define the critic-aware distribution Q as .


Character embeddings are randomly initialized with 30 dimensions.

for the proposed model, the following hyper-parameters are estimated based on the validation set and used  in the final test set.


This policy requires Wikipedia contributors to support their additions with citations from authoritative external sources (web, news, journal etc.).

it states that "articles should be based on reliable, third-party, published sources with a reputation for fact-checking and accuracy" 1 .


The task of determining the citation span for a citation c and a paragraph p, respectively ⟨p, c⟩ (or simply p c ), is subject to the citing paragraph and the citation content.

we refer with citation span to the textual fragments from p which are covered by c. The fragments correspond to the sequence of subsentences S(p) = ⟨δ 1 1 , δ 2 1 , .


Furthermore, the features f #s and f s i (the number of sentences in p together with the feature considering if δ is in the same sentence as c) are strong indicators for accurate prediction of the label of δ.

it is more likely for a fragment δ to be covered by the citation if it appears in the same sentence or sentences nearby to the citation marker.


If there exists a temporal date expression in δ i and δ i−1 and they point to different time-points, this presents an indicator on the transitioning between the states y i and y i−1 .

there is a higher likelihood of changing the state in the sequence S for the labels y i and y i−1 .


We define the citation span as the ratio of subsentences which are covered by a given citation over the total number of sub-sentences in the sentence, consequentially in the citing paragraph.

a citation is considered to have a span of one sentence if it covers all its sub-sentences.


The best performing baseline CS has an erroneous span of ∆ w = 35% and ∆ δ = 27%, in terms of extra words and sub-sentences, respectively.

nearly half of the determined span is erroneous, or in other words it is not covered in the provided citation.


This provides an amazing corpus for studying the types and effectiveness of revisions.

differences between revisions contain valuable information for modeling document quality or extracting users' expertise, and can additionally support various natural language processing (NLP) tasks such as sentence compression (Ya-mangil and Nelken, 2008), lexical simplification (Yatskar et al., 2010), information retrieval (Aji et al., 2010), textual entailment recognition (Zanzotto and Pennacchiotti, 2010), language bias detection (Recasens et al., 2013), spelling errors and paraphrases (Zesch, 2012;Max and Wisniewski, 2010).


To explore the utility of this taxonomy, we applied this model to better understand two important issues for Wikipedia: new editor retention and article quality.

we examined whether edit intentions in newcomers' first editing sessions predict their retention, and examined how edits with different intentions lead to changes in article quality.


This categorization leads to 13 distinct semantic intentions, and Table 1 provides detailed descriptions.

general revision intentions include: Clarification, Copy Editing, Elaboration, Fact Update, Point of View, Refactoring, Simplification and verification, and can be applicable to other contexts.


We used the majority voting to resolve the disagreement.

if at least 3 out of 4 annotators picked an intention for a revision, it will be selected as the ground-truth.


They are similar to textual features defined in Daxenberger and Gurevych (2013), but we considered a wider range of objects being modified.

we computed the difference in the number of characters, uppercase words, numeric chars, white-spaces, markups, Chinese/Japanese/Korean characters, HTML entity characters, URLs, punctuations, break characters, etc.


We collected 1,623,446 revisions made to these articles between January and June 2015, by randomly sampling 10% revisions that were made to these articles during that time periods.

the outcome article quality change is calculated by subtracting the previous quality score from the end quality score.


However, high-precision paraphrase generation is challenging, as most edits to a sentence do actually change its meaning.

instead of relying on paraphrasing, we use perturbations that do alter semantics to build concatenative adversaries, which generate examples of the form (p + s, q, a) for some sentence s. concatenative adversaries add a new sentence to the end of the paragraph, and leave the question and answer unchanged.


In this paper, we propose a multi-knowledge reasoning method, which can exploit heterogeneous knowledge for commonsense machine comprehension.

we first mine different kinds of knowledge (including event narrative knowledge, entity semantic knowledge and sentiment coherent knowledge) and encode them as inference rules with costs.


In general, any world knowledge whatsoever in the reader's mind can affect the choice of an interpretation (Dahlgren et al., 1989).

a person can learn any heterogeneous commonsense knowledge and make inference of given information based on all knowledge in his mind.


To address the above problems, this paper proposes a new commonsense reasoning approach, which can mine and exploit heterogeneous knowledge for commonsense machine comprehension.

we first mine different kinds of knowledge from raw text and relevant knowl-edge base, including event narrative knowledge, entity semantic knowledge and sentiment coherent knowledge.


In this section, we describe how to mine different knowledge from different sources.

we mine three types of commonly used commonsense knowledge, including: 1)Event narrative knowledge, which captures temporal and causal relations between events; 2)Entity semantic knowledge, which captures semantic relations between entities; 3)Sentiment coherent knowledge, which captures sentimental coherence between elements.


For example, if a premise document contains "Starbucks", then "coffeehouse" and "latte" will be reasonable entities in hypothesis since "Starbucks" is a possible coreference of "coffeehouse" and it is semantically related to "latte".

we identify mainly two kinds of semantic relations between entities for commonsense machine comprehension: 1) Coreference relation, which indicates that two elements refer to the same entity in environment.


This paper mines associative relations between entities from Wikipedia 1 , using the method proposed by Milne and Witten(2008).

given two entities e 1 and e 2 , we compute the semantic distance dist(e 1 , e 2 ) between them as: where E 1 and E 2 are the sets of all entities that link to these two entities in Wikipedia respectively, and W is the entire Wikipedia.


This paper proposes a commonsense machine comprehension method, which performs effective commonsense reasoning by taking heterogenous knowledge into consideration.

we mine commonsense knowledge from heterogeneous knowledge sources and simultaneously exploit them by proposing a highly extensible multiknowledge reasoning framework.


In this case, we can train a machine comprehension model to automatically attend corresponding text snippets in the review document to predict the aspect rating.

we introduce a hierarchical and iterative attention model to construct aspect-specific representations.


Thus, our model builds aspectspecific representations in a bottom-up manner.

we obtain sentence representations s k 1 , s k 2 , .


Moreover, the bottom figures in (a) and (b) show that (1) word weights of different hops are various; (2) attention values in higher hop are more reasonable.

in the first sentence, the weight of word clean is higher than the word comfortable in first hop, while comfortable surpasses clean in higher hops.


or posts without any clear argumentative purpose ("what i did with it was make this recipe for a sort of casserole/stratta (i made this up, here is the recipe) [...] butter, 4 eggs, salt, pepper, sauted onions and cabbage..add as much as you want bake for 1 hour at 350 it was seriously delicious!").

the Wikipedia talk Page corpus (WtP) contains claims typical to Wikipedia quality discussions ("why this article has NPOV issues.")


The best feature-based approach outperforms the best deep learning approach in most scenarios.

as opposed to the in-domain experiments, the difference of the Claim-F 1 measure between the feature-based approaches and the deep learning approaches is striking.


The coefficient for training data size β is statistically significantly different from zero in three out of 15 cases.

it is significantly positive in two (CNN:rand, CNN:w2vec) out of four cases for the neural networks.


By examining examples in which a model trained on OC and VG made correct predictions as opposed to a model trained on PE, we quickly noticed that lexical indicators indeed played a crucial role.

the occurrence of the word "should" (and to a lower degree: "would", "article", "one") are helpful for the detection of claims across various datasets.


Our results reveal that the best in-domain system is not necessarily the best system in environments where the target domain is unknown.

we found that mixing source domains and training on two rather noisy datasets (OC and VG) gave the best results in the cross-domain setup.


The little previous work that exists in this area currently circumvents this critical step in passagelevel question generation by assuming that such sentences have already been identified.

prior work focuses almost exclusively on sentence-level question generation: given a text passage, assume that all sentences contain a question-worthy concept and generate one or more questions for each (Heilman and Smith, 2010;Du et al., 2017;Zhou et al., 2017).


While the goal of ALA is to generate annotations of all types, it is evident from our analysis that CS annotations can not be generated by models trained solely on parallel text.

these annotations cannot be generated without background knowledge or added context.


The information distillation happens in the recaller component where only the most important information is preserved.

the recaller outputs fewer vectors s than that of the input Figure 1: Our cascaded attention based unsupervised information distillation framework.


Such distributed representation can provide better generalization than BOW representation in many different tasks (Le and Mikolov, 2014;Mikolov et al., 2013).

the input hidden layer will project the input sentence vector x j to a new space R h according to Equation 1.


The hidden attention vector a h t,i is integrated with the output attention by a weight λ a ∈ [0, 1]: The output context vector is computed as: To update the output vector s t in Equation 5, we develop a different method from that of the hidden attentions.

we use a weighted combination of the context vectors and the original outputs with λ c ∈ [0, 1].


The similarity is calculated by the Jaccard Index based method.

this objective maximizes the salience score of the selected phrases as indicated by the first term, and penalizes the selection of similar phrase pairs.


Our work is aimed at single document summarization using small amounts of reference summaries.

we address document summarization in the framework of multitask learning using curriculum learning for sentence extraction and document classification.


However, preparing such large volumes of reference summaries manually is sometimes costly.

it is infeasible for humans to create hundreds of thousands of reference summaries in cases where summarization requires domain-specific or expert knowledge.


As described earlier, we focus on single document summarization with small amounts of reference summaries, and propose a general framework for summarization that is useful for extracting sentences from a document along with its external related information.

we formalize estimation of the above-described document subjects as a document classification task and solve document summarization in the framework of multitask learning for sentence extraction and document classification.


Notice that in conventional incremental parsing, the stack stores the subtrees j .

quite shockingly, no tree structure is represented anywhere in the parser.


Furthermore, our model uses two distinct pairwise classifiers that are separately trained with features intrinsic to each type.

the WD classifier uses features based on event mentions and their arguments while the CD classifier relies on features characterizing surrounding contexts of event mentions as well.


Intuitively, if two event mentions are related to the same set of events, it is likely that the two event mentions refer to the same real world event, even when their word forms and local contexts are distinct.

we merge event clusters if their event mentions are tightly associated (i.e., having the same dependency relations) or loosely associated (i.e., co-occurring in the same sentential context) with enough (i.e., passing a threshold) other events that are known coreferent.


Next, two additional merges were made by leveraging second-order inter-dependencies.

both the event mentions released in doc 1 and presented in doc 2 are in the same dependency relation ("nmod") with a mention of the trial event cluster, therefore, a new merge was made between clusters containing the two mentions.


We train pairwise classifiers for identifying event clusters to merge.

for WD merges as indicated in lines 8-10 in Algorithm 1, we iteratively go through pairs of clusters that contain a pair of within-document event mentions, one mention from each cluster.


Subroutine ContextSimilarity (E 1 , E 2 , Θ CD ) generates a context vector (CV) for each event cluster and check whether cosine similarity between context vectors of two clusters E 1 and E 2 (cos( CV 1 , CV 2 )) is above 0.7.

we define context clusters for an event mention as the different event clusters that have event mentions co-occurring in the same sentence.


WD Classifier: the neural network based WD classifier essentially inherits the features that have been shown effective in previous event coreference studies (Ahn, 2006;, including both features for event words and features for their arguments.

the classifier includes a common neural layer shared by two event mentions to embed event lemma and partsof-speech features.


The LSTM layer is shared by both event mentions in order to calculate context word embeddings for both event mentions.

three words to each side of an event word together with the event word itself are used to calculate the context embedding for each event mention.


However, there are significantly more CD coreferent event mention pairs annotated in the ECB+ corpus, therefore, we randomly sampled 70% of all the CD coreferent event mention pairs as positive instances and randomly sampled from non-coreferent CD event mention pairs as negative instances.

number of negative instances are kept 5 times of positive instances.


As an extension, the attention mechanism enables the decoder to revisit the input sequence's hidden states and dynamically collects information needed for each decoding step.

our conversation model is established based on a combination of the models of  and (Luong et al., 2015) that we found to be effective.


We also experimented incorporating an information retrieval (IR) module to automatically collect topic hints for CG-based decoder.

a full-text index of twitter corpus is built using solr 6 , and the top 10 searched results based on the source sentence are be used to generate posterior CG distributions as hints.


The neural models are particularly successful, posting a higher accuracy on Switchboard than the HMM.

kalchbrenner and Blunsom (2013) model a DA sequence with a recurrent neural network (RNN) where sentence representations are constructed by means of a convolutional neural network (CNN); Ji et al.


In contrast, we propose to condition the next label on a predicted distribution of the current label.

we introduce two variants of this idea: the Uncertainty Propagation model and the Average Embedding model.


In this model, the distribution over the labels at the current time step is passed to the next time step.

the quantity of interest is the posterior probability of the DA of the next time step given all the utterances observed so far.


In addition to decoding, we use posterior probability when training the model.

our training objective is where D is the set of conversations in the training set, each consisting of a sequence of utterances x x x 1:T annotated with its gold sequence of DAs z z z 1:T .


This model offers a new perspective where a neural net combines an inference machine and a model (rather than simply encoding a model).

this model represents in its architecture, through a weighted sum of embeddings, the inference procedure encoded in Equation 4 for the Uncertainty Propagation model: where q(z t ) is an embedding that represents the uncertainty at time step t. q(z t ) is computed sequentially as new utterances are received, and used in both decoding and training.


We conjecture that this is because the adversarial training strategy is more beneficial to tasks in which there is a big discrepancy between the distributions of the generated sequences and the reference target sequences.

the adversarial approach is more beneficial on tasks in which entropy of the targets is high.


Different features have been explored in these models, including lexical, syntactic features, prosodic cues, and speaker interactions.

context information has been previously used in some methods.


Since the output of the model is assignments to discovered states for utterances, not pre-determined DA labels, we use a clustering evaluation method, as adopted by previous work on unsupervised DA modeling.

we use homogeneity, completeness, and v-measure as metrics (Rosenberg and Hirschberg, 2007).


Unlike the baseline models, which show a large variance in performance across corpora, our model is robust for both corpora used in the evaluation due to the model components complementing one another.

content word filtering is found to be effective when each conversation has a consistent conversational topic, and the separation between sentence-level and utterance-level DAs is beneficial for long utterances.


We propose an implicit content-introducing method which incorporates additional information into the Se-q2Seq model in a flexible way.

we fuse the general decoding and the auxiliary cue word information through our proposed hierarchical gated fusion unit.


To establish a conversation system with adequate artificial intelligence is a long-cherished goal for researchers and practitioners.

automatic conversation systems in open domains are attracting increasing attention due to its wide applications, such as virtual assistants and chatbots.


We propose a complete framework of companion teaching, and develop various teaching schemes which combine different teaching strategies and teaching heuristics together, to answer the questions of "how" and "when" to teach to achieve affordable dialogue policy learning (section 2).

a novel failure prognosis based teaching heuristic is proposed, where Mul-tiTask Learning (MTL) is utilized to predict the dialogue success reward (section 3).


Our Risk Index integrate these two contents of riskiness.

a nonnegative scalar which measures the integrated riskiness for the online training process of total length T .


To evaluate the policy learning process precisely, we proposed two measurements, Risk Index (RI) and Hitting Time (HT), to quantify the degree of safety and efficiency.

through multitask learning, we managed to optimize the predicted remaining turns and dialogue success reward explicitly, based on which we developed a novel Failure Prognosis based Teaching (FPT) heuristic to better utilize the fixed teaching budget and make the teaching affordable.


Despite promising results, there are still many challenges with this approach.

these models produce short, generic responses that lack diversity (Sordoni et al., 2015;Li et al., 2015).


For the reranking, we normalize the scores using random prompts.

suppose y k = y 1 , ..., y k−1 is a candidate segment, and (x, y 1:k−1 ) is the input to the stochastic beam search.


Conceptually, our approach exploits the structural information of composite tasks for efficient exploration.

in order to mitigate the reward sparsity issue, we equip our agent with an evaluation module (internal critic) that gives intrinsic reward signals, indicating how likely a particular subtask is completed based on its current state generated by the global state tracker.


Both the top-level and low-level policies can be learned with deep Q-learning methods, like DQN.

the top-level dialogue policy estimates the optimal Q-function that satisfies the following: where N is the number of steps that the low-level dialogue policy (intra-option policy) needs to accomplish the subtask.


We made a few changes to the schema of the data set for the composite taskcompletion dialogue setting.

we added inter-subtask constraints as well as user preferences (soft constraints).


This, to a large degree, was attributed to the use of the hierarchical structure of the proposed agent.

the top-level dialogue policy selected a subtask for the agent to focus on, one at a time, thus dividing a complex task into a sequence of simpler subtasks.


(Novikova et al., 2016), and as such, enable rapid development of NLG components in new domains.

we compare the performance of the following systems: The system by Wen et al.


We now evaluate a more coarse measure, namely the metrics' ability to predict relative human ratings.

we compute the score of each metric for two system output sentences corresponding to the same MR.


Utterances produced from inform MRs are significantly longer and have a significantly higher correlation with human ratings of informativeness and naturalness than non-inform utterance types.

bAGEL is the most complex dataset to gen-erate from.


In this work, we suggest a slightly more difficult data-to-text generation task, and investigate how effective current approaches are on this task.

we introduce a new, large-scale corpus of data records paired with descriptive documents, propose a series of extractive evaluation methods for analyzing performance, and obtain baseline results using current neural generation methods.


In this context, we believe the problem of generating multi-sentence summaries of tables or database records to be a reasonable next-problem for neural techniques to tackle as they begin to consider more difficult NLG tasks.

we would like this generation task to have the following two properties: (1) it is relatively easy to obtain fairly clean summaries and their corresponding databases for dataset construction, and (2) the summaries should be primarily focused on conveying the information in the database.


While BLEU is perhaps a reasonably effective way of evaluating short-form text generation, we found it to be unsatisfactory for document generation.

we note that it primarily rewards fluent text generation, rather than generations that capture the most important information in the database, or that report the information in a particularly coherent way.


To address this evaluation challenge, we begin with the intuition that assessing document quality is easier than document generation.

it is much easier to automatically extract information from documents than to generate documents that accurately convey desired information.


ROTOWIRE data it achieves over 90% accuracy on held-out data, and recalls approximately 60% of the relations licensed by the records.

with a sufficiently precise relation extraction system, we can begin to evaluate how well an automatic generationŷ 1:T has captured the information in a set of records s. since the predictions of a precise information extraction system serve to align entity-mention pairs in the text with database records, this alignment can be used both to evaluate a generation's content selection ("what the generation says"), as well as content placement ("how the generation says it").


we model the probability of relations r ∈ s such that r.m = y t and r.e is in the same sentence as r.m.


We train the generation models using SGD and truncated BPTT (Elman, 1990;Mikolov et al., 2010), as in language modeling.

we split each y 1:T into contiguous blocks of length 100, and backprop both the gradients with respect to the current block as well as with respect to the encoder parameters for each block.


While it is difficult in general to ascertain whether a foreign word or phrase used in an utterance is borrowed or just an instance of code-mixing , one tell tale sign is that only proficient multilinguals can code-mix, while even monolingual speakers can use borrowed words because, by definition, these are part of the vocabulary of a language.

just because an English speaker understands and uses the word "tortilla" does not imply that she can speak or understand Spanish.


: Re-annotation results H all (H most ) category.

on average, in as high as 88% of cases the annotators altered the tags of the words that are highly likely to be borrowed (i.e., T OP ) in a largely Hindi context (i.e., H all or H most ).


To model norm strength within a given demographic group or across groups, we tabulate how often respondents from a group match the most frequent answer (Primary) or one of the most frequent ten answers for that group (Top10).

given the response for one stimulus word as provided by one held-out survey respondent at a time, we determine whether that response matches the most frequent association of the remaining members of the same group (Table 3, Primary columns), or one of the top 10 associations pertaining to that same group (Table 3, Top10 columns).


In the second variation (EM B2), we also include the demographic label in the context.

for each skip-gram (c i,left , w i , c i,right ) we generate three skip-grams The two models seek to capture different scenarios.


Many of these studies have analyzed content-agnostic factors such as submission timing and author social status, as well as language factors that underlie the textual content, e.g., the topic and idiosyncrasies of the community.

there is an increasing amount of work on online discussion forums such as Reddit that exploits the conversational and community-centric nature of the usergenerated content (Lakkaraju et al., 2013;Althoff et al., 2014;Jaech et al., 2015;Tan et al., 2016;Wei et al., 2016;He et al., 2016a;, which contrasts with Twitter where the au-thor's social status seems to play a larger role in popularity.


The model aims to characterize the interdependence of comment on its global context and subsequent responses that is characteristic of multi-party discussions.

we propose a factored neural model with separate mechanisms for representing global context, comment content and response generation.


2 Furthermore, as will be seen in Section 4.1, connotation frames offer new insights that complement and deviate from the well-known Bechdel test (Bechdel, 1986).

we find that high-agency women through the lens of connotation frames are rare in modern films.


From Table 2, it seems male characters display more power and authority through their speech than their female counterparts do.

women are written to use more hedges (# Hedges) whereas men are written to use more imperative sentences (# Imperative Sent.


While this is a low bar, a surprising number of movies fail at least one of the three criteria.

as many as 42% of the movies in our dataset fail the test according to an online database of the Bechdel scores.


We validate our new frames through a case study on movie scripts.

we analyze differences in power and agency for male and female characters, and compare these dimensions to the Bechdel test.


Next, the sentences were tokenized and the tokens associated with some Part-of-Speech (POS) tags were kept.

nouns, adjectives and foreign words were kept while verbs and personal nouns were removed.


To handle covariance and avoid over-fitting with so many features, we used a series of feature selection and dimensionality reduction techniques fed into a ridge regression to create the models.

we filtered the features (which were already restricted to those present in at least 50% of the forecasting tasks) to those with a small linear correlation with the outcome, defined as having a family-wise error rate < 60 (Toothaker, 1993).


To identify distinguishing features, we use a series of multivariate linear regressions to find the relationship between each individual linguistic feature and the given quality metric, controlling for comment length -a process known as differential language analysis (Schwartz et al., 2013).

the individual linguistic feature along with the comment length (logged) are standardized and used as independent variables, and then fit to the standardized form of the given quality metric.


The coefficient given is then taken as the standardized effect size of the relationship between that feature and the quality metric, holding length constant (Fox, 1997).

it tells us how much the feature can explain the quality score, beyond what is explainable simply from length.


The burst (i.e., the rapid increase of the occurrence) of a phrase causes the burst of overlapping Ngrams including incomplete ones.

bursty incomplete N-grams inevitably overlap bursty phrases.


Based on the phenomenon described in the previous paragraph, bursty incomplete N-grams cease bursting if their original bursty phrases disappear from microblog texts.

all bursty N-grams including incomplete ones can be covered (overlapped) by bursty phrases.


Occurrence-based set covering can solve the case when a bursty N-gram is covered by multiple bursty phrases.

the bursty N-gram is not completely covered even if one of the bursty phrases is selected.


Briefly, containment relations hold when only the containment N-grams among longer Ngrams can cover the occurrence of bursty N-grams more than the shorter N-gram owing to the containment relations.

we find a stable state of containment relations.


Context variety can also be calculated using the set cover problem.

a left or right character (or word) that most covers the occurrence of the N-gram is iteratively selected until all the occurrences are covered.


Different from previous systems which treat entity linking as a static component, our search space consists of the search space of both entity linking and semantic parsing.

the search space is the union of the search space of entity linking described in Section 3.2 and the search space of the semantic parses, which we describe below.


We valid our learning framework when the explicit supervision signals are presented, as well as demonstrate the support of the scenario where supervision signals are mixed.

in this section, we report the experimental results of MMRN on named entity recognition and entity linking, both using explicit supervision, and on semantic parsing, using implicit supervision.


In our companion paper, (Iyyer et al., 2017), we used a simple form of approximated reward to get feedback as early as possible during search.

the semantic parse will be executed as soon as the parse is executable (even if the parse is still not completed) during search.


On this basis, we design an experiment to evaluate the impact of the different types of argumentative structure (Section 6).

we decompose our approach into four complementary modeling steps, both for a general model of overall argumentation and for the specific models of the given corpora.


Altogether, the visualizations give first support for the impact of modeling overall argumentation.

we hypothesize that hierarchical overall structure is decisive for myside bias, whereas a combination of sequential and hierarchical structure helps to distinguish pro-stance from con-stance texts.


(2009) below, we follow their representation of sequential structure with one exception.

the authors assigned an index to each edge that numbers the child nodes of each node ascending from 1.


All the words in the sentence are sequentially generated using the RNN, until the end-of-the-sentence symbol is generated.

each conditional p(w t y |w <t y , z), where < t = {0, .


A paragraph is modeled as a sequence of sentences, and each sentence is modeled as a sequence of words.

assume we are given a paragraph D = (s 1 , .


Our schedulers lengthen or shorten inter-repetition intervals with respect to individual training instances.

we propose using density kernel functions to estimate the latest epoch in which a given training instance can be recalled.


We estimate the optimum value of τ (line 5 of Algorithm 2) for RbF memory models using validation data.

rbF uses the loss values of validation instances and strength of the network obtained at the previous epoch to estimate network retention for validation instances at the current epoch (therefore t i = 1 for every validation instance).


delay substantial amount of instances per epoch.

cos and Qua outperform Lit in both predicting network retention and delaying items, delaying around 50% of training instances per epoch.


After that we add a bias and apply nonlinearity to obtain a feature map m i ∈ R l−k+1 .

the j-th element of m i is given by: where C k [ * , j : j + k − 1] is the j-to-(j + k − 1)-th column of C i and A, B = Tr(AB T ) is the Frobenius inner product.


Training examples contain an input goal g, specifying the available items and their values, a dialogue x, and an output decision o specifying which items each agent will receive.

we represent g as a list of six integers corresponding to the count and value of each of the three item types.


Even though failing to agree was worth 0 points, people often preferred this course over capitulating to an uncompromising opponent-a factor not well captured by the simulated partner in reinforcement learning training or rollouts (as reflected by the larger gains from goal-based models in dialogues with the LIKELIHOOD model).

the goal-based models are prone to simply rephrasing the same demand each turn, which is a more effective strategy against the LIKELIHOOD model than humans.


We multiply the performance gap between ideal performance and current performance with the episode index, thus giving later failure greater penalty.

more illustrations about safety loss and efficiency loss are given in Appendix D. Our experiments have three objectives: (1) Comparing our proposed dropout DQN in Algorithm 1 with some baselines when there is no teacher.


In order to overcome these difficulties, we propose to regularize our network based on implicit audience feedback that occurs during the debate, and is provided as metadata with the debate transcript.

provided along side each debate turn, there is a 'non-text' field that indicates if any sounds occurred during the turn such as applause or laughter from the audience.


100 Chinese to English MT output translations are assessed by 25 human judges on a five-point scale, in the form of their response (None, Little, Much, Most, or All) to the following question: how much of the meaning of the human translation is also expressed in the machine translation?.

the same 100 translations were assessed by all 25 judges.


The different system outputs for each source sentence were grouped together to reduce the burden on the annotators.

in figure 1, anno-tators were asked to answer the question for each of four outputs, rather than just one as shown.


Since our approach relies on the quality of monolingual word embeddings, we also propose to enhance vector representations of both the source and target language with linguistic information.

we augment word vectors with additional dimensions capturing corpus statistics of part-of-speech tags of words.


We go for the middle-ground, in which a word is represented as a combination of a vector for the word itself with vectors of sub-word units that comprise it.

for a given a word we learn a vector representation for the word itself, and also for each n-gram of >= 3 and < 6 in the word (Bojanowski et al., 2017).


Using a character-based cipher model provides the flexibility to generate unseen words.

the vocabulary is limited by the decoding LM, not the cipher model.


LM Language Vector Several methods have been proposed to learn multilingual language models (LMs) that utilize vector representations of languages (Tsvetkov et al., 2016;Östling and Tiedemann, 2017).

these models train a recurrent neural network LM (RNNLM; Mikolov et al.


Through this study, we have shown that neural models can learn a range of linguistic concepts, and may be used to impute missing features in typological databases.

we have demonstrated the utility of learning representations with parallel text, and results hinted at the importance of modeling the dynamics of the representation as models process sentences.


(Q3) Based on the stipulated cross-linguistic validity of VerbNet-style classification, can one exploit rich sets of readily available annotations in one language (e.g., the full English VerbNet) to automatically bootstrap the creation of VerbNets for other languages?

is it possible to exploit a cross-lingual vector space to transfer VerbNet knowledge from a resource-rich to a resource-lean language?


In light of this, we also might consider asking the system to extract some product reference from the post, rather than all of them.

we compute accuracy on a post-level by checking whether the first product type extracted by the system is contained in the annotated set of product types.


Given the data that we've already annotated, we can simulate this process by iterating through  our labeled data and collecting annotated product names that are sufficiently common.

we take all (lowercased, stemmed) product tokens and keep those occurring at least 4 times in the training dataset (recall that these datasets are ≈ 700 posts).


If we assume the domain transfer problem is more complex, we really want to leverage labeled data in the target domain rather than attempting to transfer features based only on type-level information.

we are interested in cases where a relatively small number of labeled posts (less than 100) might provide substantial benefit to the adaptation; a researcher could plausibly do this annotation in a few hours.


Different hyperedges connecting these nodes are used to represent how the semantics of a node is composed from those of its child nodes.

each A k is connected to A k+1 and E k through the hyperedge A k → (A k+1 , E k ), denoting the fact that the set of mentions that start at k or later is the union of the set of mentions that start at k + 1 or later and the set of mentions that start at k. Each E k is connected to T k 1 , T k 2 , .


Both models involve directed structures, on top of which the inference algorithm first calculates the inside score for each node from the leaf node to root, and then the outside score from the root to the leaf node, in very much the same way as how inference is done in a classic graphical model.

for our EDGE-based model, the inside scores are calculated using a bottom-up (right-to-left) dynamic programming procedure, where we calculate the inside score at each node by summing up the scores associated with each path connecting the current node to one of its child nodes.


Roughly speak- ACE-2004ACE-2005ACE-2004ACE-2005  ing, the weight of this feature controls how confident the model should be in predicting more mentions.

this is a way to balance the precision and recall of the model.


From such empirical results we can see that our proposed model using mention separators consistently yields significantly better results (p < 0.01) than the mention hypergraph model across these two datasets, under two setups (whether to optimize F 1 -score or not).

when the state-based approach is used (STATE), our approach is able to obtain a much higher recall, resulting in improved F 1 -score.


On different types of sentences: As these datasets consist of both overlapping and nonoverlapping mentions, to further understand the model's effectiveness in recognizing overlapping mentions (and non-overlapping mentions), we performed some additional experiments on the mention hypergraph model and our model.

12 we split the test data into two portions, one that consists of only sentences that contain overlapping mentions (O) and those which do not (Ø).


Instead, MinIE supports multiple minimization modes, which differ in their aggressiveness and effectively control the usefulness-precision trade-off.

minIE's complete mode (C) does not perform any minimizations.


Our goal is to retain only one of the constituents of the input clause in the argument of the extraction whenever possible, while simultaneously retaining coherence.

our approach uses the clause types detected by ClausIE to ensure that MinIE never removes obligatory constituents from a clause (which would lead to incoherent extractions); it instead may opt to move such constituents to the relation.


MinIE makes use of additional implicit extractors.

we use the patterns of FINET  to detect explicit type mentions.


We consider it safe, however, when applied to annotated extractions.

all determiners, pronouns, and adverbs indicating negation, modality, or quantities are already processed and captured in annotations.


The decision of what is considered essential and what overly specific is informed by dictionary D. Note that in order to minimize mistakes, we consider for dropping only words in instances of pattern P .

we do not touch subconstituents that contain prepositions because these are notoriously difficult to handle (e.g., we do not want to minimize Bill of Rights to Bill).


We develop a semi-supervised algorithm that extends self-training by estimating the labels of unlabeled data and then using those labels for retraining.

we use a graph-based algorithm to estimate the posterior probabilities of unlabeled data and develop a new CRF training to take the uncertainty of the estimated labels into account while optimizing the objective function.


A similar idea has been previously used in treating partially labeled data (Kim et al., 2015).

given a sentence x we define a constrained lattice Y(x), where at each position t the allowed label types Y(x t ) are: where η is the confidence threshold, y t is the prediction of posterior decoding and p(y t |x; θ) is its CRF token marginal.


Table 4 reports the results of the semi-supervised learning algorithms in different settings.

we ablate incorporating the graph-based methods of computing the posterior and CRF training (ULM vs. hard decision).


From Figure 3, we find that RLIE-A3C is able to achieve upto 6x training speedup over RLIE-DQN.

rLIE-A3C is able to achieve significant speedup in training time over rLIE-DQN, without any loss in average accuracy.


While these models are expressive and accurate, they fail to fully exploit the parallelism opportunities of a GPU, and thus their speed is limited.

they employ either recurrent neural networks (RNNs) for feature extraction, or Viterbi inference in a structured output model, both of which require sequential computation across the length of the input.


This is because a token encoded by a bidirectional RNN will incorporate evidence from the entire input sequence, but the CNN's representation is limited by the effective input width 1 of the network: the size of the input context which is observed, directly or indirectly, by the representation of a token at a given layer in the network.

in a network composed of a series of stacked convolutional layers of convolution width w, the number r of context tokens incorporated into a token's representation at a given layer l, is given by r = l(w − 1) + 1.


Our proposed model uses compositional training to ensure that the learned entity representation captures the various information sources available to it, making it quite modular.

we introduce encoders for the different sources of information about the entity, and encourage the entity embedding to be similar to all of the encoded representations.


By the virtue of our model's modular nature, it can easily incorporate new entities not present during training.

we show that our model can perform accurate linking for new entities, without having to re-train the existing entity representations, only using their description and types.


Here, we describe our model that encodes information about the set of entities E using dense unified representation for linking (v e ∈ R d , ∀e ∈ E).

we use existing mentions in Wikipedia to encode the context ( § 3.1), textual descriptions from Wikipedia to encode background information ( § 3.2), and fine-grained types from Freebase as structured topical knowledge ( § 3.3).


In order to learn the entity representation v e such that it encodes all of its mentions' contexts, we introduce an objective that encourages the context representation v m to be similar to v e (where mention m is a link to entity e), and dissimilar to other candidates 4 .

we maximize the probability of predicting the correct entity from the mentioncontext vector as P text (e|m) = where C m is the set of candidate entities.


In this section we present various experiments to evaluate the performance of our proposed entitylinking system.

we focus on the following questions: (1) how effective is our model in combining different information on standard linking benchmarks, without requiring domain specific information ( § 6.1), (2) is our model able to accommodate unseen entities by using their types, or description, without re-training the entity representations ( § 6.2), and (3) how does the model perform on fine-grained mention typing, a task it is not directly trained for, compared to approaches designed for the task ( § 6.3).


In this section, we consider the use of our approach to this setting.

for each such new entity, we need to determine their embedding using only their description and/or type information.


Compared to existing systems trained specifically for this task, embeddings from our approach (Model CDTE) performs competitively (see Table 4).

our model performs better than the neural-LSTM model of Shimaoka et al.


The attention weights should better guide the focus of the model onto important context words of the sentence.

context words that are closer to entity representation mix should have better chances to be clue words.


We present a new state-of-the-art detector that achieves a maximum performance of 0.75 F 1 (a 14% improvement), with significantly greater generalizability than previous work.

our detector achieves performance above 0.70 F 1 across a variety of combinations of lexically different corpora for training and testing, as well as dramatic improvements (up to 4,000%) in performance when trained on a small, disfluent data set.


Recent advances in neural networks enable us to jointly model charge prediction and relevant article extraction in a unified framework, where the latent correspondence from the fact description about a case to its related law articles and further to its charges can be explicitly addressed by a two-stack attention mechanism.

we use a sentence-level and a documentlevel Bi-directional Gated Recurrent Units (Bi-GRU) (Bahdanau et al., 2015) with a stack of factside attention components to model the correlations among words and sentences, in order to capture the whole story as well as important details of the case.


(3) These articles are embedded by another document encoder, and passed to an article aggregator to attentively select supportive articles, and produce the aggregated article embedding d a .

three context vectors, i.e., u aw , u as and u ad , are dynamically generated from d f , to produce attention values within the article document encoder and the article aggregator.


A promising solution is Bi-directional Gated Recurrent Units (Bi-GRU) (Bahdanau et al., 2015), which encodes the context of each element by using a gating mechanism to track the state of sequence.

bi-GRU first uses a forward and a backward GRU (Cho et al., 2014), which is a kind of RNN, to encode the sequence in two opposite directions, and then concatenates the states of both GRUs to form its own states.


Top k Article Extractor We treat the relevant article extraction task as multiple binary classifications.

we build a binary classifier for each article, focusing on its relevance to the input case, which results in 321 binary classifiers corresponding to the 321 distinct law articles appearing in our dataset.


Similar to the preliminary classification phase of , we also use word-based SVM as our binary classifier, which is fast and performs well in text classification (Joachims, 2002;Wang and Manning, 2012).

we use bag-ofwords TF-IDF features, chi-square for feature selection and linear kernel for binary classification.


In our task, bi-directional RNN can help to utilize the cooccurrence tendency of relevant articles.

we use the attentive sequence encoder in Sec.


We can also utilize the gold-standard law articles naturally in the judgement documents to supervise the article attention during training.

given the top k articles, we want the article attention distribution α ∈ R k to simulate the target article distribution t ∈ R k , where t j = 1 k if article j belongs to the gold-standard articles and t j = 0 otherwise.


We also implement an SVM model, which is effective and scales well in many fact-description-related tasks in the field of artificial intelligence and law Aletras et al., 2016).

the SVM model takes bag-of-words TF-IDF features as input, and uses chi-square to select top 2,000 features.


Existing work has studied a related albeit different task, novel document detection (Kasiviswanathan et al., 2012(Kasiviswanathan et al., , 2013Zhang et al., 2002Zhang et al., , 2004, where one aims to identify from a document stream if a newly arriving document is novel or redundant.

this task assumes all the previous documents are known to be "normal", and only checks if a new document is novel.


Instead of using the raw text, similar to (Liu et al., 2015), we use the sequence derived from SegPhrase as input to the word embedding algorithm.

word2vec (Mikolov et al., 2013) is utilized in our experiments, but can be seamlessly replaced by any other embedding results.


Both refinements of the outlierness measure benefits the performance.

by changing the average based outlierness to quantile based outlierness, the recall@1% can be improved by 50-75%, and the recall@5% can also be improved by more than 17%.


In this paper, we leverage two kinds of sequential information: 1) word sequence of subtitles and forum contents, and 2) clickstream log of learning behaviors.

different from conventional representation learning tasks, e.g.


For example, dropout predicting (Qiu et al., 2016), sentiment analysis of learning gains (Ramesh et al., 2015), instructor intervention (Chaturvedi et al., 2014) and answer recommendation (Jenders et al., 2016), etc.

(Agrawal et al., 2015) considers a similar task as ours, which is to recommend video clips to threads.


Para2Vec utilizes word and document coherence to learn their embeddings.

hierarchical Document Vector (hDV) (Djuric et al., 2015) leverages both streaming documents and their contents to achieve better representation, which is similar to our proposed model.


In this paper, we regard the subtitles or threads consistently as documents.

we discover that the log of learners' clickstream, i.e., the click records of watching videos, reading threads and posting threads in a chronological order, can reflect the document-level latent semantics.


We address this challenge of enabling user choice by automatically identifying and extracting pertinent choice language in privacy policies.

we present a two-stage architecture of classification models to identify opt-out choices in privacy policy text, labelling common varieties of choices with a mean F1 score of 0.735.


The early update strategy from Collins and Roark (2004) is applied for training.

each training sequence is decoded, and we keep track of the location of the gold path in the beam.


For better capturing non-local context information, we use LSTM structures to represent different components of each state, including buffer, action, stack, and output.

we exploit LSTM-Minus (Wang and Chang, 2016) to model the buffer segment, conventional LSTM to model the action and ouptut segment, and stack LSTM  to model the stack segments, which demonstrates highly effectively in parsing task.


More specifically, we would like to select the candidate sentence that has the smallest LD from the binary sequence created by the detection model: 1 True and false positives: Given the binary sequences described above, we also use as a feature the ratio of true positives (TP) to false positives (FP) by treating the error detection model as the "gold standard".

we count how many times the candidate hypothesis agrees or not with the detection model on the tokens identified as incorrect: TP We use a linear combination of the above three scores together with the overall score (i.e., original rank) given by the SMT system (we do not include any other SMT features) to re-rank each SMT system's 10-best list in an unsupervised way.


A neutral word r has to be manually selected to anchor the midpoint of the formality score range.

the final formality score for r is enforced to be zero: The neutral word is typically selected from function words.


1 In this paper, we propose a cross-sentence context-aware NMT model, which considers the influence of previous source sentences in the same document.

2 we employ a hierarchy of Recurrent Neural Networks (RNNs) to summarize the cross-sentence context from source-side previous sentences, which deploys an additional documentlevel RNN on top of the sentence-level RNN encoder (Sordoni et al., 2015).


Evaluating on 14 different languages, including the source language improved tagging accuracies in almost all the cases.

our model showed noticeably better performance when the source language and the target languages belong to the same language family, and competitively performed with the highest average accuracies for target languages in different families.


2 To extract image features, we used a convolutional neural network model trained on 1.2M images of 1000 class ILSVRC 2012 object classification dataset, a subset of ImageNet (Russakovsky et al., 2015).

we used VGG 19-layer CNN architecture and extracted the activations of the penultimate fully connected layer to obtain features for all images in the dataset (Simonyan and Zisserman, 2015).


3 Baselines As baselines we use monolingual models, i.e., models trained on each language separately.

we use Visual Semantic Embeddings (VSE) of Kiros et al.


The other three features capture statistics of the relationship f (I t × I h ) between the two sets of visual denotations I t and I h .

this relationship f (I t × I h ) is defined as the the matrix of image cosine similarities: these three features are: • max f (I t × I h ) returns the cosine similarity between the two most similar images.


Possible explanations for sequence effects include: Gambler's fallacy: Once annotators have developed an idea of the distribution of scores/labels, they can come to expect even small sequences to follow the distribution.

in binary annotation tasks, if they expect that True (1) and False (0) items are equally likely, then they believe the sequence 00000 (100% False and 0% True) is less likely than the sequence 01010 (50% False and 50% True).


Our strategy is to tailor our approach to firstorder QA by making strong assumptions about the problem structure.

we assume that the answer to a first-order question is a single property of a single entity in the KB, and decompose the task into two subproblems: (a) detecting entities in the question and (b) classifying the question as one of the relation types in the KB.


These models, however, may not be feasible in all computational settings.

models running on mobile devices are often constrained in terms of memory and computation.


We use a distinct feature group g for each ngram length N , and control the size V g directly by applying random feature mixing (Ganchev and Dredze, 2008).

we define the feature value v for an n-gram string x as where H is a well-behaved hash function.


Therefore, MIN selects memory segments temporally related to w t .

the opinion memory at the time step t is where h O t−i is the (t − i)-th hidden state from O-LSTM.


The work flow of memory interaction and the update process of the internal memories in O-LSTM are kept same with those in A-LSTM except the DIGEST operation.

we set m A t , the task-specific summary of A-LSTM, as The second component of MIN is a generic LSTM called S-LSTM for discriminating sentimental sentences and non-sentimental sentences.


To avoid this kind of error, we add a constraint that an aspect term should come from sentimental sentence.

s-LsTM learns the sentimental representation h s T of the sentence and then feeds it in aspect prediction as a soft constraint: On the whole, our proposed MIN framework has three LsTMs and each of them is differentiable.