2 Related Work Research on the extraction of event relations has concerned both the analysis of the temporal ordering of events and the recognition of causality relations.

the two research lines have progressed quite independently from each other.


Causality is a concept that has been widely investigated from a philosophical, psychological and logical point of view, but how to model its recognition and representation in NLP-centered applications is still an open issue.

information on causality could be beneficial to a number of natural language processing tasks such as question answering, text summarization, decision support, etc.


they involve a source and a target event.

while a list of relation types is part of the attributes for TLINKs (e.g.


Another possibility would be to automatically generate additional data from the Penn Discourse TreeBank corpus, where causality is one of the discourse relations annotated between argument pairs.

a further processing step would be needed to identify inside the argument spans the events between which a relation holds, which may introduce some errors.


to determine the source and target event.

the converse may not hold, since the causal links in the data set are very sparse, and only 2% of the total TLINKs overlap with CLINKs.


The types of entity mentions in a relation are important indicators for the very type of relation.

the coarse (only four types) entity types may not capture sufficient constraints to distinguish a relation.


In Figure 3(b), Aug top1 and Aug top2 achieve similar performances.

when adding one more type with k = 3, we obtain a lower curve, which contradicts the trend showed in the curves of the substitution method (Figure 3(a)).


Biber (1993) mentions, among other things, that it has to be decided for what target population a corpus is meant to be representative, that estimates concerning the quantities of various text types are required, and that decisions with regard to the number of individual text samples and their sizes have to be made.

there is no easy and well established way to verify the success of these measures.


To mine more event mentions, we use the simple trigger-entity pair to represent event pattern in this paper.

lots of event mentions still cannot be extracted due to the ellipsis of arguments.


The bootstrapping procedure of the document-centric view selects frequent patterns in relevant docu-ments and ignores those infrequent patterns both in relevant or irrelevant documents.

the number of infrequent patterns in Chinese is larger than that in English, due to its open and flexible sentence structure, as mentioned in Subsection 3.1.


Collections of relational paraphrases have been automatically constructed from large text corpora, as a WordNet counterpart for the realm of binary predicates and their surface forms.

these resources fall short in their coverage of hypernymy links (subsumptions) among the synsets of phrases.


WordNet (Fellbaum, 1998), on the other hand, is a very rich resource on synonymy and hypernymy.

its coverage of binary relations (as opposed to unary predicates, mostly nouns) is restricted to (mostly) single-word verbs.


SimRank with Fingerprints: Unfortunately, SimRank has very high computational complexity: the run-time of a straightforward implementation is O(Kn 4 ), where n is the number of vertices in the graph and K is the number of iterations in an iterative fixpoint computation (in the style of the Jacobi method).

there are much faster approximations of SimRank.


By the acyclicity of the WordNet hypernymy structure, the process yields a proper DAG.

the output contains redundant links (direct ones and transitive ones connecting the same pair of phrases); these are subsequently eliminated by a transitive reduction algorithm (Aho et al., 1972).


Example: in wmt12, the gold-based LMAE for the top 20% sentences is higher than 0.8 when the system is trained only on the official train set (0% of the post-edited sentences added), but reaches 0.4 when about half of the post-edited sentences are added to the training set.

the global MAE (which takes all the sentences into account) increases from 0.7 (0%) to 0.9 (50% of the post-edited sentences added): since the system assigns more scores in the top tail, it makes larger errors globally.


A head-dependents relation (HDR) is composed of a head and all its dependents, which can be viewed as an instance of a sentence pattern or phrase pattern.

since dependency trees are much flatter than constituency trees, the dependency-to-string model suffers more severe non-syntactic phrase coverage problem (Meng et al., 2013) than constituencybased models (Galley et al., 2004;Liu et al., 2006;Huang et al., 2006).


They acknowledge the utility of a graph structure in representing the content of an ontology, and assert that this structure preserves well the semantics of the ontology.

they do not proceed to examine the ontology in the context of a real-world semantic evaluation.


As traditional WSD algorithms are designed to generate output for every word in the text, recall and precision are the same value.

our algorithm works on the principle of semantic relevance, and there is no guaranteed output; senses with sufficient weight after spreading activation will be displayed.


The hyponymy relation (and converse hypernymy) which forms the ISA backbone of taxonomies and ontologies such as WordNet (Fellbaum, 1989), and determines lexical entailment (Geffet and Dagan, 2005), is asymmetric.

the cohyponymy relation which relates two words unrelated by hyponymy but sharing a (close) hypernym, is symmetric, as are synonymy and antonymy.


For example, we would expect to be able to replace any occurrence of cat with animal and so all of the contexts of cat must be plausible contexts for animal.

not all of the contexts of animal would be plausible for cat, e.g., "the monstrous animal barked at the intruder".


Looking at the results for the hyponym BLESS data set, we can see that the SVM methods do generally outperform the unsupervised methods.

the best performing model is svmSING, suggesting that, for this data set, it is best to try to learn the distributional features of more general terms, rather than comparing the vector representations of the two terms under consideration.


A common approach to tackling the MUC template filling task has involved the employment of pattern-based methods, e.g., Riloff (1996).

supervised learning approaches have constituted a more popular means of approaching the ACE tasks 2 .


(2013), that detects triggers, arguments and their roles.

in contrast to the structured perceptron employed in Li et al.


Evaluation is performed by taking into account the 33 event subtypes, rather than the 8 coarser-grained event types.

evaluation of events according to the GENIA specification considers only the correctness of complete events, after nested events have been broken down.


These results suggest that Wikipedia is a good resource from which to learn whether a food item is a brand or not.

this task could not be completely solved by WIKI since not all food items are covered by Wikipedia ( §4.9).


On the one hand, we produced an auxiliary categorization of food items according to the Food Guide Pyramid, and assumed that a food item is a type when it belongs to a category that is unlikely to contain brands.

we directly modelled the task of brand detection by using seeds provided by the output of the textual ranking features.


Note that the two contexts have no words in common, therefore syntagmatic (neighbor based) contextual features will fail to capture their similarity.

paradigmatic features such as the top substitutes "chairman", "directors", etc.


The figure shows that the performance of the instancebased induction model does not degrade as much as the word-based model as the ambiguity of the words increase.

only 14.94% of the instances in the PTB consists of words with GP greater than 1.5 and 45.71% consists of words with GP exactly 1.


Previous work tend to employ either character-level language models or dictionary-type word lists.

word-level language models have a potential of improving the accuracy and speed of decipherment.


Direct comparison of the execution times with the previous work is difficult because of variable computing configurations, as well as the unavailability of the implementations.

on ciphers of the length of 128, our MCTS version takes on average 197 seconds, which is comparable to 152 seconds reported by , and faster than our reimplementation of the bigram solver of Ravi and Knight (2008) which takes on average 563 seconds.


Various kernels, such as the convolution tree kernel (Qian et al., 2008), subsequence kernel  and dependency tree kernel , have been proposed to solve the relation classification problem.

the methods mentioned above suffer from a lack of sufficient labeled data for training.


We will see that the word representation approach can capture contextual information through combinations of vectors in a window.

it only produces local features around each word of the sentence.


For example, both of the following instances S 1 and S 2 have the relationship Component-Whole.

these two instances cannot be classified into the same category because Component-Whole(e 1 ,e 2 ) and Component-Whole(e 2 ,e 1 ) are different relationships.


Twitter is one among these microblogging services that counts about a billion of active users and 500 million of daily messages 1 .

the analysis of this huge amount of information is still challenging, as language is very informal, affected by misspelling and characterized by slang and #hashtags, i.e.


Notice how no lexical nor syntactic property allows to determine the sentiment polarity.

if we look at the entire conversation that follows: it is easy to establish that a first positive tweet has been produced, followed by a second negative one so that the third tweet is negative as well.


Performance scores report the classification accuracy in terms of Precision, Recall and standard Fmeasure.

in line with SemEval-2013, we also report the F pnn 1 score as the arithmetic mean between the F 1 s of positive, negative and neutral classes.


For such languages, it is natural to want to exploit morphologically rich data to improve parsing performance.

the effect of this is an explosion in the number of possible models due to a huge number of potential features.


The natural language processing task of grammar induction in principle should provide models for how children do this.

previous work on grammar induction has learned from small datasets, and has dealt with the resulting data sparsity by modifying the input and using careful search heuristics.


Bisk and Hockenmaier (2013) used combinatory categorial grammar (CCG) to learn syntactic dependencies from word strings.

they initialise their model by annotating nouns, verbs, and conjunctions in the training set with atomic CCG categories using a dictionary, and so do not learn from words alone.


The former are widely studied by existing methods (Gupta et al., 2007;Aker et al., 2010;Ouyang et al., 2011;Galanis et al., 2012;Hong et al., 2015).

to our knowledge, the latter are firstly incorporated into a regression model in this paper.


(2016) partially address this challenge by linking a summary back to a group of sentences that support the summary.

this linkage is weak since it tells only that there is one sentence or more supporting the summary within the group, without explicitly telling which one(s).


We propose the same hybrid approach combining rule-based and supervised classifiers for the identification of causal relations.

while temporal order has a clear formalization in the NLP community, capturing causal relationships in natural language text is more challenging, for they can be expressed by different syntactic and semantic features and involve both situation-specific information and world knowledge.


The presented approach would probably have more impact if implicit causality was also considered, which we did not take into account because it is not annotated in the Causal-TimeBank corpus.

we plan to investigate this issue in the near future.


This matrix includes several low-correlated words, making several vertically irregular lines.

the time shift operation arranges the irregular words to match the IDSC reports, producing a beautiful horizontal line, as shown in Figure 2b.


In earlier studies, Lasso was employed to model influenza epidemics by Lampos and Cristianini (2010).

in the case of vocabulary size |V |, which is much larger order than sample size T , it has been observed empirically that the prediction performance of l 1 -penalized regression, the Lasso is dominated by the l 2 -penalized one.


Consequently, it caused a considerable decrease of the forecasting accuracy.

some words, such as "fever" and "symptom", showed consistently similar time shifts.


It is therefore not sensitive to outliers, non-linear relationships, or non-normally distributed data.

most intrinsic evaluations of STS systems only report the Pearson correlation.


Note: In case dissimilar pairs are more important for the targeted task, nCG and nDCG can simply be modified by reversing the order of vector m. • Accuracy is a common evaluation measure for many tasks.

as the STS scores are continuously valued, it is unclear how to compute it.


Specifically, we consider the case of three languages or modalities X, Z and Y wherein we are interested in generating sequences in Y starting from information available in X.

there is no parallel training data available between X and Y but, training data is available between X & Z and Z & Y (as is often the case in many real world applications).


Initially we considered the corpus released as part of WMT'12 (Callison-Burch et al., 2012) which contains roughly 44M English-French parallel sentences from various sources including News, parliamentary proceedings, etc.

our initial small scale experiments showed that this does not work well because there is a clear mismatch between the vocabulary of this corpus and the vocabulary that we need for generating captions.


The need to automatically process an increasing number of languages has made obvious the extreme dependency of standard development pipelines on in-domain, annotated resources that are required to train efficient statistical models.

for most languages, annotated corpora only exist for a restricted number of domains, when they exist at all.


PoS sequences thus seem to provide an appropriate level of abstraction for cross-lingual transfer.

contrary to what this intuition suggests, the transfer of the ADJ-NOUN dependency often fails in practice.


It shows that accuracies of related sources are only marginally mod-  ified when source sentences are transformed according to WALS, which could be expected as related languages share most of their typological features.

large gains are obtained for distantly related languages.


Second, data selection mostly targets sequences very far from the target syntax: sentences that only disrespect a local preference of child position are less fluent, and consequently have a lower rank, than their hypothetical counterpart with switched positions; but they are not ungrammatical enough to be pushed into the 10% worst territory.

the data transformation approach is not restricted to preexisting n-grams, and it directly confronts the given sequence with its counterpart to keep only the most fluent, thus acknowledging local preferences.


by being based on some form of phonetic, graphematic, or semantic similarity measure (Jurish, 2010;Bollmann, 2012;Amoia and Martinez, 2013).

neural networks -and particularly deep networks with several hidden layers -are assumed to work best when trained on large amounts of data.


For bi-LSTM, this is the multi-task learning setup-using MTL improves the results by +0.7 pp (±2.8) on average, but again there is a high variance within the individual scores.

for the other methods, adding the 10,000 randomly selected samples to the training set actually decreases the average accuracy, by −0.4 pp for Norma and −2.0 pp for CRF.


The results on the Restaurant domain is similar to those on the Hotel domain, where the neural model significantly outperforms the discrete model.

the neural model gives similar results compared with the discrete model on the Doctor domain.


As shown in the figure, most black dots are on the top-right of the figure and most red dots are on the bottom-left, showing that both models are correct in most cases.

the dots are relatively more disperse in the x-axis, showing that the neural model is more confident in scoring the inputs.


First, the classifiers trained on Hotel reviews apply well to the Restaurant domain, which is reasonable due to the many shared properties among Restaurant and Hotel, such as the environment and location.

the performance on the Doctor domain is much worse, largely due to the difference in vocabulary.


An issue of G-LDA is that the word weights in Gaussian topics are measured by the Euclidean similarity between word embeddings.

the Euclidean similarity is not an optimal semantic measure, since most of word embedding algorithms use exponentiated cosine similarity as the link function (Li et al., 2016a).


The generative process of G-LDA is as follows: 3 MvTM G-LDA defines Gaussian topics, which measure word weights in topics by the Euclidean similarity between word embeddings.

the Euclidean similarity is not an optimal semantic measure of word embeddings.


The topics of LDA contain some noise words, e,g., "mr" and "don"; and G-LDA contains some less relevant words, e.g., the second topic of G-LDA is incoherent.

the topics of MvTM are more precise and clean.


Since HPYLM is based on a Bayesian framework, we can integrate other probabilistic models theoretically for other problems and apply optimization methods in accordance with a Bayesian framework.

other smoothing methods has several parameters that need to be tuned manually.


Running a language model on a recurrent neural network (RNN) (Mikolov et al., 2010) is, of course, a reasonable choice because of the good prediction performance for closed-vocabulary task.

a neural network LM usually does not include a generative process, so it is difficult to apply to unsupervised training of a language model or lexicon acquisition from speech signals.


For example, the probability of p(sing|he, will) is estimated reliably.

the probability of p(sing|will, sh..), which includes a hesitation ("sh.."), is estimated unreliably because the hesitation does not appear in the corpus.


The reason the RNN outperformed our model might be due to the closed vocabulary set in this experiment.

the context information in RNN might be  Figure 4: Weakness of segmental context model suffered from the contiguous noisy words that were caused by the combination of the noise word and OOVs, "<unk>", in the artificial noisy data, and RNN degraded prediction accuracy for artificial noisy data.


Since the type hierarchy of entities is typically built from knowledge bases such as DBpedia, which is regularly updated with new types (especially fine-grained types) and entities, it is natural to assume that the type hierarchy is growing rather than fixed over time.

current FNET systems are impeded from handling a growing type set for that information learned from training set cannot be transferred to unseen types.


WSABIE maps input features and labels to a joint space, where information is shared among correlated labels.

the joint embedding method still suffers from label noises which have negative impacts on the learning of joint embeddings.


Again, each of the contextual models perform much better than the no context baseline.

we see an interesting pattern when comparing the four contextual models; unlike in Turkish, the CRF model performs the best.


Their model was able to achieve an accuracy of 84% over ambiguous tokens in Turkish.

our proposed model uses long short-term memory (LSTM)-based architectures to capture longer range dependencies between a target word and its surrounding context.


The test set contains around 40% of the data set, with 8,055 sentences.

this data set is relatively new and we do not have good feature templates to implement the dense-CRF model.


Thus, it is required that which is solved by the condition of (11).

we analyze the requirement of the first term that (1 − cγ) t (a 0 − a ∞ ) ≤ ϵ q .


Our evaluations suggest that the model is more accurate and faster than alternative techniques.

it would still be good to analyze the performance of the model more deeply.


Incorrect dependency parsing and co-reference resolution will reduce the accuracy of extracting event information.

it also verifies that the method that summarize texts based on accurate event information is effective.


For example, the Jensen Shannon divergence (JS divergence), an information-theoretic measure not relying on human-written summaries, compares system summaries with source documents regarding their underlying probability distribution of n-grams (Lin et al., 2006).

unlike ROUGE, JS divergence is neither submodular nor factorizable (see, e.g., Louis and Nenkova (2013)), and consequently can not be optimized via ILP or submodular function optimization.


Recently, submodular functions have been extensively studied and simple algorithms have been proved to yield nice solutions (Krause and Golovin, 2014;Schrijver, 2003).

we might not want to restrict ourselves to particular kinds of functions, because often, the landscape of the objective function does not have easy to exploit properties (Bianchi et al., 2009).


For long-range search, the scout bees regularly look for new locations and investigate each new area for at least t rounds (where t is the hyper-parameter controlling the number of retry before becoming a scout bee).

in the Swarm Summarizer, the mid-range search is limited compared to the reproduction mechanism, because it is achieved only by either successfully applying several local movements, or by randomly scouting the mid-range areas, both of which are unlikely.


We know from section 4.1 that summaries produced by our algorithms and ICSI have many words and bigrams in common.

since extractive MDS is a combinatorial problem, we can compare two summaries by comparing which sentences have been selected.


All these models use word-level semantic annotations.

providing these word-level semantic annotations is costly since it requires specialised annotators.


We plan to extend our work beyond English data, as predictive parsing is language independent.

beuck and Menzel (2013) find a larger set of virtual nodes to be optimal for German for which we want to assess the merit to our method.


Another core problem for disfluency detection is to keep the generated sentences grammatical.

the sequence tagging methods and RNN method have no power of modeling the linguistic structural integrity.


The label "dislocated" is originally defined in the universal dependencies for languages such as Japanese to describe the syntactic relation of words in a topic-comment structure, but is not defined for Chinese.

in Chinese it is frequent to see the topic-comment structure in a sentence, for example: 1.


Research into character-level models is still in fairly early stages, and models that operate exclusively on characters are not yet competitive to word-level models on most tasks.

instead of fully replacing word embeddings, we are interested in combining the two approaches, thereby allowing the model to take advantage of information at both granularity levels.


Most research on SBD focus on languages that already have a well-defined concept of what a sentence is, typically indicated by sentence-end markers like full-stops, question marks, or other punctuations.

as we study more contexts of language use (e.g.


can only be decided based on capitalization, but especially in the social media datasets this signal is not very reliable.

to nouns, we do not expect adjectives to improve much, as comparative and superlative can be quite easily distinguished from the base form of the adjective and from each other.


A short sequence may temporarily be the best candidate and fulfill the goal, while longer (and possibly correct) sequences are incomplete and may be lost.

long sequences have more features, therefore their score may be arbitrarily inflated.


The preceding and following consonants defined by the onset and coda respectively may or may not be present in a syllable.


(2013) used CRF using two level tagging (phone boundary and phone distance) to perform syllabification for Romanian language.

unlike their study, our approach uses only one level tagging as discussed in the section 5.2.


In order to generate unicode compatible text corpus, we deploy a Bengali OCR 3 .

the accuracy of the OCR is very poor i.e., 52.6% words accuracy.


Learning representations of words is a pioneering study in this school of research.

paragraph (or sentence and document) embedding learning is more suitable/reasonable for some tasks, such as sentiment classification and document summarization.


Theoretically, paragraph-based representation learning is expected to be more suitable for such tasks as information retrieval, sentiment analysis and document summarization (Huang et al., 2013;Le and Mikolov, 2014;Palangi et al., 2015), to name but a few.

to the best of our knowledge, unsupervised paragraph embedding has been largely under-explored on these tasks.


After that, well-developed text processing frameworks can then be readily applied.

such imperfect transcripts usually limit the associated efficacy.


A closer look at the output reveals that the baseline marked only those boundaries that included a clear pause, i.e., "safe" candidates.

the tagger marked not only those clear pauses, but also more subtle boundaries that involved an intensity decrease and not necessarily a pause.


First, we note that our experimental settings are rather different from previously considered settings for domain adaptation in many aspects: • Sufficient target data: In a typical setting for domain adaptation, one generally assumes that the source domain has a sufficient amount of labeled data but the target domain has an insufficient amount of labeled data.

we have sufficient amounts of labeled data for all domains: our goal is to effectively utilize all of them.


The Hindi-Urdu treebanking project is one such example where the influence of differences between Hindi and Urdu texts have led to the creation of separate treebanks for Hindi and Urdu (Bhatt et al., 2009;Bhat et al., 2015).

pursuing them separately in computational linguistics makes sense.


More importantly, if we choose a third script, we have to manually develop a reasonably-sized corpus of transliteration pairs for training the transliteration models.

transliteration pairs in Devanagari and Perso-Arabic scripts can be automatically extracted from the corpora available in these scripts (see §3.1.1 for more details).


Not surprisingly, the results shown in Table 4 (all triggers) and 5 (verbal triggers) are overall lower than when using gold features.

switching from surface to deep syntax leads to higher gain for predicted data than for gold data: 5.1 points (56.7 to 61.7) for all trigger, instead of 4.2 for gold data and 6.7 points (61.3 to 68) instead of 6.4 on gold data for verbal triggers.


The key idea of annotation projection can be summarized as follows: through word alignment in parallel text corpora, the annotations are transferred from the source (resource-rich) language to the target (under-resourced) language, and the resulting annotations are used for supervised training in the target language.

automatic word alignment errors (Fraser and Marcu, 2007) limit the performance of these approaches.


Many automatic word alignment tools are available, such as GIZA++ which implements IBM models (Och and Ney, 2000).

the noisy (non perfect) outputs of these methods is a serious limitation for the annotation projection based on word alignments (Fraser and Marcu, 2007).


Unsupervised RWR -General Relations (B3) -In this approach, we form the graph G for the sentence, similar to as mentioned in Section 3.

we only use path-types 1,2 and 6 of Table 1.


With efforts from Huet, an automated analyser for exhaustive syntactically valid analysis of a Sanskrit sentence, called as the 'Sanskrit Heritage Reader' is available (Goyal et al., 2012;.

the system provides all the possible syntactically valid segmentation and it requires human assistance to choose the relevant segmentations so as to form the semantically correct sentence.


Mongolian person name always express in one word.

when transliterating Chinese person into Mongolian, the person name length unchanged.


This best F-measure even surpassed the performance of POS and ORT feature combination about F 1 is 83.35 under SE method in the same condition.

the overall performance reduced when added all type clusters features to the feature set.


Quantifying semantic relationships between linguistic terms lies at the core of many NLP applications (Pilehvar and Navigli, 2015).

hard matching between words has long been an obstacle in identifying the relatedness of two sentences (ShafieiBavani et al., 2016b).


For large and sparse inputs, sparse operations can be used to construct output layer of an autoencoder.

input layer should also be reconstructed from dense output vectors for which sparse operations cannot be used in the original algorithm.


It is possible to build different DAE models and different kNN classifiers for each level-1 category, just like we do for DBN classification models.

doing so increases the number of models twice and it requires 4 kNN searches to classify a product, instead of 2.


Paragraph vectors obtained from PV-DM and PV-DBoW are shared across context words generated from the same paragraph but not across paragraphs.


We intend to extend the gwBOVW approach to incorporate the path class label in some fashion during the embedding.

most results are shown on proprietary e-commerce datsets.


Previous supervised summarization systems often perform the two tasks in isolation.

since reference summaries are the trade-off between relevance and saliency, using them as supervision, neither of the two rankers could be trained well.


have become growingly popular in this area.

most current supervised summarization systems often perform the two tasks in isolation.


For generic summarization, people read the text with almost equal attention.

given a query, people will naturally pay more attention to the query relevant sentences and summarize the main ideas from them.


For example, (Wan and Xiao, 2009) adopted manifold ranking to make use of the within-document sentence relationships, the cross-document sentence relationships and the sentence-to-query relationships.

to these unsupervised approaches, there are also various learning-based summarization systems.


Sentence compression presupposes a prior step of sentence extraction; it is the only other way how abstractive summaries can be constructed.

this previous step is rarely talked about in the literature.


Bridging the gap between source and target language does come at an additional cost in performance.

there are a number of possible ways to attack this gap in future work, including using targetlanguage lexical resources if available, unsupervised mining of large amounts of parallel data for lexical entries, and also improving the parsing model itself with recent advances in CCG semantic parsing.


We borrow the term 'substitution' from TAG formalism (Joshi and Schabes, 1997).

when k-best constituent parsing outputs are kept for reducing error propagation, further search strategy for an optimal subset of these redundant subtrees will be discussed in Section 3.


Since k-best constituent parsing outputs are used, this work resembles the re-ranking works.

we do not perform k-best list re-ranking, e.g.


This tagging task can be performed with a precision above 93%, with a perceptron tagger.

Table 5: Results on CTB5, the same terms of Table 3. high recall of 'B' and 'H' is more crucial for our task, i.e.


This affirms that pre-word pauses carry constituency-like information.

typing behavior of users differs, as illustrated in Figure 2.


In fact, user keystroke biometrics are successfully used for author stylometry and verification in computer security research (Stewart et al., 2011;Monaco et al., 2013;Locklear et al., 2014).

also eye tracking data like scanpaths (the resulting series of fixations and saccades in eye tracking) are known to be idiosyncratic (Kanan et al., 2015).


For instance on the Ritter data, for 25/38 participants using their keystroke information as auxiliary task helps to improve overall chunking performance.

if we combine all data and train a single model, performance degrades on chunking.


(Note that all compounds are considered complex, while words that are not compounds are considered simplex-e.g., book, rain).

to previous approaches, which have primarily sought to identify the dictionary forms of compound constituents, this approach aims to identify the exact location of word boundaries in compounds.


On average, language modeling alone achieved an accuracy of ∼94%.

a language model coupled with linguistic constraints achieved a much higher accuracy, hovering around 97%.


The presence of constraint errors cautions us that a segmentation approach that uses phonotactic constraints is sensitive to loanwords.

it was only with loanwords where each candidate violated a constraint.


Were yksinomaan truly simplex, its syllabification would be *yk.si.no.maan.

it syllabifies as if it were a compound, with a syllable boundary falling in between the two stems: yk.sin.o.maan.


Despite the success of above methods in learning knowledge representations, most of them mainly consider knowledge base as a set of triples and models each triple separately and independently.

in reality, triples are connected to each other and the whole knowledge base could be regarded as a directed graph consisting of vertices (i.e., entities) and directed edges (i.e., relations).


So far, the translation of a graph context, π(•), takes the embedding results of each subject contained in the context equally.

in reality, different subjects may have different power of influence to represent the target subject.


We ascribe this result to the information bias of sentence embeddings generated by GRU, that is, GRU tends to pay more attention to the words in the end of a sentence.

for the task discussed by this paper, complete semantics provide more help to context-aware candidate selection as discussed in Section 2.1.


The primary metric for recommender system is prediction accuracy.

focusing solely on this metric is reported to limit user satisfaction by always recommending predictable items, such as a new comedy for a comedy fan who can discover it without recommendation.


We may consider selecting a typical entity among the existing entities in the category (such as dog for MAMMAL).

typical properties such as 'lifespan' can be missing with dog due to data sparsity.


Therefore, high P (lifespan|giraffe) does not mean giraffe has an unexpected property.

case B: Property is unexpected, but relevant (serendipitous) Figure 2(b) shows a distribution skewed to low P (a|e).


For example, if a method shows high performance on Q3 but not on Q2, its trivia quiz tends to deviate much from the image/topic.

the method is anyway presenting interesting trivia quiz.


Licence details: http:// creativecommons.org/licenses/by/4.0/ eyetracking data in English (Just et al., 1982).

to date, no similar correlation analyses have been conducted for Japanese.


The results have shown that the unsuitability of MLPs for cyberbullying, reaffirmed the suitability of SVMs for text classification.

in addition, the results show that decision tree based algorithms, Random Forest and J48, can also perform well in short text classification, which is contrary to most of the previously reported results.


In our Complex Word Identification study we learned that words which are simpler to non-native English speakers have much higher probabilities according to language models, both alone and in context, while those which are more complex to them tend to have a smaller number of senses in WordNet.

with what was reported by (Rello et al., 2013b) in experiments with readers who suffer from Dyslexia, we found no evidence of a relationship between the non-natives' perception of complexity and neither word length or number of syllables.


Considering 20cLM (see again Figure 5), it shows lower perplexity values for 20cA than for 19cA.

the difference is relatively small in comparison to the difference observed for 19cLM on 19th and 20th c. abstracts.


6 The program can analyze poems and check if the predominant stress pattern is iambic or anapestic.

if the input poem's meter is not one of those two, the system forces each line into one of them.


AnalysePoems is another tool for identification of metrical patterns written by Plamondon (2006).

with other programs, its main goal is not to perform a perfect scansion, but to only identify the predominant meter in a poem.


Both classifiers predict an iambic pentameter most frequently, but, in the case of the Linear Support Vector Machine there are approximately 50 analyses that only appear once (with slight differences between them).

the number of unique analyses in the CRF results is around 30.


(2014) report leading results of 0.81 F -measure (harmonic mean of precision and recall) on SU boundaries with their DNN-CRF model, outperforming a DT-CRF baseline with 0.774 F -measure.

most of the above-mentioned systems have been trained and evaluated on native speaker telephone conversation and broadcast news dialogues: the Switchboard and Broadcast News datasets prepared for the RT-03/04 shared tasks in MDE by the National Institute of Standards and Technology, U.S. Department of Commerce.


The two parallel sentences don't share any word so that the exact match strategy fails to identify the alignment between them.

the alignment can be captured based on POS, syntactic role and semantic matches.


The differences among score ranges in argumentative essays are more obvious so that high quality sentence parallelism might be a useful indicator of well written argumentative essays.

long chunks appear much less in narrative essays across all score ranges.


In particular, with this features/dataset combination, with the glm library we see slight rises in Precision@5, F1-Score@15 and MAP from 24% to 24.4%, from 19.22% to 20.30% and from 0.127 o 0.136 respectively; for C5.0, while F1-Score rises from 18.95% to just 19.29%, Precision@5 and MAP shows a more significant improvement from 22.2% to 26.8% and from 0.123 to 0.137 respectively, thus supposedly showing a better ranking of the keyphrases found.

using the same feature set over the preprocessed documents still shows an improvement from the baseline, but with slightly lower scores.


Introducing more contextual information seems necessary for dealing successfully with a construction like this one.

in this particular case it is not entirely clear how to determine the lexical aspect of the event in an automated fashion.


We speculate that if a typological feature is prone to horizontal transmission, it is likely judged to be unstable by the exclusively vertical model.

this cannot be confirmed in a straightforward manner.


Like feature stability indices explained in Section 3.2, our model assumes that if the same feature value is shared by a group, the feature in question is stable.

the collection of such groups is implicitly represented as a single neighbor graph.


Therefore, Natural Language Processing (NLP) tasks that need to be sensitive to lexical meaning should treat MWEs as single units.

this is a challenging problem since many MWEs can have multimple morphosyntactic variants, which makes them difficult to recognise or generate.


For English, our original intention was to extract combinations from the Elhuyar English-Basque dictionary, in part because the Basque translations would be useful for the translation process in the MT system.

the dictionary contained too few combinations for this study, so instead we decided to use the Oxford Collocations Dictionary (Deuter, 2008).


The precisions of methods B and C were not as good as that of method A.

the evaluation on instances identified by both B and C methods reveals that detection quality is still very high when linguistic data specific to VNCs is combined with parsing (the second row of scores in Table 5).


72.20% 99% Method B+C but not A 20.85% 97% Method B only 4.12% 93% Method C only 2.83% 83% Table 6: Identification precision for the additional VNCs detected in Spanish As the corpora and parsers we used were different for English and Spanish, the experiments in both languages are not really comparable.

it is evident that the improvement obtained for English was considerably higher than the one obtained for Spanish.


In this paper, we investigated the use of computational methods to analyze the language used in openended comments from student evaluations of teaching effectiveness in order to explore the possibility that gender bias exists in these evaluations.

to previous research that relies on numerical ratings, our results fail to reveal differences by instructor gender in overall student satisfaction, as expressed in written comments.


In the United States, Spontaneous Reporting Systems (SRSs) is the official channel supported by the Food and Drug Administration.

these system are typically under-reported and many ADRs are not recorded in the systems.


This problem can partially be solved by using bi-grams or trigrams.

this leads to the number of features exploding, and the models are thus easily overfitted.


The preposition error detection is one of error cases in NLP-TEA shared tasks.

the preposition correction is beyond the scope of NLP-TEA.


This phenomenon suggests that GRU-ME better fits the training data, but may suffer from overfitting.

the GRU model with a large hidden size is more robust when it is applied to another corpus.


Only for Zulu is the precision mediocre.

we want to choose a single grammar independently of the language, as we do not want to tune our approach to the language (we have no annotated tuning set).


As we can see, the 15 most frequent affixes found for English are indeed correct affixes of English, but among the subsequent 25 most frequent affixes, only seven are correct.

5 for German, a language with much richer inflectional and derivational morphology, all but three affixes are correct among the top 40 (and the top 25 are all correct).


We note again that the performance of LIMS-Scholar depends on the quality of the scholar-seeded knowledge, so that one should be cautious with drawing conclusions.

it appears overall that the cascaded approach of LIMS is a perfectly adequate alternative to using the scholar-seeded knowledge required for LIMS-Scholar.


They announce the best results for German and Spanish datasets.

demir and Ozgur (2014) utilize pretrained word embeddings along with their corresponding word cluster ids to achieve top performance on Czech dataset.


In general, it does this in quite reasonable ways, for instance for tagging the Latin words pastor in in a list of parish priests.

since the foreign words in the SUC training corpus are mostly in English, a language scarcely attested in our early modern corpus, it sometimes overgenerates the UO tag in incorrect contexts, for instance for the word tree 'three' (modern spelling tre).


Other classifiers could be used to model this relationship.

dBN are expected to perform well on high dimensional word-embedding and are able to model non linear relationships between word pairs.


We see from Figure 5 that the performance of KEA improves as the number of training cases increases, for both MLC and CSDS.

on MLC is does worse than the random baseline, likely because it picks keyphrases that are not even among the candidate keyphrases.


It did not take the history information of the user into account.

the microblogs a user posted in recent history can represent their interests to some degree.


(2013) proposed to use translation process to model this task.

to the methods that focuses on topic modelling, Shi et al.


The input memory representation m i of tweet t i is a matrix of size N × dim, where N is the length of the tweet.

because not all words contribute equally to the tweets meaning, and the importance degree of the word w ij should be considered in our models, we propose a probability layer to achieve the goal.


CNN-Attention is the latest method used for this task, and it showed a good performance in this work.

cNN-Attention only considers the content of the tweets and uses an attention mechanism to find important words in tweets.


From Table 3, we observe that changing the number of hops has some impact on the overall performance.

the results disprove that more hops are better, because when the number of hops is larger than 2, the performances of HMemN2N and MemN2N are both decreasing.


2011used an IR approach, by querying English Wikipedia with the top-N topic terms.

in order to do this, they required external resources (two search APIs, one of which is no longer publicly available), limiting the general-purpose utility of the method.


As shown in Table 1, initial accuracy measures are worse than all four competitors.

our system generates exclusively binary-branching output, while all competitors can produce the higher arity trees attested in the PTB-like evaluation standard (notice that our recall measure for the binary branching output beats both CCL and UPPARSE).


Unfortunately, due to the flat representation of these constructions in the gold standard trees, this insight on the part of our learner is not reflected in the accuracy measures in Table 1. short length of child-directed utterances, and therefore the right-branching baseline (RB) outperforms all systems by a wide margin on this corpus.

we argue that such utterances are a more realistic model of input to human language learners than newswire text, and therefore preferable for evaluation of systems that purport to model human language acquisition.


We note that certain error types, having recurrent error patterns, allow for straightforward artificial error data generation.

we experimentally show that quality artificial data cannot be so easily generated for content words (see §6).


This is over the human upper bound of 74%, and it shows that the topic coherence features perform well in contextualising the OOC annotation.

since a 'real-world' pipeline system does not have access to the gold annotations, we replace the gold annotations with the output of COMPDIST.


As both coherence and cohesion are important text properties that are known to influence the readability of texts, readability studies have attempted to exploited both dimensions.

most studies focused on phenomena that falls inside the category of cohesion as defined in Section 2.1 which is why we decided to focus on cohesive features in this paper.


This variable was also included in Coh-Metrix (Graesser et al., 2004), along with similar measures such as word overlap, noun overlap, stem overlap, and argument overlap.

the efficiency of this variable for readability was not assessed before Pitler and Nenkova (2008), who measured its association with text difficult and obtained a non significant correlation (r = −0.1).


(2013) computed other characteristics of lexical chains and co-reference pairs (such as the number of chains, the distance between entities, the average word length of entities, etc.).

with these features, they only reached a precision of respectively 0.367 and 0.384 for a six-class classification problem.


For example, a proper noun is the subject of sentence n and the anaphoric pronoun referring to it is often the subject of sentence n+1 (" Subject to Subject" transition).

the syntactic functions of mentions might change across sentences : the object of the sentence n becomes the subject of the next sentence.


First, all classification models perform better than their regression counterparts.

even for the former, no model using coherence or cohesive features is able to overcome a simple model based on sentence length and word frequency.


On the one hand, 6 features only were found to be significant by semi-partial correlation (when sentence length and word frequency were controlled for).

integrating the best cohesive features in a readability model did not bring significant improvement over a simple baseline on our French data.


The development of semantic resources such as FrameNet (Baker et al., 1998) and PropBank (Palmer et al., 2005) made the training of models for semantic role labelers using supervised techniques possible.

as a consequence of the considerable manual efforts needed to build proposition banks, they exist only for a few languages.


Most previous approaches have used professionally translated parallel corpora, mainly EuroParl (Koehn, 2005) and United Nations Corpora (Rafalovitch and Dale, 2009), to transfer semantic annotation.

creating these resources requires manual efforts; they are thus limited in size and in the number of languages they cover.


(2015), we introduced the concept of using entities as a method for aligning sentences and transferring semantic content in loosely parallel corpora.

the presented approach has the following limitations: (1) it was evaluated on one language only and (2) the evaluation was performed on the generated PropBank itself.


Most previous work relies on word alignments or uses bilingual dictionaries to transfer the predicate annotation between languages.

when applied to new languages and domains, these approaches face a scaling problem requiring either training on parallel corpora or otherwise dictionaries which may not be available for every language.


We assign the argument role to the governing token in the token span covered by each entity.

if the argument token in s SL is dominated by a preposition, we search for a preposition in s T L governing the entity and assign it the argument role.


We observe that the precision increases with the number of entities used in the alignments.

this increase is followed by a decrease in the number of alignments created.


The database also has a cognacy judgment for each word.

the database is not in an uniform transcription.


Comparative summarization is not a new task.

to our best knowledge there is no public benchmark data set available.


(2010) Extracts Heterogeneous en 24 352 ≈ 2,560 characters Lloret et al.

(2013) Extracts Heterogeneous en 310 10 100-200 words Our work Coherent Extracts Heterogeneous de 10 4-14 ≈ 500 words † there are corpora containing only extracts, which are suitable for evaluating extractive MDS systems.


Assuming that we extract the previous word as a feature in line 12, this would result in learning that predicting "hotel" after "dog" is beneficial.

this appears to be the case due to the the loss function and the incorrect partial prediction.


Comparing the two systems, we see that LOLS performs worse in terms of BLEU and ROUGE, but also has lower ERR(%); this indicates that our system tries to include more attributes in the NL sentences.

when looking at the scores in the unique MR subset, the difference between the two systems narrows, with almost no difference in the SF hotel dataset.


(Ganesan et al., 2012) proposed some heuristic rules to generate phrases, they used a modified mutual information function and an n-gram language model to ensure the representativeness and readability of the phrases.

their method didn't consider the descriptiveness of the phrases.


Past studies in education showed that higher-level questions, in contrast to simple factoid questions, have more educational benefits for reading comprehension (Anderson and Biddle, 1975;Andre, 1979;Hamaker, 1986).

most of existing approaches to question generation have focused on generating questions from a single sentence, relying heavily on syntax and shallow semantics with an emphasis on grammaticality (Mitkov and Ha, 2003;Chen et al., 2009;Heilman and Smith, 2010;Curto et al., 2011;Becker et al., 2012;Lindberg et al., 2013;Mazidi and Nielsen, 2014).


Therefore, our ultimate goal is to generate multiple-choice questions from plain texts in an arbitrary domain.

the state-of-the-art in extracting semantic representations of event and entity relations from text does not perform well enough to support our question generation approach.


The distractor generation component needs to generate distractors as close to a rating of 3 as possible.

distractors labeled as 2 ("easily eliminated") often occur because they come from events preceding the event described in the question or from events following the results of the events described in the question.


(ii) Different from syntactic parsers that concentrate on the syntactic accuracy of the syntax trees that their grammars derive, computational construction grammars focus on semantic accuracy as a metric that better meets their objectives.

semantic accuracy is harder to measure than syntactic accuracy because it requires textual corpora annotated with large formal meaning representations that are agreed upon by different grammar developers.


The S-score obtained for a specific phenomenon should not be presented on its own, because it would lead to an erroneous understanding of the grammar accuracy over the whole corpus.

together they give a better understanding how the grammar is working.


Methods for text simplification using the framework of statistical machine translation have been extensively studied in recent years.

building the monolingual parallel corpus necessary for training the model requires costly human annotation.


Recent studies have treated text simplification as a monolingual machine translation problem in which a simple synonymous sentence is generated using the framework of statistical machine translation (Specia, 2010;Zhu et al., 2010;Coster and Kauchak, 2011a;Coster and Kauchak, 2011b;Wubben et al., 2012;Štajner et al., 2015a;Štajner et al., 2015b;Goto et al., 2015).

unlike statistical machine translation, which uses bilingual parallel corpora, text simplification requires a monolingual parallel corpus for training.


Therefore, they may obtain similar sentence pairs effectively.

our simple proposed method achieved equivalent or higher performance than their method without considering any ordering of sentences.


Considering the average correlation scores obtained, the configurations METEOR Vector and ME-TEOR DBnary are comparable, except on German language, for which METEOR Vector obtained a better correlation score.

when we combine lexical data with Vector module (METEOR DBnary + Vector), we observe a small increase of the correlation score, in particular when threshold is tuned, which suggests a tunable version of METEOR.


We aim to capture this drift in the BTC by collecting posts over a number of years, 3 as well as being taken from different times of years, days in the month, and times of day.

previous social media NE corpora were gathered during narrow contiguous time periods (Ritter et al., 2011).


That is to say, crowd recall over the oracle annotation of the data was higher.


We also see that celebrity figures, such as @justinbieber and Kate Middleton, are more prevalent in the social media top ranking -as are journalists, such as @timhudak and David Speers.

the CoNLL data does contain a large number of sportsmen, as it is rich in cricket reportage; e.g.


In order to perform on this task with a good accuracy, the systems will be required to exhibit a deeper semantic understanding of the linguistic tail of the disambiguation tasks we analyze in this paper.

the only task that will explicitly be evaluated is the QA task itself, which means that the annotation task would be largely reduced to the components necessary for the questions and answers.


Using this cross-document information fusion, they find improved performance over monolingual systems.

this work relies on having documents across multiple languages that describe the exact same event, which is an unrealistic case in practice.


(2004) presents an approach for extracted patterns in a source language and translating these patterns for use on a target language.

these works are limited to entity extraction, whereas our focus is on event extraction.


The resultant labels are then used together with word length, existence of special characters in the word, current, previous and next words to train a CRF model that predicts the token level classes of words in a given sentence/tweet.

for within language varieties, AIDA (Elfardy et al., 2014) and AIDA2 (Al-Badrashiny et al., 2015) are the best published systems attacking this problem in Arabic.


Once we complete the analysis of a question's parse tree, not all words in the question are of further relevance to the task of QC.

so as to maximise the number of words that we have rules for, we try to create rules for all words that appear in training set.


Once we have the SM of a question, we use rules to identify the relevant QC.

before we can match appropriate words, we require a way of identifying the correct sense of a word.


(2015) all achieved not bad results using simple classifiers with manually constructed sparse features.

feature engineering is time consuming and has low extensibility to other domains.


They appended an LSTM layer on top of a Hidden Markov Model in automatic speech recognition.

the LSTM they applied was only a shallow architecture.


The best results were obtained using all the three sub-feature sets.

the contribution of the linguistic feature set was negligible (80.47% vs.  80.29%).


Consider the RC example in Figure 1(b), the context word "moved" is a strong indicator for classifying the relation of ⟨People, downtown⟩ as Entity-Destination.

most of the conventional approaches in SRL and RC only considers local context features through feature engineering, which might be incomplete.


In most of the prior work, ILP was only used for AC inference.

this approach limits the interaction of AI and AC when making decisions.


If the predicate is observed in the training data, then the syntactic patterns may still be useful.

in the worst case, if the predicate is unseen, both of the clues become weak leading to the most difficult case for SRL.


So are the verb → adjective patterns that form present participles (+end) and past participles (+t).

the agentive/instrumental nominalization pattern +er (fahren → Fahrer / drive → driver), where argument structure changes, is associated with a loss in performance.


Obvious remaining differences are the language and the type of the distributional model used.

these factors were outside the scope of the current study, so we leave them for future work.


While a lot of valuable information is contained in these linguistic studies, this information is often not readily usable by NLP due to factors such as information overlap and differing definitions across studies.

there is also a current trend towards systematically collecting typological information from individual studies in publicly-accessible databases, which are suitable for direct application in NLP (e.g., for defining features and their values).


For question answering and information retrieval tasks, sentence similarities between query-answer pairs are used for assessing the relevance and ranking all the candidate answers (Severyn and Moschitti, 2015;Wang and Ittycheriah, 2015).

sentence similarity learning has following challenges: 1.


Supervised, unsupervised, and knowledge-based approaches have been studied for WSD (Navigli, 2009).

for all-words WSD, where all words in a corpus need to be annotated with word senses, it has proven extremely challenging to beat the strong baseline, which always assigns the most frequent sense of a word without considering the context (Pradhan et al., 2007a;Navigli, 2009;Navigli et al., 2013;Moro and Navigli, 2015).


Figure 3 shows strong positive correlation between F1 and the capacity of the language model.

larger models are slower to train and use more memory.


Taking the CVC approach as an example, the adapted context vector h CV C t is only used for predicting outputs in the output layer at step t, but not in the adaptation operation at step t + 1.

the DAGRU approach can achieve sequential adaptation.


Generally, these metrics have been focused on translation into English.

there has been little attention into their direct applicability to languages with rich morphology.


System combination has also been successfully applied to statistical machine translation system (SMT) (Och and Ney, 2001;Matusov et al., 2006;Schwartz, 2008;Schroeder et al., 2009).

system combination methods in the phrase-based (PBSMT) (Koehn et al., 2003) and hierarchical (HSMT) frameworks (Chiang, 2007) tend to be rather complex, requiring potentially non-trivial mappings between the partial hypotheses across the search spaces of the individual systems.


The mixture model allows to back propagate errors both to the gating network and the experts themselves.

considering the small size of the training data and the complexity of the experts, in terms of number of parameters, full back propagation is likely to result in over-fitting.


Of course, our result are also influenced by several other factors such as the choice of languages, training data, etc.

we should point out that hierarchically combining a set of 4 systems does improve translation quality.


The drawback of LSTMs or BLSTMs is that we represent a very long sentence as a single vector which is the output of the last time step.

using BLSTM with Neural Attention (NA) mechanism, we represent the sequence of vectors as a combined weighted representation vector by selectively attending to the past outputs.


For example, the Chinese equivalent of English phrase "staged demonstrations" in Figure  4 is "进行 示威".

there is not a direct dependency relation between "进行" and "示威" in the dependency tree.


The most straightforward idea is executing the projection procedure in Section 3.1 many times.

in practice, the growth rate of the number of newly learned phrases is far beyond our imagination.


the test data is not large enough so that all newly learned phrases can be found in the test data.


(2012) produced an event-driven corpus on the ACE 2005 English corpus.

"the annotator was not required to annotate all pairs of event mentions, but as many as possible", as stated in their paper.


There are 9 directed relations and an undirected default relation Other; thus, we have 19 different labels in total.

the Other class is not taken into consideration when we compute the official measures.


Initially, the performance increases if the depth is larger in both settings with and without augmentation.

if we do not augment data, the performance peaks when the depth is 3.


They have a similar problem with ambiguous expressions which they need to solve to determine the correct class (e.g., China can belong to several classes such as LOCATION or PERSON).

they do not identify the actual real world entity of the expression (e.g., there are several cities called China in the US and other countries, but they all belong to the class LOCATION).


Like us, (Biran and McKeown, 2015) focus on DBpedia data and use bigram models.

their approach investigate discourse planning not content selection and relatedly, the basic units of their bigram models are discourse relations rather than triples.


For instance, trees such as (1a) where the subject entity is shared by two triples, will naturally induce the use of an adjective modifier (1b).

trees such as (1d) where the object entity of a triple is the subject of another triple naturally suggests the use of a participial or a relative clause (1d-e).


BL solutions also often include properties such as "source" which are generic rather than specific to the type of entity being described.

s-Model solutions often contain sets of topically related properties (e.g., birth date and birth place) while C-Model solutions enumerate facts (affiliations, mascot, president, battle) about related entities (University of Texas, Austin and United states Navy Reserve).


On the one hand, this corresponds to the monotonic nature of the OpenCCG search space, where new edges are being added without removing the old ones.

delete effects would be needed to capture empty coverage overlap in combination-rule applications.


In case 1a, "Winter comes" and "It comes" are valid solutions.

in case 1b it is impossible to convey all the semantics because it is not possible to use both "Winter" and "Summer" as required (assuming that there are no "and" connectives in our lexicon).


The above compilation provides a necessary criterion for an OpenCCG task to be solvable.

our actual purpose requires a necessary criterion for an OpenCCG edge e 0 to be feasible, i. e., to form part of a solution.


While research on MDS has also considered genres other than newswire (e.g., opinionated blog posts in TAC 2008 or biomedical research papers in TAC 2014), MDS has almost exclusively focused on homogeneous document collections that belong to the same genre.

this homogeneous nature of the existing MDS benchmark corpora does not reflect application scenarios where topically related documents from different genres need to be summarized.


In this example, it is assumed that both of these opinions are expressed about a single restaurant which is not mentioned explicitly.

take the following synthetic example that ABSA is not addressing: "The design of the space is good in Boqueria but the service is horrid, on the other hand, the staff in Gremio are very friendly and the food is always delicious."


Adding an additional aspect of misc was considered.

in the initial round of annotations, we realised that it had a negative effect on the decisiveness of annotators and it led to a lower overall agreement.


Some annotators suggested that the sentence also implies the same opinion for Islington.

at the end all annotators agreed that in such cases no implicit assumptions should be made and only confined area should be labeled.


Results on single location sentences mainly show the ability of the model to detect the correct sentiment for an aspect.

results on two location sentences demonstrate the ability of the system not only on detecting the relevant sentiment of an aspect but also on recognising the target entity of the opinion.


The system correctly identifies that a "Positive" sentiment is expressed for the general aspect about location2.

no sentiment is expressed for this aspect for location1.


(Jiang et al., 2011) was the first to propose targeted sentiment analysis on Twitter and demonstrates the importance of targets by showing that 40% of sentiment errors are due to not considering them in classification.

this task only identifies the overall sentiment and the existing corpora for the task consist only of text with one single entity per unit of analysis.


Early efforts in building them automatically also yielded lexicons of moderate sizes such as the SentiWordNet (Esuli and Sebastiani, 2006;Baccianella et al., 2010).

recent results have shown that automatically extracted large-scale lexicons (e.g., up to a million words and phrases) offer important performance advantages, as confirmed at shared tasks on Sentiment Analysis on Twitter at SemEval 2013-2016 (Nakov et al., 2013;Rosenthal et al., 2014;Rosenthal et al., 2015;Nakov et al., 2016a), where over 40 teams participated four years in a row.


We tried to download all Macedonian tweets based on the Twitter language classification.

it turned out that in many cases, the returned tweets were in Bulgarian or Russian, which are also Slavic languages and share the same alphabet with Macedonian.


The last 50 years have seen an increasing number of immigrants to other countries...

i strongly believe that they are able to sustain their cultural identities and doing so help they keep their origin values.


In general, the happiness category obtains the highest results in two datasets while fear does best on the third.

the most difficult category to be classified correctly seems to be surprise.


N-gram captures word order in short context and NB feature assigns more weights to those important words.

nBSVM suffers from sparsity problem and is reported to be exceeded by newly proposed distributed (dense) text representations learned by neural networks.


Most neural models directly learn compositions upon word embeddings, and are reported to be powerful enough to learn high-quality distributed representations without human intervention.

in sparse case, heuristic weighting techniques designed by humans are shown to be able to bring significant improvements over raw BOW representation (Wang and Manning, 2012;Martineau and Finin, 2009).


For example, NBSVM (Wang and Manning, 2012) uses the ratio of the number of words in positive texts and negative texts to weight words, and achieves competitive results on a range of text classification tasks.

traditional sparse BOW representations take each word or n-gram as a unit and ignore the internal semantics of them.


Hidden layers of RNNs can preserve the historical sequential information, and can be used as the representations of the texts (Dai and Le, 2015).

both CNNs and RNNs are essentially 'flat' models, where structural information (e.g.


In theory, accuracy should benefit from the sequential and structural information of texts.

we surprisingly find that our models outperform other approaches, even though only n-gram information is exploited in our models.


Supporting the observations we have made from the experiments on Dataset 1, we see CNN-SVM outperforming CNN on Dataset 2.

when we use all the features, CNN alone (F1-score: 89.73%) does not outperform the state of the art (Ptácek et al., 2014) (F1-score: 92.37%).


In our generalizability test, when the pre-trained features are used with baseline features, we get 4.19% F1-score improvement over the baseline features.

when they are not used with the baseline features, together they produce 64.25% F1-score.


Constrained SMT uses less data than bilingual word embeddings or stacked denoising autoencoders and still outperforms both.

this approach uses higher quality, in-domain data as well as tuning parameters which adapt it to this domain.


Microblogs such as Twitter and Facebook have gained tremendous popularity in the past decade, they often contain extremely current, even breaking, information about world events.

the writing style of microblogs tends to be quite colloquial and nonstandard, unlike the style found in more traditional, edited genres .


Most focus on content information and use models such as convolutional neural networks (CNN) (Kim, 2014) or recursive neural networks (Socher et al., 2013).

for user-generated posts on social media like Facebook or Twitter, there is more information that should not be ignored.


In the Facebook dataset we study, we use comments instead of reply links.

as the ultimate goal in this paper is predicting not comment stance but post stance, we treat comments as extra information for use in predicting post stance.


With the transformed word embedding feature, SVM can achieve comparable performance as SVM with n-gram feature.

the much fewer feature dimension of the transformed word embedding makes SVM with word embeddings a more efficient choice for modeling the large scale social media dataset.


Table 4 reports the result of their best AD setting, which represents the full joint stance/disagreement collective model on posts and is hence more relevant to UTCNN.

to their model, the UTCNN user embeddings represent relationships between authors, but UTCNN models do not utilize link information between posts.


• Most of the individual features, including the asymmetric features (posSem and dirRel) did not perform well by themselves.

somewhat surprisingly, the graph-theoretic metric lexN W played a greater role by itself (Accuracy=0.7084), implying that this simple NW-based metric could capture some aspect of the directionality of evocation relationships.


The results reported thus far were achieved by utilizing Autoextend synset semantic vectors.

alternatives  Table 7: Comparison of relational vector types.


The other extreme of the spectrum features resources compiled from user-generated content, such as micro-blogs.

these resources often suffer from grammar errors and misspellings, excessive use of acronyms and shortenings, partly due to the constrains of the publication means (e.g.


Surprisingly, we were only able to find subtitles for 12,618 movies and series.

since series are comprised of various episodes, we downloaded subtitles for each episode of every season available in OpenSubtitles.


A key challenge of text difficulty evaluation is that linguistic difficulty arises from both vocabulary and grammar (Richards and Schmidt, 2013).

most existing tools either do not sufficiently take the impact of grammatical difficulty into account (Smith III et al., 2014;Sheehan et al., 2014), or use traditional syntactic features, which differ from what language students actually learn, to estimate grammatical complexity (Schwarm and Ostendorf, 2005;Heilman et al., 2008;François and Fairon, 2012).


The relationship between text readability and reading devices was also studied in the past two years (Kim et al., 2014).

most of these approaches are intended for native speakers and use texts from daily news, economic journals or scientific publications, which are too hard to read for beginning and intermediate language learners.


L2R reaches 70.39%, 81.46%, 78.14%, and 55.54% of performances of the monolingual run in FR, DE, ES, and IT collections respectively.

the proposed L2R outperforms all the baselines with long queries in almost all the metrics.


TKs allow for using all the substructures of the relational structures as features in the learning algorithm.

in Web forums the TK performance is downgraded by the presence of noise and insignificant information, which also makes TKs too slow for processing large datasets.


In this work, we also tried to exploit neural models using their top-level representations for the (q o , q s ) pair and fed them into the TK classifier as proposed by , but this simple combination proved to be ineffective as well.

neural embeddings and weights can be useful for selecting better representations for TK models.


A Freebase predicate is a predefined relation, mostly consisting of a few words: "place of birth", "nationality", "author editor" etc.

a pattern is highly variable in length and word choice, i.e., the subsequence of the question that represents the predicate in a question can take many different forms.


(2015), we propose to apply dropout to the cell update vector g t as follows: Different from methods of (Moon et al., 2015;Gal, 2015), our approach does not require sampling of the dropout masks once for every training sequence.

as we will show in Section 4, networks trained with a dropout mask sampled per-step achieve results that are at least as good and often better than per-sequence sampling.


We first note that approximating Algorithm 1 by ignoring the rejection sampling step results in slightly worse performance.

without the rejection sampling, copLDA converges faster in terms of iterations.


Note that senLDA is the model with the fastest convergence rate with respect to the number of Gibbs iterations.

lDA, coplDA sen and coplDA np require the same number of iterations, which depends on the dataset.


In the IPMT system, the utilization of the prefix greatly affects the interaction efficiency.

state-of-the-art methods filter translation hypotheses mainly according to their matching results with the prefix on character level, and the advantage of the prefix is not fully developed.


The first interactive machine translation systems (Kay and Martins, 1973;Zajac, 1988;Yamron et al., 1993) focus on having human translators disambiguate the source texts through answering questions.

this question-answering process remains a laborious one for human translators.


Hunter and Resnik (2010) directly introduce the source-language syntactic constraints into the decoding of phrasebased MT system, which are almost the same as our work.

this method builds an independent syntactic re-ordering model and scores the hypotheses through features.


Interestingly, we also find that the topic-informed NMT system tends to produce more words in translations, namely 50,913 and 34,695 words in NIST 2004 andNIST 2005, respectively.

the NMT baseline produces 44,552 and 30,558 words in NIST 2004 andNIST 2005, respectively.


Inspecting the translation examples in Figure 5, 议长/speaker and 活跃/active fail to be translated by the baseline NMT system.

topic-informed NMT is able to use the known topic information, either from the source sentences or previous translations, to produce correct translations.


L m and L v are cross-lingual objectives which are defined as the dissimilarities between statistics: where i is index of each element in the vectors.

it is nontrivial to optimize the monolingual and cross-lingual objectives simultaneously in equation 7.


So the first approach to combine neural and statistical machine translation is not able the combine the strength of both system.

the NMT system seems to be not able to recover from the errors done by the SMT-based system.


For example, vectorial representations of the contexts are often used (Turney and Pantel, 2010), but unrelated with weighting schemes and relevance functions used in IR (with the exception of Vechtomova and Robertson (2012) in the slightly different context of computing similarities between named entities).

the weighting of contexts provides more relevant neighbors.


Moreover, we also use a least squares formulation to constrain these vectors such that w i •w j reflects the co-occurrence statistics of words i and j.

to GloVe, however, we explicitly model two factors that contribute to the residual error of this model: (i) the fact that corpus statistics for rare terms are not reliable and (ii) the fact that not all words are equally informative.


Moreover, one might presume that word associations are themselves simply derived from the distribution of words in the external language: in that case, one would expect them to be an inferior and noisy measure.

several strands of research support the idea that word associations capture representations that cannot be fully reduced to the distributional properties of the E-language environment.


This is contrasted with more statistical approaches such as Latent Semantic Analysis (Landauer and Dumais, 1997) and topic models (Griffiths et al., 2007).

it is not clear that this holds up in light of the fact that we find very little difference in performance between count models and word2vec, or previous work arguing that word embedding models perform an implicit matrix factorization (Levy and Goldberg, 2014).


When used independently, the performance is the same as always predicting the major category.

the string of the connective provides strong clues for the relation type.


The reason behind this is probably due to the fact that the existence of a connective often gives strong hint on at least one side of the interval.

there is often no explicit indication on the boundary of the other side of the interval.


Secondly, the arguments for a relation may be useful for relation type recognition.

the accuracy for argument extraction must be improved before the extracted arguments are used as features.


Moreover, factuality (Fact), aspect (Asp), modality (Mod) and polarity (Pol) information prove to be useful for discourse parsing.

the tasks derived from tense (Ten) and coreference (Coref) annotations do not lead to improvements.


Discourse parsing has proven useful for many applications (Taboada and Mann, 2006), ranging from summarization (Daumé III and Marcu, 2009;Thione et al., 2004;Sporleder and Lapata, 2005;Louis et al., 2010), sentiment analysis (Bhatia et al., 2015) or essay scoring (Burstein et al., 2003;Higgins et al., 2004).

the range of applications and the improvement allowed are for now limited by the low performance of the existing discourse parsers.


Neural models provide a solution with distributed representations, which could encode the latent semantic information, and are suitable for recognizing semantic relations between argument pairs.

conventional vector representations usually adopt embeddings at the word level and cannot well handle the rare word problem without carefully considering morphological information at character level.


Unlike the traditional model in which words usually represent as one-hot vectors and are independent with each other, vector space models reveal the relationship and capture the intuition among words which different or similar to others along a variety of dimensions (Mikolov et al., 2013).

embeddings considering only at word level is usually not good for rare words as discussed above and we introduce character-level embedding to enhance the current word embedding.


Previous work on non-cooperation has focused mainly on non-linguistic task-related non-cooperation or modelled non-cooperation in terms of special rules describing non-cooperative behaviours.

we start from rules for normal/correct dialogue behaviour-i.e., a dialogue game-which in principle can be derived from a corpus of cooperative dialogues, and provide a quantitative measure for the degree to which participants comply with these rules.


These contributions show that user-model approaches to dialogue modelling are flexible enough to account for situations of an arbitrary degree of intricacy.


A recent version of the system (Plüss et al., 2011;Traum, 2012) supports cooperative, neutral and deceptive behaviour, and also is able to reason in terms of secrecy in order to avoid volunteering certain pieces of information.

their model the adversarial scenarios by means of a set of rules that the interlocutors follow.


No specific model of events is required here.

this representation is too simplistic to describe many of the temporal relations that are often explicitly conveyed in language.


When approached as a classification task, the assumption made by most tools is that classes are independent.

this is not correct in this case, for two reasons.


Folding offers a big improvement over other methods, in terms of both error reduction above most-common-class baseline and also absolute accuracy.

it is not applicable to real-world data.


The expressivity of a representation has an inverse correlation with automatic temporal relation typing performance.

more expressive representations are required in order to accurately capture temporal structure, or even to reduce annotator confusion.


An events in ECB+ can be in one of 3 categories: (b) Event Types Figure 2: (a) Documents in ECB+ are divided into topics and sub-topics, and coreference links can exist across documents in the same sub-topic.

a coreference system is not aware of the topic (or sub-topic) partition at test time.


As a result, a system can (incorrectly) link two event mentions across topics (or sub-topics), for which it should be appropriately penalized in evaluation.

we will see in the next section that current evaluations do not meet this requirement.


Note that this behavior is not exclusive to the SIMPLE-CDEC -a system which exclusively predicts within document links can achieve such high scores under the B&H and YCF settings.

we can get a clearer assessment of cross document coreference performance by evaluating the models under the PURE-CDEC setting.


It should be noted that Lemma performs worse than Lemma-WD because it makes incorrect cross document links, due to the naive nature of the lemma match.

lemma-WD does not make any across document links, avoiding incurring these penalties.


We believe that this claim held in these works only due to the lenient evaluation settings of B&H and YCF, which did not appropriately penalize the incorrect across topic (and sub-topic) links made by the Lemma baseline.

the SIMPLE-CDEC and PURE-CDEC evaluations show that Lemma-δ is a stronger baseline.


Our task differs from this task in that we aim to estimate the segment boundaries of unsegmented lyrics using machine learning techniques.

to the segmentation of lyrics, much previous work has analyzed and estimated the segment structure of music audio signals using repeated musical parts such as verse, bridge, and chorus (Foote, 2000;Lu et al., 2004;Goto, 2006;Paulus and Klapuri, 2006;McFee and Ellis, 2014).


tends to appear at the end of a segment.

a phrase like "I'm sorry" may appear at the beginning of a segment.


This result supports the hypothesis that sequences of repeated lines (diagonal lines in the SSM) are important clues for modeling lyrics segmentation.

to results reported in text segmentation literature (Beeferman et al., 1999), TF features turned out to be ineffective for lyrics segment boundary estimation, except for TF2 unigram features.


More investigation is needed for further improvement.

to the case of Figure 5, Figure 6 shows a typical example of false negatives.


Reithinger and Klesen (1997) used a language model to predict the probability of a certain DA.

the effort to predict probability using a language model results in a severe loss of information, thereby leading to a poor result.


The audio feature based Western music mood classification system achieved better F-measure (11.5 points absolute, 19.5% relative) than the Hindi-one.

there were less number of instances present in the Western music as compared to the Hindi music.


Therefore, both SVM (classification) and SVM (regression) failed on this case.

lSTM models term dependencies in sequences with a memorizing-forgetting mechanism.


(Boyer et al., 2010) used acts in conjunction with more specific task actions, e.g., opening a specific file, etc., to discover hidden modes using a HMM.

we assume a pre-defined set of modes (see next section) that generalize across tutors and identify modes and acts jointly.


Recall that in MLNs, generally, all the groundings of a first-order formula share the exact same weight.

in several practical cases, we need to decrease the bias of the model by introducing more parameters for it.


In our experiments, we use Gurobi, a state-of-the-art ILP solver to compute the MAP solution for the MLN (Sarkhel et al., 2014).

it turns out that a naive application of approximate MAP solvers to our problem is still infeasible in practice.


Recently, a statistical based approach has been proposed for resolving NSU question (Raghu et al., 2015).

this approach only focuses on the simpler problem of (a) how old was john rolfe when he died ?


RNN encoder decoder models have been successfully trained on huge parallel corpus of millions of sentences Sutskever et al., 2014).

it is extremely hard to obtain conversation data of this magnitude.


A standard RNN encoder decoder model will end up having UNK symbols as output.

it is not possible to determine which word does this symbol correspond to, as there will be typically many UNK words in an input sequence.


One possible method to evaluate our models is to manually compare the generated output sequence to gold standard (collected from a held out set).

this method is slow, human intensive and prone to errors.


(2015) propose a taskoriented NLG model that can generate the responses providing the correct answers given the dialogue act (for instance, confirm or request some information), including the answer information.

the context information, such as the input question and dialogue act, is ignored.


To address the problem, Oh and Rudnicky 2000propose a statistical approach which can learn from data to generate variant language using a class-based n-gram language model.

due to the limits of the model and the lack of semantically-annotated corpora, the statistical approach cannot ensure the adequacy and readability of the generated language.


For ease of notation, we assume here without loss of generality that every word is modelled with a single-state Hidden Markov Model (HMM); in fact, every word is actually composed of a sequence of phonemes, and every phoneme is modelled by an HMM with 3 emitting states.

this hierarchy of models would lead to excessively long equations, and we prefer to simplify the presentation of this baseline.


In general, these methods added resource-based knowledge to their systems in order to form word vector representations, showing impressive performance gains over methods which did not address the rare words problem.

soricut and Och (2015) applied an automatic method to induce morphological rules and transformations as vectors in the same embedding space.


In comparison to LSTM and CCNN-LSTM, LBL2 SW ordSS 's lower performance on test data was expected as the former are more non-linearly complex language models.

for tasks like spoken term detection, having low perplexities on most frequent set of words is not good enough and hence, we compare LMs on the perplexity of a rare-word based test set.


Access to the displayed snippets is on demand and the user can access the information in context without the need to formulate a specific query.

these advantages are fundamentally based on how well the system is able to retrieve relevant documents, as the system's utility diminishes when proposing a lot of irrelevant documents.


they are all pretrained on the same Simple English Wikipedia dump from May 2016.

our proposed method and the TF-IDF baseline can also produce terms that are DRUID multiwords, whereas the original implementation of Habibi and Popescu-Belis (2015) can only produce single keywords.


We also saw good results using the method proposed by Habibi and Popescu-Belis (2015), with the diversity constraint (λ) set to the default value of 0.75, which was the optimal value in the domain of meeting transcripts.

we noticed that the publicly available Matlab implementation of this method 18 only removed stopwords as part of its preprocessing (5).


"in coal power plant" appears in the transcript instead of "nuclear power plant".

in this example Habibi and Popescu-Belis' system finds better matching articles, like "Microscope", "Light Microscope", "Mangrove" , "Fresh water", "River delta", "Fishing" which can be attributed to finding the keyword "microscope" and otherwise picking simpler keywords like "ocean", "sea", "fishing" and "river", which our system entirely misses.


This can change the dense vectors and IDF values for the constituents of multiwords compared to training on single words and thus affect ranking scores.

in some of the automatic transcriptions, only constituents of the correct multiwords can be found because of transcription errors, so that our method has to rank the constituent instead of the full multiword.


In previous work, NLP methods have been successfully applied to both assessing proficiency levels in L2 input texts collected from coursebooks and output texts written by learners (see section 2).

the two text types have always been considered separately, while we argue that there is a shared linguistic content between the two that can be used for knowledge transfer.


Looking into POS diversity and parser label diversity as shown in  Table 4: Accuracy from 10-folds cross validation using all features diversity; 0.37 for syntax labels diversity).

this feature may correlate with review length, which is considerably higher for experts (362 words) than for laymen (107 words).


(2014) show that structured events from open information extraction (Yates et al., 2007;Fader et al., 2011) can achieve better performance compared to conventional features, as they can capture structured relations.

one disadvantage of structured representations of events is that they lead to increased sparsity, which potentially limits the predictive power.


Second, we use a simpler neural tensor network model to learn knowledge graph embedding, which is easier to train.

the baseline event embedding model uses a recursive neural tensor network architecture to preserve the original structure of events.


This is because it is difficult to investigate the relationship among companies, and therefore news about other companies can be noise for predicting the stock prices of a company.

knowledge graph can provide attributes of entities and relations between them, hence it is possible to learn more information from related companies to help decide the direction of individual stock price movements.


Topic modeling techniques based on Dirichlet allocation are recently popular in text analytics (Blei et al., 2003), which is particularly effective on long documents to find meaningful topics (Blei et al., 2004).

we hereby emphasize that all these techniques are not directly helpful to improve recommendation systems.


Others, staying closer to the surface level, have recently experimented with sophisticated measures of textual similarity (Jimenez et al., 2013;Sultan et al., 2016) or inferring informative answer patterns (Ramachandran et al., 2015).

similar to other NLP tasks (like, for example, TE), one of the biggest challenges remains beating the lexical baseline: At SemEval-2013, the baseline consisting of textual similarity measures comparing reference and student answer frequently was not outperformed.


These corpora contain answers by mostly highly proficient speakers.

there are learner corpora assessing language students' reading comprehension by asking questions about the content of a text.


4 The results are better on the aggression-loss subset (BCS) than on the full dataset (TCF).

the aggression-loss subset does not represent real-world data, as all tweets that are not labeled loss or aggression were removed prior to the experiment.


From the statistics, we observe that the products "Samsung Galaxy" and "Xbox" are actively discussed by the users with the 39%, 37% active level respectively.

the communication on the topic "PlayStation" is not as frequent as the communication on the other two topics.


(Hu and Liu, 2004;Pang and Lee, 2008;Mukherjee et al., 2012).

all these methods fail to explore the reason why people express or change their opinions.


The results show that the positively influential users more likely utilize the words describing the facts, e.g., "security", "special" and impress".

the tweets posted by strong negative influential users are more emotional with the words like "Woo", "Wow" or the emoticons "o o".


Supervised scheme iqf • qf • icf also performs better than standard LDA at most of cases.

tf-idf-LDA gets the worst results in both datasets.


Superficially, noun synset relation structures seem largely to correspond, with hyponymy forming the backbone of a relation network.

on a closer look, various contrasts come to the fore.


However, we do not consider this approach further given the target domain and only mention it here for completeness.

as outlined above, the very nature of our problem invalidates the first line of approach.


Note that such methods assume that an exhaustive language specific list of MWEs is available.

since our task is primarily concerned with MWE "discovery", such standard lexicons may not be used.


3 System description & algorithms Thus, so far we have established that the nature and size of tweets are an hindrance for the standard tokenization process.

using word graphs would circumvent both problems.


The trigger annotation schemes are similar, the two data sets share many event types, and events occur only intrasentential.

some event types in TAC are more fine-grained, e.g., TRANSPORT in ACE was split into TRANSPORT-PERSON and TRANSPORT-ARTIFACT in TAC.


Comparing full inference, we note that Yang and Mitchell (2016) use a pipeline approach: They first tag sentences for potential entity mention and event trigger candidates, and apply label inference afterwards.


For example, if an entity is the Target of an ATTACK event, it rarely becomes the Attacker, at least not within one document.

the same entity may be the Victim of a DIE event, but never the Agent.


Recently, end-to-end memory networks have shown promising results on Question Answering task, which encode the past facts into an explicit memory and perform reasoning ability by making multiple computational steps on the memory.

memory networks conduct the reasoning on sentence-level memory to output coarse semantic vectors and do not further take any attention mechanism to focus on words, which may lead to the model lose some detail information, especially when the answers are rare or unknown words.


The main merits of these representation learning based methods are that they do not rely on any linguistic tools and can be applied to different languages or domains.

the memory of these methods, such as Long Short-Term Memory (LSTM) (Hochreiter and Schmidhuber, 1997) and Gated Recurrent Unit (GRU) (Cho et al., 2014) compressing all the external sentences into a fixed-length vector, is typically too small to accurately remember facts from the past, and may lose important details for response generation (Shang et al., 2015).


Taxonomy generation and evaluation in this submission is restricted to English Wikipedia.

it can be easily adapted to other languages by porting the heuristics, a fairly straightforward task if a dependency parser is available in the target language.


Compared to the WIBI taxonomies, HEADS shows significantly lower precision and recall scores in this evaluation.

the losses can be largely attributed to the simplification procedure (cf.


Leveraging a dependency parser is a possible way to capture more complex aspect structures, or at least to provide clues about the presence of entity modifiers (e.g., via AMOD relations).

deep parsing is known to be inaccurate on NUT, and training suitable models difficult, since NUT lacks proper grammatical structure in the first place.


We use standard recall, precision, and F 1 score metrics.

due to the different granularity of the output produced by the systems and of the GS annotations, the definition of a correct extraction varies slightly with each evaluation task.


The full model significantly outperforms all baselines.

it does not match the handcrafted approach.