Previous work on attention and pooling has only considered a small number of the possible configurations along those dimensions of attention.

the internal/external and un/weighted distinctions can potentially impact performance because external resources add information that can be critical for good performance and because weighting increases the flexibility and expressivity of neural network models.


Perhaps the combination of the external resource and the more indirect representation of the entire sentence produced by the RNN is difficult.

hedge cue patterns identified by convolutional filters of the CNN can be evaluated well based on external attention; e.g., if there is strong external-attention evidence for uncertainty, then the effect of a hedge cue pattern (hypothesized by a convolutional filter) on the final decision can be boosted.


We use the term sequence-agnostic for this.

we propose to investigate sequence-preserving attention as presented in Section 3.3.


Our motivation for introducing sequence-Model wiki bio SVM (Georgescul, 2010) 62.01 78.64 HMM (Li et al., 2014) 63.97 80.15 CRF + ling (Tang et al., 2010) 55.05 86.79 Our CNN with external attention 67.52 85.57 Table 5: Comparison of our best model with the state of the art preserving attention was that the semantic meaning of a sentence can vary depending on where an uncertainty cue occurs.

the core of uncertainty detection is keyword and keyphrase detection; so, the overall sentence structure might be less important for this task.


Attention in the literature computes a weighted average with internal attention weights.

we investigate different strategies to incorporate attention information into a neural network.


The underlying intuition is similar to attention for machine translation, which learns alignments between source and target sentences, or attention in question answering, which computes attention weights based on a question and a fact.

these sources for attention are still internal information of the network (the input or previous output predictions).


The Surface Web is the portion of the web that can be crawled and indexed by the standard search engines, such as Google or Bing.

despite their existence, there is still an enormous part of the web remained without indexing due to its vast size and the lack of hyperlinks, i.e.


These techniques can not be applied to Tor HS since the onion addresses are constructed with 16 random characters.

tools like Scallion 6 and Shallot 7 allow Tor users to create customized .onion addresses based on the brute-force technique e.g.


Multitask learning has been applied successfully to a range of tasks, mostly morphosyntactic.

little is known on when MTL works and whether there are data characteristics that help to determine its success.


The very low value for POS indicates a distribution that, although Zipfian, has very few outliers as a result of the small label set.

dEPRELS, coming from the same corpus, has about three times as many labels, yielding a distribution that has fewer mid-values while still being less than 3.


The former offers the opportunity to change the POS inventory to the three times larger PTB inventory while using the same corpus.

the characteristics of the UD/UPOS we have used as POS throughout the article makes it a more suitable auxiliary source, in fact it systematically outperforms the other two.


Most earlier work had in common that it assumed jointly labeled data (same corpus annotated with multiple labels).

in this paper we evaluate multitask training from distinct sources to address data paucity, like done recently (Kshirsagar et al., 2015;Braud et al., 2016;Plank, 2016).


This indicates a clear advantage of CBOW embeddings over count-based representations for capturing attribute meaning at the word level.

this holds only for adjectives; noun embeddings in isolation perform much worse.


The majority of existing unsupervised approaches focus on optimizing the accuracy of the method, sacrificing its interpretability due to the use of opaque models, such as neural networks.

our approach places a focus on interpretability with the help of sparse readable features.


It lies at the core of language understanding and has already been studied from many different angles (Navigli, 2009;Navigli, 2012).

the field seems to be slowing down due to the lack of groundbreaking improvements and the difficulty of integrating current WSD systems into downstream NLP applications (de Lacalle and Agirre, 2015).


Since semi-supervised models have been shown to outperform fully supervised systems in some settings (Taghipour and Ng, 2015b;Başkaya and Jurgens, 2016;Iacobacci et al., 2016;Yuan et al., 2016), we evaluate and compare models using both manually-curated and automatically-constructed sense-annotated corpora for training.

to supervised systems, knowledgebased WSD techniques do not require any senseannotated corpus.


The main disambiguation clue seems to be given by its previous and immediate subsequent words (federal and government), which tend to co-occur with this particular sense.

knowledge-based WSD systems like UKB or Babelfy give the same weight to all words in context, underrating the importance of this local disambiguation clue in the example.


As described in the Introduction, we collected the historical multiple-choice questions from Gaokao all over the country in rencent five years.

quite a lot contain graphs or tables which require the techniques beyond natural language processing(NLP).


Although (Yih et al., 2015) employed knowledge base, but still failed on multiple sentences questions which is beyond the scope of semantic parsing.

the diversity of candidates in GKHMC makes these models fail to match the question with the right candidate.


These neural approaches aim to obviate the need for any feature engineering and instead focus on developing a neural architecture that learns the representations and the ranking.

while it is possible to view a purely neural approach as an alternative to machine learning involving domain knowledge in the form of handcrafted features, there is no reason why the two approaches cannot be applied in tandem.


The approach achieves state-ofthe-art results.

it requires unsupervised pretraining of the Paragraph Vector model on a relatively big in-domain dataset.


The system described in Section 3 with no interaction transformation (only the encodings are passed to the MLP) and without any external features (x ext in Section 3 and in Figure 1), referred to as GRU-MLP, outperforms the CR and the Random baselines and the systems based on the discourse features.

it performs slightly worse than the approach of (Bogdanova and Foster, 2016).


This approach thus represents a key example of complex reasoning over Horn clause chains using neural networks.

for multiple reasons detailed below it is inaccurate and impractical.


(2) The same previous work takes only a single path as evidence in inferring new predictions.

as shown in Figure 1b, multiple paths can provide ev-idence for a prediction.


Given a vector of scores, the LSE is calculated as and hence the probability of the triple is, The average and the LSE pooling functions apply non-zero weights to all the paths during inference.

only a few paths between an entity pair are predictive of a query relation.


This means that during the back-propagation step, every path will receive a share of the gradient proportional to its score and hence this is a kind of attention during the gradient step.

for averaging, every path will receive equal p 1 N q share of the gradient.


(2012) extend PRA by augmenting KB-schema relations with observed text patterns.

these methods do not generalize well to millions of distinct paths obtained from random exploration of the KB, since each unique path is treated as a singleton, where no commonalities between paths are modeled.


For the sake of these experiments we chose to stop after an hour of an annotator time (the initial lexicon expansion bootstrap and annotating/adjudicating 1,100 sentences).

the human annotator using RASCAL gets a fairly good sense of what kind of annotations are being spotted and what is being missed.


the arguments are convincing: Predicting multiple related tasks should allow us to exploit any correlations between the predictions.

in much of this work, an MTL model is only one possible explanation for improved accuracy.


12 Computational complexity is not an issue for standard semantic benchmarks such as SimLex-999 or MEN: these data sets require only N gt similarity computations in total, where N gt is the number of word pairs in each benchmark (999 or 3000).

complexity plays a major role in the USF evaluation: the system has to compute |W c | • |V r | similarity scores, where |W c | ≈ 5, 000, and |V r | is large for large vocabularies (typically covering > 100K words).


Interestingly, the best scoring model is Glove, a model which uses document-level co-occurrence, which steers it towards learning topical similarity.

the worst performing model relies on dependency-based contexts which better capture functional similarity (Levy and Goldberg, 2014) and outperform other context choices in word similarity tasks on SimLex and SimVerb (Melamud et al., 2016;.


Quality is assessed for well-arranged discussions that seek agreement.

to the subjective nature of effectiveness, people are good in such an assessment (Mercier and Sperber, 2011).


In accordance with the moderate αvalues, full agreement ranges between 17.4% and 44.7% only.

we observe high majority agreement between 87.5% and 98% for all dimensions, even where scores are rather evenly distributed, such as for global acceptability (95.4%).


The coefficients of emotional appeal seem lower than expected, in particular for effectiveness (.31), indicating the limitation of a correlation analysis: As reflected by the 235 texts with majority score 2 for emotional appeal, many arguments make no use of emotions, thus obliterating effects of those which do.

clarity was scored 2 in most cases, too, so the very low value there (.14) is more meaningful.


The difference between the response and key is quantified by a similarity metric such as accuracy, and different system outputs are compared to each other by ranking their scores with respect to the similarity metric.

comparing the scores of the similarity metric does not paint the full picture of the differences between the outputs, as we will demonstrate.


To quantify the difference of two corefer-ence system outputs S 1 and S 2 , given a key K, we count how many of the mentions m are classified differently using a mention classification function c: The mention classification function c requires a class inventory which is not featured by the common evaluation metrics for coreference resolution.

10 Therefore, we adapt the mention classification paradigm introduced in the ARCS framework for coreference resolution evaluation (Tuggener, 2014) which assigns one of the following four classes to a mention m given a key K and a system response S: one issue with ARCS is to determine a criterion for the TP class, i.e.


All these metrics ex-cept SPICE and WMD define the similarity over words or n-grams of reference and candidate descriptions by considering different formulas.

sPICE (Anderson et al., 2016) considers a scene-graph representation of an image by encoding objects, their attributes and relations between them, and WMD leverages word embeddings to match groundtruth descriptions with generated captions.


In this example, failure of SPICE is likely due to incorrect parsing or the failure of synonym matching.

failure of CIDEr is likely due to unbalanced tf-idf weighting.


A common way of assessing the performance of a new automatic image captioning metric is to analyze how well it correlates with human judgements of description quality.

in the literature, there is no consensus on which correlation coefficient is best suited for measuring the soundness of a metric in this way.


The correlations within COMPOS-ITE dataset are even very high for all the metrics that consider n-grams, namely BLEU, CIDEr, ME-TEOR and ROUGE.

the correlations of these metrics against SPICE and WMD are not that high.


On ABSTRACT-50S dataset, the CIDEr metric outperforms all other metrics in both HC and HI cases.

on PASCAL-50S dataset, the WMD metric gives the best scores in three out of four cases.


MT evaluation metrics may attribute high scores for these pairs since they mainly focus on lexical and syntactic matching.

as our examples demonstrate, meaning could easily be lost if we rely only on form related MT system evaluation metrics.


Connotation indicates cultural or emotional association carried by words that appear in sentences (Feng et al., 2013).

to the sentiment polarity, connotation polarity indicates subtle shades of sentiment beyond denotative or surface meaning of text.


Extracting meaning related features from text and using form related features for MT evaluation have been studied separately.

integrating meaning related features into MT quality evaluation can capture the meaning preservation from source to target languages.


Thus, the interpretation of, e.g., the AUX-VERB or DET-PRON distinctions might differ across treebanks.

we ignore these differences in our analysis and consider all treebanks to be equally compliant.


Furthermore, while words tend to be hard to align in a cross-lingual setting due to homonymy and polysemy, morpho-syntactic information tends to be much more robust to language barrier (depending on typological closeness), which make them particularly relevant for cross-lingual transfer.

with delexicalized parsing approaches (Mc-Donald et al., 2011), the proposed method uses delexicalization during word embedding learning, not during parsing.


[Clubbing and putting up eyes] P 1 , [it is not violent and it does respect human rights] P 2 !!!

#irony) IMPLICIT activation arises from a contradiction between a lexicalized proposition P 1 describing an event or state and a pragmatic context P 2 external to the utterance in which P 1 is false, not likely to happen or contrary to the writer's intention.


Note that inferring irony in both types of activation requires some pragmatic knowledge.

in case of IMPLICIT, the activation of irony happens only if the reader knows the context.


Sentiment Analysis is a broad task that involves the analysis of various aspect of the natural language text.

most of the approaches in the state of the art usually investigate independently each aspect, i.e.


This constraint, in addition to the social media context, leads to a specific language rich of synthetic expressions that allow the users to express their ideas or what happens to them in a short but intense way.

the application of automatic sentiment classification approaches, in particular when dealing with noisy texts, is subjected to the presence of sufficiently manually annotated dataset to perform the training.


Along the same vein, Brown cluster assignments have also been used as a general purpose lexicon that requires no human manual annotation (Rutherford and Xue, 2014).

these solutions still suffer from the data sparsity problem and almost always require extensive feature selection to work well (Park and Cardie, 2012;Lin et al., 2009;Ji and Eisenstein, 2015).


Most previous research has focused on inducing and evaluating models from the English RST Discourse Treebank.

discourse treebanks for other languages exist, including Spanish, German, Basque, Dutch and Brazilian Portuguese.


Our system is similar to these last approaches in learning a representation using a neural network.

we found that good performance can already be obtained without using all the words in the discourse units, resulting in a parser that is faster and easier to adapt, as demonstrated in our multilingual experiments, see Section 7.


These results show that using all the words in the units (Ji and Eisenstein, 2014;Li et al., 2014), is not as useful as using more contextual information, that is taking more DUs into account (left and right children of the CDUs in the stack).

the slight drop for Relation shows that we probably miss some lexical information, or that we need to choose a more effective combination scheme than concatenation.


One possible explanation is that the Portuguese corpus is in fact a mix of different corpora, with varied domains, and possibly changes in annotation choices.

the low results for German show the sparsity issue since it is the language for which we have the fewest annotations ("#CDU", see Table 1).


This leads to a large drop in performance associated with this relation, when one of these corpora is not in the training data, especially for the source-only system for the En-DT (from 93% in F 1 to 30%).

on the En-DT, we observe improvement for other relations either largely represented in all the corpora (e.g.


The authors generate 5 artificial tasks of dialog.

the reasoning capabilities are not explicitly addressed and the author explicitly claim that the resulting dialog system is not satisfactory yet.


(2014) implement a deep learning architecture and report an 0.810 F-score and 35.9 NIST-SU error rate on broadcast news speech using prosodic and lexical features using a DNN for prosodic features, combined with a CRF classifier.

scaling this to spontaneous speech and the challenges of incrementality explained here, is yet to be tested.


For disfluency detection, standard approaches use pre-segmented utterances to evaluate performance, so this result is difficult to compare.

in the simple task, the accuracy of 0.720 repair onset prediction is respectable (comparable to (Georgila, 2009)), and is useful enough to allow realistic relative repair rates, in line with our motivation.


Utterance boundaries were detected just over a second after the end of the last word of the previous utterance.

the fact that T T D uttSeg on the word level reaches 0.283 suggests the timebased average is being weighed down by occa-sional long silences, which could be thresholded in future work.


The original Mor-phoChains system learns to identify child-parent pairs of morphologically related words, where the child (e.g., stopping) is formed from the parent (stop) by adding an affix and possibly a spelling transformation (both represented as features in the model).

these spelling transformations are never used to output underlying morphemes, instead the system just returns a segmentation by post-processing the inferred child-parent pairs.


2 The system could in principle learn that bake is the parent of baked with type suffix, which would imply the analysis bake +d.

we hope it learns instead the type delete, which implies the (correct) analysis bake +ed.


We also experiment with a larger number of topics, to see if we can profit from a finer grained topic defini-tion.

this advantage will be offset by a smaller training set since we split into more sets.


This shows that the finer grained splits model important information.

the topic expert model does not reach the accuracy of the baseline using the full training  Table 3: Results of the dependency parsing experiments using gold POS tags.


The gain in UAS is considerably smaller: The topic modeling expert reaches 90.55% as opposed to 90.26% for the full baseline.

the topic modeling setting for the 10topic setting outperforms the random baseline but does not reach the full baseline, thus mirroring the trends we have seen before.


the shared tasks on morphologically rich languages (Seddah et al., 2013;Seddah et al., 2014).

the comparison of results achieved for different languages is not straightforward as most languages and databases apply a unique tagset, moreover, they were annotated following different guidelines.


When adapting the universal dependency labels to Hungarian, we could find a one-to-one correspondence between the original labels of the Szeged Treebank and the UD labels only in most of the cases, and these labels could be automatically converted to the UD format, making use of the dependency and morphological annotations found in the original treebank.

we encountered some problematic cases during conversion, which we will discuss below in detail.


According to the UD principles, the first token of the multiword expressions should be marked as the head.

in Hungarian, it is always the last element of the multiword expression that is inflected.


We believe that this level of accuracy is not sufficient for releasing the rest of the 80,000 sentences of the automatically converted Szeged Dependency Treebank.

some of the shortcomings of the automatic conversion could be corrected by exploiting annotation found in other versions of the Szeged Treebank.


Neural attention models have achieved great success in different NLP tasks.

they have not fulfilled their promise on the AMR parsing task due to the data sparsity issue.


(2015), which does not require a dependency parser and uses SHRG to formalize the string-tograph problem as a chart parsing task.

they still need a concept identification stage, while our model can learn the concepts and relations jointly.


In this paper, we haven't used any syntactic parser.

as shown in previous works (Flanigan et al., 2014;Wang et al., 2015b;Artzi et al., 2015;Pust et al., 2015), using dependency features helps improve the parsing performance significantly because of the linguistic similarity between the dependency tree and AMR structure.


they have not been as successful on AMR parsing due to the data sparsity issue.


There is only very recent work around generation of question answer pairs from knowledge graph (Seyler et al., 2015).

there are several works around question generation that have been proposed in past with different motivations.


A rigorous approach involves setting up a hypothesis testing scenario using the performance of the systems on query documents.

often the hypothesis testing approach needs to send a large number of document queries to the systems, which can be problematic.


Intuitively, maximising cumulative rewards eventually leads to the selection of the best arm since it is the optimal decision.

(Bubeck et al., 2009) gives a theoretical analysis that any strategies for optimising cumulative reward is suboptimal in identifying the best performing arm.


We can conclude that in various NLP tasks, characters have recently been introduced in several different manners.

the models investigated in related work are either not tested on a competitive baseline (Miyamoto and Cho, 2016) or do not perform better than our models (Kim et al., 2016).


The best result is achieved by adding the first 3 and the last 3 characters to the model ('both orders'), giving a perplexity of 85.69, 3.05%/1.87% relative improvement with respect to the w475/w650 baseline.

adding more characters in both orders causes a decrease in performance.


For Dutch on the other hand, adding some random noise to the word-level model gave small improvements.

the random models perform much worse than the CW models.


This ability was harnessed by Shen and Lee (2016) for DA classification.

they ignored the conversational dimension of the data, treating the utterances in a dialogue as separate instances -an assumption that results in loss of information.


As seen in Table 4, both models exhibit accuracy drops (and small increases in negative log-likelihood) on the Switchboard development set, but small accuracy increases (and negative log-likelihood drops) on the Switchboard training set -an indication of over-fitting.

as seen in Table 5, both models yield a negligible or no drop in accuracy on the MapTask development set, while both yield a drop in accuracy on the training set.


The models that employ a DA connection to compute the attention signal (HA-RNN, woUt-tRNN, woHid2Attn, woConvRNN) show a slight improvement in accuracy when using the correct DA as input, instead of the predicted DA.

wDA2DA shows large improvements when using the correct DA (3.5% on Switchboard and 6.8% on MapTask), becoming the best-performing model for both datasets.


Since all the slotvalue specific information is delexicalised, the encoded vector can be viewed as a distributed intent .

if a value cannot be delexicalised in the input, its ngram-like embeddings will all be padded with zeros.


If multiple matches are observed, the corresponding embeddings are summed.

if there is no match for a particular slot or value, the empty n-gram embeddings are padded with zeros.


To the best of our knowledge, this is the first end-to-end NNbased model that can conduct meaningful dialogues in a task-oriented application.


Their corpus, the Columbia Quoted Speech Corpus (CQSC), is the most wellknown corpus and was used by follow-up work.

a result of their Mechanical Turk-based labeling strategy was that this corpus contains many unannotated quotes (see Table 4).


Domain dependence is a well-studied topic for PropBank SRL.

to the best of our knowledge, there exists no analysis of the performance of modern FrameNet SRL systems when applied to data from new domains.


One could optimize a frameId system to work in the no-lexicon setting which does not rely on the lexicon knowledge at all.

in this setting the classification results are currently lower.


On the one hand, annotators achieved κ = 0.345 (z = 92.2, p < 0.0001) (fair agreement) 6 when choosing targets to be added or removed.

they achieved a similar score of κ = 0.341 (z = 77.7, p < 0.0001) (fair agreement) when annotating the sentiment of the resulting targets.


A derivation precisely describes how the grounded equation system was constructed from the word problem by the automatic solver.

the grounded equation systems and the solutions are less informative, as they do not explain which span of text aligns to the coefficients in the equations.


Annotating gold derivations from scratch for all problems is time consuming.

not all word problems require manual annotation -sometimes all numbers appearing in the equation system can be uniquely aligned to a textual number without ambiguity.


There are a number of examples where an LSTM paper reports results of a CNN paper for comparison, such as (Ling et al., 2015) (POS tagging for English) and (Gillick et al., 2016) (named entity recognition for English).

there is no direct comparison between CNN and LSTM based architectures in morphological tagging.


The output is generated from this input.

multi-source morphological reinflection, the task we introduce, is a generalization in which the model receives multiple form-tag pairs.


In high-resource languages, an electronic dictionary may have near-complete coverage of the lemmata of the language.

paradigm completion is especially crucial for neologisms and lowresource languages.


(2016)'s neural architecture for MT translates from any of N source languages to any of M target languages, using language specific encoders and decoders, but sharing one single attention-mechanism.

to our work, they obtain a single output for each input.


There are two phenomena that cause reentrancies in AMR: control, where a reentrant edge appears between siblings of a control verb, and co-reference, where multiple men-tions correspond to the same concept.

6 dependency trees do not have nodes with multiple parents.


These two metrics are strictly related to the concept score.

since named entity recognition is the focus of dedicated research, we believe it is important to define a metric that specifically assesses this problem.


(2013) use a probabilistic parsing and grounding model to understand natural language instructions and extend their knowledge base by asking questions.

unlike this work, they do not use semantic parsing to leverage the compositionality of language, and also use a fixed hand-coded policy for dialog.


The HIS model allows tracking probabilities of the potentially large number of hypotheses.

it is difficult to learn a policy over this large a state space in a reasonable number of dialogs.


This can be avoided by training with a simulated user agent.

such agents are not always realistic and their design requires parameters to be set ideally from existing conversation logs.


We believe the agent performing only parser learning performs much better than the agent performing only dialog learning due to the relatively high sample complexity of reinforcement learning algorithms in general, especially in the partially observable setting.

the parser changes considerably even from a small number of examples.


This aligner achieves a 90% F 1 score on hand aligned AMR-sentence pairs.

the ISI Aligner (Pourdamghani et al., 2014) presents a generative model to align AMR graphs to sentence strings.


So instead of aligning to a concept's word lexicon, sometimes a concept aligns to its parent node (head word).

the lexicon features dominate the alignment probability in our Figure 5: The AMR annotation of sentence "In the first 6 rounds of competition, Mingxia Fu and Bin Chi are occupying the first and third positions respectively" E-M calculation.


In previous work, order information of context words (relative position of words in the contexts) was generally ignored and objectives similar to the SkipGram (henceforth: SKIP) model were used to learn v(e).

the bag-of-word context is difficult to distinguish for pairs of types like (restaurant,food) and (author,book).


Embedding models that consider words as atomic units in the corpus, e.g., SKIP and SSKIP, are word-level.

embedding models that represent words with their character ngrams, e.g., fasttext (Bojanowski et al., 2016), are subword-level.


In contrast, the ELR embedding is based on an entity's contexts, which are often informative for each entity and can distinguish politicians from athletes.

not all entities have sufficiently many informative contexts in the corpus.


Some aspects of the proposed approachnamely, the propagation of the nodes' weights through the graph, which we metaphorically represent as the flow of a contrast medium across nodes (Section 3.3) -are somewhat similar in spirit to spreading activation (Collins and Loftus, 1975) and random walks on graphs (Lovász, 1993) approaches.

in contrast to spreading activation approaches we leverage the graph directionality in order to reach all the possible nodes within the same connected components.


The key ideas behind ContrastMedium are: • Identification of important topological clues from the companion knowledge base KB in order to hierarchically sort the concepts in G. For our purposes, KB is expected to be able to provide ground-truth taxonomic relations that can be safely projected onto G to guide the cleaning process: that is, we assume it to be reasonably clean.

we do not make any assumption on how KB has been created: our approach can be used with either manually created taxonomies like WordNet or (semi-)automatically induced ones, provided they are of sufficient quality.


Concurrent to our work, Weissenborn (2016) proposes a row-less method for relation extraction considering both a uniform and weighted average aggregation function over columns.

weissenborn (2016) did not experiment with max and max-pool aggregation functions or evaluate on entity-type prediction.


(2016) treat database records and output texts as sequences, and use recurrent neural networks to encode and decode them.

our input is a set of discrete attributes instead of database records or sequences.


(2016) use variational auto-encoders to generate face images conditioned on visual attributes.

our goal is to generate texts instead of images.


In our case, the input is a shallow graph instead of a syntactic tree, and hence the search space is larger.

the same set of actions can still be applied, with additional constraints on valid actions given each configuration (Section 3.2.3).


The content selection discussed here is analogous to the selection of semantic attributes (type, color, size, etc) when generating a description of an entity (Dale and Haddock, 1991;Dale and Reiter, 1995).

instead of attributes, the content selection step in our model aims to choose the form of a proper name reference (which kind(s) of name and modifier(s) are part of the proper name reference).


Our model as presented does not make this assumption (it does not always produce a proper name reference that distinguishes the target from the distractors).

this could be incorporated into our model as well.


Therefore, a large number of dynamic oracles have been developed in recent years (Goldberg and Nivre, 2012; Goldberg and Nivre, 2013;Goldberg et al., 2014;Gomez-Rodriguez et al., 2014;Björkelund and Nivre, 2015).

the reinforcement learning approach proposed in this paper is more general and can be applied to a variety of systems.


We have shown that noisy-context surprisal derives information locality, and argued that dependency locality can be seen as a special case of information locality.

deriving dependency locality requires a crucial assumption that words linked in a dependency have higher mutual information than those words that are not.


(2015) study the characteristics of linguistic relations that may signal entailment or contradiction at subsentential level.

they ignore the logical context in which these linguistic relations occur in the entailment problem.


If all sub-goals are proved, then the theorem is proved and the entailment judgement can be produced.

there are theorems for which not all subgoals can be proved.


Henderson and Popa (2016) learn a mapping from an existing distributional vector representation to an entailment-based vector representation that expresses whether information is known or unknown.

they only evaluate on lexical semantic tasks such as hyponymy detection.


Both papers evaluate their models only on lexical relationships.

to traditional distributional similarities, Young et al.


The participants were requested to answer the PVQ questionnaire and to provide their Twitter IDs, so that their tweets could be crawled.

several challenges have to be addressed when working with Twitter, and a number of iterations, human interventions and personal communications were necessary.


(2013), speech act features were applied in order to classify personalities/values.

for this experiment the speech act classes were restricted to 11 major categories: Statement Non-Opinion (SNO), Wh  Table 5: Speech act class distributions in the corpus (in %) and speech act classifier performance Question (Wh), Yes-No Question (YN), Statement Opinion (SO), Action Directive (AD), Yes Answers (YA), Thanking (T), Appreciation (AP), Response Acknowledgement (RA), Apology (A) and others (O), hence avoiding having 43 fine-grained speech act classes.


Our results so far show that our arguments changed people's beliefs as a function of their prior beliefs and argument type.

we aim to automatically predict belief change, and hypothesize that knowing a person's personality in combination with their prior beliefs will allow us to select social-media arguments that are more persuasive for a particular individual.


This supports the results of our prior ANOVA testing over all subjects for belief change, and shows that the argument itself partially predicts belief change.

more interestingly, Table 7 also shows that providing the learner with information about personality consistently improves the ability of the learner to predict belief change.


While our results for balanced monologs suggest that summaries increase belief change, summary tools for such arguments are still under development .

perhaps high quality summaries may not be needed if compelling argument fragments can be automatically extracted (Misra et al., 2016b;Subba and Di Eugenio, 2007;Nguyen and Litman, 2015;Swanson et al., 2015).


In this model, a Multilayer Perceptron (MLP) takes as input a number of carefully hand-crafted syntactic and social behavioural features from each user and attempts to predict a label for each of the 5 personality traits.

the authors reported neither evaluation of this work, nor details of the dataset.


Also, the focus is on the prediction of trait scores on the author level based on modelling all available text from a user.

not only does our approach infer the personality of a user given a collection of short texts, it is also flexible enough to predict trait scores from a single short text, arguably a more challenging task considering the limited amount of information.


This is perhaps not the venue to consider the implications of this further, although one explanation might be that the model has uncovered a flexibility often associated with Ambiverts (Grant, 2013).

it is worth noting that the model is capable of capturing, without feature engineering, well-understood dimensions of language.


Our results suggest that apart from faster algorithms and more compact representations, recent cross-lingual word embedding algorithms are still unable to outperform the traditional methods by a significant margin.

introducing our new multi-lingual signal considerably improves performance.


If linguistic variability and noise only occur on the token level, then a tokenization-free approach has fewer advantages.

the foregoing discussion of cross-token regularities and disambiguation applies to well-edited English text as much as it does to other languages and other text-types as the example of "exchange" shows (which is dis-ambiguated by prior context and provides disambiguating context to following words) and as is also exemplified by lines 5-9 in Table 2 (right).


It is not clear to what extent blurring on the byte level is useful; e.g., if we blur the bytes of the word "university" individually, then it is unlikely that the noise generated is helpful in, say, providing good training examples in parts of the space that would otherwise be unexplored.

the text representation we have introduced in this paper can be blurred in a way that is analogous to images and speech.


We have focused on whitespace tokenization and proposed a whitespacetokenization-free method that computes embeddings of higher quality than tokenization-based methods.

there are many properties of edited text beyond whitespace tokenization that a complex rule-based tokenizer exploits.


This is true for the training of the model as well as for applying it when computing the representation of a new text.

to prior work that has assumed that the sequence-of-character information captured by character ngrams is sufficient, position embeddings also capture sequence-of-ngram information.


All three sentences S1, S2, and S3 mention same entity Barack Obama.

looking at the context, we can infer that S1 mentions Obama as a person/author, S2 mentions Obama only as a person, and S3 mentions Obama as a person/politician.


tion mechanism to allow the model to focus on relevant expressions in the entity mention's context.

the model assumed that all labels obtained via distant supervision are correct.


For noisy entity mentions, we propose a variant of a hinge loss where, like L c , score for all negative labels should go below −1.

for positive labels, as we don't know which labels are relevant to entity mention's local context, we propose that the maximum score from the set of given positive labels should be greater than one.


This indicates usefulness of the learnt feature representations.

if we repeat the same process with OntoNotes dataset, there is only a subtle change in performance.


Performance of AFET significantly drops (AFET-NoCo) when data-driven label-label correlation is ignored, which indicates that modeling data-driven correlation helps.

as shown in Figure 2a, the use of label-label correlation depends on appropriate values of parameters which vary from one dataset to another.


The strength of association between each named entity y and date d is measured based on the number of co-occurring tweets in order to form a binary tuple y, d to represent an event.

twiCal relies on a supervised sequence labeler trained on tweets annotated with event mentions for the identification of eventrelated phrases.


On Dataset II, both DPEMM and DPEMM-WE achieve better clustering results compared to LEM.

the purity of clusters generated by DPEMM is slightly higher than that generated by DPEMM-WE.


It can be observed that the purity of the cluster generated by DPEMM is 91% which is better than DPEMM-WE's 63%.

the size of the cluster returned by DPEMM is smaller and it failed to extract the location information.


boundaries as well as types) in it are known.

end-to-end relation extraction deals with plain sentences without assuming any knowledge of entity mentions in them.


As the same parameters are shared for all entity as well as relation type predictions, we expect the model to learn dependencies among relation and entity types.

as it makes separate predictions for each word pair, there might be some inconsistencies among the labels as described above.


Current global inference approaches optimize a single coherence measure, most commonly a measure of general semantic relatedness such as the Milne-Witten distance (Milne and Witten, 2008), or keyphrase overlap relatedness (KORE) (Hoffart et al., 2012).

verification allows employing many global coherence features, which we categorize according to four aspects of coherence: geographical coherence and temporal coherence, which to our knowledge have not been used before in EL, as well as entity type coherence and the general semantic relatedness mentioned above.


We have an odds ratio probability term "OR = 1.41" and two variables.

determining the boundaries of the variables is not straightforward.


We start with a linear-chain CRF: Algorithm 1 Dependence Extraction Constraints if influence term > 0 then ensure at least one A and one B label else if A > 0 then ensure at least one B label else if B > 0 then ensure at least one A label end if where T is the number of observations indexed by t, k indexes the feature function f k and weight λ k , and Z x normalizes over the entire input sequence, The CRF performs satisfactorily leading to higher precision classification of the labels than recall.

it is not able to capture some information that we know to be true.


The constraints applied for the Default constraint experiment assign the (correct) variable B and (incorrect) variable A.

the classifier in the Par-allel experiment mistakenly leads us not to label any entities in this sentence, thereby missing the relation.


However, as would be expected, the precision goes up as p 1 and p 2 increase, i.e., as we become more conservative in constraining the CRF output.

recall drops with higher values of the p threshold as the CRF is not pushed to find previously missed variables A or B.


However, achieving similar levels of performance on small real world datasets has proved difficult; major challenges stem from the large vocabulary size, complex grammar, and the frequent ambiguities in linguistic structure.

the requirement of human generated annotations for training, in order to ensure a sufficiently diverse set of questions is prohibitively expensive.


With the advent of deep learning, the state of art performance for various semantic NLP tasks has seen a significant boost (Collobert and Weston, 2008).

most of these techniques are data-hungry, and require a large number of sufficiently diverse labeled training samples, e.g., for QA, training samples should not only encompass an entire range of possible questions but also have them in sufficient quantity (Bordes et al., 2015).


An inspiration from the ripples created by the success of pre-training and as well as word2vec, this paper explores pre-training to utilize data from a related domain and also pre-trained vectors from word2vec tool (Mikolov et al., 2013).

finding an optimal dimension for these pre-trained vectors and other involved hyper-parameters requires computationally extensive experiments.


Such a training procedure allows the learner to waste less time with noisy or hard to predict data when the model is not ready to incorporate such samples.

what remains unanswered and is left as a matter of further exploration is how to devise an effective strategy for a given task?


Unlike SemEval-2007, the SemEval-2012 task is concerned exclusively with ranking substitutes; all the original participating systems were given the gold-standard substitutes and simply asked to put them in the correct order.

to score our own systems we use their own substitute lists, removing only those substitutes that do not also appear in the goldstandard list.


(2015) adapted for multiple languages also using bilingual corpora.

parallel data is an expensive resource and using parallel data seems to under-perform on the bilingual lexicon induction task (Vulić and Moens, 2015).


We also adopted this approach for multilingual environment by applying multi-view CCA to map the English part of each pre-trained bilingual word embedding to the same space.


As we keep adding more languages to the model, the hidden layer in our model -shared between all languages -might not be enough to accommodate all languages.

we can combine the strength of the linear transformation proposed in §3 to our joint model as described in Equation (3).


Most of the popular methods are based on a distributional approach: the meaning of a word is defined by the context of its use, i.e., the neighboring words.

distributional representations carry no explicit linguistic information and cannot easily represent some important semantic relationships, such as synonymy and antonymy (Nguyen et al., 2016).


(2008a), which is also used for building a semantic representation (Zesch et al., 2008b).

the level of detail and structure format obtained by such method was not deemed adequate for this work and an alternative extraction method was developed (Sections 3.2 and 3.3).


If the book is long, as the reading progresses, the guesses tend to become more accurate, as a human will try to piece together the information patterns surrounding the new words.

the definitional approach would be equivalent to reading the entire contents of a dictionary before reading the book.


It also outperforms the most popular distributional representations.

they are clearly outclassed in the semantic relatedness test, for which the distributional approaches show superior performance.


Most languages have no established writing system and minimal written records.

textual data is essential for natural language processing, and particularly important for training language models to support speech recognition.


Neural network approaches to a range of NLP problems have also been aided by initialization with word embeddings trained on large amounts of unannotated text (Frome et al., 2013;Zhang et al., 2014;Lau and Baldwin, 2016).

in the case of extremely low-resource languages we do not have the luxury of this unannotated text.


This resilience of the target English word embeddings suggests that CLWEs can serve as a method of transferring semantic information from resource-rich languages to the resource-poor, even when the languages are quite different.

the WordSim353 task is a constrained environment, so in the next section we turn to language modeling, a natural language processing task of much practical importance for resource-poor languages.


In every case where pre-trained embeddings were used, the embedding layer was held fixed during training.

we observed similar results when allowing them to deviate from their initial state.


The language models initialized with pretrained CLWEs are significantly better than their un-pre-trained counterpart on small amounts of data, reaching par performance with MKN at somewhere just past 4k sentences of training data.

it takes more than 16k sentences of training data before the plain LSTM language model began to outperform MKN.


For the Na-English lexicon, this was only 18% and 20% when lemmatized and unlemmatized, respectively.


Post-editing, the easiest one to implement, has little consideration for the words surrounding the nouns, while re-ranking works on MT hypotheses and thus ensures that a better global translation is found that is also consistent.

in some cases, no hypothesis conforms to the consistency decision, and in this case post-editing the best hypothesis appears to be beneficial.


As in Chinese semantic features are less helpful, given also the limited amount of data, combining them with syntactic ones actually decreases the performance of the syntactic ones used independently.

semantic features are more helpful on German dataset, and also improve results when we considered along with the syntactic ones together.


The analysis shows that lexical features are significantly more important than purely syntactic ones, for both languages.


Much of psycholinguistics has focused for many years on processing measures that provide difficulty estimates on a word-byword basis.

these psycholinguistic measures have not yet been tested on sentence readability ranking tasks.


Previous work on readability has classified or ranked texts based on document-level measures such as word length, sentence length, number of different phrasal categories & parse tree depth (Petersen, 2007), and discourse coherence (Graesser et al., 2004), inter alia.

not all applications that need readability ratings deal with long documents.


The cataloging of product listings through taxonomy categorization is a fundamental problem for any e-commerce marketplace, with applications ranging from personalized search recommendations to query understanding.

manual and rule based approaches to categorization are not scalable.


Then, if the specific items cannot be found in the inventory, other relevant items in the "Jeans" category are returned in the search results to encourage the user to browse further.

achieving good product categorization for e-commerce market-places is challenging.


), but with more pronounced data quality issues.

the existing methods for noisy product classification have only been applied to English.


For instance, existing approaches classify text units as argumentative or non-argumentative (Moens et al., 2007), recognize argument components such as claims or premises at the sentence-level (Mochales-Palau and Moens, 2009;Kwon et al., 2007;Eckle-Kohler et al., 2015) or clause-level Sardianos et al., 2015), or identify argument structures by classifying pairs of argument components .

these approaches are of limited use for argumentative writing support systems since they do not recognize the weak points of arguments.


The argument is a generalization from one sample to the general case.

a single sample is not enough to support the general case.


Approaches for identifying the structure of arguments recognize argumentative relations between argument components using context-free grammars (Mochales-Palau and Moens, 2009), pair classification , or maximum spanning trees (Peldszus and Stede, 2015).

none of these approaches consider the quality of arguments.


Cano-Basave and He (2016) ranked speakers in political debates by using semantic frames which indicate persuasive argumentation features, and Habernal and Gurevych (2016b) compared the convincingness of argument pairs using feature-rich SVMs and bidirectional LSTMs.

the persuasiveness score of an argument is only of limited use for argumentative writing support, since it summarizes various quality criteria and does not explain why an argument is weak.


Thus, the model is less suitable for exhaustively finding all insufficiently supported arguments.

the CNN model is more balanced with respect to precision and recall and considerably outperforms the recall of the SVM model.


Using the length features individually yields only a slight improvement of the macro F1 score over the majority baseline.

removing the length from the entire feature set causes a slight decrease of .002 in the macro F1 score compared to the system which uses all features.


Most of these arguments either refer to an example in addition to other premises which are already sufficient to support the claim or include an example for specifying another premise.

we also found several false negatives which include examples as evidence.


Using the augmented training set, consisting of both the original documents and the additional pseudodocuments, we can then employ any existing PV model (Le and Mikolov, 2014;Dai et al., 2015) to learn the document-phrase co-embeddings.

due to the significant differences in the sizes and numbers of documents and pseudodocuments, there is a danger that the addition of the pseudo-documents can have a detrimental effect on the performance of the model.


Most research efforts in this task have focused on English texts, much less attention has been given to other languages (Abdul-Mageed et al., 2011;Kapukaranov and Nakov, 2015).

spanish is the third language most used on the internet, with a total of 7.7% (more than 277 million of users) and a huge internet growth of more than 1,400%.


In addition, we introduce a novel type of LM using a modified version of the standard bidirectional LSTM called contextual bidirectional LSTM (cBLSTM).

to the unidirectional model, this model is trained to predict a word depending on its full left and right contexts.


More importantly, in (Peris and Casacuberta, 2015), bidirectional RNN LMs are used for a statistical machine translation task.

only standard RNNs but not LSTMs are utilized.


The pattern based methods (both lexical patterns and syntactic patterns) are simple, fast, and scalable on large-scale datasets.

the robustness of patterns is usually questionable in practice.


Lexical patterns either have limited coverage (e.g., fixed set of patterns (Riloff and Wiebe, 2003)), or hard-tocontrol noise (e.g., bootstrapping approaches (Qiu et al., 2011)).

supervised models can achieve better performances than patterns on manually labeled datasets, but it is often difficult to obtain large number of annotations for the relation extraction task, and the trained models are also limited to specified domains.


Because this update potentially involves concurrent reads and writes at the same memory location, we use an atomic max operation (defined as atomicMax on the NVIDIA toolkit).

atomicMax is not defined for floating-point values.


Very recently, there have been a few efforts to apply NMT to simultaneous translation either through heuristic modifications to the decoding process (Cho and Esipova, 2016), or through the training of an independent segmentation network that chooses when to perform output using a standard NMT model (Satija and Pineau, 2016).

the former model lacks a capability to learn the appropriate timing with which to perform translation, and the latter model uses a standard NMT model as-is, lacking a holistic design of the modeling and learning within the simultaneous MT context.


are trained independently and combined in a loglinear scheme in which each model is assigned a different weight by a tuning algorithm.

in NMT all the components are jointly trained to maximise translation quality.


They showed that gender differences captured by shallow syntactic features were preserved across languages, when examined by linguistic categories.

they did not study the distribution of individual gender markers across domains and languages.


Data One of the main advantages of automatic BLI systems is their portability to different languages and domains.

current standard BLI evaluation protocols still rely on generaldomain data and test sets (Mikolov et al., 2013a;Gouws et al., 2015;Lazaridou et al., 2015;Vulić and Moens, 2016, inter alia).


Sophisticated deep learning algorithms can also be applied to text clustering (Xu et al., 2015), but to date they require labeled training data, while the method proposed in this paper is unsupervised.

to bag-of-words (BOW) schemes, named entities (NEs) can be used as features (Montalvo et al., 2012).


The dominant approach for many NLP tasks are recurrent neural networks, in particular LSTMs, and convolutional neural networks.

these architectures are rather shallow in comparison to the deep convolutional networks which have pushed the state-of-the-art in computer vision.


These word embeddings are now the state-of-the-art in NLP.

it is less clear how we should best represent a sequence of words, e.g.


We obtain state-of-the-art results for all data sets, except AG's news and Sogou news which are the smallest ones.

with our very deep architecture, we get closer to the stateof-the-art which are ngrams TF-IDF for these data sets and significantly surpass convolutional models presented in .


An explanation may be that similarity rewards redundancy, which is why, e.g., all four similarity approaches falsely ranked the redundancy-free argument a 1 in Table 1 lowest.

this requires further investigation, including an analysis of more sophisticated similarity measures.


Note that in addition to the experiments reported in this table, we attempted to combine different features sets.

we did not observe   Aiming to identify potential predictors for the remaining MI codes, we conduct a set of experiments where we use our linguistic feature sets to build multiclass classifiers.


It should be mentioned that the baseline character n-gram models in most cases are more effective than baseline token n-gram models.

in average, they are worse than token n-grams due to their poor performance when the Society texts are used for training.


Note that the results for CCAT-10 are directly comparable to Table 3.

for the Guardian corpus we present the average performance of all possible 12 combinations (using one thematic category as training corpus and another thematic category as test corpus).


It is also notable that, in both corpora, the differences between DV-MA and DV-SA are not significant.

dV-MA is slightly better than dV-SA in most of the cases.


The baseline model is better only in the case of the most challenging cross-genre PAN15-DU corpus.

its performance essentially resembles random guessing (0.5).


Experimental results demonstrated a considerable gain in effectiveness when using the proposed models under the realistic cross-topic conditions in both closed-set attribution and author verification tasks.

when the corpora are too topicspecific where the texts by a given author are consistently on certain subjects different than the ones of the other candidate authors, the distortion methods seem not to be helpful.


In the I2B2 Temporal Challenge eight types of relations were initially annotated.

due to low inter-annotator agreement these were merged to three types of temporal relations, OVERLAP, BEFORE, and AFTER.


Some of the popular features used in the existing studies include linguistic features such as morphological, syntactic and semantic information of words and domain-specific features from biomedical ontologies such as Bio-Thesaurus (Liu et al., 2006) and UMLS (Unified Medical Language System) (Bodenreider, 2004).

these features heavenly account to the problem of data sparsity.


We introduce MIML algorithms for neural networks.

to most MI/MIML methods, which are applied in relation extraction, we apply MIML to the task of finegrained entity typing.


probability for e, we define: where w t ∈ R h is the output weight vector and b t the bias scalar for type t, and a t is the aggregated representation of all contexts c i of e for type t, computed as follows: where α i,t is the attention score of context c i for type t and a t ∈ R h can be interpreted as the representation of entity e for type t. α i,t is defined as: where M ∈ R h×dt is a weight matrix that measures the similarity of c and t. t ∈ R dt is the representation of type t. Table 1 summarizes the differences of our MIML methods with respect to the aggregation function they use to get corpus-level probabilities.

for optimization of all MIML methods, we use the binary cross entropy loss function, to the loss function of distant supervision in Eq.


The parameters of the PMI framework are statistically estimated using a large amount (more than 1 000 000 word pairs) of cross-linguistically diverse data.

lexStat's initial alignment algorithm is based on manually assigned parameters, and the final parameters are estimated empirically from the word pairs in the doculects being compared, and no external information is being applied.


Hauer and Kondrak (2011) were the first to introduce this measure to test the accuracy of multilingual cognate detection algorithms.

to pair scores such as ARI, B-Cubed scores have the advantage of being independent of the evaluation data itself.


Their definition of success was a function of the number of downloads from Project Gutenberg.

downloading a book is not by itself an indicator of a highly liked or a commercially successful book.


However, the results in Table 3 show an unimpressive F1-score of 0.610 for sentiment features.

the bag of sentic concepts model with average scores for sensitivity, attention, pleasantness, aptitude, and polarity gave a more impressive F1-score of 0.670, much higher than the baseline.


Table 8 shows a breakdown of fragment types in the first fold.

with n-grams, we also see   a large proportion of purely syntactic fragments, and fragments mixing both lexical elements and substitution sites.


On the one hand smaller fragments corresponding to one or two grammar productions are most common, and are predominantly positively correlated with the  literary ratings.

there is a significant negative correlation between fragment size and literary ratings (r "´0.2, p ă 0.001); i.e., smaller fragments tend to be positively correlated with the literary ratings.


In the first large-scale study of sociolinguistic variation on social media in the UK, we identify distinctively Scottish terms in a data-driven way, and find that these terms are indeed used at a higher rate by users of pro-independence hashtags than by users of anti-independence hashtags.

we also find that in general people are less likely to use distinctively Scottish words in tweets with referendum-related hashtags than in their general Twitter activity.


While some people still use distinctive elements of Scots in their speech, until recently the average Scottish person's exposure to written Scots would have been largely confined to a select few literary domains such as poetry and comic narrative (Corbett et al., 2003).

social media has given rise to a new genre of casual, communicative writing that is potentially visible to large and diverse audiences, providing both a platform and an impetus to express one's identity through the use of written language.


Perhaps other tweets with many Scottish terms were filtered out, in which case we will underestimate the probability that users choose Scottish variants.

this issue should not cause us to find differences in use between different groups where there are none.


Due to the very low rates of Scottish variants overall, our data set is too small to study differences between individual variables or even conclusively say whether there may be effects of both topic and audience size on the use of Scottish language.

we hope to be able to answer these questions in future by collecting a more complete set of data for the particular users studied here.


In a similar work, Sangati and Zuidema (2009) proposed entropy minimization and greedy familiarity maximization techniques to obtain lexical heads from labeled phrase-structure trees in an unsupervised manner.

we used neural attention to obtain the "head rules" in the GA-RNNG; the whole model is trained end-to-end to maximize the log probability of the correct action given the history.


From an empirical point of view, a transition system that over-generates is necessary for robustness, and is desirable for fast approximate linear-time inference.

from a formal point of view, the relationship of the SR-GAP transition system with automata explicitly designed for LCFRS parsing (Villemonte de La Clergerie, 2002; Kallmeyer and Maier, 2015) requires further investigations.


To overcome this, one can use a tree-structured stack (TSS) to factorize the representations of common prefixes in the stack as described by  for projective dependency parsing.

the discontinuites entail that a limited amount of copying cannot be entirely avoided.


It would be interesting to see if SR-SWAP is improved when swapping non-terminals is allowed.

it would make feature extraction more complex, because it would no longer be assumed that the buffer contains only tokens.


Neural models, with various neural architectures (Bengio et al., 2001;Mikolov et al., 2010;Chelba et al., 2014;Józefowicz et al., 2016), have recently achieved great success.

most of these neural architectures have a common issue: large output vocabularies cause a computational bottleneck due to the output normalization.


For the three noise distributions, the NCE score seems to converge.

for the unigram distribution, the log-partition function does not decrease, thus neither does the log-likelihood.


We suspect this is possible because a community's language in social media may capture economic-related community characteristics that are not otherwise easily available.

the challenge is incorporating noisy high-dimensional language features in such a way that they can contribute beyond the robust low-dimensional traditional predictors (i.e.


It shows that our approach correctly simplifies the largest number of problems, while making the fewest errors of type 3A and 4.

it can be noticed that NNLS makes many errors of type 5.


There have been recent advances in supervised summarisation mainly with respect to supervised learning using neural networks (for example (Rush et al., 2015;Chopra et al., 2016)).

due to data size requirements, these systems are constrained to title generation systems and therefore not in the scope of this work.


Evaluation against multiple, rather than single reference summaries is generally recognised as leading to fairer, better quality, evaluation: different human summaries appear to be good even though they do not have identical content (Nenkova and Passonneau, 2004).

averaging ROUGE scores across multiple summaries, as is standard practice, makes a perfect 100% score unattainable, even for abstractive systems.


Since the last step in Pipeline is to run PredPatt, Pipeline generates no malformed output.

15% of its 4 In linearized predicates, arguments are replaced by placeholders.


Both architectures use a nonlinearity to modulate the contents of a product; in an RAE this is an outer (bilinear) product while in an LSTM it is a Hadamard (element-wise) product.

lSTM memory gates represent an internal hidden state of the network, while RAE gates are part of the network input.


The model architecture is the same as an RAE with untied encoder and decoder weights (Eq 2).

the training signal differs from a classic RAE in two ways.


Distributional semantics (Turney and Pantel, 2010), and data-driven, continuous approaches to language in general including neural networks (Bengio et al., 2003), are a success story in both Computational Linguistics and Cognitive Science in terms of modeling conceptual knowledge, such as the fact that cats are animals (Baroni et al., 2012), similar to dogs (Landauer and Dumais, 1997), and shed fur (Erk et al., 2010).

distributional representations are notoriously bad at handling discrete knowledge (Fodor and Lepore, 1999;Smolensky, 1990), such as information about specific instances.


's (2014) results that DDSq works best for hypernymy detection in the NOTHYP setup.

for instantiation detection it is the concatenation of the input vectors that works best (cf.


This indicates that geospatial information does provide some useful semantic information.

the GEOD embeddings underperformed the TEXT5 embeddings in all cases.


This means that if an emoji is frequent, it is more likely to be on top of the possible choices even if it is a mistake.

the F-measure does not seem to depend on frequency, as the highest F-measures are scored by a mix of common and uncommon emojis ( , , , and ) which are respectively the first, second, the sixth and the second last emoji in terms of frequencies.


English is broadly considered to be a morphologically impoverished language, and there are certainly many regularities in morphological patterns, e.g., the common usage of -able to transform a verb into an adjective, or -ly to form an adverb from an adjective.

there is considerable subtlety in English derivational morphology, in the form of: (a) idiosyncratic derivations; e.g.


information-theoretic constraints, this should be reflected in production preferences and therefore in corpora of text types which allow for the respective omissions.

accurately finding all instances of article omission is not a trivial issue, as there are several special cases of singular nouns which allow for or even require article omission even in standard written German, e.g.


(2006), there are no bounds for LSP at all.

for CR, as it can be seen from our experiments, values of T for LSP and LSPA can be reliably selected on a validation set for a fixed training data size and a choice of features/instances.


As we show, tying the input and the output embeddings is indeed detrimental in word2vec.


In NLP, multi-task learning typically involves very heterogeneous tasks.

while great improvements have been reported (Luong et al., 2016;Klerke et al., 2016), results are also often mixed (Collobert and Weston, 2008;Martínez Alonso and Plank, 2017), and theoretical guarantees no longer apply.


As discussed in Section 1, there have been studies to reduce the time complexity of spell correction by various methods.

the recent work of de Amorim and Zampieri (2013) is closest to our work in terms of the goal of the study.


We do indeed regularize in this way throughout all our experiments, tuning the multiplier on a held-out development set.

regularization has only minor effects with large training corpora, and is not in the original word2vec implementation of skip-gram.


Having seen that skip-gram is a form of matrix factorization, we can generalize it to tensors.

to the matrix case, there are several distinct definitions of tensor factorization (Kolda and Bader, 2009).


Hence, all of the possible replies to X must be stored within the decoder's probability distribution P (Y |X), and during decoding it is hard to disentangle these possible replies.

our model contains a stochastic component z in the decoder P (Y |z, X), and so by sampling different z and then performing ML decoding on P (Y |z, X), we hope to tease apart the replies stored in the probability distribution P (Y |X), without resorting to sampling from the decoder.


For this reason we cannot directly compare the scores.

we follow the steps to replicate the challenge's pipeline on our unaltered data and use the score as our baseline.


One of the issues in the area of affective computation is that the amount of annotated data is very limited.

the number of ways that the same emotion can be expressed verbally is enormous due to variability between speakers.


Pretraining convolutional neural network (Ebrahimi Kahou et al., 2015) on an external dataset of faces improves the performance of the emotion classification model.

the problem of augmenting emotional datasets with audio samples to improve the performance of solely audio processing models remained unsolved.


In our implementation the MLP computes a whole vector with values for each v k at once.

in this notation we use just the value corresponding to v j .


Marmot's edit-tree method of candidate selection favors fewer lemmas, which allows the lemmatization module to run efficiently.

dIRECTL+ has no bias towards lemmas or tags.


Both systems correctly identify it as a plural subjunctive form of the verb lacrar.

marmot also outputs an alternative analysis that involves a bizarre lemma lacr.


15 We additionally compared with WN-Domains-3.2 (Magnini and Cavaglià, 2000;Bentivogli et al., 2004), which is the latest released version of WordNet Domains 16 .

this approach involves manual curation, both in the selection of seeds and correction of errors.


With low-resource languages, we cannot presume the availability of tag dictionaries.

we often have highquality bilingual dictionaries with translations of common words into a resource-rich language such as English.


We see the evidence for that in the very wide confidence intervals in our Wiktionary sampling.

even the smallest of frequency-aware Wiktionaries prove to be much more reliable.


By doing this, we are transferring the semantics of LKIF to Wikipedia entities and populating the LKIF ontology with Wikipedia entities and their mentions.

even using Wikipedia, many of the classes have few instances.


However, we find that curriculum learning does not produce the expected improvements.

reversed curriculum learning, learning from most specific to most general, produces better results, and it helps to indicate that there may be incoherences in the mappings between ontologies.


In contrast, the NERC does a better job at distinguishing YAGO classes, which are natively built from Wikipedia, even if the classification problem is more difficult because of the bigger number of classes.

the fact that smaller classes are recognized better than bigger classes indicates that bigger classes are ill-delimited.


This is the case, for example, of the detection of the main components in scholarly publications (Teufel et al., 2009;Liakata et al., 2012;De Waard and Maat, 2012;Burns et al., 2016) or the annotation of content zones, i.e., functional constituents of texts (Bieler et al., 2007;Stede and Kuhn, 2009;Baiamonte et al., 2016).

the notion of Content Types that we have adopted applies across genres.


The proposed model ignores long-range dependencies that could conceivably be captured using alternative architectures, such as recurrent neural networks (RNN) (Mikolov et al., 2010;Luong et al., 2013).

topical and stylistic information is contained in shorter word and character sequences for which the shallow neural network architectures with n-gram feature representations are likely to be sufficient, while having the advantage of being much faster to run.


In the real estate domain, user-generated free text descriptions form a highly useful but unstructured representation of real estate properties.

there is an increasing need for people to find useful (structured) information from large sets of such descriptions, and for companies to propose sales/rentals that best fit the clients' needs, while keeping human reading effort limited.


For example, real estate descriptions in natural language may not be directly suited for specific search filters that potential buyers want to apply.

a hierarchical data structure representing the real estate property enables specialized filtering (e.g., based on the number of bedrooms, number of floors, or the requirement of having a bathroom with a toilet on the first floor), and is expected to also benefit related applications such as automated price prediction (Pace et al., 2000;Nagaraja et al., 2011).


The conditional distribution over all dependency structures y ∈ T (x) is: normalized by the partition function Z(x; θ), which would require a summation over the exponentially large number of all possible dependency structures in T (x).

the MTT allows directly computing Z(x; θ) as det(L(θ)), in which L(θ) is the Laplacian matrix of the graph.


As the results show, the generic PBMT system outperforms its NMT counterpart in all the domains by a very large margin; and as the NMT system becomes more specific by observing more domain-specific data, the gap between the performances reduces until the NMT outperforms; which confirms the results of the previous works in this field (Luong and Manning, 2015).

it is interesting to see what is the reason behind the very low performance of the generic NMT system compared to the generic PBMT.


Recently, (Junczys-Dowmunt et al., 2016) performed a very extensive experiment in which the performance of NMT is compared with PBMT and hierarchical SMT on multiple language directions and showed that NMT systems in almost all the cases outperform their SMT counterparts and to solve the only remaining issue which is the decoding time of the NMT systems, they introduce an efficient neural decoder which makes it feasible to deploy NMT systems in-production line.

all their experiments are performed on one single domain for which there exists a very large training corpus.


A related task to TLS is the TREC update summarization task (Aslam et al., 2015).

to TLS, this task requires online summarization by presenting the input as a stream of documents.


For r, we sum up all of the features of the input given by the encoder (Line 2) and estimate the frequency.

for g, we expect Lines 5 and 6 to work as a kind of voting for both positive and negative directions since g needs just occurrence information, not frequency.


Note that its model structure is nearly identical to our baseline.

mRT trained a model with a sequence-wise min- DUC-2004 (w/ 75-byte limit) Gigaword (w/o length limit) method    imum risk estimation, while we trained all the models in our experiments with standard (pointwise) log-likelihood maximization.


They also show that the task of PP attachment with multiple noun candidates is considerably more difficult than the traditional binary classification task.


• Multipass: In the Multipass method, we train two separate models like the Monolingual method.

we apply these models on the code-mixed data differently.


At inference time, our transliteration model would predict the most likely word form for each input word.

the single-best output from the model may not always be the best option considering an overall sentential context.


For the most part, the advances are due to general improvements in parsing technology or feature representation, and do not explicitly target any specific language or syntactic construction.

despite the high overall accuracy, parsers are still persistently wrong in attaching certain relations.


Lattice minimum Bayes-risk (LMBR) decoding has been applied successfully to translation lattices in traditional SMT to improve translation performance of a single system (Kumar and Byrne, 2004;Tromble et al., 2008;.

minimum Bayes-risk (MBR) decoding is also a very powerful framework for combining diverse systems (Sim et al., 2007;de Gispert et al., 2009).


We argue that MBR-based methods in their present form are not well-suited for NMT because of the following reasons: • Previous approaches work well with rich lattices and diverse hypotheses.

nMT decoding usually relies on beam search with a limited beam and thus produces very narrow lattices (Li and Jurafsky, 2016;Vijayakumar et al., 2016).


2 allows to build up hypotheses from left to right.


1 contains a sum over the (unordered) set of all n-grams.

we can rewrite our objective function in Eq.


Rescoring a 10-best list already yields a large improvement of 1.2 BLEU.

the hypotheses are still close to the SMT baseline.


Although attention is an intuitively appealing concept and has been proven in practice, existing models of attention use contentbased addressing and have made only limited use of the historical attention masks.

lessons from better word alignment priors in latent variable translation model suggests value for modeling attention dependent of content.


While other work has sought to address this problem (Cohn et al., 2016;Tu et al., 2016;Feng et al., 2016), these models either rely on explicitly engineered features (Cohn et al., 2016), resort to indirect modeling of the previous attention decisions as by looking at the content-based RNN states that generated them (Tu et al., 2016), or only model coverage rather than coverage together with ordering patterns (Feng et al., 2016).

we propose to model the sequences of attention levels for each word with an RNN, looking at a fixed window of previous alignment decisions.


One major limitation is that attention at each time step is not directly dependent of each other.

in machine translation, the next word to attend to highly depends on previous steps, neighboring words are more likely to be selected in next time step.


Recent work has mainly focused on morphologically complex rare words has often tried to alleviate the problem by spreading the available knowledge across words that share the same morpheme (Luong et al., 2013;Botha and Blunsom, 2014;Soricut and Och, 2015).

these techniques are unable to induce representations for words whose morphemes are not seen during training, such as infrequent domain specific terms.


Dependency-based vector spaces steer the induced WEs towards functional similarity (e.g., tiger:cat) rather than topical similarity/relatedness (e.g., tiger:jungle), They support a variety of similarity tasks in monolingual settings, typically outperforming BOW contexts for English (Bansal et al., 2014;Hill et al., 2015;Melamud et al., 2016).

despite the steadily growing landscape of CL WE models, each requiring a different form of cross-lingual supervision to induce a SCLVS, syntactic information is still typically discarded in the SCLVS learning process.


Plagiarism is a statement that someone copied text deliberately without attribution, while these methods only detect textual similarities.

textual similarity detection can be used to detect plagiarism.


By removing stopwords before training, we obtain a slightly higher M I(d, k) than removing stopwords after training, in support of hypothesis 1 in Section 2.

this difference is relatively small compared to including more stopwords or changing the number of topics.


Of these, the best results have been obtained using tree models (Mou et al., 2015;Tai et al., 2015), which use sentence syntactic trees originating from grammar to help construct sentence embeddings.

noisy text (such as found in online reviews) does not always contain much grammatical structure, which reduces the effectiveness of tree models.


Applying softmax to the attention scores indicated that the receiving neural network must focus on a certain part of the input.

this assumption might not hold in the GRA framework as phrase information is not always needed at each timestep of RNN training.


For instance, in softmax, each input is simply weighted by its contribution to the sum.

in GRA, the sum of similarity scores is not a good scaling factor since phrase vectors are sometimes omitted.


We suppose that the drop of accuracy in the not-aligned version is a result of phrase vectors being over-counted with large weights, and thus reducing the effectiveness of the sequence learning ability in RNN.

with soft-alignment, GRA can incorporate CNN phrase vectors into an RNN without impacting the sequence learning effectiveness.


Latent Semantic Analysis (Deerwester et al., 1990) and Latent Dirichlet Allocation (Blei et al., 2003) are the main employed methods for this task.

these methods do not systematically yield improved performance compared to the BOW representation.


Most hyperparameters in Table 1 are also shared by the training of DV-LSTM.

due to the limitation of the current experiment platform and the cost of grid searches, we do not tune these hyperparameters in training DV-LSTM.


Another plausible reason is that PTB's genres are almost unrelated to the topics and it likely requires more abstract sequential information for their classification.

the Brown Corpus has a relatively strong overlapping between topics and genres.


Task-oriented dialogue focuses on conversational agents that participate in dialogues with user goals on domain-specific topics.

to chatbots, which simply seek to sustain open-ended meaningful discourse, existing task-oriented agents usually explicitly model user intent and belief states.


One line of work has tackled the problem using partially observable Markov decision processes and reinforcement learning with carefully designed action spaces (Young et al., 2013).

the large, hand-designed action and state spaces make this class of models brittle and unscalable, and in practice most deployed dialogue systems remain hand-written, rule-based systems.


While reasoning over memory is appealing, these models simply choose among a set of utterances rather than generating text and also must have temporal dialogue features explicitly encoded.

the present literature lacks results for now standard sequence-to-sequence architectures, and we aim to fill this gap by building increasingly complex models of text generation, starting with a vanilla sequence-to-sequence recurrent architecture.


(2017) recently showed that UTD can be improved using the translations themselves as a source of information, which suggests joint learning as an attractive area for future work.

poor precision is most likely due to the simplicity of our MT model, and designing a model whose assumptions match our data conditions is an important direction for future work, which may combine our approach with insight from recent, quite different audio-to-translation models Adams et al., 2016a;Adams et al., 2016b;Berard et al., 2016).


The description of the discourse structure is still an open issue.

some low level characteristics have already been clearly identified, e.g.


Supervised approaches to DA recognition have been successfully investigated by many authors (Stolcke et al., 2000;Klüwer et al., 2010;Kalchbrenner and Blunsom, 2013).

annotating training data is both slow and expensive process.


This encourages the embedding of a word to be representative of the context surrounding it.

a careful look reveals that the context around a word can be split into multiple categories, specifically that each word has at least 2c contexts, one for each position in the window being considered.


Le and Mikolov (2014)'s Paragraph Vector framework also focus on capturing word order in their embeddings.

our method is more efficient since ECO does not require training the n-gram embeddings.


Success is usually measured in terms of correlation with human similarity judgments.

evaluating measures of phrase-level similarity directly against human judgments of similarity ignores the problem that it is not always possible to determine meaning in a compositional manner.


Using multi-sense embeddings in a hard clustering (rather than single-sense embeddings in a soft clustering) avoids the usage of a cluster membership threshold, which most soft clustering algorithms require.

the clustering algorithm outputs a membership degree for each element and each cluster, i.e., a fuzzy membership.


Stance Detection Over the last decade, there has been active research in modeling stance (Thomas et al., 2006;Somasundaran and Wiebe, 2009;Anand et al., 2011;Walker et al., 2012a;Hasan and Ng, 2013;.

all of these previous works treat each target independently, ignoring the potential dependencies that could exist among related targets.


However, the spectrum kernel obtains lower results in all the cases.

to the other two kernels, the spectrum one assigns a high score even when only one of the texts has a high frequency for a particular n-gram (see Section 3).


Previous work focus on learning a target specific representation for a given input sentence which is used for classification.

they do not explicitly model the contribution of each word in a sentence with respect to targeted sentiment polarities.


Interestingly, compared to BILSTM-ATT without contextualized attention, BILSTM-ATT-C loses accuracies on positive (-1.1%).

bILSTM-ATT-G gives large improvements on positive (+4.2%) and neutral (+1.2%) targets but loses accuracy on negative (-2.4%).


(2016) who propose a two-view label propagation approach instead.

to our knowledge, only Mohammad and Turney (2013) investigated the effects of these perspectives on annotation quality, finding differences in interannotator agreement (IAA) relative to the exact phrasing of the annotation task.


They used hand-crafted regular expressions to identify tweets indicating that a user may have bought or wanted a product.

their dataset was biased towards bought/want tweets and their patterns covered only a subset of possible bought/want phrases.


The model translates an input graph by segmenting it into subgraphs and generates a complete translation by combining subgraph translations left-to-right.

the model treats different graph segmentations equally.


(Cao et al., 2014;Huck et al., 2013) propose different approaches to directly train LRM for Hiero rules.

these approaches are designed for CKY-decoding and cannot be directly used or adapted for LR-Hiero decoding which uses an Earley-style parsing algorithm.


Long-distance reorderings as in this example are not uncommon and their benefit on verbal translation is intuitively clear.

reordering comes at the price of separating the verb and its direct object.


NMT has been successfully tackled by different groups using the sequence-to-sequence framework (Kalchbrenner and Blunsom, 2013;Cho et al., 2014;Sutskever et al., 2014).

multi-modal MT has just recently been addressed by the MT community in a shared task .


Most existing models for multilingual natural language processing (NLP) treat language as a discrete category, and make predictions for either one language or the other.

we propose using continuous vector representations of language.


The addition of sentences with feminine forms to the language model reduced the number of feminine pronouns translated as masculine or neuter.

we still observed cases where the translation did not reflect the morphological annotation in the source.


As shown by the small oracle experiments with manually inserted annotations, the potential for improvement through co-reference resolution is significant.

pre-processing errors from tagging, parsing, and the actual co-reference resolution reduce the effect somewhat, especially for the less frequent feminine forms.


The upper bound of this ratio is 1.0, which means that ACNN would make the same mistakes as GBT would.

from our experiments, the ratio turned out to be 0.37, which means that GBT and ACNN make different mistakes more than 60% of the time.


On the other hand, removal of the rare tokens (words appearing in less than 20 titles) seem to Manual translation of the Japanese product title into English: Manual translation of the Japanese product title into English: Manual translation of the Japanese product title into English: Manual translation of the Japanese product title into English: have negligible effect on the context of the titles from the subset of 8000 titles.

the effect is a little more pronounced for the context of the titles from the subset of the 18 categories for which ACNN does better than GBT, but, with a mean error difference of only half of that for the other 17 categories on which it does worse.


Earlier work of topic modeling is fully unsupervised while recently knowledge bases (KB) begin to be incorporated into semisupervised schemes (Wang et al., 2014;Zhai et al., 2010;Chen et al., 2014).


This process is easier when abstracts are structured, i.e., the text in an abstract is divided into semantic headings such as objective, method, result, and conclusion.

a significant portion of published paper abstracts is unstructured, which makes it more difficult to quickly access the information of interest.


8 The improvement of results for the visual labels could thus be attributed to the additional hidden layer.

the performance difference for the textual labels between baseline vidual label modalities and the combined labels differ, meaning that the nDCG numbers are not directly comparable.


F-Score Trigger Function The main criterion of NER task is F-score.

high label accuracy does not mean high F-score.


The above work has shown that MTL can be effectively used to improve NNs by leveraging different kinds of data.

the obtained improvement over the base DNN was limited to 1-2 points, raising the question if this is the kind of enhancement we should expect from MTL.


Finally, a recent work close to ours is (Guzmán et al., 2016), which builds a neural network for solving Task A of SemEval.

this does not approach the problem as MTL.


Their results showed a competitive gain over strong baselines.

we have presented a model that can also exploit a joint question and comment representation as well as the dependencies among the different SemEval Tasks.


Although these two users, especially beck-s, have more folders than others and thus present more challenges for classifiers, user embeddings has potential to effectively introduce user-token interactions for organizing information.

the improvements based on embedding features are less apparent for williams-w3, whose folder categorization was the most unbalanced among all (i.e., a majority of emails belong to the same folder, making the prediction fairly easy with just few signals).


For the CR task, we are dealing with a binary classification problem for each pair of EVENT and/or TIMEX3.

considering all pairs of entities within a sentence would give us an unbalanced data set with a very large amount of negative examples.


We plan to use argumentative zoning as a first step for IR and shallow document understanding tasks like summarization.


In such cases, the training data could be separated linearly by expanding all combinations of features as new ones, and projecting them onto a higherdimensional space.

such a naive approach requires enormous computational overhead.


If we could calculate the dot products from xz and x2 directly without considering the vectors ~(xz) and ¢(x2) projected onto the higher-dimensional space, we can reduce the computational complexity considerably.

namely, we can reduce the computational overhead if we could find the function K that satisfies: since we do not need itself for actual learning and classification, 1In general, ,It(x) is a mapping into Hilbert space.


Using such information about modifiers in the training phase has no difficulty since they are clearly available in a tree-bank.

they are not known in the parsing phase of the test data.


The simplest and most effective way to achieve better accuracy is to increase the training data.

the proposed method that uses all candidates that form dependency relation requires a great amount of time to compute the separating hyperplaneas the size of the training data increases.


Corpus-based grz.mmar induction relies on using many hand-parsed sentences as training examples.

the construction of a training corpus with detailed syntactic analysis for every sentence is a labor-intensive task.


For instance, it is difficult to induce a grammar from a corpus of raw text; but the task becomes much easier when the training sentences are supplemented with their parse trees.

appropriate supervised training data may be difficult to obtain.


There has been much work done on extracting Context-Free grammars (CFGs) (Shirai et al., 1995;Charniak, 1996;Krotov et al., 1998).

extracting LTAGs is more complicated than extracting CFGs because of the differences between LTAGs and CFGs.


These look a lot like the statistics a Markov Model would use.

in the maximum entropy framework it is possible to easily define and incorporate much more complex statistics, not restricted to n-gram sequences.


However, we note that our parameter estimation algorithm directly uses equation (1).

ratnaparkhi (1996: 134) suggests use of an approximation summing over the training data, which does not sum over possible tags: we believe this passage is in error: such an estimate is ineffective in the iterative scaling algorithm.


Ideally the third item can be estimated by using the forward-backward algorithm (Rabiner 1989) recursively for the first-order (Rabiner 1989) or second-order HMMs (Watson and Chunk 1992).

several approximations on it will be attempted later in this paper instead.


Above experiments shows that adding more contextual information into lexicon significantly improves the chunking accuracy.

this improvement is gained at the expense of a very large lexicon and we fred it difficult to merge all the above context-dependent lexicons in a single lexicon to further improve the chunking accurracy because of memory limitation.


The bigram language models are popular, in much language processing applications, in both Indo-European and Asian languages.

when the language model for Chinese is applied in a novel domain, the accuracy is reduced significantly, from 96% to 78% in our evaluation.


That is, a document that contains terms at, a2 and a3 may be ranked higher than a document which contains terms at and bl.

the second document is more likely to be relevant since correct translations of the query terms are more likely to co-occur (Ballesteros and Croft, 1998).


Ballesteros, for example, used the INQUERY (Callan et al, 1995) synonym operator to group translations of different query terms.

if a term has two translations in the target language, it will treat them as equal even though one of them is more likely to be the correct translation than the other.


The field of information retrieval (IR) is the traditional discipline that addresses this problem.

most of the prior work in IR deal more with document retrieval rather than "information" retrieval.


Prior to 1999, the other notable research work on question answering that is designed to work on unrestricted text (from encyclopedia) is (Kupiec, 1993).

no large scale evaluation was attempted, and the work was not based on a machine learning approach.


It has been found that both the shallow processing techniques of IR, as well as the more linguistic-oriented natural language processing techniques are needed to perform well on the TREC-8 QA track.

for our current QA work on reading comprehension, because the answer for each question comes from the associated story, no sophisticated IR indexing and retrieval techniques are used.


1 1In an earlier version of AQUAREAS, we simply used the raw word match score m as the feature.

the learned classifiers did not perform well.


Traditional (rationalist) approaches to constructing database interfaces require an expert to hand-craft an appropriate semantic parser (Woods, 1970;Hendrix et al., 1978).

such hand-crafted parsers are time consllming to develop and suffer from problems with robustness and incompleteness even for domain specific applications.


As expected, the accuracy of all methods grows (towards the upper bound) as more tuning corpus is added to the training set.

the relation between X+%Y-Y and %Y-Y reveals some interesting facts.