This works quite well in text documents, such as news articles, as we wish to find terms that occur frequently within that document, but are not common in the other documents in that domain.

we found that this approach does not work well in Twitter as tweets tend to be short and generally most terms occur only once, including their keywords.


This is because most keywords only occur once 3 , which makes the TF component not very informative.

the MAUI baseline performs significantly better, this is because of the usage of many hand engineered features using lists of words and Wikipedia, rather than simply relying on word counts.


(2) Negation is an important feature for our task.

having it alone is not enough to find ironic instances.


These results empirically validate (H2).

even though the algorithm improves the classifier performance, the number of queries is small which suggests that a much larger dataset is needed.


In our project, we intend to build an email briefing system which extracts and summarizes important email information for the users.

we believe there are critical components missing from the current research work.


replies, opens) as the indicator of email importance.

we argue that the user action does not necessarily indicate the importance of the email.


The words "and", "is", "was" and "by" have similar geometric arrangements in Wikipedia and in Twitter, since these common words are not key differentiators for these corpora.

the pronouns "I" and "you", are heavily used in Twitter but rarely used in Wikipedia.


A negative adjusted distance value means the word is more similar than at least half of Word Twitter Most Similar Wikipedia Most Similar bc because bcus bcuz cuz cos bce macedon hellenistic euthydemus ptolemaic ill ll imma ima will youll unwell sick frail fated bedridden cameron cam nash followmecam camerons callmecam gillies duncan mckay mitchell bryce mentions unfollow reply respond strangerswelcomed offend mentions mentioned mentioning reference attested miss misss love missss missssss imiss pageant pageants titlehoder titlehoders pageantopolis yup yep yupp yeah yea yepp chevak yupik gwaii tlingit nunivak taurus capricorn sagittarius pisces gemini scorpio poniatovii scorpio subcompact sagittarius chevette Table 2: Characteristic Words in Twitter Corpora words in its bucket.

the words that are less similar than at least half of words in their buckets have positive adjusted distance values.


These supervised methods obtain high accuracies on newswire (Xue and Shen, 2003;Zhang and Clark, 2007;Jiang et al., 2009;Zhao et al., 2010;Sun and Xu, 2011).

manually annotated training data mostly come from the news domain, and the performance can drop severely when the test data shift from newswire to blogs, computer forums, and Internet literature (Liu and Zhang, 2012;).Supervised approaches often have a high requirement on the quality and quantity of annotated corpus, which is always not easy to build.


For example, in conversation 1 in Table 1, "如果(if)" which is a very important cue to predict tense of verb "废(destroy)" is omitted.

(2) Effects of interactions on tense: to other genres, conversations are interactive, which may have an effect on tense: in some cases, tense can only be inferred by understanding the interactions.


According to Table 2, n-grams and dependency parsing features 4 are useful to predict tense, and linguistic knowledge can further improve the accuracy of tense prediction.

adding conversation-specific features (interaction features) does not benefit Local(b+p+l).


In these cases, learning tense transition patterns will mislead the model and accordingly affect the performance.

our global model is more robust because it is based on our "One tense per scene" hypothesis which can be seen as prior linguistic knowledge, thus achieves good performance even when the training data is not sufficient.


The results clearly show that the thread-level features are important, providing consistent improvement for all our learning models.

the linear-chain models fail to exploit the sequential dependencies between nearby answer labels to improve the results significantly: although the labels from the neighboring answers can affect the label of the current answer, this dependency is too loose to have impact on the selection accuracy.


In the first case (Q u 1 ), the third comment is classified as good by models that only use basic features.

thanks to the thread-level features, the classifier can consider that there is a dialogue between u 1 and u 2 , causing all the comments to be assigned to the correct class: bad.


In the second example (Q u 4 ), the first two comments are classified as bad when using the basic features.

the third comment -written by the same user who asked Q u 4 -includes an acknowledgment.


Synthetic text provides a cleanroom environment for evaluating QA systems, and has spurred development of powerful neural architectures for complex reasoning.

the formulaic semantics underlying these synthetic texts allows for the construction of perfect rule-based question answering systems, and may not reflect the patterns of natural linguistic expression.


7 These three results are obtained from files at http://research.microsoft.com/en-us/ um/redmond/projects/mctest/results.html.

8 we inspected these question annotations and   Table 3: Ablation study of feature types on the dev set.


In order to identify the correct candidate answer sentences, it is crucial to match the cardinal numbers and proper nouns with those occurred in the question.

many cardinal numbers and proper nouns are out of the vocabulary (OOV) of our word embeddings.


One is the full training set containing 1229 questions that are automatically labeled by matching answer keys' regular expressions.

2 the generated labels are noisy and sometimes erroneously mark unrelated sentences as the correct answers solely because those sentences contain answer keys.


It is notable that the methods based on deep learning perform more powerful than SVM and CRF, especially for complicate answers (e.g., Potential answers).

sVM and CRF using a large amount of features perform better for the answers that have obvious tendency (e.g., Good and Bad answers).


The main reason is that the distributed representation learnt from deep learning architecture is able to capture the semantic relationships between question and answer.

the feature-engineers in both SVM and CRF suffer from noisy information of CQA and the feature sparse problem for short questions and answers.


Baroni and Lenci (2008) is the only work we are aware of that addressed different property types, while utilizing a DM for property extraction.

their approach is simple, and includes defining the properties of a concept to be the 10 neighboring words of that concept in the DM space.


For example, we can append the special beginning-of-sentence symbol s and end-of-sentence symbol /s to all sentences to increase their lengths, allowing the relaxed hybrid trees to be constructed for certain sentence-semantics pairs with short sentences.

such an approach does not resolve the theoretical limitation of the model.


Just from reading the text, it is difficult to tell whether the speaker is asking an informational question or whether they are implying that they did not say that writing a dissertation was easy.

according to our observation, which forms the basis of this work, there are two cases in which rhetorical questions can be identified solely based on the text.


We should note also that our results assume a high level of consistency of the hand annotations from the original taggging of the Switchboard Corpus.

based on our observation and the strict guidelines followed by annotators as mentioned in Jurafsky et al.


Due to the reduced size of the evenly split dataset, performing a McNemar's test with Edwards' correction (Edwards 1948) does not allow us to reject the null hypothesis that the two experiments do not derive from the same distribution with 95% confidence (χ 2 = 1.49 giving a 2-tailed p value of 0.22).

over the whole skewed dataset, we find χ 2 = 30.74 giving a 2-tailed p < 0.00001 so we have reason to believe that with a larger evenly-split dataset inte-grating context-based features provides a quantifiable advantage.


Further, transfer learning is usually inferior to traditional supervised learning when the target domain already has good training data.

our target (or future) domain/task has good training data and we aim to further improve the learning using both the target domain training data and the knowledge gained in past learning.


Multi-task learning optimizes the learning of multiple related tasks at the same time (Caruana, 1997, Chen et al., 2011, Saha et al., 2011, Zhang et al., 2008.

these methods are not for sentiment analysis.


There is no explicit incongruity here: the only polar word is 'love'.

the clause 'I made a doggy bag out of it' has an implied sentiment that is incongruous with the polar word 'love'.


Note that the 'native polarity' need not be correct.

a tweet that is strongly positive on the surface is more likely to be sarcastic than a tweet that seems to be negative.


Previous researches on emotion analysis have mainly focused on emotion expressions in monolingual texts (Chen et al., 2010;Lee et al., 2013a).

in informal settings such as micro-blogs, emotions are often expressed by a mixture of different natural languages.


A typical approach is to collect a portion of historical reviews from each user to construct a shared training corpus .

this setting is problematic: it already exploits information from every user and does not reflect the reality that some (new) users might not exist when training the global model.


We inspected the learned weights in the adapted models in each user from LinAdapt, and found the words like waste, poor, and good share the same sentiment polarity as in the global model but different magnitudes; while words like money, instead, and return are almost neutral in global model, but vary across the personalized models.

words such as care, sex, evil, pure, and correct constantly carry the same sen- timent across users.


We split the data in training (train) and evaluation (test) sets as indicated in Table 1.

the SMt system was trained on freely avail-Source: <zone> <x translation="ou1-P">x</x> <wall/> a big advantage <wall/> <x translation="/ou1">x</x> </zone> of the hostel is its placement translation: por otra parte <ou1-P>una gran ventaja</ou1> del hostal es su colocación Figure 3: Source text with reordering constraint mark-up as well as code to pass tags, and its translation.


These features enable SLU models to robustly handle unseen entities at test time.

these lists are often massive and very noisy.


This corpus provides recorded eye-tracking data, collected with a remote faceLAB system.

the evaluation presented by Engonopoulos et al.


The system with added phrasal deletion achieved the BLEU score of 60.46, while the the standard model without phrasal deletion achieved the BLEU score of 59.87.

the baseline (BLEU score when the system does not perform any simplification on the original sentence) was 59.37, indicating that the systems often leave the original sentences unchanged.


Recently, there have been several attempts at addressing the TS task as a monolingual translation problem, translating from 'original' to 'simple' sentences.

they did not try to seek reasons for the success or the failure of their systems.


Hence, Gillick and Favre (2009) were right in their assumption that syntactic and semantic concepts would not lead to performance improvements, when restricting ourselves to this dataset.

when we change domain to the legal judgments or Wikipedia articles, using syntactic and semantic concepts leads to significant gains across all the ROUGE metrics.


We note that FrameNet 1.5 covers the legal domain quite well, which may explain why these concepts are particularly useful for the ECHR dataset.

labeled (LDEP) and unlabeled (UDEP) dependencies also significantly outperform the baseline.


This project provides a starting point for developing a treebank for resource-poor languages.

a mature parser requires a large treebank for training, and this is still extremely costly to create.


All we need to do is modify the learning objective function so that it includes the regularization part.

we don't want to regularize the part related to E en word since it will be very different between source and target language.


The error rate reduction is from 15.8% down to 6.5% for training data sizes from 1k to 15k tokens.

when we use all the training data, the supervised model is slightly better.


3 Parser Extensions We previously create the learning target by representing an AMR graph as a Span Graph, where each AMR concept is annotated with the text span of the word or the (contiguous) word sequence it is aligned to.

abstract concepts that are not aligned to any word or word sequence are simply ignored and are unreachable during training.


Neural networks have also been shown to be powerful generative models for language modelling (Bengio et al., 2003;Mikolov et al., 2010) and machine translation (Kalchbrenner and Blunsom, 2013;Devlin et al., 2014;Sutskever et al., 2014).

currently these models lack awareness of syntax, which limits their ability to include longer-distance dependencies even when potentially unbounded contexts are used.


The probability p(w|t, h) can be estimated similarly.

to reduce the computational cost of normalising over the entire vocabulary, we factorize the probability as P (w|h) = P (c|t, h)P (w|c, t, h), where c = c(w) is the unique class of word w. For each c, let Γ(c) be the set of words in that class.


When the size of the beam exceeds a set threshold, the lowest-scoring derivations are removed.

in an incremental generative model we need to compare derivations with the same number of words shifted, rather than transitions performed.


Boonkwan and Steedman (2011) train a parser that uses a semi-automatically constructed Combinatory Categorial Grammar (CCG, Steedman (2000)) lexicon for POS tags, while Bisk and Hockenmaier (2012;2013) show that CCG lexicons can be induced automatically if POS tags are used to identify nouns and verbs.

assuming clean POS tags is highly unrealistic for most scenarios in which one would wish to use an otherwise unsupervised parser.


Because RNNs make very few domain-specific assumptions, they have the potential to succeed at a wide variety of tasks with minimal feature engineering.

this flexibility also puts RNNs at a disadvantage compared to standard semantic parsers, which can generalize naturally by leveraging their built-in awareness of logical compositionality.


Of course, it also does not introduce additional information about compositionality or independence properties present in semantic parsing.

it does generate harder examples for the attention-based RNN, since the model must learn to attend to the correct parts of the now-longer input sequence.


On GEO and ATIS, the copying mechanism helps significantly: it improves test accuracy by 10.4 percentage points on GEO and 6.4 points on ATIS.

on OVERNIGHT, adding the copying mechanism actually makes our model perform slightly worse.


2 After finding the set Z of all consistent logical forms, we want to filter out spurious logical forms.

to do so, we observe that semantically correct logical forms should also give the correct denotation in worlds w other than than w. spurious logical forms will fail to produce the correct denotation on some other world.


The present model shares the same encoder with the sequence-to-sequence model described in Section 3.1 (essentially it learns to encode input q as vectors).

its decoder is fundamentally different as it generates logical forms in a topdown manner.


In other words, a sequence decoder is used to hierarchically generate the tree structure.

to the sequence decoder described in Section 3.1, the current hidden state does not only depend on its previous time step.


In fact, some trigger gazetteers have already been constructed by previous work such as (Yu et al., 2015).

manual construction of these triggers heavily rely upon labeled training data and high-quality patterns, which would be unavailable for a new language or a new slot type.


As far as we know, there are no results available for comparison.

the performance of Chinese SF is heavily influenced by the relatively low performance of name tagging since our method returns an empty result if it fails to find any query metnion.


A large amount of non-parallel, domain-rich, topically-related comparable corpora naturally exist across LLs and HLs for breaking incidents, such as coordinated news streams (Wang et al., 2007) and code-switching social media (Voss et al., 2014;Barman et al., 2014).

without effective Machine Translation techniques, even just identifying such data in HLs is not a trivial task.


For example, using the textual clues above is not sufficient to find the Hausa equivalent "Majalisar Dinkin Duniya" for "United Nations", because their pronunciations are quite different.

figure 5 shows the images retrieved by "Majalisar Dinkin Duniya" and "United Nations" are very similar.


(...as the violent typhoon, which has been given the name, Haiyan, has swept through the island of Leyte and Samar.)"

• Retrieved English comparable document: "As Haiyan heads west toward Vietnam, the Red Cross is at the forefront of an international effort to provide food, water, shelter and other relief..." using face detection results we successfully remove it based on processing the retrieved images as shown in Figure 7.


This neural system provides even more accurate predictions than our improved phrase-based system.

inference is two orders of magnitude slower, which is prob-lematic for an interactive setting.


The user query may be satisfied if the machine predicts the correct completion in its top-n output.

it is well-known that n-best lists are poor approximations of MT structured output spaces (Macherey et al., 2008;Gimpel et al., 2013).


a direct correspondence in the English source, but makes the sentence feel more natural in German.

nMT sometimes drops content words, as in Ex.


(2015) in image-to-caption translation to fix Φ = 1 for all source words, which means that we directly use the sum of previous alignment probabilities without normalization as coverage for each word, as done in (Cohn et al., 2016).

in machine translation, different types of source words may contribute differently to the generation of target sentence.


(2015), which encourages the model to pay equal attention to every part of the image (i.e., Φ j = 1).

our empirical study shows that the combined objective consistently worsens the translation quality while slightly improves the alignment quality.


In (attentional) encoder-decoder architectures for neural machine translation (Sutskever et al., 2014;Bahdanau et al., 2015), the decoder is essentially an RNN language model that is also conditioned on source context, so the first rationale, adding a language model to compensate for the independence assumptions of the translation model, does not apply.

the data argument is still valid in NMT, and we expect monolingual data to be especially helpful if parallel data is sparse, or a poor fit for the translation task, for instance because of a domain mismatch.


By using phrases, PB models can capture local phenomena, such as word order, word deletion, and word insertion.

one of the significant weaknesses in conventional PB models is that only continuous phrases are used, so generalizations such as French ne .


C D S (Koehn et al., 2003) • sequence (Galley and Manning, 2010) • • sequence  and • tree  This work • • graph Table 1: Comparison between our work and previous work in terms of three aspects: keeping continuous phrases (C), allowing discontinuous phrases (D), and input structures (S).

the expressiveness of these models is confined by hierarchical constraints of the grammars used (Galley and Manning, 2010) since these patterns still cover continuous spans of an input sentence.


Galley and Manning (2010) directly extract discontinuous phrases from input sequences.

without imposing additional restrictions on discontinuity, the amount of extracted rules can be very large and unreliable.


Another argument for discontinuous phrases is that they allow the decoder to use larger translation units which tend to produce better translations (Galley and Manning, 2010).

this argument was only verified on ZH-EN.


We find that both DTU and GBMT indeed tend to use larger translation units on ZH-EN.

more smaller translation units are used on DE-EN.


Typically, trees used in SMT are either phrasal structures (Galley et al., 2004;Marcu et al., 2006) or dependency structures Xiong et al., 2007;Xie et al., 2011;Li et al., 2014).

conventional treebased models only use linguistically well-formed phrases.


as evaluated by a human partner), the hypothesis selector will be updated with the success of its prediction.

if the action has never been encountered (i.e., the system has no knowledge about this verb and thus the corresponding space is empty) or the predicted action sequence is incorrect, the human partner will provide an action sequence A i that can correctly perform command v i in the current environment.


determined based on the consistency defined previously), node t (i) and a link t → t (i) are added to the space H. The node t (i) is also added to a temporary hypothesis container waiting to be further generalized.

some children hypotheses can be inconsistent with their parents.


As shown in Figure 6, all the four curves become steady after 8 learning instances are used.

while some verb frames have final SJIs of more than 0.55 (i.e.


Words are the basic input/output units in most of the NLP systems, and thus the ability to cover a large number of words is a key to building a robust NLP system.

considering that (i) the number of all words in a language including named entities is very large and that (ii) language itself is an evolving system (people create new words), this can be a challenging problem.


In particular, applying to machine translation task, (Luong et al., 2015) learns to point some words in source sentence and copy it to the target sentence, similarly to our method.

it does not use attention mechanism, and by having fixed sized soft-max output over the relative pointing range (e.g., -7, .


In question answering setting, (Hermann et al., 2015) have used placeholders on named entities in the context.

the placeholder id is directly predicted in the softmax output rather than predicting its location in the context.


Our hope was that the improvement would be larger for the entities data since the incidence of pointers was much greater.

it turns out this is not the case, and we suspect the main reason is anonymization of entities which removed datasparsity by converting all entities to integer-ids that are shared across all documents.


Table 1 shows that easy-first is more accurate than arc-standard.

it is also more computationally expensive.


Otherwise, it would entail that there is a tree reachable from C but unreachable from t(C), for any t. Therefore, we reformulate equation 1: (3) In the transition system, the grammar is left implicit: any reduction is allowed (even if the corresponding grammar rule has never been seen in the training corpus).

due to the introduction of temporary symbols during binarization, there are constraints to ensure that any derivation corresponds to a well-formed unbinarized tree.


conjecture that CDSMs might largely avoid problems handling adjectives with multiple senses because the matrices for adjectives implicitly incorporate contextual information.

they do draw a distinction between two ways in which the meaning of a term can vary.


Frequently these global lexical models create a different idiom token classifier for each phrase.

a number of papers on idiom type and token classification have pointed to a range of other features that could be useful for idiom token classification; including local syntactic and lexical patterns (Fazly et al., 2009) and cue words (Li and Sporleder, 2010a).


The decoder is essentially a neural language model conditioned on the input sentence representation h N i .

two RNNs are used (one for the sentence s i−1 and the other for the sentence s i+1 ) with different parameters except the embedding matrix (E), and a new set of matrices (C r , C z and C) are introduced to condition the GRU on h N i .


Context dependence Both our method and the two datasets, VJ'05 and MC'07, assume that the compositionality score can be computed for each phrase with no contextual information.

in general, the compositionality level of a phrase depends on its contextual information.


The correlation score is 0.674 and that is, the two different corpora lead to reasonably consistent results, which indicates the robustness of our method.

the correlation score is still much lower than perfect correlation; in other words, there are disagreements between the outputs learned with the corpora.


Therefore, determining semantic or topical cohesion is important for metaphor detection.

even if a text is literal and cohesive, not all words within the text are semantically related.


Our topical and emotion and cognition context features are general across target words.

the specific features that are informative for metaphor identification may depend on the target word.


The topical features added to the baseline led to a significant improvement in accuracy, while emotion and cognition features only slightly improved the accuracy without statistical significance.

the combination of these emotion and cognition features with topical features (in the last row of Table 2) leads to improvement.


• To learn the sparse codes, we first train the "true" embeddings by word2vec 2 for both common words and rare words.

these true embeddings are slacked during our language modeling.


The main motivation for this approach is the assumption that single-word distributional representations cannot represent all senses of a word well (Huang et al., 2012).

li and Jurafsky (2015) show that simply increasing the dimension- Figure 5: This grammar generates nouns (x n. i ) and adjectives (x a. i ) with masculine (x .m i ) and feminine (x .f i ) gender as well as paradigm features u i .


There are test cases in analogy that hypothetically evaluate specific facets like gender of words, as in king-man+woman=queen.

it does not consider the impact of other facets and assumes the only difference of "king" and "queen" is gender.


Our results confirm our previous observation that a classification by looking at subspaces is needed to answer this question.

based on full-space similarity, one can infer little about the quality of embeddings.


Based on our results, SSKIP and CWIN embeddings contain more accurate and consistent information because MLP classifier gives better results for them.

if we considered 1NN for comparison, SKIP and CBOW would be superior.


The smoothness properties could be much more complicated.

even if this was the case, then much of the general framework of what we have presented in this paper would still apply; e.g., the criterion that a particular facet be fully and correctly represented is as important as before.


The improvements, however, are not statistically significant.

a too conservative pair selection criterion with higher threshold values significantly deteriorates the overall performance of HYBWE with HFQ+HYB+SYM.


", "gun" is identified as an Instrument for "battle" event based on the AMR relation :instrument.

dependency parsing identifies "gun" as a   Table 7: Impact of semantic information and representations on typing for ERE data.


Then the probability of the sequence is computed as k t=1 c∈c(wt) p(c|w t , θ).

to skip-gram, CBOW (Mikolov et al., 2013a) uses context to predict each token, i.e.


(2010) proposed a spectral feature alignment (SFA) algorithm to align the domain-specific words from different domains in order to reduce the gap between source and target domains.

all of these methods transfer sentiment information from only one source domain.


The word "fast" is positive when used to describe CPU.

it is frequently used as a negative word in Battery domain.


opinion role induction (Wiegand and Ruppenhofer, 2015) and effect analysis (Choi and Wiebe, 2014).

our work is the first to organize various aspects of the connotative information into coherent frames.


Sentiment inference rules have been explored by the recent work of  and .

we make a novel conceptual connection between inferred sentiments and frame semantics, organized as connotation frames, and present a unified model that integrates different aspects of the connotation frames.


The first kind of methods assume that some training data in the source domain are very useful for the target domain and these data can be used to train models for the target domain after re-weighting.

feature representation approaches attempt to develop an adaptive feature representation that is effective in reducing the difference between domains.


computational cost and lack of scalability to highdimensional features.

these methods learn the unified domain-invariable feature representations by combining the source domain data and that of the target domain data together, which cannot well characterize the domain-specific features as well as the commonality of domains.


First, we apply social dynamics motivated by social science theories to entity-entity sentiment analysis in unstructured text.

most previous studies focused on social media or dialogue data with overt social network structure when integrating social dynamics (Tan et al., 2011;Hu et al., 2013;West et al., 2014).


Following previous works that use scores of local classifiers for uncertainty measurement (Sassano and Kurohashi, 2010;Flannery and Mori, 2015), we use Score(x, d * ) to measure the uncertainty of x, assuming that the model is more uncertain about x if d * gets a smaller score.

we find that directly using Score(x, d * ) always selects very short sentences due to the definition in Eq.


Our initial assumption is that the model is more uncertain about x if d * gets a smaller probability.

we find that directly using p(d * |x) would select very long sentences because the solution space grows exponentially with sentence length.


The reason may be that under PA, annotators can be more focused and therefore perform better on the few selected tasks.

some annotators may perform better under FA.


Second, we so far assume that the selected tasks are equally difficult and take the same amount of effort for human annotators.

it is more reasonable that human are good at resolving some ambiguities but bad at others.


If a similar question is found in the database of previous questions, then the corresponding best answer can be provided without any delay.

the major challenge associated with retrieval of similar questions is the lexicosyntactic gap between them.


Previous work represents the documents with averaged vectors of words (Tang et al., 2014;Tan et al., 2015).

this may lead to the loss of detailed information of the documents.


Most of the features are extracted under both of the settings.

feature 2 is much too computation-intensive and feature 7 needs POS-tagging as the preprocessing.


WMD helps to bridge the vocabulary gap between the citing context and the cited article.

though not often, the use of distributed representation can also create mistakes.


In another example shown in Table 8, ungrounded mention SNP, though detected, contributes little to supporting the ground truth pair.

when it's grounded to the entry Scottish National Party, the system leverages world knowledge and relates it to the mention of Scotland in the citing context.


More recently, attempts have been made to automatically extract pronunciation dictionaries directly from audio data (Stahlberg et al., 2016).

the requirement of a native speaker, web resources, or audio data specific to the language still blocks development, and the number of g2p resources remains very low.


This expands 56 g2p models (the languages covered by both IPA Help and Wiktionary models) to models for 211 languages.

as shown in Table 8, results are very poor, with a very high WER of 92% using the unioned models and a PER of more than 50%.


Since Xue (2003), most methods formalize the Chinese word segmentation (CWS) as a sequence labeling problem with character position tags, which can be handled with su-pervised learning methods such as Maximum Entropy (Berger et al., 1996;Low et al., 2005) and Conditional Random Fields (Lafferty et al., 2001;Peng et al., 2004;Zhao et al., 2006a).

those methods heavily depend on the choice of handcrafted features.


In order to get exact inference, most sequence-labeling systems address this problem with a Viterbi search, which takes the advantage of their hypothesis that the tag interactions only exist within adjacent characters (Markov assumption).

since our model is intended to capture complete history of segmentation decisions, such dynamic programming algorithms can not be adopted in this situation.


The results of the discrete model and the neural model without fine-tuning are highly similar, showing the usefulness of beam-search.

with fine-tuning, the results are different.


The model with beam size 1 gives better accuracies compared to the other models with the same beam size.

as the beam size increases, the performance increases very little.


The perspectives of our model can be considered a type of feature.

they are implemented by parametric differentiable functions.


We transform the question and the answer toq m andã n analogously using distinct matrices and bias vectors.

to the semantic perspective, we keep the question and answer candidates separate in the wordby-word perspective.


Moreover, linearization of the dependency graph, because it relies on an eigendecomposition, is not differentiable.

we handle the linearization in data preprocessing so that the model sees only reordered word-vector inputs.


This suggests that, to learn true reasoning abilities, MCTest is too simple a dataset-and it is almost certainly too small for this goal.

it may be that human language processing can be factored into separate processes of comprehension and reasoning.


These inferences are feasible in a limited domain, but become difficult the more open-domain reasoning they require.

even a simple lexical overlap classifier could correctly predict the entailment.


The domain of word denotations is then or-  dered by the subset operator, corresponding to ordering by hypernymy over the words.

2 hypernymy is not the only useful partial ordering over denotations.


(2015) for their conversational dialog engine Knowbot, and therefore constitutes a more fair comparison against their results.

we also make use of the full SCITEXT corpus (Clark et al., 2014).


The main challenge in learning a curriculum is that it requires the identification of easy and hard concepts in the given training dataset.

in real-world applications, such a ranking of training samples is difficult to obtain.


These datasets can also help us understand the shaping hypothesis as we can use them to verify if easier questions are indeed getting picked by our incremental learning algorithm before harder questions.

deep learning models (Le-Cun et al., 2015) have recently shown good performance in many standard NLP and vision tasks, including QA.


SPL is different from active learning (Settles, 1995) in the sense that the training examples (and labels) are already provided and the solution only orders the examples to achieve a better solution.

active learning tries to interactively query the user (or another information source) to achieve a better model with few queries.


This process of memory pattern restructuring is difficult to achieve, so it could be the reason for the relatively poor per-formance of naive curriculum learning and SPL strategies.

as we can see from the previous results, the explore and exploit strategy of mixing in some harder examples avoids the problem of having to abruptly restructure memory patterns.


We also tried variants of our models where we used curriculum learning on grade 3 questions, followed by grade 4 and grade 5 questions.

this did not lead to significant improvements.


For instance, the question in Table 1 only contains five words, while the best answer uses 60 words for elaboration.

the best answers from a pool can also be noisy and include extraneous information irrelevant to the question.


In (Feng et al., 2015), GESD outperforms cosine similarity in their models.

the proposed models with GESD as similarity scores do not provide any improvement on the accuracy.


Note that the best result in (Severyn and Moschitti, 2015) was obtained by combining CNN-based features with additional human-defined features.

our attentive LSTM model achieves higher performance without using any human-defined features.


Pasupat and Liang (2015) use tables from Wikipedia to construct a set of QA pairs.

their annotation setup does not impose structural constraints from tables, and does not collect finegrained alignment to table elements.


In both cases only the training partition of each dataset was used to formulate and handcraft tables.

for unbounded topics, additional facts were added to each table, using science education text books and websites.


(2015) propose a neural attention model for abstractive sentence compression which is trained on pairs of headlines and first sentences in an article.

our model summarizes documents rather than individual sentences, producing multi-sentential discourse.


In the standard neural sequence-to-sequence modeling paradigm (Bahdanau et al., 2015), an attention mechanism is used as an intermediate step to decide which input region to focus on in order to generate the next output.

our sentence extractor applies attention to directly extract salient sentences after reading them.


This suggests that an alternative to training the network would be to employ a ranking-based objective or a learning to rank algorithm.


The guidelines state in fact that 'If the negated affix is attached to an adverb that is a complement of a verb, the negation scopes over the entire clause' (Morante et al., 2011, p. 21) and we annotated suffixal negation in this way.

3 out of 4 examples of suffixal negation in adverbs in the training data (e.g.


Perhaps the most common fixedlength vector representation for texts is the bag-ofwords or bag-of-n-grams (Harris, 1970).

they suffer severely from data sparsity and high dimensionality, and have very little sense about the semantics of words or the distances between the words.


Wherein, the contexts are fixedlength (length is 2k) and sampled from a sliding window over the current sentence.

the word matrix W is shared across sentences.


(2014) built a conversation system using learning to rank and semantic matching techniques.

collecting enough Q-R pairs to build chatbots is often intractable for many domains.


The Interactive Alignment Model proposes that conversational partners prime each other, causing alignment via the primed reuse of structures ranging from individual lexical items to syntactic abstractions (Pickering and Garrod, 2004).

accommodation Theory emphasizes the relatively more communicative and strategic nature of alignment (Giles et al., 1991).


If alignment is driven primarily by priming, it should be relatively consistent across different aspects of a discourse.

from a strategic or communicative perspective, alignment -in which preceding words and concepts are reused -must be balanced against a need to move the conversation forward by introducing new words and concepts.


Distributional methods such as Linguistic Style Matching (LSM) (Niederhoffer and Pennebaker, 2002;Ireland et al., 2011) or the Zelig Quotient (Jones et al., 2014) calculate the similarity between the conversation participants over their frequencies of word or word category use in all utterances within the conversation.

conditional metrics, such as Local Linguistic Alignment (LLA) (Fusaroli et al., 2012;Wang et al., 2014) and the metric used by Danescu-Nicolescu-Mizil et al.


Intuitively, a within-turn topic boundary suggests that the speaker of the current turn is initiating the topic shift.

a betweenturn boundary suggests that the following speaker who first gives substantial contribution is more likely to be the initiator of the next topic.


Several such models have been proposed (Reiter and Dale, 2000;Henschel et al., 2000;Callaway and Lester, 2002;Krahmer and Theune, 2002;Gupta and Bandopadhyay, 2009;Greenbacker and McCoy, 2009).

all of these are fully deterministic, always choosing the same referential form in the same context.


Naive Bayes model vs. RNNs Although the RNNs were able to model individual variation in the choice of referential form to some extent, they did not perform significantly better than the Naive Bayes models, which might have to do with the relatively small dataset.

we think the size of the corpus matches the relatively low complexity of the problem we address.


Intuitively, the average American reader may not know exactly how many people are in Texas, but is familiar enough with the quantity to effectively reason using Texas' population as a unit.

it is less likely that the same reader is familiar with even the concept of Angola's population.


The model is able to capture rephrasings of fact descriptions and reordering of the facts.

it often confuses prepositions and, very rarely, can produce nonsensical utterances.


Qualitatively the neural networks outperform the baseline model in cases where they are able to levage additional knowledge about the entities (see first, third and fifth example in Table 4).

for rare relationships the baseline model appears to perform better, because it is able to produce a reasonable question if only a single example with the same relationship exists in the training set (see eighth example in Table 4).


A robust model might, for example, employ one pre- dictor to copy "The Foundation" from the input, and a another one to find the answer "Issac Asimov" by searching through a database.

training multiple predictors is in itself a challenging task, as no annotation exists regarding the predictor used to generate each output token.


This is informative as it gives an intuition of how many samples can be used without further human post-editing.

it does not provide an illustration on the degree of closeness to achieving the correct code.


Some tasks, such as the generation of queries (Zelle and Mooney, 1996), have overcome this problem by executing the query and checking if the result is the same as the annotation.

we shall leave the study of these methologies for future work, as adapting these methods for our tasks is not triv-ial.


Yet, it is a promising result that the model was able to capture this difference.

in many cases, effects that radically differ from seen ones tend to be generated incorrectly.


We adopt the WAC ("words-as-classifiers") model (Kennington and Schlangen, 2015), which was originally used for reference resolution in situated dialogue.

wAC is essentially a task-independent approach to predicting semantic appropriateness of words in visual contexts and can be flexibly combined with task-dependent decoding procedures.


We observe a clear learning effect in the non-interactive system, meaning that users faced unexpected interpretation problems due to inaccurate expressions, but adapted to the situation to some extent.

both installment systems have stable performance over time, which indicates that system behaviour is immediately understandable and predictable for human users.


every annotated mention could be mapped to at least one entity, and the set of entities included the gold entity.

changes in canonical Wikipedia URLs, accented characters and unicode usually result in mention losses over time, as not all URLs can be mapped to the KB (Hasibi et al., 2016, Sec.


It is of course possible to extend the global single-link model to the multi-focus case, by modifying the model factors and resulting messages.

the simplicity of the star-shaped model, its empirical effectiveness, and ease of learning parameters make it an attractive approach for easily incorporating attention into existing resolution models.


The common practice for ranking coreference resolvers is to use the average of three different metrics.

one cannot expect to obtain a reliable score by averaging three unreliable metrics.


Therefore, m is a coreferent mention that has at least one coreference link in the response entities.

none of its detected coreference links in the response entities are correct.


As can be seen, (1) recall changes for all the metrics except for MUC; (2) both CEAF e recall and precision significantly decrease; and (3) BLANC recall notably decreases so that F 1 drops significantly in comparison to Base.

adding completely incorrect entities to the response entities should not affect the recall and it should decrease the precision.


The following observations can be drawn from these experiments: (1) MUC and LEA are the only measures which give a zero score to the response that contains no correct coreference relations; (2) in our experiments, CEAF e shows an unreasonable drop when the correct link ratio changes from 0% to 20%; and (3), in Figure 2, the BLANC F 1 values are less than or equal to those of B 3 and LEA.

in Figure 3 that contains both coreferent and non-coreferent links, BLANC F 1 is at least 20% higher than that of other metrics.


The following points are worth noting from the results of Figure 7: (1) MUC and CEAF m are the least discriminative metrics when the system output includes extra mentions; (2) except for CEAF e , other metrics rank 3-10 as the worst output;(3) CEAF e recognizes both 2-0 and 3-0 as the worst outputs.

in these outputs the extra mentions are linked together and therefore no incorrect information is added to the correctly resolved entities; and (4) LEA is the only metric that recognizes error 2-0 is less harmful than 1-2 or 1-10.


However, in these outputs the extra mentions are linked together and therefore no incorrect information is added to the correctly resolved entities; and (4) LEA is the only metric that recognizes error 2-0 is less harmful than 1-2 or 1-10.

lEA does not discriminate the different outputs in which only one extra mention is added to an entity.


We believe entity-level information is particularly useful for preventing bad merges between large clusters (see Figure 4 for an example).

it is worth noting that in practice the much more complicated cluster-ranking model brings only fairly modest gains in performance.


2016extend their mention-ranking model by incorporating entity-level information produced by a recurrent neural network running over the candidate antecedent-cluster.

this is an augmentation to a mention-ranking model, and not fundamentally a clustering model as our cluster ranker is.


For instance, when multiple puzzle-composers write crossword puzzle clues for the same word, they will try to write creative, unique clues to make the puzzle interesting and challenging; clues for "star" could be "Paparazzi's target" or "Sky light."

people writing a descriptive caption for a photograph can adopt a less creative style.


And indeed, this paper will show that, depending on the choice of clustering method, semantics-based similarity measures such as summed word embeddings and deep neural networks can have an advantage over more traditional similarity metrics, such as n-gram counts, n-gram tf-idf vectors, and dependency tree kernels, when applied to creative texts.

unlike in most text similarity tasks, in clustering the choice of similarity metric interacts with both the choice of clustering method and the properties of the text.


TTR is an obvious estimate of the width of the vocabulary of a corpus.

all other things being equal, a corpus of many very short texts triggered by the same stimulus would have more repeated words, proportional to the total number of tokens in the corpus, than would a corpus of a smaller number of longer texts.


Traditionally documents are represented as a bag-of-words (BOW) vectors.

this simple representation suffers from being high-dimensional and highly sparse, and loses semantic relatedness across the vector dimensions.


Taking the logarithm of both sides, we obtain where m ik = L i j=1 δ(z ij = k) counts the number of words assigned with the k-th topic in 6 Variational Inference Algorithm Given the hyperparameters α, γ, µ, the learning objective is to find the embeddings V , the topics T , and the word-topic and document-topic distributions Here the hyperparameters α, γ, µ are kept constant, and we make them implicit in the distribution notations.

the coupling between A, V and T , Z, φ makes it inefficient to optimize them simultaneously.


The word-to-word bilingual constraints in PCLSA are not as effective.

our MTCA model incorporates the bilingual translations using auxiliary distributions which incorporate word distributions from the other language on the topic level and can capture common semantics of multilingual reader comments.


If the dashed link does not exist, both WSBM and SCC can identify two blocks.

if the dashed link does exist, SCC will return only one big block that contains all nodes, while WSBM still keeps the nodes in two reasonable blocks.


Thus B consists of only explicit positive links.

to avoid overfitting, we sample some implicit links and add them to B as explicit negative links.


The only difference between LCH-RTM and LBH-RTM is the block detection algorithm.

their link prediction performance is poles apart-LCH-RTM even fails to outperform RTM.


For the AraSenTi-Trans lexicon, if the tweet contains a negation particle and a positive word, we do not increment the positive word counter.

for tweets containing negative words and negation particles we found that not incrementing the negative word counter degraded the accuracy, so we opted to increment the negative word counter even if a negation particle is found in the tweet.


The performance of the lexicons on external datasets also suggests their ability to be used in classifying new datasets.

there is much room for improvement given the simple method used in evaluation.


In his approach, a Bayesian segmentation algorithm is applied, which is followed by a segment clustering algorithm.

the author tested his approach by using only documents with a few transitions among authors.


Automated Text Scoring (ATS) provides a cost-effective and consistent alternative to human marking.

in order to achieve good performance, the predictive features of the system need to be manually engineered by human experts.


We report the Spearman's rank correlation coefficient ρ, Pearson's product-moment correlation coefficient r, and the root mean square error (RMSE) between the predicted scores and the gold standard on our test set, which are considered more appropriate metrics for evaluating essay scoring systems (Yannakoudakis and Cummins, 2015).

we also report Cohen's κ with quadratic weights, which was the evaluation metric used in the Kaggle competition.


As described above, the SVM model has rich linguistic knowledge and consists of hand-picked features which have achieved excellent performance in similar tasks (Yannakoudakis et al., 2011).

in terms of RMSE, it is among the lowest performing models (8.85), together with 'BLSTM' and 'Twolayer BLSTM'.


(2016) recently used eyetracking recordings to induce PoS models.

a weakness of eye-tracking data is that while eye movement surely does reflect the temporal aspect of cognitive processing, it is only a proxy of the latter and does not directly represent which processes take place in the brain.


Our best explanation for why the incorporation of preceding fMRI measurements is beneficial to our model, is that the grammatical function of a token may be predictable from a reader's cognitive state while reading preceding tokens.

note that the measurements at the far ends of the time window only factor into the token vector to a small degree as a consequence of the Gaussian weighting.


(2013) addressed the ontology mismatch problem (i.e., two ontologies using different vocabularies) by first parsing a sentence into a domainindependent underspecified logical form, and then using an ontology matching model to transform this underspecified logical form to the target ontology.

their method is still hard to deal with the 1-N and the N-1 mismatch problems between natural language and target ontologies.


Traditionally, if a sentence contains a word which is not covered by the lexicon, it will cannot be correctly parsed.

with the help of sentence rewriting, we may rewrite the OOV words into the words which are covered by our lexicons.


This is especially the case for unsupervised resolvers.

a deep network can handle difficult cases like this via learning representations that make lexically different contexts look similar.


Finally, given that our network is trained in a supervised manner, we can extensively employ lex-ical features and use them in combination with other types of features that have been shown to be useful for AZP resolution.

rather than employing words directly as features, we employ word embeddings trained in an unsupervised manner.


They showed that supplementing a Bayesian linear ridge regression model (BLRR) with data from one other source domain is beneficial when there is limited target domain data.

it was shown that simply using the source domain data as extra training data outperformed the EA domain adaptation approach in three out of four cases.


These results indicate that because the preference-ranking model performs so well, only a few target-task training instances are needed for the linear-regression step of All-MTL-cTAP.

all-MTL-TaP uses all of the training instances in its final linear regression step, and performs significantly worse on a number of prompts.


Thus, answering factoid questions will be straightforward once they are converted into the corresponding structured form.

due to complexity of language, converting natural language questions to structure forms remains an open challenge.


There is an alternative factorization p(s, r|q) = p(s|q) • p(r|s, q).

it is rather challenging to estimate p(s|q) directly due to the vast amount of entities (> 10 6 ) in a KB.


As the two problems defined by equation (16) take the standard form of classification, theoretically, cross entropy can used as the training objective.

computing the partition function is often intractable, especially for p θs (s|r, q), since there can be millions of entities in the KB.


Alternative focus with CRF RNN-CRF based models have achieved the state-of-the-art performance on various sequence labeling tasks (Huang et al., 2015;.

the labeling task we consider here is relatively unsophisticated in the sense that there are only two categories of labels -part of subject string (SUB) or not (O).


Insepcting the table horizontally, when BiGRU is employed as the subject network, the accuracy is consistently higher regardless of relation network structures.

the margin is quite narrow, especially compared to the effect of varying the relation network structure the same way.


Accordingly, various semantic classifications of AO-selecting verbs have been developed, e.g., (Kiparsky and Kiparsky, 1970;Karttunen, 1971;Karttunen, 2012), some of them explicitly in the context of NLP (Nairn et al., 2006;Saurí, 2008).

these classifications are constructed manually and often quite limited in coverage.


These classes contain many AO-selecting verbs that were not covered by Levin's classification.

verbNet does not provide information about the modal meaning of AO-selecting verbs and does not reflect fine-grained distinctions between various kinds of modality.


While approaches to exploit the syntactic behavior of verbs for the automatic acquisition of semantic verb classes from corpora have been developed in the past, they were used to recover only small verb classifications: Schulte im Walde (2006)'s work considered a semantically balanced set of 168 German verbs, Merlo and Stevenson (2001) used 60 English verbs from three particular semantic classes.

to previous work, we consider a large set of more than 600 German AO-selecting verbs and focus on their modal meaning (i.e., expressing factuality or uncertainty).


Parsing accuracy on the coordination-enhanced corpus is higher than on the original trees.

the numbers are not strictly comparable, as the test sets contain trees with somewhat different number of constituents.


Using a large-scale crowdsourcing experiment, we demonstrate that human annotators are generally accurate in assessing the traits of others.

they make systematically different types of errors compared to a prediction model trained using the bag-of-words assumption.


Recent results on social media data report a performance of over 90% for gender classification and a correlation of r ∼ 0.85 for age prediction (Sap et al., 2014).

authors can introduce their biases in text (Recasens et al., 2013).


The second is identifying topical aspects and corresponding opinions, e.g., developing topic models to predict fine-grained aspect ratings (Titov and McDonald, 2008;Wang et al., 2011).

all those works emphasize population-level analysis, which applies a global model on all users and therefore fails to recognize the heterogeneity in which different users express their diverse opinions.


(Li et al., 2010) developed an online learning algorithm to continue training personalized classifiers based on a given global model.

all of these aforementioned solutions perform model adaptation from a fixed global model, such that the learning of personalized models is independent from each other.


For example, the phrase "a waste of money" generally represents negative opinions across all users; and it is very unlikely that anybody would use it in a positive sense.

members of some segments of a social structure tend to feel certain emotions more often or more intensely than members of other segments (Hochschild, 1975).


Our motivation to build a concept graph from a technical corpus is to improve performance at the task of reading list generation.

an applied evaluation makes it harder to judge the quality of the concept graph itself.


Another natural evaluation of an automatically generated concept graph would be to compare it to a  human-generated gold standard, where an expert has created concept nodes at the optimal level of generality and linked these by her understanding of the conceptual dependencies among concepts in the domain.

there are several difficulties with this approach: (1) It is quite labor-intensive to manually generate a concept graph; (2) we expect only moderate agreement between graphs produced by different experts, who have different ideas of what concepts are important and distinct and which concepts are important to understanding others; and (3) the concept graphs we learn from a collection of documents will differ significantly from those we imagine, without these differences necessarily being better or worse.


In this work, we assume that a topic model provides a reasonable proxy for the concepts a person might identify in a technical corpus.

topic modeling approaches are better at finding general areas of research than at identifying fine-grained concepts like those shown in Figure 1.


Stolcke (2002) find that the approximation actually decreases perplexity, which we also see in the experiments ( §6).

approximation only happens when the model backs off, which is less likely to happen in fluent sentences used for perplexity scoring.


3 The log probability of tuning data w is which expands according to the definition of p LL n i The gradient with respect to λ i has a compact form where CH is cross entropy.

computing the cross entropy directly would entail a sum over the vocabulary for every word in the tuning data.


In the same direction, Rothe and Schütze (2015) trained embeddings by mixing words, lexemes and synsets, and introducing a set of features based on calculations on the resulting representations.

none of these techniques takes full advantage of the semantic information contained in embeddings.


(2003), word embeddings aim at representing, i.e., embedding, the ideal semantic space of words in a real-valued continuous vector space.

to traditional distributional techniques, such as Latent Semantic Analysis (Landauer and Dutnais, 1997, LSA) and Latent Dirichlet Allocation (Blei et al., 2003, LDA), Bengio et al.


(2014), which leverage embeddings for supervised (the former three) and knowledge-based (the latter) WSD.

to our knowledge, no previous work has investigated methods for integrating word embeddings in WSD and the role that different training parameters can play.


Among combination strategies, concatenation and average produced the smallest gain and did not benefit from embeddings of higher dimensionality.

the other two strategies, i.e., fractional and exponential decay, showed improved performance with the increase in the size of the employed embeddings, irrespective of the WSD features.


The more complex Impatient Reader computes attention over the document after reading every word of the query.

empirical evaluation has shown that both models perform almost identically on the CNN and Daily Mail datasets.


Figure 7a shows that the accuracy significantly drops as the correct answer gets less and less frequent in the document compared to other candidate answers.

the correct answer is likely to occur frequently (Fig.


More recently, methods that jointly infer the opinion entity and relation extraction tasks (e.g., using Integer Linear Programming (ILP)) have been introduced (Choi et al., 2006;Yang and Cardie, 2013) and show that the existence of opinion relations provides clues for the identification of opinion entities and vice-versa, and thus results in better performance than a pipelined approach.

the success of these methods depends critically on the availability of opinion lexicons, dependency parsers, named-entity taggers, etc.


They are capable of modeling sequences of arbitrary length by repetitive application of a recurrent unit along the tokens in the sequence.

recurrent neural networks are known to have several disadvantages like the problem of vanishing and exploding gradients.


For the first hidden layer, the computation proceeds similar to that described in Section 3.1.

for higher hidden layers i the input to the memory block is the hidden state and memory cell from the previous layer i − 1 instead of the input vector representation.


With respect to holders and targets, we find that our SLL model performs similar to the CRF baseline.

the SLL+RLL model outperforms CRF baseline.


Our sequential LSTM model is able to learn these relations thus validating that LSTMs can model long-term dependencies.

for IS-FROM relations, we find that our recall is lower than the ILP-based joint model.


Averaging the embeddings of words in a sentence has proven to be a surprisingly successful and efficient way of obtaining sentence embeddings.

word embeddings trained with the methods currently available are not optimized for the task of sentence representation, and, thus, likely to be suboptimal.


Speedups might of course be gained for all methods by presenting the sentences in batches to the models, by computing sentence representations in parallel and by running code on a GPU.

as we are interested in the differences between the systems, we run the most simple and straightforward scenario.


It is beyond the scope of this paper to provide a comprehensive analysis of all supervised methods using word or sentence embeddings and the effect Siamese CBOW would have on them.

it would be interesting to see how Siamese CBOW embeddings would affect results in supervised tasks.


With negative mappings, null maps to all:e, while each proper noun maps to its proper predicate minus all:e. There is a lot of work in semantic parsing that tackles the GeoQuery dataset (Zelle and Mooney, 1996;Zettlemoyer and Collins, 2005;Wong and Mooney, 2007;Kwiatkowski et al., 2010;Liang et al., 2011), and the state-of-the-art is 91.1% precision and recall (Liang et al., 2011).

none of these methods can guarantee 100% precision, and they perform more feature engineering, so these numbers are not quite comparable.


Another avenue for providing user confidence is probabilistic calibration (Platt, 1999), which has been explored more recently for structured prediction (Kuleshov and Liang, 2015).

these methods do not guarantee precision for any training set and test input.


Although some RNN models showed little performance gains over the SVM baselines only with doc2vec features, they were even worse than the CRF model with the same features.

the RCNN models connecting the results of CNNs to the RNNs contributed to performance improvements not only from the baselines, but also from the CNN models.


While the CNN model tended to have a higher coverage in segmentation than the others, the LRCN model produced more precise decisions to recognize the boundaries on the strength of the consideration of conversational coherences in dialogue history sequences.

the segmentation performances even with the best models were still very limited especially for inter-categorical transitions.


Secondly, the use of external knowledge could be a key to success in dialogue topic tracking, as proved in the previous studies.

this work only takes internal dialogue information into account for making decisions.


Some of these studies also demonstrated that using L1specific properties, such as the error patterns of speakers of a given L1 or L1-induced paraphrases, improves the performance of automatic error correction in non-native writing.

neither of the approaches has constructed a semantic model from L1 data and systematically studied the effects of its transfer onto L2.


The results of these studies suggest that L1 is a valuable source of information in EDC.

all these works use isolated translational equivalents and focus on error correction only.


Adding the L1 lexicosemantic features has only a minor effect on accuracy and precision, and a more pronounced effect on recall.

when we test the system on one particular related L1 (Table 8) we observe the opposite effect: with the exception of ES subj data, precision and accuracy improve, suggesting that the error detection system using L1-induced information identifies errors more precisely.


For distractors considered "Plausible" by both experts, 63.6% were selected by the learners.

for those considered "Obviously wrong" by both experts, only 11.8% attracted any learner.


Limsopatham and Collier (2015a) leveraged translations between the informal language used in social media and the formal language used in the description of medical concepts in an ontology.

we argue that effective concept normalisation requires a system to take into account the semantics of social media messages and medical concepts.


We observe that for TwADR-S and TwADR-L datasets, which are smaller datasets (dataset size of 201 and 1,436, respectively), a better performance can be achieved if the model is not allowed to update the embeddings of the input phrases.

for the AskAPatient dataset (dataset size of 8,662), allowing the model to update the embeddings results in a significantly (paired t-test, p < 0.05) better performance.


We neglected such time-related phrases when defining C 0 .

the frequencies of the listed n-grams in the business group are much higher than those in the individual group.


For example, English POS taggers benefit from carefully designed word spelling features; orthographic features and external resources such as gazetteers are widely used in NER.

such task-specific knowledge is costly to develop (Ma and Xia, 2014), making sequence labeling models difficult to adapt to new tasks or new domains.


Several RNN-based neural network models have been proposed to solve sequence labeling tasks like speech recognition (Graves et al., 2013), POS tagging  and NER (Chiu and Nichols, 2015;Hu et al., 2016), achieving competitive performance against traditional models.

even systems that have utilized distributed representations as inputs have used these to augment, rather than replace, hand-crafted features (e.g.


For many sequence labeling tasks it is beneficial to have access to both past (left) and future (right) contexts.

the LSTM's hidden state h t takes information only from past, knowing nothing about the future.


The RNNLM could be used in a KNN-style approach, where it associates each example response with its individual topic vector, using L(q) as a distance metric.

this is computationally infeasible since computing L(q) is significantly more expensive than cosine distance and the previously mentioned scalability would be lost.


SC-EX2, which relates to similar issues, is often confused with it.

sC-EX3 is rarely confused with sC-EX1 as it is about non-personal events on a larger scale.


A limitation of both the standard and proposed approach is that if a new question is created by the test-makers, then it will be necessary to collect example responses before it can be widely deployed.

since the system can be trained on ASR transcriptions, the example responses do not need to be hand-transcribed.


This makes automatic detection of sarcasm an important problem.

it has been quite difficult to solve such a problem with traditional NLP tools and techniques.


(2016) for modeling sarcasm understandability of readers.

as far as we know, these features are being introduced in NLP tasks like textual sarcasm detection for the first time.


(2013) conducted a corpus analysis and found certain similarity between Penn Discourse TreeBank relations (Prasad et al., 2008) and argumentation schemes (Walton et al., 2008).

they did not discuss how such similarity could be applied to argument mining.


Argument words (e.g., 'believe', 'opinion') signal the argumentative content and are commonly used across different topics.

domain words are specific terminologies commonly used within the topic (e.g., 'art', 'education').


If you know what labels you want, active learning can reduce the number of labeled documents needed.

establishing the label set remains difficult.


In LA and LR, the preference function directly chooses a document and directs the user to it.


It is designed to detect error patterns from a fixed window of 7 words, which is large enough to not require the use of more advanced composition functions.

the performance of the bidirectional recurrent network (Bi-RNN) is somewhat lower, especially on the test set.


It is worth noting that the full Bi-LSTM has been trained on more data than the other CoNLL contestants.

as the shared task systems were not restricted to the NUCLE training set, all the submissions also used differing amounts of training data from various sources.


Through the availability of large annotated resources, such as PropBank (Palmer et al., 2005), statistical models based on such features achieve high accuracy.

results often fall short when the input to be labeled involves instances of linguistic phenomena that are relevant for the labeling decision but appear infrequently at training time.


Analogously to how word embeddings are computed, the simplest way to embed paths would be to represent each sequence as a one-hot vector.

this is suboptimal for two reasons: Firstly, we expect only a subset of dependency paths to be attested frequently in our data and therefore many paths will be too sparse to learn reliable embeddings for them.


The prior studies attempt to predict how strongly users are currently engaged in dialogues with systems.

the goal of this study is to predict how strongly users will be engaged with intelligent assistants in the future.


chat Voice Assist can give greetings to, have conversations with, and play games with users (e.g., V2 and V6).

to device operations for accomplishing certain tasks, these functions are offered for fun or for facilitating smooth communication.


In the trigger classification stage, some previous approaches (Grishman et al., 2005;Ji and Grishman, 2008;Liao and Grishman, 2010;Huang and Riloff, 2012) use patterns to decide the types of event triggers.

pattern-based approaches suffer from low recall since real world events usually have a large variety of representations.


DMCNN is mainly used to model contextual features.

dMCNN still does not consider argument-argument interactions.


We can see that RBPB with both plain feature and pattern feature can gain  Table 3: The effect (F 1 value) of pattern feature much better performance than with two kinds of features alone.

our approach is just a pipeline approach which suffers from error propagation and the argument performance may not affect the trigger too much.


We find that JET depends too much on event patterns for event type priori and JET considers each candidate argument separately.

patterns cannot cover all events and the relationship between candidate arguments may help when identifying arguments.


Therefore, we develop an event type classifier to assign the event type, using both pattern matching information and other features, which gives our system the capability to deal with failed match cases when using patterns alone.

we train a maximum entropy classifier to predict the relationship between candidate arguments.


In summary, by using the event type classifier and the regularization method, we have achieved a good performance in which the trigger classification is comparable to state-of-theart methods, and the argument identification & classification performance is significantly better than state-of-the-art methods.

we only use sentence-level features and our method is a pipelined approach.


With self-training, we successfully exploit unlabeled data to improve over ENT by 15% F-score on the newswire domain, and 13% F-score on clinical data.

our active learning experiments demonstrate that we can match (and even beat) ENT using only 6.6% of the training data in the clinical domain, and only 5.8% of the training data in the newswire domain.


On RTE-2 their method achieves accuracy improvements of upto 10%.

hickl and Bensley (2007) achieves only a 1% accuracy improvement on RTE-3 using the same method, suggesting that it is not always as beneficial.


A token in one text passage that matches exactly, characterfor-character, with a token in another text passage would be considered a term match by this simple term matcher.

these term matchers could be more sophisticated and match pairs of terms that are synonyms, or paraphrases, or   Table 3.


Our self-training system substantially improved over ENT, achieving an F-score gain of 15% on newswire and 13% on clinical, using only additional unlabeled data.

our active learning experiments demonstrated that we could match (and even beat) the baseline ENT system with only 6.6% of the training data in the clinical domain, and only 5.8% of the training data in the newswire domain.


Existing multilingual topic models consider only document-level alignments.

most documents are hierarchically structured, i.e., a document comprises segments (e.g., sections and paragraphs) that can be aligned across languages.


BiSTM leverages existing segments from a given segmentation.

a segmentation is not always given, and a given segmentation might not be optimal for statistical modeling.


This seems to be reasonable, because some original sections include multiple topics.

tables 2 and 3 show that inferred boundaries do not work better than section boundaries.


Orthogonal to the formal semantics of DCS, distributional vector representations are useful in capturing lexical semantics of words (Turney and Pantel, 2010;Levy et al., 2015), and progress is made in combining the word vectors to form meanings of phrases/sentences (Mitchell and Lapata, 2010; Baroni and Zamparelli, 2010;Grefenstette and Sadrzadeh, 2011;Socher et al., 2012;Paperno et al., 2014;Hashimoto et al., 2014).

less effort is devoted to finding a link between vector-based compositions and the composition operations in any formal semantics.


It suggests regularities in the semantic space, especially because orthogonal matrices preserve cosine similarity -if M N is orthogonal, two words x, y and their projections π N (x), π N (y) will have the same similarity measure, which is semantically reasonable.

matrices trained by vecUD are only orthogonal for three UD relations, namely conj, dep and appos.


(2014) and the entanglement model in Kartsaklis and Sadrzadeh (2014).

these models do not show particular advantages on other datasets.


Results As shown in Table 7, vecDCS scores better than the N-gram model and demonstrates promising performance.

to our surprise, "no matrix" shows an even better result which is the new state-of-the-art.


Logic and Distributional Semantics Logic is necessary for implementing the functional aspects of meaning and organizing knowledge in a structured and unambiguous way.

distributional semantics provides an elegant methodology for assessing semantic similarity and is well suited for learning from data.


(2014), in exploring vector calculations that realize logic operations.

the previous works did not specify how to integrate contextual distributional information, which is necessary for calculating semantic similarity.


word embedding) in the answer directly, and then we use RNN to model the attentive word sequence.

this model attends a sentence word by word which may ignore the relation between words.


In the example given above, the verb corresponds quite closely to the desired target relation.

in the wild, we encounter a multitude of different ways of expressing the same kind of relationship.


(2014) proposed a deep convolutional neural network with softmax classification, extracting lexical and sentence level features.

these approaches still depend on additional features from lexical resources and NLP toolkits.


In the training phase, we could probably find that every two entities connected by this relation are also connected by the path editor/film −1 , and hence assign an extremely high weight to it.

5 in the testing phase, for any entity pair (x, y) such that (y, editor/film, x) has not been encoded, we might not even find that path and hence could always make a negative prediction.


Website privacy policies are often ignored by Internet users, because these documents tend to be long and difficult to understand.

the significance of privacy policies greatly exceeds the attention paid to them: these documents are binding legal agreements between website operators and their users, and their opaqueness is a challenge not only to Internet users but also to policy regulators.


Legal text is a familiar domain for natural language processing, and the legal community has demonstrated some reciprocal interest (Mahler, 2015).

the scale of the problem and its significance-i.e., to virtually any Internet user, as well as to website operators and policy regulators-distinguishes it and provides immense motivation (Sadeh et al., 2013).


Other approaches leverage topic mod-eling (Chundi and Subramaniam, 2014;Stamey and Rossi, 2009) or sequence alignment techniques  to analyze privacy policies or identify similar policy sections and paragraphs.

the complexity and vagueness of privacy policies makes it difficult to automatically extract complex data practices from privacy policies without substantial gold standard data.


The search is based on a so-called "floating parser" (Pasupat and Liang, 2015), a modification of a standard chart-parser, which is able to guide the search based on the similarity features.

our approach does not search among the derivations for the one that maximizes a match with the NL, but instead directly tries to predict a decision sequence that can be mapped to the LF.


number of rules) is larger than that of LF or CF.

dS is unique in allowing us to easily validate grammatical constraints.


gives small probability to ungrammatical choices, a property not shared by DSP-CL.

a more complete understanding of the difference will need more investigation.


Word embeddings-distributed representations of words-in deep learning are beneficial for many tasks in NLP.

different embedding sets vary greatly in quality and characteristics of the captured information.


Each embedding set contains information about only a part of the overall vocabulary.

it can predict what the remaining part should look like by comparing words it knows with the information other embedding sets provide about these words.


(i) Not all individual embedding sets are beneficial in this task; e.g., HLBL embeddings make FLORS perform worse in 11 out of 12 cases.

(ii) in most cases, embeddings improve system performance, which is consistent with prior work on using embeddings for this type of task (Xiao and Guo, 2013;Yang and Eisenstein, 2014;Tsuboi, 2014).


As a concrete example we collect live text data and corresponding news reports for football (called soccer more often in the United States) games and conduct our study thereby.

our methods and discussions made in this paper can be trivially adapted to other types of sports games as well.


This is used for traditional summarization problems (Kulesza and Taskar, 2011).

the problem for constructing sports news from live broadcast script is rather different.


The comparison between RF and RF+DPP shows the effectiveness of our sentence selection strategy.


All the sentences are describing a key scoring event.

none of them were selected to construct the news because our LTR model assigns low scores for these short sentences.


In our experiments, methods with higher ROUGE scores can indeed achieve better coverage of important units such as events, as shown in pyramid scores in Table 2.

we can also observe from Table 5 that automatic metrics currently cannot reflect readability factors very well.


(2010) introduced a joint rule selection model for hierarchical phrase-based translation, which also approximated the rule selection problem by a binary classification problem like our approach.

these two models adopted linear classifiers similar to those used in the MERS model , which suffers more from the data sparsity problem compared to the CSRS model.


In traditional linear models, combining scores can be implemented by including low-order features.

for neural models, this is not that straightforward because of nonlinearity.


The neural rerankers are capable of capturing global syntax features across the tree.

the most non-local neural parser with LSTM  cannot exploit global features.


These methods aim to learn common feature representations for different languages.

most of the current researches only focus on bilingual word embedding.


grSemi-CRFs use much fewer input features and most of them are task-insensitive 13 .

grSemi-CRFs achieve almost the same performance, sometimes even better.


For text chunking, grSemi-CRF outperforms all reported supervised models, except JESS-CM (Suzuki and Isozaki, 2008), a semi-supervised model using giga-word scale unlabeled data in training 14 .

the performance of our grSemi-CRF (95.01%) is very close to that of JESS-CM (95.15%).


Summarization of event descriptions can be improved by selecting causally motivated sentences.

causality is frequently expressed implicitly, which requires world knowledge and inference.


The explosion caused people to evacuate the building.

the words can not be substituted in the following sentence: The baker made a cake.


Similar to the FrameNet features, we split the example into three sections.

we also consider the dependency parse of the data.


VerbNet VerbNet is a resource devoted to storing information for verbs (Kipper et al, 2000).

to WordNet, VerbNet provides a more fine-grained description of events while focusing less on polysemy.


Modeling relation paths has offered significant gains in embedding models for knowledge base (KB) completion.

enumerating paths between two entities is very expensive, and existing approaches typically resort to approximation with a sampled subset.


Therefore prior work selected a subset of paths through sampling and pruning (Neelakantan et al., 2015;Lin et al., 2015).

the properties of the BILINEAR composition function for path representation enable us to incrementally build the sums of all path representations exactly, using dynamic programming.


In either case, the accuracy difference between ALL-PATHS and PRUNED-PATHS is small, and ALL-PATHS mainly gains in efficiency.

when nodes are modeled, the compositional learning approach gains in accuracy as well, especially when text is jointly embedded.


For the DNN models, we add the penalty term λ θ 2 to the losses, where λ is the regularization coefficient and θ contains all other parameters.

for the bilinear models we regularize the relation matrices M R toward the identity matrix instead of all zeroes, adding the following to the loss: where I r is the r × r identity matrix, the summation is performed over all relations R, and θ represents all other parameters.


Figure 6 shows how the models perform as the number of utterances per example varies.

when the search space is small (fewer number of utterances), Model B outperforms or is competitive with Model C. as the search space increases (tighter computational constraints), Model C does increasingly better.


Even though  Model A is unable to learn anything with beam size < 240.

for beam sizes larger than 240, Model A attains 100% accuracy.


Recall that projecting from Model A to C creates a more computationally tractable model at the cost of expressivity.

this is because Model C used a linear model.


Tree-structured neural networks exploit valuable syntactic parse information as they interpret the meanings of sentences.

they suffer from two key technical problems that make them slow and unwieldy for large-scale NLP tasks: they usually operate on parsed sentences and they do not directly support batched computation.


Our results show that a model that uses tree-structured composition fully (SPINN) outperforms one which uses it only partially (tree-based CNN), which in turn outperforms one which does not use it at all (RNN).

the full SPINN performed moderately well at reproducing the Stanford Parser's parses of the SNLI data at a transition-by-transition level, with 92.4% accuracy at test time.3 its transi-3Note that this is scoring the model against automatic tion prediction errors are fairly evenly distributed across sentences, and most sentences were assigned partially invalid transition sequences that either left a few words out of the final representation or incorporated a few padding tokens into the final representation.


For SG model, the max selectivity of the model neurons is only just above 0.7.

the context-based distributed models showed strong selective activation towards country names in Indonesian.


On the contrary, the context-based distributed models showed strong selective activation towards country names in Indonesian.

the selectivity of all the AE neurons is below 0.7 towards these semantically-related words.


Words like yet, even, and still are used in many diverse ways and are highly polysemous.

words like photocopying, postage, and holster tend to be used in very specific well-clustered contexts, corresponding to a single sense; for example, mail and letter are both very likely to occur in the context of postage and are also likely to co-occur with each other, independent of postage.


Previous works argued that semantic change leads to polysemy (Wilkins, 1993;Hopper and Traugott, 2003).

our results show that polysemous words change faster, which suggests that polysemy may actually lead to semantic change.


(2004b) outperformed SEMCOR for words with SEMCOR frequency less than 5.

their analysis was based on the accuracy of the first sense heuristic, rather than the entire sense distribution, and they used very different datasets to us.


The idea of a book excerpt completion task was originally introduced in the MSRCC dataset (Zweig and Burges, 2011).

the latter limited context to single sentences, not attempting to measure broader passage understanding.


In particular, when the context clearly demands a referential expression, the constraint that the blank be filled by a single word excludes other possibilities such as noun phrases with articles, and there are reasons to suspect that co-reference is easier than other discourse phenomena in our task (see below).

although co-reference seems to play a big role, only 0.3% of target words are pronouns.


Note that LAMBADA was designed to challenge language models with harder-than-average examples where broad context understanding is crucial.

the average case should not be disregarded either, since we want language models to be able to handle both cases.


An answer is correct when there is an exact string match between the predicted answer and the gold answer.

as describe in Section 2.2, some answers are composed from a set of values (e.g.


This means that spectral algorithms give a natural way for the selection of the number of latent states for each nonterminal a in the grammar.

when the data from which we estimate an L-PCFG model are not drawn from an L-PCFG (the model is "incorrect"), the number of non-zero singular values (or the number of singular values which are large) is no longer sufficient to determine the number of latent states for each nonterminal.


11 Our main focus is on comparing the coarse-to-fine Berkeley parser (Petrov et al., 2006) to our method.

for the sake of completeness, we also present results for other parsers, such as parsers of Hall et al.


In this table, we see that in most cases, on average, the optimization algorithm chooses to enlarge the number of latent states.

for German-T and Korean, for example, the optimization algorithm actually chooses a smaller model than the original vanilla model.


When estimating an HMM of a low order with data which was generated from a higher order model, EM does quite poorly.

if the number of latent states (and feature functions) is properly controlled with spectral algorithms, a spectral algorithm would learn a "product" HMM, where the states in the lower order model are the product of states of a higher order.


either using semi-supervised clustering instead of POS tags (Koo et al., 2008) or building recurrent representations of words using neural networks .

the best accuracy for these approaches is still achieved by running a POS tagger over the data first and combining the predicted POS tags with additional representations.


Researchers have used supervised learning models trained on lexical word ngram features, synsets, emoticons, topics, and lexicon frameworks to determine which emotions are expressed on Twitter Roberts et al., 2012;Qadir and Riloff, 2013;.

sentiment classification in social media has been extensively studied (Pang et al., 2002;Pang and Lee, 2008;Pak and Paroubek, 2010;Hassan Saif, Miriam Fernandez and Alani, 2013;Nakov et al., 2013;Zhu et al., 2014).


One assumption that we made about the nature of state adoption of model legislation is that the legislatures make modifications that largely preserve the model language in an effort to preserve policy.

we currently do not consider cases in which a legislature has intentionally obscured the text while still retaining the same meaning.


LOBBYBACK performs well on reconstructing model legislation from automatically generated bill clusters.

there are a number of improvements that can refine part of the pipeline.


If we take Argument 1 from Figure 1, assigning a single "convincingness score" is highly subjective, given the lack of context, reader's prejudice, beliefs, etc.

when comparing both arguments from the same example, one can decide that A1 is probably more convincing than A2, because it uses at least some statistics, addresses the health factor, and A2 is just harsh and attacks.


For comparison, the partial information classification algorithm Banditron (Kakade et al., 2008) (after adjusting the exploration/exploitation constant on the dev set) scored 0.047 on the test set.

our main interest is in convergence speed.


For the OCR task, bandit learning does decrease Hamming loss, but it does not quite achieve full-information performance.

pairwise ranking (Algorithm 2) again converges faster than the alternative bandit algorithms by a factor of 2-4, despite similar learning rates for Algorithms 1 and 2 and a compensa-  tion of smaller learning rates in Algorithm 3 by variance reduction and regularization.


Both the canonical encoder-decoder and its variants with attention mechanism rely heavily on the representation of "meaning", which might not be sufficiently inaccurate in cases in which the system needs to refer to sub-sequences of input like entity names or dates.

the copying mechanism is closer to the rote memorization in language processing of human being, deserving a different modeling strategy in neural network-based models.


(2) for the generic attention-based Seq2Seq model.

there is some minor changes in the y t−1 −→s t path for the copying mechanism.


It has been recently formulated as a Seq2Seq learning problem in (Rush et al., 2015;Hu et al., 2015), which essentially gives abstractive summarization since the summary is generated based on a representation of the document.

extractive summarization extracts sentences or phrases from the original text to fuse them into the summaries, therefore making better use of the overall structure of the original document.


In contrast COPYNET addresses the OOV problem in a more systemic way with an end-to-end model.

as COPY-NET copies the exact source words as the output, it cannot be directly applied to machine translation.


(2015) address a sequential labeling problem in NLU where the fine grained label sets across domains differ.

they assume that there exists a bijective mapping between the coarse and fine-grained label sets across domains.


They learn this mapping using labeled instances from the target domain to reduce the problem to a standard domain adaptation problem (Scenario 2).

this paper caters to multiple source domains with disparate label sets without assuming availability of any labeled data from the target domain or fine-to-coarse label mappings across domains.


The similarity matrix R associates target domain instances to the source domain clusters in proportion to their similarity.

the objective is to select the optimal K source domain clusters that fit the maximum number of target domain instances.


Several models for inducing cross-lingual embeddings have been proposed, each requiring a different form of cross-lingual supervision -some can use document-level alignments (Vulić and Moens, 2015), others need alignments at the sentence (Hermann and Blunsom, 2014; or word level (Faruqui and Dyer, 2014;Gouws and Søgaard, 2015), while some require both sentence and word alignments (Luong et al., 2015).

a systematic comparison of these models is missing from the literature, making it difficult to analyze which approach is suitable for a particular NLP task.


This can be attributed to the fact that BiSkip and BiCVM are trained on parallel sentences, and if two antonyms are present in the same sentence in English, they will also be present together in its French translation.

biCCA uses bilingual dictionary and biVCD use comparable sentence context, which helps in pulling apart the synonyms and antonyms.


Some methods do not have publicly available code (Coulmance et al., 2015;Zou et al., 2013); for others, like BilBOWA , we identified problems in the available code, which caused it to consistently produced results that are inferior even to mono-lingually trained vectors.

11 the models that we included for comparison in our survey are representative of other cross-lingual models in terms of the form of crosslingual supervision required by them.


(2015) is somewhat more constrained than the set we use, there is a good deal of overlap.

their evaluation is performed in the context of relational similarity, and they do not perform clustering or classification on the DIFFVECs.


HLBL and SENNA performed very   The lower V-measure for w2v wiki and GloVe wiki (as compared to w2v and GloVe, respectively) indicates that the volume of training data plays a role in the clustering results.

both methods still perform well above SENNA and HLBL, and w2v has a clear empirical advantage over GloVe.


Figure 2 shows the effect of sample size k on the Chinese-English validation set.

it is clear that BLEU scores consistently rise with the increase of k. we find that a sample size larger than 100 (e.g., k = 200) usually does not lead to significant improvements and increases the GPU memory requirement.


Only the unigram representation is truly open-vocabulary.

the unigram representation performed poorly in preliminary experiments, and we report translation results with a bigram representation, which is empirically better, but unable to produce some tokens in the test set with the training set vocabulary.


For OOVs, the baseline strategy of copying unknown words works well for English→German.

when alphabets differ, like in English→Russian, the subword models do much better.


The English→Russian examples show that the subword systems are capable of transliteration.

transliteration errors do occur, either due to ambiguous transliterations, or because of non-consistent segmentations between source and target text which make it hard for the system to learn a transliteration mapping.


Word pairs, which are one of the most easily accessible features between two text segments, have been proven to be very useful for detecting the discourse relations held between text segments.

because of the data sparsity problem, the performance achieved by using word pair features is limited.


The main reason is their length, as remarked above: Most sequence labeling tasks in NLP (such as most cases of named entity recognition) deal with spans of a few tokens.

the median quotation length on the Penn Attributions Relation Corpus (PARC, Pareti et al.