The fact that the former symbol should be more specific than the latter can be represented using SPEC atoms like dog n 1 dog n. Note that even a deep grammar will not fully disambiguate to semantic predicate symbols, such as WordNet senses, and so dog n 1 can still be consistent with multiple symbols like dog n 1 and dog n 2 in the semantic representation.

unlike the output of a POS tagger, an RMRS symbol that's output by a deep grammar is consistent with symbols that all have the same arity, because a deep grammar fully determines lexical subcategorisation.


This is useful for hybrid systems which exploit shallower analyses when deeper parsing fails, or which try to match deeply parsed queries against shallow parses of large corpora; and in fact, RMRS is gaining popularity as a practical interchange format for exactly these purposes (Copestake, 2003).

rMrS is still relatively ad-hoc in that its formal semantics is not defined; we don't know, formally, what an rMrS means in terms of semantic representations like (2) and (3), and this hinders our ability to design efficient algorithms for processing rMrS.


At this point, we cannot provide an efficient algorithm for testing entailment of RMRS.

we propose the following novel syntactic characterisation as a starting point for research along those lines.


To our knowledge, this is the first time that the regularity of CCG's derivational structures has been exposed.

if we take the word order into account, then the classes of PF-CCG-induced and TAG-induced dependency trees are incomparable; in particular, CCG-induced dependency trees can be unboundedly non-projective in a way that TAG-induced dependency trees cannot.


In this respect, it is the analogue of a TAG derivation tree (in which the lexicon entries are elementary trees), and we just saw that PF-CCG and TAG generate the same tree languages.

ccG and TAG are weakly equivalent (Vijay-Shanker and Weir, 1994), i.e.


We have already argued that this tree can be induced by a TAG.

it contains no two adjacent nodes that are connected by a/a b\a a/a b\b a b\b 1 2 3 4 Figure 8: The divergence between CCG and TAG an edge; and every nontrivial PF-CCG derivation must combine two adjacent words at least at one point during the derivation.


The minimization process serves to shrink down the FSM to an equivalent automata with a suitable size for parsing.

it is usually the case that the size is not small enough to meet the time and memory limitations in parsing.


Otherwise, I 1 must be the concatenation of two intervals I 1v and I 1v 1 with I 1v 2 v and 1v is also adjacent to some interval in v 2 .


Due to previous results (Rambow and Satta, 1999), we know that this is not always possible.

our algorithm may fail even in cases where a binarization exists-our notion of adjacency is not strong enough to capture all binarizable cases.


Handling terminology is an important matter in a translation workflow.

current Machine Translation (MT) systems do not yet propose anything proactive upon tools which assist in managing terminological databases.


Massive amounts of parallel data are certainly available in several pairs of languages for domains such as parliament debates or the like.

having at our disposal a domain-specific (e.g.


We trained a phrase table on TRAIN, using the standard approach.

9 because of the small training size, and the rather huge OOV rate of the translation tasks we address, we did not train translation models on word-tokens, but at the character level.


Better SMT performance could be obtained with a system based on morphemes, see for instance (Toutanova et al., 2008).

since lists of morphemes specific to the medical domain do not exist for all the languages pairs we considered here, unsupervised methods for acquiring morphemes would be necessary, which is left as a future work.


Here there is a clear distinction as raters preferred SMAC to LT, indicating that they did find usefulness in systems that modeled aspects and sentiment.

there are still 25.5% of agreement items where the raters did choose a simple leading text baseline.


Therefore, it is not straightforward to compute the precision (the ratio of correctly detected errors to all error candidates) of this method.

by ignoring variation n-grams of length ≤ 5, Dickinson and Meurers found that 2436 of the 2495 distinct variation nuclei (each nucleus is only counted for the longest n-gram it appears in) were true errors, i.e.


This process can be applied several times: once we have grouped some characters together, they become the new basic unit to consider, and we can re-run the same method to get additional groupings.

we have not seen in practice much benefit from running it more than twice (few new candidates are extracted after two iterations).


The search process can be rewritten as: Given the fact that the number of segmentations f K 1 grows exponentially with respect to the number of characters K, it is impractical to firstly enumerate all possible f K 1 and then to decode.

it is possible to enumerate all the alternative segmentations for a substring of c J 1 , making the utilisation of word lattices tractable in PB-SMT.


(Ma et al., 2007) proposed an approach to improve word alignment by optimising the segmentation of both source and target languages.

the reported experiments still rely on some monolingual segmenters and the issue of scalability is not addressed.


(Dyer et al., 2008) extended this approach to hierarchical SMT systems and other language pairs.

both of these methods require some monolingual segmentation in order to generate word lattices.


The entailment relation has been defined insofar in terms of truth values, assuming that h is a complete sentence (proposition).

there are major aspects of inference that apply to the subsentential level.


Our experiments show that several knowledgebased and corpus-based measures of similarity perform comparably when used for the task of short answer grading.

since the corpusbased measures can be improved by accounting for domain and corpus size, the highest performance can be obtained with a corpus-based measure (LSA) trained on a domain-specific corpus.


In particular, in  kernels for the processing of PASs (in PropBank 1 format (Kingsbury and Palmer, 2002)) extracted from question/answer pairs were proposed.

the relatively high kernel computational complexity and the limited improvement on bag-of-words (BOW) produced by this approach do not make the use of such technique practical for real world applications.


Therefore the answer words should be different and useless to generalize rules for answer classification.

error analysis reveals that although questions are not shared between training and test set, there are common words in the answers due to typical Web page patterns which indicate if a retrieved passage is an incorrect answer, e.g.


Most of the existing studies on linguistic networks, however, focus only on the local structural properties such as the degree and clustering coefficient of the nodes, and shortest paths between pairs of nodes.

although it is a well known fact that the spectrum of a network can provide important information about its global structure, the use of this powerful mathematical machinery to infer global patterns in linguistic networks is rarely found in the literature.


The performance of Lesk (63.89%) is also much higher than in our previous experiments, thanks to the higher chance of finding a 1:1 correspondence between the two sections.

we observed that this does not always hold, as also supported by the better results of CQC.


Word Sense Disambiguation is a large research field (see (Navigli, 2009) for an up-to-date overview).

in this paper we focused on a specific kind of WSD, namely the disambiguation of dictionary definitions.


Many parsing techniques including parameter estimation assume the use of a packed parse forest for efficient and accurate parsing.

they have several inherent problems deriving from the restriction of locality in the packed parse forest.


These learn patterns associated with individual entity classes, making use of many contextual, orthographic, linguistic and external knowledge features.

they rely heavily on large annotated training corpora.


Adjacent characters in the same orthographic class were collapsed.

we distinguish single from multiple characters by duplication.


American, Islamic) generally link to nominal articles.

they are treated by CoNLL   6 Improving Wikipedia performance The baseline system described above achieves only 58.9% and 62.3% on the CoNLL and BBN TEST sets (exact-match scoring) with 3.5million training tokens.


We plan to increase the largest training set the C&C tagger can support so that we can fully exploit the enormous Wikipedia corpus.

we have shown that Wikipedia can be used a source of free annotated data for training NER systems.


(2003) suggest combining various information sources for solving SAT analogy problems.

previous work on compound interpretation has generally used either lexical similarity or relational similarity but not both in combination.


Run Y performs better than E a for 5 of the 25 individual concepts, including NationalPark, for which no instances of national parks or related class labels are available in run E a ; and River, for which relevant instances in the labeled classes in E a , but they are associated to the class label river systems, which is incorrectly linked to the Word-Net concept systems rather than to rivers.

run E a outperforms Y on 12 individual concepts (e.g., Award, DigitalCamera and Disease), and also as an average over all classes (last two rows in Table 5).


Adding another word space model to the ensemble, either a word-based or syntaxbased model, brings down performance.

the addition of the compound model does lead to a clear gain in performance.


For example, the Pinchak and Lin (2006) model is forced to consider a question focus context (such as "X is a city") to be of equal importance to non-focus contexts (such as "X host Olympics").

we have observed that it is more important that candidate X is a city than it hosted an Olympics in this instance.


For example, "explorer Hernando Soto" is a candidate marked appropriate by both annotators to the question "What Spanish explorer discovered the Mississippi River?"

our context database does not include the phrase "explorer Hernando Soto" meaning that only a few features will have non-zero values.


Their focus is on complex-answer questions in addition to the use of a collection of user-generated answers rather than answer typing.

their use of preference ranking mirrors the techniques we describe here in which the relative difference between two candidates at different ranks is more important than the individual candidates.


A user simulation for NLG is very similar, in that it is a predictive model of the most likely next user act.

this user act does not actually change the overall dialogue state (e.g.


For example, the user is less likely to choose an item if there are more than 7 attributes, but the realizer can generate 9 attributes.

in some contexts it might be desirable to generate all 9 attributes, e.g.


Urdu is influenced from Arabic, and can be considered as having three main parts of speech, namely noun, verb and particle (Platts, 1909;Javed, 1981;Haq, 1987).

some grammarians proposed ten main parts of speech for Urdu (Schmidt, 1999).


Statistical approaches usually achieve an accuracy of 96%-97% (Hardie, 2003: 295).

statistical taggers require a large training corpus to avoid data sparseness.


The different scores of the RANDOM assignment for the two languages can be explained by their different branching factors: trees in the German treebank are typically more flat than those in the English WSJ corpus.

note that other settings of our two annotation algorithms do not always obtain better results than random.


at time-point t 2 , as additional acoustic frames have come in, it becomes clear that "forty" is a better hypothesis about the previous frames together with the new ones.


Given the robust performance of MAX when translation scores originated from the same translation model in English to Italian, it is not surprising that it favors the case where all the outputs are scored by the same model ("All tuned").

diversity amongst the system outputs has been shown to be important to the performance of system combination techniques (Macherey and Och, 2007).


the realisations are expected to be more variable than those of a rare type.

another aspect of Exemplar Theory has to be considered, namely entrenchment (Pierrehumbert, 2001;Bybee, 2006).


Baumann (2006) finds deaccentuation to be the most preferred realisation for givenness in his experimental phonetics studies on German.

baumann (2006) points out that H+L * has also been found as a marker of givenness in a German corpus study.


However, with respect to entrenchment (Pierrehumbert, 2001;Bybee, 2006), i.e.

the idea that frequently occurring behaviours undergo processes of entrenchment, in Experiment 1 one might expect to see greater similarity in the realisations of L * H. it is important to note that while tokens of L * H are not particularly similar to each other (the bulk of the distribution is around zero (see figure 1)), they are not too dissimilar either.


ABIR, compared to the image-only approach of CBIR, offers a practical advantage in that queries can be more naturally specified by a human user (Inoue, 2004).

manually annotating biomedical images is a laborious and subjective task that often leads to noisy results.


In the NUMBERS system, information is only allowed to flow from left to right, which means that the LB may be regarded as the input buffer and the RB as the output buffer.

in the general model, information may flow in both directions.


Like them, we rely on the presence or absence of cohesive links between the words in a text.

unlike Hirst and St-Onge we do not require a hand-crafted resource like WordNet or Roget's Thesaurus; our approach is knowledgelean.


At first glance, this seems to be quite natural.

when we compared this alternative method with the aforementioned approximation on search steps, we found that it worked worse than the latter, in terms of performance and speed.


NETE discovery from comparable corpora using time series and transliteration model was proposed in (Klementiev and Roth, 2006), and extended for NETE mining for several languages in (Saravanan and Kumaran, 2007).

such methods miss vast majority of the NETEs due to their dependency on frequency signatures.


For example, a template shown in Section 5, A MAY WEAR A CRASH HELMET, was supported by just two sentences in the BNC.

based on those two observations we were able to conclude that usually If something wears a crash helmet, it is probably a male person.


In initial experiments, we used relative frequencies.

for instance, the trigram filter would allow any tuple g, r i−2 , r i−1 , r i for some constant threshold τ , provided: we found that filters are more effective (and require much less space -see below), which simply require that every step has been observed often enough in the training data: In particular, the case where τ = 0 gave surprisingly good results.


On the one hand, the Alpino Treebank might not be a reliable test set for the Alpino parser, because it has been used quite intensively during the development of various components of the system.

we might regard the experiments in the previous section as development experiments from which we learn the best parameters of the approach.


They report important efficiency gains (the parser is about three times faster), coupled with a mild reduction of coverage (5% loss).

to our approach in which no manual annotation is required, Rayner and Carter (1996) report that for each sentence in the training data, the best parse was selected manually from the set of parses generated by the parser.


However, only 64 of the 100 test items in the McRae data set contain verbs that are attested in the FrameNet corpus, 8 of which involve an unattested verb sense.

the only requirement for the exemplar-based model to be able to make its predictions is that the similarities between the verbs and the nouns in the target exemplars and the memory exemplars can be computed.


Their approach hinges on the fact that similes exploit stereotypes to draw out the salient properties of a target, thereby allowing rich descriptions of those stereotypes to be easily acquired, e.g., that snowflakes are pure and unique, acrobats are agile and nimble, knifes are sharp and dangerous, viruses are malicious and infectious, and so on.

because they find that almost 15% of their web-harvested sim-iles are ironic (e.g., "as subtle as a rock", "as bulletproof as a sponge-cake", etc.


), they filter irony from these associations by hand, to yield a sizable database of stereotypical attributions that describes over 6000 noun concepts in terms of over 2000 adjectival properties.

because Veale and Hao's data directly maps stereotypical properties to simile vehicles, it does not provide a parent category for these vehicles.


Both the WordNet and ConceptNet seeds achieve comparable accuracies of 68% and 67% respectively after 5 cycles of bootstrapping, which compares well with the accuracy of 62.7% achieved by Poesio and Almuhareb.

the simile seed clearly yields the best accuracy of 84.3%, which also exceeds the accuracy of 66.4% achieved by Poesio and Almuhareb when using both values and attributes (such as Temperature, Color, etc.)


the features used in this method are only suitable for parallel corpora as the measurement is mainly based on structural similarity.


Supervised methods such as Support Vector Machine (SVM) and Maximum Entropy (ME) estimate the weight of each feature based on training data which are then used to calculate the final score.

these supervised learning-based methods may not be applicable to our proposed issue as we are motivated to build a language independent unsupervised system.


An edge is penalised if it is improbable that the head takes on yet another modifier, say in the example of an attachment to a preposition whose argument position has already been filled.

accounting for argument positions makes an edge weight dynamic and dependent on surrounding tree context.


Should an approach chance upon an alternative grammatical ordering, it would penalised.

all algorithms and baselines compared would suffer equally in this respect, and so this will be less problematic when averaging across multiple test cases.


The margin narrows between the CLE algorithm and the LMO baseline.

the AB algorithm still out-performs all other approaches by 7 BLEU points, highlighting the benefit in modelling dependency relations.


The results for the two systems are very similar since they use same kinds of features.

with Markov logic, it is easy to add predicates and formulas to allow joint inference.


Increasing the expressiveness of the argument key representation by flagging intransitive constructions would distinguish that pair of arguments.

we keep this particular representation, in part to compare with the previous work.


While RTE is outside our present scope, we do focus on QP entailment as Natural Logic does.

our evaluation differs from Chambers et al.


We feed each potential entailment pair to SVM by concatenating the two vectors representing the antecedent and consequent expressions.

2 for efficiency and to mitigate data sparseness, we reduce the dimensionality of the semantic vectors to 300 columns using Singular Value Decomposition (SVD) before feeding them to the classifier.


This success is especially impressive given our challenging training and testing regimes.

to the first study, now SVM AN |= N , the classifier trained on the AN |= N data set, and balAPinc perform no better than the baselines.


A major focus of current work in distributional models of semantics is to construct phrase representations compositionally from word representations.

the syntactic contexts which are modelled are usually severely limited, a fact which is reflected in the lexical-level WSD-like evaluation methods used.


One uses the WordSim-353 dataset (Finkelstein et al., 2002), which contains human word pair similarity judgments that semantic models should reproduce.

the word pairs are given without context, and homography is unaddressed.


Parsing accuracy has been used as a preliminary evaluation of semantic models that produce syntactic structure (Socher et al., 2010;Wu and Schuler, 2011).

syntax does not always reflect semantic content, and we are specifically interested in supporting syntactic invariance when doing semantic inference.


there are apparently verbless sentences in Hungarian: A ház nagy (the house big) "The house is big".

in other tenses or moods, the copula is present as in A ház nagy lesz (the house big will.be) "The house will be big".


In Hungarian, they are ambiguous between being adverbs and conjunctions and it is mostly their conjunctive uses which are problematic from the viewpoint of parsing.

these words have an important role in marking the information structure of the sentence: they are usually attached to the element in focus position, and if there is no focus, they are attached to the verb.


On the other hand, these words have an important role in marking the information structure of the sentence: they are usually attached to the element in focus position, and if there is no focus, they are attached to the verb.

sentences with or without focus can have similar word order but their stress pattern is different.


', where it is the subject that follows the verb for stylistic reasons.

in Hungarian, morphological information is of help in such sentences, as it is not the position relative to the verb but the case suffix that determines the grammatical role of the noun.


locative adverbs were labeled as ADV/MODE.

the frequency rate of this error type is much higher in English than in Hungarian, which may be related to the fact that in the English corpus, there is a much more balanced distribution of adverbial labels than in the Hungarian one (where the categories MODE and TLOCY are responsible for 90% of the occurrences).


Machine learning drives the process of deciding among alternative candidate splits, i.e., feature information can draw on full structural information for the entire material in the span under consideration.

due to the dynamic programming approach, the features cannot use arbitrarily complex structural configurations: otherwise the dynamic programming chart would have to be split into exponentially many special states.


As such the approach is suitable to detect paraphrases that describe the relation between two entities in documents.

the paper does not describe how the mined paraphrases can be linked to questions, and which paraphrase is suitable to answer which question type.


(Shen and Klakow, 2006) also describe a method that is primarily based on similarity scores between dependency relation pairs.

their algorithm computes the similarity of paths between key phrases, not between words.


For paragraph retrieval we use the same approach as for evaluation set 1, see Section 7.1.

in more than 20% of the cases, this method returns not a single paragraph that contains both the answer and at least one question keyword.


Normally, these pieces of information (i.e., nuggets) explain different facets of the definiendum (e.g., "ballet choreographer" and "born in Bordeaux"), and the main idea consists in projecting the acquired nuggets into the set of answer candidates afterwards.

the performance of this category of method falls into sharp decline whenever few or no coverage is found across KBs (Zhang et al., 2005;Han et al., 2006).


On the one hand, we want our model to produce a "good" translation (well-formed and transmitting the information contained in the source query) of an input query.

we want to obtain good retrieval performance using the proposed translation.


One of the reasons being the poor diversity of the Nbest list of the translations.

we be-lieve that this approach has more potential in the context of query translation.


Ideally, we would have liked to combine the two approaches we proposed: use the querygenre-tuned model to produce the Nbest list which is then reranked to optimize the MAP score.

it was not possible in our experimental settings due to the small amount of training data available.


Combining monolingually estimated reordering and phrase table features (M/M) yields a total gain of 13.5 BLEU points, or over 75% of the BLEU score loss that occurred when we dropped all features from the phrase table.

these results use "monolingual" corpora which have practically identical phrasal and temporal distributions.


This is even enforced due to the availability of toolboxes such as Moses (Koehn et al., 2007) which make it possible to build translation engines within days or even hours for any language pair provided that appropriate training data is available.

this reliance on training data is also the most severe limitation of statistical approaches.


A general condition for the pivot approach is to assume independent training sets for both translation models as already pointed out by (Bertoldi et al., 2008).

to research presented in related work (see, for example, (Koehn et al., 2009)) this condition is met in our setup in which all data sets represent different samples over the languages considered (see section 4).


The reason for this might be that the IBM models can handle noise in the training data more robustly.

in terms of unknown words, WFST-based alignment is very competitive and often the best choice (but not much different from the best IBM based models).


Rescoring of N-best lists, on the other hand, does not have a big impact on our results.

we did not spend time optimizing the parameters of N-best size and interpolation weight.


For the translation into English, the in-domain language model helps a little bit (similar resources are not available for the other direction).

having the strong indomain model for translating to (and from) the pivot language improves the scores dramatically.


Both techniques span through two orthogonal criteria when selecting bilingual sentences from the available pool: avoiding to introduce a bias in the original data distribution, and increasing the informativeness of the corpus.

we prove that among all possible subsets from the sentence pool, there is at least a small one that yields large improvements (up to 10 BLEU points) with respect to a system trained with all the data.


The same partitions as in the IWSLT2010 evaluation task (Paul et al., 2010)    effective technique that is commonly used is to reproduce out-of-vocabulary words from the source sentence in the target hypothesis.

invariable n-grams are usually infrequent as well, which implies that the infrequent n-grams technique would select sentences containing such ngrams, even though they do not provide further information.


This technique could be also applied to promote the performance of the system built by means of BSS.


As they do not specify which data they used for their held-out test set, we cannot perform a direct comparison.

our feature set is nearly a superset of their best feature set, and their result lies well within the range of results seen in our cross-validation folds.


The motivations behind that are: • Since the named entities have a tree structure, it is reasonable to use a solution coming from syntactic parsing.

preliminary experiments using such approaches gave poor results.


Since we are dealing with noisy data, the hardest part of the task is indeed to annotate components on words.

since entity trees are relatively simple, at least much simpler than syntactic trees, once entity components have been annotated in a first step, for the second step, a complex model is not required, which would also make the processing slower.


Intuitively, this representation is effective since entities annotated directly on words provide also the entity of the parent node.

this representation increases drastically the number of entities, in particular the number of components, which in our case are the set of labels to be learned by the CRF model.


Our task is similar to task A and C of TempEval-1 (Verhagen et al., 2007) in the sense that we attempt to identify temporal relation between events and time expressions or document dates.

we do not use a restricted set of events, but focus primarily on a single temporal relation t link instead of named relations like BE-FORE, AFTER or OVERLAP (although we show that we can incorporate these as well).


We plot the Cumulative distribution of frequency (CDF) of the ranks (as percentages in the mixed pools) of false negatives in figure 3.

we took similar steps for the spurious ones (false positives) and plot them in figure 3 as well (they are ranked by model-predicted probabilities of being negative).


That means the missing examples are lexically, structurally or semantically similar to correct examples, and are distinguishable from the true negative examples.

the distribution of false positives (spurious examples) is close to uniform (flat curve), which means they are generally indistinguishable from the correct examples.


It relies on the assumption that the annotators annotated all relation mentions and missed no (or very few) examples.

this is not true for training on a single-pass annotation, in which a significant portion of relation mentions are left not annotated.


Because of this, we will learn a model that can predict approximately good summaries y i from x i .

we believe that most of the score difference between manual summaries and y i (as explored in the experiments section) is due to it being an extractive summary and not due to greedy construction.


On one hand, this means that one can train MT systems on S → T data only, at the expense of only a minor loss in quality.

it is obvious that the T → S component also contributes to translation quality.


Source Cependant, je pense qu'il est prématuré de le faire actuellement,étant donné que le ministre a lancé cette tournée.

baseline I think it is premature to the right now, since the minister launched this tour.


Berend (2011) performs a form of pro/con summarization that does not rely on aspects.

most of the problems of aspect-based pro/con summarization also apply to this paper: no differentiation between good and bad reasons, the need for human labels to train a classifier, and inferior readability compared to a well-formed sentence.


The most common use of crowdsourcing in NLP is to have workers label a training set and then train a supervised classifier on this training set.

we use crowdsourcing to directly evaluate the relative quality of the automatic summaries generated by the unsupervised method we propose.


We are not assuming that all important words in the supporting sentence are nominal; the verb will be needed in many cases to accurately convey the reason for the sentiment expressed.

it is a fairly safe assumption that part of the information is conveyed using noun phrases since it is difficult to convey specific information without using specific noun phrases.


It is comparable in size to only two known lexicons: WORDNET-AFFECT (Strapparava and Valitutti, 2004) and EMOLEX (Mohammad and Turney, 2010).

to the development of these lexicons, we do not restrict our annotators to a particular set of emotions.


In particular, B&L associated a lower rank with automatically created permutations of a source document, and learned a model to discriminate an original text from its permutations (see Section 3.1 below).

coherence is matter of degree rather than a binary distinction, so a model based only on such pairwise rankings is insufficiently fine-grained and cannot capture the subtle differences in coherence between the permuted documents.


Their extended model significantly improved ranking accuracies on the same two datasets used by Barzilay and Lapata (2008) as well as on the Wall Street Journal corpus.

while enriching or modifying the original features used in the standard model is certainly a direction for refinement of the model, it usually requires more training data or a more sophisticated feature representation.


We implement our multiple-rank model with full coreference resolution using Ng and Cardie's coreference resolution system, and entity extraction approach as described above -the Coref-erence+ condition.

as argued by Elsner and Charniak (2011), to better simulate the real situations that human readers might encounter in machine-generated documents, such oracular information should not be taken into account.


Rather than rely on an ad-hoc summation of PMIs, they apply language modeling techniques (specifically, a smoothed 5-gram model) over the sequence of events in the collected chains.

they only tested these language models on sequencing tasks (e.g.


Third, we have discussed why Recall@N is a better and more consistent evaluation metric than Average rank.

both evaluation metrics suffer from the strictness of the narrative cloze test, which accepts only one event being the correct event, while it is sometimes very difficult, even for humans, to predict the missing events, and sometimes more solutions are possible and equally correct.


We think such an approach is useful because it forms a natu-ral baseline for the task (as it does in many other tasks such as named entity tagging and language modeling).

story structure is seldom strictly linear, and future work should consider models based on grammatical or discourse links that can capture the more complex nature of script events and story structure.


An AUC curve that is at least as good as a second curve at all points, is said to dominate it and indicates that the first classifier is equal or better than the second for all plotted values of the parameters, and all cost ratios.

aUC being greater for one classifier than another does not have such a property -indeed deconvexities within or intersections of ROC curves are both prima facie evidence that fusion of the parameterized classifiers will be useful (cf.


Uebersax (1987), Hutchison (1993) and Bonnet and Price (2005) each compare Kappa and Correlation and conclude that there does not seem to be any situation where Kappa would be preferable to Correlation.

all the Kappa and Correlation variants considered were symmetric, and it is thus interesting to consider the separate regression coefficients underlying it that represent the Powers Kappa duals of Informedness and Markedness, which have the advantage of separating out the influences of Prevalence and Bias (which then allows macroaveraging, which is not admissable for any symmetric form of Correlation or Kappa, as we will discuss shortly).


The other Kappa and Correlations are more complex (note the demoninators in Eqns 5-9) and how they might be meaningfully macro-averaged is an open question.

microaveraging can always be done quickly and easily by simply summing all the contingency tables (the true contingency tables are tables of counts, not probabilities, as shown in Table 1).


It is very helpful to build an automatic system to suggest latest information a user would be interested in.

unlike formal news media, user generated content in forums is usually less organized and not well formed.


Casual forums like "Wow gaming" have much more posts in each thread.


In some previous work, clustering methods were used to partition users into several groups, Then, predictions were made using information from users in the same group.

in the case of thread recommendation, we found that users' interest does not form clean clusters.


This way the confidence of "how likely" an item is interesting is preserved.

the downside is that the two different systems have different calibration on its posterior probability, which could be problematic when directly adding them together.


This observation is consistent with previous work (e.g., (Pavlov et al., 2004)).

we found that in "Fitness Forum", the performance degrades with normalization.


(red=0 being higher on the ranked list and green being lower) may be interested in a larger variety of topics and thus the user distribution in different topics is not very obvious.

people in the gaming forum are more specific to the topics they are interested in.


Arguably yes: Today the Acme Pencil Factory celebrated its one-billionth pencil.

such a contrived example is unnatural because unlike birthday, pencil lacks a strong association with celebrate.


In both sentences the agent of opened, namely Pat, must be capable of opening somethingan informative constraint on Pat.

knowing that the grammatical subject of opened is Pat in the first sentence and the door in the second sentence tells us only that they are nouns.


We quantify the selectional preference for a relative r to instantiate a relation R of a target t as the probability Pr(r | t, R), estimated as follows.

by the definition of conditional probability: We care only about the relative probability of different r for fixed t and R, so we rewrite it as: We use the chain rule: and notice that t is held constant: We estimate the second factor as follows: We calculate the denominator freq(t) as the number of N-grams in the Google N-gram corpus that contain t, and the numerator freq(t, r) as the number of N-grams containing both t and r. To estimate the factor Pr(R | r, t) directly from a corpus of text labeled with grammatical relations, it would be trivial to count how often a word r bears relation R to target word t. the results would be limited to the words in the corpus, and many relation frequencies would be estimated sparsely or missing altogether; t or r might not even occur.


To transform Pr(R | r, t) into a form we can estimate, we first apply the definition of conditional probability: To estimate the numerator Pr(R, t, r), we first marginalize over the POS N-gram p: We expand the numerator using the chain rule: Cancelling the common factor yields: We approximate the first term Pr(R | p, t, r) as Pr(R | p), based on the simplifying assumption that R is conditionally independent of t and r, given p. In other words, we assume that given a POS N-gram, the target and relative words t and r give no additional information about the probability of a relation.

their respective positions i and j in the POS N-gram p matter, so we condition the probability on them: As Figure 1 shows, we estimate Pr(R | p, i, j) by abstracting the labeled corpus into POS N-grams.


(2010) drew (R, t) pairs from each of five frequency bands in the entire British National Corpus (BNC): 50-100 occurrences; 101-200; 201-500; 500-1000; and more than 1000.

we use only half of BNC as our test corpus, so to obtain a comparable test set, we drew 20 (R, t) pairs from each of the corresponding frequency bands in that half: 26-50 occurrences; 51-100; 101-250; 251-500; and more than 500.


(2010) reported the state-ofthe-art method we used as our EPP baseline.

to prior work that explored various solutions to the generalization problem, we don't so much solve this problem as circumvent it.


Further, Wiktionary provides relations to other words, e.g., in the form of synonyms, antonyms, hypernyms, hyponyms, holonyms, and meronyms.

to GermaNet, the relations are (mostly) not disambiguated.


In order to eliminate such noise, manual post-editing is required.

such post-editing is within acceptable limits: it took an experienced research assistant a total of 25 hours to hand-correct all the occurrences of sense-annotated target words and to manually sense-tag any missing target words for the four text types.


By focusing on web-based data, their work resembles the research described in the present paper.

the underlying harvesting methods differ.


for a similar approach and results for German texts).

they failed to find an effect for lexicalized surprisal, over and above forward transitional probability.


The resulting χ 2 statistic indicates the extent to which each surprisal estimate accounts for RT, and can thus serve as a measure of the psychological accuracy of each model.

this kind of analysis assumes that RT for a word reflects processing of only that word, but spill-over effects (in which processing difficulty at word w t shows up in the RT on w t+1 ) have been found in self-paced and natural reading (Just et al., 1982;Rayner, 1998;Rayner and Pollatsek, 1987).


Both for the lexicalized and unlexicalized versions, these effects persisted whether surprisal for the previous or current word was taken as the independent variable.

the effect size was much larger for previous surprisal, indicating the presence of strong spill-over effects (e.g.


If particular syntactic categories were contributing to the overall effect of surprisal more than others, including such random slopes would lead to additional variance being explained.

this was not the case: inclusion of by-POS random slopes of surprisal did not lead to a significant improvement in model fit (PSG: χ 2 (1) = 0.86, p = 0.35; RNN: χ 2 (1) = 3.20, p = 0.07).


This is not surprising, given that while the unlexicalized models only have access to syntactic sources of information, the lexicalized models, like the human parser, can also take into account lexical cooccurrence trends.

when a training corpus is not large enough to accurately capture the latter, it might still be able to model the former, given the higher frequency of occurrence of each possible item (POS vs. word) in the training data.


8 In terms of peak accuracies, EM gives a slightly better result than the spectral method (80.51% for EM with 15 states versus 79.75% for the spectral method with 9 states).

the spectral algorithm is much faster to train.


The same dependency pattern might be constructed for multiple (positive or negative) entity pairs.

if it is constructed for both positive and negative pairs, it has to be discarded from the pattern list.


• Usually, the e-walk features are constructed using dependency types between {governor of X, node X} and {node X, dependent of X}.

we also extract e-walk features from the dependency types between any two dependents and their common governor (i.e.


One approach is to capture local symmetry of conjuncts.

this approach fails in VP and sentential coordinations, which can easily be detected by a grammatical approach.


It is therefore natural to think that considering both the syntax and local symmetry of conjuncts would lead to a more accurate analysis.

it is difficult to consider both of them in a dynamic programming algorithm, which has been often used for each of them, because it explodes the computational and implementational complexity.


Suppose we have an Italian word arcipelago, and we would like to detect its correct English translation (archipelago).

after the TI+Cue method is employed, and even after the symmetrizing re-ranking process from the previous step is used, we still acquire a wrong translation candidate pair (arcipelago, island).


1997) and Tiger (Brants et al., 2002) corpora, or those that can be extracted from traces such as in the Penn treebank (Marcus et al., 1993) annotation.

the computational complexity is such that until now, the length of sentences needed to be restricted.


5 Importantly, in the first case there is no improvement on fan-out or parsing complexity, while in the head-driven case there is a minimal improvement because of a single production with parsing complexity 15 without optimal binarization.

the optimal binarizations might still have a significant effect on the average case complexity, rather than the worst-case complexities.


Splitting discontinuous nodes for the coarse grammar introduces new nodes, so obviously we need to binarize after this transformation.

the coarse-to-fine approach requires a mapping between the grammars, so after reversing the transformation of splitting nodes, the resulting discontinuous trees must be binarized (and optionally Markovized) in the same manner as those on which the fine grammar is based.


We show that standard intrinsic metrics such as F-score alone do not predict the outcomes well.

we can build predictive performance functions that account for up to 50% of the variance in learning gain by combining features based on standard evaluation scores and on the confusion matrix entries.


We then show that standard evaluation metrics do not serve as good predictors of system performance for the system we evaluated.

adding confusion matrix features improves the predictive model (Section 4).


Its overall accuracy is 43%, the same as BEETLE II.

this is obviously not a good choice for a tutoring system, since students who make mistakes will never get tutoring feedback.


Since only a small number of features was included, this limits the applicability of the model we derived from this data set to the systems which make similar types of confusions.

it is still interesting to investigate whether confusion probabilities provide additional information compared to standard evaluation metrics.


As we are using text data, such intonational and prosodic cues are unavailable, as are the other rich sources of emotional cues we obtain from gesture, posture and facial expression in face-toface communication.

the prevalence of online text-based communication has led to the emergence of textual conventions understood by the users to perform some of the same functions as these acoustic and non-verbal cues.


The sad class also seems well distinguished when using hashtags as labels, although less so when using emoticons.

other emotion classes show a surprisingly high cross-class performance in many cases -in other words, they are producing disappointingly similar classifiers.


For the three classes happy, sad and perhaps anger, models trained using emoticon labels do a reasonable job of distinguishing classes in hashtag-labelled data, and vice versa.

for the other classes, discrimination is worse.


Results for sad and anger are reasonable, and provide a baseline for further experiments with more advanced features and classification methods once more manually annotated data is available for these classes.

hashtags give much better performance with these classes than the (perhaps vague or ambiguous) emoticons.


Second, to apply evidence collected from some annotations to a new annotation, the evidence must generalize across annotations.

collected evidence or statistics may vary widely across annotations.


Observing that 90% of all descendant instances of the concept 'Actors' match an annotation acted-in constitutes strong evidence that 'Actors' is a good concept for acted-in.

observing that only 0.09% of all descendant instances of the concept 'Football Teams' match won-super-bowl should not be as strong negative evidence as the percentage suggests.


As explained in Section 4.1, we used offline recognition results in our evaluation.

the results would be identical if we were to use the incremental speech recognition output of In-proTK directly.


This might be due to (i) English having more short nouns or verbs than German that are more likely to be confused with each other, and (ii) the English Wikipedia being known to attract a larger amount of non-native editors which might lead to higher rates of real-word spelling errors.

this issue needs to be further investigated e.g.


Given that many measures of contextual fitness allow at most one edit, many naturally occurring errors will not be detected.

allowing a larger edit distance enormously increases the search space resulting in increased run-time and possibly decreased detection precision due to more false positives.


However, allowing a larger edit distance enormously increases the search space resulting in increased run-time and possibly decreased detection precision due to more false positives.

to the quite challenging process of mining naturally occurring errors, creating artificial errors is relatively straightforward.


Thus, we also tested a trigram model based on Wikipedia.

it is much smaller than the Web model, which leads us to additionally testing smaller Web models.


Max and Wisniewski (2010) used similar techniques to create a dataset of errors from the French Wikipedia.

they target a wider class of errors including non-word spelling errors, and their class of real-word errors conflates malapropisms as well as other types of changes like reformulations.


the Cambridge Learner Corpus (Nicholls, 1999).

annotation of errors is difficult and costly (Rozovskaya and Roth, 2010), only a small fraction of observed errors will be real-word spelling errors, and learners are likely to make dif-ferent mistakes than proficient language users.


(1991) when evaluated on a corpus of artificial errors based on the WSJ corpus.

the results are not directly comparable, as Mays et al.


Translation models trained with weighted counts have been discussed before, and have been shown to outperform uniform ones in some settings.

researchers who demonstrated this fact did so with arbitrary weights (e.g.


As to the domain adaptation experiments, weights optimized through perplexity minimization are significantly better in the majority of cases, and never significantly worse, than uniform  weights.

12 the difference is smaller for the experiments with an adapted language model than for those with an out-of-domain one, which confirms that the benefit of language model adaptation and translation model adaptation are not fully cumulative.


14 A pessimistic interpretation of the results would point out that performance gains compared to the best baseline system are modest or even inexistent in some settings.


Monolingual SCF integration based on a common representation format has already been addressed by King and Crouch (2005) and just recently by  and .

neither King and Crouch (2005) nor  or  make use of existing standards in order to create a uniform SCF representation for lexicon merging.


DCs, and these DCs are not linked to any DCR: in the Syntax Extension, the standard only provides 7 class names, see Figure 1), complemented by 17 example attributes given in an informative, non-binding Annex F. These are by far not sufficient to represent the fine-grained SCFs available in such largescale lexicons as VerbNet.

the Syntax part of Subcat-LMF comprises 58 DCs that are properly linked to ISOCat DCs; a number of DCs were missing in ISOCat, so we entered them ourselves.


With respect to text documents, slightly modified passages in these documents can be identified using fingerprints (Potthast and Stein, 2008).

for data fields which contain natural language such as the assignee name field, string similarity metrics (Cohen et al., 2003) as well as spelling correction technology are exploited (Damerau, 1964;Monge and Elkan, 1997).


(2010) use a ratio of positive and negative word counts on Twitter, Kramer (2010) counts lexicon words on Facebook, and Thelwall (2011) uses the publicly available Sen-tiStrength algorithm to make weighted counts of keywords based on predefined polarity strengths.

to lexicons, many approaches instead focus on ways to train supervised classifiers.


This could explain the behavior seen in Figure 3 in which both the positive and negative sentiment scores rise over time.

further experimentation did not rectify this pattern.


The T form (German du, French tu) is employed towards friends or addressees of lower social standing, and implies solidarity or lack of formality.

english used to have a T/V distinction until the 18th century, using you as V pronoun and thou for T. in contemporary english, you has taken over both uses, and the T/V distinction is not marked anymore.


There is a large body of work on the T/V distinction in (socio-)linguistics and translation studies, covering in particular the conditions governing T/V usage in different languages  and the difficulties in translation (Ardila, 2003;Künzli, 2010).

many observations from this literature are difficult to operationalize.


We originally intended to transfer T/V labels between German and English word-aligned pronouns.

we pronouns are not necessarily translated into pronouns; additionally, we found word alignment accuracy for pronouns to be far from perfect, due to the variability in function word translation.


To investigate this hypothesis, we trained models with the best parameters as before (8-sentence direct speech context, words as features).

this time we trained novel-specific models, splitting each novel into 50% training data and 50% testing data.


In future work, we will attempt to learn social networks from novels (Elson et al., 2010), which should provide constraints on all instances of communication between a speaker and an addressee.

the big -and unsolved, as far as we know -challenge is to automatically assign turns to interlocutors, given the varied and often inconsistent presentation of direct speech turns in novels.


A few projects attempt to represent story struc-ture in terms of both characters and their emotional states.

they operate at a very detailed level and so can be applied only to short texts.


Our intuition is that the simpler method described in , which merges each mention to the most recent possible coreferent, must be even more so.

due to the expense of annotation, we make no attempt to compare these methods directly.


Dreyer and Eisner (2011) propose an infinite Diriclet mixture model for capturing paradigms.

they do not address learning of hierarchy.


Once the probability distributions G = {G s , G m } are drawn from both Dirichlet processes, words can be generated by drawing a stem from G s and a suffix from G m .

we do not attempt to estimate the probability distributions G; instead, G is integrated out.


if the word is analysed as s 2 m 2 (e.g.


Although the dataset provides word frequencies, we have not used any frequency information.

for training our model, we only chose words with frequency greater than 200.


In our experiments, we used dataset sizes of 10K, 16K, 22K words.

for final evaluation, we trained our models on 22K words.


As the prefix pre-appears at the beginning of words, it is identified as a stem.

identifying pre-as a stem does not yield a change in the morphological analysis of the word.


Clearly, adding tree cutting would improve the accuracy of the segmentation and will help us identify paradigms with higher accuracy.

the segmentation accuracy obtained without using tree cutting provides a very useful indicator to show whether this approach is promising.


At the largest training data sizes, modeling all 4 features together results in the best predictions of inflection.

using 4 separate models is worth this minimal decrease in performance, since it facilitates experimentation with the CRF framework for which the training of a single model is not currently tractable.


The BLEU score of the CRF on test is 14.04, which is low.

the system produces 19 compound types which are in the reference but not in the parallel data, and therefore not accessible to other systems.


miniature camera or miniature cameras does not occur in the training data, and so there is no appropriate phrase pair in any system (baseline, inflection, or inflection&compound-splitting).

our system with compound splitting has learned from split composita that English minia-   There has been a large amount of work on translating from a morphologically rich language to English, we omit a literature review here due to space considerations.


(2010), Clifton and Sarkar (2011), and others are primarily concerned with using morpheme segmentation in SMT, which is a useful approach for dealing with issues of word-formation.

this does not deal directly with linguistic features marked by inflection.


The F, Lemma and LMM improve over the baseline in terms of unseen words for both MLE and Yamcha techniques.

for seen words, these systems do worse than or equal to the baseline when the MLE technique is used.


The proportion of ambiguity errors is almost identical for gender, number and rationality.

rationality overall is the biggest cause of error, simply due to its higher degree of ambiguity.


The table shows that each framework assigns a single role, such as Arg0 and Agent, to each syntactic argument.

we can acquire information from this sentence that John is an agent of the throwing event (the "Affection" row), as well as a source of the movement event of the ball (the "Movement" row).


This framework was also used for text generation (Habash et al., 2003).

the problem of multiple-role assignment was not completely solved on the resource.


The funcevents in the semantic structure of a verb.

generally, a verb focuses on one of those events and this makes a semantic variation among verbs such as buy, sell, and pay as well as difference of syntactic behavior of the arguments.


For single-role assignment, Theme, in our sense, in action verbs is always duplicated with Actor/Patient.

lCS strictly divides a function for action and change; therefore the duplicated Theme is correctly annotated.


Unfortunately, we cannot discover what each argument of the semantic predicates exactly means since the definition of each predicate is not   tic properties of roles using a predicate decomposition approach, but defines specific roles for each conceptual event/state to represent a specific background of the roles in the event/state.

at the same time, FrameNet defines several types of parent-child relations between most of the frames and between their roles; therefore, we may say FrameNet implicitly describes a sort of decomposed property using roles in highly general or abstract frames and represents the inheritance of these semantic properties.


(2009), the initialization of our method depends on the correlation between DEOs and negative polarity items (NPIs).

our method trusts the initialization more and aggressively separates likely DEOs from spurious distractors and other words, unlike distillation, which we show to be equivalent to one iteration of EM prior re-estimation.


DLD09 are to be commended for having identified a crucial component of inference that nevertheless lends itself to a classification-based ap-proach, as we will show.

as noted by DL10, the performance of the distillation method is mixed across languages and in the semi-supervised bootstrapping setting, and there is no mathematical grounding of the heuristic to explain why it works and whether the approach can be refined or extended.


This method approximately captures Ladusaw's hypothesis by highly ranking words that appear in NPI contexts more often than would be expected by chance.

the problem with this approach is that DEOs are not the only words that co-occur with NPIs.


While originally developed in the bilingual context of Statistical Machine Translation, nothing prevents building such models on monolingual corpora.

in order to build reliable models, it is necessary to use enough training material including minimal redundancy of words.


Figure 2 shows how performance varies on French with number of training examples for various feature configurations.

some paraphrase types will require integration of more complex knowledge, as is the case, for instance, for paraphrase pairs involving some anaphora and its antecedent (e.g.


We apply this method on the opposite translation direction, thus having English as a source language and German as a target language.

we cannot simply invert the reordering rules which are applied on German as a source language in order to reorder the English input.


This ensures the correct placement of German verbs.

this does not ensure that the German verb forms are correct because of highly ambiguous English verbs.


Furthermore, the reordering rules are applied on a clause not allowing for movements across the clause boundaries.

we also showed that in some cases, the main verbs may be moved after the succeeding subclause.


If we take gold-standard edges as positive examples, and non-gold-standard edges as negative examples, the goal of the training problem can be viewed as finding a large separating margin between the scores of positive and negative examples.

it is infeasible to generate the full space of negative examples, which is factorial in the size of input.


From the perspective of correctness, it is unnecessary to find a margin between the sub-edges of e + and those of e − , since both are gold-standard edges.

since the score of an edge not only represents its correctness, but also affects its priority on the agenda, promoting the sub-edge of e + can lead to "easier" edges being constructed before "harder" ones (i.e.


(2003), focus on non-interactive one-shot instruction discourses.

commercially successful car navigation systems continuously monitor whether the driver is following the instructions and provide modified instructions in real time when necessary.


We first assume that episode boundaries occur when the street name changes from one segment to the next.

staying on the road may involve a driving maneuver (and therefore a decision point) as well, e.g.


We found that male users tended to look more at the navigation screen in the VCP condition than in B, although the difference is not statistically significant.

female users looked at the navigation screen significantly fewer times (t(5) = 3.2, p < 0.05, t-test for dependent samples) and for significantly shorter amounts of time (t(5) = 3.2, p < 0.05) in the VCP condition than in B.


The evaluation confirmed the importance of interactive real-time NLG for navigation, and we therefore see this as a key direction of future work.

it would be desirable to generate more complex referring expressions ("the tall church").


While this type of lexical chain is described as "reiteration without identity of referents" by Morris and Hirst (1991), it would not be captured in Centering since this is not a case of strict coreference.

lexical chains do not capture types of reiterated discourse referents that have distinct morpho-syntactic realisations, e.g.


The sentence-external features lead to an improvement when combined with the language-model based ranking.

this improvement is leveled out in the BaseSyn model.


In this experiment, we find an effect of the sentence-external features over the simple sentence-internal baselines.

in the fully spelled-out, sentence-internal model, the effect is, again, minimal.


The label was supposed to cover any criticism that is not covered by a dedicated label.

the annotators reported that they chose this label when they were unsure whether a particular criticism label would fit a certain turn or not.


the context of the discussion revealed that the author tried to draw attention to the missing figure in the article and requested it to be filled or removed.


Using this ANOVA model, we find a highly significant main effect of the Re-alVsConstructed factor that demonstrates the general ability of the models to achieve separation between Real Pairs and Constructed Pairs; on average F(1,780) = 18.22, p < .0001.

when we look more closely, we find that although the trend is consistently to find more evidence of speech style accommodation in Real Pairs than in Constructed Pairs, we see differentiation among the models in terms of their ability to achieve this separation.


Such compositions are implemented in the toolkit TIBURON.

there are translation tasks in which the used XTOPs do not fulfill this requirement.


Our approach to composition is the same as in (Engelfriet, 1975;Baker, 1979;Maletti and Vogler, 2010): We simply parse the righthand sides of the XTOP M with the left-hand sides of the XTOP N .

to facilitate this approach we have to adjust the XTOPs M and N in two pre-processing steps.


Roughly speaking, we require that the left-hand sides of N are small enough to completely process righthand sides of M .

a comparison of left-and right-hand sides is complicated by the fact that their shape is different (left-hand sides have a state at the root, whereas right-hand sides have states in front of the variables).


However, in the first pre-processing step we might have introduced some non-linear (copying) rules in N (see rule ( ) in Example 5), and it is known that "nondeterminism [in M ] followed by copying [in N ]" is a feature that prevents composition to work (Engelfriet, 1975;Baker, 1979).

our copying is very local and the copies are only used to project to different subtrees.


Especially after reading the example it might seem useless to create the rule copies in R l [in Example 6 for l = σ(z 2 , z 3 )].

each such rule has a distinct state at the root of the left-hand side, which can be used to trigger only this rule.


Claims are easiest to translate, yielding the highest overall BLEU score of 0.4879.

to that, all models score considerably lower on titles.


The results given in table 14 show that tuning on a pooled set of 6,000 text sections yields only minimal differences to tuning on 2,000 sentence pairs such that the BLEU scores for the new pooled models are still significantly lower than the best results in table 12 (indicated by "<").

increasing the tuning set to 16,000 sentence pairs for IPC sections makes the pooled baseline perform as well as the best results in table 13, except for two cases (indicated by "<") (see table 15).


Syncretism is thus considered as the accidental by-product of such forces, and German case syncretism is typically analyzed according to these lines (Barðdal, 2009;Baerman, 2009, p. 229).

these forces are not explanatory: they only describe what has happened, but not why.


Compared to other semantic concordances, the granularity of PDEV is high and thus discouraging in terms of expected IAA.

selecting among patterns does not really mean disambiguating a concordance but rather determining to which pattern it is most similar-a task easier for humans than WSD is.


Traditionally, the amount of self-information contained in a tag (as a probabilistic event) depends only on the probability of that tag, and would be defined as I(t j ) = − log p 1 (t j ).

intuitively one can say that a good measure of usefulness of a particular tag should also take into consideration the expected tagging confusion related to the tag.


Traditional machine translation (MT) systems treat LCS data as noise, or just as regular sentences.

if LCS data is processed intelligently, it can provide a useful signal for training word alignment and MT models.


They use target strings in multiple languages as different views on translation.

in our work, we treat the alignment model and language model as different views of LCS data.


We can see that the word "badminton" is aligned incorrectly with word ">™Ž(Taufik)" .

in the LCS data, we see that " >™ Ž(Taufik)" and "badminton" appear in the same sentence ">™Ž badminton x³ (Taufik plays badminton so well)" and by adding the blocked constraint into the alignment model, it correctly learns that " >™Ž(Taufik)" should be aligned with something else, and it finds "Taufik" at end.


We can see that 3 techniques we proposed for word alignment all improve the machine translation result over the baseline system as well as the IBM 3 model.

although co-training has a bigger improvement on the word alignment compared with PR + , it actually has a lower BLEU score.


The word layer is connected to a convolutional output layer y t by weights summarized in the sparse matrix C. The output layer represents all possible next minimal units, where each MTU entry is only connected to neurons in the word layer representing its source and target words.

the word and MtU layers are then computed as follows: there are a number of computational issues with this model: First, we cannot efficiently factor the word layer w t into classes such as for the atomic MtU RNN model because we require all its activations to compute the MtU output layer y t .


The results ( Table 2 and Table 3) show that RNNLM performs competitively.

our approaches model translation since we use both source and target information as opposed to scoring only the fluency of the target side, such as done by RNNLM.


By using (6) as the objective function, we observed that the resulting segmentations yield promising applications in n-gram topic modeling, named entity recognition and Chinese segmentation.


The "easiest" instances of quotation attribution problems arise when the speaker and the quote are semantically connected, e.g., through a reported speech verb like said.

in newswire text, the subject of this verb is commonly a pronoun or another uninformative anaphoric mention.


For compatibility with previous assessments, we report this score, which we call Exact Match (EM): this is the percentage of predicted speakers with the same span as the gold one.

for several quotations (about 30% in the PARC corpus) this information is of little value, since the gold mention is a pronoun, which per se does not give any useful information about the actual speaker entity.


The basic QUOTEBEFORECOREF system wrongly clusters together M 3 and M 4 as corefer-ent, and wrongly assigns M 3 as the representative speaker.

the JOINT system correctly clusters M 1 , M 2 and M 4 as coreferent.


After every n th iteration we resample η and ξ for all language models to capture the correlations.

to improve mixing time, we also resample components η k i and η l i when word i has changed event membership from type k to type l. In addition we define classes of closely related words (heuristically based on the covariance matrix) by classifying words as related when their similarity exceeds an empirically determined threshold.


(2012) extended the model of  to jointly induce semantic roles and frames using the Chinese Restaurant Process, which is also used in our approach.

they did not aim at building a lexicon of semantic frames, but at distinguishing verbs that have different senses in a relatively small annotated corpus.


To measure the precision of induced semantic frames, we adopt the purity metric, which is usually used to evaluate clustering results.

the problem is that it is impossible to assign gold-standard classes to the huge number of instances.


Unlike the hand-labelled SemCor data, our automated sense labelling method is limited to the information found in the LLR used.

there are also 330 MASC instances covered by the ALC only.


In practise, our experiments turned out to be fairly insensitive to the value of this parameter, on evaluations over rare or unseen verbs.

overall accuracy would drop slightly if this cut-off was increased.


the Bohnet parser (Bohnet, 2010) employs morphological feature value pairs similar to our feature templates and Seeker and Kuhn (2013) introduces an integer linear programming framework including constraints for morphological agreement.

these works focus on dependency parsing and to the best of our knowledge, this is the first study on experimenting with atomic morphological features and their agreement in a constituency parsing.


Prop-Bank (Palmer et al., 2005) is the corpus of reference for verb-argument relations.

relations between a verb and its syntactic arguments are only a fraction of the relations present in texts.


In (8), both 'create' and 'buy' are done due to the 'country's lack of natural resources'.

in (9), the analysts 'forecasting' and the company 'saying' do not have as their cause 'planned price cuts'.


The main challenge for such systems is translating out-of-vocabulary words (Carpuat et al., 2012).

words in biographies are closer to a training corpus of news commentaries and parlimentary proceedings and allow us to examine how well domain adaptation techniques can disambiguate lexical choices.


(2011) introduced additional caches to store (i) words and phrase pairs from training documents most similar to a current source article, and (ii) words from topical clusters created on the training set.

a central issue in these systems is that caches become noisy over time, since they ignore topic shifts in the documents.


The consistency cache's contents are computed similarly to the general domain case.

the cache gets cleared at segment boundaries.


The struct-topic cache performs much better on longer documents of over 30 sentences giving 0.3 to 0.4 BLEU points increase compared to the general domain model.

the performance worsens when the structured cache is applied on documents with less than 20 sentences.


changes the weights to learn different representations for the input layer.

the dropout for the input layer improves the performance.


The system is incremental is that each word class to be verbalised can yield a new set of utterance candidates.

it supports only addition not revisions.


While their approach is fast to execute, it is limited to a restricted set of domain specific attributes; requires a training corpus of example sentences to define the space of possible surface realisations; and is based on a large set (800 rules) of domain specific rules extracted semi-automatically from the training corpus.

we use a general, small size grammar (around 50 rules) and a lexicon which is automatically derived from the input ontologies.


The PPDB data outperforms both French literature and MSR models if we look all possible sentence pairs from test data (the column labeled "all" in the table).

when we consider whether any pair from a set of 4 translations can be translated, the PPDB models do not do as well.


The retrieval algorithm used by commercial TM systems is typically not disclosed (Koehn and Senellart, 2010;Simard and Fujita, 2012;Whyman and Somers, 1999).

the bestperforming method used in current systems is widely believed to be based on edit distance (Baldwin and Tanaka, 2000;Simard and Fujita, 2012;Whyman and Somers, 1999;Koehn and Senellart, 2010;Christensen and Schjoldager, 2010;Mandreoli et al., 2006;He et al., 2010).


As can be seen from the numerator of Equation 5, NGP is weighting the match of all n-grams as uniformly important.

it is not the case that each n-gram is of equal value to the translator.


For example, Rule (4) correctly detects the social event in sentence (5), since Semafor correctly parses the input.

semafor does not correctly parse the input sentence (1): it correctly identifies the statement frame and its Message frame element, but it fails to find the speaker.


There have been recent efforts to extract networks from text (Elson et al., 2010;He et al., 2013).

these efforts extract a different type of network: a network of only bi-directional links, where the links are triggered by quotation marks.


(2013) will extract an interaction link between Emma and Harriet in the following sentence.

their system will not detect any interaction links in the other examples mentioned in this paper.


There is a small body of recent research on automatically learning probabilistic models of scripts from large corpora of raw text (Manshadi et al., 2008;Chambers and Jurafsky, 2008;Chambers and Jurafsky, 2009;Jans et al., 2012).

this work uses a very impoverished representation of events that only includes a verb and a single dependent entity.


We claim that this is the most natural adaptation of the cloze evaluation to the multi-argument event setting.

other types of inferences would be useful as well for question-answering.


Normalisation helps with Sum throughout, with little difference in performance between Norm and Norm10, but with a slight decrease when CNorm is used.

only CNorm improves the ranking of Prod-based vectors.


The two generation algorithms presented above rely on a completed initial parsing step.

given that the complexity of the parsing stage is O(2 N • K), this may not be achievable in practice.


They can be different usages, both having a subjective meaning.

if two instances are labeled having opposing labels, we do not want them to be in the same cluster.


Customer reviews of books, hotels and other products are widely perceived as an important rea-son for the success of e-commerce sites such as amazon.com or tripadvisor.com.

customer confidence in such reviews is often misplaced, due to the growth of the so-called sock puppetry phenomenon: authors / hoteliers writing glowing reviews of their own works / hotels (and occasionally also negative reviews of the competitors).


The results suggest that both methods achieve accuracy well above the baseline.

the models trained using Learning from Crowd classes not only achieved the highest accuracy, but also outperformed the thresholds for precision and recall in detecting deceptive reviews (Table 4), while the models trained with the majority voting classes showed a very high precision, but at the expense of the recall, which was lower than the baseline (Table 3).


Our system follows much previous work by counting PPs that accompany the verb among its complements, even though they are not obligatory (so-called 'adjuncts'), because PP adjuncts are excellent clues to a verb's semantics (Sun et al., 2008).

nominal and clausal adjuncts do not count as verbal complements.


Approaching temporal link labelling as a classification task has already been explored in several works.

choosing the right feature vectors to build the classification model is still an open issue, especially for event-event classification, whose accuracy is still under 50%.


We train a classification model for each category of entity pair, as suggested in several previous works (Mani et al., 2006;Chambers, 2013).

because there are very few examples of timex-timex pairs in the training corpus, it is not possible to train the classification model for these particular pairs.


(1999) indeed shows that the accuracy of a context-sensitive word prediction system is related to how much training material is provided.

once most of the frequent combinations are covered, it takes more and more training material to improve the results a little bit.


Please note that CKS assumes that the user always accepts a prediction immediately when it is available, which might not always be the case in reality.

current popular smartphone applications suggest this approach might be too strict.


Nondeterministic variants of LR(k) parsing, for use in natural language processing, have been proposed as well, some using tabulation to ensure polynomial running time in the length of the input string (Tomita, 1988;Billot and Lang, 1989).

nondeterministic LR(k) parsing is potentially as expensive as, and possibly more expensive than, traditional tabular parsing algorithms such as CKY parsing (Younger, 1967;Aho and Ullman, 1972), as shown by for example (Shann, 1991); greater values of k make matters worse (Lankhorst, 1991).


These benchmark datasets have supported a diverse and influential line of research into semantic parsing learning algorithms for sophisticated semantic constructions, with continuing advances in accuracy.

the focus on these datasets leads to a natural question -do other natural datasets have similar syntax and semantics, and if not, can existing algorithms handle the variability in syntax and semantics?


Furthermore, they require the grammar to be binarized and linear, which means that they only support linear context-free rewriting systems (LCFRS).

our algorithm naturally supports the full power of PMCFG.


the weight of the pro- is the probability to choose this production when the result category is fixed.

in this case the probabilities for all productions with the same result category sum to one: the parsing algorithm does not depend on the probabilistic interpretation of the weights, so the same algorithm can be used with any other kind of weights.


The outside weight w o for the new active item remains the same.

we must update the inside weight since we have replaced the d-th argument in B with the newly generated categoryB d .


Since different sentiments may be expressed toward different entities in a document, fine-grained analysis may be more informative for applications.

fine-grained sentiment analysis remains a challenging task for NLP systems.


The goal of such work is to determine one overall polarity of an expression or sentence.

our framework commits to a holder having sentiments toward various events and entities in the sentence, possibly of different polarities.


This is reasonable in scenarios where available training data is fixed over long periods of time.

this approach  Figure 1: Context when translating an input sentence (bold) with simulated post-editing.


Unfortunately, in general, Bayesian techniques are computationally difficult to work with.

hierarchical Pitman-Yor process language models (HPYPLMs) are convenient in this regard since (1) inference can be carried out efficiently in a convenient collapsed representation (the "Chinese restaurant franchise") and (2) the posterior predictive distribution from a single sample provides a high quality language model.


It is expected that users with high numbers of followers are also popular in the real world, being well-known artists, politicians, brands and so on.

non popular entities, the majority in the social network, can also gain a great number of followers, by exploiting, for example, a follow-back strategy.


Avoiding mentioning (a 7 ) or replying (a 8 ) to others may not affect (on average) an impact score positively or negatively; however, accounts that do many unique @-mentions are distributed around a clearly higher impact score.

users that overdo @-replies are distributed below the mean impact score.


Vector-based models are typically used in the literature for representing documents both in monolingual and cross-lingual settings (Manning et al., 2008).

because of the large size of the vocabulary, having each term as a component of the vector makes the document representation very sparse.


This is not the case in the CosSim BN model which achieves higher results using BabelNet as a statistical dictionary, especially on the Spanish news corpus.

however, the linear projection methods as well as Full MT obtained the highest results on the English corpus.


Phrase-based models (Koehn et al., 2003;Och and Ney, 2004;Xiong et al., 2006) have been strong in local translation and reordering.

phrase-based models cannot effectively conduct long-distance reordering because they are based purely on statistics of syntax-independent phrases.


We use word alignment results for tree structure projection.

accurate word alignment is challenging when handling language pairs in which long-distance reordering is needed, and the alignment noise propagates to the tree projection.


A reasonable way out of this problem would be to save the mean and standard deviation parameters used for data standardization and use them to project the composed phrase vector outputs back to the original vector space.

enetLex obtained a stable good performance in SVD space, with the best results achieved with dimensions between 200 and 300.


At one end of the spectrum, the WMT human evaluation collects large numbers of quick judgments (approximately 3.5 minutes per screen, or 20 seconds per label) (Bojar et al., 2013).

1 HMEANT (Lo and Wu, 2011) uses a more time-consuming fine-grained semantic-role labeling analysis at a rate of approximately 10 sentences per hour (Birch et al., 2013).


Several works (e.g., Ratinov and Roth, 2009;Cohen and Sarawagi, 2004) have shown that injecting dictionary matches as features in a sequence tagger results in significant gains in NER performance.

building these dictionaries requires a huge amount of human effort and it is often difficult to get good coverage for many named entity types.


Further analysis reveals that 32 (63%) target languages for ENC, 25 (49%) target languages for EVPC, and only 5 (10%) target languages for GNC have a correlation of r ≥ 0.1 with goldstandard compositionality judgements.

8 (16%) target languages for ENC, 2 (4%) target languages for EVPC, and no target languages for GNC have a correlation of r ≤ −0.1.


The results for string similarity (CS string : r = 0.385) are similar to those for CS L2N .

as with the ENC dataset, when we combine string similarity and distributional similarity (CS all ), the results improve, and we achieve the state-of-the-art for the dataset.


Word embeddings resulting from neural language models have been shown to be a great asset for a large variety of NLP tasks.

such architecture might be difficult and time-consuming to train.


It can be computed in linear time with the Forward algorithm, which derives a recursion similar to the Viterbi algorithm (see Rabiner (1989)).

we can thus maximize the log-likelihood over all the training pairs ( 1 , the best tag path which minimizes the sentence score (6): to classical CRF, all parameters θ are trained in a end-to-end manner, by backpropagation through the Forward recursion, following Collobert et al.


The weights for the scope assignment and disambiguation task are learned in a cascaded way.

to the joint approach, the hasScope(m, s) predicate is observed during disambiguation.


Note that our definition of a topic, namely the top N features from a distributional vector, does not correspond to a topic generated by a latent variable model, because it does not have a probability distribution over words.

the TC measures we adopt do not make use of such a probability distribution except for choosing the top N words from a topic, which are then treated as an unordered set for the pairwise operations.


These idiosyncratic contexts are not mutually informative and cause a sizeable decrease in TC.

removing owl from creature does not decrease the coherence nearly as much.


In the context of hypernym detection, they could test a system's ability to find one or two good-quality hypernyms quickly from a set of candidates.

these measures are less appropriate for testing whether a system can, in general, rank hypernyms over other relations.


(2013) similarly note that generated REs should avoid information that is perceptually expensive to obtain.

these results focus on content selection rather than surface realization.


We set up the prediction task as in the previous section: Given an anchor/landmark pair, our system must decide what direction and ES-TABLISH status to assign it.

here we evaluate the system as a classifier.


On the one hand, emotions are coded by affective modalities (Scherer, 2005), among which sadness, disgust, enjoyment, fear, surprise and anger are the most usual (Ekman, 1999;Cowie and Cornelius, 2003).

an ordinal classification in a multidimensional space is considered.


Analogies are considered to be one of the core concepts of human cognition and communication, and are very efficient at encoding complex information in a natural fashion.

computational approaches towards largescale analysis of the semantics of analogies are hampered by the lack of suitable corpora with real-life example of analogies.


For example, an analogy repository containing such domain knowledge has to provide information on which attributes of source and target are generally considered comparable.

to Linked Open Data or typical ontologies, such analogical knowledge is consensual, i.e.


For example, (Bollegala, Matsuo, & Ishizuka, 2009), (Nakov & Hearst, 2008), or (Turney, 2008) approach this challenge by using pattern-based Web search and subsequent analysis of the resulting snippets.

to these approaches, we do not focus on word pair similarity, but given one entity, we aim at finding other entities which are seen as analogous in a specific domain (in our case analogies between locations and places).


These patterns were selected manually based on analysis of sample Web data by three experts.

to other approaches relying on extraction patters, e.g.


Under certain circumstances, crowd-sourcing can be very effective for handling large tasks requiring human intelligence without relying on expensive experts.

to using expert annotators, crowd-workers are readily and cheaply available even for ad-hoc tasks.


This is due to some of its necessary assumptions not holding true (see section 6.4).

our subsequence-based model achieves a higher informedness score of 0.85 and 0.87 in the best cases.


First, we compared our baseline, lexical unigrams with SVM to using lexical n-grams to test whether using n-grams actually contributed to the quality, and found the difference to be significant (sign-test p<0.024).

for SVM-based classification, the higher reported performance for also including POS features in addition to lexical n-grams could not be shown to be significant (p>0.4).


In this paper we address the problem of lexicon construction by constructing a semi-supervised system that accepts concrete inflection tables as input, generalizes inflection paradigms from the tables provided, and subsequently allows the use of unannotated corpora to expand the inflection tables and the automatically generated paradigms.

1 to many machine learning approaches that address the problem of paradigm extraction, the current method is intended to produce human-readable output of its generalizations.


In the general case, all patterns are tried for a given candidate word.

we usually have access to additional information about the candidate words-e.g., that they are in the base form of a certain part of speech-which we use to improve the results by only matching the relevant patterns.


The main source of error (334 out of 1000) is confusion with p akribi (accuracy), which has no plural.

it is on semantic grounds that the paradigm has no plural; a native Swedish speaker would pluralize akribi like akademi (disregarding the fact that akribi is defective).


Hausboot, "house boat") or only rarely have been seen in the training data.

2 most compounds consist of two (or more) simple words that occur more frequently in the data than the compound as a whole (e.g.


This approach can even produce new compounds unseen in the training data, provided that the modifiers occurred in modifier position of a compound and heads occurred as heads or even as simple words with the same inflectional endings.

as former compound modifiers were left with their filler letters (cf.


(2012) we re-implemented the approach of Stymne and Cancedda (2011), combined it with inflection prediction and applied it to a translation task.

compound merging was restricted to a list of compounds and parts.


It can be seen that using more features (SC→T→ST) is favourable in terms of precision and overall accuracy and the positive impact of using source language features is clearer when only reduced feature sets are used (TR vs. STR).

these accuracies only somewhat correlate with SMT performance: while being trained and tested on clean, fluent German language, the   models will later be applied to disfluent SMT output and might thus lead to different results there.


With accuracies of over 97%, POS-tagging of WSJ can be treated as a solved problem (Manning, 2011).

performance is still well below satisfactory for many other languages and domains (Petrov et al., 2012;Christodoulopoulos et al., 2010).


The idea is compelling: on the one hand, a list of lexicons is often available for special domains, such as bio-informatics; on the other hand, compiling a lexicon of word-tag pairs appears to be less timeconsuming than annotating full sentences.

success in type-supervised POStagging turns out to depend on several subtle factors.


Results show that BLEU score improves on a test sub-set containing only negative sentences when extra negative data is appended to the original training data and the language model is enriched as well.

system performance deteriorates on both the original test set and on positive sentences.


Also in the case of a sentence containing a subordinate clause, dependency parsing is able to correctly capture the latter as part of the scope given that the relative pronoun depends directly on the event of the main clause.

recursion from the negated event excludes coordinate clauses that are not considered part of the scope, given that the event is a dependant of the connective.


Only French setup yielded statistically significant improvement (p < .01).

if we concatenate the outputs of all languages, the improvement in translation of references with BLEU score averaged over all systems becomes statistically significant (p = .03), improving from 16.8 for the baseline system to 17.3 for the adapted MT outputs.


Early user simulation techniques are based on Ngrams (Eckert et al., 1997;Levin and Pieraccini, 2000;Georgila et al., 2005;Georgila et al., 2006), ensuring that simulator responses to a machine utterance are sensible locally.

they do not enforce user consistency throughout the dialog.


the probability of a set of waypoints given a semantic unit.

there are two problems with deriving a generative model directly over W .


The overall accuracy is slightly better when snippets from the END of the summary are chosen compared to those from the START.

with START snippets, better prediction of different length summaries was obtained, whereas the accuracy in the END case comes mainly from correct prediction of 50 and 400 word summaries.


When pragmatic information is used, 5.6 relevance variables are used on average (per dialogue snipped).

when pragmatic information is not used, this number rises to 6.3 8 .


For example, in Figure 1 banana and redberry are not directly connected but they can be reached via pear or raspberry.

by considering mediate relationships it becomes more difficult to determine the most appropriate category for each food item since most food items are connected to food items of different categories (in Figure 1, there are not only edges between banana and other types of fruits but there is also some edge to some sweet, i.e.


Sentence compression and sentence simplification also consider deleting words from input sentences.


This is because discarding "not" would flip the sentence's meaning; discarding "the" would lose a necessary determiner before a noun.

discarding "just" would hurt neither fluency nor meaning.


A linear combination of a language model score and our proposed measure based on analysis of alignments best captures redundancy.

as our experimental results suggest, it is necessary both to use alignments in translation outputs, and to use them in a good way.


Such systems are relatively easy to set up and experienced many successes: the TEES system (Björne et al., 2009;Bj[Pleaseinsertintopreamble]rne et al., 2012; Björne and Salakoski, 2013) won the BioNLP GE task in 2009 and ranked 2 nd in 2013, whereas the EVEX system won in 2013 (Van Landeghem et al., 2011;Hakala et al., 2013).

all these methods suffer from error cascading.


Words that tend to be used in the summaries, characterized by high KL(A ∥ G) scores, include locations (York, NJ, Iraq), people's names and titles (Bush, Sen, John), some abbreviations (pres, corp, dept) and verbs of conflict (contends, dies).

from KL(G ∥ A), we can see that it is unlikely for writers to include courtesy titles (Mr, Ms, Jr.) and relative time reference in summaries.


Systems such as Siddharthan (2011) use transformation rules that encode morphological changes as well as deletions, re-orderings, substitutions and sentence splitting, and are well suited to handle the voice conversion example above.

hand-crafted systems are limited in scope to syntactic simplification.


While much work in summarisation has concentrated on multi-document summarisation, where the main challenge is the detection of redundant information, the summariser presented here is a single-document summariser.

researchers have been attracted by deeper, more symbolic and thus more explanatory summarisation models that use semantic representations of some form (Radev and McKeown, 1998) and often rely on explicit discourse modelling (Lehnert, 1981;Kintsch and van Dijk, 1978;Cohen, 1984).


In natural language processing (NLP) annotation projects, we use inter-annotator agreement measures and annotation guidelines to ensure consistent annotations.

annotation guidelines often make linguistically debatable and even somewhat arbitrary decisions, and interannotator agreement is often less than perfect.


for English to (Marcus et al., 1993) or words ending in -ing (Manning, 2011).

standardized label sets have practical advantages in NLP (Zeman and Resnik, 2008;Zeman, 2010;Das and Petrov, 2011;Petrov et al., 2012;McDonald et al., 2013).


If an error occurred and the predicted tag is in the same class as the gold tag, a loss σ occurred, otherwise it counts as full cost.

to our approach, they let the learner focus on the more difficult cases by occurring a bigger loss when the predicted POS tag is in a different category.


In fact, Residual Networks can be viewed as a special case of Highway Networks where both the transform and carry gates are substituted by the identity mapping function: thereby forming a hard-wired shortcut connection x. Arguably, Equation (3) can be considered as a form of residuality with o k working as the residual function and u k the shortcut connection.

as discussed in (Srivastava et al., 2015b), in contrast to the hard-wired skip connection in Residual Networks, one of the advantages of Highway Networks is the adaptive gating mechanism, capable of learning to dynamically control the information flow based on the current input.


Notice that the highest average accuracy of the original MemN2N model on the 10k dataset is 95.8.

it was attained by a model with layer-wise weight tying, not adjacent weight tying as adopted in this work, and, more importantly, a much larger embedding size d = 100 (therefore not shown in Table 1).


Recurrent neural networks (RNNs) process input text sequentially and model the conditional transition between word tokens.

the advantages of recursive networks include that they explicitly model the compositionality and the recursive structure of natural language.


Unlike sequential models, recursive neural networks compose word phrases over syntactic tree structure and have shown improved performance in sentiment analysis (Socher et al., 2013).

its dependence on a syntactic tree architecture limits practical NLP applications.


This attention mechanism is robust as it globally normalizes the attention score m with sof tmax to obtain the weights α.

it does not consider the tree structure when producing the final representation h tree .


and this equation is similarity to the global attention.

now each non-leaf node attentively collects its own and children representations and passes towards the root which finally constructs the attentively blended tree representation.


Pooling is unweighted selection: it outputs the selected values as is.

attention can be thought of as weighted selection: some input elements are highly weighted, others receive weights close to zero and are thereby effectively not selected.