Some exceptions include recent work on learning common event sequences in news stories (Chambers and Jurafsky, 2008), an approach based on statistical methods, and the development of an event calculus for characterizing stories written by children (Halpin et al., 2004), a knowledge-based strategy.

literary theorists, linguists and others have long developed symbolic but non-computational models for novels.


They propose a logical inference model which makes use of discourse plans and coherence relations to infer categorical answers.

to adequately interpret indirect answers, the uncertainty inherent in some answers needs to be captured (de Marneffe et al., 2009).


In total, we ended up with 224 question-answer pairs involving gradable adjectives.

our collection contains different types of answers, which naturally fall into two categories: (I) in 205 dialogues, both the question and the answer contain a gradable modifier; (II) in 19 dialogues, the reply contains a numerical measure (as in (3) above and (4)).


The dialogues in (1) and 9are examples of total agreement.

(10) has response entropy of 1.1, and item (11) has the highest entropy of 2.2.


This is a common feature of such corpora of informal, userprovided reviews (Chevalier and Mayzlin, 2006;Hu et al., 2006;Pang and Lee, 2008).

since we do not want to incorporate the linguistically uninteresting fact that people tend to write a lot of ten-star reviews, we assume uniform priors for the rating categories.


Table 6 summarizes the results per response category for the WordNet-based approach (which can thus be compared to the category I results in table 5).

in contrast to the WordNet-based approach, we require no hand-built resources: the synonym and antonym structures, as well as the strength values, are learned from Web data alone.


If the user uses the turn to query or inform an unavailable filler the dialogue grows longer.

this is quite rare as shown by small difference in performance between the two models.


In our domain, performance is measured by dialogue length and solution quality.

since solution quality never affects the dialogue cost for a trained system, dialogue length is the only component influencing the mean policy cost.


First, we tried extracting grammatical roles from the parse trees which we obtained from the Berkeley parser, as this information is present in the edge labels that can be recovered from the parse.

we found that we achieved better accuracy by using RFTagger (Schmid and Laws, 2008), which tags nouns with their morphological case.


Morphological case is distinct from grammatical role, as noun phrases can function as adjuncts in possessive constructions and preposi- tional phrases.

we can approximate the grammatical role of an entity using the morphological case.


When a word conforms to the language processor's expectations, surprisal is low, and the cognitive load associated with processing that input will also be low.

unexpected words will have a high surprisal and a high cognitive cost.


In contrast, unexpected words will have a high surprisal and a high cognitive cost.

high-level syntactic and semantic factors are only one source of cognitive costs.


to more standard semantic space models where word senses are conflated into a single representation, topics have an intuitive correspondence to coarse-grained sense distinctions.


An open issue is whether a single, integrated measure (as evaluated in Table 4) fits the eyemovement data significantly better than separate measures for trigram, syntactic, and semantic surprisal (as evaluated in Table 3.

we are not able to investigate this hypothesis: our approach to testing the significance of factors requires nested models; the log-likelihood test (see Section 4) is only able to establish whether adding a factor to a model improves its fit; it cannot compare models with disjunct sets of factors (such as a model containing the integrated surprisal measure and one containing the three separate ones).


The rebanked parser performed 0.8% worse than the CCGbank parser on the intersection dependencies, suggesting that the fine-grained distinctions we introduced did cause some sparse data problems.

we did not change any of the parser's maximum entropy features or hyperparameters, which are tuned for CCGbank.


These include Eu-roWordNet (Vossen, 1998), MultiWordNet (Pianta et al., 2002), the Multilingual Central Repository (Atserias et al., 2004), and many others.

manual construction methods inherently suffer from a number of drawbacks.


The automatic evaluation quantifies how much of the gold-standard resources is covered by BabelNet.

it does not say anything about the precision of the additional lexicalizations provided by BabelNet.


(2009) presented methods to produce massive multilingual translation dictionaries from Web resources such as online lexicons and Wiktionaries.

while providing lexical resources on a very large scale for hundreds of thousands of language pairs, these do not encode semantic relations between concepts denoted by their lexical entries.


In (3), [in the park] places a well-defined situation (Yuri playing football) in a certain location.

in "The troops are based [in the park]", the same argument is obligatory, since being based requires a place to be based in.


Distinguishing between the two argument types has been discussed extensively in various formulations in the NLP literature, notably in PP attachment, semantic role labeling (SRL) and subcategorization acquisition.

no work has tackled it yet in a fully unsupervised scenario.


6 defines roles preferences local to individual arguments (r i , h i ).

an argument frame is a joint structure, with strong dependencies between arguments.


On the one hand, the semantic similarity between two nodes can be measured with any commonly adopted metric, such as cosine similarity and Jaccard coefficient (Baeza-Yates and Ribeiro-Neto, 1999).

the structural relation between a pair of nodes takes two forms as we have discussed earlier.


Nearly all semantic class taggers are trained using supervised learning with manually annotated data.

annotated data is rarely available for specialized domains, and it is expensive to obtain because domain experts must do the annotation work.


Supervised learning exploits manually annotated data, but must make do with a relatively small amount of training text because manual annotations are expensive.

seed-based bootstrapping exploits a small number of human-provided seeds, but needs a larger set of (unannotated) texts for training because the seeds produce relatively sparse annotations of the texts.


Many researchers are trying to use information extraction (IE) to create large-scale knowledge bases from natural language text on the Web.

the primary approach (supervised learning of relation-specific extractors) requires manually-labeled training data for each relation and doesn't scale to the thousands of relations encoded in Web text.


For each training sentence LUCHS first identifies subsequences of labeled words, and for each such labeled subsequence, LUCHS creates one or more seed phrases p. Typically, a set of seeds consists precisely of the labeled subsequences.

if the labeled subsequences are long and have substructure, e.g., 'San Remo, Italy', our system splits at the separator token, and creates additional seed sets from prefixes and postfixes.


Not knowing which lexicon will be most useful to the extractors, LUCHS generates several and lets the extractors learn appropriate weights.

since list similarities vary depending on the seeds, fixed thresholds are not an option.


The past methods either only learn parameters with one or two levels (e.g., in hierarchical Bayes), or requires significant amount of computation (e.g., in EM and in L 1 -regularized maxent), while also typically assuming a given hierarchy.

ontoUSP has to both induce the hierarchy and populate it, with potentially many levels in the induced hierarchy, starting from raw text with little user supervision.


On one hand, it should encourage creating abstract clusters to summarize intrinsic commonalities among the children.

this needs to be heavily regularized to avoid mistaking noise for the signal.


However, their agents tend to differ since in one case they are inducers, and in the other they are inhibitors.

aCTIVaTE and INDUCE share similar agents since they both signify positive regulation.


On the other hand, ACTIVATE and INDUCE share similar agents since they both signify positive regulation.

"activate" tends to be used more often when the patient argument is a concrete entity (e.g., cells, genes, proteins), whereas "induce" and others are also used with processes and events (e.g., expressions, inhibition, pathways).


Syntax based Statistical Machine Translation (SMT) systems allow the translation process to be more grammatically performed, which provides decent reordering capability.

most of the syntax based systems construct the syntactic translation rules based on word alignment, which not only suffers from the pipeline errors, but also fails to effectively utilize the syntactic structural features.


Monolingual Tree kernels achieve decent performance using the SSTs due to the rich exploration of syntactic information.

the sub-tree alignment task requires strong capability of discriminating the sub-trees with their roots across adjacent generations, because those candidates share many identical SSTs.


Other than the factor of the amount of training data, this is also because the plain features in Table 3 are not as effective  as those in Table 4, since they are trained on  FBIS corpus which facilitates Table 4 more with respect to the domains.

the grammatical tags and syntactic tree structures are more accurate in HIT corpus, which facilitates the performance of BTKs in Table 3.


utilizing syntactic translational equivalences alone for machine translation loses the capability of modeling non-syntactic phrases (Koehn et al., 2003).


On the one hand, a good phrase pair often fails to be extracted due to a link inconsistent with the pair.

iTG pruning can be considered as phrase pair selection, and good iTG pruning like DPDi guides the subsequent iTG alignment process so that less links inconsistent to good phrase pairs are produced.


The major reason is that we did not perform any reordering or distorting during decoding with PTT.

in both t2s and s2t systems, the BLEU-4 score benefits of PRS were covered by the composed rules: both PTT+C S 3 and PTT+C 3 performed significant better (p < 0.01) than PTT+PRS, and there are no significant differences when appending PRS to PTT+C 3 .


The definition of combinatory categorial grammar (CCG) in the literature varies quite a bit from author to author.

the differences between the definitions are important in terms of the language classes of each CCG.


On the practical side, we have corpora with CCG derivations for each sentence (Hockenmaier and Steedman, 2007), a wide-coverage parser trained on that corpus (Clark and Curran, 2007) and a system for converting CCG derivations into semantic representations (Bos et al., 2004).

despite being treated as a single unified grammar formalism, each of these authors use variations of CCG which differ primarily on which combinators are included in the grammar and the restrictions that are put on them.


NN, VBZ and DT) and supertags are CCG categories.

because the Petrov parser trained on CCGbank has no notion of Penn treebank POS tags, we can only evaluate the accuracy of the supertags.


If we obtained perfect accuracy on our new task then we would be removing all of the categories not chosen by the parser.

parsing accuracy will not decrease since the parser will still receive the categories it would have used, and will therefore be able to form the same highest-scoring derivation (and hence will choose it).


As Table 6 shows, this change translates into an improvement of up to 0.75% in F-score on Section   Tables 2, 4 and 6, are similar for all of the training algorithms.


Note that for some of the results presented here it may appear that the C&C parser does not lose speed when out of domain, since the Wikipedia and biomedical corpora contain shorter sentences on average than the news corpus.

by testing on balanced sets it is clear that speed does decrease, particularly for longer sentences, as shown in Table 9.


In short, we have shown that by combining PROMODES and PROMODES-H and finding the optimal threshold, the ensemble PROMODES-E gives better results than the individual models themselves and therefore manages to leverage the individual strengths of both to a certain extend.

can we pinpoint the exact contribution of each individual algorithm to the improved result?


(2009) achieve state-of-the-art accuracy.

these approaches dictate a particular choice of model and training regime.


(Alternately, they could keep the soft clustering, with the representation for a particular word token being the posterior probability distribution over the states.)

the CRF chunker in Huang and Yates (2009), which uses their HMM word clusters as extra features, achieves F1 lower than a baseline CRF chunker (Sha & Pereira, 2003).


This technique for turning a supervised approach into a semi-supervised one is general and task-agnostic.

we wish to find out if certain word representations are preferable for certain tasks.


We also looked at the classification accuracy for different parts of speech in Figure 5. we notice that, in the case of 10-fold cross validation, the performance is consistent across parts of speech.

when we only use 14 seeds all of which are adjectives, similar to (Turney and Littman, 2003), we notice that the performance on adjectives is much better than other parts of speech.


All these research works concentrated on attribute-based sentiment analysis.

the main difference with our work is that they did not sufficiently utilize the hierarchical relationships among a product attributes.


The algorithm H-RLS studied in (Cesa-Bianchi et al., 2006) solved a similar hierarchical classification problem as we formulated above.

the H-RLS algorithm was designed as an onlinelearning algorithm which is not suitable to be applied directly in our problem setting.


If d is set too small, important useful terms will be missed that will limit the performance of the approach.

if d is set too large, the computing efficiency will be decreased.


Since it is a discriminative approach it is amenable to feature engineering, but needs to be retrained and tuned for each task.

generative models produce complete probability distributions of the data, and hence can be integrated with other systems and tasks in a more principled manner (see Sections 4.2.2 and 4.3.1).


Subjects are not distinguished from objects and nouns may not be actual arguments of the verb.

it is a simple baseline to implement with these freely available counts.


Ideally, we would produce all possible segmentations and alignments during training.

this has been shown to be infeasible for real-world data .


Previous attempts have dealt with the overfitting problem by limiting the maximum phrase length (DeNero et al., 2006;Marcu and Wong, 2002) and by smoothing the phrase probabilities by lexical models on the phrase level (Ferrer and Juan, 2009).

(DeNero et al., 2006) experienced similar over-fitting with short phrases due to the fact that the same word sequence can be segmented in different ways, leading to specific segmentations being learned for specific training sentence pairs.


As discussed earlier, the run-time requirements for computing all possible alignments is prohibitive for large data tasks.

we can approximate the space of all possible hypotheses by the search space that was used for the alignment.


We will refer to this alignment as the full alignment.

to the method described in Section 4.1, phrases are weighted by their posterior probability in the word graph.


For example, as suggested in (Liang et al., 2008), adjacent labels do not provide strong information in POS tagging.

the applicability of this idea to other NLP tasks is still unclear.


In general, approximate algorithms have the advantage of speed over exact algorithms.

both types of algorithms are still widely adopted by practitioners, since exact algorithms have merits other than speed.


In HMMs, the score function can be written as In perceptrons, on the other hand, it is given as where we explicitly distinguish the unigram feature function φ 1 k and bigram feature function φ 2 k .

comparing the form of the two functions, we can see that our discussion on HMMs can be extended to perceptrons by substituting implementing the perceptron algorithm is not straightforward.


We consider this is because the transition information is crucial for the two tasks, and the assumption behind CARPEDIEM is violated.

the proposed algorithms performed reasonably well for all three tasks, demonstrating the wide applicability of our algorithm.


Our objective is to find the smallest supertag grammar (of tag bigram types) that explains the entire text while obeying the lexicon's constraints.

the original IP method of Ravi and Knight (2009) is intractable for supertagging, so we propose a new two-stage method that scales to the larger tagsets and data involved.


For the CCGbank test data, MIN1 yields 2530 tag bigrams.

a second stage is needed since there is no guarantee that G min1 can explain the test data: it contains tags for all word bigram types, but it cannot necessarily tag the full word sequence.


IP-minimization identifies a smaller set of tags that better matches the gold tags; this emerges because other determiners and prepositions evoke similar, but not identical, supertags, and the grammar minimization pushes (but does not force) them to rely on the same supertags wherever possible.

the proportions are incorrect; for example, the tag assigned most frequently to in is ((S\NP)\(S\NP))/NP though (NP\NP)/NP is more frequent in the test set.


An important property of CRFs is their ability to handle large and redundant feature sets and to integrate structural dependency between output labels.

even for simple linear chain CRFs, the complexity of learning and inference This work was partly supported by ANR projects CroTaL (ANR-07-MDCO-003) and MGA (ANR-07-BLAN-0311-02).


Using only unigram features {f y,x } (y,x)∈Y ×X results in a model equivalent to a simple bag-of-tokens positionby-position logistic regression model.

bigram features {f y ,y,x } (y,x)∈Y 2 ×X are helpful in modelling dependencies between successive labels.


Similar dominance mechanisms have been employed in various tree description formalisms (Rambow et al., 1995;Rambow et al., 2001;Candito and Kahane, 1998;Kallmeyer, 2001; Guillaume and Perrier, 2010) and TAG extensions (Becker et al., 1991;Rambow, 1994a).

the prime motivation for this survey is another grammatical formalism defined in the same article: multiset-valued linear indexed grammars (Rambow, 1994b, MLIGs), which can be seen as a low-level variant of UVG-dls that uses multisets to emulate unfulfilled dominance links in partial derivations.


Production factorization is very similar to the reduction of a context-free grammar production into Chomsky normal form.

in the LCFRS case some productions might not be reducible to r = 2, and the process stops at some larger value for r, which in the worst case might as well be the rank of the source production (Rambow and Satta, 1999).


This is usually done by exploiting standard dynamic programming techniques; see for instance (Seki et al., 1991).

1 the polynomial degree in the running time is a monotonically strictly increasing function that depends on both the rank and the fan-out of the productions in the grammar.


Apart from the core rules given in Figure 1, some versions of CCG also use rules derived from the S and T combinators of combinatory logic, called substitution and type-raising, the latter restricted to the lexicon.

since our main point of reference in this paper, the CCG formalism defined by Vijay-Shanker and Weir (1994), does not use such rules, we will not consider them here, either.


We have followed Hockenmaier and Young (2008) in classifying instances of generalized forward composition as harmonic if the innermost slash of the secondary argument is forward and crossed if it is backward.

generalized forward com-position is sometimes only accepted as harmonic if all slashes of the secondary argument are forward (see e.g.


Lapata and Barzilay (2005) and Barzilay and Lapata (2008) both show the effectiveness of entity-based coherence in evaluating summaries.

fewer than five automatic summarizers were used in these studies.


Since abstractive summaries would have markedly different properties from extracts, it would be interesting to know how well these sets of features would work for predicting the quality of machineproduced abstracts.

since current systems are extractive, such a data set is not available.


In other words, if a node is not part of the context itself, we assume it has no effect on its neighbors' classes.

if i is in class 1 its belief about its neighbor j is determined by their mutual lexical similarity.


If this similarity is close to 1 it indicates a stronger tie between i, j.

if i, j are not similar, i's probability of being in class 1, should not affect that of j's.


They propose a scheme which first identifies and assigns categories to the opinion segments as reporting, judgment, advice, or sentiment; and then links the opinion segments with each other via rhetorical relations including contrast, correction, support, result, or continuation.

in contrast to our scheme and other schemes, instead of marking expression boundaries without any restriction they annotate an opinion segment only if it contains an opinion word from their lexicon, or if it has a rhetorical relation to another opinion segment.


Later on, it may also include its own set of resources specifically engineered for the target language as a performance improvement.

keeping the systems up-to-date would require as much effort as the number of languages.


quality-prediction or confidence estimation at sentence-or word-level fits best a scenario in which automated translation is only a part of a larger pipeline.


In the experiments presented in this paper, we use BLEU scores (Papineni et al., 2002) as training labels.

they can be substituted with any of the proposed MT metrics that use human-produced references to automatically as-sess translation quality (Doddington, 2002;Lavie and Agarwal, 2007).


(Specia et al., 2009a) and (Specia et al., 2009b).

to date most of the research has focused on better confidence measures for MT, e.g.


Our research is more similar in spirit to the third strand.

we use outputs and features from the TM explicitly; therefore instead of having to solve a regression problem, we only have to solve a much easier binary prediction problem which can be integrated into TMs in a straightforward manner.


With this boost, the precision of the baseline system can reach 0.85, demonstrating that a proper thresholding of fuzzy match scores can be used effectively to discriminate the recommendation of the TM hit from the recommendation of the SMT output.

using the TM information only does not always find the easiest-to-edit translation.


a misleading SMT output should not be recommended if there exists a poor but useful TM match (e.g.


For example, Chinese NE boundaries are especially difficult to identify because Chinese is not a tokenized language.

english Ne boundaries are easier to identify due to capitalization clues.


[ ] Lastly, the bilingual type re-assignment factor proposed in Eq (2) is derived as follows: As Eq (4) shows, both the Chinese initial NE type and English initial NE type are adopted to jointly identify their shared NE type RType .

the monolingual candidate certainty factors in Eq (2) indicate the likelihood that a re-generated NE candidate is the true NE given its originally detected NE.


This is due to the fact that maximizing likelihood does not imply minimizing the error rate.

with additional mapping constraints from the aligned sentence of another language, the alignment module could guide the searching process to converge to a more desirable point in the parameter space; and these additional constraints become more effective as the seed-corpus gets smaller.


Comparing one thing with another is a typical part of human decision making process.

it is not always easy to know what to compare and what are the alternatives.


The same techniques can be applied to comparative question identification and comparator mining from questions.

their methods typically can achieve high precision but suffer from low recall (Jindal and Liu, 2006b) (J&L).


Of the taxonomies presented by purely linguistic studies, our categories are most similar to those proposed by Warren (1978), whose categories (e.g., MATERIAL+ARTEFACT, OBJ+PART) are generally less ambiguous than Levi's.

to studies that claim the existence of a relatively small number of semantic relations, Downing (1977) presents a strong case for the existence of an unbounded number of relations.


We tested the relation set with an initial inter-annotator agreement study (our latest interannotator agreement study results are presented in Section 6).

the mediocre results indicated that the categories and/or their definitions needed refinement.


This idea is a modification of the selectional preference view of Wilks.

by using bigram counts over verb-noun pairs Krishnakumaran and Zhu (2007) loose a great deal of information compared to a system extracting verb-object relations from parsed text.


The latest developments in the lexical acquisition technology will in the near future enable fully automated corpusbased processing of metaphor.

there is still a clear need in a unified metaphor annotation procedure and creation of a large publicly available metaphor corpus.


in many cases two discourse arguments do not occur in the same sentence.


Simple-Expansion Min-Expansion could, to some degree, describe the syntactic relationships between the connective and arguments.

the syntactic properties of the argument pair might not be captured, because the tree structure surrounding the argument is not taken into consideration.


Because a named entity should correspond to a node in the parse tree, strong evidence about either aspect of the model should positively impact the other aspect.

designing joint models which actually improve performance has proven challenging.


Furthermore, parsing accuracy degrades unless sufficient amounts of labeled training data from the same domain are available (e.g., Gildea, 2001;Sekine, 1997), and thus we need larger and more varied annotated treebanks, covering a wide range of domains.

there is a bottleneck in obtaining annotation, due to the need for manual intervention in annotating a treebank.


Dependency elements with frequency below the lowest threshold have lower attachment scores (66.6% vs. 90.1% LAS), showing that simply using a complete rule helps sort dependencies.

frequency thresholds have fairly low precision, i.e., 33.4% at their best.


To obtain diversified translation outputs, most of the current system combination methods require multiple translation engines based on different models.

this requirement cannot be met in many cases, since we do not always have the access to multiple SMT engines due to the high cost of developing and tuning SMT systems.


Following Macherey and Och (2007)'s work,  proposed a feature subspace method to build a group of translation systems from various different sub-models of an existing SMT system.

's method relied on the heuristics used in feature sub-space selection.


Our formulation can be taken as a special instance of the structural learning framework in (Tsochantaridis et al., 2005).

they concentrate on more complicated label structures as for sequence alignment or parsing.


As shown earlier, for those two setups the structural SVMs perform better than the flat approach.

for the tree hierarchies of Brown that we deformed or flattened, and also BNC and Syracuse, either or both of the two balance scores tend to be lower, and no improvement has been obtained over the flat approach.


This may indicate that a further exploration of the relation between tree balance and the performance of structural SVMs is warranted.

high visual balance and distribution scores do not necessarily imply high performance of structural SVMs, as very flat trees are also visually very balanced.


It keeps an alignment score that is over 0.99 (the maximum is 1.00) toward the plen matrix, and still has visible block patterns.

the IC-jng-word-bnc significantly adjusts the distance entries, has a much lower alignment score with the plen matrix, and doesn't reveal apparent blocks.


Together, the SMS corpus and its transcription constitute parallel corpora aligned at the message-level.

in order to learn pieces of knowledge from these corpora, we needed a string alignment at the character-level.


This is the case, for instance, for the SMS form kozer ([koze]) and its standard transcription causé ("talked"), as illustrated by Figure 2.

from a linguistic standpoint, alignment (1) is preferable, because corresponding graphemes are aligned on their first character.


Evaluated by ten-fold cross-validation, the system seems efficient, and the performance in terms of BLEU score and WER are quite encouraging.

the SER remains too high, which emphasizes the fact that the system needs several improvements.


The discriminative approach (Table 3) is flexible enough to utilize all kinds of alignments.

the M-M models perform clearly better than 1-1 models.


Most current event extraction systems rely on local information at the phrase or sentence level.

this local context may be insufficient to resolve ambiguities in identifying particular types of events; information from a wider scope can serve to resolve some of these ambiguities.


However, it does not recognize the trigger "stepped" (trigger of End-Position) because in the training corpus "stepped" does not always appear as an End-Position event, and local context does not provide enough information for the MaxEnt model to tag it as a trigger.

in the document that contains related events like Start-Position, "stepped" is more likely to be tagged as an End-Position event.


Events co-occurring with die events with conditional probability > 10% As there are 33 subtypes, there are potentially 33⋅32/2=528 event pairs.

only a few of these appear with substantial frequency.


We will not use information in the conflict table to infer the event type or argument/roles for other event mentions, because we cannot confidently resolve the conflict.

the event type and argument/role assignments in the conflict table will be included in the final output because the local confidence for the individual assignments is high.


The dialogue should be easy and intuitive, otherwise the user will not find it worth the effort and instead prefer to use manual controls or to speak to a human.

when designing an in-vehicle dialogue system there is one more thing that needs to be taken into consideration, namely the fact that the user is performing an additional, safety critical, task -driving.


Instead, the dialogue system should be able to either pause until the workload is low or change topic and/or domain, and then resume where the interruption took place.

resumption of an interrupted topic needs to be done in a way that minimizes the risk that the cognitive workload increases again.


Traditional accounts of learning typically rely on linguistic annotation (Zettlemoyer and Collins, 2009) or word distributions (Curran, 2003).

we present an apprenticeship learning system which learns to imitate human instruction following, without linguistic annotation.


The vectors in different languages are first mapped to a common space using an initial bilingual dictionary, and then compared.

there is no previous work that uses the VSM to compute sense similarity for terms from parallel corpora.


1 There has been a lot of work (more details in Section 7) on applying word sense disambiguation (WSD) techniques in SMT for translation selection.

wSD techniques for SMT do so indirectly, using source-side context to help select a particular translation for a source rule.


It could be that the gains we obtained come simply from biasing the system against such rules.

the results in table 6 show that this is unlikely to be the case: features that just count context words help very little.


In this paper, we model the union of cross-lingual links provided by all editions of Wikipedia as an undirected graph G = (V, E) with edge weights w(e) for e ∈ E. In our experiments, we simply honour each individual link equally by defining w(e) = 2 if there are reciprocal links between the two pages, 1 if there is a single link, and 0 otherwise.

our framework is flexible enough to deal with more advanced weighting schemes, e.g.


Active learning (AL) has been applied to SMT recently  but they were interested in starting with a tiny seed set of data, and they stopped their investigations after only adding a relatively tiny amount of data as depicted in Figure 1.

we are interested in applying AL when a large amount of data already exists as is the case for many important lanuage pairs.


Also note how relatively early on in the process previous studies were terminated.

the focus of our main experiments doesn't even begin until much higher performance has already been achieved with a period of diminishing returns firmly established.


The other major difference is that  measure annotation cost by # of sentences.

we bring to light some potential drawbacks of this practice, showing it can lead to different conclusions than if other annotation cost metrics are used, such as time and money, which are the metrics that we use.


From Figure 3 it might appear that VG selection is better than random selection, achieving higher-performing systems with fewer translations in the labeled data.

it is important to take care when measuring annotation costs (especially for relatively complicated tasks such as translation).


For example, in the troops stationed in Iraq, the verb stationed is a VBN; troops is the head of the phrase.

for the troops vacationed in Iraq, the verb vacationed is a VBD and also the head.


Obviously, it needs exponential time to compute the above fractional counts.

due to the property of forest that compactly represents all the parse trees, the posterior probability of a subtree in a forest, | , , can be easily computed in an Inside-Outside fashion as the product of three parts: the outside probability of its root node, the probabilities of parse hyperedges involved in the subtree, and the inside probabilities of its leaf nodes (Lari and Young, 1990;Mi and Huang, 2008).


In SRL, constituents are used as the labeling units to form the labeled arguments.

previous work  shows that if we use complete constituent (MCT) as done in SRL to represent relation instance, there is a large performance drop compared with using the path-enclosed tree (PT) 6 .


This paper defines Strictly k-Piecewise (SP k ) distributions and shows how they too can be efficiently estimated from positive data.

with the Markov assumption, our assumption is that the probability of the next symbol is conditioned on the previous set of discontiguous subsequences of length k − 1 in the string.


A straightforward way for cross-language document summarization is to translate the summary from the source language to the target language by using machine translation services.

though machine translation techniques have been advanced a lot, the machine translation quality is far from satisfactory, and in many cases, the translated texts are hard to understand.


Crossover is performed under the assumption that new solutions can be improved by re-using the good parts of old solutions.

it is good to keep some part of population from one generation to the next.


One may come up with more sophisticated base distributions.

the main point of the base distribution is to encode a controllable preference towards simpler rules; we therefore make the simplest possible assumption.


2 2 One reviewer was concerned that since we explicitly disallow insertion rules in our sampling procedure, our model that generates such rules wastes probability mass and is therefore "deficient".

we regard sampling as a separate step from the data generation process, in which we can formulate more effective algorithms by using our domain knowledge that our data set was created by annotators who were instructed to delete words only.


It is important to note that the sampler described can move from any derivation to any other derivation with positive probability (if only, for example, by virtue of fully merging and then resegmenting), which guarantees convergence to the posterior (3).

some of these transition probabilities can be extremely small due to passing through low probability states with large elementary trees; in turn, the sampling procedure is prone to local modes.


Such annotated resources are scarce and expensive to create, motivating the need for unsupervised or semi-supervised techniques (Poon and Domingos, 2009).

unsupervised methods have their own challenges: they are not always able to discover semantic equivalences of lexical entries or logical forms or, on the contrary, cluster semantically different or even opposite expressions (Poon and Domingos, 2009).


Similarly, in the example mentioned earlier, when describing a forecast for a day with expected south winds, texts in the group can use either "south wind" or "southerly" to indicate this fact but no texts would verbalize it as "wind from west", and therefore these expressions will be assigned to different semantic clusters.

it is important to note that the phrase "wind from west" may still appear in the texts, but in reference to other time periods, underlying the need for modeling alignment between grouped texts and their latent meaning representation.


An even simpler technique would be to parse texts in a random order conditioning each meaning m k for k ∈ {1,..., K} on all the previous semantics m <k = m 1 ,..., m k−1 : Here, and in further discussion, we assume that the above search problem can be efficiently solved, exactly or approximately.

a major weakness of this algorithm is that decisions about components of the composite semantic representation (e.g., argument values) are made only on the basis of a single text, which first mentions the corresponding aspects, without consulting any future texts k > k, and these decisions cannot be revised later.


Note that, when generating fields, the Markov chain is defined over fields and the transition parameters are independent of the field values r if ij .

when drawing a word, the distribution of words is conditioned on the value of the corresponding field.


When the world state is observable, learning does not require any approximations, as dynamic programming (a form of the forward-backward algorithm) can be used to infer the posterior distribution on the E-step (Liang et al., 2009).

when the state is latent, dependencies are not local anymore, and approximate inference is required.


Note that the words "sun", "cloudiness" or "gaps" were not appearing in the labeled part of the data, but seem to be assigned to correct categories.

correlation between rain and overcast, as also noted in (Liang et al., 2009), results in the wrong assignment of the rain-related words to the field value corresponding to very cloudy weather.


There are many other potential applications, including automated storytelling , anaphora resolution (McTear, 1987), and information extraction (Rau et al., 1989).

it is also commonly accepted that the large-scale manual formalization of scripts is infeasible.


For instance, Mooney (1990) describes an early attempt to acquire causal chains, and Smith and Arnold (2009) use a graph-based algorithm to learn temporal script structures.

to our knowledge, such approaches have never been shown to generalize sufficiently for wide coverage application, and none of them was rigorously evaluated.


Jones and Thompson (2003) describe an approach to identifying different natural language realizations for the same event considering the temporal structure of a scenario.

they don't aim to acquire or represent the temporal structure of the whole script in the end.


Adding the function/content word split to the HMM structure improves both EM and VB estimation in terms of both tag matching accuracy and information.

these measures look at the parser only in isolation.


Experiments show that direct multi-participant models do not generalize to held out data, and likely never will, for practical reasons.

the Extended-Degree-of-Overlap model represents a suitable candidate for future work in this area, and is shown to successfully predict the distribution of speech in time and across participants in previously unseen conversations.


This policy is simplistic, and there is significant scope for more detailed back-off and interpolation.

such techniques infer values for under-estimated probabilities from shorter truncations of the conditioning history.


This shows that its ability to generalize to unseen data is higher than that of direct models.

in the easier mismatched D+C condition, it is outperformed by the CI model due to behavior differences among participants, which the EDO model small groups and large groups, represented in their study by K = 5 and K = 10, and noted that there is a smooth transition between the two extremes; this provides some scope for interpolating small-and large-group models, and the EDO framework makes this possible.


A user simulation for NLG is very similar, in that it is a predictive model of the most likely next user act.

4 this NLG predicted user act does not actually change the overall dialogue state (e.g.


Stability Previous work on stress in N/V pairs (Sherman, 1975;Phillips, 1984) has emphasized change, in particular {2,2}→{1,2} (the most common change).

an important aspect of the diachronic dynamics of N/V pairs is stability: most N/V pairs do not show variation or change.


As in Model 4, the bifurcation structure implies all 6 possible changes between the three FPs.

change to {1,2} entails crossing the hyperplanes λ 11 =λ 12 and λ 2 =λ 12 , and is thus now frequency dependent.


(In Model 5, stable variation very near {1,1} or {2,2} is possible.)

{1,1} and {2,2} are diachronically very stable stress patterns, suggesting that at least for this model set, assuming mistransmission in the learner is problematic.


If we were to rely on Levenshtein distance, these words would seem to be a highly attractive match as cognates: they are nearly identical, essentially differing in only a single character.

no linguist would posit that these two words are related.


tween some approximating messageμ(w) and the true message µ(w).

messages are not always probability distributions and -because the number of possible strings is in principle infinitethey need not sum to a finite number.


So, the rule would yield the production q 1 w − → σ(γ(q 3 ), q 2 ) in the domain projection.

a deleting rule such as q x 2 ) necessitates the introduction of a new nonterminal ⊥ that can generate all of T Σ with weight 1.


Unfortunately, of the classes that preserve recognizability, only wLNT is closed under composition (Gécseg and Steinby, 1984;Baker, 1979;Fülöp and Vogler, 2009).

the general lack of composability of tree transducers does not preclude us from conducting forward application of a cascade.


The main purpose of this paper has been to present novel algorithms for performing application.

it is important to demonstrate these algorithms on real data.


The previously described implicit grammar G I defines a posterior distribution P (d I |s) over a sentence s via a large, indexed PCFG.

this distribution has the property that, when marginalized, it is equivalent to a posterior distribution P (d|s) over derivations in the correspondingly-weighted all-fragments grammar G. even with an explicit representation of G, we would not be able to tractably compute the parse that maximizes P (t|s) = d∈t P (d|s) = d I ∈t P (d I |s) (Sima'an, 1996).


(1998), Petrov and Klein (2007), ), that a certain amount of pruning helps accuracy, perhaps by promoting agreement between the coarse and full grammars (model intersection).

these 'fortuitous' search errors give only a small improvement and the peak accuracy is almost equal to the parsing accuracy without any pruning (as seen in Figure 5).


The implicit all-fragments approach (Section 2.2) avoids explicit extraction of all rule fragments.

the number of indexed symbols in our implicit grammar G I is still large, because every node in each training tree (i.e., every symbol token) has a unique indexed symbol.


Sutton and McCallum (2005) adopted a probabilistic SRL system to re-rank the N-best results of a probabilistic syntactic parser.

they reported negative results, which they blamed on the inaccurate probability estimates from their locally trained SRL model.


In particular, the recent shared tasks of CoNLL 2008(Surdeanu et al., 2008Hajic et al., 2009) tackled joint parsing of syntactic and semantic dependencies.

all the top 5 reported systems decoupled the tasks, rather than building joint models.


The integrated parsing approach as shown in Section 4.2 performs syntactic and semantic parsing synchronously.

to traditional syntactic parsers where no semantic role-related information is used, it may be interesting to investigate the contribution of such information in the syntactic parsing model, due to the availability of such information in the syntactic parsing process.


Therefore, such syntactic errors should be avoidable by considering those arguments already obtained in the bottom-up parsing process.

taking those expected semantic roles into account would help the syntactic parser.


After the replacement, the reviews about IBM, Apple, and Dell will not share vocabularies with each other.

for any three created words which represent the same English word, we add three edges among them, and therefore we get a simulated dictionary graph for our PCLSA model.


The words 'fault', 'mistake', 'fail' or 'miss' can be used as the nonliteral paraphrases.

it is also highly likely that these words are used to describe a scenario in a baseball game, in which 'drop the ball' is used literally.


For example θ Sentence specifies the distribution of documents in the corpus.

it is easy to see that these distributions do not influence the topic distributions; indeed, the expansions of the Sentence nonterminal are completely determined by the document distribution in the corpus, and are not affected by θ Sentence ).


For example, in an NP coreference application, if we could determine that Bill and Hillary are both first names then we could infer that Bill Clinton and Hillary Clinton are likely to refer to distinct individuals.

because Mr in Mr Clinton is not a first name, it is possible that Mr Clinton and Bill Clinton refer to the same individual (Elsner et al., 2009).


Section 2.1), and (b) given the dependency graph of the sentence embedding the annotation phrase, we consider the distance between words for each dependency link within the annotation phrase and consider the maximum over such dis- 8 In preliminary experiments our set of basic features comprised additional features providing information on the usage of stop words in the annotation phrase and on the number of paragraphs, sentences, and words in the respective annotation example.

since we found these features did not have any significant impact on the model, we removed them.


The fact that a Weakly Interactive system can simulate the result of an experiment proposed in support of the Strongly Interactive hypothesis is initially counter-intuitive.

this naturally falls out from our decision to use a probabilistic   model: a lower probability, even in an unambiguous structure, is associated with increased reading difficulty.


2 Common characteristics of RTE systems re-ported by their designers were the use of structured representations of shallow semantic content (such as augmented dependency parse trees and semantic role labels); the application of NLP resources such as Named Entity recognizers, syntactic and dependency parsers, and coreference resolvers; and the use of special-purpose ad-hoc modules designed to address specific entailment phenomena the researchers had identified, such as the need for numeric reasoning.

it is not possible to objectively assess the role these capabilities play in each system's performance from the system outputs alone.


The understanding that the second sentence of the text entails the hypothesis draws on two coreference relationships, namely that he is Oswald, and that the Kennedy in question is President Kennedy.

the utilization of discourse information for such inferences has been so far limited mainly to the substitution of nominal coreferents, while many aspects of the interface between discourse and semantic inference needs remain unexplored.


The results of recent studies, as reported in Section 2.2, seem to show that current resolution of discourse references in RTE systems hardly affects performance.

our intuition is that these results can be attributed to four major limitations shared by these studies: (1) the datasets, where discourse phenomena were not well repre-sented; (2) the off-the-shelf coreference resolution systems which may have been not robust enough; (3) the limitation to nominal coreference; and (4) overly simple integration of reference information into the inference engines.


Sometimes, these rules are generally applicable (e.g., 'Alaska → Arctic').


Template extraction We parse the corpus with a dependency parser and extract all propositional templates from every parse tree, employing the procedure used by Lin and Pantel (2001).

we only consider templates containing a predicate term and arguments 3 .


's in that both try to learn graph edges given a transitivity constraint.

there are two key differences in the model and in the optimization algorithm.


The most well known international evaluation on the factoid QA task is the Text REtrieval Conference (TREC) 1 , and the annotated questions and answers released by TREC have become important resources for the researchers.

when facing a non-factoid question such as why, how, or what about, however, almost no automatic QA systems work very well.


In other words, the coreference task as defined by MUC and ACE is geared toward only identifying coreference relations anchored to an entity within the text.

to this research trend, investigations of referential behaviour in real world situations have continued to gain interest in the language generation community (Di Eugenio et al., 2000;Byron, 2005;van Deemter, 2007;Foster et al., 2008;Spanger et al., 2009), aiming at applications such as human-robot interaction.


Thus, in addition to the artificial nature of interaction, such as using keyboard input, this corpus only records restricted types of data.

though the annotated corpus by Spanger et al.


In the former case, the model with extra-linguistic information improved by about 22% compared with the baseline model.

in the latter case, the accuracy improved by only 7% over the baseline model.


First, inference under arbitrary priors can become complex.

in the simple case of our diagonal covariance Gaussians, the gradient of the observed data likelihood can be computed directly using the DMV's expected counts and maximum-likelihood estimation can be accomplished by applying standard gradient optimization methods.


That feature will be used in the log-linear attachment probability for English.

because that feature does not show up in any other language, it is not usefully controlled by the prior.


It is reasonable to worry that the improvements from these multilingual models might be partially due to having more total training data in the multilingual setting.

we found that halving the amount of data used to train the English, Dutch, and Swedish (the languages with the most training data) monolingual models did not substantially affect their performance, suggesting that for languages with several thousand sentences or more, the increase in statistical support due to additional monolingual data was not an important effect (the DMV is a relatively low-capacity model in any case).


The ALL-PAIRS model achieved an average relative error reduction of 17.1%, certainly outperforming both the simple phylogenetic models.

the rich phylogeny of the LINGUISTIC model, which incorporates linguistic constraints, outperformed the freer ALLPAIRS model.


For this reason we do not assign a cluster to punctuation marks and we report results using this policy, which we recommend for future work.

to be able to directly compare with previous work, we also report results for the full POS tag set.


The settings of the various experiments vary in terms of the exact annotation scheme used (coarse or fine grained) and the size of the test set.

the score differences are sufficiently large to justify the claim that our algorithm is currently the best performing algorithm on the PTB-WSJ corpus for POS induction from plain text 4 .


Some of the methods are suitable for retrieval of numerical attributes.

most of them do not exploit the numerical nature of the attribute data.


All of the above numerical attribute extraction systems utilize only direct information available in the discovered object-attribute co-occurrences and their contexts.

as we show, indirect information available for comparable objects can contribute significantly to the selection of the obtained values.


In the QA scenario, if we are given the full question and not just the (object, attribute) pair we can add terms appearing in the question and having a strong PMI with the object (this can be estimated using any fixed corpus).


Since our goal is to obtain a value for the single requested object, if at the end of this stage we remain with a single value, no further processing is needed.

if we obtain a set of values or no values at all, we have to utilize comparison data to select one of the retrieved values or to approximate the value in case we do not have an exact answer.


Furthermore, part-whole relations are de-facto benchmarks for evaluating the performance of general relation extraction systems (Pantel and Pennacchiotti, 2006;Beamer et al., 2008;Pyysalo et al., 2009).

these relation extraction efforts have overlooked the ontological distinctions between the different types of part-whole relations.


5 Initially,we selected seeds from WordNet (Fellbaum, 1998) (for English) and EuroWordNet (Vossen, 1998) (for Dutch) to initialize the IE algorithm.

we found that these pairs, such as acinos-mother of thyme or radarschermradarapparatuur (radar screen -radar equipment, hardly co-occured with reasonable frequency in Wikipedia sentences, hindering pattern extraction.


Although this assumption is not always correct in practice, we consider it a reasonable approximation given what we empirically observed in our training data.

to standard CRFs, semi-Markov CRFs directly model the segmentation of an input sequence as well as a classification of the segments (Sarawagi and Cohen, 2004) In this case, the features f (s j−1 , s j , x) are defined on segments instead of on word tokens.


Although there is no significant topic drift in this case, there are not many relevant terms apart from the query terms.

the same query performs very well in English with all the documents in the feedback set of the English corpus being relevant, thus resulting in informative feedback terms such as {bovin, scientif, recherch}.


As can be observed, this causes terms like sampras to come up in the MultiPRF model.

while the MultiPRF model has some terms pertaining to Men's Winners of Wimbledon as well, the original feedback model suffers from severe topic drift, with irrelevant terms such as {telefonbuch, telekom} also amongst the top terms.


In traditional information retrieval (IR) bag-of-word representation is the most common way to express information needs.

in opinion retrieval, information need target at relevant opinion, and this renders bag-of-word representation ineffective.


In Figure 3, we can see that the word pair that has links to many documents can be assigned a high weight to denote a strong associative degree between the topic term and a sentiment word, and it likely expresses a relevant opinion.

if a document has links to many word pairs, the document is with many relevant opinions, and it will result in high ranking.


They divided the sentiment words into query-dependent and query-independent by utilizing several sentiment expansion techniques, and integrated them into a mixed model.

in this model, the contribution of a sentiment word was its corresponding incremental mean average precision value.


On the one hand, word pair can identify the relevant opinion according to intra-sentence contextual information.

it can measure the degree of a relevant opinion by considering the inter-sentence contextual information.


A Music RS could be developed along the lines of Product RSs.

music RSs recommend individual tracks, not full albums, e.g.


The utilization of tf i in classification is rather straightforward and intuitive but, as previously discussed, usually results in decreased accuracy in sentiment analysis.

using idf to assign weights to features is less intuitive, since it only provides information about the general distribution of term i amongst documents of all classes, without providing any additional evidence of class preference.


Datasets One dataset we employed is the automatic content extraction (ACE) (ACE-Event, 2005).

the utilization of the ACE corpus for the task of solving event coreference is limited because this resource provides only withindocument event coreference annotations using a restricted set of event types such as LIFE, BUSI-NESS, CONFLICT, and JUSTICE.


In ACE, nominal predicates corefer with their subject (2), and appositive phrases corefer with the noun they are modifying (3).

they do not fall under the identity relation in OntoNotes, which follows the linguistic understanding of coreference according to which nominal predicates and appositives express properties of an entity rather than refer to a second (coreferent) entity (van Deemter and Kibble, 2000).


The ACE scores for the automatically preprocessed models in Table 7 are about 3% lower than those based on OntoNotes gold-standard data in Table 5, providing evidence for the advantage offered by gold-standard preprocessing information.

the similar-if not higher-scores of OntoNotes can be attributed to the use of the annotated ACE entity types.


However, they have a major limitation that they do not have a principled mechanism to guarantee grammaticality on the target side, since there is no linguistic tree structure of the output.

string-to-tree systems explicitly model the grammaticality of the output by using target syntactic trees.


Both string-toconstituency system (e.g., (Galley et al., 2006;) and string-to-dependency model (Shen et al., 2008) have achieved significant improvements over the state-of-the-art formally syntax-based system Hiero (Chiang, 2007).

those systems also have some limitations that they run slowly (in cubic time) , and do not utilize the useful syntactic information on the source side.


We first use pattern-matching algorithm of  to convert F c into a translation forest, each hyperedge of which is associated with a constituency to dependency translation rule.

pattern-matching failure 2 at a node v f will cut the derivation path and lead to translation failure.


Of course, the characteristics of our aligned corpus may not hold for other annotated corpora or other language pairs.

we hope that the overall effectiveness of our modeling approach will influence future annotation efforts to build corpora that are consistent with this interpretation.


This model scores overlapping phrasal rules contained within terminal blocks that result from including or excluding possible links.

this model does not score bispans that cross bracketing of ITG derivations.


As can be seen in table 1, states are rather immanent operations and achievements are those occur in a single moment or operations related to perception level.

activity and accomplishment are processes (transeunt operations) in traditional philosophy.


Since each extracted action has its probability, we can use the value as a feature for state / activity verb classification.

a verb may appear in different contexts and can have multiple    Evidence of a word w being an action in eHow is denoted as E(w) where variable t is the sum of individual action probability values in D w the set of documents from which the word w has been extracted as an action.


In each group, MAD is the rightmost bar.

to graph construction from structured tables as in Sections 3.1, 3.2, in this section we use hypernym tuples extracted by TextRunner (Banko et al., 2007), an open domain IE system, to construct the graph.


We find that the objective minimized by LP-ZGL (Equation 1) is underregularized, i.e., its model parameters (Ŷ ) are not constrained enough, compared to MAD (Equation 3, specifically the third term), resulting in overfitting in case of highly connected graphs.

mAD is able to avoid such overfitting because of its minimization of a well regularized objective (Equation 3).


This further illustrates the advantage of aggregating information across sources (Talukdar et al., 2008;Pennacchiotti and Pantel, 2009).

we are the first, to the best of our knowledge, to demonstrate the effectiveness of attributes in class-instance acquisition.


(Pennacchiotti and Pantel, 2006) proposed an algorithm for automatically ontologizing semantic relations into WordNet.

despite its high precision entries, WordNet's limited coverage makes it impossible for relations whose arguments are not present in WordNet to be incorporated.


Initially, we decided to conduct an automatic evaluation comparing our results to knowledge bases that have been extracted in a similar way (i.e., through pattern application over unstructured text).

it is not always possible to perform a complete comparison, because either researchers have not fully explored the same relations we have studied, or for those relations that overlap, the gold standard data was not available.


For the "live in" relation, both repositories contain the same city and country names.

the recursive pattern learned arguments like pain, effort which express a manner of living, and locations like slums, box.


It follows immediately from Grenander's result that and hence λ opt (x, θ * ) ≤ µ x /τ for any θ * , hence Λ(x; θ I ) ≤ µ x /τ .

if we choose θ 0 = θ I , we have that p(z | θ 0 , x) > µ x for some z , hence, for θ * such that it assigns probability τ on z , we have that and hence λ opt (x, θ * ; θ ) > µ x /τ , so Λ(x; θ ) > µ x /τ .


The wide domains of the variables may be advantageous for some integer programming solvers as well.

it creates an integer programming problem of size exponential in the number of sets.


These results show some significant improvements in vector length for the larger modules.


That is a shallow minimum; both λ = 120 and λ = 160 gave vector lengths of 986, and the length slowly increased with greater λ.

the fastest bit-count on this architec- ture, using a technique first published by Wegner (1960), requires time increasing with the number of nonzero bits it counts; and a similar effect would appear on a word-by-word basis even if we used a constant-time per-word count.


Producing this kind of knowledge is extremely costly: at a throughput of one sense annotation per minute (Edmonds, 2000) and tagging one thousand examples per word, dozens of person-years would be required for enabling a supervised classifier to disambiguate all the words in the English lexicon with high accuracy.

knowledge-based approaches exploit the information contained in wide-coverage lexical resources, such as WordNet (Fellbaum, 1998).


In contrast, knowledge-based approaches exploit the information contained in wide-coverage lexical resources, such as WordNet (Fellbaum, 1998).

it has been demonstrated that the amount of lexical and semantic information contained in such resources is typically insufficient for high-performance WSD (Cuadros and Rigau, 2006).


The better performance of Wikipedia against WordNet when using ExtLesk (+3.7%) highlights the quality of the relations extracted.

no such improvement is found with De-  gree, due to its lower recall.


In this paper, we focused on English Word Sense Disambiguation.

since WordNet++ is part of a multilingual semantic network (Navigli and Ponzetto, 2010), we plan to explore the impact of this knowledge in a multilingual setting.


(2009b) for unsupervised and knowledge based approaches respectively have cast a doubt on the viability of supervised approaches which rely on sense tagged corpora.

these conclusions were drawn only from the performance on certain target words, leaving open the question of their utility in all words WSD.


Our modifications are described as follows: JCN for Verb-Verb Similarity In our implementation of the In-Degree algorithm, we use the JCN similarity measure for both Noun-Noun similarity calculation similar to SM07.

different from SM07, instead of using LCH for Verb-Verb similarity, we use the JCN metric as it yields better performance in our experimentations.


We still rely on parallel corpora, we extract typesets based on the intersection of word alignments in both alignment directions using more advanced GIZA++ machinery.

to DR02, we experiment with all four POS: Verbs (V), Nouns (N), Adjectives (A) and Adverbs (R).


In this combination scheme, the words in the typeset that result from the TransCont approach are added to the context of the target word in the RelCont approach.

the typeset words are not treated the same as the words that come from the surrounding context in the In-Degree algorithm as we recognize that words that are yielded in the typesets are semantically similar in terms of content rather than being co-occurring words as is the case for contextual words in Rel-Cont.


contiguous words belonging to the same semantic stack are modelled as an atomic observation unit or phrase.

1 with wordlevel models, a major advantage of phrase-based generation models is that they can model longrange dependencies and domain-specific idiomatic phrases with fewer parameters.


Again, we look for nodes that can be merged based on the identity of the actions involved and the (WordNet) similarity of their arguments.

we disallow the merging of nodes with focal entities appearing in the same argument slot (e.g., "[prince, princess] cries").


Ideally, the realizer should also select an appropriate tense for the sentence.

we make the simplifying assumption that all sentences are in the present tense.


Compared to previous work (e.g., Karamanis and Manurung 2002) our crossover rate may seem low and the mutation rate high.

it makes intuitively sense, as high crossover may lead to incoherence by disrupting canonical action sequences found in the plots.


However, it makes intuitively sense, as high crossover may lead to incoherence by disrupting canonical action sequences found in the plots.

a higher mutation will raise the likelihood of a lexical item being swapped for another and may improve overall coherence and interest.


Let's say, for example, that we have a knowledge base Then we can combine instances of the trees for "John", "pushes", and "the button" into a grammatically complete derivation.

because both b 1 and b 2 satisfy the semantic content of "the button", we must adjoin "red" into the derivation to make the RE refer uniquely to b 1 .


"upper left" vs. "left upper" example above).

there are also other constraints at work: "?


We did not find any significant differences in the success rates or task completion times between this system and SCRISP, but the former achieved a higher RE success rate (see Table 2).

a closer analysis shows that SCRISP was able to generate REs from significantly further away.


Dyer (2009) also employed a segmentation lattice, which represents ambiguities of compound word segmentation in German, Hungarian and Turkish translation.

to the best of our knowledge, there is no work which employed a lattice representing paraphrases of an input sentence.


However, to the best of our knowledge, there is no work which employed a lattice representing paraphrases of an input sentence.

paraphrasing has been used to enrich the SMT model.


Linguistic parse trees can provide very useful reordering constraints for SMT.

they are far from perfect because of both parsing errors and the crossing of the constituents and formal phrases extracted from parallel training data.


E n itself may have more states than E due to the association of distinct n-gram histories with states.

the counting transducer for unigrams is simpler than the corresponding counting transducer for higherorder n-grams.


The log semiring ǫ-removal and determinization required to sum the probabilities of paths labelled with each u can be slow.

if we use the proposed Ψ R n , then each path in E n • Ψ R n has only one non-ǫ output label u and all paths leading to a given final state share the same u.


These features, which we denote f (n i , g j , D), may depend on their relative position in the document D, and on any features of g j , since we have already generated its tree.

we cannot extract features from the subtree under n i , since we have yet to generate it!


We found that the interpolation approach resulted in a substantial improvement in the performance of the PCFG model for all but the Football dataset (discussed below).

for some datasets, even this improvement was not sufficient to outperform the best baseline.


This provides a measure of robustness, and previous evaluations of ASR in spoken tutorial dialogue systems indicate that neither word error rate nor concept error rate in such systems affect learning gain (Litman and Forbes-Riley, 2005;Pon-Barry et al., 2004).

limiting the range of possible input limits the contentful talk that the students are expected to produce, and therefore may limit the overall effectiveness of the system.


In informal comments after the session many students said that they were frustrated when the system said that it did not understand them.

some students in BASE also mentioned that they sometimes were not sure if the system's answer was correcting a problem with their answer, or simply phrasing it in a different way.


2 The frequency of nonunderstandings was negatively correlated with learning gain in FULL: r = −0.47, p < 0.005, but not significantly correlated with learning gain in BASE: r = −0.09, p = 0.59.

in both conditions the frequency of non-understandings was negatively correlated with user satisfaction: FULL r = −0.36, p = 0.03, BASE r = −0.4, p = 0.01.


This is provides evidence that such errors are negatively impacting the learning process, and therefore improving recovery strategies for those error types is likely to improve overall system effectiveness, The results, shown in Table 1, indicate that the majority of interpretation problems are not significantly correlated with learning gain.

several types of problems appear to be particularly significant, and are all related to improper use of domain terminology.


OntoNotes comes closest to providing a corpus with multiple layers of annotation that can be analyzed as a unit via its representation of the annotations in a "normal form".

like the Wall Street Journal corpus, OntoNotes is limited in the range of genres it includes.


Also, they can be used directly for testing paraphrase applicability (Szpektor et al., 2008), a task that has recently become prominent in the context of textual entailment (Bar-Haim et al., 2007).

polysemy is a fundamental problem for distributional models.


In predicate-argument structure analysis, it is important to capture non-local dependencies among arguments and interdependencies between the sense of a predicate and the semantic roles of its arguments.

no existing approach explicitly handles both non-local dependencies and semantic dependencies between predicates and arguments.


(2010) proposed a generative model that captures both predicate senses and its argument roles.

the first-order markov assumption of the model eliminates ability to capture non-local dependencies among arguments.


The systems by Björkelund and Zhao applied feature selection algorithms in order to select the best set of feature templates for each language, requiring about 1 to 2 months to obtain the best feature set.

our system achieved the competitive results with the top two systems, despite the fact that we used the same feature templates for all languages without applying any feature engineering procedure.


Under independence, this estimate will be high, as the itself is very frequent.

with our knowledge of English syntax, we would say p exp (the the) is low.


The event information is more effective to express the information about the term dependencies while the unigram RM ignores this information and only takes  Table 3: Performance (MAP) comparison of query expansion using the combination of RM and term dependencies the occurrence frequencies of individual words into account, which is not well-captured by the events.

the performance of Scheme 2 is more promising.


Taxonomy deduction is an important task to understand and manage information.

building taxonomies manually for specific domains or data sources is time consuming and expensive.


The quality of extraction is often controlled using statistical measures (Pantel and Pennacchiotti, 2006) and external resources such as wordnet (Girju et al., 2006).

there are domains (such as the one introduced in Section 3.2) where the text does not allow the derivation of linguistic relations.


Semi-supervised approaches start with known terms belonging to a category, construct context vectors of classified terms, and associate categories to previously unclassified terms depending on the similarity of their context (Tanev and Magnini, 2006).

providing training data and hand-crafted patterns can be tedious.


Unsupervised methods use clustering of wordcontext vectors (Lin, 1998), co-occurrence (Yang and Callan, 2008), and conjunction features (Caraballo, 1999) to discover implicit relationships.

these approaches do not perform well for small corpora.


It mostly holds that f (s) where s is a state name, d is a district name, c is a city name, and z is a ZIP code.

sometimes the name of a large city may be more frequent than the name of a small state.


In detail, these CNLs display some variations: thus an inclusion relationship between the classes Admiral and Sailor would be expressed by the pattern 'Admirals are a type of sailor' in CLOnE, 'Every admiral is a kind of sailor' in Rabbit, and 'Every admiral is a sailor' in ACE and SOS.

at the level of general strategy, all the CNLs rely on the same set of assumptions concerning the mapping from natural to formal language; for convenience we will refer to these assumptions as the consensus model.


An example of axiom-splitting rules is found in a computational complexity proof for the description logic EL+ (Baader et al., 2005), which requires class inclusion axioms to be rewritten to a maximally simple 'normal form' permitting only four patterns: A 2 , where P and all A N are atomic terms.

this simplification of axiom structure can be achieved only by introducing new atomic terms.


Our results indicate that although in principle the consensus model cannot guarantee transparent realisations, in practice these are almost always attainable, since ontology developers overwhelmingly favour terms and axioms with relatively simple content.

in an analysis of around 50 ontologies we have found that over 90% of axioms fit a mere seven patterns (table 2); the following examples show that each of these patterns can be verbalised by a clear unambiguous sentence -provided, of course, that no problems arise in lexicalising the atomic terms: since identifiers containing 3-4 words are fairly common (figure 1), we need to consider whether these formulations will remain transparent when combined with more complex lexical entries.


Part of speech tagging, constituent and dependency parsing, combinatory categorical grammar super tagging are used extensively in most applications when syntactic representations are needed.

training these tools require medium size treebanks and tagged data, which for most languages will not be available for a while.


Or, conversely, a sentence-alignment stage can be followed by a segmentation stage.

as we will see in our experiments, these strategies may result in poor segmentation and alignment quality.


Several formal evaluations have been conducted for the coreference resolution task (e.g., MUC-6 (1995), ACE NIST (2004)), and the data sets created for these evaluations have become standard benchmarks in the field (e.g., MUC and ACE data sets).

it is still frustratingly difficult to compare results across different coreference resolution systems.


2001) because of the popularity and relatively good performance of these systems.

there have been other approaches to coreference resolution, including unsupervised and semi-supervised approaches (e.g.


The performance for the location type argument improved drastically.

the total performance of the arguments was below the original TBL.


Unfortunately, it is very hard to rightly produce full parses for Chinese text.

given a constituent, SRL systems should identify whether it is an argument and further predict detailed semantic types if  it is an argument.


Our algorithm, TKA * is a variant of the kbest A * (KA *) algorithm of Pauls and Klein (2009).

to KA * , which performs an inside and outside pass before performing k-best extraction bottom up, TKA * performs only the inside pass before extracting k-best lists top down.


When building derivations bottom-up, the only way to expand a particular partial inside derivation is to combine it with another partial inside derivation to build a bigger tree.

an outside derivation item can be expanded anywhere along its frontier.


This study is preliminary, however, in that we have not yet shown improved end-to-end task performance applying this approach, such as improved BLEU scores in a machine translation task.

we believe there is reason to be optimistic about this.


FrameNet groups LUs in frames and describes relations between frames.

relations between LUs are not explicitly defined.


It is also possible that the small size of BL-I leads to overfitting and low accuracies.

pBA subset with only 151 items (only 2002 and 2003 speeches) is still 96% classifiable, so size alone does not explain low BL-I performance.


The co-training approach manages to boost the performance as it allows the text similarity in the target language to compete with the "fake" similarity from the translated texts.

the translated texts are still used as training data and thus can potentially mislead the classifier.


It is not a marginal phenomenon, since Kessler and Nicolov (2009) report that in their data, 14% of the opinion targets are pronouns.

the task of resolving anaphora to mine opinion targets has not been addressed and evaluated yet to the best of our knowledge.


A candidate selection or extraction step for the opinion targets is not required, since they rely on manually annotated targets and focus solely on the coreference resolution.

they do not resolve pronominal anaphora in order to achieve that.


(2006): The dependency paths only identify connections between pairs of single words.

almost 50% of the opinion target candidates are multiword expressions.


We observe that the MARS algorithm yields an improvement regarding recall compared to the baseline system.

it also extracts a high number of false positives for both the personal and impersonal / demonstrative pronouns.


Another direction is to divide the decoding into two steps of segmentation and conversion, which is this paper's method.

exact inference by listing all possible candidates explicitly and summing over all possible segmentations is intractable, because of the exponential computation complexity with the source word's increasing length.


Generally, the LOC and PER classes benefitted more from the head word features, SHW), than the other classes.

for the syntactic environment feature (SE), the PER class seemed not to benefit much from the presence of this feature.


Classical IE systems fill slots in domain-specific frames such as the time and location slots in seminar announcements (Freitag, 2000) or the terrorist organization slot in news stories (Chieu et al., 2003).

open IE systems are domainindependent, but extract "flat" sets of assertions that are not organized into frames and slots (Sekine, 2006;Banko et al., 2007).


This is consistent with previous work on redundancy-based extractors on the Web.

rEDUND still suffered from the problems of over-specification and over-generalization described in Section 2.


For example, when we read the title "China Tightens Grip on the Web", we can only have a glimpse of what the document says.

the key phrases, such as "China", "Censorship", "Web", "Domain name", "Internet", and "CNNIC", etc.


In large vocabulary speech recognition, a language model (LM) is typically estimated from large amounts of written text data.

recognition is typically applied to speech that is stylistically different from written language.


A LM is then built by interpolating the models estimated from large corpus of written language and the small corpus of transcribed data.

in practice, different models might be of different importance depending on the word context.


Moreover, training hierarchical ME models requires even more memory than training simple ME models, proportional to the number of nodes in the hierarchy.

it should be possible to alleviate this problem by profiting from the hierarchical nature of n-gram features, as proposed in (Wu and Khudanpur, 2002).


In fact, even when the SVM uses lexical as well as non-lexical features, its F1-score is still lower than the HGM classifier.

with the hierarchical SVM and rulebased DGM methods, the HGM method identifies decision-related utterances by exploiting not just DDAs but also direct dependencies between decision regions and UTT, DA, and PROS features.


This of course entails cognitive effort, which is very limited in the context of driving.

a dictation approach to replying to SMS messages may be far worse due to misrecognitions.


For instance, a correct dictation answer for Example (1) above was "no I'm never with my GPS".

the voice search condition had more cases (2-4 messages) in which the correct answer was not an exact copy (e.g., "no I have GPS") due to the nature of the template approach.


In other words, we could not reject the null hypothesis that the two approaches were the same in terms of their influence on driving performance.

for the SMS reply task, we did find a main effect for SMS Reply Approach (F 1,47 = 81.28, p < .001, µ Dictation = 2.13 (.19), µ VoiceSearch = .38 (.10)).


Although the reviews were randomly selected, 32 sentences extracted out of 16 reviews might seem like a small sample.

the upper time limit for reliable psycholinguistic experiments is 20-25 minute.


In 2 psycholinguistic and psychophysical experiments, we showed that rating whole customer-reviews as compared to rating final sentences of these reviews showed an (expected) insignificant difference.

rating whole customer-reviews as compared to rating second sentences of these reviews, showed a considerable difference.


We would like to do a direct comparison by simply runing the above systems on the exact same data and evaluating them the same way.

this unfortunately has to wait until new versions are released that work with the current version of the SAMA morphological analyzer and ATB.


Conceptually, these deduction rules operate by  first computing inside scores bottom-up in the coarsest grammar, then outside scores top-down in the same grammar, then inside scores in the next finest grammar, and so on.

the crucial aspect of HA * is that items from all levels of the hierarchy compete on the same queue, interleaving the computation of inside and outside scores at all levels.


This is clearly a classification problem which requires arriving at a binary decision for each entity in D (belonging to S or not).

in practice, the problem is often solved as a ranking problem, i.e., ranking the entities in D based on their likelihoods of belonging to S. The classic method for solving this problem is based on distributional similarity (Pantel et al.


If the values in V d are skewed towards the high side (negative skew), it means that the candidate entity is very likely to be a true entity, and we should take the median as it is also high (higher than the mean).

if the skew is towards the low side (positive skew), it means that the candidate entity is unlikely to be a true entity and we should again use the median as it is low (lower than the mean) under this condition.


Current techniques for this task typically bootstrap a classifier based on a fixed seed set.

our system involves the user throughout the labeling process, using active learning to intelligently explore the space of similar words.


Also, there are separate similarity lists for each of nouns, verbs, and modifiers; we only used the lists matching the seed word's part of speech.

given a seed set s and a complete target set g, it is easy to evaluate our system; we say "Yes" to anything in g, "No" to everything else, and see how many of the candidate words are in g. building a complete gold-standard g is in practice prohibitively difficult; instead, we are only capable of saying whether or not a word belongs to g when presented with that word.


Thus, flat phrases (Koehn et al., 2003), hierarchical phrases (Chiang, 2005), and syntactic tree fragments (Galley et al., 2006;Wu et al., 2010) are gradually used in SMT.

the use of syntactic phrases continues due to the requirement for phrase coverage in most syntax-based systems.


This is the only language-specific component of our translation model.

we expect this approach to work for other agglutinative languages as well.


3, all three translations correspond to the English text, 'with the basque nationalists.'

the CRF-LM output is more grammatical than the baseline, because not only do the adjective and noun agree for case, but the noun 'baskien' to which the postposition 'kanssa' belongs is marked with the correct genitive case.


1 1 One may further improve the index structure by using a trie rather than a ranking list to store βs associated with the same α.

the improvement would not be significant because the number of βs associated with each α is usually very small.


At least some of these clusters, when induced by maximizing the likelihood L(θ, α) with sufficiently large α, will be useful for the classification task on the source domain.

when the domains are substantially different, these predictive clusters are likely to be specific only to the source domain.


Minimizing the difference in the marginal distributions can be regarded as a coarse approximation to the minimization of the distance.

we have to concede that the above argument is fairly informal, as the generalization bounds do not directly apply to our case: (1) our feature representation is learned from the same data as the classifier, (2) we cannot guarantee that the existence of a domainindependent scoring function is preserved under the learned transformation x→z and (3) in our setting we have access not only to samples from P (z|x, θ) but also to the distribution itself.


The average error reductions for our method Reg+ and for the SCL method are virtually equal.

formally, these two numbers are not directly comparable.


This approach bears some similarity to the adaptation methods standard for the setting where labelled data is available for both domains (Chelba and Acero, 2004;Daumé and Marcu, 2006).

instead of ensuring that the classifier parameters are similar across domains, we favor models resulting in similar marginal distributions of latent variables.


Traditional search engines accept several terms submitted by a user as a query and return a set of docs that are relevant to the query.

for those users who are not search experts, it is always difficult to accurately specify some query terms to express their search purposes.


Regardless of different objectives, both methods derive hash functions via Principle Component Analysis (PCA) (Jolliffe, 1986).

pCA is computationally expensive, which limits their usage for high-dimensional data.


KLSH can improve hashing results via the kernel trick.

kLSH is unsupervised, thus designing a data-specific kernel remains a big challenge.


LDA seeks the best separative direction in the original attribute space.

s 3 H firstly maps data from R M to R M ×L through the following projection function where r l ∈ R M , l = 1, .


This implies FCC can also improve the satiability of S 3 H. As we see, S 3 H f ignores the contribution of features to different classes.

besides the local description of data locality in the form of object-pairs, such (global) information also provides a proper guidance for hashing.


We should emphasize that KLSH needs 0.3ms to return the results for a query document for hash lookup, and S 3 H needs <0.1ms.

iL requires about 75ms to finish searching.


It is essential for the search engine to correctly annotate the query structure, and the quality of these query annotations has been shown to be a crucial first step towards the development of reliable and robust query processing, representation and understanding algorithms (Barr et al., 2008;Guo et al., 2008;Guo et al., 2009;Manshadi and Li, 2009;Li, 2010).

in current query annotation systems, even sentence-like queries are often hard to parse and annotate, as they are prone to contain misspellings and idiosyncratic grammatical structures.


For instance, a keyword query hawaiian falls, which refers to a location, is inaccurately interpreted by a standard POS tagger as a noun-verb pair.

given a sentence from a corpus that is relevant to the query such as "Hawaiian Falls is a family-friendly waterpark", the word "falls" is correctly identified by a standard POS tagger as a proper noun.


For instance, using j-PRF always leads to statistically significant improvements over the i-PRF baseline for questions.

it is either statistically indistinguishable, or even significantly worse (in the case of capitalization) than the i-PRF baseline for the verbal phrases.


typically assume that classification models are trained and tested using data drawn from some fixed distribution.

in many practical cases, we may have plentiful labeled examples in the source domain, but very few or no labeled examples in the target domain with a different distribution.


For example, in the domain of reviews about electronics products, the words "durable" and "light" are used to express positive sentiment, whereas "expensive" and "short battery life" often indicate negative sentiment.

if we consider the books domain the words "exciting" and "thriller" express positive sentiment, whereas the words "boring" and "lengthy" usually express negative sentiment.


Methods that use related features have been successfully used in numerous tasks such as query expansion (Fang, 2008), and document classification (Shen et al., 2009).

feature expansion techniques have not previously been applied to the task of cross-domain sentiment classification.


Aue and Gammon (2005) report a number of empirical tests into domain adaptation of sentiment classifiers using an ensemble of classifiers.

most of these tests were unable to outperform a simple baseline classifier that is trained using all labeled data for all domains.


The selection of pivots is vital to the performance of SCL and heuristically selected pivot features might not guarantee the best performance on target domains.

our method uses all features when creating the thesaurus and selects a subset of features during training using L1 regularization.


However, not all words can be cleanly classified into domain specific or domain independent, and this process is conducted prior to training a classifier.

our method lets a particular lexical entry to be listed as a neighour for multiple base entries.


The semantic component of our model learns word vectors via an unsupervised probabilistic model of documents.

in keeping with linguistic and cognitive research arguing that expressive content and descriptive semantic content are distinct (Kaplan, 1999;Jay, 2000;Potts, 2007), we find that this basic model misses crucial sentiment information.


For instance, for the following tweet, which contains only three words, it is difficult for any existing approaches to classify its sentiment correctly.

relations between individual tweets are more common than those in other sentiment data.


Finally, a review is classified based on the average semantic orientation of the sentimental phrases in the review.

(Pang et al., 2002) treat the sentiment classification of movie reviews simply as a special case of a topic-based text categorization problem and investigate three classification algorithms: Naive Bayes, Maximum Entropy, and Support Vector Machines.


Many attempts have been made to extract these expressions from corpora, mainly using automated methods that exploit statistical means.

to our knowledge, no reliable, extensive solution has yet been made available, presumably because of the difficulty of extracting correctly without any human insight.


Without a rule system of semantic composition, it is difficult to evaluate the validity of the JDMWE concerning idiomaticity.

we can confirm that 3,600 Japanese standard idioms that Sato (2007) listed from five Japanese idiom dictionaries published for human readers are included in the JDMWE as a proper subset.


Although the CLC contains information about the linguistic errors committed (see Section 2), we try to extract an error-rate in a way that doesn't require manually tagged data.

we also use an error-rate calculated from the CLC error tags to obtain an upper bound for the performance of an automated error estimator (true CLC error-rate).


It is likely that a larger training set and/or more consistent grading of the existing training data would help to close this gap.

our system is not measuring some properties of the scripts, such as discourse cohesion or relevance to the prompt eliciting the text, that examiners will take into account.


In addition, the metric was shown to correlate well with human judgments.

a significant drawback of this approach is that PEM requires substantial in-domain bilingual data to train the semantic adequacy evaluator, as well as sample human judgments to train the overall metric.


For example, a paraphrase suggestion tool for a word processing software might be more concerned with semantic adequacy, since presenting a paraphrase that does not preserve the meaning would likely result in a negative user experience.

a query expansion algorithm might be less concerned with preserving the precise meaning so long as additional relevant terms are added to improve search recall.


We have so far used BLEU to measure semantic adequacy since it is the most common MT metric.

other more advanced MT metrics that have shown higher correlation with human judgments could also be used.


broadcasting news, force-protection, travel) is acceptable to users.

a remaining open question is how to predict confidence scores for machine translated words and sentences.


Experiments reporting in Section 4 indicate that the proposed confidence measure has a high correlation with HTER.

it is not very clear if the core MT system can benefit from confidence measure by providing better translations.


Exponential models and other classifiers have been used in several recent studies to condition phrase model probabilities on source-side context (Chan et al 2007;Carpuat and Wu 2007a;Carpuat and Wu 2007b).

this has been generally accomplished by training independent classifiers associated with different source phrases.


One can see that for both rule sets the estimated probabilities for rules observed a single time is only slightly  higher than probabilities for generated unobserved rules.

rules with relatively high counts in the second set receive proportionally higher estimates, while the difference between the singleton rule and the most frequent rule in the second set, which was observed 3 times, is smoothed away to an even greater extent.


With others, like ("even, number") and ("book, list"), the users seem to be encouraging collocations to be in the same topic.

the collocations may not be in any document in this corpus.


As presented here, the technique for incorporating constraints is closely tied to inference with Gibbs sampling.

most inference techniques are essentially optimization problems.


In our implementations, we found that the speed improvement from switching to a first-rest encoding and implementing more efficient queries was modest.

as we will see in Section 4.2, the last-rest encoding allows us to exploit the scrolling nature of queries issued by decoders, which results in speedups that far outweigh those achieved by reversing the trie.


In the previous section, we provided compact and efficient implementations of associative arrays that allow us to query a value for an arbitrary n-gram.

decoders do not issue language model requests at random.


In many games, it is sufficient to maintain a distinct action-value for each unique state and action in a large search tree.

when the branching factor is large it is usually beneficial to approximate the action-value function, so that the value of many related states and actions can be learned from a reasonably small number of simulations (Silver, 2009).


Given sentence y i and its dependency parse q i , we model the distribution over predicate labels e i as: Here e j is the predicate label of the j th word being labeled, and e 1:j−1 is the partial predicate labeling constructed so far for sentence y i .

in the second layer of the neural network, the units z represent a predicate labeling e i of every sentence y i ∈ d. our intention is to incorporate, into action-value function Q, information from only the most relevant sentence.