Since the number of parameters in NLM is still large, several smoothing methods are used (Chen and Goodman, 1998) to produce more accurate probabilities, and to assign nonzero probabilities to any word string.

since the probabilities in NLMs depend on the length of the sentence, two sentences of different length cannot be compared directly.


The DLM-PN can be trained by using any binary classification learning methods.

since the number of training examples is very large, batch training has suffered from prohibitively large computational cost in terms of time and memory.


For example, Ngram language model is considered to be effective in writing evaluation (Burstein et al., 1998;Corston-Oliver et al., 2001).

it becomes very expensive if N > 3 and N-grams only consider continuous sequence of words, which is unable to detect the above error "if...will...will".


Some example LSPs discovered from erroneous sentences are <a, NNS> (support:0.39%, confidence:85.71%), <to, VBD> (support:0.11%, confidence:84.21%), and <the, more, the, JJ> (support:0.19%, confidence:0.93%) 7 ; Similarly, we also give some example LSPs mined from correct sentences: <NN, VBZ> (support:2.29%, confidence:75.23%), and <have, VBN, since> (support:0.11%, confidence:85.71%) 8 .

other features are abstract and it is hard to derive some intuitive knowledge from the opaque statistical values of these features.


The performance of the statistical language models is often evaluated by perplexity or cross-entropy.

we decided to only report the real ASR performance, because perplexity does not suit well to the comparison of models that use different lexica, have different OOV rates and have lexical units of different lengths.


LERs are more comparable across some languages than WERs, as WER depends more on factors such as length, morphological complexity, and OOV of the words.

for within-language and between-model comparisons, the RWERR should still be a valid metric, and is also usable in languages that do not use a phonemic writing system.


Both WER and LER are high considering the task.

standard methods such as adaptation were not used, as the intention was only to study the RWERR of the different approaches.


In transliteration, we face similar issues as in SMT, such as lexical mapping and alignment.

transliteration is also different from general SMT in many ways.


Machine Translation (MT) quality has improved substantially in recent years due to applying data intensive statistical techniques.

state-ofthe-art approaches are essentially lexical, considering every surface word or phrase in both the source sentence and the corresponding translation as an independent entity.


This is just an approximation of the true lower bound, and bad estimates can lead to search errors.

the hope is that by choosing the right value of i, these estimates will be accurate enough to affect the search quality only slightly, which is analogous to "almost admissible" heuristics in A* search (Soricut, 2006).


Most of the previous work on statistical machine translation relies on (local) associations of target words/phrases with source words/phrases for lexical selection.

in this paper, we present a novel approach to lexical selection where the target words are associated with the entire source sentence (global) without the need to compute local associations.


Traditionally, Maxent is regularized by imposing a Gaussian prior on each weight: this L2 regularization finds the solution with the smallest possible weights.

on tasks like machine translation with a very large number of input features, a Laplacian L1 regularization that also attempts to maximize the number of zero weights is highly desirable.


Because of this synonym problem, the BOW threshold θ has to be set lower than 0.5, which is observed experimentally.

if we set the threshold to 0.3, both t 1 and t 2 will be detected in the target sentence, and we found this to be a major source of undesirable insertions.


Both constraints have been shown to be in very good fit with data from dependency treebanks (Kuhlmann and Nivre, 2006).

like all other such proposals, they are formulated on fully specified structures, which makes it hard to integrate them into a generative model, where dependency structures are composed from elementary units of lexicalized information.


Sandy gave gave dog the dog gave bone a bone This paper does not consider stochastic dependency grammars directly, but see Section 8 for an application involving them.

it is straightforward to associate weights with dependencies, and since the dependencies are preserved by the transformations, obtain a weighted CFG.


Almost linear CFLGs can represent a substantial fragment of a Montague semantics for English and such "linear" grammar formalisms as (multi-component) tree-adjoining grammars (both as string grammars and as tree grammars) and multiple context-free grammars.

iO macro grammars and parallel multiple context-free grammars cannot be directly represented because representing string copying requires multiple occurrences of a variable of type o → o.


The overall performance of our semantic role labeling approach is not competitive with leading contemporary systems, which typically employ support vector machine learning algorithms with syntactic features (Pradhan et al., 2005) or syntactic tree kernels (Moschitti et al., 2006).

our work highlights a number of characteristics of the semantic role labeling task that will be helpful in improving performance in future systems.


A strong syntactic/semantic correlation would suggest that further gains in the use of surrogate annotation data could be gained if syntactic similarity was computed between rolesets rather than their verbs.

this would first require accurate word-sense disambiguation both for the test sentences as well as for the parsed corpora used to calculate parse tree path frequencies.


These featurebased methods are considered as the state of the art methods for SRL.

as we know, the standard flat features are less effective in modeling the syntactic structured information.


In this point, one can say that the grammar-driven tree kernel is a specialization of the PT kernel.

the important difference between them is that the PT kernel is not grammar-driven, thus many nonlinguistically motivated structures are matched in the PT kernel.


The labeled order tree kernel is much more flexible than the PT kernel and can explore much larger sub-tree features than the PT kernel.

the same as the PT kernel, the labeled order tree kernel is not grammar-driven.


Their methods need to obtain a LTAG derivation tree for each parse tree before kernel calculation.

we use the notion of optional arguments to define our grammar-driven tree kernel and use the empirical set of CFG rules to determine which arguments are optional.


When presented with some previously unseen test data, we are forced to rely on its automatic parse trees.

for the best results we should take advantage of gold parse trees whenever possible, including those of the labeled training data.


(Caruana, 1997) discusses configurations where both used inputs and unused inputs (due to excessive noise) are utilized as additional outputs.

our work concerns linear predictors using empirical risk minimization.


The number of hard cases specific to the B-I classifier indicates how the features contribute to the decision of splitting or continuing back-to-back NPs.

back-to-back NPs amount to 6% of the NPs in HEb Gold and 8% of the NPs in EN G. while in English most of these cases are easily resolved, Hebrew phenomena such as null-equatives and free word order make them harder.


In some recent work (Strube and Ponzetto, 2006), it has been shown that related pairs can be generated without pre-specifying the nature of the relation sought.

this work does not focus on differentiating among different relations, so that the generated relations might conflate a number of distinct ones.


This allows the many parsers based on the Penn Treebank, for example, to be meaningfully compared.

there are two drawbacks to this approach.


The most common form of parser evaluation is to apply the Parseval metrics to phrase-structure parsers based on the Penn Treebank, and the highest reported scores are now over 90% (Bod, 2003;Charniak and Johnson, 2005).

it is unclear whether these high scores accurately reflect the performance of parsers in applications.


RASP uses an unlexicalised parsing model and has not been tuned to newspaper text.

it has had many years of development; thus it provides a strong baseline for this test set.


For the purposes of discussion, we will suppose that X = R F and that Y = {−1, +1}.

most of the techniques described in this section (as well as our own technique) are more general.


We could equally well use Φ s (x) = x, x and Φ t (x) = x, 0 .

it turns out that it is easier to analyze the first case, so we will stick with that.


Indeed, all the above methods do not make use of the unlabeled instances in the target domain.

our instance weighting framework allows unlabeled target instances to contribute to the model estimation.


In Section 4 we will develop a general framework for semi-supervised learning with constraints.

it is useful to illustrate the ideas on concrete problems.


This confirms results reported for the supervised learning case in (Punyakanok et al., 2005;.

as shown, our proposed algorithm H&W&C for training with constraints is critical when the amount labeled data is small.


For a formally organized event, such as the annual MT Evaluation sponsored by National Institute of Standard and Technology (NIST MT Eval), it may be worthwhile to recruit multiple human translators to translate a few hundred sentences for evaluation references.

there are situations in which multiple human references are not practically available (e.g., the source may be of a large quantity, and no human translation exists).


The results show that pseudo references are informative, as standard metrics were able to make use of the pseudo references and achieve higher correlations than judging from fluency alone.

higher correlations are achieved when learning with regression, suggesting that the trained metrics are better at interpreting comparisons against pseudo references.


Moreover, we can gain from performing another step.

the inclusion of the English-Chinese dictionary is harmful in this case, probably because 1-to-n alignments are less frequent for this direction, and have been captured during the first step.


This work was extended in (Rosti et al., 2007) by introducing system weights for word confidences.

the system weights did not influence the skeleton selection, so a hypothesis from a system with zero weight might have been chosen as the skeleton.


Also, the METEOR score using the METEOR optimized weights is very high.

the other scores are worse in common with the tuning set results.


They alone bring the performance in the MF up to 75%.

these two features explain only 56% of the cases in the VF.


Second, as demonstrated in the Redwood Lingo Treebank, reversibility makes it easy to rapidly create very large evaluation suites: it suffices to parse a set of sentences and select from the parser output the correct semantics.

nLG geared realisers either work on evaluation sets of restricted size (500 input for SURGE, 210 for KPML) or require the time expensive implementation of a preprocessor transforming e.g., Penn Treebank trees into a format suitable for the realisers.


, r n ) of the semantic requirement, and there is one universal quantifier for y and for each parameter x j of the action except for ref(id(r)).

a distractor a for a referring expression introduced at u is removed when we substitute or adjoin an elementary tree into u which rules a out.


One might consider using a metric based on language model probabilities for sentences: in eval-uating a language model on (already existing) test data, a higher probability for a sentence (and lower perplexity over a whole test corpus) indicates better language modelling; perhaps a higher probability might indicate a better sentence.

here we are looking at generated sentences, which have been generated using their own language model, rather than human-authored sentences already existing in a test corpus; and so it is not obvious what language model would be an objective assessment of sentence naturalness.


As in late fusion, modalityspecific classifiers are trained independently.

the Bayesian approach also learns to predict the reliability of each modality on a given instance, and incorporates this information into the Bayes net.


The gesture features described thus far capture the similarity between static gestures; that is, gestures in which the hand position is nearly constant.

these features do not capture the similarity between gesture trajectories, which may also be used to communicate meaning.


More specifically, assuming that the NM has a positive effect, the S users are asked to rate first the poorer version of the system (noNM) and then the better version (NM).

f users' task is easier as they already have a high reference point (NM) and it is easier for them to criticize the second problem (noNM).


The most salient difference is that here we investigate the benefits of displaying the discourse structure information for the users.

(Rich and Sidner, 1998) never test the utility of the SIH.


Studies have also shown that eye gaze has a potential to improve resolution of underspecified referring expressions in spoken dialog systems (Campana et al., 2001) and to disambiguate speech input (Tanaka, 1999).

to these earlier studies, our work focuses on a different goal of using eye gaze for automated vocabulary acquisition and interpretation.


Wittenburg et al experiment with unrestricted speech input for electronic program guide search, and use a highlighting mechanism to provide feedback to the user regarding the "relevant" terms the system understood and used to make the query.

their usability study results show this complex output can be confusing to users and does not correspond to user expectations.


The parser uses a representation for syntactic structure similar to dependency links which is well-suited for incremental parsing.

to previous unsupervised parsers, the parser does not use part-of-speech tags and both learning and parsing are local and fast, requiring no explicit clustering or global optimization.


A shortcoming of the DP-based approaches is that they are unable to generate nonprojective structures.

non-projectivity is necessary to capture syntactic phenomena in many languages.


The ordinary f-score is computed that way mostly in order to overcome the fact that sentences differ in length.

for applications such as IE and QA, which work at the single sentence level and which might reach erroneous decision due to an inaccurate parse, normalizing over sentence lengths is less of a factor.


Rather, as indicated by the 74% accuracy, they also consider the reputation of the merchants.

the real value of the postings relies on the text and not on the numeric ratings: the accuracy is 87%-89% when using the textual reputation variables.


The reason seems to be that the generation of a ranking by negativity seems a somehow harder task than the generation of a ranking by positivity; this is also shown by the results obtained with the uniform-valued vector e1, in which the application of PageRank improves with respect to e1 for positivity but deteriorates for negativity.

against the baseline constituted by the results obtained with the uniformvalued vector e1 for negativity, our rankings show a relevant improvement, ranging from −8.56% (e2) to −48.27% (e4).


This excerpt expresses an overall negative opinion of the product being reviewed.

not all parts of the review are negative.


The local dependencies between sentiment labels on sentences is similar to the work of Pang and Lee (2004) where soft local consistency constraints were created between every sentence in a document and inference was solved using a min-cut algorithm.

jointly modeling the document label and allowing for non-binary labels complicates min-cut style solutions as inference becomes intractable.


That is, a system could start by classifying documents, use the document information to classify sentences, use the sentence information to classify documents, and repeat until convergence.

experiments showed that this did not improve accuracy over a single iteration and often hurt performance.


Automatic sentiment classification has been extensively studied and applied in recent years.

sentiment is expressed differently in different domains, and annotating corpora for every possible domain of interest is impractical.


We believe the most important reason for this is that they explore structured prediction problems, where labels of surrounding words from the source classifier may be very informative, even if the current label is not.

our simple binary prediction problem does not exhibit such behavior.


The extraction of question-answer pairs amounted to a database of 1 million pairs in their experiment.

inspection of the publicly available Web-FAQ collection provided by Jijkoun and de Rijke 2 showed a great amount of noise in the retrieved FAQ pages and question-answer pairs, and yet the indexed question-answer pairs showed a serious recall problem in that no answer could be retrieved for many well-formed queries.


In our experiments, we employed equation 3to infer for each para-phrase pair translation model probabilities p φ (syn|trg) and p φ (trg|syn) from relative frequencies of phrases in bilingual tables.

to Bannard and Callison-Burch (2005), we applied the same inference step to infer also lexical translation probabilities p w (syn|trg) and p w (trg|syn) as defined in Koehn et al.


In order to maximize recall for the comparative evaluation of systems, we selected 60 queries that were well-formed natural language questions without metacharacters and spelling errors.

for one third of these well-formed queries none of the five compared systems could retrieve an answer.


This evaluation measure accounts for improvements in coverage, i.e., it rewards cases where answers are found for queries that did not have an adequate or material answer before.

the mean reciprocal rank (MRR) measure standardly used in QA can have the effect of preferring systems that find answers only for a small set of queries, but rank them higher than systems with  Table 4: Examples for queries and expansion terms yielding improved (+), decreased (-), or unchanged (0) retrieval performance compared to retrieval without expansion.


Such is the case with Peter's short speech in the second half of Luke 9:33 (see Table 1).

unimportant details may be deleted, and new information weaved in from other sources or oral traditions.


Second, we place no hard restrictions on the reordering of the source text, opting instead for a soft preference for maintaining the source order through the Order feature.

deviation from the source order is limited to "flips" between two sentences in (Barzilay and Elhadad, 2003), an assumption that is not valid in the Synoptics 6 .


To a certain extent, these lexical networks enable topic segmenters to exploit a sort of concept reiteration.

their lack of any explicit topical structure makes this kind of knowledge difficult to use when lexical ambiguity is high.


Other work suggests a clear utility for generating language manifesting personality (Reeves and Nass, 1996).

to date, (1) research in generation has not systematically exploited the psycholinguistic findings; and (2) there has been little evaluation showing that automatic generators can produce language with recognizable personality variation.


So, we must first map the psychological findings to parameters of a natural language generator (NLG).

this presents several challenges: (1) The findings result from studies of genres of language, such as stream-ofconsciousness essays (Pennebaker and King, 1999), and informal conversations (Mehl et al., 2006), and thus may not apply to fixed content domains used in NLG; (2) Most findings are based on self-reports of personality, but we want to affect observer's perceptions; (3) The findings consist of weak but significant correlations, so that individual parameters may not have a strong enough effect to produce recognizable variation within a single utterance; (4) There are many possible mappings of the findings to generation parameters; and (5) It is unclear whether only specific speech-act types manifest personality or whether all utterances do.


As expected, the best segmentation results are obtained using manual transcripts.

the gap between audio-based segmentation and transcript-based segmentation narrows when the recognition accuracy decreases.


Basically, the BiTAM model consists of topic-dependent translation lexicons modeling P r(c|e, k) where c, e and k denotes the source Chinese word, target English word and the topic index respectively.

the bLSA framework models P r(c|k) and P r(e|k) which is different from the BiTAM model.


Another observation is that the CH→EN bLSA model seems to give better performance than the EN→CH bLSA model.


Recently, there has been a surge of interest in improving coreference resolution by jointly modeling coreference with a related task such as MD (e.g., Daumé and Marcu (2005)).

joint models typically need to be trained on data that is simultaneously annotated with information required by all of the underlying models.


, p, where S i is the i th tree of text segments, and T i is the table-of-contents for that tree.

we cannot directly use these tablesof-contents for training our global model: since this model selects one of the candidate titles z i 1 , .


Admittedly, this method of generating training and testing data omits some dependencies at the level of the table-of-contents as a whole.

the subtrees used in our experiments still exhibit a sufficiently deep hierarchical structure, rich with contextual dependencies.


If we do not cluster the words according to their part-of-speech, we also lose some performance, obtaining 78.6% at best.

clustering all words (such as CC, DT, IN part-of-speech tags) also gives weaker results (81.1% accuracy at best).


Recent results have shown that symbolic noun compound interpretation systems using machine learning techniques coupled with a large lexical hierarchy perform with very good accuracy, but they are most of the time tailored to a specific domain (Rosario and Hearst, 2001).

the majority of corpus statistics approaches to noun compound interpretation collect statistics on the occurrence frequency of the noun constituents and use them in a probabilistic model (Lauer, 1995).


(Kim and Baldwin, 2006) and (Turney, 2006) focus on the lexical similarity of unseen noun compounds with those found in training.

although the web-based solution might overcome the data sparseness problem, the current probabilistic models are limited by the lack of deep linguistic information.


As formulated above, the learning task can be seen as an instance of multiple instance learning.

there are important properties that set it apart from problems previously considered in MIL.


present a systematic investigation of the pattern representation models and point out that substructures of the linguistic representation and the access to the embedded structures are important for obtaining a good coverage of the pattern acquisition.

all considered representation models (subject-verbobject, chain model, linked chain model and subtree model) are verb-centered.


With one randomly selected seed, we could finally extract most relevant events in some covered time interval.

it turns out that it is not just the average number of reports per events that matters but also the distribution of reportings to events.


(2002) reported that discourse structure helps to extract anaphoric relations.

their set of grammatical rules is heuristic.


If the system works without a NER component, it only knows that "Oracle" and "Peo-pleSoft" are proper noun phrases, and its confidence in correctness of a candidate relation instance Acquisition(Oracle, PeopleSoft) cannot be very high.

both entities occur many times elsewhere in the corpus, sometimes in strongly discriminating contexts, such as "Oracle is a company that…" or "PeopleSoft Inc." If the system somehow learned that such contexts indicate entities of the correct type for the Acquisition relation (i.e., companies), then the system would be able to boost its confidence in both entities ("Oracle" and "PeopleSoft") being of correct types and, consequently, in (Oracle, PeopleSoft) being a correct instance of the Acquisition relation.


Therefore, in such case, the simplest recourse is to simply label the entity as Invalid, and not to try fixing the boundaries.

if a word was missed from an entity (e.g., "Beverly O", instead of "Beverly O ' Neill"), the resulting sequence will be frequent.


The Corpusbased validator, however, works purely on the basis of context, entirely disregarding the internal structure of entities, and thus performs worst of all in this case.

the Corpus-based validator is able to improve the results for the Inventor relation, which the other two validators are completely unable to do.


Since the technical root comes before the sentence itself, no new non-projective edges are introduced.

edges from technical roots may introduce non-planarity.


They confirm the findings of Kuhlmann and Nivre (2006): planarity seems to be almost as restrictive as projectivity; well-nestedness, on the other hand, covers large proportions of trees in all languages.

to global constraints, properties of individual non-projective edges allow us to pinpoint the causes of non-projectivity.


When applying self-training to a parser trained with a small dataset we expect the coverage of the parser to increase, since the combined training set should contain items that the seed dataset does not.

since the accuracy of annotation of such a parser is poor (see the no self-training curve in Figure 1) the combined training set surely includes inaccurate labels that might harm parser performance.


This is largely because HPSG is based on a lexicalized grammar formalism, and as such its syntactic structures have an underlying dependency backbone.

hPSG syntactic structures includes long-distance dependencies, and the underlying dependency structure described by and hPSG structure is a directed acyclic graph, not a dependency tree (as used by mainstream approaches to data-driven dependency parsing).


The dynamic SBNs which we peopose, called Incremental Sigmoid Belief Networks (ISBNs) have large numbers of latent variables, which makes exact inference intractable.

they can be approximated sufficiently well to build fast and accurate statistical parsers which induce features during training.


Sigmoid Belief Networks were used originally for character recognition tasks, but later a dynamic modification of this model was applied to the reinforcement learning task (Sallans, 2002).

their graphical model, approximation method, and learning method differ significantly from those of this paper.


The assumption in this step is that transliteration of each vowel sequence of the source is a vowel sequence in the target language, and similarly for consonants.

consonants do not always map to consonants, or vowels to vowels (for example, the English letter "s" may be written as " " in Persian which consists of one vowel and one consonant).


The process of aligning words explained above can handle words with already known components in the alignment set A (the frequency of occurrence is greater than zero).

when this is not the case, the system may repeatedly insert ε while part or all of the target characters are left intact (unsuccessful alignment).


Thus in (NP the Manhattan phone book and tour guide) 6 a flat structure is given because although the is a non-nominal modifier, it is shared, modifying both tour guide and phone book, and all other modifiers in the phrase are nominal.

we found that out of 1,417 examples of NP coordination in sections 02 to 21, involving phrases containing only nouns (common nouns or a mixture of common and proper nouns) and the coordinating conjunction, as many as 21.3%, contrary to the guidelines, were given internal structure, instead of a flat annotation.


For example, there were 3,314 cases in which sentence boundary detection needs to use the results of extra line break detection, extra punctuation mark detection, and case restoration.

in the cascaded method, sentence boundary detection is conducted after extra punctuation mark detection and before case restoration, and thus it cannot leverage the results of case restoration.


In essence, such methods utilize extraction patterns to generate candidate extractions (e.g., "Istanbul") and then assess each candidate by computing co-occurrence statistics between the extraction and words or phrases indicative of class membership (e.g., "cities such as").

zipf's Law governs the distribution of extractions.


Chiang (2005) shows significant improvement by keeping the strengths of phrases while incorporating syntax into statistical translation.

the performance of linguistically syntax-based models can be hindered by making use of only syntactic phrase pairs.


These multiword phrasal units contribute to fluency by inherently capturing intra-phrase reordering.

despite this progress, interphrase reordering (especially long distance ones) still poses a great challenge to statistical machine translation (SMT).


Other further generalizations of orientation include the global prediction model (Nagata et al., 2006) and distortion model (Al-Onaizan and Papineni, 2006).

these models are often fully lexicalized and sensitive to individual phrases.


In our own exper-iment setting, the best distortion limit for Chinese-English translation is 4.

some ideal translations exhibit reorderings longer than such distortion limit.


For example, for the last two Chinese phrases in figure 1(a), simply swapping the two children of the NP node will produce the correct word order on the English side.

there are also reorderings which do not agree with syntactic analysis.


Consequently, P (t|w,θ) favors t = 1 for any sequence that does not contain exactly five heads, and assigns equal probability to t = 1 and t = 0 for any sequence that does contain exactly five heads -a counterintuitive result.

using some standard results in Bayesian analysis we can show that applying Equation 3 yields which is significantly less than .5 when n H = 5, and only favors t = 1 for sequences where n H ≥ 8 or n H ≤ 2.


Consequently, it makes sense to use a Dirichlet prior with β < 1.


For POS tagging, estimates based on multiple samples might be useful if we were interested in, for example, the probability that two words have the same tag.

computing such probabilities across all pairs of words does not necessarily lead to a consistent clustering, and the result would be difficult to evaluate.


(2003) improved the CRF method by employing the large margin method to separate the gold standard sequence la-beling from incorrect labellings.

the complexity of quadratic programming for the large margin approach prevented it from being used in large scale NLP tasks.


This simple example has shown the advantage of adopting a flexible search strategy.

it is still unclear how we maintain the hypotheses, how we keep candidates and accepted labels and spans, and how we employ dynamic programming.


This can also explain why the performance of leftto-right search with non-aggressive learning is close to bidirectional search if the beam is large enough.

with beam width = 1, non-aggressive learning over left-to-right search performs much worse, because in this case it is more likely that the gold-standard tag is not in the beam.


It is reported in (Toutanova et al., 2003) that a crude company name detector was used to generate features, and it gave rise to significant improvement in performance.

it is difficult for us to duplicate exactly the same feature for the purpose of comparison, although it is convenient to use features like that in our framework.


This would ordinarily call for a more conservative approach to avoid changes that might have unintended consequences.

evaluation methodologies must evolve to reflect the shifting interests of the research community to remain relevant.


for question classification (Zhang and Lee, 2003), but these, too are inadequate when dealing with definitional answers expressed by long and articulated sentences or even paragraphs.

shallow semantic representations, bearing a more "compact" information, could prevent the sparseness of deep structural approaches and the weakness of BOW models.


This reduces data sparseness with respect to a typical BOW representation.

sentences rarely contain a single predicate; it happens more generally that propositions contain one or more subordinate clauses.


the SSTK applied to one PAS extracted from a text fragment may not be meaningful since its representation needs to take into account all the PASs that it contains.


Results of a perception study show that contextual cues are stronger predictors of discourse function than acoustic cues.

acoustic features capturing the pitch excursion at the right edge of okay feature prominently in disambiguation, whether other contextual cues are present or not.


In our previous study (Reitter et al., 2006b), we found more syntactic priming in the task-oriented dialogues of the Map Task corpus than in the spontaneous conversation collected in the Switchboard corpus.

we compared priming effects across two datasets, where participants and conversation topics differed greatly.


From the whole interaction (very rarely more than five turns), 87% accuracy can be achieved (36% of dialogues had been hand-labeled "problematic").

the most predictive features, which related to automatic speech recognition errors, are neither available in the human-human dialogue we are concerned with, nor are they likely to be the cause of communication problems there.


Apart from subject-related factors, communicative strategies will play a role.

linguistic repetition serves as a good predictor of how well interlocutors will complete their joint task.


Of course, this correlation does not necessarily indicate a causal relationship.

participants in Map Task did not receive an explicit indication about whether they were on the "right track".


Their experiments employed up to 20 annotators, and they allowed for the explicit annotation of ambiguity.

our annotators were instructed to choose the single most plausible interpretation in case of perceived ambiguity.


Such a data set certainly contains some spurious or dubious links, while lacking some correct but more difficult ones.

we argue that it constitutes a plausible subset of anaphoric links that are useful to resolve.


Note that the five-character window surrounding " " is the same in both cases, making the tagging decision for that character difficult given the local window.

the correct decision can be made by comparison of the two three-word windows containing this character.


Collins (2002) proposed the perceptron as an alternative to the CRF method for HMM-style taggers.

our model does not map the segmentation problem to a tag sequence learning problem, but defines features on segmented sentences directly.


Rather than a finite β with a symmetric Dirichlet distribution, in which draws tend to have balanced clusters, we now have an infinite β.

most draws will have weights which decay exponentially quickly in the prior (though not necessarily in the posterior).


As can be seen in figure 2(b), this model does correct the systematic problem of pronouns being considered their own entities.

it still does not have a preference for associating pronominal references to entities which are in any way local.


Since we do not have labeled cross-document coreference data, we cannot evaluate our system's crossdocument performance quantitatively.

in addition to observing the within-document gains from sharing shown in section 3, we can manually inspect the most frequently occurring entities in our corpora.


With parallel corpora, the reason that research has been limited to only a few languages at a time -and usually just two at a time, as in the LSI work cited above -is more likely to be rooted in the widespread perception that good parallel corpora are difficult to obtain (see for example Asker 2004).


One advantage of a 'massively parallel' multilingual corpus is perhaps self-evident: within the LSI framework, the more languages are mapped into the single conceptual space, the fewer restrictions there are on which languages documents can be selected from for cross-language retrieval.

several questions were raised for us as we contemplated the use of a massively parallel corpus.


For some versions, a few of the verse translations are incomplete where a particular verse has been skipped in translation; this also explains the fact that the number of Hebrew and Greek text chunks together do not add up to 31,226.

the number of such verses is negligible in comparison to the total.


Moreover, these are the three languages of the five which are written in the Roman alphabet.

we believe the explanation for the poorer results for language pairs involving either Arabic, Russian, or both, can be pinned down to something more specific.


Recent automatic evaluation metrics typically frame the evaluation problem as a comparison task: how similar is the machine-produced output to a set of human-produced reference translations for the same source text?

as the notion of similarity is itself underspecified, several different families of metrics have been developed.


(Claveau et al., 2003) for example use Inductive Logic Programming to learn if a given verb is a qualia element or not.

their approach does no go as far as learning the complete qualia structure for a lexical element as in our approach.


Changes in the arguments or tense of the verb sometimes change the basic situation types: If SE type could be determined solely by the verb constellation, automatic classification of SEs would be a relatively straightforward task.

other parts of the clause often override the basic situation type, resulting in aspectual coercion and a derived situation type.


Evert and Baroni (2006) attempt, for the first time, to evaluate LNRE models on unseen data.

rather than splitting the data into separate training and test sets, they evaluate the models in an extrapolation setting, where the parameters of the model are estimated on a subset of the data used for testing.


The original RePortS algorithm assumes morphology to be concatenative, and specializes on prefixation and suffixation, like most of the above approaches, which were developed and implemented for English (Goldsmith, 2001;Schone and Jurafsky, 2000;Neuvel and Fulop, 2002;Yarowski and Wicentowski, 2000;Gaussier, 1999).

many languages are morphologically more complex.


The next best system obtained an F-score of 69%.

the algorithm does not perform as well on other languages (Turkish, Finnish, German) due to low recall (see (Keshava and Pitler, 2006) and (Demberg, 2006), p. 47).


In his account, competition between mid-vowel diphthongization (e.g., s[e]ntir 'to feel', s [je]nto 'I feel') and non-diphthongization (e.g., p[e]dir 'to ask ', p[i]do 'I ask') leads to paradigmatic gaps in lexemes for which the applicability of diphthongization has low reliability (e.g., abolir 'to abolish, *ab [we]lo, *ab [o]lo 'I abolish').

this approach both overpredicts and underpredicts the existence of gaps crosslinguistically.


First, it predicts that gaps should occur whenever the analogical forces determining word forms are contradictory and evenly weighted.

variation between two inflectional patterns seems to more commonly result from such a scenario.


With no analogical pressure, gaps are robustly attested (τ = 6.32).

the new gaps are not restricted to the 1sg, and under this scenario, learners are unable to generalize to a novel pairing of lexeme + IPS.


When there is weak analogical pressure, weighting for morphophonological similarity has little effect on the persistence and spread of gaps.

when there is relatively strong analogical pressure, morphophonological similarity helps atypical frequency distributions to persist, as shown in Figure 1.


The major advantage the letter-based transducer presented in Section 3 has over the Viterbi substring decoder is its word unigram language model, which allows it to reproduce words seen in the training data with high accuracy.

the Viterbi substring decoder is able to encode contextual information in the transliteration model because of its ability to consider larger many-to-many mappings.


It may seem surprising that later stages of a pipeline, already constrained to be consistent with the output of earlier stages, can profitably inform the earlier stages in a second pass.

the richer models used in later stages of a pipeline provide a better distribution over the subset of possible solutions produced by the early stages, effectively resolving some of the ambiguities that account for much of the original variation.


The oracle rate decreases under all of the constrained conditions as compared to the baseline, demonstrating that the parser was prevented from finding some of the best solutions that were originally found.

the improvement in Fscore shows that the constraints assisted the parser in achieving high-quality solutions despite this degraded oracle accuracy of the lists.


First, although the English subjectivity lexicon contains inflected words, we must use the lemmatized form in order to be able to translate the entries using the bilingual dictionary.

words may lose their subjective meaning once lemmatized.


Moreover, the lexicon sometimes includes identical entries expressed through different parts of speech, e.g., grudge has two separate entries, for its noun and verb roles, respectively.

the bilingual dictionary does not make this distinction, and therefore we have again to rely on the "most frequent" heuristic captured by the translation order in the bilingual dictionary.


In this study we develop a learning model based around the concept of iteratively predicting labels for unlabelled training samples, the basic paradigm for both co-training and self-training.

we generalise by framing the task in terms of the acquisition of labelled training data, from which a supervised classifier can subsequently be learned.


We expect the recognition and disambiguation of faces to improve if many image-text pairs that treat the same person are used.

our approach is also valuable when there are few image-text pairs that picture a certain person or object.


In addition, head and hand/forearm movements are used to detect group-action based segments (Mc-Cowan et al., 2005;Al-Hames et al., 2005).

many other features that we expect to signal segment boundaries have not been studied systematically.


In addition, many of the non-lexical feature classes, including those that have been identified as indicative of segment boundaries in previous work (e.g., prosody) and those that we hypothesized as good predictors of segment boundaries (e.g., motion, context), are not beneficial for recognizing boundaries when used in isolation.

these non-lexical features are useful when combined with lexical features, as the presence of the non-lexical features can balance the tendency of models trained with lexical cues alone to overpredict.


If the co-reference system works perfectly, the system should find a social network involving four people: {John, Tim, Mary, Mark}, and the ties: moth-erOf (Mary, John), and brotherOf (Mark, Tim).

if the co-reference system mistakenly links "John" to "his" in the second clause and links "Tim" to "his" in the first clause, then we will still have a network with four people, but the ties will be: motherOf (Mary, Tim), and brotherOf (Mark, John), which are completely wrong.


Indeed, annotation experiments using very fine-grained categories show low annotation reliability (Müller, 2006).

there is no debate over the importance nor the definition of distinguishing pronouns that refer to nouns from those that do not.


As expected, the precision of the extracted attributes as an average over all classes is best when the input instance sets are hand-picked (M ), as opposed to automatically extracted (E).

the loss of precision from M to E is small at all measured ranks.


Compared to traditional IE, the recall of our Open IE system is admittedly lower.

in a targeted extraction scenario, Open IE can still be used to reduce the number of hand-labeled examples.


Quirk and Corston-Oliver (2006) investigated the impact of parsing accuracy on statistical MT.

this work was only concerned with a single dependency parser, and did not focus on parsers based on different frameworks.


Also, this framework allows for defining arbitrary similarity functions between two matching items, and we could match arbitrary concepts (such as dependency relations) gathered from a sentence pair.

most other metrics (notably BLEU) limit themselves to matching based only on the surface form of words.


The 101 correct NO responses represent 12% of the 864 possible correct NO responses.

the systems responded correctly for 50% (2449/4920) of the cases when YES was the reference answer and for 61% (2345/3816) of the cases when UNKNOWN was the reference answer.


As the figure 1 shows, when we increase the threshold by allowing more candidate phrase pair hypothesized as valid translation, we observe the phrase table size increases monotonically.

we notice that the translation performance improves gradually.


Removing PP1 from the baseline phrase table (comparing the first group of scores) or adding PP1 to the new phrase table (the second group of scores) overall results in no or marginal performance change.

adding phrase pairs extracted by the new method only (PP3) can lead to significant BLEU score increases (comparing row 1 vs. 3, and row 2 vs. 4).


Larger window size can lead to better results as shown in Table 3 and Table 4 since more contextual knowledge is used to model measure word generation.

enlarging the window size does not bring significant improvements, The major reason is that even a small win-dow size is already able to cover most of measure word collocations, as indicated by the position distribution of head words in Table 1.


We have shown that VB is both practical and effective for use in MT models.

our best system does not apply VB to a single probability model, as we found an appreciable benefit from bootstrapping each model from simpler models, much as the IBM word alignment models are usually trained in succession.


Most statistical parsers achieve a high robustness with respect to out-of-grammar sentences by allowing for arbitrary derivations and rule expansions.

they are not suited to reliably decide on the grammaticality of a given phrase, as they do not accurately model the linguistic constraints inherent in natural language.


their grammar-based language model did not make use of a probabilistic component, and it was applied to a rather simple recognition task (dictation texts for pupils read and recorded under good acoustic conditions, no out-of-vocabulary words).


In particular it always has much lower insertion rates reflecting its supe-rior ability to remove utterances that are not typically part of the report.

the probabilistic model suffers from a slightly higher deletion rate due to being overzealous in this regard.


modelling only partial words helps the segmenter handle long, infrequent words.


Datasets around 30K words are traditional for this task.

a child learner has access to much more data, e.g.


Intuitively, compared with co-occurrence-based thesauri, hand-crafted thesauri, such as WordNet, could provide more reliable terms for query expansion.

previous studies failed to show any significant gain in retrieval performance when queries are expanded with terms selected from WordNet (Voorhees, 1994;Stairmand, 1997).


Another drawback of stemming is that it usually enhances recall, but may hurt precision (Kraaij and Pohlmann, 1996).

general Web search is basically a precision-oriented task.


2) Only the head word in the noun phrase varies in form and needs to be expanded.


If the expansion terms used are those that are variant forms of a word, then query expansion can produce the same effect as word stemming.

if we add all possible word alterations, query expansion/reformulation will run the risk of adding many unrelated terms to the original query, which may result in both heavy traffic and topic drift.


In this paper, one of the proposed methods will also use a bigram language model of the query to determine the appropriate alteration candidates.

in our approach, alterations are not limited to head words.


For simplicity, we use linear regression model here.

we denote an instance in the feature space as X, and the weights of features are denoted as w. Then the linear regression model is defined as: (3) where w T is the transpose of w. we will have a technical problem if we set the target value to the performance change directly: The range of , while the range of performance change is [-1,1].


When the query is very long and the first feature always obtains a value of log(0.5), so it does not have any discriminative ability.

the second feature helps because it can capture some co-occurrence information no matter how long the query is.


For example, in Table 1, Q2 preserves certain useful information of Q1 in the aspects of both question topic (Berlin) and question focus (fun club) although it loses some useful information in question topic (Hamburg).

questions Q3-Q5 are not related to Q1 in question focus (although being related in question topic, e.g.


Our approach follows Langkilde-Geary (2002) and Callaway (2003) in aiming to leverage the Penn Treebank to develop a broad-coverage surface realizer for English.

while these earlier, generation-only approaches made use of converters for transforming the outputs of Treebank parsers to inputs for realization, our approach instead employs a shared bidirectional grammar, so that the input to realization is guaranteed to be the same logical form constructed by the parser.


Among syntax-based translation models, the tree-based approach, which takes as input a parse tree of the source sentence, is a promising direction being faster and simpler than its string-based counterpart.

current tree-based systems suffer from a major drawback: they only use the 1-best parse to direct the translation, which potentially introduces translation mistakes due to parsing errors.


Compared with their string-based counterparts, treebased systems offer some attractive features: they are much faster in decoding (linear time vs. cubic time, see ), do not require a binary-branching grammar as in string-based models (Zhang et al., 2006), and can have separate grammars for parsing and translation, say, a context-free grammar for the former and a tree substitution grammar for the latter .

despite these advantages, current tree-based systems suffer from a major drawback: they only use the 1best parse tree to direct the translation, which potentially introduces translation mistakes due to parsing errors (Quirk and Corston-Oliver, 2006).


Ideally, a model would account for this ambiguity by marginalising out the derivations, thus predicting the best translation rather than the best derivation.


D: However, it is implemented in the Member States is still too slow.

t: the implementation measures in Member States remains too slow.


We need to do insideoutside parsing as coarse-to-fine parsers do.

we use the outside probability or cost information differently.


This figure indicates the advantage of the two-pass decoding strategy in producing translations with a high model score in less time.

model scores do not directly translate into BLEU scores.


Simply by exploring more (200 times the log beam) after-goal items, we can optimize the Viterbi synchronous parse significantly, shown in Figure 3(left) in terms of model score versus search time.

the mismatch between model score and BLEU score persists.


We show that the "dominance charts" proposed by Koller and Thater (2005b) can be naturally seen as regular tree grammars; using their algorithm, classical underspecified descriptions (dominance graphs) can be translated into RTGs that describe the same sets of readings.

rTGs are trivially expressively complete because every finite tree language is also regular.


The problem of computing the best tree is NPcomplete (Sima'an, 1996).

if the weighted regular tree automaton corresponding to the wRTG is deterministic, every tree has only one derivation, and thus computing best trees becomes easy again.


To improve results, some systems utilize additional manually constructed semantic resources such as WordNet (WN) (Beamer et al., 2007).

in many domains and languages such resources are not available.


Many other works manually develop a set of heuristic features devised with some specific relationship in mind, like a WordNet-based meronymy feature (Bedmar et al., 2007) or size-of feature (Aramaki et al., 2006).

the most prominent feature type is based on lexico-syntactic patterns in which the related words co-appear.


Either there is a failure to distinguish between these two structures, because the network fails to keep track of the fact that John is subject in one and object in the other, or there is a failure to recognize that both structures involve the same participants, because John as a subject has a distinct representation from John as an object.

symbolic representations can naturally handle the binding of constituents to their roles, in a systematic manner that avoids both these problems.


Vector addition does not increase the dimensionality of the resulting vector.

since it is order independent, it cannot capture meaning differences that are modulated by differences in syntactic structure.


Another simplification concerns K which can be ignored so as to explore what can be achieved in the absence of additional knowledge.

this reduces the class of models to: this still leaves the particular form of the function f unspecified.


This reduces the class of models to: However, this still leaves the particular form of the function f unspecified.

now, if we assume that p lies in the same space as u and v, avoiding the issues of dimensionality associated with tensor products, and that f is a linear function, for simplicity, of the cartesian product of u and v, then we generate a class of additive models: where A and B are matrices which determine the contributions made by u and v to the product p. if we assume that f is a linear function of the tensor product of u and v, then we obtain multiplicative models: where C is a tensor of rank 3, which projects the tensor product of u and v onto the space of p. Further constraints can be introduced to reduce the free parameters in these models.


The combined model is best overall with ρ = 0.19.

the difference between the two models is not statistically significant.


Only the leaf nodes of the prior's feature tree are considered, and, if no match can be found between the tuning and prior's training datasets' features, a N (0, 1) prior is used instead.

in the new approximate hierarchical model, even if a certain feature in the tuning dataset does not have an analog in the training dataset, we can always back-off until an appropriate match is found, even to the level of the root.


These methods extract lists of phrases, which are analogous to the keyphrases we use as input to our algorithm.

our approach is distinguished in two ways: first, we are able to predict keyphrases beyond those that appear verbatim in the text.


This is a reasonable assumption in the Corel dataset, where the annotations have similar lengths and the words reflect the salience of objects in the image (the multinomial model tends to favor words that appear multiple times in the annotation).

in our dataset the annotations have varying lengths, and do not necessarily reflect object salience.


To give a concrete example, let us assume that for a given image our model has produced five annotations, w 1 , w 2 , w 3 , w 4 , and w 5 .

according to the LDA model neither w 2 nor w 5 are likely topic indicators.


Naturally, parse errors result in (slightly) mislocated scopes but we had the general impression that state-of-the-art parsers could be used efficiently for this issue.

this approach requires a human expert to define the scope for each keyword separately using the predicate-argument relations, or to determine keywords that act similarly and their scope can be located with the same rules.


Space constraints do not allow us to present either the derivation or a detailed description of the sampling algorithm.

note that the conditional distribution used in sampling decomposes into two parts: where v', r' and z' are vectors of assignments of sliding windows, context (global or local) and topics for all the words in the collection except for the considered word at position i in document d; y is the vector of sentiment ratings.


finer-grained semantics are needed for the objects than subjects of verbs.

the parsing strategy is very simple, as we just substitute words by their semantic class and then train statistical parsers on the transformed input.


While it could be handled with extra categories, such as (s/(vp/np))/(s/np) for what, this is exactly the sort of strong-arm tactic that inclusion of the standard B, T, and S rules is meant to avoid.

the standard CCG analysis for English auxiliary verbs is the type exemplified in (16) (Steedman, 2000, 68), interpreted as a unary operator over sentence meanings (Gamut, 1991;Kratzer, 1991): this type is empirically underdetermined, given a widely-noted set of generalizations suggesting that auxiliaries and raising verbs take no subject argument at all (Jacobson, 1990, a.o.).


Thus, there is neither syntactic nor semantic evidence that hacer takes an object argument.

on this basis, we assign hacer the category (23): (23) hacer (s\np)/s : λP λx.cause P x Spanish has examples of cross-conjunct extraction in which hacer hosts clitics: This shows another instance of the schema in (1), which is undefined for any of the combinators in 3  The preceding data motivates adding D rules (we return to the distribution of the modalities below): To illustrate with example (10), one application of >D allows you and can to combine when the auxiliary is given the principled type assignment s/s, and another combines what with the result.


The flat structure described by the Penn Treebank can be seen in this example: (NP (NN lung) (NN cancer) (NNS deaths)) CCGbank (Hockenmaier and Steedman, 2007) is the primary English corpus for Combinatory Categorial Grammar (CCG) (Steedman, 2000) and was created by a semi-automatic conversion from the Penn Treebank.

ccG is a binary branching grammar, and as such, cannot leave NP structure underspecified.


Because this AND-OR tree represents only two different parses, the original parse and the depassivized version, only one OR node in the tree has more than one child -the root node, which has two choices, one for each parse.

the AND nodes immediately above "I" and "chance" each have more than one OR-node parent, since they are shared by the original and depassivized parses 1 .


Lexical chain is only based on similarities between lexical items in contiguous sentences.

in our approach, the linkage is based on the existing conversation structure.


Figure 1(b) shows the fragment quotation graph of the conversation shown in Figure 1(a) with all the redundant edges removed.

if threading is done at the coarse granularity of entire emails, as adopted in many studies, the threading would be a simple chain from E 6 to E 5 , E 5 to E 4 and so on.


With a strict notion of equivalence, there are no comparable rules.

the class S → PP NP ADVP VP, with 198 members, is highly similar, indicating more confidence in this correct rule.


On the surface, our model may seem as a special case of Cohen and Smith in which α = 0.

there is a crucial difference: the morphological probabilities in their model come from discriminative models based on linear context.


In the reference transcriptions and alignments used for scoring ASR systems, contractions are treated as two separate words.

aside from speech rate, our prosodic features were collected using word-by-word timestamps from a forced alignment that used a transcription where contractions are treated as single words.


Based on the original annotations, all human name translations were much better than our SMT system.

based on our re-annotation, the results are quite different: our system has a higher NEWA score and better name translations than 3 out of 4 human annotators.


There is no principled reason why this could not be done, i.e., why one could not design an HDP framework that simultaneously learns both the fragments (as in an adaptor grammar) and the states (as in an iHMM or iPCFG).

inference with these more complex models will probably itself become more complex.


In the context of tagging, there are several studies that utilized word clusters to prevent the data sparseness problem (Kazama et al., 2001;Miller et al., 2004).

these methods cannot produce the MN clusters required for constructing gazetteers.


(1999) and Torisawa (2001) showed that the EMbased clustering using verb-MN dependencies can produce semantically clean MN clusters.

the clustering algorithms, especially the EM-based algorithms, are computationally expensive.


This data splitting enables linear scalability of memory sizes.

doing so complicates the update procedure and, in terms of execution speed, may Figure 1: Parallelized inner-most routine of EM clustering algorithm.


They extracted a gazetteer for each NE category and utilized it in a NE tagger.

kazama and Torisawa (2007) extracted hyponymy relations, which are independent of the NE categories, from Wikipedia and utilized it as a gazetteer.


Wikipedia also produced a large gazetteer of more than 550,000 entries.

comparing these gazetteers and ours precisely is difficult at this point because the detailed information such as the precision and the recall of these gazetteers were not reported.


(2007), which describes how each node stores only those parameters relevant to the training data on each node.

some parameters need to be duplicated and thus their method is less efficient than ours in terms of memory usage.


Using models such as Semi-Markov CRFs (Sarawagi and Cohen, 2004), which handle the features on overlapping regions, is one possible direction.

even if we utilize the current gazetteers optimally, the coverage is upper bounded at 70%.


Specifically, if a translation entry (signatured by its Chinese and English strings) to be added is not in the baseline phrase table, we simply add the entry into the baseline table.

if the entry is already in the baseline phrase table, then we merge the entries by enforcing the translation probability as we obtain the same translation entry from two different knowledge sources (one is from parallel corpora and the other one is from the Chinese monolingual corpora).


However, they have only considered relations between single-word phrases and single-character abbreviations.

moreover, the Hmm model is computationally-expensive and unable to exploit the data co-occurrence phenomena that we 5 the Hmm model aligns the characters in the abbreviation to the words in the full-form in an unsupervised way.


(2007) demonstrates that the general feature space they devise achieves a rate of error reduction ranging from 48% to 88% over a chance baseline accuracy, across classification tasks of varying difficulty.

they also show that their general feature space does not generally improve the classification accuracy over subcategorization frames (see   In this study, we explore a wider range of features for AVC, focusing particularly on various ways to mix syntactic with lexical information.


Although it achieves the highest accuracy on the 2-way classification, its accuracy drops drastically as n gets bigger, indicating that SCF does not scale as well as other feature sets when dealing with larger number of verb classes.

the co-occurrence feature (CO), which is believed to convey only lexical information, outperforms SCF on every n-way classification when n ≥ 10, suggesting that verbs in the same Levin classes tend to share their neighboring words.


Despite the Qualifications and other measures taken in the collection phase of the corpus, we believe the quality of the data remains open to question.

the Mechanical Turk framework provided additional information for each assignment, for example the time workers spent on the task.


A high value of Time On Task thus does not necessarily mean that the worker actually spent a long time on it.

a low value indicates that he/she did only spend a short time on it.


From this we can conclude  that in the majority of cases, there was at least one quite similar answer among those for that HIT.

comparing the sentence ids is only an indicative measure, and it does not tell the whole story about agreement.


R-1 R-2 R-SU R-L NoA not included 0.56 0.46 0.37 0.52 NoA included 0.42 0.35 0.28 0.39 Random Answers 0.13 0.01 0.02 0.09 Table 4: Answer agreement: ROUGE-1, -2, -SU and -L. The sentence agreement and ROUGE-figures do not tell us much by themselves.

they are an example of a procedure that can be used to postprocess the data and in further projects of similar nature.


As Table 3 shows, this is different for SAT verbal analogy, where verbs are still the most important feature type and the only whose presence/absence makes a statistical difference.

this time coordinating conjunctions (with prepositions) do help a bit (the difference is not statistically significant) since SAT verbal analogy questions ask for a broader range of relations, e.g., antonymy, for which coordinating conjunctions like but are helpful.


(2001) showed that retrieval scores from IR systems could be modeled using a Normal distribution for relevant documents and exponential distribution for non-relevant documents.

in their study, fusion results using these comparatively complex normalization approaches achieved performance no better than the much simpler CombMNZ.


If a document's score is large in both systems, we expect it to have high probability of relevance.

as a document's score increases linearly in one source, we have no reason to expect its probability of relevance to also increase linearly.


The numbers in parentheses are the size of the clusters described by the path from the root.

we hypothesize that more informative and useful intensional summaries might be constructed from clusters of discovered associations between attributes.


To the best of our knowledge, Schulte im Walde (2006) is the only hard-clustering approach that previously incorporated selectional preferences as verb features.

her model was not soft-clustering, and she only used a simple approach to represent selectional preferences by WordNet's top-level concepts, instead of making use of the whole hierarchy and more sophisticated methods, as in the current paper.


Recent work (Talbot and Osborne, 2007b) has demonstrated that randomized encodings can be used to represent n-gram counts for LMs with signficant space-savings, circumventing information-theoretic constraints on lossless data structures by allowing errors with some small probability.

the representation scheme used by our model encodes parameters directly.


This paper focuses on machine translation.

many of our findings should transfer to other applications of language modeling.


Our model achieved smaller improvements for the phrasal system (0.43 improvement for n=1 translations and 0.72 for the selected n=100 translations).

this improvement is encouraging given the large size of the training data.


Although convex, this constraint is more expensive to enforce, therefore we drop it in our experiments below.

(adding the semidefinite connectedness constraint appears to be feasible on a sentence by sentence level.)


Problem II: Statistical transliteration always chooses the translations based on probabilities.

in some cases, the correct translation may have lower probability.


We also think that both Arg0 and Arg1 can be detected quite well relying on unlexicalized syntactic features only, that is, not knowing which are the verbal and nominal heads.

distinguishing between Arg2-4 is more dependant on the subcategorization frame of the verb, and thus more sensitive to the lack of verbal information.


use a combination of grouped VerbNet roles (for Arg2) and PropBank roles (for the rest of arguments).

our study compares both role sets as they stand, without modifications or mixing.


The motivation behind all these work is to exploit linguistically syntactic structure features to model the translation process.

most of them fail to utilize non-syntactic phrases well that are proven useful in the phrase-based methods (Koehn et al., 2003).


Chiang (2005)'s hierarchal phrase-based model achieves significant performance improvement.

no further significant improvement is achieved when the model is made sensitive to syntactic structures by adding a constituent feature (Chiang, 2005).


(2007) integrate supertags (a kind of lexicalized syntactic description) into the target side of translation model and language mod-el under the phrase-based translation framework, resulting in good performance improvement.

neither source side syntactic knowledge nor reordering model is further explored.


This solution requires larger applicability contexts .

phrases are utilized independently in the phrase-based method without depending on any contexts.


3) The tree number in a rule is not greater than d. In addition, we limit initial rules to have at most seven lexical words as leaf nodes on either side.

in order to extract long-distance reordering rules, we also generate those initial rules with more than seven lexical words for abstract rules extraction only (not used in decoding).


It thereby suggests that SCFG is less effective in modelling parse tree structure transfer between Chinese and English when using Penn Treebank style linguistic grammar and under wordalignment constraints.

formal SCFG show much better performance in the formally syntax-based translation framework (Chiang, 2005).


It clearly indicates that SRRs are very effective in reordering structures, which improve performance by 1.45 (26.07-24.62) BLEU score.

dPRs have less impact on performance in our tree sequence-based model.


(2007) show that rule-based methods are relatively ineffective for orthographic syllabification in English.

few data-driven syllabification systems currently exist.


This approach can be considered an SVM because the model parameters are trained discriminatively to separate correct tag sequences from incorrect ones by as large a margin as possible.

to generative HMMs, the learning process requires labeled training data.


Numbered NB tags are more informative than standard NB tags.

neither annotation system can represent the internal structure of the syllable.


He does not report word accuracy for his syllabification model.

his baseline L2P system is not improved by adding a syllabification model.


Dependency trees are simpler in form than CFG trees since there are no constituent labels.

dependency relations directly model semantic structure of a sentence.


Since we only have a single NT X in the formalism described above, we do not need to add the NT label in states.

we need to specify one of the three types of the dependency structure: fixed, floating on the left side, or floating on the right side.


• filtered: a string-to-string MT system as in baseline.

we only keep the transfer rules whose target side can be generated by a well-formed dependency structure.


(2006), who applied a reranked parser to a large unsupervised corpus in order to obtain additional training data for the parser; this self-training appraoch was shown to be quite effective in practice.

their approach depends on the usage of a high-quality parse reranker, whereas the method described here simply augments the features of an existing parser.


Evaluation of deep parsing results is often reported only in terms of coverage (number of sentences which re-  ceive an analysis), because, since the hand-crafted grammars are optimised for precision over coverage, the analyses are assumed to be correct.

in this experiment, we are potentially 'diluting' the precision of the grammar by using external resources to remove parses and so it is important that we have some idea of how the accuracy is affected.


Restricting by lexical types should have the effect of reducing ambiguity further than POS tags can do, since one POS tag could still allow the use of multiple lexical items with compatible lexical types.

it could be considered more difficult to tag accurately, since there are many more lexical types than POS tags (almost 900 in the ERG) and less training data is available.


Previous studies evaluate simulated dialog corpora using evaluation measures which can be automatically extracted from the dialog systems' logs.

the validity of these automatic measures has not been fully proven.


In this study, we also strive to develop a prediction model of the rankings of the simulated users' performance.

our approach use human judgments as the gold standard.


In human-computer communication, the goal of error recovery strategy is to maximize the user's satisfaction of using the system by guiding for the repair of the wrong information by human-computer interaction.

there are different approaches to improve the robustness of dialog management using n-best hypotheses.


This previous EBDM framework chose a dialog example to maximize the utterance similarity measure.

our system generates a set of multiple dialog examples with each utterance similarity over a threshold given a specific hypothesis.


Therefore, when c i is NEXT TASK, the discourse score is computed as where P (c i |c = top(S)) is a transition probability from the top node c on the focus stack S to the candidate node c i .

there is a problem for cases other than NEXT TASK because the graph has no backward probability.


In most previous work the reward function is manually set, which makes it "the most hand-crafted aspect" of RL (Paek, 2006).

we learn the reward model from data, using a modified version of the PARADISE framework (Walker et al., 2000), following pioneering work by (Walker et al., 1998).


We have also tried some other strategies that extract a larger number of templates from a DT.

the efficacy of the learned rules is quite similar to the one generated by the first method.


SVM's results are the state-of-the-art for the Text chunking task.

using a committee of ETL classifiers, we produce very competitive results and maintain the advantages of using a rule based system.


Moreover, it is necessary to split features into several sets, and then train several corresponding discriminative models separately and preliminarily.

jESS-CM is free from this kind of additional process, and the entire parameter estimation procedure can be performed in a single pass.


We used the value for the best performance with the development set 2 .

it may be computationally unrealistic to retrain the entire procedure several times using 1G-words of unlabeled data.


These facts imply that our SSL framework is rather appropriate for handling large scale unlabeled data.

aSO-semi and JESS-CM have an important common feature.


Note that ASO-semi is also an 'indirect approach'.

our approach is a 'direct approach' because the distribution of y obtained from JESS-CM is used as 'seeds' of hidden states during MDF estimation for join PM parameters (see Section 4.1).


We can calculate the precision (P ) of learned patterns for each relation by annotating the extracted patterns as correct/incorrect.

calculating the recall is a problem for the same reason as above.


While calculating the true recall here is not possible, even calculating the true relative recall of the system against the baseline is not possible as we can annotate only a small sample.


More generally, when used on top of all other components, some of the models slightly degrade performance, as can be seen by those figures in the ablation tests which are higher than the corresponding baseline.

due to their different roles, each of the matching components might capture some unique preferences.


That work, like ours, relies on pattern clusters.

it requires initial word seeds and targets the discovery of relationships specific for some given concept, while we attempt to discover and define generic relationships that exist in the entire domain.


A study can develop its own relationship definitions and dataset, like (Nastase and Szpakowicz, 2003), thus introducing a possible bias; or it can accept the definition and dataset prepared by another work, like (Turney, 2006).

this makes it impossible to work on new relationship types.


Today it is standard for web search engines to show these summaries as one or two lines of text, often with ellipses separating sentence fragments.

there is evidence that the ideal result length is often longer than the standard snippet length, and that furthermore, result length depends on the type of answer being sought.


They used RIPPER as a classifier to detect interrogative questions and their answers and used the resulting question and answer pairs as summaries.

it did not consider contexts of questions and dependency between answer sentences.


(See Section 3.4 for more about CRFs) Linear CRF model has been successfully applied in NLP and text mining tasks (McCallum and Li, 2003;Sha and Pereira, 2003).

our problem cannot be modeled with Linear CRFs in the same way as other NLP tasks, where one node has a unique label.


None of these situations is ideal: the cost of building the training corpus in the former setup is high; in the latter scenario the data tends to be domain-specific, hence unsuitable for the learning of open-domain models.

recent years have seen an explosion of user-generated content (or social media).


The general intuition is to exploit the pairwise preferences induced from the data by training on pairs of patterns, rather than independently on each pattern.

given a weight vector α, the score for a pattern x (a candidate answer) is simply the inner product between the pattern and the weight vector: the error function depends on pairwise scores.


On the one hand, this tagset is much larger than the largest tagset used in English (from 17 tags in most unsupervised POS tagging experiments, to the 46 tags of the WSJ corpus and the about 150 tags of the LOB corpus).

our tagset is intrinsically factored as a set of dependent sub-features, which we explicitly represent.


For example, in Hebrew, the preposition meaning "in", b-, is always prefixed to its nominal argument.

in Arabic, the most common corresponding particle is fy, which appears as a separate word.


In the next section we describe our model's "generative story" for producing the data we observe.

we formalize our model in the context of two languages E and F. the formulation can be extended to accommodate evidence from multiple languages as well.


At each iteration, the sampler selects a random variable X i , and draws a new value for X i from the conditional distribution of X i given the current value of the other variables: The stationary distribution of variables derived through this procedure is guaranteed to converge to the true joint distribution of the random variables.

if some variables can be jointly sampled, then it may be beneficial to perform block sampling of these variables to speed convergence.


We notice that in general, adding Englishwhich has comparatively little morphological ambiguity -is about as useful as adding a more closely related Semitic language.

once characterto-character phonetic correspondences are added as an abstract morpheme prior (final two rows), we find the performance of related language pairs outstrips English, reducing relative error over MONO-LINGUAL by 10% and 24% for the Hebrew/Arabic pair.


In this work, we use the morphological analyzer of MILA -Knowledge Center for Processing Hebrew (KC analyzer).

to English tagsets, the number of tags for Hebrew, based on all combinations of the morphological attributes, can grow theoretically to about 300,000 tags.


Uniform initialization based on the simple suffixbased ambiguity class guesser yields big improvements over the uniform all-open-class initialization.

our refined initial conditions always improve the results (by as much as 40% error reduction).


While recent work, such as GG, aim to use the Bayesian framework and incorporate "linguistically motivated priors", in practice such priors currently only account for the fact that language related distributions are sparse -a very general kind of knowledge.

our method allow the incorporation of much more fine-grained intuitions.


Two-sided class-based models received most attention in the literature.

several different types of mixed word and class models have been proposed for the purpose of improving the performance of the model (Goodman, 2000), reducing its size (Goodman and Gao, 2000) as well as lowering the complexity of related clustering algorithms (Whittaker and Woodland, 2001).


The worst case complexity of the exchange algorithm is quadratic in the number of classes.

input: The fixed number of clusters N c Compute initial clustering while clustering changed in last iteration do forall w ∈ V do forall c ∈ C do move word w tentatively to cluster c compute updated optimization criterion move word w to cluster maximizing optimization criterion Algorithm 1: Exchange Algorithm Outline the average case complexity can be reduced by updating only the counts which are actually affected by moving a word from one cluster to another.


This stems from the fact that the change in log likelihood is calculated by each worker under the assumption that no other changes to the clustering are performed by other workers in this iteration.

if in each iteration only a rather small and randomly chosen subset of all words are considered for exchange, the intuition is that the remaining words still define the parameters of each cluster well enough for the algorithm to converge.


As described in Section 5, we start out the first iteration with a random partition of the vocabulary into subsets each assigned to a specific worker.

instead of keeping this assignment constant throughout all iterations, after each iteration the vocabulary is partitioned anew so that all words from any given cluster are considered by the same worker in the next iteration.


As shown in Table 3, adding the predictive classbased model trained on the ar webnews data set leads to small improvements in dev and nist06 scores but causes the test score to decrease.

adding the class-based model trained on the ar gigaword data set to the other class-based and the word-based model results in further improvement of the dev score, but also in large improvements of the test and nist06 scores.


The paraphrase likelihood can be computed using Equation (1).

we find that using only the MLE based probabilities can suffer from data sparseness.


As described in Section 3.2, we constrain that the pattern words of an English pattern e must be extracted from a partial subtree.

we do not have such constraint on the Chinese pivot patterns.


It is advantageous to consider a space of possible narrative events and the ordering within, not a closed list.

it is worthwhile to construct discrete narrative chains, if only to see whether the combination of event learning and ordering produce scriptlike structures.


For our example, 'started' is inflected for masculine Gender, singular Number, third person.

the noun is definite and is assigned genitive Case since it is in a possessive, idafa, construction.


There is ample evidence that any NN followed by a JJ would make a perfectly valid Argument.

an AST structure would mask the fact that the JJ 'the-Chinese' does not modify the NN 'ministers' since they do not agree in Number 7 , and in syntactic Case, where the latter is genitive and the former is nominative.


We hypothesize that most 'biographical' sentences will contain a reference to the target.

some of these sentences may be irrelevant to a biography; therefore, we filter them using a binary classifier that retains only 'biographical' sentences.


One might collect training data by manually annotating a suitable corpus containing biographical and nonbiographical data about a person, as in (Zhou et al., 2004).


we should note that they may not necessarily represent the successful classification of biographical vs. non-biographical sentences but rather the classification of Wikipedia sentences vs. TDT4 sentences.


In fact, when we use only the SVM regression model to rank the hypothesis sentences, without employing any classifier, then remove redundant sentences, rewrite and trim the results, we find that, interestingly, this approach also outperforms top-DUC2004, although the difference is not statistically significant.

we believe that this is an area worth pursuing in future, with more sophisticated features.


3.1 Impact language models From the retrieval perspective, our collection is the paper to be summarized, and each sentence is a "document" to be retrieved.

unlike in the case of ad hoc retrieval, we do not really have a query describing the impact of the paper; instead, we have a lot of citation contexts that can be used to infer information about the query.


Intuitively, the impact of a paper is mostly reflected in the citation context.

thus the estimation of the impact language model should be primarily based on the citation context C. we would like our impact model to be able to help us select impactreflecting sentences from d, thus it is important for the impact model to explain well the paper content in general.


A simple way is to pool together all the sentences in C and use the maximum likelihood estimator, where c(w, s) is the count of w in s. One deficiency of this simple estimate is that we treat all the (extended) citation sentences equally.

there are at least two reasons why we want to assign unequal weights to different citation sentences: (1) A sentence closer to the citation label should contribute more than one far away.


Percentage of signature terms in vocabulary The number of signature terms gives the total count of topic signatures over all the documents in the input.

the number of documents in an input set and the size of the individual documents across different sets are not the same.


(2005) adds word repetition to their feature set.

their approach deals with all word repetitions on an equal basis, and so degrades quickly in the presence of noise words (their term for words which shared across conversations) to almost complete failure when only 1/2 of the words are shared.


The disentanglement system's test performance decreases proportionally; mean 1-to-1 falls to 36.08, and mean loc 3 to 63.00, essentially baseline performance.

mentions are not sufficient; with only name mention and time gap features, mean 1-to-1 is 38.54 and loc 3 is 67.14.


Improving the current model will definitely require better features for the classifier.

we also left the issue of partitioning nearly completely unexplored.


Inductive Logic Programming (ILP) has been applied to some natural language processing tasks, including parsing (Mooney, 1997), POS disambiguation (Cussens, 1996), lexicon construction (Claveau et al., 2003), WSD (Specia et al., 2007), and so on.

to our knowledge, our work is the first effort to adopt this technique for the coreference resolution task.


Besides, as with our entity-mention predicates described in Section 4.2, we also tried the "All-X" strategy for the entity-level agreement features, that is, whether all mentions in a partial entity agree in number and gender with an active mention.

we found this bring no improvement against the "Any-X" strategy.


Working with automatic hand tracking, Quek (2003) automatically computes perceptually-salient gesture features, such as symmetric motion and oscillatory repetitions.

our feature representation takes the form of a vector of continuous values and is not easily interpretable in terms of how the gesture actually appears.


Thus, the performance gains demonstrated in this paper cannot be explained by such punctuation-like phenomena; we believe that they are due to the consistent gestural themes that characterize coherent topics.

we are interested in pursuing the idea of visual punctuation in the future, so as to compare the power of visual punctuation and gestural cohesion to predict segment boundaries.


AL has been successfully applied already for a wide range of NLP tasks, including POS tagging (Engelson and Dagan, 1996), chunking (Ngai and Yarowsky, 2000), statistical parsing (Hwa, 2004), and named entity recognition (Tomanek et al., 2007).

aL is designed in such a way that it selects examples for manual annotation with respect to a single learning algorithm or classifier.


We refrain from calculating the overall efficiency score (Section 3) here due to the lack of generally accepted weights for the considered annotation tasks.

we require from a good selection protocol to exceed the performance of random selection and extrinsic selection.


Those approaches focus on the combination of classifiers in order to improve the classification error rate for one specific classification task.

the focus of multi-task AL is on strategies to select training material for multi classifier systems where all classifiers cover different classification tasks.


A standard approach is to use the EM algorithm to optimize the empirical likelihoodÊ log p θ (x).

1 EM only finds a local maximum, which we denoteθ EM , so there is a discrepancy between what we get (pθ EM ) and what we want (p * ).


We cannot capture these problematic higher-order dependencies in M .


In the base-Input: raw sentence sent -a list of characters Variables: candidate sentence item -a list of (word, tag) pairs; maximum word-length record maxlen for each tag; the agenda list agendas; the tag dictionary tagdict; start index for current word; end index for current word Initialization:  line POS tagger, candidates in the beam are tagged sequences ending with the current word, which can be compared directly with each other.

for the joint problem, candidates in the beam are segmented and tagged sequences up to the current character, where the last word can be a complete word or a partial word.


One likely reason is that Shi and Wang (2007) included knowledge about special characters and semantic knowledge from web corpora (which may explain the higher baseline accuracy), while our system is completely data-driven.

the comparison is indirect because our partitions of the CTB corpus are different.


the application of beam search was far from trivial because of the size of the combined search space.


Besides the usual character-based features, additional features dependent on POS's or words can also be employed to improve the performance.

as such features are generated dynamically during the decoding procedure, two limitation arise: on the one hand, the amount of parameters increases rapidly, which is apt to overfit on training corpus; on the other hand, exact inference by dynamic programming is intractable because the current predication relies on the results of prior predications.


Additional features most widely used are related to word or POS ngrams.

such features are generated dynamically during the decoding procedure so that the feature space enlarges much more rapidly.


This is a substitute method to use both local and non-local features, and it would be especially useful when the training corpus is very large.

can the perceptron incorporate all the knowledge used in the outside-layer linear model?


Effective rule-based approaches can be designed for some languages such as Spanish.

kominek and Black (2006) show that in languages with a less transparent relationship between spelling and pronunciation, such as English, Dutch, or German, the number of letter-to-sound rules grows almost linearly with the lexicon size.


These approaches include metadata extraction (Cao et al., 2005), expert profile building (Craswell, 2001, Fu et al., 2007, data fusion (Maconald and Ounis, 2006), query expansion (Macdonald and Ounis, 2007), hierarchical language model (Petkova and Croft, 2006), and formal model generation (Balog et al., 2006;Fang et al., 2006).

all of them conduct expert search with what we call a coarse-grained approach.


Macdonald and Ounis (2006) investigate the effectiveness of the voting approach and the associated data fusion techniques.

such models are conducted in a coarse-grain scope of document as discussed before.


The construction integration model described before is already making use of syntactic patterns to some extent, through the use of a shallow parser to identify noun phrases.

that approach does not cover patterns other than noun phrases.


Unary rules of the form N i → N j can form cycles, leading to infinite unary chains with infinite mass.

it is standard in the parsing literature to transform grammars into a restricted class of CFGs so as to permit efficient parsing.


The formalism was initially defined for context-free grammars and later applied to other constituencybased formalisms, such as tree-adjoining grammars (Alonso et al., 1999).

since parsing schemata are defined as deduction systems over sets of constituency trees, they cannot be used to describe dependency parsers.


Therefore, it is possible to define a variant of parsing schemata, where these structures can be defined as items and the strategies used for combining them can be expressed as inference rules.

in order to define such a formalism we have to tackle some issues specific to dependency parsers: • Traditional parsing schemata are used to define grammar-based parsers, in which the parsing process is guided by some set of rules which are used to license deduction steps: for example, an Earley Predictor step is tied to a particular grammar rule, and can only be executed if such a rule exists.


Therefore, as items for constituency parsers are defined as sets of partial constituency trees, it is tempting to define items for dependency parsers as sets of partial dependency graphs.

predictive grammar-based algorithms such as those of Lombardo and Lesmo (1996) and Kahane et al.


Parsing schemata are not suitable for directly describing deterministic parsers, since they work at a high abstraction level where a set of operations are defined without imposing order constraints on them.

many deterministic parsers can be viewed as particular optimisations of more general, nondeterministic algorithms.


For example, X → jingtian X 1 , X 1 this year , seems to capture the use of jingtian/this year as a temporal modifier when building linguistic constituents such as noun phrases (the election this year) or verb phrases (voted in the primary this year).

it is important to observe that nothing in the Hiero framework actually requires nonterminal symbols to cover linguistically sensible constituents, and in practice they frequently do not.


In detail, the monolingual parallel corpus is fairly small, thus automatical word alignment tool like Giza++ may not work well on it.

the monolingual comparable corpus is quite large, hence we cannot conduct the timeconsuming syntactic parsing on it as we do on the monolingual parallel corpus.


In (Bayraktar et al., 1998) the WSJ PennTreebank corpus (Marcus et al., 1993) is analyzed and a very detailed list of syntactic patterns that correspond to different roles of commas is created.

they do not study the extraction of entailed relations as a function of the comma's interpretation.


Pattern-based relation extraction methods (e.g., (Davidov and Rappoport, 2008;Davidov et al., 2007;Banko et al., 2007;Pasca et al., 2006;Sekine, 2006)) could in theory be used to extract relations represented by commas.

the types of patterns used in web-scale lexical approaches currently constrain discovered patterns to relatively short spans of text, so will most likely fail on structures whose arguments cover large spans (for example, appositional clauses containing relative clauses).


Note that in theory, the third relation will not be valid: one example is 'The brothers, all honest men, testified at the trial', which does not entail 'all honest men testified at the trial'.

we encountered no examples of this kind in the corpus, and leave this refinement to future work.


This gives an average precision 0.85 and an average recall of 0.36 for identifying the comma type.

this baseline does not help in identifying relations.


According to the Relation metric, there is no difference between them.

there is a semantic difference between the two sentences -the ATTRIBUTE relation says that being 59 is an attribute of John Smith while the SUBSTITUTE relation says that John Smith is the number 59.


One standard is to adopt a strict logical definition of contradiction: sentences A and B are contradictory if there is no possible world in which A and B are both true.

for contradiction detection to be useful, a looser definition that more closely matches human intuitions is necessary; contradiction occurs when two sentences are extremely unlikely to be true simultaneously.


The second sentence posits a similar relationship that includes one of the entities involved in the original relationship as well as an entity that was not involved.

different outcomes result because a tunnel connects only two unique locations whereas more than one entity may purchase food.


For instance, we detect examples 5 and 6 in table 1.

creating features with sufficient precision is an issue for these types of contradictions.


Even on the web, the doublyanchored hyponym pattern eventually ran out of steam and could not produce more instances.

all of our experiments were conducted using just a single hyponym pattern.


Results reveal the expected increase of performance, specially in terms of recall.

these results can not be directly compared with previous work on this subject, because of the different corpora used.


In this case, uncertainty does not necessarily imply subjectivity.

people sometimes explicitly indicate uncertainty to avoid being subjective.