Taneva and Weikum 2013identify relevant text snippets ('gems') by using an integer linear program to maximize relevance of selected words, and prefer the selection of contiguous content pieces since this maximizes the coherence of the content resulting in a lot of content being chosen from the same source.

the information for a specific article can come from multiple sources and hence this will not achieve information coverage.


Since all the methods optimize for the relevance, the proposed approach has comparable relevance to the baselines.

our proposed approach yields a higher information diversity and coverage as compared to the other approaches.


Standing under the old oak tree, she felt leaves tumbling down her shoulders.

while short, definite expressions signal identifiability and are thus either anaphoric expressions or familiar items, it is much harder to decide which indefinite expressions are bridging anaphors, since indefinite expressions are prototypically used to introduce new discourse referents and principally do not need an antecedent/argument in order to be interpretable.


The final performance of the adapted system (F-score of 19.5) is also given in Table 4.

to ARRAU, SciCorp is an out-of-domain corpus annotated with referential bridging.


These syntax-based models are capable of capturing long-range dependencies by modeling the repair phrases on a syntax tree.

their high requirements for training data reduce their practicality, since it is expensive to obtain training data containing both syntax trees and disfluency annotations.


Traditional output layer of Transformer is only one softmax layer.

either the word sequence or the label sequence is incapable of handling the problem of disfluency detection independently.


For the word sequence, it guides the model to treat each fluent word and disfluency word equally.

in disfluency detection task, the model just needs to distinguish the non-fluent words from the fluent words in sentence.


Specifically, the encoder is responsible for composing the high-level latent representation of the input and the decoder endeavors to decompose this representation into its corresponding input sequence.

the AE usually fails to capture any internal structure of the involved sentences under unconstrained condition.


For the weight sharing constraint based on the shared-latent space assumption, we would expect that the corresponding sentences in the labelled or unlabelled corpus will have the similar latent representations.

the weight sharing constraint alone does not necessarily guarantee the same latent representations.


The ISO standard (Bunt et al., 2010) can be seen as a generalization of all these annotation schemes.

there is no available training data for the ISO standard.


This means that once we know that an arrow falls into the direction, motion or indication category we have the interpretation of the arrow.

in order to come up with the exact relation that explains the action arrows, we need to take a further step.


(Kaufman et al., 2017) transferred the semantics of the selected reference clips to test clips, which keeps consistent and maintains temporal coherence.

the existing methods fail to take advantage of the local constraints which can extract compressed features for generating complementary syntactic elements.


In the human retina, visual features which represent natural signals formed by the peripheral receptors are very high-dimensional.

we argue that the receptors to discover where and what objects only occupy a small fraction of the space of all possible receptor activation due to the statistical regularity and redundancy (Burton and Moorhead, 1987).


The knowledge transfer is achieved by learning common semantic representations for different languages.

usually, word-aligned or sentence-aligned parallel data sets are employed for joint learning (Guo et al., 2016;Duong et al., 2016) most existing joint learning approaches focus only on cross-tasks or cross-lingual knowledge transfer.


(2012) trains an SMT model using conversational in-domain parallel data, and then translates the entire L1 SLU training corpus to L2.

bilingual in-domain data is scarce and costly, making it difficult to deliver both quality and low cost.


The authors report that the indirect alignment gives the best performance.

they also point out that distant language pairs suffer severely in word alignment.


Generative dialog models usually adopt beam search as the inference method to generate responses.

small-width beam search only focuses on the limited current optima.


Chinese word segmentation (CWS) trained from open source corpus faces dramatic performance drop when dealing with domain text, especially for a domain with lots of special terms and diverse writing styles, such as the biomedical domain.

building domain-specific CWS requires extremely high annotation cost.


These models rank the antecedents (mentions that appear earlier in discourse) and recover the full coreference clusters from local decisions.

unlike coreference relations, sequencing relations are directed.


Recent advances in event coreference have been promoted by the availability of annotated corpora.

due to the complex nature of events, approaches to event coreference adopt quite different assumptions and definitions.


For all the featuresf of the gold standard graphz and featuresf of a decoded graphẑ, the feature delta is simply: ∆ =f −f .

a decoded graph may contain links that are not directly presented but inferable from the gold standard graph.


Building these scripts by hand is labor intensive, so automatically learning script knowledge has attracted attention for decades (Balasubramanian et al., 2013;Chambers & Jurafsky, 2009;Mooney & DeJong, 1985;Pichotta & Mooney, 2014.

we are also interested in what such techniques reveal about broad, quantitative properties of discourse in general.


Thus, we draw a line after chainsim and other techniques used on the cloze task, referring to them as the candidate score.

a candidate score alone is not enough to generate schemas.


F1 for these skills can be improved by adding more examples to the k-NN classifier.

this approach does not apply to a question-answering skill.


When two triples are identical -such as twice labeling "Hillary Clinton as ":instance-of person", or adding multiple ":wiki" links to "Hillary Clinton" -we merge those redundant bits of information.

if a variable has different concepts or relations (e.g.


This further contributes to the success in numerous natural language processing (NLP) tasks such as language modeling (Cheng et al., 2016;Inan et al., 2017), sentiment analysis (Cheng et al., 2016;Kumar et al., 2016), and reading comprehension (Cheng et al., 2016;Chen et al., 2016;Shen et al., 2017).

effective and efficient grounding of distributional embeddings remains challenging.


On the other hand, for real-world datasets, taking the max in loss function tends to be very sensitive to label noise, resulting in fake negative samples.

given an image-caption pair (i, c), we employ human heuristics and WordNet knowledge base to generate contrastive negative samples C (c).


As the constitution of the dataset ensures the frequency in the entire dataset of the words used for replacement, the visual grounding of these frequent nouns is easy to obtain.

the semantics of relations (including prepositions and entity relations) or numbers are not diverse enough in the dataset, leading to the poor performance of VSE against these attacks.


As shown in Table 3, numeraltyped contrastive adversarial samples improve the counting ability of models.

it is still not clear about where the gain comes from, as the creation of numeral-typed samples may change the form (i.e., MS-COCO Test MS-COCO Test (w/.


In our experiments, we only used the stance label as attributes of text.

it may also be possible to inherit attributes from the author of the text.


To illustrate the means of persuasion, an emphasis on pathos could, for example, lead to a preference for gradually increasing the strength of emotional appeal throughout the text.

for a logos-oriented strategy, it may be important that the sequence of units coheres locally and globally (which for a pathos-oriented argument may be less relevant, or even detrimental).


c4 Proponents of the death penalty count on its deterring effect as well as and the ultimate elimination of any potential threat.

p5 despite the death penalty there are significantly more homicides in the US than in Germany.


This measure does not take into account the number of language switch points in a sentence (denoted by P (x)) and so the authors modify it further: The code-mixing in the entire corpus can then be quantified by taking an average of the above measure across all sentences in the corpus: where U is the number of sentences in the corpus.

their main assumption that the language which has the maximum number of tokens in a sentence is the Matrix language, may not always hold.


The community has already addressed this phenomenon by introducing challenges on POS-Tagging, Language Identification, Language Modeling, etc on the code-mixed corpora.

the approaches to development of dialog systems still rely on monolingual conversation datasets.


They extended Long short-term memory (LSTM) (Hochreiter and Schmidhuber, 1997) to accommodate graph structures to capture hierarchical and temporal conversations.

their model includes designs for efficiently learning the popularity of Reddit.


The broader view is implemented, inter alia, in the Latvian LVTB treebank and in the English EWT treebank (the latter henceforth abbreviated to UD EN EWT ), where in both cases about 1% of utterances contain a head with an obj and a ccomp dependent; one of the 160 such cases in UD EN EWT is: (1) Apparently the City informed [them] obj [that no permit is required] ccomp .

some treebanks -including the Finnish TDT treebank and the Polish SZ treebank (UD PL SZ , for short) -take the stronger position: if two dependents compete for the status of the direct object, only one of them may be labelled as obj or ccomp, and the other one is given the status of indirect object, iobj.


As is made clear in the description of the xcomp label, 7 secondary predicates analysed as arguments are xcomp dependents of the verb, with an enhanced nsubj relation to the nominal they predicate of, as in (9).

those analysed as adjuncts are acl dependents of the nominal elements they predicate of, as in (10).


In sentences such as "We asked the president about the hunt for Osama bin Laden", the prepositional phrase about the hunt for Osama bin Laden will receive the obl -i.e., noncore -dependency label (this is the annotation of such dependents of ASK in UD EN EWT ).

in the sentence "We asked the president whether Americans are safe at home", the subordinate interrogative clause, being an argument, is a core dependent, so it will be annotated as ccomp (again, this is the annotation of such subordinate clause dependents of ASK in UD EN EWT ).


), otherwise somewhat awkward for the basic proposal presented in the previous subsection: The labels indicated in these examples reflect the constrained understanding of cores: in both cases both conjuncts are oblique, so the whole coordinate dependent is oblique.

on the standard UD view, (34) involves a coordination of a direct object an idea and a finite clause, which -for this reason -must also be analysed as core.


The symmetry between the three types of dependents -nominal, 'closed' clausal and 'open' clausal -can be made more conspicuous by renaming the labels as in (52).

20 these new names still mix two different types of information: grammatical function (subj, obj and obl) and grammatical category and 'openness' status (n, c, x).


This task is similar to ours in the sense that a simple binary categorization of items is sought.


As do we, those authors focused on noun-noun compounds and they did not address polarity classification.

their approach targets higher frequency words as it relies on the availability of sufficient corpus data to enable the use of distributional similarity.


Table 4 shows that for most items this seems to be borne out, although the difference is not statistically significant, with the exception of -hai.

for -könig the reverse situation holds in a statistically significant way: the frequencies of affixoid formations are higher than those of the non-affixoid formations.


In regular compounds, the meaning of the complex form is more or less compositionally derived from the meaning of the components.

since affixoid uses of the morphemes in question go along with bleached meanings, the semantics of a complex word containing an affixoid use should be harder to model compositionally.


For könig the results drop down as low as the majority baseline in the present setting.

the low result is very likely also heavily driven by the fact that the instances of könig make up close to 40% of our whole dataset.


In other words, when testing on könig, we use only 60% of the dataset to train on.

we also see lower results on affixoids such as bolzen that are much less frequent than könig and for which we use more training instances in the present generalization setting than we did in the cross-validation setting in §5.1.


OLLIE (Mausam et al., 2012) follows the idea of bootstrap learning of patterns based on dependency parse paths.

while WOE relies on Wikipedia-based bootstrapping, OLLIE applies a set of high precision seed tuples from its predecessor system REVERB (see section 2.2) to bootstrap a large training set over which it learns a set of extraction pattern templates using dependency parses (see Figure 2).


In order to preserve such inter-proposition relationships, tuples may contain references to other propositions.

as opposed to OLLIE, where additional contextual modifiers are directly assigned to the corresponding relational tuples, Bast and Haussmann (2013) represent contextual information in the form of separate, linked propositions.


Finally, the problem of canonicalizing relational phrases and arguments has been hardly addressed so far.

normalizing extractions would be highly beneficial for downstream semantic tasks, such as textual entailment or knowledge base population.


to Reimers and Gurevych (2017b), who reported that SGD is the worst optimizer, our results show that SGD outperforms all other optimizers significantly (p < 0.01), with a slower convergence process during training.


All of these aim to increase dependence between topics and metadata.

our goal is to make topics independent of specified metadata.


Binary relevance (BR) (Boutell et al., 2004) is one of the earliest attempts to solve the MLC task by transforming the MLC task into multiple single-label classification problems.

it neglects the correlations between labels.


(2017) use CNN and recurrent neural network (RNN) to capture the semantic information of texts.

they either neglect the correlations between labels or do not consider differences in the contributions of textual content when predicting labels.


Our proposed methods are developed based on traditional Seq2Seq models.

the mask module is added to the proposed methods, which is used to prevent the models from predicting repeated labels.


In general, even when a simple linear function generates the training data, these methods are not guaranteed to be consistent with it (Rot99a).

if the feature space is chosen so that they are, the robustness properties shown above become significant.


For the sake of sireplicity we opted for a linear combination.

simply maximizing correct parses and minimizing incorrect ones would most likely lead to overfitting.


Instead, a better (but disappointing) strategy would be simply using the tuning corpus.

this is not the situation of LB --plot (d)--for which a moderate, but consistent, improvement of accuracy is observed when retaining the original training set.


On the one hand, the type of features used in the rules was significantly different between corpora and, additionally, there were very few rules that applied to both sets.

the sign of the prediction of many of these common rules was somewhat contradictory between corpora.


One obvious strategy is to use a linear combination of separate language and translation components, of the form: where p(w[hi) is a language model, p(wli , s) is a translation model, and A E [0, 1] is a combining weight.

this appears to be a weak technique (Langlais and Foster, 2000), even when A is allowed to depend on various features of the context (hi, s).


Because different corpora are used to obtain the various results reported in the literature and the problem is often defined differently, detailed comparison is difficult.

the accuracy achieved appears to approach the accuracy results achieved with handwritten rules.


When a ~ and e are both high, the STL is consuming an unreasonable number of input sentences.

the problem is not intrinsic to the STL model of acquisition.


For example, concern's conflation set contains itself as well as "concerned," "concerns," and "concerning" (or, in shorthand notation, the set is {a,b,c,d}).

to evaluate an algorithm, we sum the number of correct (C), inserted (Z), and deleted (D) words it predicts for each hypothesized confla- in making these computations, we disregard any CELEX words that are not in the algorithm's data set and vice versa.


Furthermore, the WSME model is easier to train than the conditional one, because in the WSME model we don't need to estimate the normalization constant Z during the training time.

for each event (x, y) in the training corpus, we have to calculate Z(x) in each iteration of the MEC model.


The first sum in 3is the expected value of fie ~::#, and it is obviously not possible to sum over all the sentences.

we can estimate the mean by using the empirical expected value: means of the sample mean with respect to P0 (Chen and Rosenfeld, 1999).


The transformation between T t and T3 should be one of the simplest (accordingly to the operations cost; see section 3.3), and then the search would be limited to the retrieval of a pair (T1,T2) sharing an equivalent transformation.

this is still time-consuming, and we are trying to define a general way to limit the search in such a tree structure space, for example based on tree indexing for efficiency (Daelemans et al., 1997).


We find that GA feature selection always significantly outperforms the MBLP variant without selection and that feature ordering and weighting with CA significantly outperforms a situation where no weighting is used.

gA selection does not significantly do better than simple iterative feature selection methods, and gA weighting and ordering reach only similar performance as current information-theoretic feature weighting methods.


when we used these in the ten-fold cv setup of our main experiments, this gave a mean score of 97.4 (± 0.9) for IBi-IG with CA-selection and a mean score of 97.1 (± 1.1) for IGTREE with GA-selection.


we have found no results (on these particular data) that indicate an advantage of evolutionary feature selection approach over the more classical iterative methods.


Such property seems important for IR purposes, where we might prefer noise rather than silence in the recall process.

it must remain optional, as some other tasks (such as the NP extraction, or the phrase chunking (Abney, 1991)) may need a full disambiguation.


In recent years, a variety of Machine Learning (ML) techniques has been used to improve the portability of IE systems to new domains, as in SRV (Freitag, 1998), RAPIER (Califf and Mooney, 1997), LIEP (Huffman, 1996), CRYSTAL (Soderland et al., 1995) and WHISK (Soderland, 1999) .

some drawbacks remain in the portability of these systems: a) existing systems generally depend on the supported text style and learn IE-rules either for structured texts, semi-structured texts or free text , b) IE systems are mostly single-concept learning systems, c) consequently, an extractor (e.g., a rule set) is learned for each concept within the scenario in an independent manner, d) the order of execution of the learners is set manually, and so are the scheduling and way of combination of the resulting extractors, and e) focusing on the training data, the size of available training corpora can be inadequate to accurately learn extractors for all the concepts within the scenario 2.


As argumented in an earlier paper (van Halteren, 2000a), a theory-based feature weight determination would have to take into account each feature's decisiveness and reliability.

clear definitions of these qualities, and hence also means to measure them, are as yet sorely lacking.


In practice, due to the finite size of the training corpus, the number of rules is always moderate.

as higher values of k lead to an enormous number of possible rules, huge data sets would be necessary in order to have a reliable estimate of the probabilities for values above k = 3.


Our NP chunks are very similar to the ones of Ramshaw and Marcus (1995 (NP-SBJ *-1) (VP to (VP be (ADJP-PRD excellent)))))) , but ... [CONJP Not only ] does [NP your product ] [vP have to be ] [ADJP excellent ] , but ... ADVP chunks mostly correspond to ADVP constituents in the Treebank.

aDVPs inside aDJPs or inside VPs if in front of the main verb are assimilated into the aDJP respectively VP chunk.


The initial performance of the tagger was improved by a post-process correction method based on error driven learning and by 4In the literature about related tasks sometimes the tagging accuracy is mentioned as well.

since the relation between tag accuracy and chunk precision and recall is not very strict, tagging accuracy is not a good evaluation measure for this task.


The evaluation program shows that this simple procedure reaches its best result for 5-contexts (table 1) with 92.46% label accuracy and phrase correctness measured by FZ=i = 87.23.

the improvement from 3-contexts to 5contexts is insignificant, as 3-contexts reached 92.41% accuracy and F~=1=87.09.


It is 1Precisely, the number of combination becomes 23.

we do not consider I-LST tag since it dose not appear in training data.


This extra information can be encoded in a state.

one must balance this approach with the fact that as the amount of information in a state increases, with limited training material, the chance of seeing such a state again in the future diminishes.


The training sets are very small, however, so that the need for feature restrictions disappears and the full model can be used.

the limited size of the training sets has as a disadvantage that hill-climbing becomes practically useless.


Using such tools has simplified ontology construction.

the wide-spread usage of ontologies is still hindered by the timeconsuming and expensive manual construction task.


In contrast to some other approaches we include into the groups also some verbs which are in fact infinitive participants of verb valencies.

we are able to detect such cases and recognize the "pure" verb groups afterwards.


Memory-based learning (MBL) has enjoyed considerable success in corpus-based natural language processing (NLP) tasks and is thus a reliable method of getting a high-level of performance when building corpus-based NLP systems.

there is a bottleneck in MBL whereby any novel testing item has to be compared against all the training items in memory base.


This is provided to give a more direct comparison with the results in (Daelemans et al., 1999a).

for many NL tasks it is not a good measure and the fscore is more accurate.


Sure enough both smoothing and cut-off values will allow to improve the precision.

literature has shown that the most frequent sense baseline needs less training data.


We h a ve presented some preliminary results demonstrating the bene t of using multilingual data.

we conducted our experiments only on a small test set of 32 verbs in one language pair.


The PP attachment problems we tackle are those involved in analysing such NPs for languages relying on composition of Romance type (as French or Italian), as well as those present in verbal configurations for languages relying on composition of Germanic type (as German or English).

our model can easily be extended to deal with other cases as well.


The results obtained in previous works relying on semantic classes are above ours (around 0.82 for (Brill and Resnik, 1994) and 0.77 for (Lauer and Dras, 1994)), but a direct comparison is difficult inasmuch as only three-word sequences (V N P, for (Brill and Resnik, 1994) and N N N for (Lauer and Dras, 1994)) were used for evaluation in those works, and the language studied is English.

it may well be the case that the semantic resource we use does not compare well, in terms of coverage and homogeneity, with WordNet, the semantic resource usually used.


Our results show that the complete model, making use of all the available information yields the best results.

these results are still low, and we still need to precisely identify how to improve them.


results of Section 2.1) and may learn a same concept in different ways.

9 the combination of classifiers using different oversampling and undersampling rates may be useful since we may not be able to predict, in advance, which rate is optimal (cf.


For instance, a simple sentence like "Show me the meal" has the gold standard parse: (S (VP (VB Show) (NP (PRP me)) (NP (DT the) (NN meal)))) and is parsed by this algorithm as (ROOT (VVB Show) (PNP me) (NP (AT0 the) (NN1 meal))) According to this evaluation scheme its recall is only 33%, because of the presence of the nonbranching rules, though intuitively it has correctly identified the bracketing.

the crossing brackets measures overvalues these algorithms, since they produces only partial parses -for some sentences my algorithm produces a completely flat parse tree which of course has no crossing brackets.


It can follow a verb, begin a sentence, end a sentence, and so on.

iN DT is generally followed by some kind of a noun or adjective.


This quantity was not used in practice because, although it is an excellent indicator of NP, PP, and intransitive S constituents, it gives too strong a bias against other constituents.

neither system is driven exclusively by the entropy measure used, and duplicating the above rankings more accurately did not always lead to better end results.


The "compact grammar" aspect of MDL is perhaps closer to some traditional linguistic argumentation which at times has argued for minimal grammars on grounds of analytical (Harris, 1951) or cognitive (Chomsky and Halle, 1968) economy.

some CFGs which might possibly be seen as the acquisition goal are anything but compact; take the Penn treebank covering grammar for an extreme example.


For example, DT NN is an extremely common NP, and when it occurs, it is a constituent around 82% of the time in the data.

when it occurs as a subsequence of DT NN NN it is usually not a constituent.


As we are considering each pair independently from the rest of the parse, this model does not correspond to a generative model of the kind standardly associated with PCFGs, but can be seen as a random field over the possible parses, with the features being the sequences and contexts (see (Abney, 1997)).

note that we were primarily interested in the clustering behavior, not the parsing behavior, and that the random field parameters have not been fit to any distribution over trees.


Applied to a system for automatic pronominal anaphora resolution, it led to a substantial improvement in the ratio of suitable and unsuitable candidates in the sets considered by the anaphora resolver (Evans and Orȃsan, 2000).

the previous system has two main weaknesses.


The classification described in the previous section is useful for determining the animacy of a sense, even for those which were not previously found in the annotated corpus, but which are hyponyms of a node that has been classified.

nouns whose sense is unknown cannot be classified directly and therefore an additional level of processing is necessary.


This was expected given that our method is based on the idea that in most of the cases the number of animate and inanimate senses determines the animacy of a noun.

this would mean that the same noun will be classified in the same way regardless of the text.


Several studies rely on the linguistic intuition that the head of the noun phrase makes a greater contribution to the semantics of the nominal group than the modifiers.

for some specific tasks in NLP , the head is not necessarily the most semantically important part of the noun phrase.


Rules can be of various normal forms and can be ordered.

the appropriate ordering can be hard to find and the key point of many rule induction algorithms is to minimize the search strategy through the space of possible rule sets and orderings.


Hence, we have defined a learning setting that is both positive examples only and unsupervised.

there has been some suggestion that negative evidence may be available in the form of parental correction.


The most effective learning model appears to have been a combination of symbolic and stochastic techniques, like the approach presented here.

a full lexicon is supplied to the learner, so that the problem is reduced to one of disambiguating between the possible supertags.


For example, one could model P(T money order) via P(T AE AE µ and P(T manifest itself) via P(T Î È Ö Ó µ.

because English words often have multiple parts of speech (e.g.


Given a dictionary marked with core parts of speech, it is trivial to generate hypothesized inflected forms following the regular paradigms, as shown in the left size of Figure 3.

due to irregularities and semi-regularities such as stem- a$ e$ Noun-Nomin-p3-fem-plur-indef e$ i$ Noun-Nomin-p3-fem-plur-indef ea$ ele$ Noun-Nomin-p3-fem-plur-indef i$ ile$ Noun-Nomin-p3-fem-plur-indef a$ ale$ Noun-Nomin-p3-fem-plur-indef $ $ Adj-masc,neut-sing $ā$ Adj-fem-sing $ i$ Adj-masc,neut,fem-plur $ e$ Adj-fem,neut-plur ru$ ra$ Adj-fem-sing ru$ ri$ Adj-masc,neut,fem-plur ru$ re$ Adj-fem-plur ... ... ... e$ $ Verb-Indic_Pres-p1-sing e$ i$ Verb-Indic_Pres-p2-sing e$ e$ Verb-Indic_Pres-p3-sing e$ em$ Verb-Indic_Pres-p1-plur e$ eţi$ Verb-Indic_Pres-p2-plur e$ $ Verb-Indic_Pres-p3-plur changes, such generation will clearly have substantial inaccuracies and overgenerations.


As theory predicts, unigram probability estimates for many words do converge as corpus size grows.

contrary to intuition, we found that for many commonplace words, for example tightness, there was no sign of convergence as corpus size approaches one billion words.


This shows the improvement in stability of the estimates from using large corpora, although ±20% is a considerable deviation from the goldstandard estimate.

other words, for instance tightness, fail spectacularly to converge to their final estimate before the influence of the forced convergence of the ratio starts to take effect.


There are also some content words that appear to be distributed evenly.

some words appear often in the first 5 million word sample but are not seen again in the remainder of the corpus.


The most obvious and expected conditioning of the random variables is the topic of the text in question.

it is hard to envisage seemingly topicneutral words, such as tightness and newly, being conditioned strongly on topic.


ρ is not a probability measure, but just a function of y and c into [0,1].

since a geometric mean reflects "uniformity" of the averaged values, ρ captures the degree to which p(y,c,X i ) values are high across all datasets.


This causes two important problems that motivate our work: 1) annotating data is often a di cult and costly task involving a lot of human work 1 , such that large collections of labelled data are di cult to obtain, and 2) interannotator agreement tends to be low i n e g g enomics collections (Krauthammer et al., 2000), thus calling for methods that are able to deal with noise and incomplete data.

unsupervised techniques do not require labelled data and can thus be applied regardless of the annotation problems.


With the introduction of unknown labels as supplementary optimisation variables, the constraints of the quadratic optimisation problem are now nonlinear, which m a k es solving more di cult.

approximated iterative algorithms exist which can e ciently train Transductive SVMs.


The context we consider contains the four preceding and the four following words.

we did not take into account the position of the words in the context, but only their presence in the right o r left context, and in addition we replaced, whenever possible, each w ord by a feature indicating (a) whether the word was part of the gene lexicon, (b) if not whether it was part of the protein lexicon, (c) if not whether it was part of the species lexicon, (d) and if not, whenever the candidate was neither a noun, an adjective nor a v erb, we replaced it by its part-of-speech.


One one hand, Fisher kernels are derived from information-geometric arguments (Jaakkola and Haussler, 1999) which require that the kernel reduces to an inner-product of Fisher scores.

polynomial and RBF kernels often display better performance than a simple dot-product.


Important issues wide open to discussion are: validation of results, psycho-linguistic relevance of the experimental setup, principled ways of surpassing the contextfree limitations of Lambek grammar (inherited in GraSp), just to mention a few.

already the spin-offs of our project (the collection of non-linguistic learners) do inspire confidence in our tenets, we 12 The learning experiment sketched in Moortgat (2001) shares some of GraSp's features.


• No assumption on the independence between dependency relations The probabilistic model assumes that depen-dency relations are independent.

there are some cases in which one cannot parse a sentence correctly with this assumption.


Japanese dependency relations are heavily constrained by such static features since the inflection forms and postpositional particles constrain the dependency relation.

when a sentence is long and there are more than one possible dependency, static features, by themselves cannot determine the correct dependency.


For the first impression, it may seems natural that higher accuracy is achieved with the probabilistic model, since all candidate dependency relations are used as training examples.

the experimental results show that the cascaded chunking model performs better.


A leading advantage of ME models is their flexibility: they allow stochastic rule systems to be augmented with additional syntactic, semantic, and pragmatic features.

the richness of the representations is not without cost.


Any degree of precision desired could be reached by any of the algorithms, with the appropriate value of ε.

gIS, say, would require many more iterations than reported in Table 2 to reach the precision achieved by the limited memory variable metric algorithm.


We typically think of computational linguistics as being primarily a symbolic discipline.

statistical natural language processing involves non-trivial numeric computations.


Perhaps the discrepancy (especially for the F l 8 9 G 6 E class) reflects differences in the way the training data was annotated; F l 8 9 G 6 E is a highly heterogenous class, and the criteria for distinguishing between  The models described here are very simple and efficient, depend on no preprocessing or (with the exception of s w v ¦ ) external databases, and yet provide a dramatic improvement over a baseline model.

the performance is still quite a bit lower than results for industrial-strength language-specific named entity recognition systems.


Moreover, some elements of such multiwords (like president) are high frequency items, well known and of high significance for identifying a named entity.

there are elements that often appear in named entities, but are not characteristic for them (like the first name Israel).


During the run the error rate increases due to finding candidates and verification through misclassified items.

as the "era" example (see section 4) illustrates, misclassified items support the classification of goal items.


The maximum entropy models bene ted from extra feature which encoded capitalization information, positional information and information about the current word being part of a person name earlier in the text.

incorporating a list of person names in the training process did not help (Spanish test set: F =1 =73.66 Dutch test set: F =1 =68.08) Jansche (2002) employed a rst-order Markov model as a named entity recognizer.


One might conjecture that a supertagging approach could go a long way toward parse disambiguation.

an upper bound for such an approach for our corpus is below 55 percent parse selection accuracy, which is the accuracy of an oracle tagger that chooses at random among the parses having the correct tag sequence (Oepen et al., 2002).


Then, for instance, in the sentence: •(3)John talks to Mary with logical form •(4) talk-communicative-act(e,x,y), john(x), comm-to(y), mary(y) the verb talks has two arguments, the NP subject John, and the PP to Mary, as represented in the logical form associated with the verb, where the PP is the second argument and as such should be included in the subcategorisation frame of the verb: (S\NP)/PP.

in the sentence: •(5)Bob eats with a fork with logical form •(6) eat-ingest-act(e,x), bob(x), instrwith(e,y), a(y), fork(y) the PP with a fork is not an argument of the verb eat as reflected in its logical form and should not be included in its subcategorisation frame, which is S\NP.


In this case, the PP is an adjunct to the verb.

this preposition can be selected as the argument of the verb swim in sentence 9, which denotes a motion event and is an instance of type motion-act.


Recent studies on the automatic extraction of lexicalized grammars (Xia, 1999;Chen and Vijay-Shanker, 2000;Hockenmaier and Steedman, 2002a) allow the modeling of syntactic disambiguation based on linguistically motivated grammar theories including LTAG (Chiang, 2000) and CCG (Clark et al., 2002;Hockenmaier and Steedman, 2002b).

existing models of disambiguation with lexicalized grammars are a mere extension of lexicalized probabilistic context-free grammars (LPCFG) (Collins, 1996;Collins, 1997;Charniak, 1997), which are based on the decomposition of parsing results into the syntactic/semantic dependencies of two words in a sentence under the assumption of independence of the dependencies.


The first problem with LPCFG is partially solved by this model, since the dependencies not represented in LPCFG (e.g., long-distance dependencies and argument/modifier distinctions) are elegantly represented, while some relations (e.g., the control relation between "want" and "student") are not yet represented.

the other two problems remain unsolved in this model.


This model can represent all the syntactic/semantic dependencies of words in a sentence.

the statistical model is still a mere extension of LPCFG, i.e., it is based on decomposition into primitive lexical dependencies.


A possible solution to this problem is to directly estimate p(A|w) by applying a maximum entropy model (Berger et al., 1996).

such modeling will lead us to extensive tweaking of features that is theoretically unjustifiable, and will not contribute to the theoretical investigation of the relations of syntax and semantics.


Consequently, the probability model takes the following form.

this model has a crucial flaw: the maximum likelihood estimation of semantics probability is intractable.


A possible solution to this problem is to map SVMs' results into probabilities through a Sigmoid function, and use Viterbi search to combine those probabilities (Platt, 1999).

this approach conflicts with SVMs' purpose of achieving the so-called global optimization 1 .


As far as SVMs are concerned, the use of parses or pairs of parses both maximize the margin between x i1 and x ij , but the one using a single parse as a sample needs to satisfy some extra constraints on the selection of decision function.

these constraints are not necessary (see section 3.3).


E is the space of event of the classification problem, and For classifier g f on space E, its loss function is x 2 =x 1 and g f (x 1 , x 2 ) = +1 0 otherwise Therefore the expected risk R vote (f ) for the voting problem is equivalent to the expected risk R class (g f ) for the classification problem.

the definition of space E violates the independently and identically distributed (iid) assumption.


Variants of the Perceptron algorithm, which are known as Approximate Maximal Margin classifier, such as PAM (Krauth and Mezard, 1987), ALMA (Gentile, 2001) and PAUM (Li et al., 2002), produce decision hyperplanes within ratio of the maximal margin.

almost all these algorithms are reported to be inferior to SVMs in accuracy, while more efficient in training.


This seems to nullify our effort to create this new algorithm.

our new algorithm is still useful for the following reasons.


The main difference between Basilisk and Meta-Bootstrapping is that Basilisk scores each noun based on collective information gathered from all patterns that extracted it.

meta-Bootstrapping identifies a single best pattern and assumes that everything it extracted belongs to the same semantic class.


To find candidate seed words, we automatically identified 850 nouns that were positively correlated with subjective sentences in another data set.

it is crucial that the seed words occur frequently in our FBIS texts or the bootstrapping process will not get off the ground.


Bagga and Baldwin, 1998) has primarily approached personal name disambiguation using document context profiles or vectors, which recognize and distinguish identical name instances based on partially indicative words in context such as computer or car in the Clark case.

in the specialized case of personal names, there is more precise information available.


This would yield a set of cluster seeds, divided by features, which could then be used for further clustering.

this method relies on having a number of pages with extracted features that overlap from each referent.


The original EM method failed to boost the precision (76.78%) of the rule learned through only labeled data.

cV-EM and cV-EM2 boosted the precision to 77.88% and 78.56%.


Following this procedure, we should look up the thesaurus ID of the word 'suru'.

we do not look up the thesaurus ID for a word that consists of hiragana characters, because such words are too ambiguous, that is, they have too many thesaurus IDs.


In the many cases, the CV-EM2 outputs the same number as the CV-EM.


The EM method failed to boost it, and degraded it to 73.56%.

by using CV-EM the precision was boosted to 77.88%.


In CV-EM, the best point is selected in cross validation, that is 3.

cV-EM2 estimates 0 by using the relation of three precisions: the initial precision, the precision for the iteration 1 and the precision at convergence.


For one word 'mae' in nouns and three words 'kuru ', 'koeru' and 'tukuru' in verbs, CV-EM was superior to CV-EM2.

for four words 'atama ', 'kaku n', 'te' and 'doujitsu' in nouns and four words 'ukeru', 'umareru', 'toru' and 'mamortu' in verbs, CV-EM2 was better to CV-EM.


In ideal estimation, the precision for noun words was boosted from 76.78% to 79.64% by the EM method, that is 1.037 times.

the precision for verb words was boosted from 78.16% to 79.92% by the EM method, that is 1.022 times.


Naive Bayes assumes the independence of features, too.

this assumption is not so rigid in practice.


Further results show that this form of co-training considerably outperforms self-training.

we find that simply retraining on all the newly labelled data can, in some cases, yield comparable results to agreement-based co-training, with only a fraction of the computational cost.


In speech processing, various adaption techniques have been proposed for language modeling.

the language modeling problem is essentially unsupervised (density estimation) in the sense that it does not require any annotation.


In addition, we also manually labeled the following data: It is perhaps not surprising that a sentence boundary classifier trained on WSJ does not perform nearly as well on some of the other data sets.

it is useful to examine the source of these extra errors.


Since is only an approximation of the conditional probability, this estimate may not be entirely accurate.

one would expect it to give a reasonably indicative measure of the classification performance.


If we can identify those data, then a natural way to enhance the underlying classifier's performance would be to include them in the training data, and then retrain.

a human is required to obtain labels for the new data, but our goal is to reduce the human labeling effort as much as possible.


This method helps to some extent, but not as much as we originally expected.

we have only used the simplest version of this method, which is susceptible to two problems mentioned earlier: it tends (a) to select data that are inherently hard to classify, and (b) to select redundant data.


using six types of editing (editing based on low and high value for all three criteria).

with the previous study, where for all tasks even the smallest editing led to significant accuracy decreases, for our task there was no clear decrease in performance.


First, we run the predictors on all tasks using all features.

with the previous study which favored the memory-based learner for almost all their tasks, our results favor IB1-IG for only two of the four tasks (ISCORR and STATUS).


We have previously shown that a broad set of 220 noisy features performs well in supervised verb classification (Joanis and Stevenson, 2003).

to Merlo and Stevenson (2001), we confirmed that a set of general features can be successfully used, without the need for manually determining the relevant features for distinguishing particular classes (cf.


Dorr and Jones, 1996;Schulte im Walde and Brew, 2002).

in contrast to Schulte im Walde and Brew (2002), we demonstrated that accurate subcategorization statistics are unnecessary (see also Sarkar and Tripasai, 2002).


3 In tests of the m easure on some contrived clusterings, we found it quite conservative, and on our experimental clusterings it did not often attain values higher than .25.

it is useful as a relative measure of goodness, in comparing clusterings arising from different feature sets.


The measure is independent of the true classification, and could be high when the other dependent measures are low, or vice versa.

it gives important information about the quality of a clustering: The other measures being equal, a clustering with a higher value indicates tighter and more separated clusters, suggesting stronger inherent patterns in the data.


This performance comparison tentatively suggests that good feature selection can be helpful in our task.

it is important to find a method that does not depend on having an existing classification, since we are interested in applying the approach when such a classification does not exist.


Many feature sets performed very well, and some far outperformed our best results using other feature selection methods.

across our 10 experimental tasks, there was no consistent range of feature ranks or feature set sizes that was correlated with good performance.


For FrameNet, the combined use of both collocation types achieves better performance for the individual classifiers: 70.3% versus 68.5%.

classification using a single classifier is not effective due to confusion among the fine-grained roles.


It enhances a WordNet synset with its contextual information and refines its relational structure, including relations such as hypernym, hyponym, antonym and synonym, by maintaining only those links that respect contextual constraints.

phraseNet is not just a functional extension of WordNet.


There is some evidence (Wessel et al., 2001) that this approach can give results that are at least as good as those obtainable with an external CE model.

cE as we present it here is not incompatible with traditional techniques, and has several practical advantages.


In order to abstract away from approximations made in deriving character-based probabilities p(k|x, h, s) used in the benefit calculation from word-based probabilities, we employed a specialized user model.

to the realistic model described in (Foster et al., 2002b), this assumes that users accept predictions only at the beginnings of words, and only when they are correct in their entirety.


Various morpheme extraction algorithms can be applied to the data.

the main advantage of our framework is that it presents the morphology algorithm of choice with a training set for particular linguistic features.


7 This accuracy compares favorably with the accuracy of 40% obtained for the raw hyponymy extraction experiments in Section 2, suggesting that inferring new relations by using corpus-based similarities to previously known relations is more reliable than trying to learn completely new relations even if they are directly attested in the corpus.

our accuracy falls way short of the figure of 82% reported by Widdows and Dorow (2002).


State-of-theart machine learning techniques including Support Vector Machines (Vapnik, 1995), AdaBoost (Schapire and Singer, 2000) and Maximum Entropy Models (Ratnaparkhi, 1998;Berger et al., 1996) provide high performance classifiers if one has abundant correctly labeled examples.

annotating a large set of examples generally requires a huge amount of human labor and time.


Combining a naive Bayes classifier with the EM algorithm is one of the promising minimally supervised approaches because its computational cost is low (linear to the size of unlabeled data), and it does not require the features to be split into two independent sets unlike cotraining.

the use of unlabeled data via the basic EM algorithm does not always improve classification performance.


We first transform the probability P (c k | x) using Bayes' rule, Class probability P (c k ) can be estimated from training data.

direct estimation of P (c k | x) is impossible in most cases because of the sparseness of training data.


For example, word occurrence is a commonly used feature for text classification.

obvious strong dependencies exist among word occurrences.


As described in the previous section, the naive Bayes classifier can be easily extended to exploit unlabeled data by using the EM algorithm.

the use of unlabeled data for actual tasks exhibits mixed results.


Their results showed that they gained a slight improvement by using a certain amount of unlabeled data.

test set accuracy began to decline as additional data were harvested.


The naive Bayes classifier can be combined with the wellestablished EM algorithm to exploit the unlabeled data .

the use of unlabeled data sometimes causes disastrous degradation of classification performance.


(2003) achieved the highest overall F β=1 rate.

the difference between their performance and that of the Maximum Entropy approach of Chieu and Ng (2003) is not significant.


While in the development set the NER module achieves a very good balance between precision and recall, in the test set the precision drops almost 4 points, being the F 1 results much worse.

development and test sets for German are much more similar.


A simple combination method is the equal voting method (van Halteren et al., 2001;Tjong Kim Sang et al., 2000), where the parameters are computed as λ i (w) = 1 n and P i (C|w, C i ) = δ (C, C i ), where δ is the Kronecker operator (δ (x, y) := (x = y?1 : 0)) -each of the classifiers votes with equal weight for the class that is most likely under its model, and the class receiving the largest number of votes wins.

this procedure may lead to ties, where some classes receive the same number of votes -one usually resorts to randomly selecting one of the tied candidates in this case - Table 2 presents the average results obtained by this method, together with the variance obtained over 30 trials.


In the voting methods, each classifier gave its entire vote to one class -its own output.

equation (2) allows for classifiers to give partial credit to alternative classifications, through the probability P i (C|w, C i ).


We did also try to incorporate gazetteer information by adding ¤ -gram counts from gazetteer entries to the training counts that back the above character emission model.

this reduced performance (by 2.0% with context on).


Typically, efficient inference procedures in both frameworks rely on dynamic programming (e.g., Viterbi), which works well in sequential data.

in many important problems, the structure is more general, resulting in computationally intractable inference.


It also trains the two classifiers separately as the basic approach.

it assumes that the entity classifier knows the correct relation labels, and similarly the relation classifier knows the right entity labels as well.


Therefore, the quality is not always good.

our global inference procedure, LP, takes the natural constraints into account, so it never generates incoherent predictions.


The aim of bootstrapping is to compensate for the paucity of labeled examples.

its potential danger is label 'contamination'namely, wrongly (automatically) labeled examples may misdirect the succeeding iterations.


This is not surprising since a small number of randomly chosen seeds provide much less information (corpus statistics) than high frequency seeds.

spectral's perfor- Medium Low All (spectral) 100 seeds 61.3 47.8 30.6 48.1 300 seeds 64.5 57.6 37.9 53.9 500 seeds 64.9 57.0 37.9 54.3 Figure 4: F-measure results (%) in relation to the choice of input vectors for spectral analysis.


Previous work has reached mixed conclusions; some suggest that combinations of syntactic and lexical features will perform most effectively.

others have shown that simple lexical features perform well on their own.


We used the training and test data divisions that already exist in the SENSEVAL-2 and SENSEVAL-1 data.

the line, hard, serve and interest data do not have a standard division, so we randomly split the instances into test (20%) and training (80%) portions.


If the target word line is used in the plural form, is preceded by a personal pronoun and the word following it is not a preposition, then it is likely that the intended sense is line of text as in the actor forgot his lines or they read their lines slowly.

if the word preceding line is a personal pronoun and the word following it is a preposition, then it is probably used in the product sense, as in, their line of clothes.


Such features are not expected to perform much better than the majority classifier.

when used in combination with other features, they may be useful.


Under some ideal conditions, where the optimal parameters can be identified, this is the highest improvement that can be achieved for the given labeled set.

most of the times, it may not be possible to find these optimal parameter values.


Even though we use a "local versus topical" feature split, which divides the features into two separate views on sense classification, there might be some natural dependencies between the features, since they are extracted from the same context, which may weaken the independence condition, and may sometime make the behavior of co-training similar to a self-training process.

as theoretically shown in (Abney, 2002), and then empirically in (Clark et al., 2003), co-training still works under a weaker independence assumption, and the results we obtain concur with these previous observations.


This occurs because in small and sparse data, direct first order features are seldom observed in both the training and the test data.

the indirect second order co-occurrence relationships that are captured by these methods provide sufficient information for discrimination to proceed.


The function we want to approximate is a mapping f from parser configurations to parser actions, where each action consists of a transition and (unless the transition is Shift or Reduce) a dependency type: Here Config is the set of all possible parser configurations and R is the set of dependency types as before.

in order to make the problem tractable, we try to learn a functionf whose domain is a finite space of parser states, which are abstractions over configurations.


Since parsing is a sentence-level task, we believe that the overall attachment score should be computed as the mean attachment score per sentence, which gives an estimate of the expected attachment score for an arbitrary sentence.

since most previous studies instead use the mean attachment score per word (Eisner, 1996;, we will give this measure as well.


Although some variants to the usual dot-product are sometimes used (for example, higher-order polynomial kernels and RBF kernels), the distribution of examples is not taken into account in such kernels.

new types of kernels have more recently been proposed; they are based on the probability distribution of examples.


So far, we have discussed the computational time for the kernel constructed on the Gaussian mixture.

the computational advantage of the kernel, in fact, is shared by a more general class of models.


The total number of categories is actually 116.

for small categories, reliable statistics cannot be obtained.


Before the first iteration the seed points are initialized to random values.

a bad choice of initial centers can have a great impact on performance, since -means is fully deterministic, given the starting seed points.


This remains to be proven conclusively (Dagan et al., 1999).

similarity-based techniques do not discard any data.


The backing off steps are shown in Figure 1.

if we use the similarity-based language model shown in (2) as a guide, then we can create a smoothed version of Collins' model using the weighted probability of all similar PPs (for brevity we use c in to indicate the context, in this case an entire PP quadruple): to the language model shown in (2), the set of similar contexts S(c) and similarity function α(c, c ) must be defined for multiple words (we abuse our notation slightly by using the same α and S for both PPs and words, but the meaning should be clear from the context).


Part of the problem is that words in the PP are replaced independently and without consideration to the remaining context.

we had hoped the specialist thesaurus might alleviate this problem by providing neighbours that are more appropriate for this specific task.


Our results show that the similarity-based smoothing of frequency estimates significantly improves an already respectable probabilistic PP attachment model.

our hypothesis that a task-specific thesaurus would outperform a generic thesaurus was not borne out by our experiments.


Semantic similarity measures have focused on individual word senses.

in many applications, it may be informative to compare the overall sense distributions for two different contexts.


For both the original and modified , SPD has the same value because we are moving a total probability mass of 0.5 from E and F to B, with the same semantic distance (since E and F are at the same level in the tree).

we consider that, at the node B subtree, .


In the presence of more noise, her method performs quite well in many cases; it is best or tied for best on the development verbs, medium frequency (70%) and on the test verbs, all verbs (67%), high frequency (80%), and medium frequency (80%).

it does not do well on low frequency verbs at all (below chance at 40%).


6 Stemming from this analysis, a possible refinement to separating the frequency bands is to use a different classifier in each frequency band, then combine their performance.

we observe that the best SPD performer in one frequency band tends to be the best performer in other bands (development: SPD without entropy, w h i k j P h ; test: SPD without entropy, w § ¦ © ).


All systems captured some kind of clause information through feature codification.

some of the systems restrict the search for arguments only to the immediate clause Williams et al., 2004) and others use the clause hierarchy to guide the exploration of the sentence (Lim et al., 2004;Carreras et al., 2004).


All studies of semantic role labelling we are aware of have used constituents as instances for classification.

constituents are not available in the shallow syntactic information provided by this task.


In the second change, it is possible to miss some information in cases where the semantic chunks do not align with the sequence of BPs.

in Section 3.2 we show that the loss in performance due to the misalignment is much less than the gain in performance that can be achieved by the change in representation.


(2003), who apply the IOB2 tagging scheme in their word-by-word models, as shown in the second row of Figure 1.

two aspects of the problem at hand make this tag assignment difficult to use for TBL.


Gildea (2002) proposed a probabilistic discriminative model to assign a semantic roles to the constituent.

it needs a complex interpolation for smoothing because of the data sparseness problem.


A semantic role label(r i ) is represented by using a BIO notation such as B-A*, I-A*, etc.

o is too frequently occurred than other semantic role labels, it can have a somewhat high probability than others.


synset codes (Smeaton, 1999;Sussna, 1993;Voorhees, 1993;Voorhees, 1994;Moschitti and Basili, 2004).

even the state-of-the-art methods for WSD did not improve the accuracy because of the inherent noise introduced by the disambiguation mistakes.


In general, the use of 10 disjoint training/testing samples produces a higher variability than the nfold cross validation which insists on the same document set.

this does not affect the t-student confidence test over the differences between the Mi-croAverage of SK and bow since the former has a higher accuracy at 99% confidence level.


A WN-based semantic similarity function between noun pairs is used to improve indexing and document-query matching.

the WSD algorithm had a performance ranging between 60-70%, and this made the overall semantic similarity not effective.


On the same corpus, SCISSOR obtains 91.5% precision and 72.3% recall.


The string-based version of SILT uses no syntactic information while the tree-based version generates a syntactic parse first and then transforms it into an MR.

sCIssOR integrates syntactic and semantic processing, allowing each to constrain and inform the other.


Other measures of word association are possible, such as mutual information (MI), which we can use with the dependency and the adjacency models, similarly to #, χ 2 or Pr.

in our experiments, χ 2 worked better than other methods; this is not surprising, as χ 2 is known to outperform MI as a measure of association (Yang and Pedersen, 1997).


Recent work on the problem of detecting synonymy through corpus analysis has used the Test of English as a Foreign Language (TOEFL) as a benchmark.

this test involves as few as 80 questions, prompting questions regarding the statistical significance of reported results.


It has been shown that measures based on the pointwise mutual information (PMI) between question words yield good results on the TOEFL (Turney, 2001;Terra and Clarke, 2003).

ehlert (2003) shows convincingly that, for a fixed amount of data, the distributional model performs better than what we might call the pointwise co-occurrence model.


If cen i is small, so that the n th i occurrence of the term is near the end of the document, then it is not surprising that w in i +1 is censored.

if cen i is large, so the n th i occurrence is far from the end of the document, then either it is surprising that the term did not re-occur, or it suggests the term is rare.


E.g., labels are not taken into account when measuring the quality of the partition.

in many cases, supervision is used at the application level when determining an appropriate distance metric (e.g., (Lee, 1997;Weeds et al., 2004;Bilenko et al., 2003) and more).


A good metric is one in which close proximity correlates well with the likelihood of being in the same class.

when applying clustering to some task, people typically decide on the clustering quality measure q S (h) they want to optimize, and then chose a specific clustering algorithm A and a distance metric d to generate a 'good' partition function h. it is clear that without any supervision, the resulting function is not guaranteed to agree with the target function p (or one's original intention).


Starting from uniform parameters it climbs from a 40% baseline to a 60% accurate model.

the initializer can do slightly better with precise but sparse gender/number information alone.


We follow IBM Model 1 (Brown et al., 1993) and assume that each word in an utterance is generated by exactly one role in the parallel frame Using standard EM to learn the role to word mapping is only sufficient if one knows to which level in the tree the utterance should be mapped.

because of the vertical ambiguity inherent in intentional actions, we do not know in advance which is the correct utterance-to-level mapping.


Given our model, there exists the possibility of employing intrinsic measures of success, such as word alignment accuracy.

we choose to measure the success of learning by examining the related (and more natural) task of language understanding.


The use of statistical methods in computational linguistics has produced advances in tasks such as parsing, information retrieval, and machine translation.

most of the successful work to date has used supervised learning techniques.


Not surprisingly, the categorical parser does not perform as well as the supervised statistical parser: only 92.7% of German words and 94.9% of English words (85.7% and 86.8%, respectively, of multisyllabic words) are syllabified correctly.

a more important result of parsing the corpus using the categorical parser is that its output can be used to define a model class (i.e., a set of PCFG rules) from which a model can be learned using EM.


This is because the categorical parser produces a wider range of onsets and codas than there are in the true parses.

the induced model is not a superset of the supervised model.


Ultimately, of course, we will want to be able to capture not only the main effects in the data, but some of the subtler effects as well.

we believe that the way to do this is not by introducing completely free parameters, but by using a Bayesian prior that would enforce a degree of similarity between certain parameters.


Supervised training of named entity recognition (NER) systems requires large amounts of manually annotated data.

human annotation is typically costly and time-consuming.


1 These experiments prove the utility of selective sampling and suggest that parameters for a new domain can be optimised in another domain for which annotated data is already available.

there are some provisos for active learning.


What is more, we speculate that annotators might have an even higher error rate on the supposedly more informative, but possibly also more difficult examples.

this would not be reflected in the carefully annotated and verified examples of a gold standard corpus.


So far, the behaviour we have observed is what you would expect from selective sampling; there is a marked improvement in terms of cost and error rate reduction over random sampling.

selective sampling raises questions of cognitive load and the quality of annotation.


This demonstrates clearly that tokens with higher KLdivergence have lower inter-annotator agreement.

as discussed in sections 2.3 and 2.4, we decided on sentences as the preferred annotation level.


The tagger follows the CoNLL-2003 task setting (Tjong Kim Sang and De Meulder, 2003), and thus is not developed with WSJ data.

we allowed its use because there is no available named entity recognizer developed with WSJ data.


Our model aims to capture such dependencies among the labels of nodes in a syntactic parse tree.

building such a model is computationally expensive.


The predicate widen shares the trade gap with expect as a A1 argument.

as expect is a raising verb, widen's subject is not in its typical position either, and we should expect to find it in the same positions as expected's subject.


We should finally specify the distributions of the s i .

we make the simplifying assumption that their distribution is flat (noninformative).


This heuristic works well with the correct parse trees.

one of the errors by automatic parsers is due to incorrect PP attachment leading to missing arguments.


From the initial 4,683,777 nodes (of sections 02-21), the heuristic removed 1,503,100 nodes with a loss of 2.6% of the total arguments.

as we started the experiments in late, we used only the 992,819 nodes from the sections 02-08.


Most current semantic role labeling (SRL) approaches can be classified in one of two classes: approaches that take advantage of complete syntactic analysis of text, pioneered by (Gildea and Jurafsky, 2002), and approaches that use partial syntactic analysis, championed by the previous CoNLL shared task evaluations .

to the authors' knowledge, a clear analysis of the benefits of using full syntactic analysis versus partial analysis is not yet available.


There is a significant gap both between parse-F1-reranked trees and SRL-F1-reranked trees, which shows promise for joint reranking.

the gap between SRL-F1-reranked trees and gold parse trees indicates that reranking of parse lists cannot by itself completely close the gap in SRL performance between gold and predicted parse trees.


(2005), the boundary agreement of Charniak is higher than that of Collins; therefore, we choose the Charniak parser's results.

there are two million nodes on the full parsing trees in the training corpus, which makes the training time of machine learning algorithms extremely long.


A parser therefore is trained on this new corpus and should be able to serve as an SRL system at the same time as predicting a parse.


These new parameters all have the same values as their associated unknown words, so the probability distribution specified by the model does not change.

when a kernel is defined with this reparameterized model, the kernel's feature extractor includes features specific to these words, so the training of a large margin classifier can exploit differences between these words in the target domain.


Once we have the trained parsing model, our proposed porting method proceeds the same way in this scenario as in transferring.

because the original training set already includes the vocabulary from the target domain, the reparameterization approach defined in the preceding section is not necessary so we do not perform it.


For this reason we do not run experiments on the task considered in (Gildea, 2001) and (Roark and Bacchiani, 2003), where they are porting from the restricted domain of the WSJ corpus to the more varied domain of the Brown corpus as a whole.

to help emphasize the success of our proposed porting method, it is relevant to show that even our baseline models are performing better than this previous work on parser portability.


They propose to use the statistics from a source domain to define priors over weights.

in their experiments they used only trivial sub-cases of this approach, namely, count merging and model interpolation.


The EM algorithm is guaranteed to continuously increase the likelihood on the training set until convergence to a local maximum.

the likelihood on unseen data will start decreasing after a number of iterations, due to overfitting.


When we learn a single markovized PCFG from the treebank, that grammar gives a likelihood ratio of only 61.

when we train with a hierarchical model composed of a shared grammar and four individual grammars, we find that the grammar likelihood ratio for these rules goes up to 126, which is very similar to that of the empirical ratio.


Ignoring the non-crossing condition, the constraint set is exact.

because of the non-crossing condition, the constraint set is more restrictive than necessary.


rather than impose the constraints exactly, we enforce them approximately through the introduction of slack variables .


It seems that the global loss model may have been over-regularized (Table 3).

we have picked the t parameter which gave us the best resutls in our experiments.


Finally, we compared our results to the probabilistic parsing approach of (Wang et al., 2005), which on this data obtained accuracies of 0.7631 on the CTB test set and 0.6104 on the development set.

we are using a much simpler feature set here.


However, such statistics are typically gathered on a case-by-case basis, and no reliable procedure exists to automatically identify constructions.

in computational linguistics, many automatic procedures are studied for identifying MWEs (Sag et al., 2002) -with varying success -but here they are treated as exceptions: identifying multi-word expressions is a pre-processing step, where typically adjacent words are grouped together after which the usual procedures for syntactic or semantic analysis can be applied.


For example, "corruption" is refered to as a "tool" in the actual corpus anaphora, a metaphoric usage that would be difficult to predict unless given the usage sentence and its context.

a human agreement of 79% indicate that such instances are relatively rare and the task of predicting a definite anaphor without its context is viable.


In such cases, such as in example 2 from Table 1, answer can be successfully replaced with reply yielding a substitution which conveys the original meaning.

in situations such as in example 1 the word answer is in the sense of a general solution and cannot be replaced with reply.


Furthermore, modeling the a priori substitution likelihood captures the majority of cases in the evaluated setting, mostly because Word-Net provides a rather noisy set of substitution candidates.

successfully incorporating local and global contextual information, as similar to WSD methods, remains a challenging task for future research.


In addition, we can see that the best setting for γ is somewhere around γ = 4, 000.

in this experiment, we could only test up to 1,000 sentences due to the cost of SVM training, which were  where L is the number of training examples, regardless of the use of the speed-up method (Kazama and Torisawa, 2005), we can observe that the WMOLT kernel achieves a high accuracy even when the training data is very small.


Depending on the types of verb classes to be induced, the automatic approaches vary their choice of verbs and classification/clustering algorithm.

another central parameter for the automatic induction of semantic verb classes is the selection of verb features.


The adverbial features outperform the frame-based features in any clustering.

none of the differences between the frame-based clusterings and the grammar-based clusterings are significant (χ 2 , df = 1, α = 0.05).


Especially the all results demonstrate once more the missing correlation between association/feature overlap and clustering results.

it is interesting that the clusterings based on window co-occurrence are not significantly worse (and in some cases even better) than the clusterings based on selected grammar-based functions.


The associations therefore did not help in the specific choice of corpus-based features, as we had hoped.

the assumption that window-based features do contribute to semantic verb classes -this assumption came out of an analysis of the associations -was confirmed: simple window-based features were not significantly worse (and in some cases even better) than selected grammar-based functions.


One problem is that there are so many words and so many senses that it is hard to make available a sufficient number of labeled training examples for each of a large number of target words.

this indicates that the total number of available labeled examples (irrespective of target words) can be relatively large.


Since a "bank" as in "money bank" and a "save" as in "saving money" may occur in similar global contexts, certain global context features effective for recognizing the "money bank" sense may be also effective for disambiguating "save", and vice versa.

with respect to the position-sensitive local context features, these two disambiguation problems may not have much in common since, for instance, we sometimes say "the bank announced", but we rarely say "the save announced".


As in the above example of "bank" (noun) and "save" (verb), the predictive structure of global context features may be shared by the problems irrespective of the parts of speech of the target words.

the other types of features may be more dependent on the target word part of speech.


It may be evident that had we only the sentence Investors suffered heavy losses in our corpus, there would be no difference in probability between the five parse trees in figure 1, and U-DOP would not be able to distinguish between the different trees.

if we have a different sentence where JJ NNS (heavy losses) appears in a different context, e.g.


On the one hand, fitting exactly to the training data may lead to overfitting.

dismissing true properties of the data as sampling bias in the training data will result in low accuracy.


This has the advantage of easing the data-sparsity issues described above because infrequent sequences are clustered into more frequent non-terminal symbols.

in incremental systems, constituents are compared directly, which can lead to a bias towards shorter constituents.


One method for achieving such an internal relationship might be to attach contexts to the expressions with which they co-occur, and we propose using such a method here.

this requires that we have some criterion for deciding when and how expressions should be attached to their contexts.


They claim that local statistics, effectively n-grams, can be sufficient to indicate to the learner which alternative should be preferred.

this argument has been carefully rebutted by (Kam et al., 2005), who show that this argument relies purely on a phonological coincidence in English.


First of all, observe that syntactic congruence is a purely language theoretic notion that makes no reference to the grammatical representation of the language, but only to the set of strings that occur in it.

there is an obvious problem: syntactic congruence tells us something very useful about the language, but all we can observe is weak substitutability.


These models impressively manage to extract significant structure from raw data.

for our purposes, neither of these models is suitable.


In general gazetteers are thought to provide a useful source of external knowledge that is helpful when an entity cannot be identified from knowledge contained solely within the data set used for training.

some research has questioned the usefulness of gazetteers (Krupka and Hausman, 1998).


In each case, the standard+g model outperforms the standard model at a significance level of p ¡ 0¢ 02.

these results camouflage the fact that the gazetteer features introduce some negative effects, which we explore in the next section.


The simplest pruning method is to set a count threshold c below which transitions are removed.


One way to handle this problem is to build a language model of content tokens and retain only the maximum likelihood token sequence.

in the current work, the following heuristic which worked well in practice is used.


The downside of supervised learning is expensive training data.

massive amounts of unlabeled data are readily available.


(2005) use language-specific information (for example, chunks).

the method presented here is language independent.


variations are consistent enough to allow us to draw some general conclusions.


For this shared task we decided to ignore any additional relations.

the data format could easily be extended with additional optional columns in the future.


its average as well as top score far exceed those for Arabic and Turkish, which are larger.


Tests on the PDT (Böhmovà et al., 2003) show that the added actions are sufficient to handle all cases of non-projectivity.

since the cases of non-projectivity are quite rare in the corpus, the general learner is not supplied enough of them to learn how to classify them accurately, hence it may be worthwhile to exploit a second classifier trained specifically in handling nonprojective situations.


In principle, one pass through the dependency array would suffice to parse a sentence.

due to linguistic constraints like uniqueness princi ple, barrier tags and "full" heads 5 , some words may be left unattached or create conflicts for their heads.


7 As a general rule, non-projective arcs were only al lowed if no other, projective head could be found for a given word.

linguistic knowledge suggests that non-projective arcs should be particu larly likely in connection with verb-chain-depen dencies, where subjects attach to the finite verb, but objects to the non-finite verb, which can create crossing arcs in the case of object fronting, chain inversion etc.


We saw the same relation in this evaluation: for Turkish, Arabic, and Slovene, languages with limited number of training sentences, our system obtains accuracies below 70%.

one can not argue that the training size is the only cause of errors: Czech has the largest training set, and our accuracy is also below 70%.


The pure Wait action was suggested in (Yamada and Matsumoto, 2003).

here we come up with these five actions by separating actions Left into (real) Left and WaitLeft, and Right into (real) Right and WaitRight.


The advantage of a pipeline model is that it can use more information that is taken from the outcomes of previous prediction.


It is expected that we selected the FORM and CPOSTAG of each nodes as features in the preceding work.

the POSTAG is also a useful feature for Chinese, and we grouped the original POS tags of Sinica Treebank from 303 to 54 in our preceding work.


For example, there were no features that considered subject-verb agreement nor agreement of an adjective with the number or lexical gender of the noun it modified.

it is possible that morphological information influenced the training of edge weights if the information was implicit in the POS tags.


As with the regular shift-reduce, it uses a stack S and a list of input words W .

instead of finding constituents, it builds a set of arcs G representing the graph of dependencies.


With the availability of resources such as the Penn WSJ Treebank, much of the focus in the parsing community had been on producing syntactic representations based on phrase-structure.

recently their has been a revived interest in parsing models that produce dependency graph representations of sentences, which model words and their arguments through directed edges (Hudson, 1984;Mel čuk, 1988).


Ideally one would like to make all parsing and labeling decisions jointly so that the shared knowledge of both decisions will help resolve any ambiguities.

the parser is fundamentally limited by the scope of local factorizations that make inference tractable.


In our case this means we are forced only to consider features over single edges or pairs of edges.

in a two stage system we can incorporate features over the entire output of the unlabeled parser since that structure is fixed as input.


Table 2 shows that each component of our system does not change performance significantly (rows 2-4 versus row 1).

if we only allow projective parses, do not use morphological features and label edges with a simple atomic classifier, the overall drop in performance becomes significant (row 5 versus row 1).


Another source of potential error is that the average sentence length of Arabic is much higher than other languages (around 37 words/sentence).

if we only look at performance for sentences of length less than 30, the labeled accuracy is still only 71%.


This framework is efficient for both projective and non-projective parsing and provides an online learning algorithm which combined with a rich feature set creates state-of-the-art performance across multiple languages (McDonald and Pereira, 2006).

mcDonald and Pereira (2006) mention the restrictive nature of this parsing algorithm.


Thirdly, inference timed out (Chinese) and fourthly, constraints were not violated that often in the first place (Japanese).

the effect of the first problem might be reduced by training with a higher k. The second problem could partly be overcome by using a better tagger or by a special treatment within the constraint handling for word types which are likely to be mistagged.


At the third phase, the post-processor (which is another learner) recognizes the still un-parsed words.

in this paper, we aim to build a multilingual portable parsing model without employing deep language-specific knowledge, such as lemmatization, morphologic analyzer etc.


We build a classifier, which learns to find root word based on encoding context and children features.

most of the dependency relations were constructed at the first stage.


For example, the error rate of Noun in Turkish is 39% which is the highest error rate.

the head error rates fall in the middle rank for the other two languages.


In Japanese, al-though our parser achieves more than 97% left/right arc rates.

for the root word precision rate is quite lower (85.97).


Barzilay and Elhadad's scoring function aims to identify sentences (for inclusion in a summary) that have a high concentration of chain members.

we are interested in chains that span several sentences.


Our approach to document compression differs from most summarisation work in that our summaries are fairly long.

we believe this is the first step into understanding how compression can help summarisation.


In the example given above, the predicate beat evoques a single frame (i.e., Cause harm).

predicates often have multiple meanings thus evoquing more than one frame.


We can also observe a trend in recent work in textual entailment that more emphasis is put on explicit learning of the syntactic graph mapping between the entailed and entailed-by sentences (Mac-Cartney et al., 2006).

relatively fewer attempts have been made in the QA community.


First, the U-SVM classifies questions into a question-dependent set of clusters, and the answer is the name of a question cluster.

most previous models have classified candidates into positive and negative.


However, most of previous research has focused on using multilingual resources typically used in SMT systems to improve WSD accuracy, e.g., Dagan and Itai (1994), Li and Li (2002), Diab (2004).

this paper focuses on the converse goal of using WSD models to improve actual translation quality.


The purpose of the study was to lower the annotation cost for supervised WSD, as suggested earlier by Resnik and Yarowsky (1999).

this result is also encouraging for the integration of WSD in SMT, since it suggests that accurate WSD can be achieved using training data of the kind needed for SMT.


These properties may be maintained by examining sentences adjacent to each potential insertion point.

a local sentence comparison method such as this may fail to account for global document coherence (e.g.


Currently, our system is trained on insertions in which the sentences of the original text are not modified.

in some cases additional text revisions are required to guarantee coherence of the generated text.


A χ 2 test on overall frequencies of aggregated versus non-aggregated disjunctives showed that the non-aggregated descriptions ('true' partitions) were a significant majority (χ 2 = 83.63, p < .001).

the greater frequency of aggregation in VS compared to VDS turned out to be significant (χ 2 = 15.498, p < .001).


IA part initially performs partitioning based on the basic-level TYPE of objects, in line with the evidence.

later partitions can be induced by other properties, possible yielding partitions even with same-TYPE referents (e.g.


Figure 2 shows that the performance is almost constant on ungrammatical data in the important sentence length range from 5 to 40.

there is a negative correlation of accuracy and sentence length for grammatical sentences.


Adpositions do tend to have a high number of siblings on average, which could explain MSTParser's performance on that category.

adjectives on average occur the furthest away from the root, have the shortest dependency length and the fewest siblings.


It was already known that the two systems make different errors through the work of Sagae and Lavie (2006).

in that work an arc-based voting scheme was used that took only limited account of the properties of the words connected by a dependency arc (more precisely, the overall accuracy of each parser for the part of speech of the dependent).


A naïve method for computing the partition function is therefore to evaluate The above would require calculating n determinants, resulting in O(n 4 ) complexity.

as we show below Z(θ) may be obtained in O(n 3 ) time using a single determinant evaluation.


In response, several researchers have developed algorithms for automatically learning inference rules from textual corpora.

these rules are often either imprecise or underspecified in directionality.


A proper treatment of polysemy is essential in the area of lexical acquisition, since polysemy repre-sents a pervasive phenomenon in natural language.

previous approaches to the automatic acquisition of semantic classes have mostly disregarded the problem (cf.


These results suggest that morphology is the best single source of evidence for our task.

recall from Section 3 that the sampling procedure for the Gold Standard explicitly balanced for morphological factors.


For instance, according to t-tests performed on the Gold Standard (α = 0.05), only three of the 18 semantic features exhibit significant mean differences for classes basic and event.

aNOVa across the 6 classes (α = 0.05) yields significant differences for 16 out of the 18 features, which indicates that most features serve to distinguish object adjectives from basic and event adjectives.


This distributional approach works under the assumption that the context vector of each word encodes sufficient information for enabling accurate word clustering.

many words are distributionally unreliable: due to data sparseness, they occur infrequently and hence their context vectors do not capture reliable statistical information.


add-on's in existing POS induction algorithms, which remain primarily distributional in nature.

our approach more tightly integrates morphology into the distributional framework.


We have also experimented with a very simple method for handling ambiguity in our bootstrapping algorithm: when augmenting the seed set, instead of labeling a word with a tag that receives 9 votes from the 45 pairwise classifiers, we label a word with any tag that receives at least 8 votes, effectively allowing the assignment of more than one label to a word.

our experimental results (not shown due to space limitations) indicate that the incorporation of this method does not yield better overall performance, since many of the additional labels are erroneous and hence their presence deteriorates the quality of the bootstrapped data.


We have proposed a new bootstrapping algorithm for unsupervised POS induction.

to existing algorithms developed for this problem, our algorithm is designed to (1) operate under a resource-scarce setting in which no languagespecific tools or resources are available and (2) more tightly integrate morphological information with the distributional POS induction framework.


A number of (mostly manually curated) databases such as MINT (Zanzoni et al., 2002), BIND , and SwissProt (Bairoch and Apweiler, 2000) have been created to store protein interaction information in structured and standard formats.

the amount of biomedical literature regarding protein interactions is increasing rapidly and it is difficult for interaction database curators to detect and curate protein interaction information manually.


One of them is based on matching pre-specified patterns and rules (Blaschke et al., 1999;Ono et al., 2001).

complex cases that are not covered by the pre-defined patterns and rules cannot be extracted by these methods.


For instance, our example sentence is a positive interaction sentence for the KaiC and SasA protein pair.

it is a negative interaction sentence for the KaiA and SasA protein pair, i.e., it does not describe an interaction between this pair of proteins.


Intuitively, the unlabeled data pushes the decision boundary away from the dense regions.

unlike SVM, the optimization problem now is NP-hard (Zhu, 2005).


(2004) proved that edit kernel is not always positive definite.

it is possible to make the kernel matrix positive definite by adjusting the γ parameter, which is a positive real number.


Supervised machine learning approaches have been applied to this domain.

they rely only on labeled training data, which is difficult to gather.


Al-Onaizan and Knight (2002) report a 1best accuracy of 0.199 on a corpus of Arabic person names (but an accuracy of 0.634 on English names), using a "spelling-based" model, i.e., a model which has no access to phonetic information.

the details of their experiment and model differ from ours in a number of respects.


Similarly, "h" is more common following "ya" in the target string (often as part of the larger suffix "iyah").

the preceding context "rya" is usually observed in the word "qaryat", meaning "village" as in "the village of ..." In this grammatical usage, the tah marbouta is spoken and therefore rendered with a "t".


Thus, Klementiev and Roth, in common with the two MT approaches described above, carefully control the features used by the perceptron.

to these approaches, our algorithm discovers latent alignments, essentially selecting those features necessary for good performance on the task at hand.


State-of-the-art NER systems are characterized by high accuracy, but they require a large amount of training data.

domain specific ontologies generally contains many "fine-grained" categories (e.g., particular categories of people, such as writers, scientists, and so on) and, as a consequence, supervised methods cannot be used because the annotation costs would become prohibitive.


In the first example (see row 1), the word job is strongly associated to the word position, because the contexts of the latter in the examples 1 and 2 are similar to the context of the former, and not to the word task, whose contexts (4, 5 and 6) are radically different.

the second example (see row 2) of the word job is similar to the occurrences 4 and 5 of the word task, allowing its correct substitution.


Capturing non-local dependencies is crucial to the accurate and complete determination of semantic interpretation in the form of predicateargument-modifier structures or deep dependencies.

with few exceptions (Model 3 of Collins, 1999;Schmid, 2006), output trees produced by state-of-the-art broad coverage statistical parsers (Charniak, 2000;Bikel, 2004) are only surface context-free phrase structure trees (CFG-trees) without empty categories and coindexation to represent displaced constituents.


The automatic generation grammar transform presented in (Cahill and van Genabith, 2006) provides a solution to coarse-grained and (in fact) inappropriate independence assumptions in the basic generation model.

there is a sense in which the proposed cure improves on the symptoms, but not the cause of the problem: it weakens independence assumptions by multiplying and hence increasing the specificity of conditioning CFG category labels.


SMT has evolved from the original word-based approach (Brown et al., 1993) into phrase-based approaches (Koehn et al., 2003;Och and Ney, 2004) and syntax-based approaches (Wu, 1997;Alshawi et al., 2000;Yamada and Knignt, 2001;Chiang, 2005).

much important work continues to be carried out in Example-Based Machine Translation (EBMT) (Carl et al., 2005;Way and Gough, 2005), and many existing commercial systems are rule-based.


Since the posterior samples are produced as a byproduct of Gibbs sampling while maximum poste-rior decoding requires an additional time consuming step that does not have much impact on scores, we used the posterior samples to produce the results in Table 1.

to MCMC, Variational Bayesian inference attempts to find the function Q(y, θ, φ) that minimizes an upper bound of the negative log likelihood (Jordan et al., 1999): The upper bound in (3) is called the Variational Free Energy.


Thus VB can be viewed as a more principled version of the wellknown ad hoc technique for approximating Bayesian estimation with EM that involves adding α−1 to the expected counts.

in the ad hoc approach the expected count plus α − 1 may be less than zero, resulting in a value of zero for the corresponding parameter (Johnson et al., 2007;.


To measure the similarity between potential preand post-conjuncts, a lot of work on the coordination disambiguation used the similarity between conjoined heads.

not only the conjoined heads but also other components in conjuncts have some similarity and furthermore structural parallelism.


(2005) enabled the use of non-local features by using Gibbs sampling.

it is unclear how to apply their method of determining the parameters of a non-local model to other types of non-local features, which they did not used.


Krishnan and Manning (2006) divided the model into two CRFs, where the second model uses the output of the first as a kind of non-local information.

it is not possible to use non-local features that depend on the labels of the very candidate to be scored.


Nakagawa and Matsumoto (2006) used a Bolzmann distribution to model the correlation of the POS of words having the same lexical form in a document.

their method can only be applied when there are convenient links such as the same lexical form.


Such methods include ALMA (Gentile, 2001) used in (Daumé III and Marcu, 2005) 2 , MIRA (Crammer et al., 2006) used in (McDonald et al., 2005, and Max-Margin Markov Networks (Taskar et al., 2003).

to the best of our knowledge, there has been no prior work that has applied a perceptron with a margin (Krauth and Mézard, 1987) to structured output.


We use the superscript, l, for local features as Φ l i (x, y) and g for non-local features as Φ g i (x, y).

then, feature mapping is written as Here, we define: Ideally, we want to determine the labels using the whole feature set as: Algorithm 4.1: Candidate algorithm (parameters: if there are non-local features, it is impossible to find the highest scoring candidate efficiently, since we cannot use the Viterbi algorithm.