Basically, instead of updating all λ's simultaneously, we will loop over each indicator function, and compute an update for that indicator function, in turn.

the first change we make is that we exchange the outer loops over training instances and indicator functions.


SCGIS also needs to store the training data, albeit in a different form, but one that is also of size T .

the matrix is interchanged so that its outermost index is over indicator functions, instead of training data.


Notice that for GIS, the top loops end up iterating over all non-zero indicator functions, for each output, for each training instance.

they examine every entry in the training matrix T once, and thus require time T .


The models no longer satisfy the constraints exactly, but work much better on test data.

instead of attempting to maximize the probability of the training data, they maximize a slightly different objective function, the probability of the training data times the prior probability of the model: where In other words, the probability of the λs is a simple normal distribution with 0 mean, and a standard deviation of σ. Chen and Rosenfeld describe a modified update rule in which to find the updates, one solves for δ i in SCGIS can be modified in a similar way to use an update rule in which one solves for δ i in Although sequential updating was described for joint probabilities in the original paper on GIS by Darroch and Ratcliff (1972), GIS with sequential updating for conditional models appears previously unknown.


Instead of treating f # as a constant, we can treat it as a function of x j and y.

let f # (x, y) = i f i (x, y) Then, solve numerically for δ i in the equation Notice that in the special case where f # (x, y) is a constant f # , Equation 4 reduces to Equation 2.


While this work is very preliminary, and the experimental setting somewhat unrealistic (dense features artificially generated), especially for many natural language tasks, the results are dramatic enough to be worth noting.

minka found that a version of conjugate gradient descent worked extremely well -much faster than GIS.


there were many thousands of features in the model (depending on the exact model), but at most 55 could be "true" for a given training or test instance.


"Simple" means that propagations take only polynomial time; the combinatorics is in the distribution steps alone.

it can still happen that a search tree of exponential size has to be explored, but the time spent on propagation in each of its node is only polynomial.


its labels value is In either case, T will require precisely one outgoing substitution edge for each of its substitution nodes, and it will allow arbitrary numbers of outgo-ing adjunction edges for each node where we can adjoin.

the valency value is as follows: is label, i is index of N , and N is pth substitution node for A:i in T } ∪ {adj A,i * | ex.


They can be classified as follows: General feature: • overall sentence length • presence of verb-final and verb-second ancestor nodes • "heaviness" both in number of characters and number of tokens  During testing of the extraposition model, the model was consulted for each extraposable clause to find the highest node to which that clause could be extraposed.

the target node for extraposition is the highest node in the parent chain for which the answer to the classification task "Should the clause move from node X to the parent of node X?"


The precision of the patterns obtained from one QA-pair example in algorithm 1 is calculated from the documents obtained in algorithm 2 for other examples of the same question type.

the precision scores are calculated by cross-checking the patterns across various examples of the same type.


In this case Smolensky's proposal can be unworkable.

§4 will show that a finite-state production grammar in classical OT need not be invertible by any finite-state comprehension grammar.


1) lays out a generalized version of OT's theory of production, introducing some notational and representational conventions that may be useful to others and will be important below.

all objects are represented as strings, or as functions that map strings to strings.


The generating function GEN maps any x ∈ Σ * to the (nonempty) set of candidates y whose underlying form is x.

gEN just inserts Figure 1: This paper's view of OT production.


The above notation is general enough to handle some of the important variations of OT, such as Paradigm Uniformity and Sympathy Theory.

one can define GEN so that each candidate y encodes not just an alignment between x and z, but an alignment among x, z, and some other strings that are neither underlying nor surface.


and you frequently find that mice occupy an important part of the modern medical laboratory.

mice are especially necessary for diagnosing human cancers .


(van Deemter, 2001) generalises the D&R algorithm first, to plural definite descriptions and second, to disjunctive and negative properties as indicated in Figure 2.

the algorithm starts with a distractor set C which initially is equal to the set of individuals present in the context.


make case distinctions) over the cardinality of the output description starting with the lowest possible value.

first the algorithm will try to find a description with cardinality one, then with cardinality two etc.


To take into account disjunctive properties, the constraints used can be modified as indicated in Figure 8.

the algorithm looks for a tuple of sets such that their union  is a basic distinguishing description for E 9 É  where each ¹ is a conjunctive description.


Generation ends when the resulting tree (i) satisfies all communicative goals and (ii) is syntactically complete.

the goal of describing some discourse old entity using a definite description is satisfied as soon as the given information (i.e.


In practice, these goals contribute to updating a "goal semantics" which the generator seeks to realise by building a phrase structure tree that (i) realises that goal semantics, (ii) is syntactically complete and (iii) is pragmatically appropriate.

if an entity must be described which is discourse old, a DD will be computed for that entity and added to the current goal semantics thereby driving further generation.


Aone and Bennett (1995), McCarthy and Lehnert (1995)).

a pair of NPs is classified as co-referring or not based on constraints that are learned from an annotated corpus.


Exact string match is likely to be a better coreference predictor for proper names than it is for pronouns, for example.

we replace the SOON STR feature with three features -PRO STR, PN STR, and WORDS STR -which restrict the application of string matching to pronouns, proper names, and non-pronominal NPs, respectively.


Our results provide direct evidence for the claim (Mitkov, 1997) that the extra-linguistic strategies employed to combine the available linguistic knowledge sources play an important role in computational approaches to coreference resolution.

our results suggest that additional performance gains might be obtained by further investigating the interaction between training instance selection, feature selection, and the coreference clustering algorithm.


The features were not derived empirically from the corpus, but were based on common-sense knowledge and linguistic intuitions regarding coreference.

we increase the number of lexical features to nine to allow more complex NP string matching operations.


(See the indented All Features results in Table 2.

7 ) the classifiers acquire a number of lowprecision rules for common noun resolution, presumably because the current feature set is insufficient.


our tree makes use of many of the features that are not present in the original Soon feature set.


General CFGs have unfavourable computational properties with respect to intersection.

the problem of deciding whether the intersection of two CFGs is non-empty is undecidable (Harrison, 1978).


Our efforts are devoted to two aspects: first, we believe that the selected samples should reflect the underlying distribution of the training corpus.

the selected samples need to be representative.


We further define the distance between two parsing actions as follows: it is either  is the Hamming distance, and Computing the editing distance (3) requires dynamic programming and it is computationally extensive.

to speed up computation, we can choose to ignore the difference in contexts, or in other words, (4) becomes  the sample density is defined as the inverse of its average distance to other samples.


We can turn what at first seems to be distributional clustering into tree induction by confining P(B) to put mass only on tree-equivalent bracketings.

consider P bin (B) which is uniform over binary bracketings and zero elsewhere.


Our results show that the combined criterion consistently leads to smaller models than the models pruned using either of the criteria separately.

the combination of rank and entropy achieves the smallest models at a given CER.


These measures are based on the probability distribution over the whole vocabulary.

if the test text is w 1 n , then perplexity is based on the values of P(w i |w i-1 ), and the new measures will be based on the values of P(w|w i-1 ) for all w in the vocabulary.


Stolcke used the Kullback-Leibler divergence between the pruned and un-pruned model probability distribution in a given context over the entire vocabulary.

the increase in relative entropy from pruning a bigram is computed by where the summation is over all word pairs (w i-1 w i ).


The back-off bigram models we generated in this study are character-based models.

the training and test corpora are not word-segmented.


We can see that the percentage of the number of bigrams pruned by both criteria seems to increase as the model size decreases, but all criterion-pairs have overlaps much lower than 100%.

we find that the average overlap between probability and entropy is approximately 71%, which is the biggest among the three pairs.


As we expected, in all three cases, using a combination of two pruning criteria achieves consistently better performance than using either of the criteria separately.

using the combination of rank and entropy, we obtained the best models over a wide large of CER values.


Results show that using two pruning criteria simultaneously achieves better bigram models than using either of the criteria separately.

the combination of rank and entropy achieves the smallest bigram models at the same CER.


In constructing the ACM, we used asymmetric clustering, in which different clusters are used for predicted and conditional words.

for clustering conditional words, we try to minimize the perplexity of training data for a bigram of the form where N is the total number of words in the training data.


Since the tree is not balanced, the actual number of clusters may be somewhat smaller.

we use w l to represent the cluster of a word w using a tree cut at level l. if we set l to the value "all", it means that the tree is cut at infinite depth, i.e.


the predictive cluster model P(w i |w i-2 w i-1 W i )× P(W i |w i-2 w i-1 ) has the best performance when the cluster W i is the predictive cluster PW i generated by using the metric of Equation (4).

the IBM model achieved the best results when predicted and conditional clusters were used for predicted and conditional words respectively.


In particular, the IBM model achieved the best results when predicted and conditional clusters were used for predicted and conditional words respectively.

the IBM model is of the form P(w i |PW i )× P(PW i |CW i-2 CW i-1 ).


Rather than plot all points of all models together, we show only the outer envelope of the points.

if for a given model type and a given point there is some other point of the same type with both lower perplexity and smaller size than the first point, then we do not plot the first, worse point.


We see that not only is the optimal setting for k not 0, but also typically the exact opposite is the optimal: k=all in which case , or k=14, 16, which is very similar.

we see that words depend on the previous words and that an independence assumption is a poor one.


Then when two most recent exposed heads are adjoined together under some enriched NT label, the NT label has to match the parent NT information carried in both of the exposed heads.

if the two exposed heads bear different information about their parents, they can never be adjoined.


Although the N-best lists are already highly restricted by the 3-gram model during the first recognition pass, this fact still shows the potential of a good grammar-based language model.

we should notice that the SLM was trained on 20M words of WSJ while the lattice 3gram was trained on 45M words of WSJ.


These results are interesting in their own right, and may have practical implications as well.

they suggest that language modeling may be a fruitful way to approach issues of contextual influence in text.


Among the children of H01 we find Chemistry (parent of Biochemistry), Electronics (parent of Amplifiers and Robotics), Mathematics (Fractals, Game Theory and Fourier Analysis).

we find a wide range of concepts that are not described by a simple relationship.


The relationships are different depending on whether the person is a patient, a physician or a donor.

we therefore descend the M01 sub-hierarchy, obtaining the following clusters of NCs: to correctly assign a relationship to these NCs, we needed to descend one level for the second word.


To assess the degree of this generalization we calculated how many CPs are accounted for by the classification rules created above for the Anatomy categories.

if we know that A01 A07 unequivocally determines a relationship, how many possible (i.e., present in our collection) CPs are there that are "covered by" A01 A07 and that we do not need to consider explicitly?


Ideally, however, we would like to reduce the lexical ambiguity for similar senses and to retain it when the senses are semantically distinct (like, for example, for diet in case 4).

ideally, the ambiguity left at the levels of our rules accounts for only (and for all) the semantically different senses.


However, instead of retreating to unsupervised estimation techniques or creating small LFG treebanks by hand, we use the labeled bracketing of the WSJ training sections to guide discriminative estimation.

discriminative criteria are defined with respect to the set of parses consistent with the WSJ annotations.


If y is a string, then let Ω(y) = {ω ∈ Ω|Y (ω) = y} and F(y) = ω∈Ω(y) {f ∈ ω}.

ω(y) is the set of parses of a string y and F(y) is the set of features appearing in the parses of y.


We require that each parse be identified by a unique value satisfying the no-goods.

we require that: Finally, a packed representation R represents the set of parses Ω(R) that are identified by values that satisfy the no-goods, i.e., Ω(R) = {ω(x)|x ∈ X , N (x) = 1}.


However, the dynamic programming algorithms presented here require the information encoded in properties to be local with respect to the features F used in the packed parse representation.

we require that properties be defined on features rather than parses, i.e., each feature f ∈ F is associated with a finite vector of real values (g 1 (f ), .


, H n+1 , where H j is the subset of H that depend on X j but do not depend on any variables ordered before X j , and H n+1 is the subset of H that do not depend on any variables at all (i.e., they are constants).

3 H j = {H ∈ H|X j ∈ d(H), ∀i < j X i ∈ d(H)} and H n+1 = {H ∈ H|d(H) = ∅}.


The high correlation coefficient of 0.99 indicates that BLEU tracks human judgment well.

interesting is how well BLEU distinguishes between S2 and S3 which are quite close.


The second implementation is different from the first one only in (1).

it employs as a classifier a decision list instead of an ensemble of NBCs.


This process is repeated either for a given number of iterations I or until all the unlabeled data has been labeled.

the definition of the two data views appears to be a crucial factor which can strongly influence the behaviour of Co-Training.


The following theorem is based on (Dasgupta et al., 2001).

theorem 4 If view independence is satisfied, and if F and G are rules based on different views, then one of the following holds: for a given binary rule F , a pessimistic estimate of the number of errors made by F is δ/(µ − δ) times the number of instances labeled by F , plus the number of instances left unlabeled by F .


Defining n = ∆A + ∆B, the precision of the rule is q = ∆A/n.

the precision of the new classifier is Q t+1 = (A + ∆A)/N t+1 , which can be written as: the precision of the new classifier is a weighted average of the precision of the old classifier and the precision of the new rule.


In other respects though, our approach is more ambitious because it focuses on the problem of recognizing such discourse relations in unrestricted texts.

given as input sentence pairs such as those shown in (1)-(2), we develop techniques and programs that label the relations that hold between these sentence pairs as CONTRAST, CAUSE-EXPLANATION-EVIDENCE, CONDITION, ELABO-RATION or NONE-OF-THE-ABOVE, even when the discourse relations are not explicitly signalled by discourse markers.


For example, we consider that a CONTRAST relation holds between two text spans if one of the following relations holds: CONTRAST, ANTITHESIS, CONCESSION, or OTH-ERWISE, as defined by Mann and Thompson (1988), CONTRAST or VIOLATED EXPECTATION, as defined by Hobbs (1990), or any of the relations characterized by this regular expression of cognitive primitives, as defined by Knott and Sanders (1998) -NEGATIVE.

in our approach, we do not distinguish between contrasts of semantic and pragmatic nature, contrasts specific to violated expectations, etc.


Since RST (Mann and Thompson, 1988) employs a finer grained taxonomy of relations than we used, we applied the definitions shown in Table 1.

we considered that a CONTRAST relation held between two text spans if a human annotator labeled the relation between those spans as ANTITHESIS, CONCESSION, OTHERWISE or CONTRAST.


A metric called Feature Usage Frequency indicates which features are the most discriminatory in the CART tree.

feature Usage frequency counts how often a feature is queried for each data point, normalized so that the sum of feature Usage frequency values for all the features sums to one.


If there have been more than three acknowledgments of bookings, then several legs of a journey have been successfully booked, therefore User Satisfaction is high.

user Satisfaction is high if the system has asked if the user would like a price for their itinerary which is one of the final dialogue acts a system does before the task is completed.


We demonstrate a problem faced by the Competitive Linking algorithm and present an approach to ameliorate it.

we rebuild the bilingual dictionary by clustering similar words in a language and assigning them a higher cooccurrence score with a given word in the other language than each single word would have otherwise.


between oil and et over a link between oil and pétrole.

word variations can be due to inflectional morphology (e.g.


Furthermore, we have decided to exclude words from clustering that account for only few of the cooccurrences of ¢ ¡ .

a separate threshold,  Similarly to the most frequent words, dictionary scores for word pairs that are too rare for clustering remain unchanged.


We wanted to check performance variation based on different types of TREC-2001 questions.

we have separated definition and generic questions from true named entities questions.


We compare the algorithm's performance on this parsing task with varying training set sizes (yielding learning curves) and different input representations.

we compare input consisting of words only, a variant that includes word form information for lowfrequency words, gold-standard POS only, and combinations of these.


We present an architecture for the integration of shallow and deep NLP components which is aimed at flexible combination of different language technologies for a range of practical current and future applications.

we describe the integration of a high-level HPSG parsing system with different high-performance shallow components, ranging from named entity recognition to chunk parsing and shallow clause recognition.


The job of the source model is to assign a score ( % ) 0 ' @ 4 t o a compression independent of the original document.

the source model should measure how good English a summary is (independent of whether it is a good compression or not).


The paper proposes and empirically motivates an integration of supervised learning with unsupervised learning to deal with human biases in summarization.

we explore the use of probabilistic decision tree within the clustering framework to account for the variation as well as regularity in human created summaries.


Find-Diversity: find clusters of lexically similar sentences in text.

(we represent a sentence here a vector of tfidf weights of index terms it contains.)


The rest of the features distinguish types of capitalization and all other words such as punctuation marks.

the FirstWord feature arises from the fact that if a word is capitalized and is the first word of the sentence, we have no good information as to why it is capitalized (but note that AllCaps and CapPeriod are computed before FirstWord, and take precedence.)


We use Japanese word segmentation as a test case.

we discuss how the size of a pool affects the learning curve.


One reason for this could be that support vectors in selected examples at each iteration from a larger pool make larger clusters than those selected from a smaller pool do.

in the case of a larger pool, more examples selected at each iteration would be similar to each other.


With these modifications, we can use all of the standard HMM algorithms.

we can use this as the basis of a parameter estimation algorithm using the expectation-maximization theorem.


However in other cases, where the morphology is very irregular the performance will be poor.

with the Arabic data sets, the NAKISA data set is very small compared to the complexity of the process being learned, and the MCCARTHY data set is rather noisy, with a large number of erroneous transcriptions.


This paper defines HDAG as a Directed Acyclic Graph (DAG) with hierarchical structures.

certain nodes contain DAGs within themselves.


Although the precision for 1996-2001 was higher than that for 1989-1996, it is still too low to use them as NLP resources.

the article alignments included many incorrect alignments.


This section reveals that not only rank order but also absolute values of AVSIM are reliable for discriminating between correct and incorrect alignments.

they correspond well with human evaluations.


This indicates that AVSIM can discriminate between differences in judgment.

the AVSIM values correspond well with human judgment.


They revealed that DTL is superior to MT at high-recall levels.

if we want to obtain many article alignments, then DTL is more appropriate than MT.


In this paper, we concentrate on the usage of shallow information to increase the efficiency, and potentially the quality, of HPSG parsing.

we want to use analyses delivered by an efficient shallow parser to pre-structure the search space of HPSG parsing, thereby enhancing its efficiency, and guiding deep parsing towards a best-first analysis suggested by shallow analysis constraints.


During resolution, we filter the candidates for each encountered pronoun in the same way as during training.

we only consider the NPs in the current and the preceding 2 sentences.


Besides, compared with Strube's S-list algorithm, our approach also achieves gains in the F-measure by 3.2 (MUC-6), and 1.6 (MUC-7).

our approach obtains significant improvement (21.1 for MUC-6, and 13.1 for MUC-7) over Connolly et al.


We introduced how to adopt the twincandidate model in our competition learning ap-proach to resolve the coreference problem.

we proposed a candidate filtering algorithm that can effectively reduce the computational cost and data noises.


It would be impractical to move to a context-free technology in these areas; at the same time it is desirable to include an account of word formation in these tasks.

it is important to be able to break down complex compounds into the individual components, in order to reach an effect similar to the way compounds are treated in English orthography.


We conducted a study to determine how political speeches have been tailored over the course of U.S. history throughout changing climates of military action.

we wondered if politicians were more likely to talk about goals having to do with conflict, competition, and aggression during wartime than in peacetime.


The features of an example are derived from the treebank-style parse tree output by an implementation of Collins' parser (Collins, 1999).

we traverse the full parse tree to determine the verbs, agents, patients, and indirect objects related to a noun phrase candidate ¥ ¡ .


While a machine learning approach is used in (Gildea and Jurafsky, 2000) to determine general semantic roles, we used a simple rule-based traversal of the parse tree instead, which could also reliably determine the generic agent and patient role of a sentence, and this suffices for our current purpose.

for a given noun phrase candidate ¥ ¡ , the following groups of features are used: Verb of Agent NP (VAg) When ¢ ¡ is an agent in a sentence, each of its associated verbs is a VAg feature.


As the field of IE has developed, the focus of study has moved towards automatic knowledge acquisition for information extraction, including domain-specific lexicons (Riloff, 1993;Riloff and Jones, 1999) and extraction patterns (Riloff, 1996;Yangarber et al., 2000;Sudo et al., 2001).

methods have recently emerged for the acquisition of event extraction patterns without corpus annotation in view of the cost of manual labor for annotation.


There are several ways in which our pattern model may be further improved.

we would like to relax the restraint that all the fills must be tagged with their proper NE tags by introducing a GENERIC place-holder into the extraction patterns.


Unlike previous studies that focus on user's knowledge or typical kinds of users, the user model we propose is more comprehensive.

we set up three dimensions of user models: skill level to the system, knowledge level on the target domain and the degree of hastiness.


The current state of the art in NLG can be described as small pipelined generation systems that incorporate some, but not all, of the available pipelined NLG modules.

there is no system todate which both revises its output and inserts appropriate discourse markers.


For instance, in our corpora we found that these interference effects occurred 23% of the time for revised clauses and 56% of the time with discourse markers.

almost one of every four clause revisions potentially forces a change in discourse marker lexicalizations and one in every two discourse markers occur near a clause revision boundary.


Therefore, in addition to the dictionary, many systems also contain special components for unknown word identification.

statistical methods have been widely applied because they utilize a probabilistic or cost-based scoring mechanism, instead of the dictionary, to segment the text.


The basic idea behind the metric is that a Chinese word should appear as a stable sequence in the corpus.

the components within the word are strongly correlated, while the components at both ends should have low correlations with words outside the sequence.


Then, given a FN candidate, three named entity candidates, each for one category, are generated in the lattice, with the class probabilities |FN).

we delay the determination of its type until decoding where the context model is used.


Therefore, we annotate the text corpus based on the lexicon.

we segment each sentence as much as possible into words that are stored in our lexicon, and tag only the new words, which otherwise would be segmented into strings of one -character words.


We also did not differentiate LNs and ONs in evaluation.

we only checked the word boundaries of LNs and ONs and treated both tags exchangeable.


Linguistic count data often violate the simplistic assumptions of standard probability models like the binomial or Poisson distribution.

the inadequacy of the Poisson distribution for modeling word (token) frequency is well known, and robust alternatives have been proposed (Mosteller and Wallace, 1984;Church and Gale, 1995).


This last model has relaxed the relationship between the probability of the outcome 0 and the probabilities of the other outcomes.

we obtain appropriate counts for the outcome 1 by pretending that the outcome 0 occurs only about 71 times, compared with an actual 405 observed occurrences.


Our evaluation metric is human assessment: Can the translation provided by the system be part of an acceptable translation of the whole sentence?

the noun phrase has to be translated correctly given the sentence context.


This decision is based on the frequency of the words involved.

we pick the splitting option with highest geometric mean of word frequencies of its The German side of both the training and testing corpus is broken up in this way.


Instead of making an assumption on the features, bilingual bootstrapping makes an assumption on the classes.

it assumes that the classes of the classifiers in bootstrapping do not overlap.


In the algorithm, the collaboration between the classifiers is driven by uncertainty reduction.

one classifier always selects the most uncertain unlabelled instances for it and asks the other classifier to label.


Although the parsing-based NE learner is found to suffer from the recall problem, we can apply the learned rules to a huge parsed corpus.

the availability of an almost unlimited raw corpus compensates for the modest recall.


• We prove that {a k bc | 0 ≤ k ≤ n} ⊆ L(G n ) using the following properties: we get: • The condition A B is crucial for strictness of language inclusion.

: • This construction is in some sense more complex than those for the other systems (Foret and Le Nir, 2002a;Foret and Le Nir, 2002b) since they do not directly translate as limit points in the more restricted system NL.


However, in that model, only counts from the same type of construction could be used, whereas in our model, the functor probability for a verb in a zero or reduced relative clause can be estimated from all occurrences of the head noun.

all instances of the noun and verb occurring together in the training data (with the same predicate-argument relation between them, but not necessarily with the same surface configuration) are taken into account by the new model.


The experiments presented here demonstrate that the performance of a simple baseline model can be improved significantly if long-range dependencies are also captured.

our results indicate that it is important not to restrict the model to local dependencies.


The subtrees are then combined by a simple unification process and syntactic features are percolated down.

movement chains are represented as traditional "slash" features which are passed (locally) from parent to daughter; when unification is complete, there is a trace at the bottom of each slash-feature path.


The results from the TLA variants operating in the LDD are presented in Table 3.

99%  Average  TLA-SVC  67,896  11,273  TLA-Greed  19,181  4,110  TLA  16,990  961  Table 3: TLA variants, # of sentences consumed interesting is that contrary to results reported by Niyogi & Berwick (1996) and Sakas & Nishimoto (2002), the SVC and Greediness constraints do help the learner achieve the target in the LDD.


We describe several simple, linguistically motivated annotations which do much to close the gap between a vanilla PCFG and state-of-the-art lexicalized models.

we construct an unlexicalized PCFG which outperforms the lexicalized PCFGs of Magerman (1995) and Collins (1996) (though not more recent models, such as Charniak (1997) or Collins (1999)).


One successful method of combating sparsity is to markovize the rules (Collins, 1999).

we follow that work in markovizing out from the head child, despite the grammar being unlexicalized, because this seems the best way to capture the traditional linguistic insight that phrases are organized around a head (Radford, 1988).


Since we expressly do not smooth the grammar, not all splits are guaranteed to be beneficial, and not all sets of useful splits are guaranteed to co-exist well.

while v = 3, h ≤ 2 markovization is good on its own, it has a large number of states and does not tolerate further splitting well.


To allow the finite/non-finite distinction, and other verb type distinctions, SPLIT-VP annotated all VP nodes with their head tag, merging all finite forms to a single tag VBF.

this also accomplished Charniak's gerund-VP marking.


In this paper, we explore a novel approach for finding long-distance dependencies.

we detect such dependencies, or discontinuities, in a two-step process: (i) a conceptually simple shallow tagger looks for sites of discontinuties as a preprocessing step, before parsing; (ii) the parser then finds the dependent constituent (antecedent).


without using phrase structure information.

we develop a finite-state tagger which inserts EEs at the appropriate sites.


Subjective Quality A total of 510 sentences from the test corpus were evaluated by paired comparison.

the source sentences were translated using the base rule set, and the same sources were translated using the rules after the cleaning.


Therefore, if we regard the bilingual corpus of the new domain as the evaluation corpus and carry out feedback cleaning, the rule set will be adapted to the new domain.

our method can be applied to adaptation of an MT system by using a smaller corpus of the new domain.


In this paper, we used the WSD program reported in (Lee and Ng, 2002).

our method made use of the knowledge sources of part-of-speech, surrounding words, and local collocations.


In addition, the difference between the accuracy figures of M3 and P2 averaged over the set of all difficult nouns is 0.065.

eliminating the advantage that manually sense-tagged data have in their sense coverage would reduce the performance gap between the two approaches from 0.140 to 0.065.


In order to get an idea of the intrinsic difficulty of the countability learning task, we tested the agreement between the two resources in the form of classification accuracy.

we calculate the average proportion of (both positive and negative) countability classifications over which the two methods agree.


In order to discourage the classifiers from overtraining on negative evidence, we constructed the gold-standard training data from unambiguously negative exemplars and potentially ambiguous positive exemplars.

we would like classifiers to judge a target noun as not belonging to a given countability class only in the absence of positive evidence for that class.


The final make-up of the gold-standard data for each of the countability classes is listed in Table 2, along with a baseline classification accuracy for each class ("Baseline"), based on the relative frequency of the majority class (positive or negative).

for bipartite nouns, we achieve a 99.4% classification accuracy by arbitrarily classifying every training instance as negative.


We restricted these selections and settings within reasonable bounds to keep our experiments computationally feasible.

feature selection was limited to varying the size of the window that was used to model the local context of an instance.


However, these methods are not appropriate for chunking Korean and Japanese, because such languages have a characteristic of partially free wordorder.

there is a very weak positional constraint in these languages.


The reason why the performance on Korean is lower than that on English is the curse of dimensionality.

the wider context is required to compensate for the free order of Korean, but it hurts the performance (Cherkassky and Mulier, 1998).


Since the memory-based learning is an efficient method to handle exceptional cases of the rules, it supports the rules by making decisions only for the exceptions of the rules.

the memorybased learning enhances the rules by efficiently handling the exceptional cases of the rules.


However, for each word in a given sentence, only a small part of features in the space are related to the decision on supertag.

the supertag of a word is determined by the appearances of certain words, POS tags, or supertags in its context.


The fact that PVs are separable hurts recall.

for Type II, a Noun Phrase (NP) object can be inserted inside the compound verb.


This backoff scheme recursively estimates the probability of an unseen n-gram by utilizing (n-1)-gram estimates.

the probability of Equation (11) backs off to the estimate of P(w j |R), which is computed as: where N is the total number of dependencies in training data, and C(w j , R) is the number of dependencies that contains w j .


The optimal value of N is 3 in our experiments.

given a headword trigram (h 1 , h 2 , h 3 ), there are 3 initial dependencies: d 12 , d 13 , and d 23 .


There are many possibilities for future improvements.

as discussed in Section 6, syntactic dependency structure is believed to capture useful information for informed language modeling, yet further improvements may be possible by incorporating non-syntax-based dependencies.


Other inputs indicate that the last utterance is still pending, and allow the agent to wait further.

task attention over an interval following the utterance triggers grounding.


In a multimodal conversation, the way a user interacts with a system is dependent not only on the available input channels (e.g., speech and gesture), but also upon his/her conversation goals, the state of the conversation, and the multimedia feedback from the system.

there is a rich context that involves dependencies from many different aspects established during the interaction.


The research reported here uses a methodological approach which is quite common in Artificial Life research but still relatively novel in (computational) linguistics: Rather than attempting to develop simulations that generate natural phenomena directly, as one does when using Newton's equations to simulate the trajectory of a ball falling from a tower, we engage in computational simulations and robotic experiments that create (new) artificial phenomena that have some of the characteristics of natural phenomena and hence are seen as explaining them.

we implement artificial agents with components modeling certain cognitive operations (such as introducing a new syntactic category, computing an analogy between two events, etc.


Mathematical language, often semi-formal, is interleaved with natural language informally verbalizing proof steps.

mathematical expressions (or parts thereof) may lie within the scope of quantifiers or negation expressed in natural language: For parsing, this means that the mathematical content has to be identified before it is interpreted within the utterance.


The relationship between reparandum and repair seems to be quite different: the repair is a "rough copy" of the reparandum, often incorporating the same or very similar words in roughly the same word order.

they seem to involve "crossed" dependencies between the reparandum and the repair, shown in Figure 1.


However, it is easy to modify the TAG so it does include a bigram model that does generate the source words, since each nonterminal encodes the preceding source word.

we multiply the weights of each TAG production given earlier that introduces a source word R i by P n (R i |R i−1 ).


Since the immediate goal of this work is to produce a program that identifies the words of a sentence that belong to the reparandum of a repair construction (to a first approximation these words can be ignored in later processing), our evaluation focuses on the model's performance in recovering the words in a reparandum.

the model is used to classify each word in the sentence as belonging to a reparandum or not, and all other additional structure produced by the model is ignored.


There is no guarantee that parses favored by the second stage will be generated by the first stage.

because the first stage model prunes the space of parses from which the second stage rescores, the first stage model may remove solutions that the second stage would have assigned a high probability.


In the next section of the paper we describe how global linear models can be applied to speech recognition.

we focus on how the decoding and parameter estimation problems can be implemented over lattices using finite-state techniques.


We pruned this feature set to include all unigrams and bigrams, but only those trigrams with an expected count of greater than 0.01 in the training set.

to be included, a  trigram must occur in a set of paths, the sum of the conditional probabilities of which must be greater than our threshold θ = 0.01.


The starting point is to assume that we have a training corpus of utterances labeled with the appropriate unit sequences.

for each training utterance, we assume available a sequence of feature vectors f = f 1 .


The basis for this stems from the hypothesis that locations populated with more restaurants are likely to be queried.

the statistics of the database instances can directly reflect on the distribution of user queries.


In general we would expect that estimating the joint probability would be harder than estimating the conditional probability, because the joint probability contains more information than the conditional probability.

the probability distribution over sentences can be derived from the joint probability distribution, but not from the conditional one.


We will now give a first theorem regarding the convergence of this algorithm.

first, we need the following definition: GEN(x i ) is the set of incorrect candidates for an example x i .


Convolution kernels allow us to treat structural features without explicitly representing the feature vectors from the input object.

convolution kernels are well suited to NLP tasks in terms of both accuracy and concept.


Unfortunately, experiments have shown that in some cases there is a critical issue with convolution kernels, especially in NLP tasks (Collins and Duffy, 2001;Cancedda et al., 2003;Suzuki et al., 2003b).

the over-fitting problem arises if large "substructures" are used in the kernel calculations.


In some cases we already know that larger sub-structures are significant features as regards solving the target problem.

these significant larger sub-structures,  which the conventional methods cannot deal with efficiently, should have a possibility of improving the performance furthermore.


The PrefixSpan algorithm enumerates all the significant sub-sequences by using a depth-first search and constructing a TRIE structure to store the significant sequences of internal results efficiently.

prefixSpan algorithm evaluates uw, where uw represents a concatenation of a sequence u and a symbol w, using the following three conditions.


Moreover, the experiments confirmed one important fact.

in some cases maximum performance was achieved with n = ∞.


Actually, to determine the local focus, we only need to check the entities in a short discourse segment.

for a candidate, the number of its adjacent antecedents to be checked is limited.


candi SubjNP rank top in the decision tree.

for the reference determination, the subject roles of the candidate's referent within a discourse segment will be checked in the first place.


This should be owed to the fact that for a candidate, its adjacent antecedents, even not the closest one, could give clues to reflect its salience in the local discourse.

the model prefers a high precision to a high recall, which copes well with the capability of the existing non-pronoun resolution module.


The differences between WN distance and Google distance, and that between our own MLP and the Weka implementation of Naive Bayes, are also significant (by a sign test, p ≤ .05), whereas the pairwise differences between our own MLP and Weka's NN, and between this and the Naive Bayes classifier, aren't.

although we find little difference between using WordNet and Google to compute lexical distance, using WordNet leads to slightly better results for BDs.


This paper examines several key issues in computing and using anaphoricity information to improve learning-based coreference systems.

we present a new corpus-based approach to anaphoricity determination.


Our goal in this paper is to improve learningbased coreference systems using automatically computed anaphoricity information.

we examine two important, yet largely unexplored, issues in anaphoricity determination for coreference resolution: representation and optimization.


(2002) and Ng and Cardie (2002a)), leaving the remaining three possibilities largely unexplored.

to our knowledge, there have been no attempts to (1) globally optimize an anaphoricity determination procedure for coreference performance and (2) incorporate anaphoricity into coreference systems as a feature.


Consequently, as part of our investigation, we propose a new corpus-based method for achieving global optimization and experiment with representing anaphoricity as a feature in the coreference system.

we systematically evaluate all four combinations of local vs. global optimization and constraint-based vs. feature-based representation of anaphoricity information in terms of their effectiveness in improving a learning-based coreference system.


The classification associated with a training instance -one of ANAPHORIC or NOT ANAPHORIC -is derived from coreference chains in the training documents.

a positive instance is created for each NP that is involved in a coreference chain but is not the head of the chain.


To achieve global optimization, we construct a parametric anaphoricity model with which we optimize the parameter 1 for coreference accuracy on heldout development data.

we tighten the connection between anaphoricity determination and coreference resolution by using the parameter to generate a set of anaphoricity models from which we select the one that yields the best coreference performance on held-out data.


cr := cost of misclassifying a positive instance cost of misclassifying a negative instance Inspection of this definition shows that cr provides a means of adjusting the relative misclassification penalties placed on training instances of different classes.

the larger cr is, the more conservative the classifier is in classifying an instance as negative (i.e., non-anaphoric).


Based on this definition of conservativeness, we can construct an anaphoricity model parameterized by cr.

the parametric model maps a given value of cr to the anaphoricity classifier trained with this cost ratio.


We can also define conservativeness in terms of the number of NPs classified as non-anaphoric for a given set of NPs.

given two anaphoricity models A and B and a set of instances I to be classified, we say that A is more conservative than B in determining an NP as non-anaphoric if A classifies fewer instances in I as non-anaphoric than B.


The reason is that we have no control over whether or how the anaphoricity feature is used by the coreference learner.

the behavior of the coreference system is less predictable in comparison to a constraint-based representation.


Moreover, CBLO does not seem to be very effective in improving the baseline, in part due to the dramatic loss in recall.

although we see improvements in F-measure in five of the 12 experiments in this group, only one of them is statistically significant.


The experimental setting employed here is essentially the same as that in CBLO, except that anaphoricity information is incorporated into the coreference system as a feature rather than as constraints.

each training/test coreference instance i (N P i ,N P j ) (created from NP j and a preceding NP NP i ) is augmented with a feature whose value is the anaphoricity of NP j as computed by the anaphoricity classifier.


As discussed above, we optimize the anaphoricity model for coreference performance via the conservativeness parameter.

we will use this parameter to maximize the F-measure score for a particular data set and learner combination using held-out development data.


The experimental setting employed here is essentially the same as that in the CBGO setting, except that anaphoricity information is incorporated into the coreference system as a feature rather than as constraints.

each training/test instance is augmented with a feature whose value is the computed anaphoricity of NP j .


In this section, we examine which anaphoricity features are important in order to gain linguistic insights into the problem.

we measure the informativeness of a feature by computing its information gain (see p.22 of Quinlan (1993) for details) on our three data sets for training anaphoricity classifiers.


We have examined two largely unexplored issues in computing and using anaphoricity information for improving learning-based coreference systems: representation and optimization.

we have systematically evaluated all four combinations of local vs. global optimization and constraint-based vs. feature-based representation of anaphoricity information in terms of their effectiveness in improving a learning-based coreference system.


Unlike the noisy-channel model, the joint source-channel model does not try to capture how source names can be mapped to target names, but rather how source and target names can be generated simultaneously.

we estimate a joint probability model that can be easily marginalized in order to yield conditional probability models for both transliteration and back-transliteration.


Table 4 shows that using word-aligned data leads to better translation quality than using sentencealigned data.

significantly less data is needed to achieve a high Bleu score when using word alignments.


We assume a simple hypothesis: the best metric should be the one that best discriminates between manual and automatically generated reports.

a similarity metric that cannot distinguish manual and automatic reports cannot be a good metric.


There may be better static analyses, and better uses of this static analysis.

future work will focus on using static analysis to determine smaller representations (by cutting nodes in Static Cuts) of the chart edges themselves.


The training corpora were modified using a utility by Brian Roark to convert newspaper text to speechlike text, before being used as training input to the model.

all numbers were converted to words (60 ¡ sixty) and all punctuation was removed.


Since training data are labeled as the context unit, we employ a Naive Bayes classifier because it can be built by estimating the word probability in a category, but not in a document.

the Naive Bayes classifier does not require labeled data with the unit of documents unlike other classifiers.


The classification of a test document is based on the voting of each feature of that test document.

the final prediction score is calculated by accumulating the voting scores of all features.


In order to overcome this problem, we propose a hybrid method for choosing keywords.

a developer obtains 10 candidate keywords from our keyword extraction method and then they can choose proper keywords from them.


We also create a family of cut-based subjectivity detectors; these take as input the set of sentences appearing in a single document and determine the subjectivity status of all the sentences simultaneously using per-item and pairwise relationship information.

for a given document, we use the construction in Section 2.2 to build a graph wherein the source s and sink t correspond to the class of subjective and objective sentences, respectively, and each internal node v i corresponds to the document's i th sentence s i .


In the future, we will perform a large scale evaluation on domain specific corpora.

we will use balanced and domain specific corpora to isolate words having very different neighbours, and therefore rankings, in the different corpora and to detect and target words for which there is a highly skewed sense distribution in these corpora.


The intuition here is that not all of the senses that are possible for a word will be relevant for a concept.

the distribution over the senses of a word given a concept can be expected to have a lower entropy than the distribution over the senses of the word in the language as a whole.


We use the shorthand The Sense Model makes the assumption, inspired by ideas in Diab and Resnik (2002) and Bengio and Kermorvant (2003), that the English word 3 ¡ and the Spanish word 7 £ in a translation pair share the same precise sense.

the set of sense labels for the words in the two languages is the same and may be collapsed into one set of senses that is responsible for both English and Spanish words and the single latent variable in the model is the sense label for both words .


The assumption of a one-to-one association between sense labels made in the Sense Model may be too simplistic to hold for arbitrary languages.

it does not take into account that translation is from sentence to sentence (with a shared meaning), while the data we are modeling are aligned single-word translations in each language given their senses.


On the PARSEVAL metric, our algorithm performed particularly well on null complementizer and control locus insertion, and on S node relocation.

johnson noted that the proper insertion of control loci was a difficult issue involving lexical as well as structural sensitivity.


The main drawback is that structures may not be properly represented by flat features.

these problems affect the processing of predicate argument structures annotated in PropBank (Kingsbury and Palmer, 2002) or FrameNet (Fillmore, 1982).


In this paper, we select portions of syntactic trees, which include predicate/argument salient sub-structures, to define convolution kernels for the task of predicate argument classification.

our kernels aim to (a) represent the relation between predicate and one of its arguments and (b) to capture the overall argument structure of the target predicate.


Table 1 shows that a major problem with the baseline system is that it accepts too many user utterances.

the baseline system accepts the wrong interpretation for 22 in-grammar utterances, 45 utterances which it should have rejected as outof-grammar, and 12 utterances which it should have ignored.


Even a textstructuring algorithm functioning solely on the basis of the Centering constraints might therefore still choose the particular order in (1).

a metric of text coherence purely based on Centering principles-trying to minimize the number of nocbs-may be sufficient to explain why this order of clauses was chosen, at least in this particular genre, without need to involve more complex explanations.


A -B and C -D would be in the same relation, in this case, merger and acquisition (M&A).

we could discover the relation between these ORGANIZATIONs.


If most of the NE pairs in the same cluster had words in common, the common words would represent the characterization of the cluster.

we can regard the common words as the characterization of a particular relation.


We found that all large clusters had appropriate relations and that the common words which occurred frequently in those clusters accurately represented the relations.

the frequent common words could be regarded as suitable labels for the relations.


The features are the same as those used for the models in Section 4.1, but are represented with indicator variables.

for each feature we calculated the number of possible values and then represented an observation of the feature as a sequence of binary values in which one value is set to and the remaining values are set to .


By using n-grams for grammar induction one addresses the two parts in one go.

the use of n-grams implies that the solution will be searched for in the class of probabilistic regular languages, since n-grams induce probabilistic automata and, consequently, probabilistic regular languages.


These observations give rise to a natural question: can we gain anything in parsing from using general methods for inducing regular languages instead of methods based on n-grams?

can we describe the bodies of grammatical rules more accurately and more concisely by using general methods for inducing regular languages?


The MDI algorithm traverses the lattice of all possible partitions for this general automaton, attempting to merge states that satisfy a trade-off that can be specified by the user.

assume that A 1 is a temporary solution of the algorithm and that A 2 is a tentative new solution derived from A 1 .


However, as one might expect, it is easier to cluster word sequences (or word class sequences) than to tell how to put them together into trees.

if one is given all contiguous subsequences (subspans) from a corpus of sentences, most natural clusters will not represent valid constituents (to the extent that constituency of a non-situated sequence is even a well-formed notion).


In the basic model, P(B) is uniform over binary trees.

then, for each i, j , the subspan and context pair is generated via a classconditional independence model: all spans guess their sequences and contexts given only a constituency decision b.


A key reason that these models are capable of recovering structure more accurately than previous work is that they minimize the amount of hidden structure that must be induced.

neither model attempts to learn intermediate, recursive categories with no direct connection to surface statistics.


At least in this experiment, its initial bias toward such models carried over to the final learned model.

7 the higher-entropy local maximum found by DA, in this case, explained the observed data almost as well without overcommitting to particular tag sequences.


We touted DA's insensitivity to those parameters as an advantage, but in scenarios where well-chosen initial parameters can be provided (as in §4.3), we wish for DA to be able exploit them.

there are at least two cases where "good" initializers might be known.


ALM extends FLM to include some information on fidelity.

it pays some attention to how faithful a translation is to its source text.


We assume here that the MEMT works on a sentence-by-sentence basis.

it takes as input a source sentence, gets it translated by several OTSs, and picks up the best among translations it gets.


Now the question we are interested in asking is whether the choice of LM really matters.

does a particular choice of LM gives a better performing FLM ψ or ALM ψ than something else, and if it does, do we have a systematic way of choosing one LM over another?


Tables 2 and 3 show that ONMF improves on several of these results.

we get better recall and F -score on the probable alignments (and even a better precision than Ralign.EF.1 in table 2).


The correct translation t can then be extracted if it can be found as a translation of m. The transitive translation model, therefore, combines the processes of both direct translation and indirect translation, and is defined as:  One major challenge of the transitive translation model is the propagation of translation errors.

incorrect m will significantly reduce the accuracy of the translation of s into t. A typical case is the indirect association problem (Melamed, 2000), as shown in Figure 2 in which we want to translate the term s 1 (s=s 1 ).


R χ 2(s,t) (or R CV (s,t)) represents the similarity ranking of each translation candidate t with respect to s and is assigned to be from 1 to k (number of output) in decreasing order of similarity measure S X 2 direct (s,t) (or S CV direct (s,t)).

if the similarity rankings of t are high in both of the context-vector and chi-square methods, it will be also ranked high in the combined method.


One of the advantages of the use of PDAs is that these devices provide an operational specification that determines which steps must be performed when parsing an input string, something that is not offered by CFGs.

pDAs can be associated to parsing strategies for contextfree languages.


Also, observe that our results do not depend on the general behaviour of a parsing strategy S, but just on its "point-wise" behaviour on each input CFG.

if S does not have the CPP and the SPP, but for some fixed CFG G of interest we obtain a PDT A that has the CPP and the SPP, then we can still apply the construction in Theorem 5.


These approaches are nonoptimal, in the sense that a collection of probability distributions cannot be obtained.

some probability distributions expressible in terms of a context-free grammar cannot be expressed in terms of the LR parser constructed from that grammar, under the restrictions of the existing approaches to training of LR parsers.


Noteworthy is that (Sornlertlamvanich et al., 1999) showed empir-ically that an LR parser may be more accurate than the original CFG, if both are trained on the basis of the same treebank.

the resulting probability function p A on transitions of the PDT allows better disambiguation than the corresponding function p G on rules of the original grammar.


Properness for PCFGs does not restrict the space of probability distributions on the set of parse trees.

if a probability distribution can be defined by attaching probabilities to rules, then we may reassign the probabilities such that that PCFG becomes proper, while preserving the probability distribution.


Although CFGs and PDTs are weakly equivalent, they behave very differently when they are extended with probabilities.

there seems to be no notion similar to PCFG properness that can be imposed on all types of PDTs without losing generality.


This is surprising, as only properness has been described in existing literature on probabilistic PDTs (PPDTs).

all existing approaches to probabilistic LR parsing have assumed properness rather than anything related to reverse-properness.


capabilities with the possibility of processing complex questions.

questions whose answer needs to be gathered from pieces of factual information scattered in different documents.


That is, questions whose answer needs to be gathered from pieces of factual information scattered in different documents.

we have designed a layer oriented to process the different types of temporal questions.


systems can deal with simple factual temporal questions.

questions requiring to be answered with a date, e.g.


Finally, he would give as a final answer one of the answers to the first question (if there is any), whose associated date stays within the period of dates implied by the answer to the second question.

he would obtain the final answer by discarding all answers to the simple questions which do not accomplish the restrictions imposed by the temporal signal provided by the original question (during).


System Architecture Current Question Answering system architectures do not allow to process complex questions.

questions whose answer needs to be gathered from pieces of factual information that is scattered in a document or through different documents.


We have found empirically that when our own QA system's (Prager et al., 2000;Chu-Carroll et al., 2003) top answer is wrong, the correct answer is often present later in the ranked answer list.

the correct answer is in the passages retrieved by the search engine, but the system was unable to sufficiently promote the correct answer and/or deprecate the incorrect ones.


But such morphological variations are inapplicable to Chinese verbs; instead, they are conveyed lexically (Li and Wong, 2002).

tense and aspect in Chinese are expressed using a combination of time words, auxiliaries, temporal position words, adverbs and prepositions, and particular verbs.


In our task, we also use it to quantify the similarity between two words.

the calculation in SVM need be projected to a higher dimensional space by using a certain kernel function (,) ij K ww .


Based on these benchmarks, using either manually truthed corpora or automatically constructed corpora, using either ambiguous corpora or unambiguous corpora, our algorithm consistently and significantly outperforms the existing algorithm.

our system achieves a very high precision (0.96 precision).


The weighting function f must have the property that f(x+y) > f(x) + f(y) for any positive integers x and y.

consecutive matches are awarded more scores than non-consecutive matches.


Then kmeans clustering was conducted on latent semantic space transformed from context vector matrix, using normalized Euclidean distance.

context vectors were reduced to 100 dimensions using SVD.


As it was shown in Table 5, with both χ 2 and f req based feature ranking, F SGM M algorithm performed better than CGD term and CGD SV D if we used average accuracy to evaluate their performance.

with χ 2 based feature ranking, F SGM M attained 55.4% average accuracy, while the best average accuracy of CGD term and CGD SV D were 40.9% and 51.3% respectively.


Informally stated, the generative capacity of the class of all component grammars of a GMTG exactly corresponds to the class of all projected languages.

the interaction among different grammar components in the rewriting process of GMTG does not increase the generative power beyond the above mentioned class.


Additionally, whether the immediately surrounding words bear pitch accent also affect the likelihood of accentuation.

a word that might typically be accented may be unaccented because the surrounding words also bear pitch accent.


what good classifiers for the task are like) through jointly learning multiple classification problems on unlabeled data.

we systematically create thousands of problems (called auxiliary problems) relevant to the target task using unlabeled data, and train classifiers from the automatically generated 'training data'.


Second, we report higher performance than the previous best results on syntactic chunking (the CoNLL'00 corpus) and named entity chunking (the CoNLL'03 English and German corpora).

our results are obtained by using unlabeled data as the only additional resource while many of the top systems rely on hand-crafted resources such as large name gazetteers or even rulebased post-processing.


We present a linear prediction model for structural learning, which extends the traditional model to multiple problems.

we assume that there exists a low-dimensional predictive structure shared by multiple prediction problems.


For semi-supervised learning, the idea is to create many auxiliary prediction problems (relevant to the task) from unlabeled data so that we can learn the shared structure ¢ (useful for the task) using the ASO algorithm.

we want to create auxiliary problems with the following properties: Automatic labeling: we need to automatically generate various "labeled" data for the auxiliary problems from unlabeled data.


Relevancy: auxiliary problems should be related to the target problem.

they should share a certain predictive structure.


We show that our ASO-based semi-supervised learning method (hereafter, ASO-semi) can produce results appreciably better than all of the top systems, by using unlabeled data as the only additional resource.

we do not use any gazetteer information, which was used in all other systems.


Lexical-semantic resources have been applied successful to a wide range of Natural Language Processing (NLP) problems ranging from collocation extraction (Pearce, 2001) and class-based smoothing (Clark and Weir, 2002), to text classification (Baker and McCallum, 1998) and question answering (Pasca and Harabagiu, 2001).

wORDNET (Fellbaum, 1998) has significantly influenced research in NLP.


Our WSD system was built to combine information from many different sources, using as much linguistic knowledge as could be gathered automatically by NLP tools.

our goal was to see the extent to which sense-tagging of verbs could be improved by adding features that capture information about predicate-arguments and selectional restrictions.


We found that the generator which intuitively produces the best language does engender the most learning.

it appears that functional aggregation is responsible for the improvement.


Many of today's most popular natural language applications -Machine Translation, Summarization, Question Answering -are text-to-text applications.

they produce textual outputs from inputs that are also textual.


We note here, however, that depending on the input IDL-expression, the task addressed can vary in complexity from linear to exponential.

for the intersection of an IDL-expression complexity.


Though we made no language specific model changes, we did need to make some data specific changes.

we used the method of  to simplify part-of-speech tags since the rich tags used by Czech would have led to a large but rarely seen set of POS features.


However, many semantic information-oriented applications like Question Answering, Information Extraction and Paraphrase Acquisition require a tighter similarity criterion, as was also demonstrated by papers at the recent PASCAL Challenge on Recognizing Textual Entailment (Dagan et al., 2005).

all these applications need to know when the meaning of one word can be inferred (entailed) from another word, so that one word could substitute the other in some contexts.


Possible sources of noise in their experiment could be ignoring word polysemy and data sparseness of word-feature cooccurrence in the corpus.

extending the rationale of Weeds et al., we suggest that if the meaning of a word v entails another word w then it is expected that all the typical contexts (features) of v will occur also with w. the characteristic contexts of v are expected to be included within all w's contexts (but not necessarily amongst the most characteristic ones for w).


Although the results were encouraging our manual data analysis shows that we still have to handle word ambiguity.

this is important in order to be able to learn the direction of entailment.


Instead, we examined people's ability to discern relative differences, because by varying the rating differences represented by the test instances, we can evaluate multiple granularities in a single experiment.

at intervals over a number of weeks, we authors (a non-native and a native speaker of English) examined pairs of reviews, attemping to determine whether the first review in each pair was (1) more positive than, (2) less positive than, or (3) as positive as the second.


5 If we choose R from a family of sufficiently "gradual" functions, then similar items necessarily receive similar labels.

we consider linear, S -insensitive SVM regression (Vapnik, 1995;Smola and Schölkopf, 1998); the idea is to find the hyperplane that best fits the training data, but where training points whose labels are within distance S of the hyperplane incur no loss.


Hence, PSP appears to be a promising basis for computing document similarity for our rating-inference task.

we defined  But before proceeding, we note that it is possible that similarity information might yield no extra benefit at all.


When the algorithms being compared are statistically indistinguishable on Triangles point towards significantly better algorithms for the results plotted above.

if the difference between a row and a column algorithm for a given author dataset (a, b, c, or d) is significant, a triangle points to the better one; otherwise, a dot (.)


We suppose that a small number of seed words are given.

we know beforehand the semantic orientations of those given words.


The threshold of the magnetization for hyper-parameter estimation is set to 1.0 × 10 −5 .

the predicted optimal value of β is the largest β whose corresponding magnetization does not exceeds the threshold value.


To measure the effect of fullyfledged coreference resolution, we constructed entity classes simply by clustering nouns on the basis of their identity.

each noun in a text corresponds to a different entity in a grid, and two nouns are considered coreferent only if they are identical.


However the nature of the relationship between the coherence relations signalled by discourse connectives and their empirical distributions has to date been poorly understood.

one might wonder whether connectives with similar meanings also have similar distributions.


These statistics show that our theoretically motivated expectations are supported.

(1) SYNONYMOUS connectives have the least distributional divergence and EXCLUSIVE connectives the most, (2) CONT.


We demonstrated the new function's utility in helping to predict the substitutability of connectives, and it can be expected to have wider applicability to lexical acquisition tasks.

it is expected to be useful for learning relationships which cannot be characterised purely in terms of similarity, such as hyponymy.


The central idea behind the majority of these learningbased approaches is to recast coreference resolution as a binary classification task.

a classifier is first trained to determine whether two NPs in a document are co-referring or not.


Our goal in this paper is to improve the robustness of the standard approach by addressing the above weaknesses.

we propose the following procedure for coreference resolution: given a set of NPs to be clustered, (1) use pre-selected learningbased coreference systems to generate candidate partitions of the NPs, and then (2) apply an automatically acquired ranking model to rank these candidate hypotheses, selecting the best one to be the final partition.


Now we need to define the set of allowable values for each of these elements.

we want to define them in such a way that the resulting coreference systems can potentially generate good candidate partitions.


's method creates a smaller number of training instances than McCarthy and Lehnert's.

a positive instance is created for each anaphoric NP, NP¡ , and its closest antecedent, NP ; and a negative instance is created for NP¡ paired with each of the intervening NPs, NP .


Following previous work on using global features of candidate structures to learn a ranking model (Collins, 2002), the global (i.e., partition-based) features we consider here are simple functions of the local features that capture the relationship between NP pairs.

we define our partition-based features in terms of the features in the Ng  .


Method-based features, on the other hand, are used to encode the identity of the coreference system that generated the candidate partition under consideration.

we have one method-based feature representing each pre-selected coreference system.


In contrast to the MUC results, the B-CUBED results for the two baseline systems are mixed (see rows 1 and 2 of Table 4).

while there is no clear winner for the NWIRE data set, N&C performs better on BNEWS but worse on NPAPER than the Duplicated Soon system.


Unlike the MUC results, using more features to train the ranking model does not always yield better performance with respect to the B-CUBED scorer (see rows 3-5 of Table 4).

the best result for BNEWS is achieved using only method-based features, whereas the best result for NPAPER is obtained using only partitionbased features.


This is essentially an upper limit on how much our approach can improve upon the baselines given the current set of candidate partitions.

the performance of our approach is limited in part by the quality of the candidate partitions, more so with B-CUBED than with the MUC scorer.


We investigate two unexplored issues that influence the effectiveness of such information: statistics source and learning framework.

we for the first time propose to utilize the web and the twin-candidate model, in addition to the previous combination of the corpus and the single-candidate model, to compute and apply the semantic information.


Corpus-based knowledge usually suffers from data sparseness problem.

many predicate-argument tuples would be unseen even in a large corpus.


As "NP collected" should occur less frequently than "NP said", the candidates of it 1 would generally have predicate-argument statistics lower than those of it 2 .

a positive instance for it 1 might bear a lower semantic feature value than a negative instance for it 2 .


In fact, our analysis of the current data set reveals that most P-Prons refer back to a P-Pron or NE candidate whose semantic category (human) has been determined.

simply using features NE and Pron is sufficient to guarantee a high success, and thus the relatively weak semantic feature would not be taken in the learned decision tree for resolution.


We explored two issues that affect the utility of the semantic information: statistics source and learning framework.

we proposed to utilize the web and the twin-candidate model, in addition to the common combination of the corpus and single-candidate model, to compute and apply the semantic information.


Our reranking parser associates a parse with a score v θ (y), which is a linear function of the feature values f (y).

each feature f j is associated with a weight θ j , and the feature values and weights define the score v θ (y) of each parse y as follows: Given a string s, the reranking parser's outputŷ(s) on string s is the highest scoring parse in the n-best parses Y(s) for s, i.e., The feature weight vector θ is estimated from the labelled training corpus as described in section 4.


The Collins parser (Collins, 1997) does use dynamic programming in its search.

whenever a constituent with the same history is generated a second time, it is discarded if its probability is lower than the original version.


Rather than storing a single best parse of each edge, one stores n of them.

when using dynamic programming, rather than throwing away a candidate if it scores less than the best, one keeps it if it is one of the top n analyses for this edge discovered so far.


Here the parser first creates a parse forest based upon a much less complex version of the complete grammar.

it only looks at standard CFG features, the parent and neighbor labels.


We experimented with various kinds of feature selection, and found that a simple count threshold performs as well as any of the methods we tried.

we ignored all features that did not vary on the parses of at least t sentences, where t is the count threshold.


Work on kernel methods in natural language has focussed on the definition of appropriate kernels for natural language tasks.

most of the work on parsing with kernel methods has focussed on kernels over parse trees (Collins and Duffy, 2002;Collins and Roark, 2004).


The dynamic programming approach requires the feature vector of a tree to be decomposable into a sum over parts of the tree.

this is impossible with the TOP and Fisher kernels derived from the SSN model.


We believe that this approach to defining kernels would simplify the problem of defining kernels for these tasks, and could be very useful for many of them.

maximum entropy models also use a normalized exponential function to estimate probabilities, so all the methods discussed in this paper would be applicable to maximum entropy models.


Question Answering in the QBTE Model 1 involves directly classifying words w i in the document into answer words or non-answer words.

given input x (i) for w i , its class label is selected from among {I, O, B} as follows: I: if the word is in the middle of the answer word sequence; O: if the word is not in the answer word sequence; B: if the word is the start word of the answer word sequence.


We evaluate the protocols that are generated by these transformations with reference to the benchmarks extracted in section 2.

we show that one such transformation, dubbed Add Side Participants(ASP), yields protocols for querying and assertion that fulfill these benchmarks.


On this view each dialogue participant's view of the common ground, their Dialogue Gameboard (DGB), is structured by a number of attributes including the following three: FACTS: a set of facts representing the shared assumptions of the CPs, LatestMove: the most recent grounded move, and QUD ('questions under discussion'): a partially ordered set-often taken to be structured as a stack-consisting of the currently discussable questions.

querying and Assertion Both querying and assertion involve a question becoming maximal in the querier/asserter's qUD: 3  Following (Larsson, 2002;Cooper, 2004), one can 3 pushed onto the stack, if one assumes qUD is a stack.


For this we created an "accessibility window" going back from each utterance.

we included for each utterance a) all previous utterances of the same speaker from the same turn; and b) the three last utterances of every speaker, but only until one speaker took the turn again and up to a maximum of 6 previous utterances.


Under the assumption that constituents generally move as a whole, we predict the probability of each given ordering of modifiers independently.

we make the following simplifying assumption (where c is a function returning the set of nodes modifying t): Furthermore, we assume that the position of each child can be modeled independently in terms of a head-relative position: Figure 3a demonstrates an aligned dependency tree pair annotated with head-relative positions; Figure 3b presents the same information in an alternate tree-like representation.


• Peers which are very far from the set of models all receive QUEEN = 0.

qUEEN does not distinguish between very poor automatic summarisation strategies.


We can formalise this idea by introducing a universal quantifier on the variable x in the QUEEN formula.

qUEEN X,M (a) can be defined as the probability, measured over M × M × M , that for every metric in X the automatic summary a is closer to a model than two models to each other.


We have created a supervised version of the noisychannel model with some improvements over the K&M model.

we learned that adding an additional rule type improved compression, and that enforcing some deletion constraints improves grammaticality.


This property makes summarization in this area even more difficult than traditional summarization.

topic "drift" occurs more radically than in written genres, and interpersonal and pragmatic content appears more frequently.


We investigate both approaches in this paper.

we develop a parser for German which attains the highest performance known to us by making use of smoothing and a highly-tuned suffix analyzer for guessing part-of-speech (POS) tags from the input text.


As explained above, this paper focuses on unlexicalized grammars.

we make use of probabilistic context-free grammars (PCFGs; Booth (1969)) for our experiments.


This is consistent with the finding in RY: their gains come from the joint learning of the probabilities P (w|s m 1 ) and P (s m 1 , t n 1 ).

the word model accounts for much of their gains over the  Equivalent to the E2 lexicon in RY.


To apply this idea, one needs to observe that Stochastic OT grammars are learned from ordinal data, as seen in (1).

only one aspect of the structure generated by those normal distributions -the ordering of constraints -is used to generate outputs.


Furthermore, the chain is defined in very simple terms: from each state sequence we can only transition to a state se-quence obtained by changing the state at any one position i, and the distribution over these possible transitions is just ( 1)where s −i is all states except s i .

the transition probability of the Markov chain is the conditional distribution of the label at the position given the rest of the sequence.


Due to the lack of a development set, our consistency model for the CMU Seminar Announcements is much simpler than the CoNLL model, the numbers where selected due to our intuitions, and we did not spend much time hand optimizing the model.


Our model generalizes naturally to other statistical models and other tasks.

it could in the future be applied to statistical parsing.


The lexical choices are made in a way that heavily prefers phrasal cohesion in the output target sentence, as scored by the language model.

the predictions benefit from the sentential context of the target language.


The third voting model is a boosting model (Freund and Schapire, 1997), since has consistently turned in very competitive scores on related tasks such as named entity classification (Carreras et al., 2002) .

an AdaBoost.MH model was used (Schapire and Singer, 2000), which is a multiclass generalization of the original boosting algorithm, with boosting on top of decision stump classifiers (i.e., decision trees of depth one).


The reason is that the classification hyperplane was learned only from labeled data.

the coherent structure (two-moon pattern) in unlabeled data was not explored when inferring class boundary.


Then at the ninth iteration, A 9 was tagged as +1 since the label of A 9 was determined only by labeled points in kNN model: A 9 is closer to any point in {A 0 ∼ A 8 } than to any point in {B 0 ∼ B 8 }, regardless of the intrinsic structure in data: A 9 ∼ A 10 and B 9 ∼ B 10 are closer to points in lower moon than to points in upper moon.

bootstrapping method uses the unlabeled data under a local consistency based strategy.


The reason of this self-correcting behavior is that with the push of label information from nearby points, the value of Y B,+1 became higher than Y B,−1 .

the weight of edge B − C is larger than that of edge B − A, which makes it easier for +1 label of point C to travel to point B.


Then example A will compete with C and other unlabeled examples around B when determining the label of B.

the labels of unlabeled examples are determined not only by nearby labeled examples, but also by nearby unlabeled examples.


In this paper we present a supervised Word Sense Disambiguation methodology, that exploits kernel methods to model sense distinctions.

a combination of kernel functions is adopted to estimate independently both syntagmatic and domain similarity.


Exploring this direction is the main focus of this paper.

we acquire a Domain Model (DM) for the lexicon (i.e.


Section 2 introduces the notion of Domain Model.

an automatic acquisition technique based on Latent Semantic Analysis (LSA) is described.


In Section 3 we present a WSD system based on a combination of kernels.

we define a Domain Kernel (see Section 3.1) and a Syntagmatic Kernel (see Section 3.2), to model separately syntagmatic and domain aspects.


Each kernel adds some additional dimensions to the feature space.

we have defined two families of kernels: Domain and Syntagmatic kernels.


The associate gap-weighted subsequence kernel is defined by We modified the generic definition of the string kernel in order to make it able to recognize collocations in a local window of the word to be disambiguated.

we defined two Syntagmatic kernels: the n-gram Collocation Kernel and the ngram PoS Kernel.


In this paper we presented a supervised algorithm for WSD, based on a combination of kernel functions.

we modeled domain and syntagmatic aspects of sense distinctions by defining respectively domain and syntagmatic kernels.


Experimental results show that the use of Domain Models allows us to reduce the amount of training data, opening an interesting research direction for all those NLP tasks for which the Knowledge Acquisition Bottleneck is a crucial problem.

we plan to apply the same methodology to Text Categorization, by exploiting the Domain Kernel to estimate the similarity among texts.


Are the observed gender linguistic differences valuable from an engineering perspective as well?

can a natural language processing task benefit from modeling these differences?


Our computational tools have allowed us to quantitatively show that the gender of one speaker influences the linguistic patterns of the other speaker.

classifying same-gender conversations can be done with almost perfect accuracy, while evidence of some convergence of male and female linguistic patterns in cross-gender conversations was observed.


For our models, gain(a, l) can be obtained in a more efficient way 3 : Note that we restrict that h(a, e, f ) ≥ 0 for all feature functions.

the original terminational condition for greedy search algorithm is: : By introducing gain threshold t, we obtain a new terminational condition: Note that we restrict h(a, e, f ) ≥ 0 for all feature functions.


The base feature of our log-linear models, IBM Model 3, takes the parameters generated by GIZA++ as parameters for itself.

our loglinear models share GIZA++ with the same parame-ters apart from POS transition probability table and bilingual dictionary.


In this paper, we first train two models using the out-of-domain training data and the in-domain training data, and then build a new alignment model through linear interpolation of the two trained models.

we make use of the out-of-domain training data and the in-domain training data by interpolating the trained alignment models.


For the special case of Rote extractors, a more attractive alternative has been proposed by Brin (1998), Agichtein and Gravano (2000), and Ravichandran and Hovy (2002).

essentially, for any text snippet of the form A 1 pA 2 qA 3 , these systems estimate the probability that a relationship r(p, q) holds between entities p and q, given the interstitial context, as 2 the probability of a relationship r(p, q) is the number of times that pattern xA 2 y predicts any relationship r(x, y) in the training set T .


After fusion, the system returns one consensus target for each person and thus the evaluation is on the accuracy of those targets.

missing tar- Additionally, since the targets are ranked, we also calculated the mean reciprocal rank (MRR).


One possibility is that higher ranked retrieved documents are more likely to contain biographical facts, while in later documents it is more likely that automatically annotated training instances are in fact false positives.

higher ranked documents are cleaner training data.


This paper has presented new experimental methodologies and results for cross-document information fusion, focusing on the task of biographic fact extraction and has proposed a new method for crossfield bootstrapping.

we have shown that automatic annotation can be used effectively to train statistical information extractors such Naïve Bayes and CRFs, and that CRF extraction accuracy can be improved by 5% with a negative example model.


From the speech production point of view, thousands of spoken languages from all over the world are phonetically articulated using only a few hundred distinctive sounds or phonemes (Hieronymus, 1994).

common sounds are shared considerably across different spoken languages.


German has substantially freer word order than English.

note that while the verb comes second in matrix clauses, essentially any element can be in the first position.


Typically, reordering models in phrase-based systems are based solely on movement distance.

at each point in decoding a "cost" is associated with skipping over 1 or more German words.


We suffered a slight loss in the speed by using the BiVN(see Table 4), but we could improve performance in all of the translation systems(see Table  3).

the performance improvement in CBIBM with BiVN was remarkable.


The candidate argument sequence that corresponds to the correct assignment in Figure 1 will be: Features from Local Models: All features included in the local models are also included in our joint models.

each template for local features is included as a joint template that concatenates the local template and the node label.


Typically they range from about two to six words in length, but critically we cannot judge whether a phrase is lexical without considering both shorter and longer sequences.

the statistical comparison that needs to be made must apply in principle to the entire set of word sequences that must be distinguished from phrasal terms, including longer sequences, subsequences, and overlapping sequences, despite the fact that these are not statistically independent events.


It is of course an empirical question how well mutual expectation performs (and we shall examine this below) but mutual expectation is not in any sense a significance test.

if we are examining a phrase like the east end, the conditional probability of east given [__ end] or of end given [__ east] may be relatively low (since other words can appear in that context) and yet the phrase might still be very lexicalized if the association of both words with this context were significantly stronger than their association for other phrases.


That is, if we are examining a phrase like the east end, the conditional probability of east given [__ end] or of end given [__ east] may be relatively low (since other words can appear in that context) and yet the phrase might still be very lexicalized if the association of both words with this context were significantly stronger than their association for other phrases.

to the extent that phrasal terms follow the regular patterns of the language, a phrase might have a relatively low conditional probability (given the wide range of alternative phrases following the same basic linguistic patterns) and thus have a low mutual expectation yet still occur far more often than one would expect from chance.


Since we do not know what the expected shape of this distribution is going to be, a nonparametric method using ranks is in order, and there is some reason to think that frequency rank regardless of n-gram size will be useful.

ha, Sicilia-Garcia, Ming and Smith (2002) show that Zipf's law can be extended to the combined frequency distribution of n-grams of varying length up to rank 6, which entails that the relative rank of words in such a combined distribution provide a useful estimate of relative probability.


We then apply the fast hamming distance search algorithm as described in Section 3.

it involves steps 3 to 6 of the algorithm.


They all studied the effects of various kinds of tokenization, lemmatization and POS tagging and show a positive effect on SMT quality.

considering Arabic, Lee (2004) investigated the use of automatic alignment of POS tagged English and affix-stem segmented Arabic to determine appropriate tokenizations.


Our experiments show that precision-oriented alignments yield better phrases for MT than recalloriented alignments.

they cover a higher percentage of our test sets and result in fewer untranslated words and selection of longer phrases during decoding.


To demonstrate how alignment-related components of the MT system might change the translation quality significantly, we did an additional experiment to compare different techniques for extracting phrases from a given alignment.

we are comparing two techniques for phrase extraction: 1.


According to Bayes rule we have: Evaluating P (c) requires integrating over π.

we have: where n 1 is the number of utterances for which c u = 1, and n 0 is the number of utterances for which c u = 0.


We aim to jointly maximize the intra-segmental similarity and minimize the similarity between different segments.

we want to find the segmentation with a maximally homogeneous set of segments that are also maxi-mally different from each other.


This is not surprising, since we based the acquisition of C(p) on gender.

5 We calculate significance with McNemar's test, p=0.05.


In the paper, we propose a kernel-based method that can automatically mine the syntactic information from the parse trees for pronoun resolution.

we utilize the parse trees directly as a structured feature and apply kernel functions to this feature, as well as other normal features, to learn the resolution classifier.


In this paper we will explore how to utilize the syntactic parse trees to help learning-based pronoun resolution.

we directly utilize the parse trees as a structured feature, and then use a kernel-based method to automatically mine the knowledge embedded in the parse trees.


For a pronoun to be resolved, Hobbs' algorithm works by searching the parse trees of the current text.

the algorithm processes one sentence at a time, using a left-to-right breadth-first searching strategy.


In the paper, we propose a kernel-based method to incorporate the information from parse trees.

we directly utilize the syntactic parse tree as a structured feature, and then apply kernels to such a feature, together with other normal features, to learn the decision classifier and do the resolution.


Empirically, it is attested as comparatively slower reading time or longer eye fixation at a disambiguating region in an ambiguous sentence compared to its control sentences (Frazier and Rayner, 1982;Trueswell, 1996).

the garden-path effect detected in many human studies, in fact, is measured through a "comparative" method.


Under this general assumption, the overall difficulty of a sentence is expected to be measured or predicted by the mean size of probability decrease.

probability will drop faster in garden-path sentences than in control sentences (e.g.


However, the test results in this study provide a simpler account for the effect.

the comparative difficulty of an object-relative clause might be attributed to its less frequent POS sequence.


The tagger predicted the processing difficulty of the RR ambiguity with probability re-ranking.

the tagger initially favors the main-verb interpretation for the ambiguous -ed form, and later it makes a repair when the ambiguity is resolved as a past-participle.


Such capabilities render it possible to arrive at a precise evaluation of the grammaticality of the input.

instead of deciding on the grammaticality of the input, we can give an indication of its grammaticality, quantified on the basis of the description of the properties of the input.


Although they are "on-topic", these two documents are non-parallel.

they contain no parallel sentence pairs; methods designed to extract full parallel sentences will not find any useful data in them.


eM ¡ © S achieves the highest accuracy among the methods described so far.


(2004) and performs domain driven disambiguation (IRST-DDD).

the approach com-  Table 6: Results of individual disambiguation algorithms and rank-based ensemble on Senseval-3 nouns pares the domain of the context surrounding the target word with the domains of its senses and uses a version of WordNet augmented with domain labels (e.g., economy, geography).