The above approach does not guarantee, however, that all "impossible n-grams" are considered.

any "impossible trigram" [First,Second,Third] cannot be detected as such (i.e.


However, since no morphological clue is available either to tell that an icon is, e.g., accusative or dative, we have to rely on semantic knowledge to guide role assignment.

an icon parser has to know that drinking is something generally done by living beings on liquid objects.


To support context-based multimodal interpretation in conversational systems, we have developed a semantics-based representation to capture salient information from user inputs and the overall conversation.

we present three unique characteristics: finegrained semantic models, flexible composition of feature structures, and consistent representation at multiple levels.


Inspired by earlier works on multimodal interfaces (e.g., Bolt, 1980;Cohen el al., 1996;Wahlster, 1991;Zancanaro et al., 1997), we are currently building an intelligent infrastructure, called Responsive Information Architect (RIA) to aid users in their information-seeking process.

rIA engages users in a full-fledged multimodal conversation, where users can interact with rIA through multiple modalities (speech, text, and gesture), and rIA can act/react through automated multimedia generation (speech and graphics) (Zhou and Pan 2001).


Furthermore, MIND also identifies how an input relates to the overall conversation discourse through discourse understanding.

mIND uses a representation called conversation segment to group together inputs that contribute to a same goal or sub-goal (Grosz and Sidner, 1986).


In this paper, we focus on representing user inputs and the conversation context.

we discuss two aspects of representation: semantic models that capture salient information and structures that represent those semantic models.


This representation is inspired by the earlier works (Johnston et al., 1997;Johnston, 1998) and offers a flexibility to accommodate complex inputs.

mIND represents intention, attention and constraints identified from user inputs as a result of both unimodal understanding and multimodal understanding.


The key difference between MIND and earlier works is the use of rich contexts to improve understanding.

multimodal understanding consists of two sub-processes: multimodal fusion and context-based inference.


In the multi-step movement approach, the question to model for each potential attachment site of an extraposable clause is whether the clause should move up to its grandparent (a "yes" answer) or remain attached to its current parent (a "no" answer).

we have cast the problem as a staged classification task.


We use the hypothesis introduced by  that although a verb in a particular class can occur in all of the syntactic contexts as verbs from other classes the statistical distributions can be distinguished.

verbs from certain classes will be more likely to occur in some syntactic contexts than others.


The method works as follows: for each verb, we need to associate a score to the hypothesis that a particular set of dependents of the verb are arguments of that verb.

we need to assign a value to the hypothesis that the observed frame under consideration is the verb's SF.


We then check the verbs and objects of each sentence to see if they match these words.

if a category word or a lexically related word appears as the verb or the object of a sentence, the sentence is classified as a member of that group.


However, with discounting, the value becomes positive for uniformly dense clusters, because the frequencies of individual cells are always smaller than their agglomeration and so the discounting effect is stronger for the former.

using the same formula, we calculated the significance weights t i in c = (S T , S D ) as: and the significance weights of d j as: all the terms and documents in a cluster can be jointly ordered according to their contribution in the entropy calculation given by Eq.


The goodness of a wordmatch depends on the following factors: (1) level of synonymy -the number of synonyms the words have in common (according to WordNet, Miller et al., 1990); (2) position in sentence; and (3) partof-speech (PoS) -obtained using MINIPAR (Lin, 1998).

a word matches perfectly a word Arg in position in sentence Arg , if both words are exactly the same, they are in the same sentence position, and they have the same PoS.


Our results show that the two main factors that affect recognition performance are BN size and word noise, while the average edit distance remains stable for belief and arc noise, as well as for node insertions (the only exception occurs for 40% arc noise and size 9 BNs).

for arc noise, belief noise and node insertions, the average Figure 4: Effect of arc noise on performance (1560 trials) Figure 5: Effect of belief noise on performance (2040 trials) edit distance was 3 or less for all noise percentages, while for word noise, the average edit distance was higher for several word-noise and BN-size combinations.


The purpose of drawing charts and graphics, however, is not restricted just to use in such presentations.

as the drawing of quantitative charts helps to analyze huge amount of data and to find out its characteristics, they can be a useful means for interactive exploratory data analysis.


This paper discusses automatic quantitative chart design that supports such interactive exploratory data analysis.

a methodology is proposed for taking queries and requests expressed in natural language as input and answering them in charts through organizing that interaction into felicitous dialogue.


Descriptions describe constraints or conditions that the objects related to the action should satisfy, which has the form, s Quantifier, Var/Class, Restrictionw , where Quantifier is a generalized quantifier, Var is the variable of quantification, and the quantification ranges over the objects each of which is a member of Class and satisfies Restriction.

this logical form is a flattened version of Woods' MRL (Woods, 1978), and as in Woods' MRL, Class can be a function.


Now, chart realization is reduced to aspect assignment.

you can realize an appropriate chart from a given user utterance by obtaining its logical form, and, if some of the aspects are implicit in the utterance, by completing the perspective using the characteristics of the data concerned, the current context and so on.


In order to reduce those loads, we made the process of change visually understandable.

the system shows animations that represent which component of the current chart moves and changes to which component of the new one.


In addition to statistical constraint, our proposed statistical method requires that a candidate string must contain detected morphemes.

the statistical rules are triggered by detected morphemes only.


In order to test how useful countability would be in predicting the countability of unknown words, we tested the system using stratified ten-fold cross validation.

we divided the common noun dictionary into ten sets, then tested on each set in turn, with the other ninetenths of the data used as the training set.


Here, we employ the "one class versus all others" approach.

each classifier The consistency problem is solved by the Viterbi search.


Since SVMs do not output probabilities, we use the SVM+sigmoid method .

we use a sigmoid function to a probability-like value.


The performance of our method on six out of the 60 test words is summarized in Table 1.

the instances are classified according to the correct sense and the sense selected by our method.


Since our model allows only a verb between the RPD and the RP (see section 3.2) and \mae ni" is not a verb, this example cannot be handled by our method.

\mae ni " b l o c ks the rst \kuroi" from the second \kuroi" and they are never next to each other on the stack.


That leads us to define this function as follows: We crafted the definition of the weight P after several experiments.

we noticed that the function couldn't be symmetric (we cannot reason- why we introduce this power, to stress more on the ideas present in the vector we want to oppose.


how to choose the right relation between an item and a vector.

how to choose the good meaning of the word.


Regarding the level of texts, reformulating this hypothesis is straightforward: Contextual hypothesis for texts: the contextual similarity of the lexical constituents of two texts contributes to their semantic similarity.

: the more two texts share semantically similar words, the higher the probability that they deal with similar topics.


This demand decreases as the path distance of nodes increases so that topic changes latently controlled by preceding nodes can be realized.

: adding text z to the hierarchically structured connotation of x, we do not simply look for an already inserted text y, to which z is most similar, but to a path P , which minimizes the loss of cohesion in the overall tree, when z is attached to P .


Furthermore, we observe that model M3 induces trees of highest cohesion and lowest variance, whereas VS shows the highest variance and lowest cohesion scores in case of STs and CTs.

: based on semantic spaces, models M1, M2, and M3 produce more stable results than the vector space model.


There are several possible directions for our future research.

• We would like to expand the generation rules.


Since cooccurring words in different languages cannot be directly compared, a bilingual dictionary is used as a bridge across the corpora.

some other co-occurring words have similar meanings or are related to the same concept - (price competition of overseas travel has become intense.


According to Kuhn (1999), "The Off-line parsability condition is a restriction on allowable cstructures excluding that for a given string, infinitely many c-structure analyses are possible".

kuhn assumes that OLP is, in fact, a condition that is intended to guarantee finite ambiguity.


When a symbol is found that does not match a valid transition, the PTA is augmented with a new path to accept the rest of the string.

a Probabilistic Finite State Automaton (PFSA) is merely an automaton with probabilities associated with each transition and final state.


To measure the quality of the inferred DTD, we use the concept of Minimum Message Length (MML) (Georgeff & Wallace 1984).

we adapt the formula developed for PFSA (Raman 1997) as below: where: • N is the number of states in the PFSA • V is the cardinality of the alphabet plus one • tj is the number of times the jth state is visited • mj is the number of arcs from the jth state (plus one for final states) • m j is the number of arcs from the jth state (no change for final states) • nij is the frequency of the ith arc from the jth state • M is the sum of all mj values and • M is the sum of all m j values One of the goals of this work was to both produce new inference methods and make comprehensive comparisons with existing techniques.


The key of this new method is a new algorithm for the ant move selection as shown in algorithm 2.

in line 4, a merge must pass the sk-strings criterion to be considered.


The best overall performance is most important, whilst the best average indicates stability across different input parameters.

sk-ALL denotes the implementation of sk-strings with five variants from its family (refer to (Raman 1997) for details of different variants).


Among these learning algorithms, we focus on the Nearest Neighbor algorithm.

the k-Nearest Neighbor (k-NN) classifier in text categorization is one of the state-of-the-art methods including Support Vector Machine (SVM) and Boosting algorithms.


TF-IDF values of two terms t i and t j , which occur in a test document d, are modified by reflecting the co-occurrence frequency value.

the terms with a high co-occurrence frequency value and a low category frequency value could have higher term weights as follows: where i) tw(t i ,d) denotes a modified term weight assigned to term t i , ii) cf denotes the category frequency, the number of categories in which t i and t j co-occur, and iii) is the maximum value among all co-occurrence frequency values.


Even more, the time complexity of TCFP without contextual information is O(mc), where c is the number of categories.

the classification of TCFP requires a simple calculation in proportion to the number of unique terms in the test document.


Each performance was calculated as the average of the performance over the four sets.

the performance of the second proposed method is calculated according to four-fold cross validation (Mitchell, 1997).


The extra circles are called support vectors, and their removal would change the decision surface.

the decision surface for linearly separable space is a hyperplane which can be written as w•x+b = 0, where x is an arbitrary data point(x∈R n ) and w and b are learned from a training set.


'Perfect analysis' stands for the F1 achieved using the perfect (post-edited) output of the tracking method, i.e., the errors by both tracking and detecting shifts were corrected.

the documents judged as Yes but were not evaluated as Yes were eliminated, and the documents judged as No but were evaluated as Yes were added.


In our case x s has 8 elements, as has x.

the document vector d is represented as d=[x s x], e.g.


1 In German, as opposed to English, the auxiliary can be separated from the verb/verb group it belongs to.

a complement or modifier can split the verb chunk, and consequently the chunker builds two separate verb chunks.


It is clear that further work in this area is required.

we plan to experiment with different measures of association to determine the degree of relatedness of words, and with alternative clustering techniques.


Clearly, further work in this area is also necessary.

as suggested by a reviewer, we will try to exploit morpho-syntactic differences between simple nouns and modifiers to help distinguishing between the two types.


From the viewpoint of robust processing, an amount of lost information can be regarded as a cost (i.e., penalty) of robust processing.

default unification tries to minimize the cost.


3) It reduces to standard unification in the case of F and G being consistent.


Ideal lenient default unification tries to maximize the amount of information of the result, subsuming the result of forced unification.

ideal lenient default unification tries to generate a result as similar as possible to the result of forced unification such that the result is defined in the type hierarchy without the top type.


4) It reduces to standard unification in the case F and G are consistent (unifiable).

f Our algorithm for lenient default unification proceeds in the following steps.


In the example above, the structure-sharing tagged as 2 in the original grammar rule R is decomposed into the structuresharings 3 , 4 , 5 , 6 .

the structure-sharing tagged as 2 is preserved except HEAD:CASE:.


The importance value of a sentence by formula (3) is used for modifying TF value of a term.

since a TF value of a term in a document is calculated by the sum of the TF values of terms in each sentence, the modified TF value (WTF(d,t)) of the term t in the document d / is calculated by formula (4).


We achieved the similar results in our experiments.

k-NN produced a better performance by using Cen(S) and Rocchio produced a better performance by using Sim(S,T).


We describe in this paper different parsing techniques, from shallow to deep one, with this originality that they rely on the same formalism, described in the previous section.

in our approach, one can choose the granularity level of the parse without modifying linguistic resources In this technique, we get hierarchical and grammatical information while preserving robustness and efficiency of the processing.


In order to raise the accuracy of the intention inference, moreover, the context information is taken into consideration.

according to the occurrence probability of a sequence of intentions learned from a dialogue corpus with the intention tags, the degree of similarity with each utterance is weighted based on the intentions of the last utterances.


A dependency relation expresses a part of syntactic and semantic characteristics of the sentence, and can be strongly in relation to the intentional content.

it can be expected that two utterances whose dependency relations are similar each other have a high possibility that the intentions are also so.


Then, in our technique, the degree of similarity β, which has been explained in Section 3, is weighted based on the intentions of the utterances until it results in a user's utterance.

we consider the occurrence of a utterance intention I n at a certain time n to be dependent on the intentions of the last N − 1 utterances.


The weight assignment based on the intentions sequences is accomplished by reducing the value of the degree of similarity when the intentions N-gram probability is smaller than a threshold.

a formula (5) defines the degree of similarity γ using the weight assignment by intentions N-gram probability.


An in-car speech dialogue corpus which has been constructed at CIAIR (Kawaguchi et al.,  , was used as a corpus with intention tags, and analyzed based on Japanese dependency grammar (Matsubara et al., 2002).

the intention tags were assigned manually into all sentences in 412 dialogues about restaurant search recorded on the corpus.


We have divided 1,609 driver's utterances of 238 dialogues, which is not used for learning the intentions 2-gram probability, into 10 pieces equally, and evaluated by cross validation.

the inference of the intentions of all 1,609 sen- Figure 5: Relation between the weight coefficient ω and the accuracy tences was performed, and the recall and precision were calculated.


Other parsers rather than Minipar could be used.

we found out that Minipar always returns unambiguous trees, often making erroneous attachment decisions.


Updating removes all discourse entities from the DM which are not realized in the current utterance.

there is an assumption that referents mentioned in the previous utterance are much more salient than older ones.


We rely entirely upon the graph to deduce the relative importance of relationships.

our algorithm is designed to reduce so-called 'infections' (Roark and Charniak, 1998, §3) where the inclusion of an out-of-category word which happens to co-occur with one of the category words can significantly distort the final list.


Except for the unigrams, the shapes of the other TREC n-gram Zipfian curves are similar to but not quite the same as those for the English corpora.

the bigram curve for Mandarin is more curved than the English curve because there are more compound words in Mandarin than English.


To deal with this problem, we distinguish predicateargument examples, which are collected from a large corpus, by coupling a verb and its closest case component.

examples are not distinguished by verbs such as naru 'make/become' and tsumu 'load/accumulate', but by couples such as "tomodachi ni naru" 'make a friend', "byouki ni naru" 'become sick', "nimotsu wo tsumu" 'load baggage', and "keiken wo tsumu" 'accumulate experience'.


Their contribution accumulates as background noise in M(D(T)), which is the part to be offset by the baseline function.

if M(D(T)) were to exclude the contribution of non-typical words, it would not need to be normalized and would be more precise.


Now we can define SAL(D(T), s) using the saliency measure defined above and a parameter s ≥ 0: where DIFFNUM(X) stands for the number of distinct items in set X.

sAL(D(T), s) is the number of distinct words in D(T) whose saliency of occurrence is not less than s. For instance, using the 1996 archive of Nihon Keizai shimbun (a Japanese financial newspaper), sAL(D("Aum ３ "), 110) = 74, sAL(D("Aum"), 200) = 50, sAL(D*("do"), 110) = 1, and sAL(D*("do"), 200) = 0, where D*("do") is a set of N 0 randomly chosen articles from D("do") and N 0 is the threshold value stated in subsection 1.2.


By considering the meaning of the baseline function, we can solve the problem of determining the optimal value of saliency parameter s without approximating baseline functions.

since the baseline function is considered as background noise to be offset, the best situation should be that the baseline function is a constant-valued function while SAL(D(T), s) is a non-trivial function (i.e., not a constant function).


Second, we explore a novel use of Expectation-Maximization (Dempster et al., 1977) that iteratively reestimates a parsing model using the augmenting heuristics as a starting point.

the EM algorithm we use is a variant of the Inside-Outside algorithm (Baker, 1979;Lari and Young, 1990;Hwa, 1998).


We introduce a notion of light semantics, which corresponds to a restricted form of semantic interpretation for the document allowing exchange of information between the document and the knowledge base and permitting knowledge acquisition during the authoring process.

the knowledge gained during the authoring of a document can be re-used for simplifying the authoring of other documents.


Let's now look again at the abstract tree AT1: This abstract tree is valid relative to G1 (it corresponds to a possible complete derivation) but it is not necessarily valid relative to the combination <G1,KB1>; this notion is defined in the following way: because the abstract tree uniquely determines the set of rules which have been used for building the derivation, it also uniquely determines a set of associated KB constraints; thus AT1 is associated with the set of constraints: fdf(diprox,tablet), da(diprox,swallow)g. Now we say that AT1 is valid relative to the combination <G1,KB1> if and only if it is both valid relative to G1 and if its associated set of constraints is compatible with KB1.

we need to show that the addition of the two constraints df(diprox,tablet), da (diprox,swallow) to the ABOX still leads to a satisfiable knowledge base.


This means that when checking life/death we do not ever need to consider a repetitive derivation during the enumeration of derivations.

because we are dealing with a finite parameter domain, the derivations that we need to consider have a bounded depth (otherwise we would necessarily encounter repetitive situations), and the decidability of the process follows.


The selected keywords or title strongly reflect the author's wordings.

this approach is an "author-centered approach".


The other is that in the obligatory component the title explains the concrete content of the tech-nology while the headline expresses the purpose of the technology, which general readers can understand.

we identified two expressing techniques of newspapermen.


Our method relaxes the constraints for the purpose of robust dependency parsing.

our method considers that the bunsetsus, which don't have the head bunsetsu, such as fillers and hesitations, depend on themselves (relaxing the constraint that each bunsetsu depends on another only one bunsetsu).


• Each bunsetsu doesn't no more than one head bunsetsu.

our method considers the dependency structure whose probability is maximum to be the most plausible one.


We used the same data as that for our investigations in Section 2.2.

among all driver's utterance units of 81 dialogues, 100 turns were used for the test data, and 5978 turns for the learning data.


We conduct experiments that show that this goal appears to be realizable.

we discuss a natural language generation system that is composed of SPoT, a trainable sentence planner, and FER-GUS, a stochastic surface realizer.


In this section, we address the issue of the amount of effort that is required to port a sentence planner to new domains.


Finally, we have shown that a statistical NLG component can be integrated into a dialog system in real time.

we replace the hand-crafted surface generation of Communicator with FERGUS.


Whereas our system is generally able to return all reading-variants for a given kanji string and therefore provide the full set of translations for the kanji string, conventional systems return only the translation for the given reading.

with our system, the learner will be able to determine which of the readings is appropriate for the given context based on the translation, whereas with conventional systems, they will be forced into attempting to contextualize a (potentially) wrong translation.


In this paper, we will address three issues in the development of the Chinese Treebank: annotation speed, annotation accuracy and usability of the corpus.

we attempt to answer four questions: (i) how do we speed up the annotation process, (ii) how do we maintain high quality, i.e.


To monitor our annotation accuracy and interannotator consistency, we randomly selected 20% of the files to double annotate.

for these files, each annotator annotates them independently.


Foo and Li (2001) have conducted a series of experiments which suggests that the word segmentation approach does indeed have effect on IR performance.

they observe that the recognition of words of length two or more can produce better retrieval performance, and the existence of ambiguous words resulting from the word segmentation process can decrease retrieval performance.


The remaining two algorithms were selected because they can both be altered by simple parameter settings to obtain different word segmentation accuracies.

the second Chinese word segmenter we investigated was the minimum description length algorithm of Teahan et al.


In investigating this phenomenon, one finds that the performance drop appears to be due to the inherent nature of written Chinese.

in written Chinese many words can often legally be represented their subparts.


The coarse classifier determines a set of preferred labels, ½ Then each coarse class label in ½ is expanded to a fixed set of fine classes determined by the class hierarchy.

suppose the coarse class is mapped into the set ½ ¾ Ñ of fine classes, then ¾ Ë ¾ ½ .


For the sake of the experimental comparison, we define six feature sets, each of which is an incremental combination of the primitive feature types.

feature set 1 (denoted by Word) contains word features; feature set 2 (Pos) contains features composed of words and pos tags and so on; The final feature set, feature set 6 (RelWord) contains all the feature types and is the only one that contains the related words lists.


Identifying the labels of entities and relations is treated here as the target of our learning problem.

we learn these target properties as functions of all other "simple to acquire" properties of the sentence.


In the context of this work, for algorithmic reasons, we model only some of the conditional probabilities.

the probability P (R ij |E i , E j ) has the following properties.


Under this framework, finding the most suitable coherent labels becomes the problem of searching the most probable assignment to all the E and R variables.

the global prediction e 1 , e 2 , ..., e n , r 12 , r 21 , ..., r n(n−1) satisfies the following equation.


We use softmax (Bishop, 1995) over the raw activation values as probabilities.

suppose the number of classes is n, and the raw activation values of class i is act i .


The structure of our belief network, which represents the constraints is a bipartite graph.

the variable E's and R's are the nodes in the network, where the E nodes are in one layer, and the R nodes are in the other.


All the experiments of these approaches are done in 5-fold validation.

these datasets are randomly separated into 5 disjoint subsets, and experiments are done 5 times by iteratively using 4 of them as training data and the rest as testing.


All experiments were performed with 10-fold cross validation.

90% of documents are kept as a training set while the rest 10% are used for testing.


The ECG formalism is designed to allow constraints on x-schema simulation to be expressed.

the Event schema in Figure 3 has roles that refer to event phases; these correspond to x-schema places and transitions.


In order to perform an accurate mapping the toolkit needs to get information such as the sentence type, the main predicate, the arguments of the predicate, and the modifications of the predicate and arguments from a sentence.

it mostly needs local dependency relationships.


Table 2 shows that the error detector mistakenly judges some wrong trees as correct, but not the other way round.

when the detector says a tree is wrong, we have high confidence that it is indeed wrong, but when it says a tree is correct, there is some chance that the tree is actually wrong.


To address this problem, we assigned a penalty frequency of AbsFreq = 0.1 to a lemma-pair in a paraphrase that does not appear in the dictionary.

the score of a paraphrase is divided by 10 for each of its lemma-pairs that is absent from the dictionary.


For the question answering task, under the same retrieval conditions, recall improved from 294 correct documents without paraphrasing to 337 with a maximum of 20 paraphrases per query.

the number of queries for which correct documents were retrieved improved from 169 to 182.


For instance, a value of 10 means that the score for an absent adjacent lemma-pair is 1/10 of the score of an absent non-adjacent lemma-pair.

the score of a paraphrase is divided by 100 for each absent adjacent lemma-pair.


This leads us to making the most of our mother language, even in conducting translation.

when we translate our language into a foreign one unfamiliar to us, we may try to paraphrase the source input into easier expressions we can translate.


(1993), paraphrasing is performed in these models because it is necessary to prepare for the subsequent bilingual process.

the paraphraser operates as a sub-module for successful transfer.


We want to show in this paper how previously proposed methods can be extended to and improved for specialized domains.

we will focus on the use and enrichment of multilingual thesauri, which, even though partially related they may be to the texts under consideration, are nonetheless an available and valuable resource for the task.


The ↑-arrow refers to the f-structure of Figure 1: LFG c-structure and f-structure analysis of Maria liest oft Bücher the mother node (= the CP), the ↓-arrow to the fstructure of the node itself (= NP, C ).

2 the above rule states that CP dominates an NP and a C node; the NP functions as the subject (SUBJ) of CP, and C is the head of CP (sharing all features, by unification of their respective fstructures).


Modules serve to encapsulate data and are "black boxes" to each other.

the input and output of each module (i.e.


they perform similar actions on the input).

the code is structured according to functional considerations.


Moreover, macros/templates can be nested, and can thus be used to model constraints similar to type hierarchies (Dalrymple et al., To Appear).

nPfunc is used to encapsulate the alternative nP functions in German.


In contrast to documentation of other types of software, grammar documentation has to focus both on "why" and "how", due to the fact that in a grammar implementation the code in and of itself represents important information, as argued above.

the details of the input-output mapping represent the actual linguistic analysis.


In contrast to grammar modules, however, these modules are (usually) black boxes.

a reader of the documentation is not forced to follow all the links documentation should be independent of the structure of the grammar code.


Moreover, documentation may refer to partial rules only (or macros).

the complete documentation of an entire rule can be spread over different sections of the documentation.


User-friendly documentation therefore has to include an index that associates a grammar rule (or macro) with the documentation sections that comment on this rule.

besides referencing from the documentation to the grammar (by copying), the documentation must also support referencing (indexing) from various parts of the grammar to the relevant parts of the documentation.


However, in many NLP tasks, the goal is to predict the state sequence given the observation sequence.

the generative approach inappropriately applies a generative joint probability model for a conditional probability problem.


In principle, given an observation sequence , the goal of a conditional probability model is to find a stochastic optimal By applying the Bayes' rule, we can rewrite the equation (1) as: Obviously, the second term MI captures the mutual information between the state sequence and the observation sequence o .

to compute efficiently, we propose a novel mutual information independence assumption: we assume a state is only dependent on the observation sequence o and independent on other states in the state sequence s .


The main difference between a GHMM and a LSD-DHMM lies in their output models in that the output model of a LSD-DHMM directly captures the context dependence between successive observations in determining the "hidden" states while the output model of the GHMM fails to do so.

the output model of a LSD-DHMM overcomes the strong context independent assumption in the GHMM and becomes observation context dependent.


For efficiency, we can always assume , where the pattern entry .

we only consider the observation dependence in a window of 2N+1 observations (e.g.


In this paper, we focus exclusively on inferring high-level structure.

we investigate ways to automatically determine the correct attachment site for paragraph and multi-paragraph segments.


It remains to be seen how improving part of speech tagging, syntactic analysis and discourse modeling can influence the outcome of the task.

some work needs to be done to evaluate the detection of temporal adjuncts, a major source of information in the process.


Linearity is a restriction on CFTGs that requires the number of occurrences of every variable in the right-hand side of a rule be no more than 1, and nondeletion requires all variables in the left-hand side of a rule occur at least once in the right-hand side.

linear, nondeleting, monadic CFTGs are those with nonterminals of rank 0 and 1 only and with exactly one occurrence of a variable in every right-hand side of a rule for a nonterminal of rank 1.


The ngram model captures the short distance context dependency within an n-words window while the MI model captures the context dependency between the word pairs over a long distance by using the concept of mutual information.

the MI-Ngram model incorporates the word occurrences beyond the scope of the normal ngram model.


Compared with the normal word ngram model, the novel MI-Ngram model also incorporates the long distance context dependency by computing the mutual information of the distance dependent word pairs.


Obviously, Equation (3.12) takes the joint probability into consideration.

those frequently occurring word pairs are more important and have much more potential to be incorporated into the MI-Ngram model than less frequently occurring word pairs.


Example-based transfer utilizes the source examples for disambiguation of mapping and parsing.

the semantic distance (Sumita and Iida, 1991) is calculated between the source examples and the headwords of the input sentence, and the transfer rules that contain the nearest example are used to construct the target tree structure.


The most appropriate sequence is determined from the product of the translation model and the language model in the same manner as statistical MT.

when F and E denote the channel target and channel source sequence, respectively, the output word sequenceÊ that satisfies the following equation is searched for.


When the translation probability is calculated, the source word sub-sequence is obtained by tracing transfer mapping, and the applied translation model is restricted to the source sub-sequence.

the translation probability is locally calculated between the corresponding phrases.


Bottom-up generation locally applies the translation model to a partial tree.

the probability is calculated without word alignment linked to the outside of the tree.


In such a case, by splitting the input, translation may be successfully performed for each portion.

in a dialogue, sentences tend not to have complicated nested structures, and many long sentences can be split into mutually independent portions.


The measure is defined as Score in equation 5, where λ, ranging from 0 to 1, gives the weight of Sim.

the method uses only P rob when λ is 0, and the method generates candidates by P rob and selects a candidate by only Sim when λ is 1.


However, newswire does not offer direct access to facts, events, and opinions; rather, journalists report what they have experienced, and report on the experiences of others.

facts, events, and opinions are filtered by the point of view of the writer and other sources.


Wiebe (1994), studies methods to track the change of "point of view" in narrative text (fiction).

the "writer" of one sentence may not correspond to the writer of the next sentence.


Therefore, rather than trying to handcraft a more complex collection of heuristics, we chose to adopt a supervised machine learning approach that relies on features identified in this analysis.

we will first train a binary classifier to make pairwise decisions as to whether a given pse is the immediate parent of another.


We therefore include several features that encode the relative position of pse parent and pse target in the sentence.

we add a feature that is 1 if pse parent is the root of the parse (and similarly for pse target ).


Discourse structure relations here refer to informational relations that hold between sentences or other non-overlapping segments in a discourse monologue.

discourse structure relations reflect how the meaning conveyed by one discourse segment relates to the meaning conveyed by another discourse segment (cf.


Another important question is whether certain types of coherence relations participate more or less frequently in crossed dependencies than other types of coherence relations.

the question is whether the frequency distribution over types of coherence relations is different for arcs participating in crossed dependencies compared to the overall frequency distribution over types of coherence relations in the whole database.


We refer to PRECISE's use of semantic information to correct parser errors as semantic over-rides.

pRECISE detects that an attachment decision made by the parser is inconsistent with the semantic information in its lexicon.


These results show that the titles composed using the wizard could stimulate the lay readers' interest the most among each subject's seven titles.

the authors can stably compose the more appealing titles for lay readers with using the wizard than without using the wizard.


As a result, the titles composed using the wizard can stimulate the lay reader's interest more than the titles composed without using the wizard.

our title revision wizard can help the users who cannot compose effective titles by their own effort.


LP Locality: Precedence statements only constrain the order among elements within the same compacted domain.

precedence constraints cannot look into a compacted domain.


The SC-structure is still mostly undetermined, but by the co-and contra-dominance principles, the solver has already established that some nodes must dominate others: A dotted edge with label s in the picture means that the solver knows there must be a path between these two nodes which starts with an s-edge.

the solver has computed a large amount of semantic information from an incomplete syntactic analysis.


Note that we do not formally distinguish event variables from individual variables.

l(with) can be applied to either nouns or verbs, which both have type e,t .


Using alignment clues, this can be expressed as the following optimization problem: is the combined clue value for the linked items s n and t a D n .

word alignment is the search for the best link for each source language word.


On the Canadian Hansards task, we achieved further significant reduction of the AER.

we reached an AER of 6.6% by performing only the HMM training.


The main idea of most supervised learning approaches is to recast this task as a binary classification problem.

a classifier is learned and then used to determine whether or not two NPs in a document are co-referring.


Recognition failure often involves pronouns in contexts which are not covered by the discriminating rules.

dar fails to resolve singular neuter gender pronouns with distant antecedents and to identify vague anaphors, because it always "finds" an antecedent in the context ranking.


While having a simple (though non-standard) definition, this function yields improved performance relative to the two suggested evaluation criteria -for vector quality and for word similarity.

the underlying idea is that a good characteristic feature for a word w should characterize also multiple words that are highly similar to w. such feature should have a substantial "focus" within the close semantic vicinity of w. Applying RFF weighting achieved about 10% improvement in predicting meaning entailing substitutability (Section 5).


Yet, similarity measures that utilize MI showed good performance.

a common practice is to filter out features by minimal frequency and weight thresholds.


The semantic neighborhood of w, denoted by N(w), is defined as the set of all words v which are considered sufficiently similar to w, satisfying sim(w,v)>s where s is a threshold (0.04 in our experiments).

rFF is then defined by: we identify all words v that are in the semantic neighborhood of w and are also characterized by f and sum their similarities to w. Notice that rFF is a sum of word similarity values rather than being a direct function of wordfeature association values (which is the more common approach).


Disambiguation rules, as well as the rules for syntactic mapping (not discussed here) and for identifying idioms, were written within the framework of constraint grammar by using the CG-2 parser 5 .

morphological disambiguation and semantic disambiguation were implemented within a single rule system.


It sometimes happens that linguistic disambiguation rules cannot be written.

problematic is the noun of the Class 9/10 in object position without qualifiers, many of which would help in disambiguation.


Instead of building a word dependency corpus from scratch, we use the standard data set for comparison.

we use Penn Treebank's Wall Street Journal data (Marcus et al., 1993).


the sentence looks like: The following features of neigbors are also used as w i 's features.


The novel parsing strategy of allowing the grammar to decide if the supertagging is likely to be correct suggests a number of interesting possibilities.

we would like to investigate only repairing those areas of the chart that are most likely to contain errors, rather than parsing the sentence from scratch using a new set of lexical categories.


Phrase marked categories can only combine with other phrase marked categories, and hence avoid combination over intonational phrase boundaries.

boundary tones function like "stoppers" of theme and rheme categories, preventing theme and rheme to be further spread along sub-phrases of the sentence.


We choose polarization followed by destructuring as abstraction.


The basic idea of this RSCM is to roughly calculate the word posterior probability by using ranks of MT outputs in the N-best list of an SMT system.

the ranks of probabilities of MT outputs in the N-best list were used instead of the probabilities of the outputs themselves, as in the non-parametric statistical test.


This sorted mixture is alternatively used instead of the system's N-best list in the existing RSCM.

the proposed RSCM checks whether it accepts/rejects each top MT output in the original M-best lists by using the sorted mixture; on the other hand, the existing RSCM checks whether it accepts/rejects the top MT output in the system's N-best list by using the system's N-best.


In order to plot the ROC Curve, the authors conducted the same experiment as shown in Figure 1.

in the case where the grade of satisfactory translations is only grade A, each of the proposed and existing RSCMs tried to accept grade A MT outputs and to reject grade B, C, or D MT outputs.


If <priest> is used for "gobou" in a cooking domain text, though "gobou" means 'burdock' (a genus of coarse biennial herbs) in its context, "gobou" is identified as an agent.

"gobou" is incorrectly analyzed as antecedents of the following nominative zero pronouns.


To deal with this problem, predicate-argument examples which are collected from a large corpus are distinguished by coupling a verb and its closest case component.

examples are not distinguished by verbs (e.g.


The closest case component plays an important role to determine the usage of a verb.

when the closest case is "wo" or "ni ", this trend is clear-cut.


The result of case analysis tells if the zero pronouns exist.

vacant case slots in the case frame, which have no correspondence with the input case components, mean zero pronouns.


In addition, the verb sense disambiguation results are cached and applied globally in the same text.

the selected case frames are cached for each verb, and only the case frames that are similar to the cache are used for the same verb following in the same text.


On the other hand, our proposed method deals with only the similar case frames to the cached "orosu (1)".

the case frame "orosu (2)", which means "withdraw money from bank or post", is not similar to "orosu (1)", and is not used.


Furthermore, the extended MBN will provide opportunities for a new type of research in the field of consumer health.

we envisage experiments that investigate how the domain of medical phenomena is conceptualized by non-expert human subjects.


It has been reported that stochastic parsers degrade in performance on domains different than what they were trained on (Hwa, 1999;Gildea, 2001), so there really was an issue whether the output would be good enough.

we are taking the Collins parser trained on the Wall Street Journal and applying it unchanged to spontaneous human-human dialog in an emergency rescue task domain.


However, 82% of the sentences have no crossing brackets in the Collins parse.

while the parser may not generate the same set of constituents, it generates very few constituents that straddle the boundaries of any constituent in the TRIPS parse.


This is the case with many simple features, such as words and sentence position.

then, we define the total information covered by any given subset S of t (a proposed summary or answer) as where w i is the weight of the concept c i and the information contained in a summary is the sum of the weights of the conceptual units covered by at least one of the textual units included in the summary.


The results confirm empirically that the (r) = 0 condition (1) gives the most consistent results, and (2) generally yields accuracies among the highest, regardless of how long training is allowed to continue.

the presence of any negative application during the training phase will cause the error corrector to behave unpredictably, and the more complex model of greater error tolerance is unnecessary in practice.


Rule applications only update the output classification of a sample, and do not update the current state of the corpus.

the feature values for a sample are initialized once, at the beginning of the program, and not changed again thereafter.


In Step 5 of the algorithm, the current state of the corpus is updated with the latest rule on each iteration.

in each given iteration of the outer loop, the learner considers the corrected training data obtained by applying rules learned in the previous iterations, so the learner has access to the labels that result from applying the previous rules.


cally supervised", in fact this supervision comes in the form of the annotation of the Wall Street Journal treebank on which the parser is trained, rather than parses for our parallel training corpus.

the text we are parsing has a different vocabulary and style of prose from the WSJ treebank, and often the fluency of the English translations leaves something to be desired.


Rather than attempting an automatic disambiguation, however, ConAno merely makes suggestions to the human analyst, which she might follow or discard.

conAno loads a list of (potential) connectives, and when annotation of a text begins, highlights each candidate so that the user can either confirm (by marking its scope) or discard (by mouse-click) it if it not used as a connective.


To handle the ASR results of disfluent utterances, we employ SRI's Gemini robust language parser (Dowding et al., 1993).

we use Gemini to retrieve the longest strings of valid S and NP fragments in each ASR result.


The system accomplishes this by extending the CIA to maintain multiple competing Information States.

the DMT has been extended to allow for the same parse to attach in multiple locations-these multiple attachments are eventually pruned as more evidence is accumulated in the form of further speech or gestures-that is, as hypotheses are confirmed or rejected over time.


These tables show that sentences are often split and reconstructed.

multiple document summarization data exhibit    very complex correspondence because various summarization techniques such as sentence compaction, sentence combination, and sentence integration are used.


What our method is doing for unknown words can be interpreted as follows: The method examines all possible unknown words in a sentence, and probability for an unknown word of length k, w i =   where h is a history of the sequence.

the probability of the unknown word is approximated by the product of the probabilities of the composing characters, and this calculation is done in the framework of the word-level Markov model-based method.


In this paper, we report on a preliminary study where we cluster discussions carried out in electronic newsgroups.

we report on the influence of the above features on the clustering process, and describe a filtering mechanism that identifies and removes undesirable influences.


Secondly, the fact that the performance with filtering converges for such a low value of suggests that there is little true topical overlap between the newsgroups, and the fact that the performance is significantly better for suggests that there is some overlap, possibly created by a sub-topic of one of the newsgroups.

although there are only three newsgroups, four centroids are better at finding them than three centroids, because the fourth centroid may correspond to an overlap region between two clusters, which then gets assigned to the correct newsgroup.


Retrieval performance was measured as follows: correct documents in the selected cluster total correct documents in the dataset where a correct document is one that contains all the words in a query, and the selected cluster is that whose centroid has the highest average value for the query terms.

if a query comprises the words , and cluster 7 has a centroid value © ¢ for word , then the cluster that best matches the query is the cluster Our measure for retrieval performance considers only recall (i.e.


It does not have a precision component, because the system retrieves only documents that contain all the words in the query.


Intuitively, key terms in a document are some conceptual terms that are prominent in document and play main roles in discriminating the document from other documents.

a key term in a document can represent part of the content of the document.


We aimed at the high precision extraction of sentiment units.

our SA system attaches importance to each individual sentiment expression, rather than to the quantitative tendencies of reputation.


Using the translation architecture, any forms can be chosen as the predicates and arguments by adjusting the patterns and lexicons.

monolingual word translation is done in our method.


Two types of factors, 'filters' and 'preferences', were discussed in detail.

their restrictive potential and effect on success rate were assessed on the evaluation corpus.


For example, coreference resolution is a key task in natural language interfaces, machine translation, text summarization, information extraction and question answering.

information extraction systems like those built in the DARPA Message Understanding Conferences (MUC) have revealed that coreference resolution is such a crucial component of an information extraction system that a separate coreference task has been defined and evaluated in MUC-6 (1995) and MUC-7 (1998).


2) The definite noun phrase will be coreferential with a named entity if the head word of the definite noun phrase is only modified by the determiner "the".

the definite noun phrase is of type "the HEADWORD", e.g.


In our strategy, this is done through a common preference agent based on the principle of proximity.

our common preference agent takes advantages of the relative locations of the remaining antecedent candidates in the text.


Another objective of this work is to examine how a spelling system can be implemented using a normal Thai continuous speech corpus.

as the preliminary stage, we investigate the effects of spelling using such existing corpus.


Suppose the first sentence is ranked A, the second B, and the third C in Table 1  Sometimes question-answering (QA) by human subjects is used for evaluation (Morris et al., 1992;Hirao et al., 2001).

human subjects judge whether predefined questions can be answered by reading only a machine generated summary.


This means that the extracted sentences include redundant ones.

the difference between "Precision" and "Coverage" is large in "Pattern."


In the test phase, we apply both the hard and soft pattern rules to match against test instances.

soft matching pattern rules would assign a probabilistic score to an instance that is not matched by any of the hard matching pattern rules.


Many hard pattern rule inductive learning systems have been developed for information extraction from free texts or semi-structured texts.

autoSlog-TS (Riloff, 1996) generates extraction patterns using annotated text and a set of heuristic rules and it eliminates the dependency on tagged text and only requires the pre-classified texts as input.


As a subset of email management, we address the problem of constructing a summary of email discussions.

we examine ongoing discussions which will ultimately culminate in a consensus in a decision-making process.


However, in contrast to a conventional sentence extraction summary in Figure 1, the ideal summary ought to provide sufficient information about the current state-of-affairs of the discussion, in order to minimize any further delay in the matter being resolved.

this might include a description of the matter being discussed and the responses received so far.


(2003) in testing their supervised tagging algorithm.

we allocated sections 00-18 for training, 19-21 for development, and 22-24 for testing.


We then simulated, in a similar fashion, the construction of higher quality lexicons by using relative frequencies of tags for each word from the tagged Treebank to limit allowable word-tag assignments.

tags that appeared the tag of a particular word less than X% of the time were omitted from the set of possible tags for that word.


First, we used our unfiltered lexicon along with our tagged corpus to extract non-ambiguous tag sequences.

we looked for trigrams in which all words contained at most one possible part-of-speech tag.


We translated Chinese words into English.

chinese is the source language and English is the target language.


If a word e in English is indeed the translation of a word c in Chinese, then we would expect e to be ranked very high in both lists in general.

our combination method is as follows: we examine the top M words in both lists and find k e e e ,..., that appear in top M positions in both lists.


Since we are using comparable corpora, it is possible that the translation of a new word does not exist in the target corpus.

our experiment was conducted on comparable corpora that are not very closely related and as such, most of the Chinese words have no translations in the English target corpus.


The sign + means that the identified pattern is of type verb-direct object: the graph can then be applied to deal with passive structures.

the graph can only be applied in a sign + appears in the @ A column of the constraints table.


Here, N is the same number in the second column.

the compression ratio of the VBS and lead methods was standardized, and we compared the coverage of both methods.


It may be argued that the compression ratio is exaggerated.

although paragraphs ranked higher than 50 can potentially provide the sufficient viewpoints, the top 50 paragraphs were always used to calculate the dominator of the compression ratio.


Since Japanese is a head-final language, the order of linguistic expressions for groups are retained in the final linguistic expression for the SOG representation.


We presumed that, when a speaker tries to narrow down an object group from G i to G i+1 , there is an optimal ratio between the dimensions of G i and G i+1 .

narrowing down a group from a very big one to a very small one might cause hearers to become confused.


Output the triggered vocabulary set T ; We use the similar strategy with Ponte language model method (Ponte and Croft 1998) to compute the similarity between the query and the document.

we firstly construct the simple language model according to the statistical information of vocabulary and then compute the generative probability of the query.


tense and aspect in Chinese are expressed using a combination of, for example, time words, auxiliaries, temporal position words, adverbs and prepositions, and particular verbs.


Elements of b(y) that cannot be translated into elements of a'(y) constitute a residual second-language vector, denoted as b'(y) = (b' 1 (y), b' 2 (y), …, b' n (y)).

the similarity of candidate translation equivalent y to target word x is then defined as Note that a'(y)+b'(y) is a concatenation of a'(y) and b'(y) since they have no elements in common.


For a first-language word association (x, x'(j))-where a target word is given as x and its j-th associated word is given as x'(j)-a set consisting of second-language word associations alignable with it, denoted as Y(x, x'(j)), is constructed.

where R 2 is the collection of word associations extracted from a corpus of the second language, and D is a bilingual dictionary to be adapted.


An accompanying associated word is a word that is associated with both words making up the word association in question.

where R 1 is the collection of word associations extracted from a corpus of the first language.


A translingually alignable accompanying associated word is a word that is an accompanying associated word of the first-language word association making up the alignment in question and, at the same time, is alignable with an accompanying associated word of the second-language word association making up the alignment in question.


The plausibility factor is defined as the weighted sum of two component plausibility factors.

where α is a parameter adjusting the relative weights of the component plausibility factors.


The first component plausibility factor, PL 1 , is defined as the sum of correlations between the translation equivalent and the accompanying associated words.

x Select translation equivalents to which more than a certain percentage of associated words are assigned Figure 2: Bilingual-dictionary adaptation using the ratio of associated words word usually correlates closely with the translation equivalent that correlates closely with a majority of its accompanying associated words.


The second component plausibility factor, PL 2 , is defined as the maximum plausibility of alignment involving the translation equivalent, where the plausibility of alignment of a first-language word association with a second-language word association is defined as the mutual information of the second-language word association multiplied by the sum of correlations between the translation equivalent and the translingually alignable accompanying associated words.

x This is based on the assumption that correct alignment of word associations is usually accompanied by many associated words that are alignable with each other as well as the assumption that alignment with a strong word association is preferable to alignment with a weak word association.


The above definition of the correlations between translation equivalents and associated words is recursive, so they can be calculated iteratively.

initial values are set as the mutual information between the target word and an associated word is used as the initial value for the correlations between all translation equivalents of the target word and the associated word.


It should be mentioned that the above evaluation was done under strict conditions.

two out of three pseudo senses of each pseudo target word were assumed to occur in the corpus, while many real target words have only one sense in a specific domain.


The method is thus evaluated by recall and precision of selecting pseudo senses.

where S' is a set consisting of pseudo senses corresponding to the first two constituent words, and T' is a set consisting of pseudo senses relevant to translation equivalents selected by the method.


The average number of "true analogies" per sentence over all the corpus is: 162, 318 / 68, 184 = 0.42.

in this corpus, one sentence is involved in about half a "true analogy" in average, taking it for granted that the linguistic differences between Chinese, English and Japanese filter real oppositions in meaning out of the oppositions captured by analogies of form.


This is because sentence b is written on the presupposition that the reader may know that Dolly had a child.

it is more fitting to assume sentence b to be an elaboration of sentence c. As one may easily imagine, there are some precedent sentences prior to sentence b in the original document.


Instead of using a syntactically motivated co-occurrence approach as above, our system uses lexico-syntactic rules.

it finds lexico-POS patterns by making modifications to the basic edit distance algorithm.


If nothing about the data is known, estimating a probability distribution using the principle of maximum entropy involves selecting the most uniform distribution where all events have equal probability.

it means selecting the distribution which maximises the entropy.


The intuition behind Gaussian priors is that the parameters in the maximum entropy model should not be too large because of optimization problems with infinite feature weights.

: we enforce that each parameter will be distributed according to a Gaussian prior with mean µ and variance σ 2 .


The cognitive relevance of the UER is outlined in Section 3 -graphical modeling elements in general represent prominent kinds of concepts, or, respectively, structural or metaconcepts.

we will discuss the importance of these graphical modeling elements for the cognitive adequacy of representational frameworks such as the UER.


Roughly, verbs encode events or similar entities, entities that are called eventities within the UER.

1 the semantics of a verb or, as in most cases, one of its readings, corresponds to the eventity that is encoded by the verb.


The specification of both an ontological category as well as a property (technically captured in the UER in form of a type-expression and an attribute) rules out that SLEEP, as an eventity, or a stone, as an inanimate ineventity, could be potential undergoers of the WAKE UP 1 eventity.

the participant class specification works like a filter on the set of all entities, ruling out those entities which cannot be participants of the eventity in question because they do not fulfil the required characteristics.


The undergoer y essentially endures the same course of events, although this time the transition has a trigger in that it is caused by the active instigator, the actor.

the transition is triggered by a cause signal, the receipt of which is rendered in the concave pentagon in y's swimlane.


The signal sending is the result of an unspecified action state.

the actor performs some action (where action is broadly understood and does not necessarily entail intention), the nature of which is not conceptualized and irrelevant, thus leaving the specifics of the action state irrelevant (which is indicated by the underscore).


the cognitive structuring is reflected in the diagrammatic representation in a straightforward way, which is not the case in the linearized representation in (2).


2), but no content of the structure is given.

the exact concept is irrelevant and underspecified, the only thing that matters is structure: in Fig.


Examples in the UER we have seen are the octagon for eventities, the rectangle for participants, and the rectangle with rounded corners for passive simple states.

the structure is passed on into the graphical-visual domain, whereas the contents are kept in linearized form.


Thirdly, case studies applying the UER framework have shown a strong potential of the UER in capturing structural relations between different readings of one lexical item and also between different lexical items.

the UER is a practical tool for the study of semantic relationships.


In addition, it might be interesting to ask what degree of complexity eventity structures can maximally obtain while still being lexicalized in a single verb.


Pointwise MI between class c and feature f measures how much information presence of f contains about c: Gain Ratio (GR) is a normalized variant of Information Gain (IG), introduced into machine learning from information theory (Quinlan, 1993).

iG measures the number of bits of information obtained about presence and absence of a class by knowing the presence or absence of the feature 1 : Gain Ratio aims to overcome one disadvantage of iG which is the fact that iG grows not only with the increase of dependence between f and c, but also with the increase of the entropy of f. why features with low entropy receive smaller iG weights although they may be strongly correlated with a class.


Comparing discriminative and characteristic weighting functions we see that the supervised variants of TS frequently perform on a par with MI, GR, and OR.

tSL 2 was the best performer on Naive Bayes, BNC and the second best on kNN, AP.


Compared to relationally richer, e.g., WORDNET based, interlinguas as applied for cross-language information retrieval (CLIR) (Gonzalo et al., 1999;Ruiz et al., 1999), we use a rather limited set of semantic relations and pursue a more restrictive approach to synonymy.

we restrict ourselves to the specific sublanguage used in the context of the medical domain.


Since we tackle texts from administrative and legal domains we principally assume unspecified references.


In this paper, we explore the use of automatic syntactic simplification for improving content selection in multi-document summarization.

we show how simplifying parentheticals by removing relative clauses and appositives results in improved sentence clustering, by forcing clustering based on central rather than background information.


In contrast, the syntactic simplification literature deals with syntactic constructs that can be interpreted from a rhetorical perspective.

appositives and non-restrictive relative clauses are considered parentheticals in RST (Mann and Thompson, 1988).


On the other hand, human summaries contain only one parenthetical unit per 8.9 sentences on average.

human summaries contain fewer parenthetical units per sentence than the original reports; this appears to be a deliberate attempt at including more events and less background information in a summary.


By saving this information, we cannot only make use of information of sense selection for the sentence, but also information for this collocation.

the sentence S1 gives us two samples for different senses of the target word.


Optimal context size is criteria, target part-ofspeech and n-gram size dependent.


As shown in Figure 4, "SVM+NB" significantly outperformed "SVM+BL" for low frequency words, and the number of word types (N (o)) became obviously greater when o was small.

the Naive Bayes classifier proposed here could probably handle many more low frequency words than the baseline classifier.


Step (9d) must include extensive filtering procedures to check for shared semantic features, looking for commonality between the members.

there must be some threshold overlap between subgroups of the candidate lexical set and and the existing semantic classes.


Recognition of place names was done by an existing morphological analyzer.

the procedure discards the remaining m − k pairs in the ranking because they tend to include erroneous hypernyms.


We checked if the above alternatives can acquire the correct pairs of a hypernym and a hyponym obtained by HEAIH.

we counted how many correct pairs produced by HEAIH were also acquired by the alternatives when using the same document set.


From the data, we eliminated the pairs of a hypernym and an HCS that were not included in the top five for each hypernym in the ranking of the HEAIH output.

each hypernym could have a maximum of only five HCSs in the evaluation data.


Only when results are significant can we expect them to be reproducible, but even then a second experiment may draw a different picture.

the influence of qualitatively different source material or different evaluation criteria can never be predicted by statistical means alone.


Question A can be rephrased in the following way: How much does the observed precision value for an acceptance region A differ from the true average precision π A ?

our goal here is to make inferences about π A , for a given SF g and threshold γ.


It is obvious that a repetition of the evaluation experiment may lead to quite different precision values, especially for n < 1 000.

there is a considerable amount of uncertainty in the evaluation results for each individual measure.


(This question is normally asked when g 1 performed better than g 2 in the evaluation.)

our goal is to test whether π A > π B for given acceptance regions A of g 1 and B of g 2 .


In addition, it violates a desirable requirement of any similarity measure that the maximum similarity of 1 should only result when comparing identical words.

non-identical pairs 4 like Xanex/Nexan-where all bigrams are shared-are assigned a similarity value of 1.


Assuming that the segments appended to the beginning of each string are chosen according to the rule specified in Section 4.2, the returned value of BI-SIM always falls in the interval .

it returns 1 if and only if the strings are identical, and 0 if and only if the strings have no segments in common.


Table 2 (rows 2, 5 and 7) indicates that the number of POS trigram and bigram types common to both corpora was much smaller for the NEGRA-MED comparison than it was for NEGRA-NEWS.

more of the NEGRA POS n-gram types appeared in the NEWS corpus as well, whereas far less showed up in the MED corpus.


he checks the existence and mutual information values of phrases obtained by substituting the words with similar ones, which results in the classification of the phrase as being compositional or noncompositional.


In order to solve this problem, automatic summarization methods have been studied (Mani and T.Maybury, 1999).

the necessity for a multiple-document summarization has been increasing and the multiple-document summarization technology has been intensively studied recently (Mani, 2001).


To avoid a skewed recall, the number of test set words used in the corpus is given in table 2.

the recall can be calculated by the total number of words used in the corpus, but not by that recorded in the dictionaries.


However, the recall drastically increases from 45.56%, or 59.57% under the 40-word condition, to 85.03%.

the algorithm run with 20 Chinese paradigm morphemes resulted not only in high precision but also much higher recall than Chinese paradigm words in the same corpus.


Further, the more biased the measure is, the more highly ranked these high frequency words will tend to be.

there will be high overlap between neighbour sets generated considering all 2000 nouns as potential neighbours and neighbour sets generated considering just the nouns in WS high as potential neighbours.


Such a formulation permits use of a new and efficient algorithm for greedy morphological segmentation of the corpus in a recursive manner.

our method does not restrict words to be segmented only once, into a stem+affix form, as do many extant techniques.


We thus provide a search framework method which avoids some restrictions needed in previous work for efficiency.

our method segments words in the corpus recursively, enabling multiple morphs to be extracted from a single word, rather than just allowing a single stem+affix pair for a given word, as in many previous approaches.


Initial experimental results for the different models using our algorithm are given in Section 5.

as we show below, the key to efficiency is deriving local expressions for the change in coding length that will be caused by resegmentation on a particular prefix p. this coding length change, ∆CODE p , should depend only on direct properties of p, those morphs V p = {v k = ps k } for which it is a prefix, and those strings S p = {s k |ps k ∈ V p } (p's continuations).


The value is defined by the equation, Here, res(n) denotes a set of retrieval results for the n-th candidate, and |res(n)| denotes the number of elements in the set.

the significance score decreases if the retrieval results have a large common part.


This means that our method was effective not only for utterances with high ASR accuracy but also for those with around 50% accuracy.

an appropriate confirmation was generated even for utterances whose ASR accuracy was not very high.


Given input sequence X, the goal of statistical machine translation is to find the best output sequence, Y , that maximizes conditional probability P (Y |X): The problem of sentence boundary detection can be reduced to the problem of translating a sequence of words, X, that does not include periods but instead includes pauses into a sequence of words, Y , that includes periods.

in places where a pause might be converted into a period, which means P (X|Y ) = 1, the decision whether a period should be inserted or not is made by comparing language model scores P (Y ) and P (Y ).


A pause preceding or following surface expressions can be converted into a period.

pauses following expressions " (to)", " (nai)", and " (ta)", and pauses preceding expression " (de)", can be converted into a period when these pauses are longer than average.


In the future, we would like to solve the problems that we found in our experiments.

we want to reduce the number of errors due to inserted structures and solve other problems described in Section 2.1.


This framework extends previous work in Bayesian model merging for lexical acquisition (Bailey, 1997) and the induction of context-free grammars (Stolcke 1994) to handle the relational structures and usage-based considerations madepossible with ECG.

the criteria employed favor constructions that have simple descriptions (relative to the available meaning representations and current set of constructions) and are frequently employed.


The behaviour of the objects in cluster 0 (the biggest cluster by far) presents more cohesion than that of the objects in cluster 1, which have a medium mean value for most features.

binary adjectives do not have low values in those features that characterize unary ones, but still significantly lower.


In this paper, we explore the applicability of PageRank to semantic networks, and show that such graph-based ranking algorithms can be successfully used in language processing applications.

we propose and experiment with a new unsupervised knowledge-based word sense disambiguation algorithm, which succeeds in identifying the sense of all words in open text with a precision significantly higher than other previously proposed knowledge-based algorithms.


In this paper, we showed that iterative graphbased ranking algorithms -originally designed for content-independent Web link analysis or for social networks -turn into a useful source of information for natural language tasks when applied on semantic networks.

we proposed and evaluated a new approach for unsupervised knowledge-based word-sense disambiguation that relies on PageRankstyle algorithms applied on a WordNet-based concepts graph, and showed that the accuracy achieved through our algorithm exceeds the performance obtained by other knowledge-based algorithms.


Step 2 Concept extraction from the corpus: The node is defined as an operative node when the Tfidf value of word i W in the domain corpus is higher than that in its corresponding contrastive (out-of-domain) corpus.

Figure 2(a) represent the operative nodes.


The first row and first column of the matrix can be initially set to 0.

( , ) 0, m 0 n 0 Sim m n if or = = = Matrix filling: Assign the values to the remnant elements in the matrix as the following equation: There are some synonyms belonging to the same concept represented in one vertex.


The performance improvement was confirmed by consistent experimental results and measured by using various objective translation metrics.

bLEU score was improved by 7.9% absolute.


If one assumes an annotation strategy where Skogfoss and the predicative NP your parents' farm are marked as part of the same reference chain, then the descriptive content contributed by farm is added to the reference chain, which in turn makes it highly likely that the later expression that farm belongs to the same reference chain, as desired.

the descriptive content contributed by a predicative phrase -if assumed to be coreferential with its subjectcan be useful for identifying desired subsequent coreferences.


This is shown in (4b), where we see that an indefinite predicative NP can very well be preceded by a pronoun that describes the same individual as the predicative NP.

if one assumes that predicative NPs are anaphoric to their subject, one has to explain why predicative NPs do not have the expected forms of nominal anaphora.


Then an edge is added between each ∆-node and the ∆-nodes which represent the sub-senses of the next lower level.

the graph includes a tree of ∆-nodes which encodes the hierarchical structure of each entry.


Obligatory case information of nouns in an ordinary dictionary can be utilized to solve the difficult problem in the semantic analysis of "N m no N h " phrases.


For example, "rugby no coach" can be interpreted by the definition of coach as follows: the dictionary describes that the noun coach has an obligatory case sport, and the phrase "rugby no coach" specifies that the sport is rugby.

the interpretation of the phrase can be regarded as matching rugby in the phrase to some sport in the coach definition.


The top-5 lists show two interesting points: one is that precision is very high, and the other is that Fscore including role tagging is not much different from F-score of only FE identification.

there are a few (not 120) confusing roles for a given frame element, and we have many frame elements that are not identified even in n-best lists.


Importantly, grammar says nothing about how standards for positive gradable adjectives are derived.

contra Cresswell (1977) and others, the interpretation of adjectives is not computed relative to a grammatically-specified "comparison class" of related objects.


2004describes more recent advances made to both acoustic and language modeling for oral-reading recognition of children's speech.

that work describes the use of cross-utterance word history modeling, positionsensitive dynamic n-gram language modeling, as well as vocal tract length normalization, speakeradaptive training, and iterative unsupervised speaker adaptation for improved recognition.


The aim of this paper is to address the recall problem by using extraction methods that are linguistically more sophisticated than surface pattern matching.

we use dependency parsing to extract syntactic relations between entities in a text, which are not necessarily adjacent on the surface level.


Framing is not attempting to capture the entire meaning of the passage; it needs to be just sufficient enough to communicate with the user about the differences in their question and the returned text.

the framing process may uncover topics or aspects within the answer space which the user has not explicitly asked for, and thus may be unaware of their existence.


Despite the excellent performance of the supervised KPCA-based WSD model on average, though, our further error analysis investigations have suggested certain limitations.

the supervised KPCA-based model often appears to perform poorly when it encounters target words whose contexts are highly dissimilar to those of any previously seen instances in the training set.


On the other hand, if we are not confident about the supervised KPCA model's prediction, we then turn to the semi-supervised KPCA model and take its classification as the predicted sense.

the composite model uses the following algorithm to combine the sense predictions of the supervised and semi-supervised KPCA models in order to disambiguate the target word in a given test instance x: 1. let s 1 be the predicted sense of x using the supervised KPCA baseline model 2. let c be the similarity between x and its most similar training instance 3. if c ≥ t or s 1 = s mf (where t is a preset threshold, and s mf is the most frequent sense of the target word): • then predict the sense of the target word of x to be s 1 • else predict the sense of the target word of x to be s 2 , the sense predicted by the semisupervised KPCA model The two conditions checked in step 3 serve to filter those instances where the supervised KPCA baseline model is confident enough to skip the semi-supervised KPCA model.


Specifically, the composite model uses the following algorithm to combine the sense predictions of the supervised and semi-supervised KPCA models in order to disambiguate the target word in a given test instance x: 1. let s 1 be the predicted sense of x using the supervised KPCA baseline model 2. let c be the similarity between x and its most similar training instance 3. if c ≥ t or s 1 = s mf (where t is a preset threshold, and s mf is the most frequent sense of the target word): • then predict the sense of the target word of x to be s 1 • else predict the sense of the target word of x to be s 2 , the sense predicted by the semisupervised KPCA model The two conditions checked in step 3 serve to filter those instances where the supervised KPCA baseline model is confident enough to skip the semi-supervised KPCA model.

: • The threshold t specifies a minimum level of the supervised KPCA baseline model's confidence, in terms of similarity.


The threshold t is defined to rise as the relative frequency of the most frequent sense falls.

t = 1 − P (s mf ) + c where P (s mf ) is the probability of most frequent sense in the training corpus and c is a small constant.


Although the asserted type information for Water does not convey any properties that would suggest a mass noun lexicalization, the genls hierarchy of collections does.

the collection Chemical-CompoundTypeByChemicalSpecies is known to be a specialization of the collection ExistingStuffType, via the transitive properties of genls.


In our previous work (Tufiş and Ion, 2003), to approximate Z we imposed an exit condition for the clustering algorithm based on distance heuristics.

the algorithm stops when the minimal distance between the existing classes increases beyond a given threshold level: where dist(k) is the minimal distance between two clusters at the k-th iteration step and α is an empirical numerical threshold.


One of the greatest advantages of our approach is that it can be used to automatically sense-tag corpora in several languages at once.

if we have a parallel corpus in multiple languages (such as the Orwell corpus), disambiguation performed on any one of them propagates to the rest via the ILI linkage.


For 45.4% of the confirmed relations both the headword and its genus term were in the same Goi-Taikei semantic class.

many classes contain a mixture of class names and instance names: buta niku "pork" and niku "meat" are in the same class, as are ¡ doramu "drum" and dagakki "percussion instrument", which we can now distinguish.


As well as extending the coverage of the grammar, we are investigating making the semantics more tractable.

we are investigating the best way to represent the semantics of explicit relations such as 3 5 4 isshu "a kind of".


One such possibility is prepositional phrase attachment.

following only the grammar rules, many times a prepositional phrase could be an adjunct/ argument of several preceding components.


The PropBank project (Kingsbury and Palmer, 2002) provides a large human-annotated corpus of semantic verb-argument relations.

we use the data provided in the CoNLL-2004 shared task of semantic-role labeling (Carreras and Màrquez, 2003) which consists of a portion of the PropBank corpus, allowing us to compare the performance of our approach with other systems.


In this work, following the CoNLL-2004 shared task definition, we assume that the SRL system takes as input only partial syntactic information, and no external lexico-semantic knowledge bases.

we assume as input resources a part-of-speech tagger, a shallow parser that can process the input to the level of based chunks and clauses (Tjong Kim Sang and Buch-holz, 2000;Tjong Kim Sang and Déjean, 2001), and a named-entity recognizer (Tjong Kim Sang and De Meulder, 2003).


It incorporates several improvements over the basic Winnow multiplicative update rule.

a regularization term is added, which has the effect of trying to separate the data with a thick separator (Grove and Roth, 2001;Hang et al., 2002).


We use softmax (Bishop, 1995) over the raw activation values as conditional probabilities, and also the score of the target.

suppose the number of classes is n, and the raw activation values of class i is act i .


Therefore, the goal here is to output a superset of the correct arguments by filtering out unlikely candidates.

we learn two classifiers, one to detect beginning argument locations and the other to detect end argument locations.


If we assume that the classifier returns a score, score(S i = c i ), corresponding to the likelihood of seeing label c i for argument S i , then, given a sentence, the unaltered inference task is solved by maximizing the overall score of the arguments, (1) In the presence of global constraints derived from linguistic information and structural considerations, our system seeks for a legitimate labeling that maximizes the score.

it can be viewed as the solution space is limited through the use of a filter function, F, that eliminates many argument labelings from consideration.


If there is an R-XXX argument, then there has to be an XXX argument.

if an argument is a reference to some other argument XXX, then this referenced argument must exist in the sentence.


This is done by setting z to a set of indicator variables.

let z ic = [S i = c] be the indicator variable that represents whether or not the argument type c is assigned to S i , and let p ic = score(S i = c).


, S j k occupy the same word in a sentence, then this constraint restricts only one arguments to be assigned to an argument type.

k − 1 arguments will be the special class null, which means the argument candidate is not a legitimate argument.


What about locative uses of the pronominal adverb "y" 1 ?

when it is used to replace the locative prepositional complement of a verb (Locative Prepositional Phrase, or LPP), like in: (3)Cette ville est belle.


…considerations as nutrient availability.

the panel concluded that the fear of creating hazardous microorganisms, or pathogens, is overstated.


All the experiments with machine learning configurations were performed with 10-fold crossvalidation.

the question set was divided into 10 parts, and each experiment was repeated 10 times.


Given a new word, we use WordNet again to obtain a synonym set of the unseen word to determine how it interacts with our sentiment seed lists.

we compute where c is a sentiment category (positive or negative), w is the unseen word, and syn n are the WordNet synonyms of w. To compute Equation 1, we tried two different models: where f k is the k th feature (list word) of sentiment class c which is also a member of the synonym set of w, and count(f k ,synset(w)) is the total number of occurrences of f k in the synonym set of w. P(c) is the number of words in class c divided by the total number of words considered.


Clearly, the advantage of such classification relies on having the ability of extracting from the documents such instances.

a good question classification module may be useless if we lack an accurate named entity extractor for the document collection.


The reason most likely lies in the fact that the styles of the objective summaries written by humans vary a lot or even conflict with each other.

they do not possess many common features that are a must for high value of similarity between two texts.


They employed a CCG (Steedman, 2000) or LTAG (Schabes et al., 1988) parser to acquire syntactic/semantic structures, which would be passed to statistical classifier as features.

they used deep analysis as a preprocessor to obtain useful features for training a probabilistic model or statistical classifier of a semantic argument identifier.


Although the statistical approach seems a reasonable way for developing an accurate identifier of PropBank annotations, this study aims at establishing a method of directly comparing the outputs of HPSG parsing with the PropBank annotation in order to explicitly demonstrate the availability of deep parsers.

we do not apply statistical model nor machine learning to the post-processing of the output of HPSG parsing.


We prepared two grammars for the evaluation: ¡ penn was extracted from the Penn Treebank with the original algorithm (Miyao et al., 2004), and ¡ prop was extracted using the PropBank annotations for argument/modifier distinction by a method similar to Chen and Rambow (2003).

constituents annotated with ARG were treated as an argument in the grammar extraction.


"Labeled" here means the label of argument positions.

a predicateargument relation was judged to be correct if w as correctly output.


The "Gold parses" row represents the accuracy attained when correct HPSG derivations are given.

it represents the accuracy when Section 23 of the HPSG treebank was given.


Conventional parameter estimates are affected (in some way or other) by the likelihood function for a parameter, and the likelihood function is such that longer documents exert a greater influence on the overall likelihood for a parameter.

we note that if the true binomial parameter θ ij for the j-th word in the i-th document were known, then the most sensible expected value for the distribution over θ j would be: Whereas the expected value of conventional method-of-moments estimate is: a weighted mean of the maximum likelihood estimates of each of the θ ij , with weights given by p (θ ij ), i.e.


Since all elements of the proposed EVD-based approach can rely entirely on lexical statistics, hardly any price is paid for the improved empirical results.

the approach, like LSA or SMT, can still be generalized to virtually any language(s); computation of the EVD takes similar resources to that of the SVD since all the blocks are sparse; It is close to two decades now since Deerwester et al.


As with many problems, there is pressing need to find strategies for reducing the amount of supervision required for producing accurate supertaggers, but as yet, no one has explored the use of weak supervision for the task.

there are many dialog systems which rely on hand-crafted lexicons that both provide a starting point for bootstrapping a supertagger and which could benefit greatly from supertag pre-parse filter.


Here, I investigate such weakly supervised learning for supertagging and demonstrate the importance of proper initialization of the tag transition distributions of the HMM.

such initialization can be done using inherent properties of the CCG formalism itself regarding how categories 1 may combine.


POS tags are relatively inert, passive elements in a grammar, whereas CCG categories are the very drivers of grammatical analysis.

syntax is projected, quite locally, by lexical categories.


The sense with the highest overall similarity is the first sense.

the approach makes use of two similarity measures which complement each other and provide a large amount of data regarding the word senses.


We also created training data sets using collocational neighbors.

using the InfoMap toolkit 2 , we constructed vector-based representations for individual words from the BNC using a term-document matrix and the cosine similarity measure.


(3) How does the proposed approach compare against other unsupervised methods?

we are interested to find out whether we outperform Mc-Carthy et al.


This result may even point to a general characteristic of tutorial dialogue which makes computational modelling challenging.

the Neutral category resulted in conflicting annotations because it is often unclear, as in the examples shown above, whether Requests are being answered or merely addressed.


As such, we do not use corpora as a means of characterizing noun-compounds themselves, but as a means of characterizing the action possibilities of the individual nouns that can participate in a compound.

we attempt to characterize those properties of different concepts denoted by nouns to help predict how those nouns will combine with others, and through which relations.


The antecedent of an other-anaphor is a complement expression to the anaphor in a super set.

an other-anaphor is a set of elements excluding the element(s) specified by the antecedent.


We restrict the patterns so that no pattern span across two or more sentences.

the pattern shall not contain the symbol ".".


Our approach is based on synchronous tree substitution grammar (STSG, Eisner (2003)), a formalism that can account for structural mismatches, and is trained discriminatively.

we generalise the model of Cohn and Lapata (2007) to our abstractive task.


Cohn and Lapata (2007) extract a STSG from a parsed, word-aligned corpus of source and target sentences.

they extract the minimal set of synchronous rules which can describe each tree pair.


The analyses expose a number of challenges and several surprising results.

while ROUGE has very strong correlation with responsiveness for both human and system summaries, there is a significant gap in responsiveness between humans and systems which is not accounted for by the ROUGE metrics.


The instructions for assessing content responsiveness implicitly acknowledges this; what is surprising is the extent to which linguistic quality does influence content responsiveness.

we demonstrated that content responsiveness in DUC 2006 and 2007 correlated with the linguistic quality questions of grammar (Q1) and focus (Q4), and that systems were significantly penalized in content responsiveness when their summary ended Table 6: Correlation and p-values between content responsiveness and various metrics for each "Test" year of DUC.


In a subcorpus containing NP internal agreement errors, the average processing time per sentence increases from 8.3 seconds with the selective activation of MALrules to 31.4 seconds with the activation of all MAL-rules.

the MAL-rules modeling the error subject verb agreement are a source of ambiguity.


For instance, it always provides the minimum distance between two synsets.

the algorithm always provides an answer being the minimum distance close or far.


In this framework all knowledge resources are evaluated on a common WSD task.

we used the noun-sets of the English Lexical Sample task of Senseval-3 and SemEval-2007 exercises which consists of 20 and 35 nouns respectively.


All knowledge bases are evaluated as topic signatures.

word vectors with weights associated to a particular synset which are obtained by collecting those word senses appearing in the synsets directly related to the topics.


We evaluated KnowNet using the same framework explained in section 4.

the noun part of the test set from the English Senseval-3 and SemEval-2007 English lexical sample tasks.


Learners, however, although they do make this kind of mistake, are much more hampered by the confusion between for and to, as in She was helpful for me or This is interesting for you.

for learners it seems that the abstract use of this preposition, its benefactive sense, is much more problematic than the spatial sense.


The use of these patterns is motivated by the intuition that noisy expressions found with DNVs, such as expressions about locations or instruments, will also frequently appear with non-negated verbs.

if you observe "cannot go to Y (by / because of) X" and X is not a trouble expression, then you can expect to find "can go to Y (by / because of) X" as well.


As it serves as a basis for POS tagging, category induction, and human category acquisition, we investigate the information needed to disambiguate a word in a local context, when using corpus categories.

we increase the recall of an error detection method by abstracting the word to be disambiguated to a representation containing information about some of its inherent properties, namely the set of categories it can potentially have.


We do not know for sure whether frames induce coherent sets of words or whether they accurately disambiguate a word, or both.

can frames be used to group the target words (induction) or to group the contexts (disambiguation)?


The variation all CC/RB disappeared points to an error in the word but, yet there is also a noticeable inconsistency in the word all.

it is often the case that we should ignore the POS of the context words, due to the fact that erroneous contexts exist and, more importantly, that not all categories aid in disambiguation.


As mentioned by (Al-Onaizan and Papineni, 2006), it can be problematic that these deterministic choices are beyond the scope of optimization and cannot be undone by the decoder.

there is no way to make up for bad information in later translation steps.