Note, however, that R-Precision requires the number of gold labels per document to be known beforehand, which is unrealistic in practical applications.

we propose using R-Precision@K (RP @K), where ALL LABELS FREQUENT FEW ZERO RP @5 nDCG@5 Micro-F 1 RP @5 nDCG@5 RP @5 nDCG@5 RP @5 nDCG@  K is a parameter.


Evaluation metrics such as topic coherence, perplexity, and log-likelihood measure how well topics model data, but are not sufficient to measure whether user feedback is incorporated as expected.

we propose new control metrics to measure how well models reflect users' refinement intentions.


Our "good user" reflects a realistic user behavior pattern: identify a mixed category topic and apply refinements to focus the topic on its most dominant category.

the "good user"-with access to true document categories-first chooses a topic associated with multiple categories of documents and determines the dominant category of the top documents for the topic.


, K} is drawn according to the multinomial distribution θ and the word is then drawn according to the multinomial distribution β zn .

the marginal likelihood of the document p Given a document w, the inference task is to determine the conditional distribution p(θ|w).


For example, visible actions from our datasets such as "get up", "cut them in half", "getting ready", and "chopped up" cannot be correctly labeled using only object detection.

we need to use additional video information such as Inception and C3D information.


This is not surprising given that ELMo uses two bidirectional LSTMs and has improved the state-of-the-art in many NLP tasks (Peters et al., 2018).


However, in contrast to previous work, we realize this performance improvement without having to observe the segment boundaries.

our model may be applied straightforwardly to Chinese, where word boundaries are not part of the orthography.


Representing history To represent x <t , we use an LSTM encoder to read the sequence of characters, where each character type σ ∈ Σ has a learned vector embedding v σ .

the history representation at time t is This corresponds to the standard history representation for a character-level language model, although in general, we assume that our modelled data is not delimited by whitespace.


Finally, we also show that our model has superior performance, in terms of heldout perplexity, compared to a character-level LSTM language model.

overall, our results show that we can obtain good segmentations on a variety of tasks, while still having very good language modeling performance.


The categories where we see the largest improvement in recall correspond to those that are likely a priori to correlate most reliably with observable features.

this result is consistent with a hypothesis that the lexican is successfully acquiring knowledge about how words idiosyncratically link to visual features.


dataset (De Vries et al., 2017), a standard dataset for goal-oriented visual dialogue that focuses on identifying an undisclosed object in the image through a series of questions.

our contribution is threefold: • An end-to-end architecture for goal-oriented visual dialogue combining Information Gain with Reinforcement Learning.


For a given question q, the Regularized Information Gain is given as: where τ (q) is the expected information gain associated with asking a question and C(q) is the cost of asking a question q ∈ Q in a given game.

the information gain is measured as the KL divergence between the prior and posterior likelihoods of the scene objects before and after a certain question is made, weighted by a skewness coefficient β(q) over the same posterior.


We use the posterior distribution provided by the guesser to extract the Pearson's second skewness coefficient (i.e., the median skewness) and create the β component.

assuming a sample mean µ, median m, and standard deviation σ, the skewness coefficient is simply given by: Some questions might have a high information gain, but at a considerable cost.


Regularized information gain as a reward function can help the questioner to come up with an efficient strategy to narrow down a large search space.

the reward function is given by: the reward function is the sum of the tradeoff between the information gain τ (q) and the cost of asking a question C(q) for all questions Q in a given game.


in most neural speech translation, such as that of (Bojar et al., 2018), the input to the translation system is usually the 1-best hypothesis from the ASR instead of the word lattice output with its corresponding probability scores.


Compared to visual dialog (Das et al., 2017a,b;Strub et al., 2017;Mostafazadeh et al., 2017) tasks, agents need to additionally cooperate to change the environment with actions (e.g., move pieces around).

the agents have to possess the ability to adapt and hold a dialog about partially-constructed scenes that will occur over the course of their interactions.


However, this reward scheme alone has an issue: once all objects in the scene are described, any further messages will not result in a change in the scene and have a reward of zero.

there is no incentive to end the conversation.


When considering past history, we make the Markovian assumption that the current state of the Drawer's canvas captures all information from the previous rounds of dialog.

the Drawer need only consider the most recent utterance from the Teller and the current canvas to decide what to draw next.


We assume that the directly-related words can be determined based on the visual information, while the identification of words in the second group requires the consideration of their relationship with those in the first group.

we propose a two-step strategy to construct the image-grounded vocabulary.


The worst case of N-gram problem is that the model directly generated the same frequent captions in the training set (Devlin et al., 2015a).

the capability of generating novel captions for an image that is not seen in the training set indicates that the generator is able to understand a given image better instead of simply generating frequent Ngrams found in the training set.


Previous work on multimodal machine translation has shown that visual information is only needed in very specific cases, for example in the presence of ambiguous words where the textual context is not sufficient.

models tend to learn to ignore this information.


Previous works (Yin and Ordonez, 2017; have found explicit object detections to be informative for image description generation.

we base our intuition on the hypothesis that a thorough and true description of the image should consist of information about objects and their interactions (frequencies, etc.)


With 5 references (a feasible number from a practical perspective), VIFIDEL achieves the highest accuracy with a slight improvement in score over zero or one reference.

the object weighting scheme can be seen to help VIFIDEL focus on which objects are more important.


This happens because such additional objects do not seem too relevant to humans, as these are mostly not mentioned in the references.

the scores from VIFIDEL change with both number of annotations and the reference descriptions, while metrics like METEOR (0.354) and SPICE (0.320) would remain constant.


On the other hand, for the unaligned version, we keep the original audio and visual features as extracted, without any word-segmented alignment or manual subsampling.

the lengths of each modality vary significantly, where audio and vision sequences may contain up to > 1, 000 time steps.


Following the standard REINFORCE algorithm (Williams, 1992), the gradient for the expectation E n,sn∼π θ [R(s n )] in Equation 16 can be written as: is joint negative loglikelihood of sentence s n and its indicator idx n , and it can be decomposed as: where L AW , L N W and L P L are negative loglikelihoods; p AW , p N W and p P L are probabilities of taking an action; 1 denotes indicator function.

equation 17 can be re-written as: It is very hard to train agents using reinforcement learning from scratch, therefore a good initialization for policy network is usually required (Bahdanau et al., 2016;Silver et al., 2016;Wang et al., 2018b).


For reference, the average T T R noun of a human-written story (the entire VIST dataset) is 0.86.

we hypothesize workers prioritized their efforts in deleting repetitive words for AREL, resulting in the reduction of story length.


Since, to the best of our knowledge, this work is the first to introduce the task of weakly supervised relationship prediction solely using captions and bounding boxes, we do not have any directly comparable baselines, i.e., all other work is either completely supervised or relies on all ground truth entity-relation triplets being present at train time.

we construct baselines relying solely on captions and ground truth bounding box locations that are comparable to our task.


A straightforward modification of both algorithms is possible to ensure that these states are not present in the output.

efree t,c a a a (4 (5) Illustration of the difference in size between two variants of efree.


However, in these experiments all states in the input are co-accessible by construction; and moreover, all states in the input are final states.

the advantages of the pergraph t'c algorithm could not be observed here.


While the planner is specialized for planning communicative actions, it can in principle draw on anything in the system's general body of knowledge and belief that might support the goals it is given.

(as we shall see) in the second example in Section 4.3, the planner decides to mention the rain not because it bears any special relationship to anything in the input goal, but rather because mentioning it is part of a plan that achieves the goal.


The planner can reorganize the plan to take advantage of these kinds of situations.

12 the communicative plan can be modified based on linguistic information, again without violating the strict modularity of the generator.


This phenomenon is modeled in IGEN by having a special word #<w:IT> whose meaning can range over the various items constituting the weather; this word is constrained to appear only as the subject of an appropriate verb.

as we saw in Section 4.1, #<w: IT> is represented as meaning #<TEMPERATURE> and annotated appropriately.


The #<FEW-DAYS> sequence activated by the option #<w : TOMORROW>, on the other hand, largely overlaps with the plan's #<SEQUENCE>, so no adjustment is made.

#<w:TOMORR0W> becomes the preferred option.


2.5.2 Translation from G into P. As mentioned in Section 1 in relation to the scheme in Figure 3, the purpose of this translation is to draw the graphical symbols that are referred to in G. To picture the full map, the only symbols that must be drawn are the symbols of the base (Pbase), as emerging symbols do not have an independent pictorial realization.

the only translations that have to be defined are the translations of the symbols contained in the expression ~ (i.e., the map).


The syntactic and semantic structures of the three languages permit expression and interpretation of information in each of the modalities, and the ability to systematically find correlated expressions in different modalities with the same semantic values.

it is possible to state formally what it means to resolve a multimodal reference: according to this theory, to resolve a multimodal reference is to find the semantic value of an expression using either the information expressed in the modality or information expressed through other modalities with the help of the translation functions.


However, note that the expressions of P are the graphical symbols on the drawings and parsing a drawing (an expression of type map) produces a syntactic structure of P whose translation into G is the expression ~ (which we called the map).

emergent objects can also be represented in G as long as they can be produced from the base through syntactic rules of P and their translations into G. expressions of G that refer to graphical objects stand in a one-to-one relation with the corresponding objects in P. Although, theoretically, expressions in G and P are different representational objects, in actual interpretation processes they always come packed together.


Accordingly, if there are n names and m graphical objects, the first column of a function table can be filled up in m different ways, the second in m -1 different ways, and so on, until n graphical objects have been assigned.

each function table with n names and m graphical objects defines m!/(m -n)!


Clearly, the above characterization is very wide ranging.

fusion has been applied to a wide variety of pattern recognition and decision theoretic problems-using a plethora of theories, techniques, and tools--including some applications in computational linguistics (e.g., Brill and Wu 1998;van Halteren, Zavrel, and Daelemans 1998) and speech technology (e.g., Bowles and Damper 1989;Romary and Pierrel 1989).


The number of points given to a candidate for scoring strategy si is inversely related to its rank order on the basis of si.

the total number of points (T) awarded for each strategy is: where N is the number of candidate pronunciations (N = 6 in our longevity example, so that T(6) = 21).


The various combinations are denoted as a five-bit code where 1 at position i indicates that strategy si was included in the combination (i.e., 6si = 1 in Equations (4) and (5)) and a 0 indicates that it was not.

as an example, the code 00100 indicates that Strategy 3 (FSP) was used alone.


For the purposes of this article, however, words are the tokens generated by my tokenizers and stemmers for the languages in question.

the models in this article are only a first approximation to the vast complexities of translational equivalence between natural languages.


On the other hand, noise can reduce the strength of an indirect association without affecting any direct associations.

direct associations are usually stronger than indirect associations.


Our layout style does not allow a section to start on a page unless both the section header and two lines of section text can fit on the page.

increasing the size of this paragraph to four lines causes Word to start the section headed "You could do it ..." on the next page; this makes the leaflet overflow onto an additional page, and thus violate the overall size constraint.


Therefore, increasing the size of this paragraph to four lines causes Word to start the section headed "You could do it ..." on the next page; this makes the leaflet overflow onto an additional page, and thus violate the overall size constraint.

a very small change in a document (such as changing a few words from italics to bold) can cause significant changes in a document's size.


The fact that a Pipelines and Size Constraints document's size is so sensitive to its exact surface form is what makes size estimation difficult.

of such problems, although the size estimator soon grew in complexity considerably beyond what we had originally intended, it still made mistakes.


Stede is specifically interested in verbalizations of situations, to use his own neutral term.

the domain model contains a representation for categories such as states, activities, and events that Stede nicely synthesizes from linguistic work on aspect.


The third is a collection of evidence from Dutch and Swiss German (Huybregts 1976(Huybregts , 1984Shieber 1985) for the non-context-freeness of at least some natural languages.

while MSO may be a very elegant choice, it does not quite appear to be a sufficient choice.


The common trade-off between recall and precision is clearly present for the smaller window sizes, with the F-measure providing a kind of average.

far, we have applied a common collocation extraction technique to a semantic association task.


In fact, 4.27 is the maximum MI score for this corpus size (42,825) and the frequency of the seed term af (2,206), irrespective of the frequency of the target word, reached when all occurrences of the target word are concentrated in the window (see the appendix for details).

any hapax legomenon appearing in the window will automatically be assigned the maximum value of MI, along with target words with the most extreme W/C distributions (Window-Complement: 2-0, 3-0, 4-0, etc.).


traveling up at one end of an axiom link, continue down at the other end; traveling down at an (input) daughter of an input division link, perform the functional application of the result of traveling down at the mother to the result of traveling up at the other (output) daughter; traveling down at the left (respectively, right) daughter of an input product link, take the first (respectively, second) projection of the result of traveling down at the mother; traveling down at the (input) daughter of an output division link, return the associated index; traveling down at a root, return the associated lexical semantics.

for our example we obtain (26a), which is logically equivalent to (26b).


A DA represents the meaning of an utterance at the level of illocutionary force (Austin 1962).

a DA is approximately the equivalent of the speech act of Searle (1969), the conversational game move of Power (1979), or the adjacency pair part of Schegloff (1968) and Saks, Schegloff, and Jefferson (1974).


Each utterance is assigned a unique DA label (shown in column 2), drawn from a well-defined set (shown in Table 2).

dAs can be thought of as a tag set that classifies utterances according to a combination of pragmatic, semantic, and syntactic criteria.


However, for higher-order discourse grammars we found that forward-backward decoding consistently gives slightly (up to 1% absolute) better accuracies, as expected.


We need to allow for the fact that the models combined in Equation 8 give estimates of differing qualities.

we introduce an exponential weight a on P(Fi[Ui) that controls the contribution of the prosodic likelihood to the overall likelihood.


The problem with applying Equation 11, of course, is that the DA type Ui is generally not known (except maybe in applications where the user interface can be engineered to allow only one kind of DA for a given utterance).

we need to infer the likely DA types for each utterance, using available evidence E from the entire conversation.


It is generally not possible to directly compare quantitative results because of vast differences in methodology, tag set, type and amount of training data, and, principally, assumptions made about what information is available for "free" (e.g., hand-transcribed versus automatically recognized words, or segmented versus unsegmented utterances).

we will focus on the conceptual aspects of previous research efforts, and while we do offer a summary of previous quantitative results, these should be interpreted as informative datapoints only, and not as fair comparisons between algorithms.


The first character returns documents about such things as "evidence," "products," "body, .... image," "prices"; while the second returns documents about "theory, .... barber," and so on.

many documents that are completely irrelevant to the query will be returned, causing the precision of information retrieval to decrease greatly.


The character t is finally encoded in this model, with probability 5/26.

three encodings occur for this one character, with probabilities 1/2, 1/2, and 5/26 respectively, which together amount to just over 5 bits of information.


Experiments show that in deterministic contexts, for which d = 1, the probability that the single character that has occurred before reappears is greater than the 1 -1/(2n) implied by the above estimator.

in this case the probability is increased in an ad hoc manner to 1 -1/(6n).


For example, whenever an analyst used a JOINT relation to connect two clause-like units separated by an and, the rhetorical parser failed to identify the two units; it often treated them as a single elementary unit.

the recall figures at all levels were significantly lower than those specific to the humans.


The nuclearity preferences associated with the discourse markers eliminate many of the invalid interpretations.

the discourse structures built for sentences that have clause-like units as leaves are correct in most of these cases.


For example, the rhetorical parser is not able to identify that a discourse boundary should be inserted before the occurrence of and in the sentence "[Surface temperatures typically average about -60 degrees Celsius (-76 degrees Fahrenheit) at the equator] [and can dip to -123 degrees C near the poles.]."

the recall figure with respect to identifying the elementary units of this sentence is 0.


The NLP tool is not considered a black box.

full access to its source code is required in order to define and measure analysis-level style markers.


Moreover, their training procedures do not require excessive time or computational cost.

they can be easily incorporated into a real-time application.


For comparison purposes, we employed two sets of common words based on 30 and 50 most frequent words of the training corpus, respectively.

for each text a vector of 30 (or 50) parameters indicating the frequencies of the most frequent words of the training corpus (normalized by the text length) are calculated.


However, the identification error rate does not continuously decrease from 11 to 15 training texts; the identification error rate using 12 as well as 15 training texts for each category is greater than the rate attained by using 10 texts.

it is clear that satisfactory accuracy can be achieved with only 10 training texts.


The pronoun it here will be interpreted as (say) il, equivalent to: giving an RLF existsl(Af.pos(past(disappear(f, il))).

it, in this context, is interpreted, roughly, as 'the computer that figures in the eventuality of being owned by Smith'.


Recall that we are assuming that the various conditions in equivalences are goals to be proved in the manner of Prolog or similar inference methods.

a predicate like parallel can be called with or without arguments instantiated.


At a later stage the quantifier assumption is "discharged," capturing all occurrences of the free variable.

their analysis of something like every manager disappeared would proceed as follows: every manager disappeared every manager disappeared -discharge the assumption: In the final logical form, I am using an informal representation of their "pair" notation for generalized quantifiers, which uses the same variable in both the restriction and the body, unlike the one we have been using.


For 22b With the every unscoped, there is no choice of context which contains a parallel element to bill that does not also contain the VPELL functor, leading to an infinite regress, analogous to the "occurs check" failure in DSP.

we correctly cannot produce the unavailable reading.


Unfortunately, in our analysis, quantified NPs are also of type e and thus we will produce invalid interpretations in cases where the equivalence applies before the quantifier has been scoped.

as well as the correct bound variable interpretation for a sentence like every manager likes his secretary, we will also produce the structure corresponding to every manager likes every manager's secretary.


Some simplified examples will give the flavor of this theory.

a pronoun is represented at QLF by a term containing essentially a syntactic category, an index, a restriction predicate, and a metavariable; schematically: term(pro, <idx>, <restriction>, <met avrble>) a sentence like he sneezed will, ignoring tense and aspect, be represented as follows at the QLF level; and, when the metavariable has been instantiated to the contextually preferred candidate referent, at the resolved quasi-logical form (RQLF) level: QLF = sneeze(term(pro,+l,masc,Referent)) KQLF = sneeze (term (pro, +I, masc, john) ) Scoping of quantifiers is also a matter of instantiating metavariables.


It was stated earlier that the grammatical formalism used was not an essential component of the theory.

any alternative robust parsing system could be plugged in instead.


These advantages are thought by many to offset some of the obvious disadvantages of this way of developing NLP theories--in particular, the fact that, given the current state of language processing technology, many hypotheses of interest cannot be tested yet (see below).

quantitative evaluation is now commonplace in areas of language engineering such as parsing, and quantitative evaluation techniques are being proposed for semantic interpretation as well, for example, at the Sixth and Seventh Message Understanding Conferences (MUC-6 and MUC-7) (Sundheim 1995;Chinchor 1997), which also included evaluations of systems on the so-called coreference task, a subtask of which is the resolution of definite descriptions.


Attributive readings may also be proposed during QLF resolution; some of these seem to correspond to our unfamiliar uses.

the CLE seems to account for discourse-new descriptions, although they are not explicitly mentioned, and the methods used for choosing a referential or an attributive interpretation are not discussed.


In addition to the two distinctions described in the previous section, atomicity distinguishes punctual events (e.g., She noticed the picture on the wall) from extended events, which have a time duration (e.g., She ran to the store).

four classes of events are derived: culmination, culminated process, process, and point.


Because the genetic algorithm is stochastic, each run may produce a different function tree.

performance is evaluated over the models produced by multiple runs.


Clauses with be as their main verb, composing 23.9% of the corpus, always denote a state.

we can focus our efforts on the remaining clauses.


Of clauses with main verbs other than be and have, 83.8% are events.

simply classifying every verb as an event achieves an accuracy of 83.8% over the 739 test cases.


This rule cannot increase accuracy over the baseline without the remaining 13 indicators, since it does not positively identify any states--it only identifies events, which are all correctly classified by the baseline.

it is only useful because the overall model also correctly identifies some stative clauses.


The system can automatically classify all verbs appearing in a corpus, including those that have not been manually classified for supervised training data.

we have demonstrated a much-needed full-scale method for aspectual classification that is readily expandable.


When speakers/writers use an NP to refer to an object or a set of objects, they try to single out the entity uniquely.

when someone utters the NP the tenant of the house, the speaker may aim to single out a unique person, say Mr. X.


These algorithms are much faster than the obvious straightforward implementation when the corpus contains long repeated substrings, though for many corpora, the complications required to avoid quadratic behavior are unnecessary.

far we have seen how to compute tf for a single n-gram, but how do we compute tf and df for all n-grams?


That is, the LBL (= 1) is less than the SIL (= 4).

there is room for m between the LBL of s[10]m and the SIL of s[10]m. The endpoints m between LBL and SIL are highlighted in gray.


These results are also presented for the nontrivial intervals.

far we have seen how to compute term frequency, tf, for all substrings (n-grams) in a sequence (corpus).


A straightforward method for computing df for an interval is to enumerate the suffixes within the interval and then compute their document ids, remove duplicates, and return the number of distinct documents.

for example, df("o') in Figure 6, can be computed by finding corresponding interval, (11,14}, where every suffix within the interval starts with "o" and no suffix outside the interval starts with "o'.


Topic segmentation research based on prosodic information has generally relied on hand-coded cues (with the notable exception of Hirschberg and Nakatani [1998]), or has not combined prosodic information with lexical cues (Litman and Passonneau [1995] is one example where lexical information was combined with hand-coded prosodic features for a related task).

the work presented here is the first that combines automatic extraction of both lexical and prosodic information for topic segmentation.


In preliminary experiments, we observed a 5% relative reduction in segmentation error with initial and final states over the baseline HMM topology of Figure 2.

all results reported later use an HMM topology with initial and final states.


We conclude that a simple prosodic feature, pause duration, is an excellent criterion for the chopping step, giving comparable or better performance than standard sentence boundaries.

we used pause duration as the chopping criterion in all further experiments.


be reasonable for a small root word list, the potential for "noisy" or incorrect alignments is quite high when the corpus of inflected forms is large and the procedure is not given any prior knowledge of possible segmentations.

automatically selecting the "correct" segmentation becomes nontrivial.


The surfacy feature to feature mapping module maps the surfacy representation of the affixes to symbolic feature names; as a result, no surface information remains except for the citation form.

for instance, allomorphs that encode the same feature and map to different surfacy features now map to the same feature symbol.


For example, one citation form might be used in the present tense and another in the past tense; or one might be used with multisyllable affixes and another with single-syllable affixes.

a given lexical item can have multiple citation forms, each of which gets associated with a mutually exclusive subset of inflectional forms.


Note that we now use the citation form C, and not the stem S, as a part of the segmented form.

at the end of the process we generate pairs of inflected forms and their corresponding segmented forms to be used in the derivation of the morphographemic rules.


By definition of d-trees, each component of a d-tree is fully specified with respect to immediate domination.

all immediate domination relations between nodes in a component are indicated by i-edges.


Given the above requirements, at the end of the composition process, we can, when possible, get a minimal reading of a d-edge to be a domination relation of length zero.

we obtain the following procedure for removing d-edges: Consider a d-edge with a node x as the d-parent and with a d-child y.


In our proposed tree description language, we provide for underspecified dominance but not for underspecified linear precedence.

in the graphical representations of d-trees, we assume that sister nodes are always ordered as shown.


Raising verbs simply lack a subject of their own, but the S node is justified by the finiteness of the verb, not by the presence or absence of a subject.)

dSG can be used to develop grammars in which the derivation faithfully and straightforwardly reflects syntactic dependency.


However, studying the effect of context expansion for a PN tagging task in particular is relevant because: PNs may be hypothesized to have a unique sense in a text, and even in a domain corpus.

we can reliably consider as potential sense indicators all the contexts in which a PN appears.


The main asset of categorial grammar has traditionally been the close association between syntactic description on the one hand and a transparent and well-founded semantics on the other.

categorial grammar has primarily been favored by semantically oriented theorists and has arguably played an important role in the development of formal semantics for natural languages (see, for example, Lewis [1970] and Montague [1973]).


The morphology assigns a probability to each word w as the product of the probability of w's signature times w's stem, given its signature, and w's suffix, given its signature: prob (w = t +f) = prob (c 0 prob (t I or) prob (f ] or), where cr is the signature associated with t: cr = sig(t).

while stems and suffixes, which are defined relative to a particular morphological model, are assigned their empirical frequency as their probability, words are assigned a probability based on the model, one which will always depart from the empirical frequency.


However, they also appear as freestanding words, and so we use the word NULL, to indicate a zero suffix.


As we noted briefly above, the existence of a regular pattern of suffixation with n distinct suffixes will generally give rise to a large set of stems displaying all n suffixes, but it will also give rise in general to stems displaying most possible combinations of subsets of these suffixes.

if there is a regular paradigm in English consisting of the suffixes NULL, -s, -ing, and -ed, we expect to find stems appearing with most possible combinations of these suffixes as well.


We may represent a subsignature of that signature as a string of 0s and ls (a Boolean string b, of the form {0,1}*, abbreviated bk) indicating whether (or not) the ith suffix is contained in the subsignature.

if a stem t occurs [t] times, then the probability that it occurs without a particular suffix~ is (1 -prob(fi))[tJ; the probability that it occurs without all of the suffixes missing from the particular subsignature b = {bk} is i-i(1 --bk)(1 -prob(fi)) [t]; k and the probability that the particular subsignature b will arise at all is the sum of those values over all of the stems in the signature: all that is necessary is to estimate the hidden parameters of the frequencies of the individual suffixes in the entire paradigm.


Although the possibility of overcoming a majority is present with the arbiter type combiners, the situation is rather improbable.

we ought to be more than satisfied if any combiners approach the level corresponding to the projected combiner which resolves all ties correctly.


In a typical learning scenario, an automated procedure iteratively proposes rule modifications and adopts them only in those cases in which an objective function--that is, the evaluation method--indicates that performance has improved.

it is absolutely crucial that both positive and negative system changes are reflected as such in the feedback provided by the scoring mechanism.


This would cause the F-score to rise from 0.450 to 0.571, implicitly instructing the developer to keep it removed.

in all of these scenarios, the feedback provided by the evaluation method may steer our system developer off of the path to the optimal system state.


For these, we would expect the restrictive algorithm to correctly indicate a reduction.

we are interested in identifying those documents for which one algorithm signaled an improvement and the other signaled a reduction in score between a pair of system states.


In the case of the 25.00 point difference, the standard algorithm indicated a change from 59.62 to 51.69, whereas the restrictive algorithm indicated a change from 34.62 to 51.69.

while both algorithms assigned the same score to the second system state, the standard algorithm had assigned undeserved credit to the first system state, and what should have resulted in a 17-point improvement was shown instead as an 8-point reduction.


In our experience, by the time an IE system is in the 50-point performance range on the MUC-6 task, the majority of further progress results from a series of incremental changes that have a relatively small affect on the overall score.

a change that decreases performance by 1.26 points will almost certainly be removed from the system, whereas a change that increases performance by 1.29 will almost certainly be kept.


It should be borne in mind that it would not be difficult to construct a scenario in which the ramifications for a MUC-like task would have been far more severe--with a different task specification, template structure, set of slot definitions, or scoring scheme, for instance--to the extent that the integrity of such an evaluation could be compromised.

these results serve to highlight a potential pitfall to be avoided in future evaluations of IE and other high-level language processing tasks, which, like the MUCs, may be the principle driving forces behind technology development for extended periods of time.


A top-down parser will, in contrast, derive an efficiency benefit from precisely the information that is underspecified in these other approaches.

our top-down parser makes it very easy to condition the probabilistic grammar on an arbitrary number of values extracted from the rooted, fully specified derivation.


Since ,/4init~i+l is produced by expanding only analyses on priority queue HI '~it, the set of complete trees consistent with the partial derivations on priority queue ~4i,,~t is a subset of the "i+1 set of complete trees consistent with the partial derivations on priority queue H~ nit, that is, the total probability mass represented by the priority queues is monotonically decreasing.

conditional word probabilities defined in a way consistent with Equation 14 will always be between zero and one.


In other words, Shieber's algorithm is correct up to Tomuro and Lytinen Nonminimal Derivations licensing, but incorrect on the basis of a stronger criteria of minimality.

to guarantee correctness based on minimality, we need another algorithm; such an algorithm is exactly the solution to the nonminimal derivation problem.


First, whenever Shieber's algorithm produces a nonminimal derivation, it also produces a corresponding minimal derivation (Tomuro 1999).

one possible solution is to use subsumption to discard items that are more specific than any other items that are produced.


Hence the book by Reiter and Dale is biased to a certain extent towards standardized, formatted document generation, mostly in English.

some NLG-related application-oriented topics are underrepresented, prominently issues of multilinguality.


Chapter 4 is devoted to a careful comparison of L 2 with SnS, a logic studied earlier by Rabin, and Rogers shows that in a specific sense they are strongly equivalent.

there is an exact descriptive-complexity characterization of the context-free languages.


He then shows (by reduction to the origin-constrained tiling problem) that the resultant theory is undecidable.

the device of coindexing (not present in GPSG) increases expressive power to the extent that there is no general algorithm to determine whether a given theory is satisfiable.


L 2 gives a characterization of the strong generative capacity (SGC) of context- free languages, in the sense of Miller (1999), since it provides a logical characterization of the properties that are assigned to a string in virtue of the fact that it has a given derivation.

it does in a sense show a certain GB theory to be strongly context-free (p. 6).


The sentence can be disambiguated on pragmatic grounds because it is far more likely that sports equipment will be bought in a sports shop.

pragmatic information can be used to disambiguate bat to its "sports equipment" sense.


If we assume that every word of this type is assigned a part of speech which disambiguates it (i.e., verb in the case of beam), then an additional 7% of words in LDOCE could, potentially, be disambiguated.

up to 95% of word types in LDOCE can be disambiguated to the homograph level by part-of-speech information alone.


However, the codes are clearly not of equal levels of generality; for example, the code H is used to represent all humans, while M represents human males.

for a restriction with type H, we would want to allow words with the more specific semantic class M to meet it.


However, course(2) is marked abstract, which does not comply with this restriction.

assuming that run is being used in its second sense leads to a situation in which there is no set of senses which comply with all the restrictions placed on them; therefore run(l) is not the correct sense of run and the partial tagger marks this sense as wrong.


However, LDOCE has a higher level of average ambiguity and does not contain as complete a thesaural hierarchy as Roget, so we would not expect such good results when the algorithm is adapted to LDOCE.

we implemented the approach as a partial tagger.


If more than one sense has been assigned the maximum value, the tie is again broken by choosing the first sense.

each partial tagger returns a single sense and the exact match metric is used to determine the proportion of tokens for which that tagger returns the correct sense.


This fact is appreciated even in Brent (1999), which states "From a cognitive perspective, we know that humans segment each utterance they hear without waiting until the corpus of all utterances they will ever hear becomes available" (p. 89).

although the incremental algorithm in Brent (1999) is consistent with a developmental model, the formal statistical model of segmentation is not.


For example, "look", "drink", and "doggie" were always transcribed "lUk", "driNk", and "dOgi" regardless of where in the utterance they occurred and which mother uttered them in what way.

transcribed, the corpus consists of a total of 9790 such utterances and 33,399 words, and includes one space after each word and one newline after each utterance.


MBDP-1 assigns a score of 5.3+5.95 = 11.25 to the segmentation out + lets versus a score of 11.76 to outlets.

out + lets is the preferred segmentation.


In the 1-gram language model, the segmentation out + lets scores 5.31 + 5.97 = 11.28, whereas outlets scores 11.09.

it selects outlets as the preferred segmentation.


Then using the discounting scheme described, we have P(damnbritish) = 1/(x + 6) P(damn) = 2/(x+6) P(british) = x/(x + 6) and We seek an x for which P(damn)P(british) > P(damnbritish).

we get 2x/(x + 6) 2 > 1/(x q-6) ~ x > 6 The actual scores for MBDP-1 when presented with damnbritish for a second time are -logP(damnbritish) = 2.8 and -logP(D&m) -logP(brItIS) = 1.8 + 0.9 = 2.7.


By contrast, our method assigns a classification to verbs that have not previously been seen in the training data.

while we do not as yet assign different classes to the instances of a verb, we can assign a single predominant class to new verbs that have never been encountered.


For our purposes, these roles can simply be thought of as semantic labels which are non-decomposable, but there is nothing in our approach that rests on this assumption.

our approach would also be compatible with a feature-based deftnition of participant roles, as long as the features capture such general distinctions as, for example, the doer of an action and the entity acted upon (Dowty 1991).


We therefore expect that it will be less common for the same noun to occur in subject and object position across instances of the same object-drop verb.

we hypothesize that this pattern of thematic role assignments will be reflected in a differential amount of usage across the classes of the same nouns as subjects and objects for a given verb.


However, since the causative is a transitive use, and the transitive use of unergatives is expected to be rare (see above), we do not expect unergatives to exhibit a high degree of detectable overlap in a corpus.

this overlap of subjects and objects should primarily distinguish unaccusatives (predicted to have high overlap of subjects and objects) from the other two classes (each of which is predicted to have low [detectable] overlap of subjects and objects).


This particular sense extension has a transitive alternant, but not a causative transitive (i.e., Consumer spending jumped the barrier .... but not Low taxes jumped consumer spending... ).

while the possible subcategorizations remain the same, rates of transitivity and causativity may be different than for the literal mannerof-motion sense.


After eliminating repetitions, 94 occurrences were left, which were classified by hand, yielding 71 animate pronouns, 11 inanimate pronouns and 12 unclassified occurrences (for lack of sufficient context to recover the antecedent of the pronoun with certainty).

at least 76% of usages of they were animate; we assume the percentage of animate usages of the other pronouns to be even higher.


In the experiments below, we follow two methodologies in training and testing, each of which tests a subset of cases held out from the training data.

in all cases, the results we report are on test data that was never seen in training.


Although the precise make-up of the test cases vary, on average the test set will represent the class membership proportions of the entire set of verbs.

as with the single hold-out approach, chance accuracy corresponds to a maximum of 20/59, or 33.9%, of the test set being labelled correctly.


The difference in classifications between each four-feature subset and the full set of features indicates the changes in class labels that we can attribute to the added feature in going from the four-feature to five-feature set.

we can see whether the features indeed contribute to discriminating the classes in the manner predicted in Section 2.2, and summarized here in Table 13.


We can infer that the accuracy in our classification is due to argument structure information, as subcategorization is the same for all verbs.

we observe that the content of the thematic roles assigned by a verb is crucial for classification.


This is an important concept, which proposes that "regular" ambiguity in classification--i.e., sets of verbs that have the same multi-way classifications according to Levin (1993)--can be captured with a finer-grained notion of lexical semantic classes.

subsets of verbs that occur in the intersection of two or more Levin classes form in themselves a coherent semantic (sub)class.


For example, Merlo and Stevenson (1998) show that, contrary to the naive assumption, speakers' preferences in syntactic disambiguation are not simply directly related to frequency (i.e., a speaker's preference for one construction over another is not simply modelled by the frequency of the construction, or of the words in the construction).

the kind of corpus investigation we are advocating--founded on in-depth linguistic analysis--holds promise for building more natural NLP systems which go beyond the simplest assumptions, and tie together statistical computational linguistic results with experimental psycholinguistic data.


The extension is a subset A of the set of objects G and the intension is a subset B of the set of attributes M. We call the pair (A,B) a formal concept if each object of the extension has all the properties of the intension.

for the data shown in Figure 1, the pair ({Gropius, Breuer}, {Urban Planner, Architect}) represents a formal concept: each of the members of the extension possesses all the attributes mentioned in the intension.


natural layouts produced for the kind of magazine from which the original page was taken.

while conformance to the RST structure may prove itself a necessary (or at least, very desirable) condition, it is by no means sufficient for the construction of appropriate layouts.


The fact that there are so many covarying dimensions is, however, readily available from the dependency lattice: when a dataset has extensive and simultaneous regularities then these are present as nodes in the lattice.

when there are co-varying dimensions of potential aggregation, a medium allocation of diagram is made, when not, text is selected.


In the DArtbio case, we set up our heuristics, rather arbitrarily, to give headers to textblocks only; a more realistic decision-making process here can only be constructed after considerably more empirical investigation.

since the individual biography layout units are still complex, we do not immediately construct a header element but pass this option down so that it may be picked up by the first nuclear textblock that Partial layout structure following decomposition of the presentation plan according to summary and biographies.


Both patterns are abstractions over cases of fragment embedding in which the root of the dominating fragment constitutes one of the relevant occurrences.

the scope of the i-within-i patterns is rather restricted.


Psycholinguistic research supports the claim that listeners process utterances one word at a time.

when a listener hears a pronoun, he or she will try to resolve it immediately; if new information appears that makes the original choice incorrect (such as a violation of binding constraints), the listener will go back and find a correct antecedent.


Nested phrases can include prenominal modifiers, possessive pronouns, and so forth.

the number of candidate markables to be used for coreference is small.


In this example, the pronoun ~I 'him' is included in a PP (which is not included in another constituent) and the NP el padre de Luis is not included in another NP or PP.


The total number of pronoun occurrences in the study was 575.

we were able to define the different patterns of Spanish pronoun resolution and apply them in order to obtain the evaluation results that are presented in this paper.


Computational processing of semantic and domain information is relatively expensive when compared with other kinds of knowledge.

current anaphora resolution methods rely mainly on constraint and preference heuristics, which employ morpho-syntactic information or shallow semantic analysis (see, for example, Mitkov [1998]).


In general, it appears that cheapness is most likely to be an available option--the expectation that the current Cp will be the next Cb is generally plausible--but salience and cohesion are not always available.

if we take account of the options available to a reader at each stage, both versions of the discourse conform as far as possible to the principles of cohesion, salience, and cheapness.


Although the book is targeted at the nonspecialist, there is a difference in overall style between, roughly, the earlier chapters on speech and those on language processing.

chapters 2--4 consist of surveys and empirical comparisons of a number of different techniques, whereas the succeeding chapters place more emphasis on the authors' own work and less on comparing and contrasting competing approaches.


Anaphor resolution typically builds on many sources of information-among them, information about the grammatical structure of the sentence-so that the different filters and preferences may be used.

it must in general be regarded as a postgrammatical process, in the sense that it is completed after sentences are parsed.


For average segment size and a uniform distribution of false positives, the average penalty is k 2 , as described earlier.

in general, in large enough segments, the penalty when the false positive is a distance d < k from a boundary is d, and the penalty when the false positive is a distance d > k from a boundary is k. for larger segments, the average penalty assuming a uniform distribution becomes larger, because there are more places in the segment that are at least k positions away from a boundary.


Suppose we have a false positive in Segment A.

as Size(a) decreases from 2k to k, the average false positive penalty decreases linearly with it, because when Size(a) decreases below 2k, the maximum distance any sentence can be from a boundary becomes less than k. the maximum possible penalty for a false positive in a is less than k, and this number continues to decrease as Size(a) decreases.


Algorithms A-0, A-1, and A-2 receive the same penalty (about k), and Algorithm A-3 receives the smallest penalty (2e, where e is the offset from the actual boundary, presumed to be much smaller than k).

although it makes the mistake of penalizing Algorithm A-1 as much as Algorithms A-0 and A-2, it correctly recognizes that the error made by Algorithm A-3 is a near miss and assigns it a smaller penalty than Algorithm A-1 or any of the others.


Similarly, the FP segmentation decreases from 0.128 for the (20, 30) range to 0.107 for the (5, 45) range, and the FNP segmentation decreases from 0.317 for the (20, 30) range to 0.268 for the (5, 45) range.

variation in segment size has an effect on P k , as predicted.


Both P k and WD appear to solve the problem of underpenalizing false positives, but WD has the added benefit of being more stable across variations in segment size distribution.

wD essentially solves Problems 1, 2, and 3.


However, P k is more lenient for pure false positives that occur close to boundaries.

it is not immediately clear why P k is harsher in this situation, but a more detailed look provides the answer.


We can thus substitute either b or 2a for c. We choose to substitute 2a, because P k is strongly affected by segment size variation for Type A and Type C errors, but not for Type B errors.


In FP/FNP1, false positive probability is uniformly distributed throughout each segment, whereas in FP/FNP3, false positive probability is uniformly distributed throughout the entire document.

the FP/FNP3 segmentations are more likely to have boundaries that are very close to each other, since they are not segment dependent, while FP/FNP1 are limited to at most one false positive per segment.


A further notational convention will be useful: Values will be identified by two indices, the first of which identifies the Attribute.

to denote Value j of Attribute A i , we write V i,j .


For example, selection of BEING OF THE SAME AGE removes all those sets whose elements are not of the same age as each other, selection of FORMING A FOOTBALL TEAM removes all sets that do not make up a football team, and so on.

the algorithm generates descriptions of sets of collective entities (i.e., sets of sets).


For instance, phases might run separately before running in combination: first (as usual) Phase 1, then 2, then (as usual) 1&2, then 3, then 1&3, then 2&3, then (as usual) 1&2&3, and so on.

8 of this adaptation, the description Y ∪ Z would be generated because of Phase 2 alone.


Further, note that the estimated distribution P E(ŵ) does not approach P w as the sample size increases, so the expected loss does not converge to 0 as the sample size n increases.

the DOP1 estimator is also inconsistent.


Moreover, choosing the wrong word can convey an unwanted implication.

lexicographers have compiled many reference books (often styled as "dictionaries of synonyms") that explicitly discriminate between members of near-synonym groups.


Many near-synonyms differ in their marking as to the speaker's attitude to their denotation: good thing or bad thing.

the same person might be described as skinny, if the speaker wanted to be deprecating or pejorative, slim or slender, if he wanted to be more complimentary, or thin if he wished to be neutral.


Lexicographers, for instance, whose job it is to categorize different uses of a word depending on context, resort to using mere "frequency" terms such as sometimes and usually (as in Figure 1).

we cannot yet make any claims about the influence of context on nearsynonymy.


But on the other hand, no two words could ever be known to be cognitive synonyms, because, even at a fine grain, apparent cognitive synonyms might be fur-ther distinguishable by a still more fine-grained representation.

granularity is essential to the concept of cognitive synonymy, as which pairs of words are cognitive synonyms depends on the granularity with which we represent their propositional meanings.


So in this view, near-synonyms share the same propositional meaning just up to the point in granularity defined by language dependence.

we have an operational definition of near-synonymy: If the same concept has several reasonable lexicalizations in different languages, then it is a good candidate for being considered a language-independent concept, its various lexicalizations forming sets of near-synonyms in each language.


(We don't rule out other elements in the combination, but these are the main two.)

word meaning is not explicitly represented in the lexicon but is created (or generated, as in a generative model of the lexicon [Pustejovsky 1995]) when a word is used.


In this way, a cluster of near-synonyms is not a mere list of synonyms; it has an internal structure that encodes fine-grained meaning as differences between lexical entries, and it is situated between a conceptual model (i.e., the ontology) and a linguistic model.

the model has three levels of representation.


To account for fine-grained meanings and near-synonymy, we postulate a third, intermediate level (or a splitting of the conceptual-semantic level).

the three levels are the following: So, taking the conventional ontological model as a starting point, we cut off the ontology at a coarse grain and cluster near-synonyms under their shared concepts rather than linking each word to a separate concept.


In the figure, each set of near-synonyms forms a cluster linked to a coarse-grained concept defined in the ontology: Generic-Error, Generic-Order, Person, and Object, respectively.

the core denotation of each cluster is the concept to which it points.


When the problem of context dependence is better understood, this part of the formalism will need to be changed.

a denotational distinction of a word w is a quadruple of components as follows: w: (frequency strength indirectness concept) The first part of Table 2 gives some examples for the distinctions of Figures 7 and 8.


As for the attitude itself, we take a conservative approach, for now, and define only three possible attitudes: favorable, neutral, and pejorative.

an expressive distinction has the following form: w: (frequency strength attitude entity) Frequency and strength have the same role as above.


From two such sets of distinctions, one can compute, or build, an explicit representation of the difference between two near-synonyms.

the difference between, or similarity of, two near-synonyms depends on the semantic content of their representations on the subconceptual/stylistic level (cf.


The interaction of the dimensions within a cluster is not yet very well studied, so for a partial solution, we make the simplifying assumptions that the dimensions of a cluster are independent and that each can be reduced to a true numeric dimension.

16 two distinctions d 1 and d 2 are commensurate if the following two conditions hold: • d 1 and d 2 are of the same type (i.e., stylistic, expressive, or denotational).


So, assuming that each option has an associated numeric score (the degree to which it satisfies all of the preferences), we can simply choose the set of options that maximizes the sum of the scores, subject to the other constraints of building a proper sentence plan.

we do not provide a solution to how the context affects the combination of the scores.


Each word in SP has to be chosen from a distinct cluster.

given • a particular cluster c in the set of all cluster options • a list W of candidate near-synonyms of c, ordered according to a prespecified criterion (some candidates of the cluster might have already been ruled out because of collocational constraints) • a set P c ⊂ P of compatible preferences, each of which can potentially be satisfied by a word in c, with associated importances: Imp: P → [0, 1] find the first candidate w ∈ W such that WSat(P, w ) is maximized.


That is, whenever a (local) decision is made between several options, choose the option with the highest score.

we postulate that the most preferred sentence plan will be one of the first few sentence plans generated, though we offer no proof beyond our intuition that complex global effects are relatively rare, which is also a justification for the simplifications we made above.


Carpenter (1997) points out that nonintersective adjectives (e.g, toy, fake, alleged) are unlike numerical modifiers and intersective adjectives in that their semantics requires phrasal (wide) scope for -PLU, corresponding to the "set of things" interpretation of the plural noun.

toy guns is interpreted as plu(toy gun) because the plural outscopes the modification.


Continuing with the dog example, the proposal is to estimate p(run | dog , subj) using a relative-frequency estimate of p(run | animal , subj) or an estimate based on a similar, suitably chosen class.

assuming this choice of class, p( dog | run, subj) would be approximated as follows: The following derivation shows that if p(v | c i , r) = k for each child c i of c , and Note that the proof applies only to a tree, since the proof assumes that c is partitioned by c and the sets of concepts dominated by each of the daughters of c , which is not necessarily true for a directed acyclic graph (DAG).


Assuming a level of significance of α = 0.05, the critical value is 12.6 (for six degrees of freedom).

for this α value, the null hypothesis would not be rejected for either statistic, and the conclusion would be that there is no reason to suppose that p(run | canine , subj) is not a reasonable approximation of p(run | dog , subj).


Furthermore, we show in Section 7 that to generalize unnecessarily can be harmful for some tasks: If we already have lots of data regarding sandwich , why generalize any higher?

the purpose of this section is not to show that the acquired levels are "correct," but simply to show how the levels vary with α and the sample size.


The figures in each column arise from using the contingency tables based on the complete training data, but with each count in the table multiplied by the percentage at the head of the column.

the 50% column is based on contingency tables in which each original count is multiplied by 50%, which is equivalent to using a sample one-half the size of the original training set.


The uniform-distribution assumption (20) means that cuts close to the root of the hierarchy result in a greater smoothing of the probability estimates than cuts near the leaves.

there is a trade-off between choosing a model that has a cut near the leaves, which is likely to overfit the data, and a more general (simple) model near the root, which is likely to underfit the data.


, (5, ⊥ w )}, two cloned states ((0, ) and (1, b)) and two queue states ((⊥, br) and (⊥, bra)).

of the designation of (0, ) as the algorithm addstring Input: [create cloned and queue states; add clones of accepting states to F]  [check queue and cloned states one by one] end for return M = (Q, Σ, δ, q 0 , F) end algorithm Algorithm to add a string w to the language accepted by a finite-state automaton while keeping it minimal.


The less-specific distributions were used only when no data were present for any more-specific distribution.

the distributions selected are arranged in a cut across the lattice representing the most-specific distributions for which data are available.


In sentences with only one candidate frame element (the subject in The door closed), the use of the FEG prior will cause it to be labeled Theme, even when the feature probabilities prefer labeling a subject as Agent.

the FEG prior, by representing the set of arguments the predicate is likely to take, essentially already performs the function of the subcategorization feature.


The FEG prior allows us to introduce a dependency between the classifications of the sentence's various constituents with a single parameter.

it can handle the alternation of our example without, for example, introducing the role chosen for one constituent as an additional feature in the probability distribution for the next constituent's role.


There are several factors that may account for this: the FrameNet domains were intended primarily as a way of organizing the project, and their semantics have not been formalized.

it may not be surprising that they do not correspond to significant generalizations about argument structure.


The main difficulty is that when dealing with real-world texts, we have to identify abbreviations and proper names ourselves.

estimates based on the application of our method when using 100% correctly disambiguated capitalized words and abbreviations can be considered as the upper bound for the SBD approach, that is, the top performance we can achieve.


The simplest strategy for deciding whether a word that is followed by a period is an abbreviation or a regular word is to apply well-known heuristics based on the observation that single-word abbreviations are short and normally do not include vowels (Mr., Dr., kg.).

a word without vowels can be guessed to be an abbreviation unless it is written in all capital letters and can stand for an acronym or a proper name (e.g., BBC).


We also would like to boost the abbreviation handling with a domain-independent method that enables the system to function even when the abbreviation list is not much of a help.

in addition to the list of known abbreviations and the guessing heuristics, we decided to apply the DCA as described below.


Also, the documents in that corpus are distributed very densely around their mean.

not much can be inferred about the dependency of the performance of the method on document length apart from the observation that documents 2,000-3,000 words long are handled well by our approach.


In our markup convention (Section 2), periods are tokenized as separate tokens regardless of whether they stand for fullstops or belong to abbreviations.

a POS tagger can naturally treat them similarly to any other ambiguous words.


Although it needs such a corpus to be relatively large (a few hundred thousand words), this is normally not a problem, since when the system is targeted to a new domain, such a corpus is usually already available at no extra cost.

there is no trade-off between the amount of human labor and the performance of the system.


For example, connectives like because direct attention to the cause of the previously described event, and connectives like so direct attention to the consequences of the event.

in a sentence like (17), the verb projects a focus preference for Bill, because Bill is the person associated with the endpoint of the event of criticizing.


The grammar designer has a notion of what paths can be equated, but this notion is not formally defined.

it is possible to find erroneous path equations such as VP.b:<assign-case> = V.t:<tense>.


(9605013) • Our method combines similarity-based estimates with Katz's back-off scheme, which is widely used for language modeling in speech recognition.

• we base our model on the work of Clark and Wilkes-Gibbs (1986), andHeeman andHirst (1992) .


Paice and Jones (1993) report that they abandoned an informal sentence selection experiment in which they used agriculture articles and experts in the field as participants, as the participants were too strongly influenced by their personal research interest.

of subjectivity, a number of human sentence extraction experiments over the years have resulted in low agreement figures.


This contrast in agreement might have to do with the location of the rhetorical zones in the paper: AIM and TEXTUAL zones are usually found in fixed locations (beginning or end of the introduction section) and are explicitly marked with metadiscourse, whereas CONTRAST sentences, and even more so BASIS sentences, are usually interspersed within longer OWN zones.

these categories are more exposed to lapses of attention during annotation.


Recall is lower at 70%, 24%, and 39%, but low recall is less of a problem in our final task.

the main bottleneck is correct rhetorical classification.


Of the 632 REF AGENTs, 436 (69%) were classified as US AGENT, 175 (28%) as THEM AGENT, and 20 (3%) as GENERAL AGENT.

of this manual disambiguation, the performance of the Agent feature increased dramatically from K = .08 to K = .14 and that of SegAgent from K = .19 to K = .22.


For our data, however, sentence boundaries are typically not available in the first place.

we devise a word-based evaluation metric derived from an average relevance score from human relevance annotations (section 6.2).


This mismatch between acoustic and linguistic segmentation is reflected in the output of a speech recognizer, which typically generates a sequence of speaker turns whose boundaries are marked by periods of silence (or nonspeech).

one speaker's turn may contain multiple sentences, or, on the other hand, a speaker's sentence might span more than one turn.


5 There were six naive human annotators performing the task; 6 only four, however, completed the entire set of dialogues.

the number of annotations available for each dialogue varies from four to six.


The overall runtime performance of this algorithm is given by the sum of the steps listed above, for an overall runtime of Initially, we may be greatly concerned with the size of these constants; upon further analysis, however, we see that most synsets have very few parent-child relations.

the worst-case values may not reflect the actual performance of our application.


If both of these give 100% coverage, this would mean that all and only the concepts identified by strong lexical chains in the document occur in the summary.

the higher these numbers turn out to be, the more likely it is that lexical chains are a good intermediate representation of the text summarization task.


The benefit of this linear-time algorithm is its ability to compute lexical chains in documents significantly larger than could be handled by Barzilay and Elhadad's implementation.

our algorithm makes lexical chains a computationally feasible intermediate representation for summarization.


Neither the abstracts nor the source documents were electronically available, so the information was collected through photocopies.

we do not have information regarding number of sentences and words in the source document.


Many modern theories of mental categorization (Rosch 1978;Smith and Medin 1981) assume that mental categories are represented by prototypes or exemplars.

if different people are exposed to different category prototypes and exemplars, they are likely to have different rules for evaluating category membership.


He does show, however, that actually a nonlinear representation is harmful everywhere but in the stem, for example, leading to duplication of rules when coverage is extended to affixed forms (page 112f).

he must weaken his architecture to provide a second stage in which rules postprocess fully linearized verb stems; the same setup is proposed for broken plural formation in nouns, because vowel length and prosodic shape transfer from singular to plural and cannot be read off the components alone.


To be sure, the book does have its strong sides, including good reviews of related work and an exposition of a particular multitape finite-state formalism that is detailed enough to allow the interested reader to implement it, and-if so desiredcreate a working morphology system for Semitic and other languages.

i would recommend it as a useful source of inspiration for researchers in the field, as long as their foreseen applications are unaffected by the criticism presented above.


"The book offers a combined discussion of the main theoretical, methodological and application issues related to corpus work.

starting from the definition of what is a corpus and why reading a corpus calls for a different methodology from reading a text, the underlying assumptions behind corpus work are discussed.


It requires much effort on the part of a speaker, however, to produce a perfectly level tone.

of accommodating a speaker's effort, the listeners will shift their linguistic perception boundary between level and rising tones away from the psychophysical boundary, to allow the speaker some freedom in articulation (Wang 1976).


If the case marking system is too complex, however, it will be hard for children to learn as they acquire the language.

there may exist a balance point among the three different constraints involved.


(1997a), to examine the effect of different metrics of calculating perceptual contrast.

we will have two fitness functions regarding the first criterion, perceptual contrast, denoted as F 1F and F 1S , with the latter based on Schwartz's method and the former based on Fant's.


This knowledge derives primarily from observations of three sorts: the frequency of distribution of the sounds in the languages of the world, the patterns of historical change in sound systems, and the acquisition of sounds in children and the dissolution of sounds in linguistic pathology.

the complexity assigned to tones based on markedness may reflect an integrated effect of perception, production, and learnability.


Very often different researchers have their own individual strategies of transcription, and different informants employed for the same dialect also have individual differences in their speech.

controversies often arise, such as whether a particular tone in Cantonese is 13 or 23.


Moreover, the markedness complexity we hypothesize is an abstract measure that incorporates many factors, including perception and production.

the consideration of perception contrast may have been duplicated in the fitness function.


Often, an alignment includes effects such as reorderings, omissions, insertions, and word-tophrase alignments.

we need a very general representation of alignment.


They are very easy to implement and understand.

variants of the heuristic models described above are widely used in the word alignment literature.


Therefore, we modify Models 3 and 4 slightly by replacing φ 0 !

in equation 27with J φ 0 : of this modification, the alignment models for both nonempty words and the empty word are deficient.


We expect to achieve better alignment quality by using a model that takes into account both types of dependencies.

we combine HMM and Model 4 in a log-linear way and call the resulting model Model 6: Here, the interpolation parameter α is employed to weigh Model 4 relative to the hidden Markov alignment model.


Hence, the EM algorithm can increase likelihood by simply aligning more and more words with the empty word.

3 we modify Models 3 and 4 slightly, such that the empty word also has a deficient alignment model.


This count is then used instead of N(e, f) in the EM algorithm as shown in equation 35.

• Och and Ney (2000) suggest that the effective multiplicity of a dictionary entry be set to a large value µ + 1 if the lexicon entry actually occurs in one of the sentence pairs of the bilingual corpus and to a low value otherwise: µ(e, f) = µ + if e and f co-occur only dictionary entries that indeed occur in the training corpus have a large effect in training.


The baseline alignment model does not allow a source word to be aligned with more than one target word.

lexical correspondences like that of the German compound word Zahnarzttermin with the English dentist's appointment cause problems, because a single source word must be mapped to two or more target words.


Therefore, lexical correspondences like that of the German compound word Zahnarzttermin with the English dentist's appointment cause problems, because a single source word must be mapped to two or more target words.

the resulting Viterbi alignment of the standard alignment models has a systematic loss in recall.


It is well known that manually performing a word alignment is a complicated and ambiguous task (Melamed 1998).

in performing the alignments for the research presented here, we use an annotation scheme that explicitly allows for ambiguous alignments.


corpus, however, there still is an unacceptably large training time.

we report the results for only up to 128,000 training sentences.


Let us assume for the sake of illustration that the cost function is defined in such a way that adding a vertex or an edge always costs one point.

for each v ∈ V H and for each e ∈ E H : cost(v) = cost(e) = 1.


Subsequently, in the sketch of the basic algorithm (Figure 6), the set of distractors should be redefined as follows: That is, the distractor set is restricted to those vertices n in the scene graph G that currently are at least as salient as the target object v. For target objects that are linguistically salient, this will typically lead to a reduction of the distractor set.

distinguishing graphs for these target objects will generally be smaller than those for nonsalient objects.


Because of the way the classifier is built, predictor features for class 2 need to have higher loadings to overcome the lower prior probability of the class.

we actually rank the features in Table 6 according to αP(c) k (which we denote as α c,k ).P(c) represents the empirical prior probability of a class c, and k is simply a constant (.25 in this case) chosen to try to get a mix of features for different classes at the top of the list.


Since class 0 is simply more frequent in the training data than the other two classes, the weights for this class tend to be higher.

we also list some of the best predictor features for classes 1 and 2 in the table.


The probability distributions that our classifiers define for quantifier scope structures are conditional on syntactic phrase structure, because they are computed on the basis of syntactically provided features, such as the number of nodes of a certain type that intervene between two quantifiers in a phrase structure tree.

the combined language model that we define in this article assigns probabilities according to the pairs of structures that may be assigned to a sentence by the Qstructure and phrase structure syntax modules.


This is referred to as presuppositional accommodation.

presuppositions are, under certain circumstances, able to present new information to the discourse.


Proper names, once introduced in discourse, always seem available for future anaphoric reference, and so do first-and secondperson pronouns.

in DRT, their discourse referents are introduced at the global level of DRS, even in cases in which they are realized as part of a subordinated context, as for the proper name Butch in Example (15).


On the other hand, the local constraints help in dealing with the presupposition project problem.

we take a suggestion made by Beaver (2002) and use the local constraints as a further criterion for ranking potential solutions of the resolution algorithm.


Sometimes the two categories of sentence coincide, but often they do not.

in the following passage: (3) He entered the office.


In practice, this separation of abstract structure from visual realization is not carried through consistently; for reasons of convenience, authors sometimes prefer to have direct control over appearance.

l a T E X, for example, allows both the abstract tag em, meaning emphasis, and the visual tag it, meaning italic face.


Note that rhetorical structures are always unordered, whereas document structures are ordered trees.

the two unordered propositions in the rhetorical structure of Figure 3 are realized in different orders in the two document structures.


S, where S is the total number of sisters (including N R ).

if N R is an argument in a nucleus-satellite relation, the range will be 1 .


The rhetorical structure will comprise just two propositions linked by a concession relation: concession(ban(fda, elixir), approve(fda, elixirplus)) The method works by computing the options for realizing each constituent of the rhetorical structure, where a constituent is any node in the rhetorical structure tree along with its descendents.

in the present example there are three constituentsthe two propositions and the whole relationship-and for convenience we will label them as follows: A: approve(fda, elixirplus) B: ban(fda, elixir) C: concession(ban(fda, elixir), approve(fda, elixirplus)) We can now begin to characterize the units in the document structure that will realize the rhetorical constituents A, B, and C. For each unit, four choices must be made: its level, its indentation, its position in relation to its sisters, and its connective.


Subordinating Conjunction: The units connected by although may occur in any order, but they must be text-phrases.

if connective(C) = C alt , then level(A) = level(B) = L 0 .


It is worth noting that similar distinctions are made in other branches of linguistics.

in phonology, a distinction is made between a phonemic level and a phonetic level.


However, • the medicine has been thoroughly tested • it has no significant side effects.


The adjectivenoun combinations are treated along similar lines.

the logical polysemy of words like finish and fast is not accounted for by exhaustive listing.


Indeed, one can predict that problems with finding evidence for begin V NP will occur on the basis of Gricean principles of language production, where the heuristic be brief (which is part of the maxim of manner) will compel speakers to utter begin coffee as opposed to begin V coffee if V is one of the plausible interpretations of begin coffee.

on the basis of this Gricean reasoning, one might expect metonymies like (7a) and (8a) to occur with greater frequencies than their respective paraphrases (see (7b) and (8b)).


The adverbial function of the adjective difficult is expressed only periphrastically (i.e., in a difficult manner, with difficulty).

we obtained the frequency f (difficult, e) only on the basis of infinitival constructions (see the examples in (54)).


The editors' introduction is a step in the right direction, but a small one, and given the wide range of topics addressed, some further analysis and guidance would have been welcome.

although this is Didier Bourigault, Christian Jacquemin, and Marie-Claude L'Homme (editors) (Université Toulouse-le-Mirail, CNRS Orsay, and Université de Montréal) Amsterdam: John Benjamins (Natural language processing series, edited by Ruslan Mitkov, volume 2), 2001, xviii+379 pp; hardbound, ISBN 1-58811-016-8, $99.00 This collection of papers derives from the Proceedings of the First Workshop on Computational Terminology (Computerm '98), held at COLING-ACL '98 in Montreal, but is a substantial revision thereof.


Furthermore, of the remaining papers, most have been significantly extended.

this book should not be thought of as a simple reissue, in hardcover, of the workshop proceedings.


Intuitively, if a word pair (x, y) is expected to be a likely lexical word pair in the dictionary and cognate set, then models that make (x, y) probable are more likely (according to the prior).

the expected count of (x, y) is increased at each iteration of training to the extent that the prior favors it.


• most search engines allow users to retrieve a limited number of documents (e.g., 1,000 in AltaVista).

we continue our search with a host crawler, which uses the Web pages found by the search engines as seeds.


We will do this by decomposing the problem into two components that are easier to estimate: where L is the size of the source vocabulary.

p(t i | M Qs ) can be approximated by combining the translation model p(t i | s j ), which we can estimate on the parallel Web corpus, and the familiar p(s j | M Qs ), which can be estimated using relative frequencies.


We checked the assumptions for an analysis of variance (by fitting a linear model for a withinsubjects design) and found that indeed the distribution of the residual error was quite skewed, even after transformation of the data.

we resorted to a nonparametric alternative for the analysis of variance, the Friedman test (Conover 1980).


In contrast, EBMT systems have overcome this constraint by storing subsentential translational correspondences in addition to the sententially aligned pairs from which they are derived.

where a TM system can only propose a number of close-scoring matches in its database for the translator to adapt into the final translation, an EBMT system can produce translations itself by automatically combining chunks from different translation examples stored in its memories.


In some cases, an improvement of 50% was seen when EBMT was used.

of the methodology chosen, we were able to perform a detailed evaluation of the strengths and weaknesses of the three Web-based systems used in our research, with Logomedia clearly outranking the other systems used.


The four main chapters of the book then cover document retrieval from collections and from the Web, information extraction, text categorization, named-entity and coreference recognition, and automatic summarization.

unlike the standard large and comprehensive computational linguistics textbooks (Jurafsky and Martin 2000;Manning and Schütze 1999) and handbooks (Dale, Moisl, and Somers 2000;Mitkov 2003), Jackson and Moulinier's book is concise and focused on the goal of building applications.


This is achieved by means of composition, with the transducer TopChannelNotEmpty defined as follows: TopChannelNotEmpty =~[ ["<" LeftChannelSymbols* "0" "(" AnyIG ("@") ")" "0" RightChannelSymbols* ">" ] *] ; This filter would be read as "It is not the case that all topmost channel symbols in a configuration are all 0s."

configurations in which all most recent channel symbols are 0 are filtered.


We can augment the pseudocode given in section 6.6 as follows: This will have the parser produce outputs with up to three unlinked word-final IGs when there are no outputs with a smaller number of unlinked word-final IGs.

it is possible to recover some of the partial-dependency structures when a full-dependency structure is not available for some reason.


Rather, donkey sentences show that discourse semantics has to provide variables to translate pronouns, and that discourse mechanisms must interpret these variables as bound-even though the pronouns appear "free" by syntactic criteria.

it is significant that discourse adverbials can appear in their own version of donkey sentences, as in 24 The examples in (24) involve binding in the interpretation of discourse adverbials.


whereas in (39), it is resolved to e 1 :red(light1).

the resulting interpretations of (38b) and (39b) are, respectively, if(e 4 ,e 3 ), where complement(e 2 ,e 4 ) and e 2 :stop(you) (If you do something other than stop, you'll get a ticket.)


(58) If they're drunk and they're meant to be on parade and you go to their room and they're lying in a pool of piss, then you lock them up for a day.

(The Independent, June 17, 1997) it becomes harder and harder to distinguish the scope of discourse-level syntax and semantics from that at the sentence-level.


In DLTAG, however, it is predicates on clausal arguments (such as coordinate and subordinate conjunctions) that define the domain of locality.

at this level, these predicates anchor initial trees into which clauses substitute as arguments.


As we noted, our view is that there are three mechanisms at work in discourse semantics, just as there are in clause-level semantics: Inference isn't the only process involved.

the focus of our presentation here is on how compositional rules and anaphor resolution (which itself often appears to require inference) operate together with inference to yield discourse semantics.


As in example (66b), an inferred relation is triggered between the interpretations of T2 and T1, namely, explanation(interp(T2),interp(T1)).

then, as a result of β:for ex1 adjoining at t2 and β:for ex2 adjoining at the root of the higher (66c) and (66d) differ only in the derivation of the interpretation that for example then abstracts over.


To this, i2 adds the somewhat redundant information that i1 serves to elaborate the advice in T1.

(69a) and (69b) receive similar interpretations but by different means.


Each subcategorization frame is a multiset 10 specifying the complements that the head requires in its left or right modifiers.

generate the left and right modifiers with probabilities the subcategorization requirements are added to the conditioning context.


Let AC be the set of adjective classes (which subsume WordNet synsets; we elaborate further on this subsequently).

the TCMs define a probability distribution over NS that is conditioned on a verb class (vc) or adjective class (ac) and a particular grammatical relation (gr): Acquisition of a TCM for a given vc and gr proceeds as follows.


The probability at the hypernym nc will necessarily total the probability at all hyponyms, since the frequency credit of hyponyms is propagated to hypernyms.

to disambiguate a noun occurring in a given relationship with a given verb, the nc ∈ C n that gives the largest estimate for p(nc|vc, gr) is taken, where the verb class (vc) is that which maximizes this estimate from C v .


Unfortunately, in the book no empirical analysis of different feature representations is even attempted.

whenever empirical findings do not fit well with theoretical estimates, weak explanations are given.


Paşca's approach is to extract the words connected to the wh-word in the dependency representation of the question and look up those words in WordNet conceptual hierarchies that have been hand-mapped onto specific answer types.


(Note that the basic algorithm of BB performs disambiguation on all the words in English and Chinese.)

in the case of plant, for example, the classifiers with respect to gongchang and zhiwu make classification decisions only on D and E and not C and F (in Figure 10), because it is not necessary to make classification decisions on C and F. In particular, it calculates λ * (c) as λ * (c) = P(c | t) and sets θ = 0 in the right-hand side of step 2.


We have noticed that a key to BB's performance is the asymmetric relationship between the classes in the two languages.

we tested the performance of MB and BB when the classes in the two languages are symmetric (i.e., one-to-one mapping).


Possible modules involved are the lexicon, speech recognition, syntactic parsing, speech synthesis, semantic interpretation, and so on.

the identification of lexical words and/or the delimitation of words in running texts is a prerequisite of NLP.


To recognize proper names is the first task for Chinese word extraction, because their meanings cannot be obtained through the combination of smaller words, as in the compound words that are described next.

a reasonable way to approach them is to deduce them from Chinese text collections.


Furthermore, if is removed from the dictionary, then another, different segmentation pattern will be achieved.

the dictionary is an important factor in these methods.


For example, , 'suddenly', is often used separately as a short sentence.

"S" and "E" will be counted multiple times, and we regard as a meaningful word, although there are probably very rarely other characters preceding or succeeding it.


In each experiment, all of the values of precision and partial recall of both IT and AV were counted by the same person.

the evaluation results should be reasonably credible.


Note that all of the results were obtained only on new words.

the size of the system dictionary will affect the result of the experiment.


Moreover, the assumption of equal distributions over the categories masks the exact source of disagreement among the coders.

such an assumption is detrimental if such systematic disagreements are to be used to improve the coding scheme (Wiebe, Bruce, and O'Hara 1999).


Each has been extended to more than two coders, for example, respectively Fleiss (1971) and Bartko and Carpenter (1976).

without loss of generality, our examples involve two coders.


For κ Co , P(E) is computed from each coder's individual probabilities.

the less two coders agree in their overall behavior, the fewer chance agreements are expected.


(In fact, I was sent a preliminary version by the publisher in 1999.)

the editors' assessment of the most significant recent trends in MT is not entirely up to date.


She points out as well, however, that a number of other types of semantic relationships manifested in bridging references would not be identifiable from lexical-conceptual information.

although work on bridging references has attempted to provide a characterization of the possible semantic relationships involved, what success has been achieved is limited to a small subset of cases.


Under this assumption, this corpus would be characterized as extremely incoherent, a claim belied both by native-speaker intuitions (an acceptable level of coherency for these texts was confirmed by three native speakers) and also by the fact that the task that was the central concern of these messages was successfully completed; the group exchanged a number of opinions and pieces of information and came to a consensus regarding their sports outing, with no message showing confusion about information contained in previous messages.

the solution of allowing only explicit centers does not yield an accurate characterization of the coherence of this corpus.


This is actually closely tied to another advantage: There is no need to limit the types of semantic relationships into which the Cfs can enter.

there is no need to restrict our analysis to a subset of the phenomenon; our account will handle inferable centers having any kind of semantic relationship to the centers in the previous utterance.


The recall estimate was produced by manually inspecting 6,000 of the initial 14,383 candidate terms, asking the experts to mark all the terms judged as "good" domain terms, and comparing the obtained list with the list of terms automatically filtered by OntoLearn (the phase of terminology filtering described in Section 3.1).

of the feedback obtained from the tourism experts, we decided that experts' interpretation difficulties could indeed be alleviated by associating a textual definition with each new concept proposed by OntoLearn.


But once an agreement is reached, filling the lower levels of the ontology can still take a long time, simply because it is a tedious and time-consuming task.

we think that OntoLearn revealed itself indeed to be a useful tool within Harmonise.


Apart from the theoretical problem of how to incorporate external dictionaries in a mathematically sound way into a statistical framework for machine translation (Brown, Della Pietra, Della Pietra, and Goldsmith 1993) there are also some pragmatic difficulties: As discussed in Section 2.2, one of the disadvantages of these conventional dictionaries as compared to full bilingual corpora is that their entries typically contain single words or short phrases on each language side.

it is not possible to distinguish among the translations for different readings of a word.


However, the Viterbi score VT P for the first pair is 1.0 • 0.3 • 1.0 = 0.3, which is lower than the Viterbi score VT P for the second pair, 1.0 • 0.4 • 1.0 = 0.4.

this second pair will be the approximate result given by equation 7.


The particular case a(j) = 0 means that the position j in t is not aligned with any position in s. All the possible alignments between t and s are denoted by A(s, t), and the probability of translating a given s into t by an alignment a is Pr(t, a | s).

an optimal alignment between s and t can be computed aŝ Different approaches for estimating Pr(t, a | s) were proposed in Brown et al.


They all change their right languages as the result of adding a new string, but only the last prefix state (the most distant from the initial state) is sure to change its transitions.

it should be removed from the register before adding a new string.


Since the automaton is deterministic, it cannot hold more than one copy of the same string.

we need only to show that no other strings are erroneously added to the automaton.


The detailed examples provide real-life morphological problems and the correct way to solve them.

a considerable subset of Esperanto morphotactics is actually covered by examples in chapter 4.


Unsurprisingly, the maximum-likelihood hypothesis is the one in which each tree was generated in a single, atomic substitution; this hypothesis wastes no probability mass on unseen sentences.

bonnema and Scha turn to the uniform distribution over derivations, which is also well-founded but has the opposite bias (smaller subtrees take more probability mass because they can combinatorially occur in more derivations).


The framework itself seems to cover most, if not all, prior work.

the author's goal seems to be to provide a way of unifying prior approaches, instead of choosing among them.


For instance, we know that the most problematic test cases for the PP-attachment model are those for which it has no statistics at all.

those data that the system has not yet encountered might be good candidates.


By selecting either as the first training example, we could ignore all but the four-word characteristic tuple for both u 1 and u 2 (a saving of seven tuples each); since u 3 and u 4 each have three words in common with the first two candidates, they would no longer depend on their lower four tuples; and although we would also improve the statistics for one of u 5 's tuples (on), nothing could be pruned from u 5 's characteristic tuples.

f backoff (u 1 , C) = f backoff (u 2 , C) = 7 + 7 + 4 + 4 = 22 (see Figure 4).


In the early stage, f conf picks the same examples as f novel , because we have no confidence in the statistics of novel examples.

f conf is also prone to chase after examples that rarely occur to build up the confidence of some unimportant parameters.


A second problem is that f conf ignores the output of the model.

if candidate A has a confidence interval around [0.6, 1] and candidate B has a confidence interval around [0.4, 0.7], then f conf will prefer candidate A, even though training on A will not change the hypothesis's performance, since the entire confidence interval is already in the noun zone.


This classification task is more difficult than PP-attachment, since the number of possible parses for a sentence grows exponentially with respect to its length.


Then, there are a total of Y × Z parses, and the probability of each parse is P(x)P(y)P(z), where y ∈ Y and z ∈ Z.

to compute h Y,Z,k , we need to sum over all possible parses: the tree entropy of the entire sentence can be recursively computed from the entropy values of the substrings.


Maher is not directly speaking to us but is being quoted by the writer.

the source is a nested source, which we notate (writer, Maher); this represents the fact that the subjectivity is being attributed to Maher by the writer.


In (5), they are used subjectively, but one can easily imagine objective usages, in a scientific domain, for example.

an NLP system may not merely consult a list of lexical items to accurately identify subjective language but must disambiguate words, phrases, and sentences in context.


There are techniques for analyzing agreement when annotations involve segment boundaries (Litman and Passonneau 1995;Marcu, Romera, and Amorortu 1999), but our focus in this article is on words.

our analyses are at the word level: Each word is classified as either appearing in a subjective element or not.


For example, there are articles whose purpose is to present an argument rather than cover a news story, but they are not explicitly labeled as editorials by the Wall Street Journal.

the opinion piece annotations of data sets OP1 and OP2 in Table 1 have been manually refined.