Given a term, the path from the root to the corresponding leaf is unique.

given the term frequencies n j of document j as defined in (1), the number of times n jl child l ∈ C(k) was picked at node k is known and fixed.


Firstly, smaller values for both tend to be associated with lower levels of the trees, while large values are with higher levels of the trees.

we see that PCluster tend to have subtrees of vocabulary terms that are positively correlated with each other-i.e.


This has also been observed for other morphological learners (Goldwater et al., 2006).

we use types in learning and inference, and effectively enforce the constraint that words can have only one segmentation per type.


However, the lexicon prior alone incorrectly favors the trivial segmentation that shatters each word into characters, which results in the smallest lexicon possible (single characters).

we also impose a corpus prior: an exponential prior on the number of mor-phemes used to segment each word in the corpus, which penalizes over-segmentation.


This is particularly true for languages with predominantly inflectional morphology, such as Arabic, Hebrew, and English.

our model uses separate lexicons for prefixes, stems, and suffixes.


For example, deleting l from the Hebrew word lyhwh (to the lord) results in yhwh (the lord).

dELORTRANS1 forces legal words to compete against each other for probability mass, which seems like a misguided objective.


Thus DELORTRANS1 forces legal words to compete against each other for probability mass, which seems like a misguided objective.


If, however, lAlrb remains a single morpheme, and we still segment Alrb into Al-rb, then we introduce two new morphemes into the lexicons, and we will be penalized by the lexicon prior accordingly.

we must segment the whole corpus jointly, making exact inference intractable.


Many of the new features use syntactic information, and in particular depend on information that is available only inside a syntax-based translation model.

they widen the advantage that syntaxbased models have over other types of models.


Finally, we note that several of the features (the third-and eighth-ranked reward and twelfthranked penalty) shape the translation of shuo 'said', preferring translations with an overt complementizer that and without a comma.

these features work together to attack a frequent problem that our targetsyntax features also addressed.


Meanwhile, translation grammars have grown in complexity from simple inversion transduction grammars (Wu, 1997) to general tree-to-string transducers (Galley et al., 2004) and have increased in size by including more synchronous tree fragments (Galley et al., 2006;DeNeefe et al., 2007).

of these trends, the syntactic component of machine translation decoding can now account for a substantial portion of total decoding time.


Deterministic reordering based on syntactic analysis for the input sentences provides a good way of resolving long distance reordering, without introducing complexity to the decoding process.

it can be efficiently incorporated into phrase-based systems.


Normally in a phrase-based decoder, very long distance reordering is not allowed because of efficiency considerations.

it is very difficult in general to translate English into Korean with proper word order.


In such cases, the change in order from English to Korean is similar to the verb rule, except that the head adjective itself should be in front of the verbs.

in our adjective precedence rule in the second panel of Table 1, we group the auxiliary verb, the passive auxiliary verb and the negation and move them together after reversing their order.


In order to do this, we disallow any movement across punctuation and conjunctions.

for sentences like "John hit the ball but Sam threw the ball", the reordering result would be "John the ball hit but Sam the ball threw", instead of "John the ball but Sam the ball threw hit".


A key difference between their approaches and ours is that they do not perform reordering during training.

they would need to rely on reorder units that are likely not violating "phrase" boundaries.


They come from various sources with a substantial portion coming from the web after using simple heuristics to identify potential document pairs.

for some documents in the training data, we do not necessarily have the exact clean translations.


One of our motivations of using the precedence reordering rules is that English will look like SOV languages in word order after reordering.

even monotone decoding should be able to produce better translations.


Another possible reason is that after the reordering rules apply in preprocessing, English sentences in the training data are very close to the SOV order.

eM training becomes much easier and word alignment quality becomes better.


As a result, EM training becomes much easier and word alignment quality becomes better.

a hierarchical phrase-based system can extract better rules and hence achievesbetter translation quality.


However, distortion model can not model long-distance reordering, due to the lack of context information, thus is difficult to predict correct orders under different circumstances.

while phrase-based SMT moves from words to phrases as the basic unit of translation, implying effective local reordering within phrases, it suffers when determining phrase reordering, especially when phrases are longer than three words (Koehn et al., 2003).


During decoding, the blocks are produced by applying either one of two reordering rules on two smaller blocks, or applying lexical rule (3) on some source phrase.

the score of a block A is defined as are respectively the usual and incremental score of language model.


As for target side, we ran the Moses scripts to classify target language words using the mkcls toolkit before running GIZA++.

we directly use its classification result, which generate 50 classes with 2 optimization runs on the target sentences.


By grouping words into concepts, we effectively overcome the data distribution difference of both domains.

we may reduce the number of parameters required to model the target domain data, and improve the quality of the estimated parameters in the domain transfer.


Our empirical study shows that better adaptation is obtained with a 50-topic LaSA model.

we set the number of topics N as 50, and define the context view window size as {-3,3} (i.e.


Each NE type has different domain features.

the performance stability of each NE type recognition is very important in the domain transfer.


In other words, a syllable is not allowed to begin with a consonant cluster that is not found at the beginning of some word, or end with a cluster that is not found at the end of some word (Goslin and Frauenfelder, 2001).

a word like admit [dmÁt] must be syllabified [d-mÁt] because [dm] never appears word-initially or word-finally in English.


When syllabifying a word, SSP states that sonority should increase from the first phoneme of the onset to the syllable's nu-cleus, and then fall off to the coda (Selkirk, 1984).

in a word like vintage [vÁntÁ], we can rule out a syllabification like [vÁ-ntÁ] because [n] is more sonorant than [t].


We also discovered that accuracy can be improved by numbering the tags.

in our tagging scheme, the single-syllable word strengths [str AEÌs] would be labeled with the sequence {O1 O2 O3 N1 C1 C2 C3}.


The Learned EBG approach of van den Bosch (1997) achieves 97.78% word accuracy when training on approximately 60K examples.

our results represent a nearly 50% reduction of the error rate.


2007report a disappointing word accuracy of 54.14% for their legality-based implementation, which does not accord with the results of our categorical approaches on English CELEX.

we also apply our methods to the dataset they used for their experiments: the NETtalk dictionary.


Because of this, for all pairs of adjacent words within an entity, there will be a binary rule applied where one word will be under the left child and the other word will be under the right child.

we compute features over adjacent words/labels when computing the features for the binary rule which joins them.


Early experiments showed that it was difficult for even our baseline named entity recognizer, based on a state-of-the-art CRF, to learn these types of entities.

3 we decided to merge all but the three most dominant entity types into into one general entity type called misc.


But the CCG account is a competence model as well as a performance model, in that it seeks to unify category representations used in processing with learned generalizations about argument structure; whereas the model described in this paper is exclusively a performance model, allowing generalizations about lexical argument structures to be learned in some other representation, then combined with probabilistic information about parsing strategies to yield a set of derived incomplete constituents.

the model described in this paper has a freer hand to satisfy strict working memory bounds, which may not permit some of the alternative composition operations proposed in the CCG account, thought to be associated with available prosody and quantifier scope analyses.


The number of language models is written K; the number of words is W ; the length of the document is T ; and the depth of the hierarchy is L. For hierarchical segmentation, the vector y t indicates the segment index of t at each level of the topic hierarchy; the specific level of the hierarchy responsible for w t is given by the hidden variable z t .

is the index of the language model that generates w t .


Here we use a symmetric Dirichlet prior α, though asymmetric priors can easily be applied.

far we have treated the hidden variables z as observed.


The matrix A can be constructed by iterating through the layers and then iterating: u from 1 to T ; v from u+1 to T ; and t from u to v + 1.

the time cost for filling A is O(LT 3 ).


Intuitively, efficient inference is possible because the location of each segment boundary affects the likelihood of only the adjoining segments at the same level of the hierarchy, and their "children" at the lower levels of the hierarchy.

the observation likelihood at each level decomposes across the segments of the level.


In contrast, our model captures the entire sequence of topics using a compact representation.

we can explicitly and tractably model global discourse-level constraints.


Also, our ASR only predicts words once there is acoustic evidence for several phonemes and every phoneme (being modelled by 3 HMM states) must have a duration of at least 3 frames.

some errors relative to the final hypothesis occur because the ASR 7 In figure 1, hypotheses 1, 2 and 3 would be cropped away.


Obviously, using only information up to t − ∆ has averse effects on correctness as well, as this measure evaluates the word sequences up to w gold t which may already contain more words (those recognised in ]t − ∆, t]).

to be more fair and to account for the lag when measuring the module's correctness, we additionally define fair r-correctness which restricts the evaluation up to time t − ∆: w hypt t−∆ = w gold t−∆ .


However, as noted in Section 3, short vowels and other diacritic markers are typically not orthographically represented in MSA texts.

aSR systems typically do not output fully diacritized transcripts.


Diacritization is generally not necessary to make the transcript readable by Arabicliterate readers.

entries in the decoding pronunciation dictionary consist of undiacritized words that are mapped to a set of phoneticallyrepresented diacritizations.


Since, as noted above, it is standard to produce unvocalized transcripts when recognizing MSA, we must map word pronunciations to unvocalized orthographic output.

for each diacritized word in our training pronunciation dictionary, we remove diacritic markers and replace Hamzat-Wasl ({), <, and > by the letter 'A', and then map the modified word to the set of pronunciations for that word.


We find that the difference between the two is only 1.03% (with 246,376 phones, 557 deletions, 1696 substitutions, and 277 insertions).

adding the BASEPR rules to XPR does not appear to contribute a great deal to the representation chosen by forced alignment.


1 It should be noted that phrase-based models can translate multiple words as a unit, and therefore capture non-compositional meaning.

by default if the training data is processed such that, for example, aufnahme, in its sense of recording, is segmented into two words, then more paths in the lattices be-  come plausible translations.


Our model defines a conditional probability distribution over virtually all segmentations of a word w. To train our model, we wish to maximize the likelihood of the segmentations contained in the reference lattices by moving probability mass away from the segmentations that are not in the reference lattice.

we wish to minimize the following objective (which can be computed using the forward algorithm over the unpruned hypothesis lattices): The gradient with respect to the feature weights for a log linear model is simply: To compute these values, the first expectation is computed using forward-backward inference over the full lattice.


Furthermore, when even this minimal data cannot be acquired for a particular language (as was the situa-tion we faced with Hungarian and Turkish), we have demonstrated that the parameters obtained in one language work surprisingly well for others.

with virtually no cost, this model can be used with a variety of diverse languages.


Starting from an SMT model trained initially on bilingual data, the problem is to minimize the human effort in translating new sentences which will be added to the training data to make the retrained SMT model achieves a certain level of performance.

given a bitext L := {(f i , e i )} and a monolingual source text U := {f j }, the goal is to select a subset of highly informative sentences from U to present to a human expert for translation.


Compared to the corresponding features computed using the FA results based on transcriptions, these promising pronunciation features that can be obtained practically, show some reduction in quality (from 13.4% to 21.1%) but are still usable.

our proposed twostage method for pronunciation feature extraction is proven to be a practical way for the computation of features that have acceptable performance.


The recognized words with high confidence scores tend to be correctly recognized.

focusing on words recognized with high confidence scores may reduce the negative impact caused by recognition errors on the quality of the likelihood related features.


Clearly, with the increase of the confidence score, more words tend to be accurately recognized.

it is reasonable to only use likelihoods estimated on the hypothesized words with high confidence scores for extracting likelihood related features.


We find that the best RMSE for two variables (0.042) is only slightly lower than that for one (0.043), so it is doubtful that a second variable helps.

our analysis suggests that among our candidates, the best predictor of optimism is simply where γ = 0.938, with this value being independent of domain, training set size, and n-gram order.


The correlation between the actual and predicted cross-entropy on the evaluation data is 0.9997; the mean absolute prediction error is 0.030 nats; the RMSE is 0.043 nats; and the maximum absolute error is 0.166 nats.

on average we can predict test performance to within 3% in perplexity, though in the worst case we may be off by as much as 18%.


If we examine how much different ranges ofλ i contribute to the overall value of , we find that the great majority of the mass (90-95%+) is concentrated in the rangeλ i ∈ [0, 4] for all ten models under consideration.

to a first approximation, the reason that eq.


Surprisingly, for exponential language models, the number of model parameters seems to matter not at all; all that matters are the magnitudes of the parameter values.

one can improve such models by adding features (or a prior model) that reduce parameter values while maintaining training performance.


Let's begin with a word trigram model First, we would like to convert this model into a class-based model.

without loss of generality, we have it seems reasonable to use the distributions p ng (c j |w j−2 w j−1 ) and p ng (w j |w j−2 w j−1 c j ) as the starting point for our class model.


(2) tells us that with respect to test set performance, the number of model parameters seems to matter not at all; all that matters are the magnitudes of the parameter values.

one can improve exponential language models by adding features (or a prior model) that shrink parameter values while maintaining training performance, and from this observation we develop Heuristics 1 and 2.


Note that the clusters delicately balance the information in the siteunaware labels nb(•) and the site structure encoded in the URLs (given by g i 's).

the label cls(p) can be thought of as a smoothed version of nb(p).


When the websites are as diverse as represented in S all , there is no perfect automatic way to generate the ground truth labels.

we sampled a number of pages for human labeling as follows.


On a uniform sample of 100 such pages in S all , 90% were found to be genuine reviews.

we obtained a collection of webpages with slightly noisy positive labels.


Furthermore, it provides support for the idea that this influence is mediated by "grammatically relevant" semantic properties, demonstrating that these accounted for approximately 75% of the variance in implicit sentiment expressed by alternative headlines describing the same event.

far, we have established a predictive connection between syntactic choices and underlying or implicit sentiment, mediated by grammatically relevant semantic properties.


However, even here we note that the MSE is low, suggesting that the frequencies of stream counts (found in top K true counts) are very close to the true counts.

we conclude that the quality of the 5-gram stream counts produced for this value of ǫ is quite high (in relation to the true counts).


However, search engines are not designed for NLP research and the reported hit counts are subject to uncontrolled variations and approximations.

several researchers have collected corpora from the Web by themselves.


In contrast, less content-loaded words like "somewhat" tend to occur on average once in documents, and thus have similar values for collection and document frequencies ( f w ≈ df w ).

more informative words can be less accurately estimated by the Poisson distribution, which is based on the simplistic assumption that the expected number of occurrences of word in a document can be estimated by its total number of occurrences in the corpus.


One practical problem with this approach, apart from the sheer number of the rules that result from the extraction procedure, is that the parsing complexity of all synchronous formalisms that we are aware of is exponential in the rank of a rule, defined as the number of nonterminals on the righthand side.

it is important that the rules of the extracted grammar are transformed so as to minimise this quantity.


However, the number of nonterminals required to recreate our TSG grammars in a PCFG would be exorbitant.

our model should be better able to learn specific lexical patterns, such as full noun-phrases and verbs with their sub-categorisation frames, while theirs are better suited to learning subcategories with larger membership, such as the terminals for days of the week and noun-adjective agreement.


Our problem therefore becomes one of identifying the posterior distribution of e given t, which we can do using Bayes' Rule: Since the sequence of elementary trees can be split into derivations, each of which completely specifies a tree, P (t|e) is either equal to 1 (when t and e are consistent) or 0 (otherwise).

the work in our model is done by the prior distribution over elementary trees.


The type of compositional semantics we are interested in is heavily rooted in verb semantics, which is particularly appropriate for the instructional text we are working with.


Now, recall from Step 1 that the mincut finder classifies a node n as anaphoric if and only if w sn ≥ 0.5.

hence, to achieve the aforementioned goal, we just need to ensure the property that w sn ≥ 0.5 if and only if P A (m n ) ≥ c. we compute w sn using a sigmoid function: where α is a constant that controls the "steepness" of the sigmoid.


Interdisciplinary review panels are often called upon to review proposals in a wide range of areas, some of which may be unfamiliar to panelists.

they must learn about a new discipline "on the fly" in order to relate their own expertise to the proposal.


The key to a more efficient solution is to note that θ(µ) is invariant to scaling of µ.

we can store S = instead of µ and make the following sparse update: For both iEM and sEM, we also need to efficiently compute θ(µ).


There are many differences between written text and speech -meetings in particular.

our goal in this paper is to investi-gate whether we can successfully apply some existing techniques, as well as propose new approaches to extract keywords for the meeting domain.


If the total term frequency of all the words in one cluster is high, it is likely that this cluster contributes more to the current topic from a thematic point of view.

we want to assign higher weights to the words in this cluster.


When human annotators choose key phrases (containing more than one word), we split them into words and measure the matching words.

when the system only generates five keywords, the upper bound of the recall rate may not be 100%.


Specifically, we know that d ≥ τ , where d is the total duration of the pause (with d = ∞ at the end of a turn 1 ).

p (F |O t ) can be rewritten using Bayes rule as where p (F |O) is the probability that the user released the floor without any knowledge of the duration of the pause, and p (d ≥ τ |O) is the probability that the pause will last at least τ ms. We further decompose p (d ≥ τ |O) into 1 Note that this is an approximation since the user could start speaking again after releasing the floor to reestablish the channel (e.g.


However, in the vast majority of cases, the time after which the user resumes speaking is significantly longer than the time the system takes to endpoint.

p (F |O t ) is a function of p (F |O) and p (d ≥ τ |O, U ).


Then the top ten percent of the respondent-centered meaned Likert ratings were marked as positive for the trait, and the bottom ten percent were marked as negative for a trait.

each respondent labels the other speaker as either positive, negative, or NA for each of the three traits.


Since the start and end of each turn were timemarked by hand, each feature was easily extracted over a turn, and then averages and standard deviations were taken over the turns in an entire conversation side.

the feature F0 MIN for a conversation side was computed by taking the F0 min of each turn in that conversation side (not counting zero values of F0), and then averaging these values over all turns in the side.


However, cell (1, 2) is closed, hence there is only one midpoint at which (1, 3) can be built (m = 1).

the amount of work to parse the sentence will be less than the worst-case quadratic bound based on this processing savings in open cells.


By Lemma 2, there are at maximum kN cells that fall in either case 2 or case 3.

by Lemma 1, the amount of work for each of these cells is bounded by ck for some constant c. the theorem is proved.2 If |E| ≤ k for a constant k, the theorem proves the complexity will be O(N ).


These models are developed to test the intuition that the erroneous parts of speech repairs (reparanda) are not generated or recognized as such while occurring, but only after they have been corrected.

they are designed to minimize the differences in grammar rule applications between fluent and disfluent speech containing similar structure.


One important thing to notice is that the EDITED label contains no information about the structure beneath it.

a parser trained on this default annotation has no information about the attempted constituent type, which, in the case of restarts would obviously be beneficial.


A second problem with this representation is evident in certain multi-word repairs such as the one in Figure 2 that require an extra right branch off of the main left branching structure of the tree.

a multi-word reparandum structure requires an extra unary rule application at the left-corner of the sub-tree, in this case S/VP, relative to the inline structure of the fluent version of that phrase.


That is, given the preceding context, the player tossed a frisbee must begin a reduced RC, such that there is no local ambiguity.

if comprehenders are making full use of the linguistic context, (1a) should be no more difficult than the other examples, except insofar as ambiguous verbs are harder than unambiguous verbs, and reduced RCs are harder than unreduced RCs, predicting there would be only the two main effects of RC reduction and verb ambiguity on reading times for the tossed a frisbee region.


To calculate the posterior, we first use the definition of conditional probability to rewrite it as In a context-free grammar, given the syntactic category that dominates a string of words, the words' probability is independent from everything outside the category.

this is equivalent to 5.1.1 Posterior: the numerator's first term The first term in the numerator P (S * ⇒ w i 0 Xν) can be computed from a parse of w i 0 by summing forward probabilities of the form 5.1.2 Posterior: the denominator Similarly, the denominator P (S * ⇒ w j 0 λ) can be computed from a parse of w j 0 by summing forward probabilities of the form for all Y.


Switching back from event space notation and substituting this term into Eq.

7, we now have by conditioning all terms on E 0 , the presence of at least i words, we have transformed the probabilities we need to calculate into these four terms, which are easier to calculate from the parser.


We show that using exact inference provides an increase of up to one BLEU point on two language pairs (French-English and German-English) in comparison to decoding using the ReWrite decoder.

the ReWrite decoder performs respectably but can be improved slightly, albeit at the cost of efficiency.


Oracle extraction is a nontrivial task because computing the similarity of any one hypothesis requires information scattered over many items in the hypergraph, and the exponentially large number of hypotheses makes a brute-force linear search intractable.

efficient algorithms that can exploit the structure of the hypergraph are required.


The length penalty in the BLEU metric, in particular, is only an indirect and weak indicator of adequacy.

sMT work has been driven to optimize systems such that they often produce translations that contain significant role confusion errors despite reading fluently.


The trained UBM has 64 mixture components.

the dimension of the GMM mean supervectors is 2560.


There are 66 one-word sentences in the video data (18.6% of the total).

the length of the sentences from the video data tends to be short.


We had SINCA acquire noun concepts using both video and questionnaire data.

we have showed that spoken language is well suited to SINCA's algorithm for acquiring noun concepts.


Similarly, the suite can predict a negative assign- ment for each of the 49 classes.

precision is calculated as the fraction of correctly predicted instances to the number of examplars where a prediction was made.


Also, many classifiers that are available in Malay are not used in Indonesian, and the small set of Indonesian count classifiers that are not used in Malay do not form part of the query set, so no confusion results.

it seems that greater difficulty would arise when attempting to predict count classifiers for Indonesian nouns, as their optionality and blocking from Malay cognates would introduce noise in cases where language identification has not been used to generate the corpus (like the Web) -hand-constructed corpora might be necessary in that case.


This process results in a synthetic children's speech model designed on the basis of the female model.

for a new language an effective synthetic children's acoustic model can be derived without the need of children's data (Hagen et al., 2008).


The task of FrameNet-based shallow semantic parsing can be implemented as a combination of multiple specialized semantic labelers as those in (Carreras and Màrquez, 2005), one for each frame.

the general semantic parsing work-flow includes 4 main steps: (i) Target Word Detection, where the semantically relevant words bringing predicative information (the frame targets) are detected, e.g.


Nonetheless, since we aim at modeling an SRL system for a new language (Italian) and a new domain (dialog transcriptions), the above features may result ineffective.

to achieve independence on the application domain, we exploited Tree Kernels (Collins and Duffy, 2002) over automatic structural features proposed in (Moschitti et al., 2005;.


However, manually building such graphs for various applications may be labor intensive and time consuming.

we have tried to investigate how to build this graph automatically.


However, each feature vector contains small number of non-zero terms (<20 features) compared to the feature space (>2000 features).

most pairs of utterances share no common feature, and their distance is close to 1.0.


These words are likely to contribute terms with high partial scores.

the early-stopping step fires earlier and becomes more effective.


The second inequality arises from the fact that the value of a combination feature is equal to or less than the values of its original features.

we examine (5) and check whether or not |v k | will be larger than C. If not, we can remove the feature safely.


Since the examples are sorted in the order of their y |α i,y |, the bound will become tighter quickly.

many combination features are filtered out in the early steps.


Each experiment fits a model using data from 19 speakers and tests on the remaining speaker.

when we test our models, we are testing the ability to classify utterances of an unseen speaker.


But for some languages, collecting parallel corpora is not easy.

many researchers paid attention to bilingual dictionary extraction from comparable corpora (Fung, 2000;Chiao and Zweigenbaum, 2002;Daille and Morin, 2008;Robitaille et al., 2006;Morin et al., 2007;Otero, 2008), in which texts are not exact translation of each other but share common features.


Our approach uses data-driven syntactic analyzers for acquiring dependency relations automatically.

it is easier to adapt our approach to other language pairs.


It is clear that the distance between the vector of '经济学(economics)' and 'economics' is much smaller than that between '经济学' and 'medicine'.

the pair '经济学/economics' is extracted successfully.


Domain-dependent training data are particularly difficult to obtain.

in conventional system development, system developers determine the types of LM and LUM by trial and error.


Every LM and LUM has some advantages and disadvantages, so it is difficult for a single combination of LM and LUM to gain high accuracy except in a situation involving a lot of training data and effort.

using multiple speech understanding methods is a more effective approach.


For example, finitestate grammar LMs and FST-based LUMs achieve high accuracy in recognizing and understanding ingrammar utterances, whereas out-of-grammar utterances are covered by N-gram models and LUMs based on WFST and keyphrase-extractors.

it is more possible that the understanding results of MLMU will include the correct result than a case when a single understanding model is used.


Actively sampled data can have very different characteristics than passively sampled data.

it's promising to investigate using different inference procedures during AL than are used during passive learning (PL).


The trend toward balanced datasets during AL could mislead and cause us to underestimate the PA.

our next algorithm aims to set the PA based on the ratio of neg to pos instances in the entire corpus.


Increasing the relative penalty for slack error on positive training points will not have much impact if (nearly) all of the pos train points are already classified correctly.

in situations where models are already achieving nearly 100% recall on their train data, InitPA is not expected to outperform CurrentPA.


If the left span is relatively short, the contribution of the left NP to the total score of the new edge is small, so this prominence is misplaced.

we repeat the above process with the higher-level lists organized by right span instead of left.


Nevertheless, the distributions are not identical.

again, while we predict that using this style of dependency syntax features might improve performance of compression systems within a single corpus, we would not expect trained models that rely on these syntactic dependency features to generalize in an ideal way between corpora.


However, our goal is to make a better system on day 2 but try to minimize human efforts on day 1.

we raise two questions: 1) Can we still obtain improvements by not using all of day 1 data?


Explicit YQQs are less interesting, because the user's temporal intention is clearly specified in the query.

ranking for these types of queries is relatively straightforward.


Then, we removed all the years from explicit YQQs.

implicit YQQs are obtained from explicit YQQs.


Not surprisingly, using adaption achieved much better results than without using adaption.

these experiments prove the effectiveness of our proposed methods.


Syntactic parsing is typically one of the first steps in a text mining pipeline.

any improvement in performance would have a ripple effect over all processing steps.


This is time consuming and prone to error.

unlike typical text simplification that emphasizes syntactic transformation of sentences, our approach utilizes a named entity recognition engine, BANNER (Leaman & Gonzalez, 2008), to replace multi-word gene names with single-word placeholders.


When "η=0", it indicates that no HFDS features are used to build the correspondence vectors; while "η=1" indicates that all features are equally used to build the correspondence vectors, that is to say, proposed FW-SCL algorithm is simplified as traditional SCL algorithm.

proposed FW-SCL algorithm could be regarded as a generalized version of traditional SCL algorithm.


• MICA is based on an explicit phrase-structure tree grammar extracted from the Penn Treebank.

mICA can associate dependency parses with rich linguistic information such as voice, the presence of empty subjects (PRO), wh-movement, and whether a verb heads a relative clause.


Chinese, and some other languages, e.g., Japanese do not have delimiters between words.

segmentation is usually an indispensible step for further processing, e.g., Part-of-Speech tagging, parsing, etc.


Our explanation might be that the trigger, unlike the predicate in the semantic role labeling task, can not only be a verb, but also can be a noun or other types.

the syntactic position for the trigger in the parse tree is much more flexible than the predicate in Semantic Role Labeling.


Both rules and lexicons are grounded on linguistic knowledge and generated by expert linguists.

the build process is expensive and the system is difficult to maintain (Bennett and Slocum, 1985).


In our case, an APE system based on statistical models will be trained to correct the translation errors made by a RBMT system.

both RBMT and SMT technologies will be combined in order to increase the overall translation quality.


One of the desired qualities for this measure was that it should pose little effort to the human evaluator.

a binary measure was chosen, the suitability, where the translations are identified as suitable or not suitable.


Changes between RBMT and APE were minor but useful.

aPE did not pose a system degradation with respect to the RBMT.


The need for two distinct classifiers can be illustrated with the following example: ( (1a,b) are grammatical while (1c) is hardly acceptable, and no simple precedence rule can be learned from pairs of constituents in (1a) and (1b): the temporal adjunct earlier today can precede or follow each of the other constituents dependent on the verb (she, him, an email).

the classifier which determines the precedence relation is not enough.


The question in the example in the Introduction section is concerned with the object of the predicate "is" rather than of the other predicates "understand" or "was".

a pre-processing step is needed to correctly identify the object (in this example) that is being asked, which is described next.


(v) Then we perform recognition with the new anchored network and extract the best path through it.

we enforce partial alignment of the audio with the question given, while the regular recognition LM is still used for the parts outside the question.


Since the application acts like an interactive dictionary of sign language, primary concern is to return correct results no matter how few they are.

high precision is appreciated much more than high recall rates.


Long named entities are frequently abbreviated in oral Chinese language for efficiency and simplicity.

abbreviation modeling is an important building component for many systems that accept spoken input, such as directory assistance and voice search systems.


This multistream approach enabled a later integration of feature streams and increased flexibility in modelling the interdependences between the different streams, enabling some accommodation for asynchrony and multiple time scales.

use of the richer DBN multistream model resulted in a significant lowering of the action error rate to around 13%.


Because of the ambiguity packing, zero or more local analyses may exist for each passive edge on the chart, and the cross-combination of the packed daughter edges is not guaranteed to be compatible.

expensive unification operations must be reapplied during the unpacking phase.


), regardless of case, number or gender.

the grammar can generate millions of (incorrect) parse trees for a given sentence, and has a flat posterior distribution.


Tested across six domains, our system outperforms all non-oracle baselines including the best domain-independent parsing model.

we are able to demonstrate the value of customizing parsing models to specific domains.


Given that some aspects of syntax are domain dependent (typically at the lexical level), single parsing models tend to not perform well across all domains (see Table 1).

statistical parsers inevitably learn some domain-specific properties in addition to the more general properties of a language's syntax.


We feed these similarity measures into a regression model which learns how domain dissimilarities hurt parse accuracy.

to parse a target domain, we need only find the input that maximizes the regression function -that is, the highest scoring mixture of source domains.


We preprocessed the corpora to standardize many of the annotation differences.

our results on them may be slightly different than other works on these corpora.


According to the independence assumption in Equation 2, with the addition of prosodic edge X → b j , the same sub-tree also has the highest probability, denoted by P (i, j, X, 1), of generating word sequence w j i together with the break index sequence b j i .

we have: The structural constraint that a break index is only generated by the highest nonterminal that covers the preceding word as the right-most terminal enables a dynamic programming algorithm to compute P (i, j, X, 0) and thus P (i, j, X, 1) efficiently.


Using the adjective unigram probability as a learning feature will mislead the system into trusting an adjective that is common but has a poor bigram affinity to the noun in the phrase.

we eliminate this feature for both the SVMs and the decision tree learning.


Graph creation & clustering: In the setting of KM, a collocation is a juxtaposition of two nouns within the same paragraph.

each noun is combined with any other noun yielding a total of N 2 collocations for a paragraph with N nouns.


In addition, a context-dependent meaning for a word is determined by choosing one of the vectors in its set based on minimizing the distance to the vector representing the current context.

the model supports judging the similarity of both words in isolation and words in context.


bat and club) at the cost of lower robustness to noisy clusters that might be introduced when K is large.

when contextual information is available, AvgSim and MaxSim can be modified to produce more precise similarity computations: , the maximum likelihood cluster for w in context c. AvgSimC corresponds to soft cluster assignment, weighting each similarity term in AvgSim by the likelihood of the word contexts appearing in their respective clusters.


Of course, real translations are almost always at least partially syntactically divergent.

it is unreasonable to expect perfect matches of any kind between the two sides' syntactic trees, much less expect that those matches be well explained at a word level.


We cannot efficiently compute the model expectations in this equation exactly.

we turn next to an approximate inference method.


For a single-word source phrase, if it can be translated separately, it is always translated as a unit in the context of phrase-based decoding.


A word c k can be possibly labeled as B y by the first classifier and E y by the second classifier.


Such an approach is only tractable for small bilingual parallel corpora that would result in a small H, but not for large corpora containing millions of sentences, such as those described in Section 4.1.

we approximate H with a graph H that only contains nodes "near" to the node representing the query phrase.


One reason for this is that about 90% of articles are used correctly by ESL learners; this is higher than the performance of state-of-the-art classifiers for article selection.

not using the writer's article, when making a prediction, may result in making more mistakes than there are in the data.


So the real baseline on the task is "do nothing".

to determine the baseline for a correction task, one needs to consider the error rate in the data.


Since the methods of error generation described in Section 5 rely on the distribution of articles and article mistakes and these statistics are specific to the first language of the writer, we conduct evaluation separately for each source language.

for each language group, we train five system types: one system is trained on clean English data without errors (the same classifier for the three language groups) and four systems are trained on data with errors, where errors are produced using the four methods described in Section 5.


CLC is commercially available, but it is obvious that for many researchers such a corpus will be too expensive and they will have to create or license their own error-annotated corpus.

the question of whether one could use less annotated data to train a meta-classifier and still achieve reasonable results becomes important.


Since the language model is generally more accurate than the error-specific classifier, the meta-classifier tends to trust its score more than that of the classifier.

we see this kind of error quite frequently.


The multinomials θ k , π k and ψ E create non-local dependencies in our model, breaking our HMM dynamic programing.

we adopt Gibbs sampling as our inference engine.


The majority of tweets are not real stories, but rather updates on one's personal life, conversations, or spam.

simply running a first story detection system on this data would yield an incredible amount of new stories each day, most of which would be of no interest to anyone but a few people.


Once we have threads of tweets, we are interested in which threads grow fastest, as this will be an indication that news of a new event is spreading.

for each time interval we only output the fastest growing threads.


Unfortunately, the latter assumption is not fulfilled in many applications.

model adaptation is necessary when training data is not matched (not drawn from same distribution) with test data.


In many cases shifting from one domain to another domain, initial trained decision boundaries (on out-of-domain data) result in high conditional entropy for the new domain, due to mismatch between distributions.

there is a need to adjust model parameters such that decision boundaries goes through low-density regions of the distribution.


Although this is a suitable/ideal assumption for classification, in most practical problems this assumption is not satisfied and often classes overlap.

we can not expect the conditional entropy to be convex in this situation and to achieve minimization w.r.t parameters (other than the trivial solutions).


Also ∂H θ (Y|X) ∂θ p Y|X) ∂λ | because we only have one parameter.

interpolation weight will be chosen by the following criterion For the purpose of estimating one parameter λ, we use γ = 1 in the above equation The large vocabulary continuous speech recognition (LVCSR) system used throughout this paper is based on the 2007 IBM Speech transcription system for GALE Distillation Go/No-go Evaluation (Chen et al., 2006).


As it can be seen from this figure just considering the conditional entropy will result in a non-convex objective function whereas adding the entropy-stability term will make the objective function convex.

for the purpose of the evaluation, we show the results for estimating λ directly on the tran- it can be seen that the new unsuper-vised method results in the same performance as supervised adaptation in speech recognition task.


However, there is no such corpus available.

we follow the approach of Gravano et al.


The F-measure scores for commas and periods are also comparable to the prosody-based work of (Christensen et al., 2001), with the precision of the period slightly lower, but compensated by recall.

our system can perform additional formatting, while retaining a reasonable capitalization and punctuation performance.


Additionally, the gold standard is itself imperfect, with typical estimates of professional disagreement around five percent.

we judge the quality of Mechanical Turk data by comparing the performance of one LVCSR system trained on Turker annotation and another trained on professional transcriptions of the same dataset.


This suggests that sequence models, which jointly assign all labels in a sequence, may be more appropriate.

we begin incorporating context by moving from classification to sequence models.


We found similar results in the test data.

we expect that even a minimal amount of context based on the labels of adjacent bins will help.


However, we are interested in language identification of web documents, which can be in any language, including languages that do not overtly mark word boundaries, such as Japanese, Chinese and Thai; while relatively few languages fall into this categories, they are among the most populous web languages and therefore an important consideration.

approaches that assume a language is spacedelimited are clearly not suitable for our purposes.


Phrasal translations might indirectly capture MWEs, but they are not distinguished from any other n-gram.

the usefulness of explicitly modeling MWEs in the SMT framework has not yet been studied systematically.


In F • G • E the NTs (nodes) are the cross product of pairs of states from E, the NTs from G, and pairs of states in F .

synchronous parsing is the task of computing F • G • E. Since composition is associative, we can compute this quantity either as (F .


a string e in the target will result in the will result in a parse forest that exactly derives f, e , which is the goal of synchronous composition.

in synchronous parsing, we seek to compute F • G • E. Since composition is associative, we can compute this quantity either as (F .


As these nodes correspond to concatenation and wrapping operations, each inner-node production has length O(ϕ(p)).

the sum of the lengths of the productions created from |p| is O(|p| + ρ(p)ϕ(p)).


Since the rank of a production is always smaller than its length, this is reduced to O(|p|ϕ(p)).

the size of the normal-form transformation of an LCFRS G of fan-out ϕ is O(ϕ|G|) in the worst case, and linear space in practice, since the fan-out is typically bounded by a small integer.


(2) Remote Evaluation An effective utility evaluation will require users with a diversity of prior knowledge and computer experience.

we asked the second group of 11 users in a remote usability testing mode (Hammontree et al., 1994).


In fact, the average time the users spent using (III) was only about 7.2 minutes.

we can conclude that crossdocument IE can produce more informative summaries in a more efficient way.


In the Remote Testing, the accuracy results from the three levels are as follows: 21/37, 28/37 and 31/36.

both user groups benefited from using IE techniques, but the enhancements vary a lot.


One would expect the literal part of the expression to exhibit cohesive ties with the context, hence the cohesion based features may fail to detect this type of figurative usage.

the performance of the GMM is lower for figuratively used words ('nw') than for idioms ('nsa', 'nsu').


Notice that after achieving some improvements the performance on blogs is still much worse than on review data.

we performed a manual error analysis for a better understanding of the difficulties of sentiment analysis on blog data, and identified the following challenges.


Alternatively, we may want to treat each instance of w in a document that contains an instance of w to be a co-occurrence event.

if w appears three times in a document that contains two instances of w, the former method counts it as one co-occurrence, while the latter as six co-occurrences.


But the Chomskyan approaches fit the same paradigm, as do many other theories of syntax.

there is no theory-neutral representation of a sentence or a set of sentences, because every representation needs a theory for us to extract its meaning!


Similarly, for the word collaborate, the sense "work together on a common enterprise of project" (objective) corresponds to "Ü , " in Chinese translation, and "cooperate as a traitor" (subjective) corresponds to "£(, H r".

subjectivity information should be effective for improving lexical translation for what we previously (Su and Markert, 2008) termed subjectivity-ambiguous words, i.e.


However, we do not necessarily need dedicated human translated data as we could also use existing parallel texts in which the target word occurs.

we think that a supervised approach to lexical substitution is feasible.


Queries with click frequency under 10 are discarded.

84,703 unique queries remained, which form our universe of queries.


This involves the desired parameters, which we solve for by estimating the others from data, as described next.

if we have estimates for o 1 -related probabilities, we can derive o 2 -related probabilities via Equation 1.


Moreover, emphasis is placed on performing well across all documents and across all genres.

it is important for a research team to be able to evaluate their system using HTER, or at least determine the ranking of the documents according to HTER, for purposes of error analysis.


While HTER has been shown to correlate quite well with human judgment of MT quality, it is quite challenging to obtain HTER scores for MT output, since this would require hiring and training human subjects to perform the editing task.

other metrics such as BLEU or TER are used as proxies for HTER.


Moreover, after this kind of generalization, the syntactic structure is used as a whole in exact matching all the same.

as an alternative to the exact matching, tree kernel based methods can be used to implicitly explore the substructures of the syntactic structure in a highdimensional space.


The model is modularized in such a way that running EM on all variables simultaneously would be very difficult.

we adopt a variational approach which optimizes various subgroups of the variables in a round-robin fashion, holding approximations to the others fixed.


Although from these figures reliable statistics cannot be estimated, a reasonable assumption could be that a global noise level of 10% 6 might contain a 1/10 ratio for real-word vs. non-word errors.

looking at the real-word error curve of Figure 3, the inability to recover errors for noise levels greater than 2-5% should actually be acceptable given this empirical observation.


Most state of the art approaches for machine transliteration are data driven and require significant parallel names corpora between languages.

developing transliteration functionality among n languages could be a resource intensive task requiring parallel names corpora in the order of n C 2.


Arabic has a reduced set of vowels and, unlike English, in most contexts short vowels are optional.

when Arabic is used as the bridge language the loss of information (in terms of vowels) is large Pair ACC-1 Relative change in ACC-1 Hin-Eng 0.422 Hin-Ara-Eng 0.155 -64.28% Table 6: Incorrect choice of bridge language and the second stage system has no possibility of recovering from such a loss.


This is because inside the inner loop, the best representation for each negative example must be found.

we need to perform inference for every negative example when updating the weight vector solution.


It turned out, however, that a 3-Binomial model with a good fit on one of the samples was rejected with high probability for the other.

1 on the one hand, the variations between annotators in each sample were not as high as to preclude a model that captures only instance variability from fitting well; on the other hand, evidently, the two samples did not come from the same annotator distribution, but differed systematically due to factors we did not control for.


The disadvantage of iterative parameter mixing, relative to simple parameter mixing, is that the amount of information sent across the network will increase.

if network latency is a bottleneck, this can become problematic.


The second inequality is true by the generalized arithmetic-geometric mean inequality and the final inequality is Theorem 3.

the worst-case number of epochs is identical for both the regular and distributed perceptron -but the distributed perceptron can theoretically process each epoch S times faster.


To parse all the sentences in the PDT, one must use a non-projective parsing algorithm, which is a known NP-complete inference problem when not assuming strong independence assumptions.

the use of approximate inference techniques is common in order to find the highest weighted tree for a sentence.


Instead, consider the uni- form mixture case.

theorem 3 implies: for cases where training errors are uniformly distributed across shards, it is possible that, in the worst-case, convergence may slow proportional the the number of shards.


The correction, coupled with unit insertion and deletion cost, guarantees that for an unrelated term pair a combination of insertion and deletion will always be less costly then a substitution.

in the computation of the optimal alignment, each operation cost ranges between 0 and 2.


Note that several English Wikipedia names sometimes get the same score for a query.

we used a tie-aware mean-reciprocal rank measure to evaluate the performance (McSherry and Najork, 2008).


The utterances in our corpus of child directed speech are often only four or five words long, whereas the general LDA model assumes documents are much larger.

there may not be enough words to infer a useful utterance specific distribution over topics.


In contrast, consonant-based approaches rarely face the analogous problem of previously unseen consonants.

one low-resource solution to data sparsity is to use consonant-based techniques for unknown words (Ananthakrishnan et al., 2005;Nelken and Shieber, 2005).


This entails that space cannot be used as a reliable delimiter for words.

urdu shares the word segmentation challenge for language processing, like other Asian languages.


As an Urdu learner, a person is not taught to leave a space between words, but only to generate correct shaping while writing.

the concept of space is only learnt later on when the person learns how to use a computer.


However, Unicode terms ے as a nonjoiner with no medial shape.

the Urdu writers use ی to generate the medial position of ے in combined form.


During each iteration of MERT, the random seed is based on the MERT iteration number.

while a different set of random points is selected during each MERT iteration, on any given iteration all models use the same set of points.


Repeated draws from G exhibit the "clustering property," which means that they will be assigned to the same value with positive probability.


Therefore, the likelihood of the new assignment b S depends only on the new counts: Using these new counts in place of the ones in (4), we get the following expression: .

(9) far, we have shown how to sample b S given a set S ⊂ S of non-conflicting sites with the same type.


Each edge of the tree specifies a directed dependency from a head token to a dependent, or argument token.

the number of dependencies in a parse is exactly the number of tokens in the sentence, not counting the artificial root token.


If a sentence contains at least one paraphrasable phrase, we call the sentence a paraphrasable sentence or a non-paraphrasable sentence otherwise.

let D be the maximum number of sentence boundaries between two subsequent paraphrasable sentences in T. for every D sentences within a cover text T, there will be at least one paraphrasable sentence.


Lacking this prior knowledge, we have no optimal alignment order and we must in effect learn the substitution costs as we construct the MSA.

we choose to add instances in the order they occur in the corpus, and to iterate over the entire MSA, re-introducing each sequence.


However, since many phenomena pertaining to question generation are not so easily encoded with rules, we include statistical ranking as an integral component.

we employ an overgenerate-andrank approach, which has been applied successfully in areas such as generation (Walker et al., 2001;Langkilde and Knight, 1998) and syntactic parsing (Collins, 2000).


For example, noun phrases are "islands" to movement, meaning that constituents dominated by a noun phrase typically cannot undergo WHmovement.

from John liked the book that I gave him, we generate What did John like?


While 27.3% of all test set questions were acceptable, 52.3% of the top 20% of ranked questions were acceptable.

the quality of the top fifth was nearly doubled by ranking with all the features.


That is, for every ambiguous term, new kinds of information about the world become important, and the knowledge that a system may have acquired for previously-studied ambiguous terms may have little or no impact on the next ambiguous term.

open-ended knowledge acquisition is a fundamental obstacle to strong performance for this disambiguation task.


Features that are independent of any particular sense of the keyword, on the other hand, have no such restriction, and are just as likely to appear in the context of one sense as any other.

a low count for the co-occurrence of two potential features over a large corpus of documents for keyword k is a reliable indicator that the two features are part of the glosses of two distinct senses of k. GLOSSY's collocation detector begins by indexing the corpus and counting the frequency of each vocabulary word.


However, step 1 of the query processing does not actually check this, since it is simply going through each template, without examining any anchors, to determine which have the appropriate structure to match a query.

we store in another table the information that for a (template, query) to match it must be the case that an anchor at a particular address in that template satisfies a particular lexical restriction.


The corpus under examination was collected in a previous study on expert human tutors.

what follows is a brief synopsis of how this corpus was collected.


These suggestions of the tutor's may be framed using language like "maybe" and "perhaps" to make them more polite, and the student echoes this language in return.

tentativeness may not come from uncertainty, but from suggestions couched in polite language.


In this thread, the reply expresses an opinion against the first level quote, but not the second level quote.

we used segments of text with single quotes as an initiation and the immediately following non-quoted text as the response.


We also stipulated that R cannot be the parent, grandparent, sibling, or child of P. Together the non-quoted text of P and R forms a negative instance.

the final dataset consists of pairs of message pairs ((p i , p j ), (p i , p k )), where they have the same reply message p i , and p j is the correct quote message of p i , but p k is not.


Words that are closer to the "semantic prototypes" represented by each of the k dimensions of the reduced vector space will have vectors with longer lengths than words that are less prototypical.

those words that are closer to those prototypes will have a larger effect on the direction of the resulting vector and therefore on the comparison with other texts.


Although some of them help express user's sentiment on certain topics, they are not directly helpful for keyword analysis and may interfere with POS tagging in the preprocessing pipeline.

we designed a set of regular expressions to detect and remove them.


This is because a higher "gtmin" value causes more backoff in cases of insufficient data leading to better estimates.

we select F LM 4 for the coming experiments.


This morphological complexity also has consequences for the part-of-speech (POS) annotation of Arabic: Since words can be complex, POS tags refer to segments rather than to whole words.

the word wsyrfEwnhA (in Buckwalter transliteration; engl.


The first approach is surprisingly successful given the complex-ity of the task, reaching an accuracy on the word level of 94.74%, as compared to 93.47% for the segmentation-based approach.

the result for the whole word approach is very close to the result obtained by using gold standard segmentation (94.91%).


The extracted phrase patterns have variable length.

longer patterns may contain shorter patterns.


However, no forced-alignment phone information is available for the BDC data.


Unlike Hidden Markov Models (HMMs), CRFs are direct discriminative models: they predict the probability of a label sequence conditioned on the input.

cRFs can capture long-range dependencies in the data and avoid the need for restrictive independence assumptions.


There are several attempts at incorporating arbitrary tree-based features but these involve either solving an ILP problem (Riedel and Clarke, 2006) or using computa- tionally intensive sampling-based methods (Nakagawa, 2007).

these models, while accurate, are slow (O(n 3 ) for projective, first-order models, higher polynomials for higher-order models, and worse for richer tree-feature models).


However, these changes are limited to a fixed local context around the attachment point of the action.

we observe that the feature extraction and score calculation can be performed once for each action/location pair in a given sentence, and reused throughout all the iterations.


(2004), as well as mildly context-sensitive monolingual formalisms such as Tree Adjoining Grammar (Joshi and Schabes, 1997).

work on transforming general LCFRS grammars promises to be widely applicable in both understanding how these formalisms interrelate, and, from a more practical viewpoint, deriving efficient parsing algorithms for them.


The computational complexity of the resulting parser is polynomial in the length of the input string, with the degree of the polynomial being the number of distinct endpoints in the most complex production.

for input of length n, the complexity is O(n c ) for some constant c which depends on the grammar.


This is because that is, our newly created nonterminal X may join spans from B r−1 and B r , but can never introduce new spans.

as similar result holds for the second newly created production: In this case, the fan-out of the newly created nonterminal, ϕ(X) may be greater than ϕ(a).


One can produce arbitrarily complex grammars that generate the simple language a * .

the parsing complexity of a grammar, like its rank and fan-out, can be said to categorize its strong generative capacity.


Whereas past efforts have used linguistic resources -e.g., Word-Net -to construct the lexical graph over which propagation runs, our lexicons are constructed using a graph built from co-occurrence statistics from the entire web.

the method we investigate can be seen as a combination of methods for propagating sentiment across lexical graphs and methods for building sentiment lexicons based on distributional characteristics of phrases in raw data (Turney, 2002).


Our work, on the other hand, does not rely on syntactic parsers or restrict the set of candidate lexicon entries to specific syntactic classes, i.e., adjective phrases.

the lexicon built in our study is on a different scale than that examined in Kaji and Kitsuregawa (2007).


If we plot the precision-recall graphs using purity to classify sentences -as opposed to the vote-flip algorithm, which only provides an unweighted classification -we can see that at almost all recall levels the web-derived lexicon has superior precision to the other lexicons (Figure 4).

even though the web-derived lexicon is constructed from a lexical graph that contains noise, the graph propagation algorithms appear to be fairly robust to this noise and are capable of producing large and accurate polarity lexicons.


Subjective sentences often contain words which reverse the sentiment polarities of other words.

interactions between words need to be considered in sentiment classification, which is difficult to be handled with simple bag-of-words approaches, and the syntactic dependency structures of subjective sentences are exploited in our method.


However, the polarities are reversed by modifying the word prevents, and the dependency subtree "prevents cancer and heart disease" has positive polarity.

the whole dependency tree "It prevents cancer and heart disease."


In our preliminary experiments, L-BFGS often did not converge and classification accuracy was unstable when the initial values of parameters were randomly set.

in later experiments, we set the initial values in the following way.


they can be opinion words (see Sentences 1-3), communication words, such as maintained in Sentence 2, or other lexical cues, such as according in Sentence 3.

the definition of boundaries of the structures for the convolution kernels is less straightforward in opinion holder extraction.


In this work, we consider all noun phrases (NPs) as possible candidate opinion holders.

the set of all data instances is the set of the NPs within the MPQA 2.0 corpus.


For the background scopes SEM and SEN T , however, there is no second expression in focus.

grammatical relation paths cannot be defined for these scopes.


However, for the purpose of document ranking, this will not make any difference.


Moreover, a conventional distortion limit (e.g., 5) would likely force translation of the verb prior to the full subject unless the exact subject phrase existed in the phrase table.

1 the correct re-ordering is either improbable or impossible, depending on the choice of distortion parameters.


2 By definition, the model has a least cost translation path: monotone.

we can add to the baseline calculation D(s, t) the cost of skipping back to the first uncovered source word and then translating the remaining positions monotonically.


The procedure presented there runs in linear time in the size of the input MBOT.

we can transform each STSG to an equivalent MBOT in one-symbol normal form in linear time.


The idea of this construction is illustrated in Figure 5.

our BAR-HILLEL construction has the well-known complexity O(|M | • |S| 3 ).


However, MBOT can also compute transformations, of which the domain or range cannot be represented by a TSG.

if we train MBOT directly and utilize their full expressive power, then we might not be able to perform forward and backward application.


8 Hence, we start generating sentence combinations n r in lexicographic order, for r = 1...n, and for each combination E i k = S i 1 , S i 2 , ..., S i k where k > 1, we generate additional extracts E ′ i k by successfully swapping S i j with S i k for j = 1, ..., k − 1 and checking to see if the above constraints are still satisfied.

from a combination with k sentences that satisfies the constraints, we might generate up to k − 1 additional extracts.


Furthermore, we see from Table 2 that the upper bounds corresponding to these scores are 65.7, 37.4, and 38.1 respectively, which are well above both the TextRank and the Lead systems.

the percentile rankings of the Lead and the Tex-tRank systems for this domain do not seem to give us clues about how the two systems compare to each other, nor about their actual distance from the upper bounds.


Second, recent work by Chambers and Jurafsky (2009) has induced "narrative chains", i.e., likely sequences of events, by their use of similar head words.

we expect that all k nearest neighbors of a source predicate s are informative for the selectional preferences of s (like schießen) as long as they are either synonyms of its literal translation (shoot/hit) or come from the same narrative chain (stalk/kill/.


The maximum-likelihood parameters can then be obtained by finding the θ maximizing: And the gradient is given by: where the first term is the conditional expected feature count and the second term is the expected feature count.

far, we have written the equations in a generic form for arbitrary conditional random fields with latent variables (Sutton and McCallum, 2007).


Unfortunately, this is not possible, as we are using the designated CoNLL training set as our test set, and any supervised system trained on this data would achieve unfairly high scores.

we approximate the performance of a supervised system by clustering instances according to their gold standard role after introducing some noise.


(2000) note, results from studies of causality in narrative texts might not carry over to scientific writing, because inferences are made more spontaneously during the reading of narrative than expository texts.

comparing expert vs lay readers on the comprehension of causal relations in scientific writing is a most timely investigation.


The problem is that some propositional content is harder to formulate than others.

good formulations of some propositional content might have much lower scores than even mediocre formulations of other propositional content.


We then run the machine learner again, with this new feature relative:  Table 3: Precision, Recall and F-measure of classifier (second run) relative to human judgement.

• The ratio of the test-set probability (from the first run) to the highest of the 8 test-set probabilities for the different formulations of that sentence: probabilities for acceptability are normalised such that the best score for a given propositional content is 1 and the other 7 formulations score less than or equal to 1.


However, the ultimate objective of automatic metaphor processing is a type of interpretation that can be directly embedded into other systems to enhance their performance.

we define metaphor interpretation as a paraphrasing task and build a system that automatically derives literal paraphrases for metaphorical expressions in unrestricted text.


literally (ask in the context of "a person asking another person a question or a favour"; post in the context of "a person posting/sending a letter"; buy in the sense of "making a purchase").


The basic sense of kill implies an end or termination of life.

termination is the shared element of the metaphorical verb and its literal interpretation.


Indeed, the authors explicitly make the assumption that a single protagonist participates in all of the events forming a narrative chain.

the discovered chains de-scribe actions experienced by the protagonist localized within a particular narrative structure.


Shell language typically occurs as fairly long sequences of words, but identifying the exact span of a sequence of shell seems less important than in related tagging tasks, such as named entity recognition.

rather than evaluating based on spans (either with exact or a partial credit system), we measured performance at the word token-level using standard metrics: precision, recall, and the F 1 measure.


Furthermore, because we have posterior marginals rather than a single Viterbi derivation, we can explicitly force the aligner to choose denser extraction sets simply by lowering the marginal threshold.

we also show substantial improvements over DeNero and Klein (2010) in recall-heavy objectives, such as F 5 .


The key observation is that for any legal value σ e i = [k, ], S e i and U e i require that a ij = off for all j / ∈ [k, ].

6 we start by computing the product of all the off beliefs: b = j q a ij (off).


When co-referent text mentions appear in different languages, these techniques cannot be easily applied.

we develop new methods for clustering text mentions across documents and languages simultaneously, producing cross-lingual entity clusters.


7 is that 3,251 of the 7,655 automatic coreference chains are not in the reference.

the evaluation is dominated by the penalty for spurious system coreference chains.


While these high-order potentials encode important linguistic properties of valid assignments, they greatly complicate learning and inference.

our method estimates the parameters of the local sequence models and the global potentials separately.


For example, if the Tactic field in a gold event record contains the words "bombing" and "blast", the measure is expected to give a perfect score to a learned record that contains one of these words.

as in the MUC-4 guidelines, we count pre-specified synonyms and morphological derivations of the same word only once.


we replace each group with a placeholder.


We noticed that the scope of a given reference often consists of units of higher granularity than words.

in this method, we split the sentence into segments of contiguous words and, instead of labeling individual words, we label the whole segment as inside or outside the scope of the target reference.


However, despite these difficulties, it is necessary for an automated assessment system to address the high level information of a spoken response in order to fully cover all aspects that are considered by human raters.

in this paper we focus on exploring features to represent the high-level aspect of speech mainly on the accuracy of the content.


As expected, the results show that the performance on ASR output is lower than when human transcripts are used.

we propose improved content features that take into account ASR confidence scores to emphasize responses whose estimated word accuracy is comparatively higher than others.


The reason for this is that  the contextual-based tasks are more constrained to the materials provided with the test item, whereas the opinion-based tasks are relatively open-ended.

it is easier for the similarity measures to track the content, the topics, or the vocabulary usage of the contextual-based topics.


However, in practical automated speech scoring systems, the only available text is the output of the ASR system, which may contain a large number of recognition errors.

in this section we show the performance of the content features extracted using ASR hypotheses.


total value of a path accepting (w, t) is log q φ (t | w).

let the value of each arc in cq φ be the log of its weight in the isomorphic FST q φ (t | w).


If n p > n q , then a's start state does not store such a long history.

the value of a actually depends on how one reaches a!


The reason for all this past research is that (Merialdo, 1994) was a negative result: while EM is guaranteed to improve the model's likelihood, it degrades the match between the latent states and true parts of speech (if the starting point is a good one obtained with some supervision).

for the task of POS induction, there must be something wrong with the HMM model, the likelihood objective, or the search procedure.


Algorithm 6 implements the FINDVIOLATION function for the generic Local Violation-Fixing Perceptron in Algorithm 2.

algorithm 5 is equivalent to algorithm 6 plugged into algorithm 2.


And the problem of misspelling in PAN could be remedied with automatic spelling correction.

it is possible to improve the MT metrics based approach further by utilizing certain NLP systems as pre-processing modules for the text.


In order to do this well, it is not enough to simply ask the initiating question, because students need the experience of engaging in meaningful dialogue about the instructional content.

systems must respond appropriately to student explanations, and must provide detailed, flexible and appropriate feedback (Aleven et al., 2002;.


These sessions were completed prior to the beginning of the experiment during which the BEETLE corpus was collected, and are not included in the corpus presented here.

the dataset used in the paper constitutes unseen data for the BEETLE II system.


While PRO exhibits a small variance for a given initializer, as also reported by Hopkins and May (2011)  trouble recovering from one of the random initializers.

while the within-initializer variance for PRO tended to be smaller than that of MERT, PRO's overall range was larger.


5 We will define these "active" features of a string x by constructing some unweighted deterministic FSA, G (described in §4).

the active features of x are determined by the collection of arcs on the accepting path of x in G. to satisfy the agreement constraint, x i and x j must be accepted using the same arcs of G (or more generally, arcs that have the same features).


Fixing a uniform threshold would have caused the model to filter out long transliterations.

we tied the threshold to the length of transliterated words.


In graph reinforcement, observed mappings were modeled as a bipartite graph with source (S) and target (T) character sequences and weighted with the learnt alignment probabilities (M).

the mapping between s ∈ S and t ∈ T was m(s,t).


Setting the larger batch size implies the more accurate corpus-BLEU, but at the cost of slower convergence of SGD.

we propose an optimized update method inspired by the passive-aggressive algorithm (Crammer et al., 2006), in which each parameter update is further rescaled considering the tradeoff between the amount of updates to the parameters and the ranking loss.


(2011), setting batch size to a smaller value, such as |b| = 1, does not work well in practice, since BLEU is devised for a corpus based evaluation, not for an individual sentence-wise evaluation, and it is not linearly decomposed into a sentence-wise score (Chiang et al., 2008a).

the smaller batch size may also imply less accurate batch-local corpus-BLEU and incorrect oracle translation selections, which may lead to incorrect sub-gradient estimations or slower convergence.


The behavior is justified by our optimized update procedure, in which the learning rate η k is used as a tradeoff parameter.

it tries a very aggressive update at the early stage of training, but eventually becomes conservative in updating parameters.


Since M can only produce the output tree u , this also limits the total number of rule applications in any successful derivation.

m can only apply a copying rule at most |u | times, which shows that m is finitely copying (see Definition 8).


Since M is sensible, its associated unweighted XTOP R N , which is obtained by setting all weights to 1 and computing in the BOOLEAN semiring, is sensible.

n is finitely copying by Theorem 7.1 of Engelfriet and Maneth (2003).


Consequently, N is finitely copying by Theorem 7.1 of Engelfriet and Maneth (2003).

also M is finitely copying, which concludes the proof.


The first XTOP R annotates the input tree with the rules that the second XTOP R is supposed to use.

the first XTOP R remains nondeterministic, but the second XTOP R , which simply executes the annotated rules, is now deterministic.


A segment that frequently appears on the Web tends to be seen as: (1) popular, (2) structurally sound.

the frequency of appearance of n-grams on the Web (via the Google search engine) appears as a good indicator of the n-gram popularity/soundness (Keller and Lapata, 2003) .


Although there are a handful of corpora that include punctuation errors in their annotation scheme, such as NUCLE (Dahlmeier and Ng, 2011) and HOO (Dale and Kilgarriff, 2010), there are none to our knowledge that focus specifically on commas.

we designed and implemented our own annotation scheme on a set of essays to allow us the freedom to identify the most important aspects of comma usage for our work.


( 4)This way of using paraphrases basically achieves expansion of the terms in documents with their paraphrases.

if two documents have no terms in common, but one has the term explosion and the other has the term blast, by knowing that the two terms are paraphrases, their similarity will be different from zero, which would have been the case if no paraphrasing was used.


In Table 2, note that each potential boundary position is considered k (the window size) times.

each positive or negative boundary assignment is counted k times; near misses producing a blend of values: TP, FP, FN.


What is not apparent in Table 2, is that WinPR is insensitive to window size, with the exception of near misses.

adjusting the window size can be used to adjust the tolerance or sensitivity to near misses.


2008 WinPR counts boundaries, not windows, which has analytical benefits, but WindowDiff's counting of windows provides an evaluation of segmentation by region.

windowDiff is more appropriate when an evaluator is less interested in the types and the number of errors and more interested in the percentage of the sequence that is correct.


However, for ASR transcripts, although topical similarity is not as accurate as lexical similarity, it can compensate for recognition errors, so that the approaches have similar performance.

graph-based approaches can significantly improve the baseline results.


A word is segmented by identifying the longest of the φ suffixes that it ends with.

each word has one suffix feature if it ends with one of the φ suffixes and none otherwise.


We therefore exclude all frequent words from clustering on the assumption that enough training data is available for them.

clustering of words is restricted to those below a certain token frequency threshold θ.


The success of MERT, PRO and MR indicates that their shared search approximation is actually quite reasonable.

we propose Batch MIRA, which sits exactly where MERT sits in the standard tuning architecture, greatly simplifying parallelization: where BatchMIRA() trains the SMT-adapted MIRA algorithm to completion on the current approxima-tionẼ, without parallelization.


Moreover, complex hierarchical temporal expressions, such as the Tuesday before last or the third Wednesday of each month, and ambiguous expressions, such as last Friday, are difficult to handle using deterministic rules and would benefit from a recursive and probabilistic phrase structure representation.

we attempt to learn a temporal interpretation system where temporal phrases are parsed by a grammar, but this grammar and its semantic interpretation rules are latent, with only the input phrase and its grounded interpretation given to the learning system.


Conversely, the falsity of any of them is sufficient to make (1) true: (1) would be true if nobody got anything, we didn't get, an offer wasn't gotten or the offer wasn't for more than $40.

all four statements (a-d) are inside the scope of (1).


sentences are fully interpreted to identify positive implicit meaning.


The average sentence length across all languages is 18 words.

receiving feedback regarding a single word in a sen-tence equals to about 1/18 ≈ 5.5% of the information provided by a fully annotated sentence.


Thus, receiving feedback regarding a single word in a sen-tence equals to about 1/18 ≈ 5.5% of the information provided by a fully annotated sentence.

we may view the light-feedback provided for 99% of the dataset as about equal to additional 5.5% of fully annotated data.


In constituency parsing, exhaustive inference for all but the simplest grammars tends to be prohibitively slow.

most high-accuracy constituency parsers routinely employ a coarse grammar to prune dynamic programming chart cells * Research conducted at Google.


Detecting such power relations and determining their effect on dialog is a hard problem, and it is the ultimate goal of our research.

we do not use knowledge of power relations as features in performing a finergrained analysis of dialog acts.


This is the essential purpose of the trait features -using the Output Length feature, the optimizer has a "knob" with which it can control the trait value independently of everything else.

3 the new hypotheses that we select are "optimal" in terms of our existing MT model probabilities, but have trait values which vary from the baseline in a precise way.


Once the weight sets are generated, the hypotheses for each trait condition can be generated by rescoring the forest inside of the decoder.

all 28 trait hypotheses can be generated for almost no cost over a single decode.


In contrast, Hiero-style systems typically use CKY chart-parsing decoders which can freely combine target hypotheses generated in intermediate cells with hierarchical rules in the higher cells.

the generation of the target hypotheses are fragmented and out of order compared to the left to right order preferred by n-gram language models.


In the first column, we show the average over the eight initial weight vectors of the BLEU score achieved, while in the second column we show the results from the initial weight vector with the highest BLEU score on the development set.

while the second column corresponds to a tuning process where the single best result is retained, the first column shows the expected behavior of the procedure on a single initial weight vector.


The overall precision and recall are calculated by plugging these values in equations 1 and 2.

52.4 is the macro-average F-score obtained by the I.a.


The immediate audience of a user x u is his followers.

incorporating social features into our model is likely to contribute to its prediction ability.


In contrast, the "car" sense of "jaguar" dominates the vector created using the search engine.

incorporating models from different sources could be more effective than relying on word sense discovering algorithms operating solely on one corpus.


A key factor deciding the validity of our conclusion is the reliability of the annotations derived via MTurk.

our first step is to evaluate the ability of Turkers to perform WSD.


We perform experiments on German and English Wikipedia (two largest available), but our technique is general and does not depend on sparse, manually-created interwiki links.

compared to those approaches, we achieve much higher recall.


Looking at battles at a broad time granularity applicable to wars may not produce a meaningful clustering.

we should estimate the temporal structure of each article before clustering.


In a traditional generation pipeline, referring expression generation happens at the sentence planning level.

its output is not a textual nugget but a description employed later on by the surface realizer.


The final ranked list consists of the merged timespans.

although our timespans are defined as hour intervals, it is possible for our system to return longer (merged) timespans.


If all of the expansion terms exhibit burstiness during the time interval, it strongly suggests the timespan may be relevant to the query.

to measure the relevance of the timespan, we first compute the burstiness scores for all of the terms within the time interval.


Anything that follows a node with an inert label is by definition not reachable from the root of the tree.

any node that is part of a sentiment flow that follows an inert node is not reachable along a path and is actually inert itself.


Our work is part of a larger collaboration with social scientists to study the diffusion of information technology (IT) innovations through society by identifying opinion leaders and IT-relevant opinionated language Rogers (2003).

we focus on a collection of articles from the IT professional magazine, Information Week, from the years 1991 to 2008.


Our IT concept target list (59 terms) comes from our application.

we construct a trie for each appearance of any of these possible target terms.


UEM and CoDL (γ = −∞) When γ → −∞ then due to an infinite penalty on the entropy of the posterior, the entropy must become zero.


For γ = 1 with agreement constraints, our algorithm specializes to an earlier proposed technique by (Ganchev et al., 2008).

our algorithm puts these dual decomposition techniques with Algorithm 3 Projected Subgradient-based Lagrange Relaxation Algorithm that optimizes Eq.


Lines 6-9 show that the required computation is decomposed over each sub-component.

if computing the posterior and expected values of linear functions over each subcomponent is easy, then the algorithm works efficiently.


The difficulty in accurately computing features translates to difficulties in composing coreference rules: for example, the aforementioned rule involving gender and number agreement, as well as rules that implement traditional linguistic constraints on coreference, may no longer be accurate and desirable to have if the features involved cannot be accurately computed.

we believe that research in multilingual coreference resolution will continue to be dominated by supervised approaches.


The classifiers trained on each view complement each other in the learning process.

we can leverage the predictions made by each classifier on the unlabeled dataset to augment the training data of both classifiers.


We demonstrate the effectiveness of the extracted features in a supervised binary classification task for MCCR with MaxEnt classifiers (using the combined feature set) as well as using semi-supervised methods of co-training MaxEnt classifiers and training MaxEnt models using posterior regularization (using two independent views of the data -semantic view and temporal view).

we show that MCRR can be performed using semi-supervised learning with semantic and temporal views of the data.


Assessment has also been shown an important part of learning, as testing assists retention and can be used to guide learning (Anderson & Biddle, 1975).

as learners move on from an educational setting to unstructured self-learning settings, they would still benefit from having the means for assessment available.


This is fortunately very similar to the goals of automatic summarization, in which the selected sentences should be ordered by how central they are to the article.

we make use of our own implementation of SumBasic (Nenkova et al., 2006), a simple but competitive document summarization algorithm motivated by the assumption that sentences containing the article's most frequently occurring words are the most important.


2 2 Heilman and Smith (2010a and b) asked raters to identify question deficiencies, including vague or obvious, but raters were not asked to differentiate between Good and Okay.

questions considered Good in their study would include Okay.


These features characterize the way in which a word is translated, an indication of its meaning.

of soft K-Means clustering, each word in the target-language vocabulary is assigned to a list of up to M clusters.


Much less attention has been given to the underlying structure of the topics themselves.

most topic models generate topics independently from a single underlying distribution and require millions of parameters, in the form of multinomial distributions over the vocabulary.


Unfortunately, CD training cannot infer this structure since the CD derivation makes use of a fixed structure in the one-step reconstruction.

we have taken a MCEM approach, first sampling the PoE structure in the E-step, then fixing these samples for Z and B when computing the one-step reconstruction of the data, X. Contrastive Divergence Gradient We provide the approximate derivative of the contrastive divergence objective, where Z and B are treated as fixed.


The IOMM generates a row for each data point, whereas the SCTM generates a row for each topic.

the SCTM goes beyond the IOMM by allowing the rows to be shared among documents and models document-specific mixtures over the rows of the matrix.


The operation model does not have the information that "Wie heißen Sie" may be translated as a phrase during decoding.

the future-cost estimate available to the operation model for the span covering "Wie heißen Sie" will have unigram probabilities for both the translation and language model: the future-cost estimate in the operation model is much worse than that of the phrase-based model.


Their distributions should be more informative than the so-called "lexical weighting" models, and less sparse than relative frequency estimates, though potentially not as effective for truly non-compositional units.

we explore the inclusion of all such information.


The score of a translation for a fixed decomposition is a linear function of the features, but the score of a translation is a maximum of linear functions (over decompositions).

if we define hypotheses as just containing translations, MERT training does not work directly for optimizing the weights of the dynamic combination method.


For each candidate translation, we have access to the phrase pairs it used as well as the alignments inside each phrase pair.

each source sentence and its candidate translation form a word-aligned parallel sentence pair.


However, a morpheme alignment induces a unique word alignment.

if a morpheme alignment a m and a word alignment a w are compatible, then the word alignment is a w is recoverable from the morpheme alignment a m .


The size of the word vocabulary is larger than the size of the morpheme vocabulary.

the number of rare words is larger for words than it is for morphemes.


We define an event sentence as one that contains both a learned event phrase and a purpose phrase, based on the dictionaries at that point in time.

the number of event sentences increases as the bootstrapped dictionaries grow.


Chinese word i is tagged with a named entity type X (z .

these hard bilingual constraints guarantee that when two words are aligned, they are tagged with the same named entity tag.


In Figure 2(b), the English word "Agency" is wrongly aligned with the Chinese word "电 (report)".

these two words cannot be assigned with the same tag.


This condition can be expressed as the following constraints: When the tag pair of a word alignment pair is determined, the corresponding monolingual named en-tity tags can also be identified.

this rule can be expressed by the following constraints: if z ycye a = 1, then z yc i and z ye j must be both equal to 1.


However, in this paper, we assume that no named entity annotated bilingual corpus is available.

the above method is only used as Oracle.


We also found that automatically estimated bilingual tag PMI scores (Auto) gave comparable results  to Oracle.

this technique is effective for computing the PMI scores, avoiding the need of manually annotating named entity bilingual corpus.


The N ranges from 1 to 3, and the context is either left-side or right-side.


McElree (2001; 2006) has found that retrieval of any non-focused (or in this case, unconnected) element from memory leads to slower processing.

integrating two disjoint connected components should be expected to incur a processing cost due to the need to recall the current state of the superordinate sequence to continue the parse.


We have j representing language L j , and for each topic z k ∈ Z.

the latent cross-lingual topics also span a shared cross-lingual semantic space.


The novel response-based approach of similarity removes the effect of high-frequency words that tend to appear higher in the lists of semantic word responses.

the real synonyms and translations should occur as top candidates in the lists of similar words obtained by the response-based method.


In a typical setting for mining semantically similar words using latent topic models in both monolingual Dinu and Lapata, 2010) and cross-lingual setting (Vulić et al., 2011), the best results are obtained with the number of topics set to a few thousands (≈ 2000).

our bilingual LDA model on all corpora is trained with the number of topics K = 2000.


Though a standard dictionary probably will not use the same POS tag set that we are targeting, it nevertheless provides information about the relatedness of various word types.

we use nodes DICTPOS p that indicate that a particular word type is listed as having POS p in the dictionary.


Our fix is to define the output of the parser to be arg max t (a,i,j)∈t |µ(a, i, j)| rather than arg max t (a,i,j)∈t µ(a, i, j) as defined in Goodman's algorithm.

if a marginal value µ(a, i, j) is negative, we simply replace it with its absolute value.


However, these parameters imply a latent division of responsibility for observed words: some are present because of the tag while others are general words in the corpus.


Our hypothesis is that f-LDA could be used as an exploratory tool to expedite the creation of these reports.

in our evaluation we want to measure how useful the extracted snippets would be in informing the writing of such reports.


However, since our dataset is conversational (more details in Section 3), and the text segments are not long, we aim for a classifier with high recall.

we only use TFxIDF (Salton and McGill, 1986), position of the first occurrence (Frank et al., 1999) and phrase length as our features.


However, annotators write labels of their own and may use words that are not directly from the conversation but are semantically related.

we propose to also use another variant of R-f1 that incorporates semantic relation between words.