Rule-driven Hypothesis Generation: One can use the rules described in Section 2 to generate a set of hypotheses.

this is intuitive as the feature functions may differ in the scale of the feature values or in their importance in ranking the hypotheses.


On the other hand, if the word " " does appear in the training data but it has only a translation "gruel" as that is the meaning in the formal text, the translation system may wrongly translate " " into "gruel" for the informal text where the word " " is more likely to mean "like".

clearly, the  larger the N is, the higher the precision is.


• LD-PinYin(x, y): the Levenshtein distance on PinYin of x and y.

we use 252 examples to train the log-linear model that is described in Section 4, and use 249 examples as test data to compute the precision.


Tables 7-9 show the precision results for different categories as described in Section 2, using the ruledriven, data-driven, or both rule and data-driven features, respectively.

the transformation path for this category is "Formal → PinYin → Informal (similar or same PinYin as the formal phrase)".


For example, the presence of an article in English greatly reduces the ambiguity of the succeeding tag.

a higher acceptance rate and faster convergence is achieved when the proposal Q is a close approximation of P .


One last remark is regarding our decision to find a method for successfully exploiting the existing taggers for POS tagging Spanglish text.

we believe that we can improve results further by using Machine Learning (ML) algorithms trained specifically for this task.


We cannot easily classify the gap between " " and " " as segmented or unsegmented.

these rule-based methods are fast, however, their performances depend on the dictionary which cannot include all words, and also on the rules which cost a lot of time to make and must be updated frequently.


Where the N -best approximation is used N = 15, and for all QBC methods C = 3; these figures exhibited a good balance of accuracy and training speed in preliminary work.

furthermore, the total token-level strategies (TTE, TVE, TKL) outperform their length-  Table 2: Detailed results for all query strategies on all evaluation corpora.


A typical active learner begins with a small labeled set L, selects one or more informative query instances from a large unlabeled pool U, learns from these labeled queries (which are then added to L), and repeats.

this can be thought of as a QBC generalization of N -best entropy, where each committee member casts a vote for the posterior label distribution.


By analogy, it would be a poor idea for a language model to score a string highly if it could be segmented into independently frequent n-grams.

since each character in Σ is actually a 4-tuple, this trigram machine is fairly large.


If the label for a given labeled data point consists only of a single integer, then the entropy of the corresponding r j is zero (the probability of that integer will be unity, with the remaining probabilities being zero).

label propagation (lP) (Zhu and Ghahramani, 2002) is a graph-based SSl algorithms that performs Markov random walks on the graph and has a straightforward extension to multi-class problems.


Form(p, f) is true iff part p has core form f, and ArgForm(p, i, f) is true iff the ith argument in p has form f. 3 The "f!"

cOMPOSE creates clusters of large lambda-forms if they tend to be composed of the same subforms (e.g., the lambda form for "is next to").


(Xue and Palmer, 2004) fount out that different features suited for different sub-tasks of SRL, i.e.

only 1-best outputs of the parser before reranking are used for our evaluation.


It turns out that these semirings can also compute first-and second-order partial derivatives of all the above results, with respect to a parameter vector θ ∈ R m .

the root node corresponds to the goal item.


Entropy on a Hypergraph The entropy of the distribution of derivations in a hypergraph 14 is provided that we define r e def = log p e (so that r(d) = e∈d r e = log p(d)).

when using this semiring to compute second derivatives (Case 2) or covariances, one may exploit the invariant that r = s, e.g., to avoid storing s and to compute r1s2 + s1r2 in multiplication simply as 2 • r1r2.


This would allow us to judge the validity of label percolation, and under different maxLen values.

cast this way, a probabilistic approach is actually quite appealing, and one could use collected data to train a prediction model (such as a Markov random field).


for these reasons, researchers resort to optimizing an automatic metric (almost always BLEU) as a proxy for human judgment.


The fourth feature exploits structural information on the source side.

the estimation of the context LMs is very efficient.


With GE, the retraining times are on average 6 minutes on cora and 4 minutes on apartments.

we also add a normalization constant Under this construction the expectation of G yk is the predicted conditional probability that the label at some arbitrary position i is y when the observational test at i succeeds,p( If we have a set of constraint functions {G yk : y ∈ Y}, and we use the score function in Equation 1, then the GE objective function specifies the minimization of the KL divergence between the model and target distributions over labels conditioned on the success of the observational test.


is the total weight of all paths in the F 3 's WFSA that accept the string x ∈ Σ * 2SA .

belief propagation translates to our setting as follows: • Each message is a WFSA.


Concerning RM, sequential training of the 6 models took 2,596 minutes, while testing took 27 minutes.

the chance of this resulting in an improvement is rather low.


We reduced the number of schema labels by: (1) mapping the labels address, booktitle, journal and school to venue, (2) mapping month and year to date, and (3) dropping the fields url, ee, cdrom, note, isbn and chapter, since they never appeared in citation texts.

the labeling function employs high-precision alignment rules to assign labels to text tokens using labeled record tokens.


In the AnCora corpus of Spanish and Catalan newspaper text (Martí et al., 2007), nearly half of the entities are embedded.

weak named entities, "consist of a noun phrase, being it simple or complex" and must contain a strong entity.


In Section 4.4 we will show how our unified IE model compares to these baselines.


For example, a statement that Al Qaeda is believed to be responsible for an attack would typically appear after the event description.

our unified probabilistic model described in this paper does not draw a hard line between relevant and irrelevant sentences, but gently balances the influence of both local and sentential contexts through probability estimates.


To determine whether a noun phrase instance NP i should be extracted as a filler for an event role, GLACIER computes the joint probability that NP i : (1) appears in an event sentence, and (2) is a legitimate filler for the event role.

any classifier that can generate probability estimates, or similar confidence values, could be plugged into our model.


Formally, the baseline NB classifier uses the formula: where F is the set of local features, P (PlausFillr (NP i )) is the prior probability, and Z is the normalizing constant.

to the MUC-4 data, this improvement was mostly due to an increase in precision (up to 0.41), indicating that our unified IE model was effective at eliminating many false hits.


On the other hand, the frequencies of adverbs in free-style texts are much higher than those in pros/cons, as pros/cons mostly contain adjective-noun patterns.

this shows that the scoring of sentiment from our system is quite close to human annotation.


This allows us to pool all instances of a given adverb regardless of which adjective it is associated with, in order to compute the absolute value of the perturbation score for that adverb.

also, we can tell from the predicate sub-frame that there is an adverb "quite" modifying the head word "good".


The automation of the discourse-relation recognition is the next step in this research.

in the iterative phase, we create a random ordering of all the instances and, in turn, apply the relational classifier to each instance where the relational features, for a given instance, are computed using the most recent polarity assignments of its neighbors.


Also note that, by virtue of the constraint in Equation 7, ILP does not classify any connected instance as neutral; thus the precision is NaN, recall is 0 and the f-meaure is NaN.

for our experiments, the DAs with no opinion annotations as well as those with neutral opinions are considered as neutral.


We observed that the ILP system performs better than the ICA system on instances that are connected, while ICA performs better on singletons.

notice also, that even though Local predicts only DA-4 correctly, this correct classification finally influences the correct choice for all the instances, including the remotely connected DA-2.


The participles supposing and assuming create conditional sentences where the conditional clause and the consequent clause can be syntactically independent.

the results indicate that the wholesentence-based classifier performs the best.


to compare SWSD with WSD, we re-ran the 10-fold cross validation experiments, but this time using the original sense labels, rather than S and O.


We borrow machine learning features which have been successfully used in WSD.

because we do not have data labeled with the S/O coarse-grained senses and we did not want to undertake the annotation effort at this stage, we created an annotated corpus by combining two types of sense annotations: (1) labels of senses within a dictionary as S or O (i.e., subjectivity sense labels), and (2) sense tags of word instances in a corpus (i.e., sense-tagged data).


The classifier, which is a sentence-level S/O classifier, has low subjective and objective recall but high subjective and objective precision.

the presence of another lexicon entry "explains away" the presence of the O sense in the subjective expression, and we do not want SWSD to overrule the contextual classifier.


Due to trimming of triplets with small probabilities after each iteration, the version based on 10 iterations was slightly smaller, having 164 million triplets but also performed slightly worse.

in future work, we plan to extend the discriminative word lexicon model in two directions: extending context to the document level and feature engineering.


Figure 2: Precision at rank for the different systems on the Athletes class.

we use a Gradient Boosted Decision Tree regression model -GBDT (Friedman, 2001), which consists of an ensemble of decision trees, fitted in a forward step-wise manner to current residuals.


And with improved inference for unsupervised Λ, Labeled LDA lends itself naturally to modeling semi-supervised corpora where labels are observed for only some documents.

as a baseline, we use a set of multiple one-vsrest SVM classifiers which is a popular and extremely competitive baseline used by most previous papers (see (Kazawa et al., 2004;Ueda and Saito, 2003) for instance).


Supervised LDA posits that a label is generated from each document's empirical topic mixture distribution.

we expect Labeled LDA to do better credit assignment on this subset and consequently to show improved performance as a classifier, and indeed this is the case.


In (Mihalcea and Tarau, 2004), due to the unsupervised method, only the test set was used for comparing the performance of Tex-tRank and Hulth's method.

find the distance or similarity between every pair of data points in the dataset; 2.


Each cluster is represented by an exemplar term, which is also the centroid of each cluster.

none of these previous works has overall consideration on the essential properties of appropriate keyphrases mentioned in Section 1.


One of the main problems with the pattern-based graph is transitivity of connections.

we selected three different cases (in Nepal, Israel, wales) where the obtained information can be reliably verified, and applied our framework on these settings.


In general, articles on the same object in different languages are edited independently and do not have to be translations of one another, but are linked to each other by their authors.

claudio Giuliano is supported by the ITcH project (http://itch.fbk.eu), sponsored by the Italian Ministry of University and Research and by the Autonomous Province of Trento and the X-Media project (http://www.


As syntagmatic relations hold among words collocated in a particular temporal order, they can be modeled by analyzing sequences of words.

the syntagmatic kernel is defined by where K n is the string kernel defined in Equation 5 and the parameter n represents the length of the subsequences analyzed when estimating the similarity between contexts.


Besides, many verbal and adjectival concepts in Wikipedia are redirected to nominal identifiers.

kernel methods are theoretically well founded in statistical learning theory and shown good empirical results in many applications (Shawe-Taylor and Cristianini, 2004).


Wikipedia structure and quality make this resource particularly suitable for information extraction and word sense disambiguation tasks (Csomai and Mihalcea (2008) and Milne and Witten (2008)).

the sets of Italian sentences extracted for every (F, l), i.e.


Our results suggest some possible prescriptions for tuning techniques according to annotator expertise.

this work is funded by NSF grant BCS 06651988 "Reducing Annotation Effort in the Documentation of Languages using Machine Learning and Active Learning."


Annotation task: language documentation The amount of money spent on obtaining human annotations is an extremely important concern in much language annotation.

another factor which must be considered when annotation is done by human annotators (rather than being simulated) is the accuracy of the humans' labels.


This sub-set is then passed on as training data to the supervised classifier, which then labels the remainder of the data set.

related Words (relW) This feature set is a variant of the previous one.


we show that it is possible to improve on Sporleder and Li's (2009) results by employing a two-level strategy, in which a cohesion-based unsupervised classifier is combined with a supervised classifier.


Interestingly, when going from the first to the second iteration the training set nearly doubles (from 396 to 669 instances), while the proportion of errors is also reduced by a third (from 7% to 5%).

4 if we had labelled data we could train a supervised classifier to learn these and other contextual clues.


It is important to keep in mind that this distinction between types of prediction (and as a consequence, prediction error) is not equivalent to the one drawn in the traditional cognitive science modularity debate, which has focused on the source of these predictions.

the right-hand side sequences consist of either (1) exactly one terminal item, or (2) zero or more non-terminals.


Since the start and end of each turn were timemarked by hand, each feature was easily extracted over a turn, and then averages and standard deviations were taken over the turns in an entire conversation side.


In addition they were also asked to rate their own intentions: "How often did you behave in the following ways on this 'date'?"

we do find gender differences; men ask more questions when they say they are flirting, women ask fewer, although they do use more repair questions, which men do not.


With these extensions we expect that our model of grammar induction has the potential to greatly improve translation output.

the operator considers pairs of nodes, v, w, in one tree and attempts to swap their alignment values.


This is further complicated by the language model integration.

with no limit on rule size this method will learn a single rule for every training instance, and therefore will not generalise to unseen sentences.


The alignment variable may be empty, a v = ∅, while non-empty values must obey the tree constraints.

this model leverages a hierarchical Bayesian prior to induce a compact translation grammar directly from a parsed parallel corpus, unconstrained by word-alignments.


The set of valid labels is constrained by the other alignments in the tree, specifically that of the node's closest aligned ancestor, a p , its closest aligned descendants, a d , and its aligned siblings, a s (the aligned descendants of a).

if the grammar itself is extracted using wordalignments induced with models that are unable to capture such reorderings, it is unlikely that the grammar will live up to expectations.


data analysis shows that "NP 会" is a frequent pattern in the training corpus, and there are 874 binary rules of which the source language sides are "NP 会".


Therefore, if we can find a method to greedily reduce the size of each bucket ( , ′), we can reduce the overall expected edge competitions when parsing with ′.

to achieve acceptable decoding speed, we range the beam size from 10 to 70.


This property of synchronous binarization guarantees the polynomial time complexity of SCFG parsers even when an n-gram language model is integrated, which has been proved to be one of the keys to the success of a string-to-tree syntax-based SMT system.

the costs for binarization trees are computed based on the cost table given in table 1.


Though enlarging beam size can reduce the search errors and improve the system performance, the decoding speed of string-to-tree SMT drops dramatically when we enlarge the beam size.

some binary rules with different source-sides may also have competitions in a chart cell.


One advantage of the bitext-dependent pivoting approach is the use of the additional human knowledge that is encapsulated in the parallel sentence alignment.

these results also show that our method is especially useful in settings involving low-density languages or special domains: the smaller subset models, emulating a resource-poor language situation, show higher gains than larger models (which are supersets of the smaller subset models), when augmented with paraphrases derived from the same paraphrase training set.


word association profiles, or cooccurrence vectors) of out-of-vocabulary words and phrases in the source language; then, generating paraphrase candidates from phrases that cooccur in similar contexts, and assigning them similarity scores.

these approaches still require bitexts where one side contains the original source language.


The sliding window and word association (SoA) measures.

we conducted experiments to find out how these relative advantages play out.


Parting ways with previous work on sentence compression, which heavily relied on humans to create gold standard references, this work has a particular focus on using data gathered from RSS feeds, which if successful, could open a door to building gold standard data in large quantities rapidly and with little human effort.

nYT-RSS has a heavy concentration of scores in a stretch between -5 and -10, indicating that for the most of time, the overlap between compression and its source is rather modest compared to CLwritten and CLspoken.


The direct inversion model fails to capture the transitional behavior from one phrase to another, which makes it more vulnerable to n-gram mismatch, especially when evaluated on the GEOQUERY corpus where phrase-to-phrase transitions are more frequent.

we call this model the direct inversion model.


The generic averaged perceptron training algorithm appears in Figure 3.

averaged perceptron models are very simple, just requiring a decoder and a simple update function, yet despite their simplicity they have been shown to achieve state-of-the-art results in Treebank and CCG parsing (Huang, 2008;Clark and Curran, 2007a) as well as on other NLP tasks.


Event triples generated include <PER bush,worked,ORG amoco> and <PER bush,started,ORG jnb>.

analysis shows that the representations compared are complementary, suggesting that extraction performance could be further improved through system combination.


This is problematic for modeling language which has regular structure.

a simple and popular (McDonald, 2000;Bullinaria and Levy, 2007;Lowe, 2000) way to construct a semantic space model is to associate each vector component with a particular context word, and assign it a value based on the strength of its co-occurrence with the target (i.e., the word for which a semantic representation is being constructed).


This is not to say that more sophisticated approaches have not been developed or that they are not useful.

the composition models yield reductions in perplexity when combined with a standard n-gram language model over the n-gram model alone.


Ideally, we would have liked to train Roark's (2001) parser on the same data as that used for the semantic models.

as more vectors are combined, vector addition will lead to greater generality rather than greater specificity.


The simplification reduces the number of free parameters.

essentially, these models provide a simple procedure for constructing spatial representations of word meaning.


Dividing through by the overall probability of each context word prevents the vectors being dominated by the most frequent context words, which will often also have the highest conditional probabilities.

in this case we use linear interpolation (equation 18) because the models are roughly equally predictive and also because linear interpolation is widely used when structured language models are combined with n-grams and other information sources.


Much work in linguistic theory (Partee, 1995;Montague, 1974) has been devoted to compositionality, the process of determining the meaning of complex expressions from simpler ones.

the products h i • p(c i ) must themselves be a valid distribution over context words.


To ensure that our model is competitive, we tested our system on the data set from the SemEval 2007 preposition WSD task (Litkowski and Hargraves, 2007).

a joint learning model for argument identification and preposition sense is unlikely to perform better than the independent models.


Wordnet-dependent parameters depend on the graph based structure of Wordnet whereas the Corpus-dependent parameters depend on various statistics learnt from a sense marked corpora.

for example, consider the f-score as given by IWSD for Marathi.


The sentence data used contains examples of either one or the other.

for the training/test set there were 139 (73-150) sentences per MWE on average, totaling 3350, with a 40% average positive ratio.


In addition, many studies have shown the importance of order and distance in MWE extraction in English (two recent examples are (Dias, 2003;Deane, 2005)).

the developed features are not specific to a particular type of expression.


The advantage of such methods is that association relations are established at the phrase level instead of the lexical level, so they have the potential to resolve the above-mentioned translation problem.

our MWE translation extraction method is similar to the two-phase approach proposed by Smadja et al.


Even if all collocations in the gold set are included in the 200Kbest list, the precision is only 28%.

according to the analysis in subsection 3.2, we need to penalize the aligned word pairs with lower frequencies.


Classifier parameters (C for PA/SVM and maxent's Gaussian prior) and number of iterations (up to 10) for the online methods were optimized using a single randomized iteration.

(2008), where exact updates were shown to over-fit but converged after a single iteration of training.


Each article contains one or more labels describing its general topic, industry, and region.

the CW classifier shows the opposite behavior after one epoch of training (the leftmost blue bar), assigning low prediction probability (less than 0.3) to more than 1,200 examples and prediction probability of at least 0.9 to only 100 examples.


When the relevance labels are assigned by humans, it is reasonable to assume that queries with the lowest information entropy of labels are the least noisy.

we evaluated the ranking model adaptation methods on two web search domains, namely (1) a name query domain, which consists of only person name queries, and (2) a Korean query domain, which consists of queries that users submitted to the Korean market.


this strongly motivates the adoption of an approach similar to the Natural Language Processing (NLP) technique of using n-grams to deal with unseen words.


As shown in Table 11, NavClick (NavClick Feature Set) best improves relevance for two word queries.

in Section 2, we introduce our overall methodology.


As there is a match between the question head word film and the dictionary named film, each noun phrase in the candidate sentence is checked.

the experiment shows that the proposed head word and its hypernym features play an essential role in building an accurate question classifier.


Specific features are question dependent.

if the question is ABBR type, we tag CoNLL LOC, ORG and MISC entities as candidate answers; If the question is HUMAN type, we tag CoNLL PER and ORG entities; And if the question is LOC type, we tag CoNLL LOC and MISC entities.


The search method commonly used in dependency parsers is a chart-based dynamic programming algorithm that finds the highest-scoring projective dependency tree under an edge-factored scoring function.

we believe that for any interesting linguistic structure, the second layer will be highly dependent on the structure of the first layer.


Our goal in this paper is to propose a novel mechanism allowing a user to cluster a set of documents along the desired dimension, which may be a hidden dimension, with very limited user feedback.

each of them can be thought of as revealing an important dimension of the data.


We discussed betweenness centrality before.

egory to itself called a self-edge, indicating that a word and its antonym (with opposite semantic orientation) both exist in the same category.


The extrinsic evaluation shows that our thesaurusand affix-based lexicon is significantly more accurate than SentiWordNet.

by removing edges that had a weight less than 6, we can visualize a smaller and more understandable 540 edge network of the core categories and any new isolates created.


This is expected because it is generated by largely automatic means.

6 Visualizing the semantic orientation of thesaurus categories In recent years, there have been substantial developments in the field of information visualization, and it is becoming increasingly clear that good visualizations can not only convey information quickly, but are also an important tool for gaining insight into an algorithm, detecting systematic errors, and understanding the task.


Each node is colored by its semantic orientation (red for negative, blue for positive) and edges are colored by their weight, from red to blue.

the method assigns a semantic orientation to a word-sense combination similar to the SentiWordNet approach and differing from the General Inquirer and Turney-Littman lexicons.


Some of the errors in MSOL were due to nonantonymous instantiations of the affix patterns.

the goal of this work is to produce a highcoverage semantic orientation lexicon and so we additionally evaluate the lexicons on the extrinsic task described below.


On the other hand, the word "Fondue" is more likely to retain higher probability in P (•) since it tends to appear as dish names.

let P e (•) denote a distribution whose support is text(e); this corresponds to the distribution of words specific to the object e, taken from the description text(e).


Suppose we impose a threshold and return the most likely object only when its score is above threshold, we can then compute precision and recall at different thresholds.


Krenn & Evert (2001) experiment with one-tailed variants of LL and Chi-Squared measures, although they do not define these variants.

rather, it might be most useful to predict common or majority human responses.


Inspired by the success of English grapheme-to-phoneme research in speech synthesis, many researchers have proposed phoneme-based English-to-Chinese transliteration models.

chinese phonemebased transliteration models in the literature fall into either M (E G , c P ) or M (E P , c P ) (Meng et al., 2001;Gao et al., 2004;Jiang et al., 2007;Lee and chang, 2003;Wan and Verspoor, 1998;Virga and Khudanpur, 2003).


We use a bottom up, minimum distance clustering for valid affix pairs (rules).

the count of a rule candidate r={b 1 , b 2 } in a trie is the number of stem candidates it appears with: We also use CAff(s) for the set of affix candidates of stem s across several tries, and CRule(s) for the set of rule candidates of a stem s across several tries.


Then we ran an experiment removing the PoS tags altogether, and the performances were 93.25% accuracy and 88.58% F-measure respectively.

the fact that different authors use different versions of the same gold standard to evaluate similar experiments (e.g.


The closest experiment to ours is the one by Gao & Johnson (2008) who run their Bayesian HMM over the whole WSJ and evaluated against the full gold standard, the only difference being is that we exclude the CoNLL shared task sec-    Table 2: Performance of the output of the three iHMM runs when used in shallow parsing against number of iteretions (in thousands).

since the sum of the entries must sum to one, the rows of the transition matrix will be zero everywhere and all its mass in a random location.


the performance of our algorithm on the Brown corpus is as strong as on the WSJ corpus.

for example, in the sequence "...for his stepping...", the pair (for his) is unambiguous in that for has only one possible tag 'IN' and both possible tags of his, 'PRP' and 'PRP$', fall into the Nominal category, then a rule (IN : N ) is extracted; but (his about) is not usable since his has more than one possible tag and about has two possible tags, 'RB' and 'IN', which are neither both 'N' nor both 'V'.


In this section we describe the setup that was used for all experiments reported in this paper.

given the many choices available in mapping features to weights, we will describe the mapping function in general terms.


A Korean morphological analysis tool 4 was used to create the supervised tokenization.

we also used the most simple word-alignment model, but more complex word alignment models could be incorporated into our bilingual model.


For both language pairs, this accurately reflects the empirical distribution of token length, as can be seen in Figure 2.

c j is the word formed by concatenating characters i through j, and a is a variable indicating which English position generated f .


The number of bucket cells also effects the overall error rate significantly since smaller ranges reduce the probability of a collision.

if a collision occurs that n-gram is diverted to the overflow dictionary; otherwise the fingerprint is stored in the bucket.


performance from the batch retrained and online models.

the O-RLM is incrementally retrained online.


Taking advantage of the dual annotations provided for each sentence in the SSR corpus, we can report double-reference evaluation.

while the SwBD corpus has been used in some previous simple disfluency labeling work (e.g., Johnson and Charniak, 2004;Kahn et al., 2005), we consider the SSR for its fine-grained error annotations.


New Methods: The hybrid method variants proposed in this paper (hybrid-prop-cos-ll and hybrid-filt-cos-ll) were the best performers on the RG-65 test set.

we showed that the hybrid method correlates with human judgments of semantic distance in most cases better than any of the other methods we replicated.


If the corresponding strings are matched or the relations mentioned above exist, the two predicates, arguments, or dependencies are related; otherwise, not.

then we define a new measurement for a pair of texts, called textual Relatedness, which is a weaker concept than semantic similarity or paraphrase.


Therefore, another focus of this paper will be to apply SRL to the RTE task.

then we define the relatedness function between two trees as the minimum value of the relatedness scores of all the triple pairs from the two trees.


New alignment algorithm has to be designed to explore other strategies.

figure  2 illustrates the PAG for both T and H after semantic parsing, and the resulting P-Trees and A-Trees after applying the decomposition algorithm.


Even more, we also plan to compare/combine it with other methods which are not based on overlapping information between T and H.    The first author is funded by the PIRE PhD scholarship program sponsored by the German Research Foundation (DFG).

we simplify these basic relatedness functions into a binary decision.


From an engineering perspective, adopting the new term-weighting scores produced by our model is straightforward.

in addition, when more information is available, such as the position of a term in the document or whether a term is part of an anchor text, incorporating it in the similarity measure in a principled manner may not be easy.


for this purpose, we propose a pattern extraction algorithm using snippets retrieved from a web search engine.


Table 1 compares the proposed method against Miller-Charles ratings (MC), and previously proposed web-based semantic similarity measures: Jaccard, Dice, Overlap, PMI (Bollegala et al., 2007), Normalized Google Distance (NGD) (Cilibrasi and Vitanyi, 2007), Sahami and Heilman (SH) (2006), co-occurrence double checking model (CODC) (Chen et al., 2006), and support vector machine-based (SVM) approach (Bollegala et al., 2007).

our wildcard query retrieves snippets in which a and b appear within a window of seven words.


An unlimited number of clause boundaries may intervene between the moved element and the gap (hence "unbounded").

the DCU post-processor is ideal because it does produce dependencies in a GR format.


For a statistical parser to cope with nonconfigurational phenomena as observed in, for instance, Hebrew or German, it should allow for flexibility in the form of realization of the grammatical functions within the phrase-structure representation of trees.

in our case, there is more statistical evidence in the data for, e.g., case marking patterns, than for association of grammatical relations with structurally-marked positions.


Some papers directly target specific applications, and build lexical resources as a side effect.

we can see that in all cases, utilization of the latter greatly improves recall.


Concept definitions were taken from existing WordNet subtrees, and the obtained new terms were manually scored by human judges.

hence, we take a simple but effective query-based approach.


Great efforts have been invested in manual preparation of concept resources such as WordNet (WN).

second, we would like to favor languages with less resources for a given concept, since noise empirically appears to be less prominent in such languages 10 .


these so-called pivot language based methods rely on the idea that the lookup of a word in an uncommon language through a third, intermediated language can be automated.


However, this discrepancy is not reflected in the traditional recall evaluation, since each word has an equal weight, regardless of its frequency of use.

euroWordNet was specifically designed for word disambiguation purposes in cross-language information retrieval (Vossen 1998).


For our experiment, we use the first normalization method, though other methods can be applied as well.

the transition probabilities can be interpreted as edge flows in Markov random walk over graph vertices (documents in collections).


Spectral clustering is the task of finding good clusters by using information contained in the eigenvectors of a matrix derived from the data.

the number of edges between documents in different languages is about as many as the number of must-linked constraints given.


Figure 1 gives an illustration of edge-creation process when two multilingual documents (nodes) are connected.

k represents the number of documents that are affected by pairing up two multilingual documents.


If the model assigns all tokens in a tuple to a single topic, the maximum posterior topic probability for that tuple will be near to 1.0.

the growth of the web, and in particular Wikipedia, has made comparable text cor-  pora -documents that are topically similar but are not direct translations of one another -considerably more abundant than true parallel corpora.


The above results suggest that the Web-based suggestions system performs at least as well as the Aspell system.

damerau-Levenshtein edit distance was introduced as a way to detect spelling errors (damerau, 1964).


A critical issue for reading difficulty prediction is how to handle words that appear in a new document that have never been seen in the training/development texts.

there are some substantial differences between these previous studies and our approach.


Approximately 30 years have passed since Gilhooly and Logie's study was conducted.

sVR was applied twice using different sets of features -once with the document word frequencies and once with the esti- Figure 6: The scatter plot demonstrates the strong relationship between predicted and actual global readability levels.


Table 2 shows that average performance without n-grams (across two-, three-and nine-zone tasks) for line-based classification drops by 4.67%.

this makes the zone fragment approach potentially more portable for use across email data from different enterprise domains.


Sender zones contain text written by the current email sender.

jangada did not accurately identify forwarded or reply content in email data from the Enron email corpus.


Regarding Precision, simply judging the correctness of terms and their individual classification links may provide high scores, but this doesn't help with the eventual assembly of terms into a single coherent taxonomy.

the algorithm was allowed to run for 10 iterations.


Indeed, the whole point of our work is to learn concepts and taxonomies that go above and beyond what is currently available.

table 1 shows a comparison of the harvested terms against the terms present in WordNet.


As we show in this paper, WordNet (Fellbaum, 1998), the most obvious contender because of its size and popularity, is deficient in various ways: it is neither complete nor is its taxonomic structure inarguably perfect.

this requires human judgment, which we could not afford.


Overall, these results demonstrate that our algorithm produced many high-quality intermediate concepts, with good precision.

it is necessary to compare with something, and it is important not to skirt the issue by conducting evaluations that measure subsets of results, or that perhaps may mislead.


our work can be viewed as a new method for acquiring such bias from text; moreover, our approach is not specialized for ILP and may be used to transform the feature space in other forms of relational learning as well (Roth and Yih, 2001;Cumby and Roth, 2003;Richardson and Domingos, 2006).


Unlike a mention-pair model, an entity-mention model is a classifier that decides whether or not an active mention m k is coreferent with a partial cluster c j that precedes m k .

as mentioned before, Denis and Baldridge (2008) train a mention-ranking model.


Like the mention-pair model, each training instance i(m j , m k ) represents m k and a preceding mention m j .

the instance representation employed by a cluster ranker is identical to that used by an entity-mention model, where each training instance i(c j , m k ) represents a preceding cluster c j and a discourse-old mention m k and consists of cluster-level features formed from predicates.


As noted before, a mention-pair model is a classifier that decides whether or not an active mention m k is coreferent with a candidate antecedent m j .

this is not the case when system mentions are used.


Section 5 shows the experimental results.

their experiments assume true ACE mentions(i.e.


Our further evaluation reveals that the performance improvement difference among different domains of the ACE 2003 corpus is due to the distribution of pronouns' antecedents occurring over different sentence distances, as shown in Table 4.

their experiments on the ACE 2003 corpus shows that, while the two semantic role features much improve the performance of common noun resolution by 3.8 and 2.7 in F-measure on the BNEWS and NWIRE domains respectively, they only slightly improve the performance of pronoun resolution by 0.4 and 0.3 in F-measure on the BNEWS and NWIRE domains respectively.


subject and object), using some heuristic rules.

the ability of linking referring expressions both within a sentence and across the sentences in a text is critical to discourse and language understanding in general.


Moreover, given a pronominal anaphor, its relative ranking among all the pronouns in a sentence, according to relevant semantic role information and its surface position, is incorporated.

table 4 lists the pronoun distributions with coreferential relationships in the training data and the test data over pronominal subcategories and sentence distances.


The statistics in the CoNLL 2004 and 2005 shared tasks (Shen and Lapata, 2007) shows that the semantic roles of ARG0/agent and ARG1/patient account for 85% of all arguments and most likely act as the centers of the local focus structure in discourse due to the close relationship between subject/object and agent/patient.

given the centering theory as described above, we can draw the following conclusions: 1) The centering theory is discourse-related and centers are discourse constructs.


The chance that many different persons carry this name is high.

this happens because the prior probability of coreference of any two "John Smiths" mentions is low, as this is a very common name and none of the "John Smith" has an overwhelming number of mentions.


Let us recall the "John Smith" and "Barack Obama" example from the previous section.

two professional contexts are present in only 4% of the cases.


Our positive results in even this situation suggest that in future, pairwise reordering costs should probably be integrated into MT systems.

neither alternative produced improvements in our ultimate translation quality.


We used GIZA++ (Och and Ney, 2003) to align approximately 751,000 sentences from the German-English portion of the Europarl corpus (Koehn, 2005), in both the German-to-English and English-to-German directions.

the results in the table also vary the reordering model used in Moses, set to either a single-parameter distance-based model, or to the lexicalized bidirectional msd model.


We ran the algorithm multiple times over the training sentences, 4 We tried two other methods for deriving English word order from word alignments.

it might be better in future work to still allow the local search to take more than one step, but to penalize the second step.


The choice of modelling P (φ(x, a, y)|x) instead of modelling P (y|x) directly is crucial, both from a modelling and from a computational perspective.

second, using a heuristic proposed in (Och et al., 1999), all the aligned phrase pairs (x , a , y ) satisfying the following criteria are extracted: (1) x and y consist of consecutive words of x and y, and both have length at most k, (2) a is the alignment between words of x and y induced by a, (3) a contains at least one link, and (4) there are no links in a that have just one end in x or y .


The general process is that for each rule we convert it into a hyper-path and then add the hyper-path into a hyper-tree incrementally.

finally we discuss the top-down rule matching algorithm between forest and hyper-tree.


This section studies the impact of the forest pruning threshold on the rule matching speed when setting the maximum rule height to 5.

it does not consume real translation time.


In order to produce the ranked lists of candidate suggestions for each query, due to the lack of training data, we have opted for the unsupervised procedure described in the previous section: 1.

this work is also related to the analysis of temporal properties of information streams in data mining (Kleinberg, 2006) and information retrieval from time series databases (Agrawal et al., 1993).


The latter is a powerful predictor: baselines based on class frequency outperform most of the systems that participated in the KDD Cup (Lin and Wu, 2009).

we expect them to be useful for solving other applications that require handling entities, such as named entity recognition and classification, relation extraction or disambiguation.


These results illustrate a typical contribution of current knowledge sources for current RTE systems.

the resulting size is O(|T |+k), as we can see from Figure 3.


Accuracy on the development set is used to select the number of times we pass through all Given the above framework, we require training derivations D, feature vectors F , and a decoder to carry out the argmax over all d reachable from a particular source word.

the + perceptron line shows that setting the three weights with perceptron training results in a huge boost in accuracy, nearly matching our indicator system.


The different behavior of the re-ranking model in the LUNA and MEDIA corpora is due partially to the task complexity, but it is mainly due to the fact that CRFs have been deeply studied and experimented (see (Hahn et al., 2008)) on MEDIA.

table 4 shows the results of the SLU experiments on the MEDIA and LUNA test sets using the manual transcriptions of spoken sentences and a re-ranker based on PtK and the FEAtURES structure (already reported in the previous table).


In order to have highly accurate skip-above pairs, a set of thresh- Table 2: Skip-above pairs count vs. human judgements (e.g., the element in the third row and second column means we have 40 skip-above pairs with "excellent" url 1 and "perfect" url 2 ).

the reality is that dedicated training data is usually insufficient, while it is possible that non-dedicated data can also help the learning.


An important observation is that the relevance patterns for the ranking within a specific category may have some unique characteristics, which are different from generic relevance ranking.

4) The utility of click pairs: by our study, the quality of click pairs extracted by SSL approach is comparable to those extracted by heuristic rule approach.


For example, in Table 5 (a), the NDCG 5 values (0.7822 and 0.7834) are very close to each other.

web users' query intentions are extremely heterogeneous, which makes it difficult for a generic ranking model to achieve best ranking results for all queries.


All bilingual data are used to extract phrases up to length 3 on the source side.

in this section, we will present in detail the method for systematically deriving SMT systems from a standard linear SMT model based on feature subspaces for system combination.


In order to decide the sequence of sub-systems to add, we first evaluate the performance of pair-wise combinations between each sub-system and its baseline system on the development set.

also some of the feature tweaking in this solution is system or language specific, thus for any new systems or language pairs, human engineering has to be involved.


In following experiments, we only consider removing common features shared by both baseline systems for feature subspace generation.

since lower-order models have already been contained in higher-order model for the purpose of smoothing in almost all statistical language model implementations, there is also no extra training cost.


The SuperARV essentially organizes information concerning one consistent set of dependency links for a word that can be directly derived from its syntactic parse.

the context space is partitioned by means of binary questions.


Having found this value, we compare the hierarchical real-time detector's performance with alternative detectors, these being: • an off-line detector applied to off-line ASR transcripts, • a flat real-time detector, • an off-line detector applied to the real-time ASR transcripts.

section 2 describes related work, and section 3 describes our annotation scheme for decision discussions, and our experimental data.


We showed that this difference can be bridged to some extent by a binary feature relating to idiomatic word usage, and that the difference vanishes when testing outside of the annotated corpus.

ideally, we would like our classifiers to learn only from robust features: consider only features that appear at least k times in the training data (rarefeature pruning).


This was recently followed by (Matsuzaki et al., 2005;Petrov et al., 2006) who introduce state-of-the-art nearly unlexicalized PCFG parsers.

as discussed above, all state-of-the-art published methods rely on lexical features for such tasks (Zhang et al., 2001;Sha and Pereira, 2003;Finkel et al., 2005;Ratinov and Roth, 2009).


This setting is shown to produce good results for sequence labeling tasks in previous work (Kudo and Matsumoto, 2000), and is what most end-users of SVM classifiers are likely to use.

for the case of non-linear kernels, anchored and L2-SVM produce different results, as the anchoring is applied prior to the kernel expansion.


This yields 58.5 pairwise F1 (see SEM-COMPAT in Table 2) as well as similar improvements across other metrics.


This change by itself yields 51.7 pairwise F1 (see BASE-TREE in Table 2), which is small overall, but reduces pairwise pronoun antecedent selection error from 51.3% to 42.5%.

to most recent research, our pairwise decisions are not made with a learned model which outputs a probability or confidence, but instead for each mention m i , we select an antecedent amongst m 1 , .


In the experiments, with each of our approaches, we successfully obtained relevant errors.

in the experiments, some errors could not be forcibly corrected by our approach.


Torisawa (2005) has pointed out that when r expresses the purpose of using a tool f , P (r|f ) tends to be large.

shows the accuracy figures for the N-best entailment pairs for each method, with N being 1,000, 2,000, .


(People do not usually repeat v r and v l in the same document to avoid redundancy.)

note that P (v r |v l ) should be 1 when entailment v l → v r holds (i.e., v r is observed whenever v l is observed) and we have reliable probability values.


Standard Phrase-based SMT decoders merge translation hypotheses if they cover the same source words and share the same n-gram language model history.

we tuned this on the MT03 DevSet for the best accuracy while maintaining a manageable search space.


Fortunately, CCG specifies the desired kind of lexical categories (supertags) st i for every word and a small set of combinatory operators o i that combine the supertag st i with a previous parse state S i−1 into the next parse state S i .

the baseline produced better translation in 8% of the analysis sample.


We have presented a novel parsing paradigm, bilingually-constrained monolingual parsing, which is much simpler than joint (bi-)parsing, yet still yields mild improvements in parsing accuracy in our preliminary experiments.

the search space of joint parsing is inevitably much bigger than the monolingual case, forcing existing approaches to employ complicated modeling and crude approximations.


Due to the nature of the rule based coreference system (in contrast to machine learning based), the weights of relations may not be accurately estimated.

this research was supported by British telecom grant Ct1080028046 and BISC Program of UC Berkeley.


2 For example, the head word company of the NP the company immediately determines its SC being organization.

we present four binary lexical feature sets as follows.


Table 4 shows the SC classification accuracy of all NPs (all) and non-pronoun NPs (non-PN) on the train-ing+development (we refer it as training hereafter) and test datasets.

the difficulty is to accurately compute the semantic class features.


Several types of labeled ordered trees are possible (Figure 3).

next, we give a qualitative analysis for better interpretation of our problem and results.


In both labeled and unlabeled sets, the most 5 Our mapping tables are available at http://home.postech.ac.kr/∼stardust/acl09/.

more recently, using speech acts has become an appealing approach in summarizing the discussions (Galley et al., 2004;mcKeown et al., 2007).


We also conjecture that the discrepancy between EMAIL and FORUM is probably due to the more heterogeneous nature of the FORUM data where anyone can post and reply while EMAIL (Enron or The improvement of less frequent tags is prominent, for example 25% for action motivator (AC), 40% for polite mechanism (P), and 15% for rhetorical question (QR) error rate reductions were achieved in FORUM data ( Figure 5(b)).

f1∼3, f4∼5, and f6 are three posts in a forum thread.


(2006)'s work, Ravi and Kim (2007) applied speech act classification to detect unanswered questions.

these methods create a large number of lexical features that might not be necessary for speech act identification.


The remaining type 8 words are usually proper names.

since the opinion extraction at word level concerns only word structure, no retraining for the best threshold is need when domain shifts, which is a superiority of our method.


Figure 1 shows that "冒汗" (sweat) is an example word listed under the verb sense of the character "冒" (perspire), thus the character "冒" (perspire) in the word "冒 汗" (sweat) functions as a verb.

the structure is like a subject-verb sentence condensed in one word.


The longest common subsequence is divided into consecutive matches, producing in our example A, B|C|D .

in our experiments, r = 10, f = 5, k = 5.


This T − value corresponds negative precision 0.70 and negative recall 0.02.

despite its simpler nature, we believe the task we address is of practical use: a list of single-snippet definitions from Web pages accompanied by the source URLs is a good starting point for users seeking definitions of terms not covered by encyclopedias.


We then compute the similarity between each training window and the centroid of its target term.

some Web search engines provide special facilities (e.g., Google's "define:" query prefix) that seek definitions of user-specified terms in online encyclopedias or glossaries; to save space, we call both "encyclopedias".


Our evaluation on the CLASSIFIEDS data showed that the learning objective presented here, combined with the use of collocation-based VE, yielded remarkably good accuracy performance.

our model is desired to be consistent with such prototype information.


Given a set of labeled examples , we can estimate model parameters in a supervised machine learning setting.

we can optimize p λ (y|x, v) instead of p λ (y|x) during learning.


The work was supported in part by the National Natural Science Foundation of China (60535030; 60605016), the National High Technology Research and Development Program of China (2006AA010103), the National Key Basic Research Program of China (2004CB318005,  2004CB318105).

section 2 reviews some closely related works, including the lexical semantic related parsing and the hierarchical state-split unlexicalized parsing.


Results show that the words in the same subcategory of POS tags are semantic consistent in some cases.

in this section, we designed several experiments to investigate the validity of refining grammars with semantic knowledge.


However, restricted by the strong contextfree assumptions, the original PCFG model which simply takes the grammars and probabilities off a treebank, does not perform well.

based on the hand-craft sense information, this model is proved to be effective for parse selection.


If more special kinds of semantic classes are chosen, it will make the structural information weaker.

we refine the subcategories "PN-0", "PN-Event-0" and "PN-AttributeValue-2" from the good starting annotations.


Figure 2.1 shows valid and invalid TTS templates which can be extracted from an example hosting TTS template.

a few cross-boundary noisy alignments could disable a big portion of a training syntax tree, while for a phrase-based SMT system, their effect is limited to the phrases they align.


Our results also show a substantial improvement on an existing automatic tagging approach based on TF×IDF, and the results compare well to other systems.


In fact, however, their methods relied on the existence of a full-coverage lexicon, both in training and running time.

the measures for evaluation are also introduced.


With this assumption, probabilistic modeling can be reduced to a classification problem.

since VerbNet classes are designed to represent the distinctions in the syntactic frames that verbs can take, features, representing the statistics of syntactic frames, are extracted from the unannotated corpora.


A major portion of the errors were caused by confusing the classes that take the same prepositions.

when strong clues do not exist, verbs tend to be misclassified into this class.


We particularly analyze the contribution of SR of query and document terms to this task.

it is not yet clearly understood, what the most beneficial method is to employ SR using these resources in IR.


We hypothesize that our trigram approach may be more robust to disfluent and fragmented meeting speech and emails   (Brill, 1992) over all sentences in a document.

(2006) employed several types of features including dependency structures in which words can be backed off to POS tags.


We set both the maximum phrase length and the language model order to ten; this value was found by tuning on the development dataset.

10 unlike Spanish and Portuguese, there were far less systematic spelling variations between Malay and Indonesian.


different NEs associated with different relations are described by the same parse tree.

every word in a dependency tree has exactly one parent except the root.


In the case of logical metonymy an implicit predicate is inferable from particular verbnoun and adjective-noun pairs in a systematic way.

in FN information about syntactic realization patterns of frame elements as well as information about frequency of occurrences of these patterns in corpora is provided.


It has been claimed that information about predicates associated with nouns can be helpful for a variety of tasks in natural language processing (NLP), see for example (Pustejovsky et al., 1993;Voorhees, 1994).

finally, among all possible n (numbers of the considered relations with the highest ranking) we selected that one which provides the maximal f max value.


For the scientific abstracts data set, we defined the following boolean features to represent the position of the sentence the keyword occurs in.

materials and methods or Introduction and Background sections).


We formulate the first sub-problem as a supervised classification task, where we classify the potential keywords as real speculation keywords or not by using a diverse set of linguistic features that represent the contexts of the keywords.

previous studies focus on identifying speculative sentences (Light et al., 2004;Medlock and Briscoe, 2007;Szarvas, 2008;Kilicoglu and Bergler, 2008).


The keywords were annotated with a minimalist strategy.

kilicoglu and Bergler (2008) showed that manually identified syntactic patterns are effective in classifying sentences as speculative or not.


The best F-measure (82.82%) for the full text articles is achieved by using all the feature types except the co-occurring keywords.

"appear" is a real speculation keyword in 86% of its occurrences in the abstracts and in 83% of its occurrences in the full papers, whereas "can" is a real speculation keyword in 12% of its occurrences in the abstracts and in 16% of its occurrences in the full papers.


Our approach is to develop probabilistic models that would provide a good framework for such studies.

perhaps a less trivial observation is that the words restaurant and chef are frequent in UK blogs, but the Indian word distribution is dominated by words pertaining to recipes.


Gibbs sampling is a type of Markov chain Monte Carlo algorithm and is what we employ in this paper, as it is simple to derive, comparable in speed to other estimators, and it approximates a global maximum (whereas EM algorithms may only converge to a local maximum).

we examine the topic of food and eating.


e m be a corpus of translations, one for each source sentence.

this drop in tuning performance comes with a corresponding increase of 0.6 on the test set, relative to MERT training.


We compare two practical sampling strategies as follows: (1) Randomly sampling, which picks the initial seeds from the training data using a random scheme.

sampling is a part of statistical practice concerned with the selection of individual observations, which is intended to yield some knowledge about a population of interest.


With the plenitude of unlabeled natural language data at hand, semi-supervised learning can significantly reduce the need for labeled data with only limited sacrifice in performance.

actually, for some settings the recalls even decrease slightly.


Experiments on the ACE RDC 2003 and 2004 corpora show the stratified sampling strategy contributes more than the bootstrapping procedure itself.

these relation instances are first divided into 5 sets, then, one of them (about 860 instances) is used as the test data set, while the others are regarded as the training data set, from which the initial seed set is sampled.


By contrast, even small size of labeled instances can be fully utilized to correctly induce the unlabeled instances via LP algorithm due to its ability to exploit manifold structures of both labeled and unlabeled instances (Chen et al., 2006).

when m is equal to 100, the whole set of the base example is added to the labeled data, i.e.


An emotion corpus for Japanese was built for rec-ognizing emotions and emotion estimation (Ren, 2009;Matsumoto, 2006).

the statistics show that emotion of love has high independence, however, joy, surprise and angry has relative low independence.


Note that the Corrected  Table 5: Our trained MLN outperforms two other systems at predicting whether sentences of the form "Every <class 1> <relation> a/some <class 2>" have a plausible indirect reading or not.

some of our technique's errors highlight interesting difficulties with function detection.


We do not match numbers expressed in digits (e.g., 1234) because we found that they produced too many noisy extractions, such as dates and times.

crucially, our approach, which we call ZIPF FLATTENING, has only two parameters that need to be trained using a supervised approach, and these parameters do not depend on the relation R. the approach minimizes the amount of training data we need to a practical level.


Additionally, we discuss a language special problem, i.e.

the first four are used in semantic chunking task, and all are used in SRC task.


Most existing systems for automatic Chinese SRL make use of a full syntactic parse of the sentence in order to define argument boundaries and 2 Our system is available at http://code.google.com/p/csrler/ to extract relevant information for training classifiers to disambiguate between role labels.

the majority of Chinese nouns are of type attributive-head.


Previous work on English suggests that even good labeling performance has been achieved by full parse based SRL systems, partial parse based SRL systems can still enhance their performance.

for any given word, it is very hard to accurately find the head.


Finally, the character type features also contributed to reducing the error rate (lines A and E).

alterations are particularly common among hiragana, katakana and kanji (e.g.


As illustrated, the more restrictive a wrapper is, the worse it performs.

sEAL then statistically ranks the candidate instances (and wrappers), using the techniques outlined below, and outputs a ranked list of instances to the user.


In this paper, we are interested in the problem of recognition of the proper names (the named entity recognition task), and hence use only entities labeled with the type NAM (LDC, 2005).

also, a random partition is used to ensure that the data in each partition is representative of D u .


Retrieving this information and analyzing this content are impossible tasks if they were to be manually done.

compromising between the two, we extend the dependency tree node with phrases.


11 The SVM-KE classifiers without fstries were still faster than PKI, but as expected from a large |x d | value, the classifiers with higher conjunctive features were much slower than the classifier with only primitive features by factors of 13 (d = 2), 109 (d = 3) and 738 (d = 4) and the classification time accounted for most of the parsing time.

to minimize this computational cost, x c is to be chosen from V c as follows: There are two issues with speeding up the classifier by the computation shown in Eq.


From many previous works, redundancy precision (RP ), redundancy recall (RR) and redundancy F Score (RF ) are used to evaluate the performance of document-level novelty mining .

losee also mentioned that POS tagging shows a great potential to avoid lexical ambiguity and it can help to improve the performance of information retrieval (losee, 2001).


Previous studies for novelty mining have been conducted on the English and Malay languages .

after POS filtering using Rule2, the remaining words are: " ('n'), ('v'), ('v'), ('n')".


In addition, there is no obvious inflected or derived words in Chinese so that word stemming is not applicable.

the detected document which is very similar to any of its history documents is regarded as a redundant document.


TREC 2004 Novelty Track data is developed from AQUAINT collection.

we used machine translation to translate the remaining documents/sentences to Chinese.


This indicates that the noise in machine translation for Chinese had little impact on our actual results.

in order to reduce the noise brought by Chinese word segmentation and get a better word list for one document or sentence, we firstly apply word segmentation on the Chinese text and then utilize Part-of-Speech (POS) tagging to select the meaningful candidate words.


This paper proposes a novel document reranking method based on Latent Dirichlet Allocation (LDA) which exploits the implicit structure of the documents with respect to original queries.

in the vector space model, this distance is normally the cosine or inner product measure between two vectors.


LDA2 performed better in precision at top documents while LDA4 showed promising results in terms of more general evaluation metrics.

(2009), re-ranking algorithms that rely only on the structure of the global graph are likely lead to the problem of topic drift.


Both types of studies, however, have 1 Whether or not corpus-based studies of consistency have any bearing on syntactic priming as a reality in the human mind limited the constructions that are examined to particular syntactic constructions and alternations.

also, the previous sentence also contains a conjunction CC followed by a S node under a S node, which the reranker prefers.


They have provided evidence that syntactic consistency exists not only within coordinate structures, but also in a variety of other contexts, such as within sentences, between sentences, within documents, and between speaker turns in the Switchboard corpus.

(2005) for several English corpora, including the WSJ, Brown, and Switchboard corpora.


Were loopy BP exact, these beliefs would be the true marginals and hence a point in the marginal polytope M(G x ).

24 consists of a MAP computation and the solution corresponds to one outputŷ t ∈ Y(x t ).


Using the structure of WordNet, we can now describe a generative process that produces a distribution over a multilingual vocabulary, which encourages correlations between words with similar mean-ings regardless of what language each word is in.

on a small monolingual corpus, prediction is very poor.


A simple way to set π d,s,n is to draw it from a symmetric Dirichlet prior.

we used an nDCG-like metric to compare the performance of our model and of BL-2.


An alternative would be to rewrite the individual word w i as ¬w i .

we will measure this as the inverse of the above metric, i.e.


Higher number of bits are needed to encode these improbable n-grams which results in increased perplexity.

the upper bound UB on the score of Q is Let Q * be the question in C having the maximum score.


The similarity scores are weighted by the corresponding idf's, and normalized.

this is contrary to the actual ordering Q 25 ≻ Q 26 |Q 24 , which reflects the fact that Q 25 , which expects the same answer type as Q 24 , should be deemed more useful than Q 26 , which has a different answer type.


This may well be true in the near or even long term.


With a trivial modification we can output translations that can fit any one of the patterns, as long as we do not care which.

we cannot afford to explore a part of the hypothesis space which has no possible solutions for our constraints, we must be able to prune a hypothesis as soon as it leads us to such a subspace, otherwise we will end up on an unrecoverable garden path.


This aggregation step allows instances of the same trigram in different sentences to share information and works better in practice than direct selftraining on the output of the supervised CRF.

for example, the word 'unrar' will most likely not occur in the labeled training data.


This implies that vertices u and v are likely to have similar marginals over POS tags.

the gains from self-training are more modest (0.2%) on the evaluation (test) set.


Different from previous work, our proposed approach is designed to jointly perform both sentence boundary and sentence type prediction, and punctuation prediction on speech utterances.

the majority of the sentences are declarative sentences.


The minimum error rate training (MERT) (Och, 2003) procedure is used for tuning the model parameters of the translation system.

we start with surveying related work in Section 2.


We build two layers of labels for this task, as listed in Table 2.

this motivates us to build a joint model for performing such a task, to be presented in the next section.


For example, in the recent evaluation campaign of the International Workshop on Spoken Language Translation (IWSLT) (Paul, 2009), only manually transcribed or automatically recognized speech texts are provided but the original raw speech waveforms are not available.

these experiments can be divided into two categories: with or without duplicating the ending punctuation symbol to the start of a sentence before training.


On the other hand, joint learning and decoding using cross-product of segmentation states and tagging states does not suffer from error propagation problem and achieves higher accuracy on both subtasks (Ng and Low, 2004).

one shortcoming of this method is that calculation of marginal probabilities of features may be inefficient and some approximations are required for fast computation.


Due to the nature of the models, document boundaries are retained.


the joint probability of any word occurring with NN is much less likely to be zero and the distribution is much less likely to be peaked.


This results in a huge number of features.

the first technique adds a new class of pattern based features to learning, which are not used in any existing work.


Moreover, having many feature classes is very useful as they provide features with varied granularities and diversities.

most existing works deal with formal writings, e.g., essays of people, the Reuters news corpus and the British National Corpus (BNC).


At the end of each scan, it determines which candidate sequences have minsup and minadherence (lines 12 -13).

we proposed a new class of features which are POS sequence patterns that are able to capture complex stylistic regularities of male and female authors.


Sample selection bias was first introduced in the econometrics by Heckman (1979).

this assumption may not hold in practice, i.e., the training and the test distributions can be different.


Since we have three ways of aggregating the scores of a paragraph's nearest neighbors and three ways of aggregating the resulting paragraph scores, this second method H s for scoring organization has nine variants.

table 3 shows the number of essays that received each of the seven scores for organization.


In fact, to our knowledge, there is no publicly-available corpus that is annotated with sentence function labels.

the intuition is that at least some portion of an essay's organization score can be attributed to the organization of the sentence sequences of its component paragraphs.


Log-likelihoods of the held-out set are plotted with respect to the WER observed for each OCR engine.

finally, conclusions and future work are presented in Section 5.


Figures 4 and 5 show the results of the document clustering experiments with and without feature selection, respectively.

this may be an artifact of the relatively poor starting performance for this dataset, a result of the fact that the gold standard labels do not align  Figure 4: Results for the clustering experiments on the three synthetic datasets without feature selection.


When compared to other techniques, OPCA had the highest accuracy while still having a run-time that allowed scaling to large data sets.

if we post-process full translation by an LSI model trained on the English training set, full translation is the most accurate.


Furthermore, language model integration becomes more expensive here since the decoder now has to maintain target-language boundary words at both ends of a subtranslation (Huang and Chiang, 2007), whereas a phrase-based decoder only needs to do this at one end since the translation is always growing left-to-right.

but unlike in phrase-based decoding, we can not simply choose any arbitrary uncovered subtree for the next step, since rules already dictate which subtree to visit next.


This research was funded in part by the Army Research Laboratory through ARL Cooperative Agreement W911NF-09-2-0072 and by the Office of the Director of National Intelligence (ODNI), Intelligence Advanced Research Projects Activity (IARPA), through the Army Research Laboratory.

the overall experimental setup and corpus preparation process is presented in Figure 3.


Note that the prefixes have a long tail: these ten prefixes cover only 303 of the 359 test examples.

one source of information that has not previously been exploited is the "lexical fixedness" (Fazly et al., 2009) of noncompositional prefix verbs.


If a prefix verb is compositional, it is more likely to occur near its separated stem in text.

there is still much to be gained from applying SCD, which makes a third as many errors as the system which always assigns compositional (95.7% for SCD vs. 85.3% for Base1).


In recent years, there has been an increasing interest in SRL on several languages.

it contains parallel proposition annotations for 325 files (chtb 0001.fid to chtb 0325.fid) from Chinese-English parallel Treebank.


Using our joint inference model, F1 scores of SRL results on Chinese and English text achieve 79.53% and 77.87% respectively, which are 1.52 and 1.74 points higher than the results of baseline monolingual SRL combination systems respectively.

for both Chinese and English SRL systems, we use the 3-best parse trees of Berkeley parser (Petrov and Klein, 2007) and 1-best parse trees of Bikel parser (Bikel, 2004) and Stanford parser (Klein and Manning, 2003) as inputs.


For example, given And they continue [anonymously]  We have presented a supervised method for the automatic discovery of MANNER.

adding the specialized features is justified.


Among these are various classifiers, ensemble methods combining several supervised classifiers, bootstrapping and semi-supervised learning methods, using the Web as a corpus and knowledge-based methods relying mainly on machine readable dictionaries.

as far as we know, this is the first paper to address the TSD task.


Genre-specific training is typically infeasible due to such application-specific data sets being unannotated, motivating this line of research.

we assume that the data source is unknown, which is our claim in this work, and we show that we obtain better performance than using source-specific MD models.


For each word, we run 1-through 6-preceding-word windows through the classifier, and 1-through 6-word windows beginning with the word, for a total of 12 windows, yielding for each window a result like: For each of the 12 results, we extract three features: the identity of the top-scoring language, here, Swedish; the confidence score in the top-scoring language, here, 0.235; and the score difference between the target language (English for these experiments) and the top-scoring non-target language, here, 0.148 − 0.235 = −0.087.

the classifier for "mixed"-quality data and the "gazetteer" model were each trained on that set plus the "Latin" training set and the supplemental set.


For this reason, we do not train on Transactions, letting our testing on this data set serve as an example of testing on such data formats unseen.

real-world data genres inevitably include some mixed-language/non-linguistic input.


A sentence with a P P lower than a threshold θ 1 is considered "clean" and hence the "clean" baseline MD model described in Section 3 is used to detect mentions of this sentence.

deterministic characterset segmentation is applied, to mark non-targetcharacter-set passages for non-processing by the remainder of the system, or, in a multilingual system, to be diverted to a subsystem suited to process that character set.


Table 2 shows that 1) RAND outperforms SEQ by 1.2 units in F1score.

table 1 lists the major relation types and subtypes, including their corresponding instance numbers and ratios in our evaluation set.


Normally, this number is proportional to the size of that stratum in the whole data set.

the improvement comes not only from significant precision boost, but also from moderate recall increase.


Each method has equal average precision over the first 200 terms, as semantic drift does not typically occur in the early iterations.

in McIntosh (2010), a biomedical expert spent considerable time (∼15 days) and effort identifying potential negative categories and subsequently optimising their associated seeds in trial and error bootstrapping runs.


This observation is consistent with the findings reported in (Wan and Xiao, 2008b Similarly, when W ranges from 2 to 10, the performance on RESEARCH does not change much.


Moreover, jointly learning content and task parameters yields additional gains over independently learned models.

this architecture does not permit the usage of standard discriminative models which condition freely on textual features.


In fact in our corpus we found an average quotation usage of 6.44 per thread.

as different annotations (or system's output) can group sentences in different number of clusters, metrics widely used in classification, such as the κ statistic, are not applicable.


Implicitly by doing this we want two sentences in the same or adjacent fragments to have similar topic distributions, and fall in the same topical cluster.

topics in an email thread may not change in sequential way.


Furthermore, when employing the proposed method, certain relations have relatively high F-scores even with very little labeled data: Still, in this case, the extended classifier's accuracy is higher than the baseline (0.213 versus 0.122).

in Siolas and d'Alché-Buc (2000), a semantic proximity measure based on WordNet (Fellbaum, 1998) is defined, as a basis to create a proximity matrix for all terms of the problem.


Table 1 contains the F-score for each RSTDT relation, using training sets containing respectively one, two, three, five and seven instances of each relation.

each instance from the RSTDT is labeled with a single discourse relation, from which a single feature vector is created.


This shows that the proposed method is useful when large numbers of features are missing from the training set, which corresponds in practice to small training sets, with few training instances for each relation type.

the classification algorithm does not have sufficient information to correctly predict the relation type of the given test instance.


We created the following two Amazon Mechanical Turk tasks, which enable humans to play the language game on the scenarios: Speaker Task In this task, human annotators play the role of speakers in the language game.

utility (Communicative Success) We primarily evaluate a speaker by its ability to communicate successfully with a human listener.


This section focuses on an orthogonal direction: improving literal strategies with learning.

we train the speaker, S:LEARNED, (similarly, listener, L:LEARNED) on training examples which comprise the utterances produced by the human annotators (see Section 6.1 for details on how this data was collected).


He generates nearly-equivalent syntactic paraphrases of the source-side training sentences, then pairs each paraphrased sentence with the target translation associated with the original sentence in the training data.

from Table 3, it can be seen that the "Lattice" system achieves an improvement of 0.51 absolute (3.63% relative) BLEU points and a decrease of 1.6 absolute (2.14% relative) TER points compared to the baseline.


The probability p 2 is slightly greater than p 1 .

the BLEU score of the "Lattice" system is much higher than the baseline, and the tER score is quite a bit lower than the baseline.


Figure 5 plots the number of hits returned for the names from Google search engine.

this section describes (1) how we collect the English and Chinese EntityCube datasets, (2) how to build ground-truth test datasets for evaluating our framework, and (3) how to set up three parameters λ, θ, and δ.


For the first issue, a number of models have been proposed to model the structure mapping between tree and string (Galley et al., 2004;Liu et al., 2006;Yamada and Knight, 2001;DeNeefe and Knight, 2009) and between tree and tree (Eisner, 2003;Zhang et al., 2007Zhang et al., & 2008Liu et al., 2009).

mapping from tree-to-tree sequence into tree-to-string rule In this subsection, we describe what the parallel search spaces are and how to construct them.


we include results for B-Cubed F1, the harmonic mean of precision and recall.


The boxed diagonal line segment arises from the repeat of the word "processing," while the main diagonal line trivially arises from self-similarity.

this baseline serves as an upper bound of what large vocabulary recognition can provide for this task.


It is given two inputs: a query (place name) associated with an image and a set of documents.

table 2 summarizes these input documents.


The summarizer uses the following features, as reported in previous work (Edmundson, 1969;Brandow et al., 1995;Radev et al., 2001;Conroy et al., 2005;Aker and Gaizauskas, 2009; Aker and Gaizauskas, 2010a): • querySimilarity: Sentence similarity to the query (cosine similarity over the vector representation of the sentence and the query).

these sentenceLimit summaries have an average of 221 words, which explains the higher ROUGE recall scores seen in training compared to testing (where a 200 word limit was imposed).


This helps to address the overestimate of h 2 and should therefore lead to a smaller search graph and faster runtime due to its early elimination of dead-ends.

this is not the case for regression results.


• We demonstrate how far one can get with simple, deterministic coreference systems that do not require machine learning or detailed semantic information.

this cluster can later be merged with both singular and plural pronouns.


To evaluate macro content selection, we measured the F 1 score (the harmonic mean of precision and recall) of the set of records chosen with respect to the human-annotated set of records.

the generated text w is a deterministic function of the decisions.


In WEATHERGOV, numeric field values are often rounded or noisily perturbed, so it is difficult to generate precisely matching numbers.

see Figure 1(c) for an example of a scenario.


Following the evaluation methodology of Chen and Mooney (2008), we trained our system on three  Table 1: ROBOCUP results.

we used a modified BLEU score where numbers differing by at most five are treated as equal.


Non-overlapping phrases were given negative labels.

like headlines, captions have to be short and informative.


When it comes to generating creative language (poems, stories, jokes, etc), people have massive advantages over machines: • people can construct grammatical, sensible utterances, • people have a wide range of topics to talk about, and • people experience joy and heart-break.

figure 8 shows our concept of generation as a cascade of weighted fSTs.


Over 94% of word tokens are assigned a syllablestress pattern that matches the pattern transcribed from audio.

there are difficulties: • The test data contains many words that are unknown to the CMU dictionary.


empirical results show significant improvements in alignment quality as well as in translation performance over baselines in a large-scale Chinese-english translation task.


The alignment configuration internal to allmonotone phrase pair blocks is, obviously, monotonic, which is a configuration that is effectively modeled by traditional alignments models.

for translation experiments, we used cdec (Dyer et al., 2010), a fast implementation of hierarchical phrase-based translation models (Chiang, 2005), which represents a state-of-the-art translation system.


Common practice (Koehn et al., 2003) takes a set of word alignment links L and defines the alignment constraints C A so that there is a consistency between the links in the (f .

bayesian methods have been recently developed to induce a grammar directly from an unaligned parallel corpus (blunsom et al., 2008;blunsom et al., 2009).


With this new rule type, G 2 allows phrase/rule concatenation before reordering with another hierarchical rule.

alignment models estimated over the parallel text are used to generate these alignments, but these models are then typically used no further in rule extraction.


We classify translation rules into several reordering patterns, and build a MaxEnt model for each pattern based on various contextual features.

hierarchical rule, consisting of both words and variables, is of great importance for the hPB model.


To train a MaxEnt classifier for glue rules, we extracted 65.8M reordering (monotone and inverse) instances from the training data, using the algorithm described in Xiong et al.

analogously, for the hierarchical rule that contains two variables, the source phrases are classified into 4 patterns, while the target phrases are classified into 14 patterns, as shown in Table 3.


The first rule generates a monotone translation, while the second rule swaps the source phrases covered by X 1 and X 2 on the target side.

the training examples for each MaxEnt model is small and the model maybe unstable.


Source pattern Target pattern terns (Table 2).

the example is not a noun possessive case because the subphrase covered by X 1 is not a noun but a prepositional phrase.


Initial experiments showed effectiveness with the unsigned (absolute value of the) margin, which we use in all experiments.

this problem of domain shift is a pervasive problem in NLP in which any kind of model -a parser, a POS tagger, a sentiment classifier -is tested on data that do not match the training data.


When CWPM is slow to detect a change (over 400 examples), the domain classifier is the clear winner.

horizontal dotted lines indicate the mean of the accuracy/margin before and after the domain shift.


When all of the documents come from one data distribution, like newswire articles, this tends to work well.

sentiment and spam are binary and ACE is multi-class.


The width of each box is n, the number of examples used to estimate P (A) and P (A ) for A ∈ A, where the real interval A corresponds to the vertical span of the box.

in practical settings, we do not know if the data distribution will change, and certainly not when.


In fact, we will show that it is also faster than the HMM, and has lower alignment error rate than the HMM.

we use Gibbs sampling for learning and the HMM Viterbi decoder for testing.


Indeed, it seems intuitive that a good translation pool should be sufficiently varied, as argued in Section 3.1.

after collecting samples for each sentence, the whole list is used to grow the translation pool.


This is the XTAG English Grammar (XTAG-Group, 2001).

the system using factorized grammars shows BLEU improvement in all conditions.


This is why we named the two families Transitive 3 and Intransitive 2 in Figure 4.

according to our own experience, we obtain about 200GB of rules from training data of about 50M words on each side.


Our English-Pashto (E2P) data originates from a two-way collection of spoken dialogues, and consists of two parallel sub-corpora: a directional E2P corpus and a directional Pashto-English (P2E) corpus.

2010, allowing us to compare various strategies under the same conditions.


We present a novel, discriminative sample selection strategy that preferentially selects batches of candidate sentences with constructs that lead to erroneous translations on a held-out development set.

the binary pairwise comparator can be constructed from the permuted classifier outputs.


Words that have non-existent counterparts can be aligned to the NULL word.

first, we deal with different languages.


We tried a variation of the experiment where the CRF method is used to recover *pro* and the pattern matching is used to recover *PRO*, since these represent the best methods for recovering the respective empty categories.

we examine effects that empty categories have on machine translation.


As we have discussed in Section 2, we are interested in recovering dropped pronouns (*pro*) and control construction markers (*PRO*).

we are dealing with a different language and different kinds of empty categories.


the total loss of the method is |MinDCost(R, s)−MinDCost(R , s)|.


In other words, we hypothesize that there may be more training data to learn a phrase's translations in a bilingual corpus than what SMT approaches typically use.

automatic evaluation results are reported in Table 8 for various configurations.


The constraint that the parser keeps one or more fully connected syntactic trees is intended to operationalize this cognitive fact.

human syntactic parsing must be fully connected (or close to it) as people are able to apply vast amounts of real-world knowledge to the process as it proceeds from word-toword (van Gompel and Pickering, 2007).


The results reported for RZL are averaged over 10 runs.

the PUPA score for P F (s) is computed using the parses created for the entire test set by the fully trained parser.


For WSJ, F-score improvement is up to 1.32% (WSJ10), 3.13% (WSJ20), 3.35% (WSJ40) and 3.23% (the entire WSJ).

the only parameter it has is N H but ZL improves parser performance for most N H values.


Confidence-based ZL is based on the idea that sentences for which the fully-trained parser provides parses of similar quality manifest similar syntactic patterns.

the second protocol is the 'less is more protocol of Spitkovsky et al., (2010) in which we parsed each test corpus using a parser that was trained with all training sentences of a bounded length.


In the Dridan (2009) work, parse ranking showed some improvement when morphological information was added to the tags.

in this case, we looked at the edges of the parsing chart.


Since many of the languages that people are building new grammars for are under-resourced, we can't depend on having any external information or NLP tools, and so the methods we examine are purely unsupervised, using nothing more than the grammars them-selves and raw text.

there needs to be both gold and non-gold trees for any sentence to be used in training the discriminative model.


Acc (%) Negation scope identification 83.10 Speculation scope identification 86.41 Table 5: Accuracy (%) of scope identification with golden parse trees and golden cues on the abstracts subcorpus using 10-fold cross-validation It is worth nothing that we adopted the postprocessing algorithm proposed in Section 4.4 to ensure the continuousness of identified scope.

since a cue is deemed inside its scope in the Bioscope corpus, our post-processing algorithm first includes the cue in its scope and then starts to identify the left and the right scope boundaries, respectively.


In the following, we first report the results of cue recognition and then the results of scope identification with automatic cues.

the cue is recast as the predicate and the scope is recast as the arguments of the cue.


Moreover, few if any taggers do any lemma disambiguation; that is partly because one must start with some standard inventory of lemmas, which are not available for most languages, perhaps because the importance of lemma disambiguation has been underestimated.

the accuracy achieved by our M f ull model is, to the best of our knowledge, higher than prior approaches have been able to achieve so far for the problem of combined morphological and lemma disambiguation.


For a given token t i and potential label l k , our goal is to approximate the indicator function g(i, k), which is 1 if the k th label of token t i is present, and 0 otherwise.

instead of having each observation be a token, we now let each observation be a (token, label) pair, but only include those labels that were output by SAMA.


We note in passing that the existence of set of stable "lemma IDs" is an unusual feature of SAMA, which in our opinion ought to be emulated by approaches to lexical analysis in other languages.

this broad coverage comes at a cost; the list of analytic alternatives must include a long Zipfian tail of rare or contextually-implausible analyses, which collectively are correct often enough to make a large contribution to the coverage statistics.


We do not learn separate models for each label.

morphological disambiguation mainly depends on features of nearby words, which help to character-ize how inflected forms of these lemmas might fit into local phrasal structures.


The similarity between two vertices is calculated according to the degree of connectedness among their direct neighbours.

choose an internal node, D k ∈ D curr uniformly.


In both figures, we observe that HRGs outperform the CWU baseline under all parameter combinations.

a number of NLP problems can be cast into a graphbased framework, in which entities are represented as vertices in a graph and relations between them are depicted by weighted or unweighted edges.


Figure 3 shows an example of alignment between the conversation terms and hypothesis terms in Example 2.

an edge between vertices captures a dependency relation and is represented as predicates in the clause representation.


Unlabeled adaptation: Adding cluster-like features Recall that unlabeled adaptation frameworks find the features that "work" across domain.

second, we also show that applying an adaptation algorithm that finds shared representation between domains often impacts the choice in adaptation algorithm that makes use of target labeled data.


First, unlabeled adaptation frameworks are necessary since many features only exist in one domain.

we find that the combined approach often outperforms the individual adaptation approaches.


In order to have a deep analysis, it is necessary to choose specific adaptation algorithms for each aspect of adaptation framework.

it is not clear how these newly added features will impact the behavior of the labeled adaptation frameworks.


This relationship is however not expressed through the tying of the corresponding parameters; instead we let the prediction space guide the convergence of the context space.

the LBL model estimates the n-gram parameters by: In this equation, E is an energy function defined as: where R is the projection matrix introduced above, (v k ) l−n+1≤k≤l−1 are the 1-of-V coding vectors for the history words and v l is the coding vector for w l ; C k ∈ R m×m is a combination matrix and b r and b v denote bias vectors.


We therefore introduce a different initialization strategy, called one vector initialization.

this method fails to deliver any probability estimate for words outside of the prediction vocabulary, meaning that a fall-back strategy needs to be defined for those words.


Apart from the supervised approach, Sun et al.

we use the following feature templates, as are widely used in most segmentation work: Here C refers to a character; n refers to the position index relative to the current character.


One of the main reasons of such performance degradation lies in the model's ability to cope with OOV words.

we propose to use χ 2 statistics together with bootstrapping strategies to build Chinese word segmentation model.


For a word type w 1 , the left-word context of w 1 , L(w 1 ), is defined as the N-dimensional vector whose n-th component is the number of bigrams, i.e., pairs of consecutive tokens (t i-1 , t i ) in the corpus, such that w(t i ) = w 1 and w(t i-1 ) = n. Similarly, we define the right-word context of w 1 , R(w 1 ), as the N-dimensional vector whose n-th component is the number of bigrams (t i , t i+1 ) such that w(t i ) = w 1 and w(t i+1 ) = n. We let L (resp.

for a probabilistic assignment P, we define G(P) to be the weighted average, over all w and all k, of ||L P (w) -µ L k || 2 + ||R P (w) -µ R k || 2 ; the weight used in this average is P wk f(w), just as in the computation of the Gaussian means.


in each iteration, given the assignment A obtained from the previous iteration, one first computes the latent descriptors for all word types as defined above, and then proceeds in the usual way to update cluster centroids and to find a new assignment A to be used in the next iteration.