In fact, this method is commonly used in event ordering (Mani et al., 2003;Lapata and Lascarides, 2004;Boguraev and Ando, 2005).

many segment pairs lack temporal markers and other explicit cues for ordering.


Programming (ILP) We can cast the task of constructing a globally optimal TDAG as an optimization problem.

to the previous approaches, the method is not greedy.


It is also strictly domain-dependent and hence difficult to be adapted to new domains.

although addressing such drawbacks associated with knowledge-based approaches, the latter approach often suffers the data sparseness problem and hence needs a fully annotated corpus in order to reliably estimate an accurate model.


We currently do not apply any special filters to remove non-verbal sounds or background noise (other than laughs) that overlap with speaker turns.

if artificial laughs overlap with a speaker turn (there were only few such instances), the speaker turn is chopped by marking a turn boundary exactly before/after the laughs begin/end.


This can be also be thought of as squeezing together the four outside corners, creat- ing a new cell whose probability is estimated using IBM Model 1.

the inside Viterbi alignment satisfies the ITG constraint, implying only one solid cell in each column and each row.


Many of the features of the AEP prediction model are specifically tuned to the choice of German and English as the source and target languages.

it should be easy to develop new feature sets to deal with other languages or treebanking styles.


Because Kanji comprises ideograms, an individual pronunciation can potentially be represented by more than one character.

if several Kanji strings are related to the same pronunciation of the source word, their meanings will be different and will therefore convey different impressions.


Existing transliteration methods for Chinese (Haizhou et al., 2004;Wan and Verspoor, 1998) aim to spell out foreign names of people and places, and do not model impression.

as exemplified by "Coca-Cola" in Section 1, the impression of words needs to be modeled in the transliteration of proper names, such as companies and products.


A simple implementation is to segment each impression keyword into characters.

because it is difficult for a user to provide an exhaustive list of appropriate keywords and characters, our impression model derives characters that are not included in the impression keywords.


These co-occurrences can potentially be collected from existing language resources, such as corpora in Chinese.

it is desirable to collect an association between a word and a character, not simply their co-occurrence in corpora.


We believe our hand-assigned costs are a reasonable starting point if one knows nothing about the particular pair of languages in question.

one could also train such costs, either from an existing list of known transliterations, or as part of an iterative bootstrapping method as, for example, in Yarowsky and Wicentowski's (2000) work on morphological induction.


Expanding a foreign word to its possible variants in a query has been shown to increase the precision of search results (Abduljaleel and Larkey, 2003).

ooV words in the query are easily recognised based on English rules and an English-Arabic dictionary: capitalised words are marked as nouns, and the remaining words are translated using the dictionary.


We built the Arabic language model using 100 000 words extracted from the TREC collection using the same spell-checker.

we excluded any word that could be a proper noun, to avoid involving foreign words.


This could be due to the limited size of the training corpus.

we expect that improvements to this approach will remain limited due to the fact that many Arabic and foreign words share the same trigrams.


They investigate the impact of different features and data size, and report results significantly better than a simple baseline.

their results vary considerably between the languages and the domains.


Techniques based on machine learning (Zhou et al., 2005;Hao et al., 2005;Bunescu and Mooney, 2006) are expected to alleviate this problem in manually crafted IE.

in most cases, the cost of manually crafting patterns is simply transferred to that for constructing a large amount of training data, which requires tedious amount of manual labor to annotate text.


Their approaches were similar to our approach using PASs derived from full parsing.

one problem with their systems is that they could not treat non-local dependencies such as semantic subjects of gerund constructions (discussed in Section 2), and thus rules acquired from the constructions were partial.


these results included pairs that Reactome missed or those that only cooccurred and were not interacting pairs in the text.


Parsing errors are intrinsic problems to IE methods using parsing.

from Table 3, we can conclude that the key to gaining better accuracy is refining of the method with which the PAS patterns are constructed (there were 46 related FNs) rather than improving parsing (there were 35 FNs).


as enzymes that catalyze certain reactions) depends on their three-dimensional structure.

genes only specify the linear, sequence of the amino acids, and the ribosome (the cell's "protein factory") uses this information to assemble the polypeptide chain.


Ultimately, models which explicitly capture all atoms and their physical interactions are required to study the folding of real proteins.

since such models often require huge computational resources such as supercomputers or distributed systems, novel search strategies and other general properties of the folding problem are usually first studied with coarse-grained, simplified representations, such as the HP model (Lau and Dill, 1989;Dill et al., 1995) used here.


What CKY by itself does not give us is an accurate prediction of the rates that govern the folding process, including misfolding and unfolding events.

we believe that it is possible to obtain this information from the chart by extracting all tree cuts (which corresond to the states of the chain at different stages during the folding process) and calculating folding rates between them.


They guide the search using the performance on parsing (and several other tasks) of the grammar at each stage in the search.

our approach explores the space of grammars by starting with few nonterminals and splitting them.


If the reevaluated candidate is no longer better than the second candidate on the queue, we reinsert it and continue.

if it is still the best on the queue, and it improves the model, we enact the split; otherwise it is discarded.


they have a better performing approximation which is used in their reported score.


The INHERIT model may be regarded as containing all the same rules (see (1)) as the PCFG-LA model.

these rules' probabilities are now collectively determined by a smaller set of shared parameters.


The choice of a particular passpattern, for example, depends on all and only the three nonterminals X, Y, Z.

given sparse training data, sometimes it is advantageous to back off to smaller amounts of contextual information; the nonterminal X or Y might alone be sufficient to predict the passpattern.


Indeed, Table  3 shows that we can improve agreement precision by setting θ agr to the (positive) mean agreement score µ assigned by the SVM agreement-classifier over all references in the given debate 12 .

this comes at the cost of greatly reducing agreement accuracy (development: 64.38%; test: 66.18%) due to lowered recall levels.


We compute transitive closure by using a Union-Find structure, which runs in time O(log * n), which for practical purposes can be considered linear (O(n)) 3 .

when computing the best information gain for a nominal feature, StRip has to make a pass over the data for each value that the feature takes, while RIPPER can split the data into bags and perform the computation in one pass.


StRip's performance is all the more impressive considering the strength of the SVM and RIPPER baselines, which which represent the best runs across the 336 different parameter settings tested for SV M light and 144 different settings tested for RIPPER.

all four of the StRip runs using the full MPQA corpus (we vary the loss ratio for false positive/false negative cost) outperform those baselines.


By carefully selecting the smoothing parameters, the model can preserve dependencies between topic and sentiment words, and is quite capable of distinguishing the positive sentiment of 'unpredictable plot' from the negative sentiment of 'unpredictable steering'.

the model does ignore the ordering of the words, so it will not be able to differentiate the negative phrase 'gone from good to bad' from its exact opposite.


One possible approach is to set the threshold values for frequency in a polar context, max(p(a), n(a)) and for the ratio of appearances in polar contexts among the to-tal appearances, max(p(a),n(a)) f (a) .

the optimum threshold values should depend on the corpus and the initial lexicon.


Summarization of meetings faces many challenges not found in texts, i.e., high word error rates, absence of punctuation, and sometimes lack of grammaticality and coherent ordering.

meetings present a rich source of structural and pragmatic information that makes summarization of multi-party speech quite unique.


the two sentences have contradictory meanings, and it would be unfortunate to increase the score of a peer summary containing the former sentence because the latter is included in some model summaries.


While the contingency counts in Table 2 only hinted a limited benefit of linear-chain features, empirical results show the contraryespecially for order k = 2.

the further increase of k causes overfitting, and skip-chain features seem a better way to capture non-local dependencies while keeping the number of model parameters relatively small.


Incorporating syntactic features into the context has been at the forefront of recent research (Collins et al., 2005;Rosenfeld et al., 2001;Chelba and Jelinek, 2000;Hall and Johnson, 2004).

much of the previous work has focused on English language syntax.


Factoring a word into the semantics-bearing lemma and syntaxbearing morphological tag alleviates the data sparsity problem to some extent.

the number of possible factorizations of n-grams is large.


The space of features spanned by the crossproduct space of words, lemmas, tags, factoredtags and their n-gram can potentially be overwhelming.

not all of these features are equally important and many of the features may not have a significant impact on the word error rate.


Transductive learning, first described by Vapnik (Vapnik, 1998) also describes a setting where both labeled and unlabeled data are used jointly to decide on a label assignment to the unlabeled data points.

the goal here is not to learn a general classification function that can be applied to new test sets multiple times but to achieve a high-quality onetime labeling of a particular data set.


As seen in Table 4, POSset sizes of machine-learned lexicon is a factor of 2 or 3 smaller than that of the baseline lexicons.

recall is better for the baseline lexicons.


Because reviews can be numerous and varying in quality, it is important to rank them to enhance customer experience.

with ranking search results, assessing relevance when ranking reviews is of little importance because reviews are directly associated with the relevant product or service.


Pang and Lee (2005) have studied prediction of product ratings, which may be particularly relevant due to the correlation we find between product rating and the helpfulness of the review (discussed in Section 5).

a user's overall rating for the product is often already available.


The exact scoring approaches developed in commercial systems are often not disclosed.

more recent work on one of the major systems, e-rater 2.0, has focused on systematizing and simplifying the set of features used (Attali and Burstein 2006).


later formulated global inference using integer linear programming, which is the approach that we apply here.

to our work,  operated in the domain of factual information extraction rather than opinion extraction, and assumed that the exact boundaries of entities from the gold standard are known a priori, which may not be available in practice.


Since both features represent a positive sentiment and the bigram matches fewer contexts than the unigram, it is probably sufficient just to have the unigram.

there are many cases where a feature captures a subtlety or non-compositional meaning that a simpler feature does not.


9) as an extension to the Dirichlet prior method ).

we have introduced a multinomial approximation of the Polya distribution.


Table 3 also shows that Polya-CF outperformed CProb when the dataset was ML and CProb was better than Polya-CF in the other cases.

the differences in precision were small.


We actually tried to learn α ω and α µ from the training data by using an EM method (Minka, 2003;Yamamoto and Sadamitsu, 2005).

the estimated parameters were about 0.05, too small for better recommendations.


We see that the very large corpus has reduced the accuracy of frequency weighted Random Indexing.

our two top performers have both substantially increased in accuracy, presenting a 75-100% improvment in performance over FREQ.


We have shown that order-1 semi-Markov conditional random fields are strictly more expressive than order-1 Markov CRFs, and that the added expressivity enables the use of features that lead to improvements on a segmentation task.

markov CRFs can more naturally incorporate certain features that may be useful for modeling sub-chunk phenomena and generalization to unseen chunks.


The results show that 2nd order chains of characters generally obtain the best performance.

the difference in performance between 1st order and 2nd order chains could be considered as statistically insignificant due to the large overlap of the error bars.


Some machine learning algorithms also have been investigated in Chinese NER, including HMM (Yu et al., 1998;, class-based language model (Gao et al., 2005;, RRM (Guo et al., 2005;, etc.

when a machine learning-based NER system is directly employed in a new domain, its performance usually degrades.


date, time, numeral expression) are encoded into pattern-specific class labels aligned with the tokens.

the performance usually becomes unstable when NER models are applied in different domains.


Experimental results show that the performance of the general NER model is significantly enhanced in the first several retraining cycles since more training data are used.

when the general training data set size is more than 2.4M, the performance enhancement is very slight.


More training data are used, higher NER performance can be achieved.

it is difficult to significantly enhance the performance when the training data size is above a certain threshold.


Domain-specific models usually could achieve a higher performance in its corresponding domain after being trained with a smaller amount of domainspecific annotated data (see Table 2 in Section 3.2).

the performance stability of domain-specific NER model is poor across different domains.


One is chronological ordering (Barzilay et al., 2002;Bollegala et al., 2005), which is based on time-related features of the documents.

such temporal features may be not available in all cases.


Unlike in MO, selection of the next sentence here is based on the most recent one.


In fact, it can be seen as merge of the original c 0 and c k , and in this sense the updated c 0 represents the history of selections.

to the MO algorithm, the ordering algorithm here (HO) uses immediate back-front co-occurrence, while the MO algorithm uses relative back-front locations.


The synset having higher overlapping word counts (or weights) is selected for a particular test example.

for TSWEB and TSBNC the better results have been obtained using occurrences (the weights are only used to order the words of the vector).


Therefore, it is desirable to crawl the web and to develop specific search engines for NLP applications (Cafarella and Etzioni, 2005).

considering that great efforts are taken in commercial search engines to maintain quality of crawling and indexing, especially against spammers, it is still important to pursue the possibility of using the current search engines for NLP applications.


Detailed analyses of the results revealed that word groups such as bacteria and diseases are clustered correctly.

word groups such as computers (in which homepage, server and client are included) are not well clustered: these words tend to be polysemic, which causes difficulty.


This phenomenon depends on how a search engine processes AND operator, and results in unstable values for the PMI.

our method by the chi-square uses a co-occurrence matrix as a contingency table.


Co-occurrence analysis has been used to determine related words or terms in many NLP-related applications such as query expansion in Information Retrieval (IR).

related words are usually determined with respect to a single word, without relevant information for its application context.


Query expansion thus aims to improve query expression by adding related terms to the query.

the effect of query expansion is strongly determined by the term relations used (Peat and Willett, 1991).


Another often used resource is associative relations extracted from co-occurrences: two terms that co-occur frequently are thought to be associated to each other (Jing and Croft, 1994).

co-occurrence relations are noisy: Frequently co-occurring terms are not necessarily related.


One naturally would suggest that compound terms can be used for this purpose.

for many queries, it is difficult to form a legitimate compound term.


To some extent, the proposed approach is also related to (Schütze and Pedersen, 1997), which calculate term similarity according to the words appearing in the same context, or to second-order co-occurrences.

a key difference is that (Schütze and Pedersen, 1997) consider only separate context words, while we consider multiple context words together.


could be built as a traditional bigram model.

this is not a good approach for IR because two related terms do not necessarily co-occur side by side.


This shows that the CDQE method does not increase recall to the detriment of precision, but both of them.

cIQE increases precision at all but 0.0 recall points: the precision at the 0.0 recall point is 0.6565 for cIQE and 0.6699 for UM.


Among these methods, supervised learning is usually more preferred when a large amount of la-beled training data is available.

it is time-consuming and labor-intensive to manually tag a large amount of training data.


Most of semi-supervised methods employ the bootstrapping framework, which only need to pre-define some initial seeds for any particular relation, and then bootstrap from the seeds to acquire the relation.

it is often quite difficult to enumerate all class labels in the initial seeds and decide an "optimal" number of them.


(2004)'s method is to use a hierarchical clustering method to cluster pairs of named entities according to the similarity of context words intervening between the named entities.

the drawback of hierarchical clustering is that it required providing cluster number by users.


Some of these synsets are expressed by very general nouns such as "biont", "benthos", "whole", and "nothing".

others undoubtedly refer to other supersenses, for which they provide the label, such as "food", "person", "plant" or "animal".


After merging together the 333 and 444 numbers, B ij will recompute the new inter-cluster compatibility as 0.51, the average of the inter-cluster edges.

the cluster compatibility function C ij can represent the fact that three numbers with different area codes are to be merged, and can penalize their compatibility accordingly.


It can also associate a weight with the fact that several fields overlap (e.g., the chances that a cluster has two first names, two last names and two cities).

the binary classifier only examines pairs of fields in isolation and averages these probabilities with other edges.


We present two methods for creating and applying transliteration models.

to most previous transliteration approaches, our models are discriminative.


At first glance, k-best lists may seem like they should outperform sampling, because in effect they are the k best samples.

there are several important reasons why one might prefer sampling.


Thus, the aims of his work are similar to ours.

he is not concerned with the more fine-grained elements, and also uses a different machinery.


Whereas HMMs are generative models, CRFs are discriminative models that can incorporate rich features.

other approaches to text segmentation have also been pursued.


There exist clear recommendations on structuring medical reports, such as E2184-02 (ASTM International, 2002).

actual medical reports still vary greatly with regard to their structure.


Again, naive computation of (9) is intractable.

the max-product variant of loopy belief propagation can be applied to approximately find the MAP assignment of y (maxproduct can be seen as a generalization of the wellknown Viterbi algorithm to graphical models).


This information can be described in a DTD as book ← title author + price .

as shown in Example 1, regexes for information extraction rely on more complex constructs.


Typically, the restrictions involve either disallowing or limiting the use of Kleene disclosure and disjunction operations.


A simple pattern for this task is: "one or more capitalized words followed by a version number", represented as When applied to a collection of University web pages, we discovered that R 5 identified correct instances such as Netscape 2.0, Windows 2000 and Installation Designer v1.1.

r 5 also extracted incorrect instances such as course numbers (e.g.


Such features would undoubtedly do a better job predicting the rationales and hence increasing equation 1.

crucially, our true goal is not to predict the rationales but to recover the classifier parameters θ.


We show that the proposed measure is optimal for constructing the core cluster among documents of equal length.

our method is not useful in a setup where some long documents have a topical portion: such documents should be considered on-topic, but their heavy tail of background words overcomes the topical words' influence.


If |G| |R| 0, and if π 0, then topical words would tend to appear more often than non-topical words.

we cannot simply base our conclusions on word counts, as some words are naturally more frequent than others (in general English).


Intuitively, this means that the new feature could no longer be used to discriminate between sentences sampled from P i+1 and real sentences.

setting 1 i rej p + in this manner may violate the constraints 18associated with the features already existing in the model, thus hampering the model's performance.


Given that in each iteration we generate 12,045 sentences, and that in the n'th iteration each sentence has to be classified by n features, this gives a total of roughly 10 7 10 ⋅ classifications after 21 iterations.

using rejection sampling, we used only 7 6.7 10 ⋅ classifications in total -a difference of over three orders of magnitude.


(1999) define the similarity-weighted probability, Pr SIM , to be: where Sim(v ′ , v) returns a real-valued similarity between two verbs v ′ and v (normalized over all pair similarities in the sum).

erk (2007) generalizes by substituting similar arguments, while Wang et al.


In a similar vein, Swanson and Gordon (2006) reported that parse tree path features extracted from a rule-based dependency parser are much less reliable than those from a modern constituent parser.

we recently carried out a detailed comparison (Johansson and Nugues, 2008b) between constituent-based and dependency-based SRL systems for FrameNet, in which the results of the two types of systems where almost equivalent when using modern statistical dependency parsers.


For parsers that consider features of single links only, the Chu-Liu/Edmonds algorithm can be used instead.

this algorithm cannot be generalized to the second-order setting - McDonald and Pereira (2006) proved that this problem is NP-hard, and described an approximate greedy search algorithm.


To compare our results with previously published results in SRL, we carried out an experiment comparing our system to the top system (Punyakanok et al., 2008) in the CoNLL-2005 Shared Task.

comparison is nontrivial since the output of the CoNLL-2005 systems was a set of labeled segments, while the CoNLL-2008 systems (including ours) produced labeled semantic dependency links.


To handle this situation fairly to both types of systems, we carried out a two-way evaluation: conversion of dependencies to segments for the dependency-based system, and head-finding heuristics for segmentbased systems.

the latter is difficult since no structure is available inside segments, and we had to resort to computing upper-bound results using gold-standard input; despite this, the dependencybased system clearly outperformed the upper bound of the performance of the segment-based system.


We note that inference is particularly helpful with rarely mentioned instances.

inference can lead to errors when the proof tree contains joins on generic terms (e.g., "company") or common extraction errors (e.g., "LLC" as a company name).


To see why, we re-examine the proof: the large factor comes from assuming that all of R's first arguments which meet the PF definition are associated with exactly K min distinct second arguments.

in our corpus 83% of first arguments are associated with only one second argument.


HOLMES is also related to open-domain questionanswering systems such as Mulder (Kwok et al., 2001), AskMSR (Brill et al., 2002), and others (Harabagiu et al., 2000;Brill et al., 2001).

these Q/A systems attempt to find individual documents or sentences containing the answer.


They often perform deep analysis on promising texts, and back off to shallower, less reliable methods if those fail.

hOLMES utilizes TI and attempts to combine information from multiple different sentences in a scalable way.


Typically, a translation rule consists of a source-side and a target-side.

the source-side of a rule usually corresponds to multiple target-sides in multiple rules.


The advantage of the MERS model is that it uses rich contextual information to compute posterior probability for e given T .

the translation probabilities and lexical weights in Lynx ignore these information.


Considering the following examples: The syntactic tree of the Chinese phrase " " is shown in Figure 6.

there are two TATs which can be applied to the source tree, as shown in Figure 7.


2, the value of distortion score peaks at d=1, i.e., the monotonic alignment, and decays for non-monotonic alignments depending on how far it diverges from the monotonic alignment.

following Och and Ney (2003), we use a fixed value p 0 for the probability of jumping to a null state, which can be optimized on held-out data, and the overall distortion model becomes 0 0 if state ( | , ) (1 ) ( | , ) otherwise Given an HMM, the Viterbi alignment algorithm can be applied to find the best alignment between the backbone and the hypothesis,   the alignment produced by the algorithm cannot be used directly to build a confusion network.


It uses a similarity model for synonym matching and a distortion model for word ordering.

to previous methods, the similarity model explicitly incorporates both semantic and surface word similarity, which is critical to monolingual word alignment, and a smoothed distance-based distortion model is used to model the first-order dependency of word ordering, which is shown to be better than simpler approaches.


As shown in Figure 2, they simply strip off the appropriate words from each state, collapsing dynamic programming items which are identical from the standpoint of their left-to-right combination in the lower order language model.

having only orderbased projections is very limiting.


It is also worth noting that the number of search errors incurred in the coarse-to-fine approach can be dramatically reduced (at the cost of decoding time) by increasing the pruning thresholds.

the fortuitous nature of coarse-tofine search errors seems to be a substantial and desirable effect.


(Mao et al, 2007) used a sequential CRFs regression model to measure the polarity of a sentence in order to determine the sentiment flow of the authors in reviews.

this method must manually select a word set for constraints, where each selected word achieved the highest correlation with the sentiment.


In that work, we found that the projection of annotations across parallel texts can be successfully used to build a corpus annotated for subjectivity in the target language.

parallel texts are not always available for a given language pair.


As Joachims (2002) points out, the above type of problem is NP-Hard.

an approximate solution to finding g ik can be obtained by solving the following SVM optimization problem: In the second approach to ranking emotions, we use regression to model f i directly.


If max-product BP converges, we may simply output each variable's favorite value (according to its belief), if unique.

max-product BP tends to be unstable on loopy graphs, and we may not wish to wait for full convergence in any case.


m1 is usually not preferred in practice (Goodman, 1997).


In this paper, for simplicity reasons we do not consider the effect of for-statement implementations on the optimal binarization.

it is well known that reducing the number of constituents produced in parsing can greatly improve CKY parsing efficiency.


For example, in Figure 2, if we use left binarization, then [A B]:[0, 2] can be shared to generate both X :[0, 4] and Y :[0, 3], in which we can save one IC overall.

if right binarization is used, there will be no common ICs to share in the generation steps of X :[0, 4] and Y :[0, 3], and overall there are one more IC generated.


When combined with exact inference algorithms, like the iterative CKY (Tsuruoka and Tsujii, 2004), the accuracy will be the same.

if combined with other inexact pruning techniques like beam-pruning (Goodman, 1997) or coarse-to-fine parsing (Charniak et al., 2006), binarization may interact with those pruning methods in a complicated way to affect parsing accuracy.


So while for children or less educated adults these constructions might pose difficulties, they were favored by our assessors.

the average parse tree height negatively correlated with readability as expected, but surprisingly the correlation is very weak (-0.06).


To alleviate this problem, an obvious idea is to extract rules from k-best parses instead.

a k-best list, with its limited scope, has too few variations and too many redundancies (Huang, 2008).


The basic idea is to decompose the source (Chinese) parse into a series of tree fragments, each of which will form a rule with its corresponding English translation.

not every fragmentation can be used for rule extraction, since it may or may not respect the alignment and reordering between the two languages.


In tree-based extraction, for each sentence pair, each rule extracted naturally has a count of one, which will be used in maximum-likelihood estimation of rule probabilities.

a forest is an implicit collection of many more trees, each of which, when enumerated, has its own probability accumulated from of the parse hyperedges involved.


Their approach was globally optimised and discriminative trained.

a language model, an information source known to be crucial for obtaining good performance in SMT, was notably omitted.


These n-best lists are produced using algorithms tuned to remove multiple derivations of the same translation (which have previously been seen as undesirable).

it would be simple to extend our sampling based decoding algorithm to calculate the MBR estimate using BLEU, in theory providing a lower variance estimate than attained with n-best lists.


Since its introduction by Och (2003), minimum error rate training (MERT) has been widely adopted for training statistical machine translation (MT) systems.

mERT is limited in the number of feature weights that it can optimize reliably, with folk estimates of the limit ranging from 15 to 30 features.


This feature can be viewed as a soft syntactic constraint: it biases the model toward translations that respect syntactic structure, but does not force it to use them.

this more syntactically aware model, when tested in Chinese-English translation, did not improve translation performance.


On one hand, each conversational system has a domain model, which is the knowledge representation about its domain such as the types of objects and their properties and relations.

there are available resources about domain independent lexical knowledge (e.g., WordNet (Fellbaum, 1998)).


Now, the quantity AHT (T, r) is logged as part of the email management system and corresponds to the time taken to respond to a customer's email.

we do not have access to AHT (T , r) for any T = T .


Furthermore document-level entities can include more than one name string.

once a document-level entity has been clustered, it remains linked to entities that were a part of that initial clustering.


As we observed, the exact match baseline has fairly high accuracy but is obviously also too aggressive of a strategy.

for certain very famous global entities, any reference to the name (especially in corpora made of primarily news text) is likely to be a reference to a single global entity.


'Lady Thatcher' with 'Margaret Thatcher'); and linking entities despite spelling mistakes in a document (e.g linking 'Avenajado' with 'Robert Aventajado').

as we have already seen, the cross-document co-reference system does make mistakes and these mistakes can propagate to the within-document output.


We have noticed that the performance of the crossdocument co-reference system on organizations lags behind the performance of the system on people.

for LEDR, the extraction system's performance is quite similar between the two entity classes.


The ACE guidelines (LDC, 2008a) suggest that this distinction can be difficult to make, and in fact have a lengthy set of rules for classifying such cases.

these rules can seem unintuitive, and may be difficult for machines to learn.


This is also the only algorithm in the name variation component that scales quadratically with the number of name strings.

each calculation is independent, and could be done simultaneously (with enough machines).


Since different feature-sets and different ML approaches are used and combined for each experiment, it is not possible to present the number of features used in each experiment in Table 4.

table 5 shows the number of features and the ML approach used for each genre and NE class.


Moreover the quality of the performance of the different feature extraction tools such as AMIRA (for POS tagging and BPC) and MADA (for the morphological features) are optimized for NW data genres, thereby yielding suboptimal performance on the WL genre, leading to more noise than signal for training.

comparing relative performance on this genre, we see a significant jump from the most frequent baseline FreqBaseline (F β=1 =27.66) to the best baseline MLBaseline CRF (F β=1 =55.32).


In this paper, we experimented with one empirical and two well-known unsupervised statistical machine learning techniques: k-means and EM and evaluated their performance in generating topicoriented summaries.

the performance of these approaches depends entirely on the feature set used and the weighting of these features.


Previous phrase alignment work has primarily mitigated this tendency by constraining the inference procedure, for example with word alignments and linguistic features (Birch et al., 2006), or by disallowing large phrase pairs using a noncompositional constraint (Cherry and Lin, 2007;Zhang et al., 2008).

the problem lies with the model, and therefore should be corrected in the model, rather than the inference procedure.


(2005) has made some preliminary attempt on the idea of hierarchical semantic role labeling.

without considerations on how to utilize the characteristics of linguistically similar semantic roles, the purpose of the hierarchical system is to simplify the classification process to make it less time consuming.


Previous SRC systems treat all the tags equally, and view the SRC as a multi-category classification task.

we have different opinions of the traditional architecture.


Previous semantic role classifiers always did the classification problem in one-step.


The proposal distri-bution does not affect the underlying probabilistic model -Metropolis-Hastings will converge to the same underlying distribution for any non-degenerate proposal.

a well-chosen proposal distribution can substantially speed convergence.


In all but the simplest models there is no known closed form for the posterior distribution.

the Bayesian literature describes a number of methods for approximating the posterior P(θ | d).


In theory, the Gibbs samplers produce streams of samples that eventually converge on the true posterior distribution, while the Variational Bayes (VB) estimator only produces an approximation to the posterior.

as the size of the training data distribution increases the likelihood function and therefore the posterior distribution becomes increasingly peaked, so one would expect this variational approximation to become increasingly accurate.


This is not surprising, as the α and α specify how likely the samplers are to consider novel tags, and therefore directly influence the sampler's mobility.

in our experiments the best results are obtained in most settings with small values for α and α , usually between 0.1 and 0.0001.


representations, training a model on it, and using the resulting model as a new objective function.

it turns out that after a single round, improved weights due to additional training do not change the feature representation; the inference process does not yield a different outcome.


Table 6: Final results on EPPS English-Spanish, constrained triplet models, 10 EM iterations, compared to standard IBM model 1. eration using less than 2 GB of memory.

this shows that triggers outside the immediate context help overall translation quality.


Research on non-task-oriented dialogue systems like casual conversation dialogue systems ("chatbots") is on the other hand not very common, perhaps due to the many amateurs who try to build naturally talking systems using sometimes very clever, but rather unscientific methods although there are systems with chatting abilities as (Bickmore and Cassell, 2001) but concentrate on applying strategies to casual conversation rather than their automatic generation of those conversations.

we believe that the main reason is that an unrestricted domain is disproportionately difficult compared to the possible use such a system could have.


The proposition templates are applied in a predetermined order: for example, first a template "(noun) (wa) (adjective)" is used; next a template "(noun) (ga) (adjective)" is used.

since the generated proposition is not always a natural statement, the system uses exact matching searches of the whole phrases in a search engine to check the naturalness of each proposition.


In most NLP problems, the data is present in structured forms, like strings or trees, and this structural information can be effectively passed to a kernel-based learning algorithm using an appropriate kernel, like a string kernel (Lodhi et al., 2002) or a tree kernel (Collins and Duffy, 2001).

feature-based methods require reducing the data to a pre-defined set of features often leading to some loss of the useful structural information present in the data.


it is not obvious what the implicit features are and the authors do not describe it either.

our dependency-based word subsequence kernel, which also computes similarity between two dependency trees, is very transparent with the implicit features being simply the dependency paths.


In the query expansion viewpoint, an attempt to identify and decrease the proportion of unnecessary translations in a translation model may produce an effect of "selective" implicit query expansion and result in improved retrieval.

prior work on translation-based Q&A retrieval does not recognize this issue and uses the translation model as it is; essentially no attention seems to have been paid to improving the performance of the translation-based approach by enhancing the quality of translation models.


In terms of weighting scheme, the TextRank approach, which is more "strict" than tf-idf in eliminating unimportant words, has led comparatively higher retrieval performances on all levels of removal quantity when the translation model has been trained from the "noisy" (Q Q) corpus.

the "less strict" tf-idf approach has led better performances when the translation model has been trained from the "less noisy" (Q A) corpus.


Since co-occurrence is a symmetric relation, it suffices to compute half of the matrix.

for conceptual clarity and to generalize to instances where the relation may not be symmetric, the algorithm computes the entire matrix.


On such tasks, feature selection algorithms based on feature-class correlation have been very successful.

in the current problem, which we call 'semantic classification', there seem to be a fixed number of domain specific operative words such as 'grant', 'deny', 'moot', 'strike', etc., which, almost entirely decide the class of the docket entry, irrespective of the existence of other highly correlated features.


In this example, although 'deny' is rightly assigned a positive weight and 'moot' is rightly assigned a negative weight, when both features co-occur in a docket entry (as in 'deny as moot'), it makes the label negative.

8 the combined weight of the linear SVM is positive since the absolute value of the weight assigned to 'deny' is higher than that of 'moot', resulting in a net positive score.


As we have discussed earlier, this docket entry belongs to the OSJ category since it contains at least one OSJ event.

we see that the negative weights assigned by the SVM to the second and third sentences result in an overall negative classification.


Our intuition was that a decision tree makes a categorical decision at each node in the tree, hence it could capture the binary-switch like behavior of features.

the performance of the decision tree is found to be statistically indistinguishable from the linear SVM as shown in entry 4 of table 5.


So far, the classifiers we considered received a performance boost by piggybacking on the human selected features.

they did not take into account the polarity of these features.


The L 1 -regularized logistic regression was comparable to the SVM with linear kernel in this experiment.

the presented model presents the advantage that it can reduce the number of active features (features with non-zero weights assigned); the L 1 regularization can remove 74%, 48%, and 82% of substitution rules in each dataset.


For example, by discarding LUs occurring less than 200 times in the corpus, we obtain a +0.12 improvement in accuracy, but the coverage decreases to 57%.

uncovered LUs are also the most rare ones and their relevance in an application may be negligible.


We will be modelling this as another kind of MA, interaction-explicit MA, since the user needs to indicate explicitly that he wants a given suffix to be replaced, in contrast to the non-explicit positioning MA.

if the underlying MT engine providing the suffixes is powerful enough, the user would quickly realise that performing a MA is less costly that introducing a whole new word, and would take advantage of this fact by systematically clicking before introducing any new word.


This yielded a further average improvement in WSR of about 16% (25% relative improvement) when considering a maximum of 5 explicit MAs.

relative improvement in WSR and uMAR increase drop significantly when increasing the maximum allowed amount of explicit MAs from 1 to 5.


It is well known that richer context representation gives rise to better parsing performance (Johnson, 1998).

the need for tractability does not allow much internal information to be used to represent a hypothesis.


One way to estimate the tag pair generative probability Pr( , ′) is to manually align nodes between parallel trees, and use the manually aligned trees as the training data for maximum likelihood estimation.

this is a time-consuming and errorprone procedure.


For example, the compound noun, ' (corporate buyout)' contains an event noun ' (buyout)' and its accusative, ' (corporate).'

compound nouns provide no information about syntactic dependency or about case markers, so it is difficult to specify the predicate-argument structure.


This means players make a similar number of guesses even for difficult initial letters.

the distribution of initial letters for Free Association data reflects the relatively frequency of initial letters in English.


The quality of data for hard letters is considerably worse than that for easy letters.

to Categorilla and even Categodzilla, we found that the Free Association data was quite clean.


For example, for the word "crime", WordNet has as hyponyms "burglary" and "fraud".

it doesn't have "arson", "homicide", or "murder", which are among the 871 new pairs.


For example, the category "Things that accumulate in your body" is both easier to think of answers for and probably collects more useful data.

automatically creating categories with the right level of specificity is not a trivial task; our initial experiments suggested that it is easy to generate too much context, creating an uninteresting category.


Representative of each method, MSTParser and MaltParser gave comparable accuracies in the CoNLL-X shared task (Buchholz and Marsi, 2006).

they make different types of errors, which can be seen as a reflection of their theoretical differences (McDonald and Nivre, 2007).


In fact, due to parser's performance and word alignment ac-curacies, the statistics we collected from the GALE dataset, containing 10 million sentence-pairs, show that the children in the subtree VP(PP,VP) is translated monotonically 126310 times, while reordered of only 22144 times.

the hand-aligned data support the swap for 1245 times, and monotonically for only 168 times.


Due to its high coverage, WT assigns labels to a larger number of the instance in WN-gold than any other method.

the average rank of the correct class assignment is lower, resulting is  lower MRR scores compared to Adsorption.


If there are no a priori preferences for cluster exemplars, the preferences are set to the median similarity (which can be thought of as the 'knee' of the objective function graph vs. number of clusters), and exemplars emerge from the message passing procedure.

shorter RP are more likely to contain base forms of relations (because longer phrases likely contain additional words specific to the sentence).


Results: Discarding clusters below a certain size had no significant effect on precision.

this step is still necessary for bootstrapping RE, since machine learning approaches require a sufficient number of positive examples to train the extractor.


However, this step is still necessary for bootstrapping RE, since machine learning approaches require a sufficient number of positive examples to train the extractor.

(Yakushiji et al., 2005) 33.70 33.10 33.40 (Mitsumori et al., 2006) 54.20 42.60 47.70 (Erkan et al., 2007 59.59 60.68 59.96 our results confirm the observation that frequently co-occurring pairs of entities are likely to stand in a fixed relation.


As expected, RD alone does not match combined precision and recall of state-of-the-art supervised systems.

we show better performance than expected.


We believe that it is important for the research community to continue to invest in building better resources in "source" languages, as it looks the most promising approach.

using a propagation approach can definitely help bootstrap the process.


The baseline system attempts to translate the first two phonetic characters as "wheat Georgia," whereas the other system simply deletes them.

the second sentence shows how word deletion can sacrifice adequacy for the sake of fluency, and the third sentence shows that sometimes word deletion removes words that could have been translated well (as seen in the baseline translation).


Statistical language processing systems for speech recognition, machine translation or parsing typically employ the Maximum A Posteriori (MAP) decision rule which optimizes the 0-1 loss function.

these systems are evaluated using metrics based on string-edit distance (Word Error Rate), ngram overlap (BLEU score (Papineni et al., 2001)), or precision/recall relative to human annotations.


Because a lattice may represent a number of candidates exponential in the size of its state set, it is often impractical to compute the MBR decoder (Equation 1) directly.

if we can express the gain function G as a sum of local gain functions g i , then we now show that Equation 1 can be refactored and the MBR decoder can be computed efficiently.


Relative to the N -best list, the translation lattice provides a better estimate of the expected BLEU score.

there are few hypotheses outside the 1000-best list which are selected by lattice MBR.


The results (Table 5) show that on aren, there is no degradation if we limit the maximum order of the n-grams to 3.

on zhen/enzh, there is improvement by BLEU(%) Max n-gram order aren zhen enzh 1 38.7 26.8 40.0 2 44.1 27.4 42.2 3 44.9 28.0 42.4 4 44.9 28.5 42.6 considering 4-grams.


This is a significant development in that the MBR decoder operates over a very large number of translations.

the current N -best implementation of MBR can be scaled to, at most, a few thousands of hypotheses.


This way, phrase pair extraction goes handin-hand with estimating the probabilities.

in practice, due to the huge number of possible phrase pairs, this task is rather challenging, both computationally and statistically.


They formulate a joint phrase-based model in which a source-target sentence pair is generated jointly.

the huge number of possible phrase-alignments prohibits scaling up the estimation by Expectation-Maximization (EM) (Dempster et al., 1977) to large corpora.


For example, one way to binarize the permutation {2, 1, 3, 4} is to introduce a proper split into {2, 1; 3, 4}, then recursively another proper split of {2, 1} into {2; 1} and {3, 4} into {3; 4}.

the permutation {2, 4, 1, 3} is non-binarizable.


In this work we also start out from a generative model with latent segmentation variables.

we find out that concentrating the learning effort on smoothing is crucial for good performance.


Given a small number of coreference-annotated documents and a large number of unlabeled documents, these weakly supervised learners aim to incrementally augment the labeled data by iteratively training a classifier 1 on the labeled data and using it to label mention pairs randomly drawn from the unlabeled documents as COREFERENT or NOT COREFERENT.

classifying mention pairs using such iterative approaches is undesirable for coreference resolution: since the non-coreferent mention pairs significantly outnumber their coreferent counterparts, the resulting classifiers generally have an increasing tendency to (mis)label a pair as non-coreferent as bootstrapping progresses (see Ng and Cardie (2003)).


A natural way to extend these unsupervised coreference models is to incorporate additional linguistic knowledge sources, such as those employed by our fully supervised resolver.

feature engineering is in general more difficult for generative models than for discriminative models, as the former typically require non-overlapping features.


Relations that have been exploited in supervised coreference resolution include transitivity (McCallum & Wellner, 2005) and anaphoricity (Denis & Baldridge, 2007).

there is little work to date on joint inference for unsupervised resolution.


In the model above, only the best annotationẑ produced by upstream stages is used for determining the optimal outputŷ.

,ẑ may be an incorrect annotation, while the correct annotation may be ignored because it was assigned a lower confidence value.


This relationship is use-ful, since an agent of the predicate thought is likely to be a person entity.

the nodes sailors and thought are adjacent in the dependency tree of the sentence.


In particular, it tells us nothing about phrases other than NPs.

for NER, we see that both self-training and learning with hints improve over the baseline.


This is related to joint inference (Daumé III et al., 2006).

we do not require that that a single data set be labeled for multiple tasks.


A common technique for learning such experts in the Weighted Majority algorithm (Littlestone and Warmuth, 1994), which weighs a mixture of experts (classifiers).

since we require a hard assignment -pick a single shared parameter set s -rather than a mixture, the algorithm reduces to picking the classifier s with the fewest mistakes in predicting domain d. This requires tracking the number of mistakes made by each shared classifier on each domain once a label is revealed.


These methods largely require batch learning, unlabeled target data, or available source data at adaptation.

our algorithms operate purely online and can be applied when no target data is available.


Starting with 5,947 different types of relations, transitive rules increase the dataset to approximately 12,000.

this increase wasn't enough to be effective in global reasoning.


(2006) used these latter relations in their work.

we also add new time-time links that are deduced from the logical time intervals that they describe.


Both before and after also showed increases in precision and recall in the three-way evaluation.

unknown did not parallel this improvement, nor are the increases as dramatic as in the twoway evaluation.


Chinese is a language that does not have morphological tense markers that provide explicit grammaticalization of the temporal location of situations (events or states).

in many NLP applications such as Machine Translation, Information Extraction and Question Answering, it is desirable to make the temporal location of the situations explicit.


Negation signals in the BioScope corpus always have one consecutive block of scope tokens, including the signal token itself.

the scope finding classifier can make predictions that result in nonconsecutive blocks of scope tokens: we observed that 54% of scope blocks predicted by the system given gold standard negation signals are nonconsecutive.


As a consequence, the optimization procedure may stop in a poor local optimum.

it is difficult to compute a direction that decorrelates two or more correlated feature functions.


For instance, the syntactic approach of Marcu et al.

(2006) can learn unlexicalized rules that insert function words in isolation, such as: as discussed in (Wang, Knight & Marcu, 2007), joint modeling of structure and lexical choice can exacerbate data sparsity, a problem that they attempt to address by tree binarization.


Furthermore, the most common deletion is of quotation marks, which is incorrect in most cases, even though such deletion is evidenced in the training corpus 5 .

the next most common deletions "I" and "it" are linguistically well grounded, since Spanish often drops pronouns.


A particular language pair could have alignments that are very unsuited to the stochastic assumptions of the IBM or HMM alignment models.

manually aligning 110 language pairs is impractical.


The sentences with large rank scores are chosen into the summary.

the model makes uniform use of the sentences in different documents, i.e.


Because the work by Murray and Renals used the same dataset, we can compare our scores directly.

rambow carried out summarization work on a different, unavailable email corpus, and so we re-implemented their summarization system for our current email data.


Therefore model parameters can be directly estimated from the training corpus by counting.

in our task, the correct correspondence between NL words and MR structures is unknown.


Model III, as an interpolation of the above two models, achieves a much better F-measure on GEO-QUERY corpus.

it is shown to be less effective on ROBOCUP corpus.


each of these systems has pursued alignment in idiosyncratic and poorly-documented ways, often using proprietary data, making comparisons and further development difficult.


The low recall figures are particularly noteworthy.

a partial explanation is readily available: by design, the Stanford system ignores punctuation.


12 Because punctuation tokens constitute about 15% of the aligned pairs in the MSR data, this sharply reduces measured recall.

since punctuation matters little in inference, such recall errors probably should be forgiven.


Thus, these knowledge was integrated in the representation level, and then the joint probabilities of words and corresponding SuperARVs were estimated.

in the class-based language models, words are taken as the model units, while other units smaller or larger than words are unfeasible for modeling simultaneously, such as the Chinese characters for Chinese names.


Complementary to our work, this technique also utilizes segmentation and metadata information.

our model enables the simultaneous use of all metadata attributes by combining features derived from different partitions of the training documents.


In theory, n-gram weighting can be applied to any smoothing algorithm based on counts.

because many of these algorithms assume integer counts, we will apply the weighting factors to the smoothed counts, instead.


The interpolated model with n-gram weighting achieves perplexity improvements roughly additive of the reductions obtained with the individual models.

the 1.0% WER drop for the interpolated model significantly exceeds the sum of the individual reductions.


In terms of WER, the Random feature again shows no effect on the baseline WER of 33.7%.

to our surprise, the use of the simple log(c) feature achieves nearly the same WER improvement as the best segmentation-based feature, whereas the more sophisticated features computed from HMM-LDA labels only obtain half of the reduction even though they have the best perplexities.


As expected, the use of more sophisticated interpolation techniques decreases the perplexity and WER reductions achieved by n-gram weighting by roughly half for a variety of feature combinations.

all improvements remain statistically significant.


As expected, GLI outperforms both LI and CM.

whereas LI and CM essentially converge in test set perplexity with only 100 words of devel- opment data, it takes about 500 words before GLI converges due to the increased number of parameters.


A manual approach might take the category NP and subdivide it into one subcategory NPˆS for subjects and another subcategory NPˆVP for objects (Johnson, 1998;Klein and Manning, 2003).

rather than devising linguistically motivated features or splits, latent variable parsing takes a fully automated approach, in which each symbol is split into unconstrained subcategories.


(2006), where categories were repeatedly split and some splits were re-merged if the gains were too small.

while the grammars are indeed compact at the (sub-)category level, they are still dense at the production level, which we address here.


Since it can occur in subject and object position, the production NP → it has remained unsplit.

in a single-scale grammar, two productions NP 0 → it and NP 1 → it would have been necessary.


The original unsplit production (at top) would naively be split into a tree of many subproductions (downward in the diagram) as the grammar categories are incrementally split.

it may be that many of the fully refined productions share the same weights.


It is possible to directly define a derivational semantics for multi-scale grammars which does not appeal to the underlying single scale grammar.

in the present work, we use our multiscale grammars only to compute expectations of the underlying grammars in an efficient, implicit way.


Head word alignment features: When considering a node pair (n, n ), especially one which dominates a large area, the above measures treat all spanned words as equally important.

lexical heads are generally more representative than other spanned words.


Then, we simplify (2) by fixing the alignments a 0 : This optimization has no latent variables and is therefore convex and straightforward.

while we did use this as a rapid training procedure during development, fixing the alignments a priori is both unsatisfying and also less effective than a procedure which allows the alignments a to adapt during training.


Although a real estimate of the impact of a parser design decision in this scenario can only be gauged from the quality of the translations produced, it is impractical to create such estimates for each design decision.

estimates using the solution proposed in this paper can be obtained fast, before submitting the parser output to a costly training procedure.


Note that this feature is meant as a refinement of the previous UNK feature, in the sense that perplexity numbers are meant to signal the occurrence of unknown words, as well as rare (from the training data perspective) words.

the correlation we observe for this feature is similar to the correlation observed for the UNK feature, which seems to suggest that the smoothing techniques used by the parsers employed in these experiments lead to correct treatment of the rare words.


The experiments shown in the paper were limited to readily available statistical parsers (which are widely deployed in a number of applications), and certain domains/genres (because of ready access to gold-standard data on which we could verify predictions).

the features we use in our predictor are independent of the particular type of parser or domain, and the same technique could be applied for making predictions on other parsers as well.


To avoid overfitting, we consider only the two models that performed optimally in in the SYN space in Experiment 1 (SELPREF-POW with n=30 and M&L).

since we found that vectors with raw frequency components could model the data, while PMI components could not, we only report the former.


In this work, the features encoded are binary.

features can be assugned numeric weights that corresponds to the probability of the indicator being true for any path between x and z ij (Cohen and .


3 Obviously, the notion of local syntactic coherence only captures some aspects of syntax -e.g., it does not capture long-distance dependencies.

it is a plausible component of syntactic competence and a plausible intermediate step in the acquisition of syntax.


This is surprising because linguistically and computationally syntagmatic and paradigmatic relations are fundamentally different.

on closer inspection, we observe that limiting the number of iterations is often beneficial when computing solutions to a problem iteratively.


Features are the key to obtain an accurate question classifier.

to Li and Roth (2002)'s approach which makes use of very rich feature space, we propose a compact yet effective feature set.


The word of turkeys in the chunk (or span) contributes to the classification of type ENTY:animal if the hypernyms of WordNet are employed (as described in next section).

the extra word group would introduce ambiguity to misclassify such question into HUMAN:group, as all words appearing in chunk are treated equally.


It is not surprising that the word shape feature only achieves small gain in question classification, as the use of five shape type does not provide enough information for question classification.

this feature is treated as an auxiliary one to boost a good classifier, as we will see in the second experiment.


For instance, the head word extracted from What is the speed hummingbirds fly is hummingbirds (the correct one should be speed), thus leading to the incorrect classification of ENTY:animal (rather than the correct NUM:speed).

to Li and Roth (2006)'s approach which makes use of very rich feature space, we proposed a compact yet effective feature set.


The relatively high ratio of subjective questions in the Science category is surprising.

we find that users often post polemics and statements instead of questions, using CQA as a forum to share their opinions on controversial topics.


We can use both of them to obtain better results in the offline setting, while in online setting, we can use the text of the question alone.


Question analysis, especially question classification, has been long studied in the question answering research community.

most of the previous research primarily considered factual questions, with the notable exception of the most recent TREC opinion QA track.


SEAL's extractor requires the longest common contexts to bracket at least one instance of every seed per web page.

when seeds are noisy, such common contexts usually do not exist or are too short to be useful.


The improvements are most apparent for the TREC 13 dataset, where Ephyra has a much higher performance compared to TREC 14 and 15.

the best-configured SEAL did not improve the F 1 score on TREC 14, as reported in Table 5.


When predicted segments were clustered, the quality of the output (2 nd row) is not as good as when the reference segments were used (1 st row) as inaccurate segment boundaries affected the performance of the clustering algorithm.

the qualities of subtasks that occur frequently are not much different.


As we can see, in clean conditions the accuracy of the 3 features is quite high but as the noise conditions increase the accuracy of the 3 techniques decreases substantially.

the TF feature is much more sensitive to noise than the other two techniques.


Otherwise, if we look also into the future, we could just do language identification to extract the CS points.

our goal is to provide methods that can be used in real time applications, where we do not have access to observations beyond the point of interest.


It should be noted that the Spanglish data set is a transcription of spoken CS.

this new evaluation set contains only written CS.


1 It should be noted that absolute values in the range between 0 and 1 are meaningless by themselves.

if a set of word pairs is shown to consistently have higher values than another set, then we can conclude that the members of the former set tend to be semantically closer than those of the latter.


Of course, not all of them are antonymous, for example sect-insect and coy-decoy.

these are relatively few in  Table 1: Sixteen affix rules to generate antonym pairs.


If antonymous occurrences are to be exploited for any of the purposes listed in the beginning of this paper, then the text must be sense disambiguated.

word sense disambiguation is a hard problem.


Previous studies have mostly focused on the idiom type identification (Lin, 1999;Krenn and Evert, 2001;Baldwin et al., 2003;Shudo et al., 2004;Fazly and Stevenson, 2006).

there has been a growing interest in idiom token identification in recent times (Katz and Giesbrecht, 2006;Hashimoto et al., 2006b;Hashimoto et al., 2006a;Birke and Sarkar, 2006;Cook et al., 2007).


This suggests that when one wishes to apply a WSD system to a new domain of interest, it is worth the effort to annotate a small number of examples gathered from the new domain.

instead of randomly selecting in-domain examples to annotate, we could use active learning (Lewis and Gale, 1994) to help select in-domain examples to annotate.


Also, these prior research efforts only experimented with a few word types.

we perform active learning experiments on the hundreds of word types in the OntoNotes data, with the aim of adapting our WSD system trained on SEMCOR to the WSJ domain represented by the OntoNotes data.


Indeed, the ranking of the instances after convergence was identical to the HITS authority ranking computed from instance-pattern matrix M (i.e., the ranking induced by the dominant eigenvector of M T M ).

filtered Espresso suffers less from semantic drift.


The final recall achieved was 0.773 after convergence on the 20th iteration, outperforming the most-frequent sense baseline by 0.10.

a closer look reveals that the filtering heuristics is limited in effectiveness.


Filtered Espresso halted after the seventh iteration (Filtered Espresso (optimal stopping)) is comparable to the proposed methods.

in bootstrapping, not only the number of iterations but also a large number of parameters must be adjusted for each task and domain.


Since search queries are a fundamental part of the information retrieval task, it is essential that we interpret them correctly.

the variable forms queries take complicate interpretation significantly.


We can conclude from this that capitalization in a mixed-case query is a fair indicator that a word is a proper noun.

the great majority of queries contain no informative capitalization, so the great majority of proper nouns in search queries must be uncapitalized.


We did not see a significant improvement in this metric.

we feel that our feature's high ranking warrants reporting and hints at a potentially genuine boost in retrieval performance in a system less feature-rich.


For example, in a machine translation system (Koehn et al., 2007), if the bilingual training data does not contain the word " " (the second example in Table 1), it leaves the word untranslated.

if the word " " does appear in the training data but it has only a translation "gruel" as that is the meaning in the formal text, the translation system may wrongly translate " " into "gruel" for the informal text where the word " " is more likely to mean "like".


Rule-driven Hypothesis Generation: One can use the rules described in Section 2 to generate a set of hypotheses.

with this approach, one may generate an exponential number of hypotheses.


For each candidate relation (x, y), we use the pair as a query to search the web, and treat the number of pages returned by the search engine as a feature value.

3 these features are quite expensive as millions of queries may need to be served.


More formally, if we denote the proposal distribution as Q(z), the target distribution as P (z), and the previous sample as z, then the probability of accepting a new sample z * ∼ Q is set at: Theoretically any non-degenerate proposal distribution may be used.

a higher acceptance rate and faster convergence is achieved when the proposal Q is a close approximation of P .


We also explore the use of different language identification methods to select POS tags from the appropriate monolingual tagger.

the best results are achieved by a machine learning approach using features generated by the monolingual POS taggers.


To evaluate their language model, they asked a human subject to judge sentences generated by a PCFG induced from training data and the language model.


Our original motivation came from the lack of linguistic resources to process Spanglish text.

we did train from scratch a sequential model for POS tagging Spanglish, namely Conditional Random Fields (CRFs) (Lafferty et al., 2001).


In the error analysis we showed that most of the mistakes made by the language identification method, and the oracle itself, occur in sentences with intrasentential code-switching, showing the difficulty of the task.

our machine learning approach was less sensitive to the complexity of this alternation pattern.


There are some researches show that when compound words are split into smaller constituents, better retrieval results can be achieved (Peng et al., 2002a).

it is reasonable that the longer the word which co-exists in query and corpus, the more similarity they may have.


Since A is on the decision boundary, it will be queried as the most uncertain.

querying B is likely to result in more information about the data as a whole.


For pool-based active learning, we often assume that the size of U is very large.

these densities only need to be computed once, and are independent of the base information measure.


In most interesting natural language applications, K is very large, making this algorithm intractable.

it is common in similar situations to approximate the Fisher information matrix with its diagonal (Nyffenegger et al., 2006).


Though the time complexity of the algorithm given by (Jiang and Ng, 2006) is also linear, it should assume all feature templates in the initial selected set 'good' enough and handles other feature template candidates in a strict incremental way.

these two constraints are not easily satisfied in our case, while Algorithm 1 may release these two constraints.


A syntactic-semantic reranking was performed to output the final results according to (Johansson and Nugues, 2008).

only 1-best outputs of the parser before reranking are used for our evaluation.


Under the first-order expectation semiring E R,R n , the inside algorithm of Figure 2 will return Z, r where r is a vector of n feature expectations.

eisner (2002, section 5) observes that this is inefficient when n is large.


Thus, we must be finding ∇Z by formulating it as a certain expectation r. Specifically, to be rs T , a matrix.

when using this semiring to compute second derivatives (Case 2) or covariances, one may exploit the invariant that r = s, e.g., to avoid storing s and to compute r1s2 + s1r2 in multiplication simply as 2 • r1r2.


18 This shows that DA helps with the local minimum problem, as hoped.

dA's improvement on the dev set did not transfer to the test set.


Whereas this is composable in MERT, as they can be recalculated at each critical point.

this would slow down the optimization process quite a bit, since one cannot traverse the dimension by simply adjusting the sufficient statistics to reflect changes in 1-best candidates.


The cost would further grow linearly with the number of MERT iterations and the n-best list size.

optimizing for BLEU takes on the order of minutes per iteration, and costs nothing.


This method is very attractive, since it opens the door to rich lexical features.

in order to robustly optimize the feature weights, one has to use a substantially large development set, which results in significantly slower tuning.


A possible generic solution is to cluster the lexical features in some way.

how to make it work on such a large space of bi-lingual features is still an open question.


This method avoids the over-fitting problem, at the expense of losing the benefit of discriminative training of rich features directly for MT.

the feature space problem still exists in these published models.


Similar ideas were explored in (He et al., 2008).

their length features only provided insignificant improvement of 0.1 BLEU point.


Some candidate rules were thrown away due to the source side constraint.

one string-to-dependency rule may split into several dependency-to-dependency rules due to different source dependency structures.


introduced features defined on constituent labels to improve the Hiero system (Chiang, 2005).

due to the limitation of MER training, only part of the feature space could used in the system.


Features similar to those already labeled are likely to be discriminative, and therefore likely to be labeled (rather than skipped).

insufficient diversity may also result in an inaccurate model, suggesting that coverage should select more useful queries than similarity.


In order to better understand differences in feature query selection methodology, we proposed a feature query selection method motivated 6 by the method used in Tandem Learning in Section 4.1.

this method performs poorly in the experiments in Section 5.


This is typically done between string pairs, where a pronunciation is mapped to its spelling, an inflected form to its lemma, a spelling variant to its canonical spelling, or a name is transliterated from one alphabet into another.

many problems involve more than just two strings: • in morphology, the inflected forms of a (possibly irregular) verb are naturally considered together as a whole morphological paradigm in which different forms reinforce one another; • mapping an English word to its foreign transliteration may be easier when one considers the orthographic and phonological forms of both words; • similar cognates in multiple languages are naturally described together, in orthographic or phonological representations, or both; • modern and ancestral word forms form a phylogenetic tree in historical linguistics; • in bioinformatics and in system combination, multiple sequences need to be aligned in order to identify regions of similarity.


The alignment CRF (AlignCRF) model described in Section 3.1 is able to predict labels for a text sequence given a matching DB record.

without corresponding records for texts the model does not perform well as an extractor because it has learned to rely on the DB record and alignment features (Sutton et al., 2006).


Both our models have a high F1 value for the other label O because we provide our algorithm with constraints for the label O.

since there is no realization of the O field in the DB records, both M-CRF and M+R-CRF methods fail to label such tokens correctly.


In the AnCora corpus of Spanish and Catalan newspaper text (Martí et al., 2007), nearly half of the entities are embedded.

work on named entity recognition (NER) has almost entirely ignored nested entities and instead chosen to focus on the outermost entities.


They collapsed all DNA subtypes into DNA; all RNA subtypes into RNA; all protein subtypes into protein; kept cell line and cell type; and removed all other entities.

they also removed all embedded entities, while we kept them.


For example, some phrases explicitly refer to an event, so they almost certainly warrant extraction regardless of the wider context (e.g., terrorists launched an attack).

1 some phrases are potentially relevant but too general to warrant extraction on their own (e.g., people died could be the result of different incident types).


For example, most readers would agree that sentence (1) below is describing a terrorist event, while sen-   Table 4: Sentential Event Recognizer Results for MUC-4 using 1300 Documents for Training tence (2) is not.


For the Disease role, the Fscore jumped 6%, from 0.43 for the best baseline systems (AutoSlog-TS and the NB baseline) to 0.49 for GLACIER NB/NB .

to the MUC-4 data, this improvement was mostly due to an increase in precision (up to 0.41), indicating that our unified IE model was effective at eliminating many false hits.


On this event role, the F-score of GLACIER NB/NB (0.44) matches that of the best baseline system (Sem Affinity, with 0.44).

note that GLACIER NB/NB can achieve a 5% gain in recall over this baseline, at the cost of a 3% precision loss.


Researchers, such as (Polanyi and Zaenen, 2006), have discussed how the discourse structure can influence opinion interpretation; and previous work, such as (Asher et al., 2008;Somasundaran et al., 2008), have developed annota-tion schemes for interpreting opinions with discourse relations.

they do not empirically demonstrate how automatic methods can use their ideas to improve polarity classification.


Here, in interpreting the ambiguous opinion a bit different as being positive, we use the knowledge that it participates in a reinforcing discourse, and that all its neighbors (e.g., ergonomic, durable) are positive opinions regarding the same thing.

if it had been a non-reinforcing discourse, then the polarity of a bit different, when viewed with respect to the other opinions, could have been interpreted as negative.


Our first baseline, Base, is a simple distributionbased classifier that classifies the test data based on the overall distribution of the classes in the training data.

in Table 3, the class distribution is different for the Connected and Singleton conditions.


Similarly, "being different" could be deemed negative in other discourse contexts.

iLP is able to arrive at the correct predictions for all the instances.


For example, in the sentence, if your Nokia phone is not good, buy this great Samsung phone, the author is positive about "Samsung phone" but does not express an opinion on "Nokia phone" (although the owner of the "Nokia phone" may be negative about it).

if the sentence does not have "if", the first clause is clearly negative.


A large majority of conditional sentences are introduced by the subordinating conjunction If.

there are also many other conditional connectives, e.g., even if, unless, in case, assuming/supposing, as long as, etc.


Popular types of conditionals include actualization conditionals, inferential conditionals, implicative conditionals, etc (Declerck and Reed, 2001).

these classifications are mainly based on semantic meanings which are difficult to recognize by a computer program.


The lexicon covers a substantial subset of the subjective expressions in the corpus: 67.1% of the subjective expressions contain one or more lexicon entries.

fully 42.9% of the instances of the lexicon entries in the MPQA corpus are not in subjective expressions.


For our experiments, OP gives a better idea of the impact of SWSD, because most of the keyword instances SWSD disambiguates are weaksubj clues, and weaksubj keywords figure more prominently in objective classification.


The overall improvements for this language pair are +1% BLEU and -1.4% TER.

to the GALE Chinese-English task, the triplet lexicon model for the Arabic-English language pair performs slightly better than the discriminative word lexicon.


For example, in a given bitext sentence, the Arabic word AlAw-DAE might be translated as situational, for which there might be no support in the alignment lexicon.

the PMI between AlAwDAE and situation might be sufficiently high.


Alignment in that case is straightforward.

some Arabic prepositions and even more English prepositions do not have an explicit counterpart on the other side.


Our approach needs and uses a parser for only one side (English) and not for the other (Arabic).

some of the components of this aligner are language-specific, such as word order heuristics, the list of specific function words, and morphological variation lists.


The derivation of the algorithm so far has focused on its relationship to LDA.

labeled lDA can also be seen as an extension of the event model of a traditional Multinomial Naive Bayes classifier (McCallum and Nigam, 1998) by the introduction of a mixture model.


However, inferring the best set of labels for an unlabeled document at test time is more complex: it involves assessing all label assignments and returning the assignment that has the highest posterior probability.

this is not straight-forward, since there are 2 K possible label assignments.


Because only a processed form of the documents was released, the Yahoo dataset does not lend itself well to error analysis.

only 33% of the documents in each top-level category were applied to more than one sub-category, so the credit assignment machinery of L-LDA was unused for the majority of documents.


For the supervised keyphrase extraction approach, a document set with human-assigned keyphrases is required as training set.


In recent years, a number of systems are developed for extracting keyphrases from web documents (Kelleher and Luz, 2005;Chen et al., 2005), email (Dredze et al., 2008) and some other specific sources, which indicates the importance of keyphrase extraction in the web era.

none of these previous works has overall consideration on the essential properties of appropriate keyphrases mentioned in Section 1.


Our study is related to geographical information retrieval (GIR) systems.

our problem is very far from classic GIR problem settings.


Most of the tourists going through this path spend weeks in prior information mining and preparations.

even when using the most recent maps and guides, they discover that available geographical knowledge is far from being complete and precise.


For instance, despite the fact that the sentences "he is affected by AIDS" and "HIV is a virus" express concepts closely related, their similarity is zero in the VSM because they have no words in common (they are represented by orthogonal vectors).

due to the ambiguity of the word "virus" , the similarity between the sentences "the laptop has been infected by a virus" and "HIV is a virus" is greater than zero, even though they convey very different messages.


This repository of sentences is already divided by sense and can significantly speed-up manual annotation.

the extracted sentences could enrich the training set of machine learning systems for frame annotation to improve the frame identification step.


Exploiting redirections and anchoring strategies, our induction method can account for orthographical variations, for example it acquires both memorize and memorise.

also misspelled words may be collected, for instance gynaecologial instead of gynaecological.


On the one hand, the retrieved data could speed up human annotation, requiring only a manual validation.

the extracted sentences could provide enough training data to machine learning systems for frame assignment, since insufficient frame attestations in the FrameNet database are a major problem for such systems.


If multiple ref-erences were used that Bleu would likely have stronger correlation.

it is clear that the cost of hiring professional translators to create multiple references for the 2000 sentence test set would be much greater than the $10 cost of collecting manual judgments on Mechanical Turk.


The expert actually gains very little from ds for both rand and unc: adding suggestions gave OP-ERs of just 1.83 and .39, respectively.

the non-expert obtains an improvement of 5.93 OPER when suggestions are used with rand, but performs worse when used with unc (-9.91 OPER).


Our results suggest some possible prescriptions for tuning techniques according to annotator expertise.

even if we can estimate a relative level of expertise, following such broad prescriptions is unlikely to be more robust than an approach which adapts selection and suggestion to the individual annotator, perhaps working within an annotation group.


Our results provide validation of several features that can be optimized in the development of new summarization systems when the objective is to improve content selection on average, over a collection of test inputs.

none of the features is consistently predictive of good summary content for individual inputs.


The predictiveness of features like ours will be limited for such inputs 4 .

model summaries written for the specific input would give better indication of what information in the input was important and interesting.


When reference summaries are available, ROUGE provides scores that agree best with human judgements.

when model sum-maries are not available, our features can provide reliable estimates of system quality when averaged over a set of test inputs.


For instance, in (3) there are several strong cues which suggest that play with fire is used literally.

because the unsupervised classifier only looks at lexical cohesion, it misses many other clues which could help distinguish literal and nonliteral usages.


Conversely, words with a low sal lit should be more indicative of the non-literal class.

we found that, in practice, the measure is better at picking out indicative words for the literal class; non-literal usages tend to co-occur with a wide range of words.


As a consequence its confidence function may also not be very accurate.

we know from Sporleder and Li (2009) that the unsupervised classifier has a reasonably good performance.


We could counteract this by selecting a higher proportion of examples labelled as 'literal'.

given that the number of literal examples in our data set is relatively small, we would soon deplete our literal instance pool and moreover, because we would be forced to add less confidently labelled examples for the literal class, we are likely to introduce more noise in the training set.


Note that this data set is potentially noisy as not all non-canonical form examples are used literally.

when checking a small sample manually, we found that only very small percentage (<< 1%) was mis-labelled.


Intuitively, it is plausible that the saliency feature performs quite well as it can also pick up on linguistic indicators of idiom usage that do not have anything to do with lexical cohesion.

a combination of the first three features leads to an even better performance, suggesting that the features do indeed model somewhat different aspects of the data.


Their experiments show that discourse connectives and the distance between the two text spans have the most impact, and event-based features also contribute to the performance.

their system may not work well for implicit relations alone, as the two most prominent features only apply to explicit relations: implicit relations do not have discourse connectives and the two text spans of an implicit relation are usually adjacent to each other.


These problems can be ameliorated by imposing limits on rule size or early stopping of EM training, however neither of these techniques addresses the underlying problems.

our model is trained in a single step, i.e., the alignment model is the translation model.


Generally, two edges can be re-combined if they satisfy the following two constraints: 1) the LHS (left-hand side) nonterminals are identical and the sub-alignments are the same (Zhang et al., 2006); and 2) the boundary words 1 on both sides of the partial translations are equal between the two edges (Chiang, 2007).

as shown in Figure 2, the decoder still generates 801 edges after the hypothesis re-combination.


Therefore, if we can find a method to greedily reduce the size of each bucket ( , ′), we can reduce the overall expected edge competitions when parsing with ′.

it can be easily proved that the numbers of binary rules in any ′ ∈ ℬ are same, which implies that we cannot reduce the sizes of all buckets at the same timeremoving a rule from one bucket means adding it to another.


In our experiments, the linear binarization method is just 2 times faster than the CKY-based binarization.

(•) cannot be easily predetermined in a static way as is assumed in Section 3.3 because it depends on ′ and should be updated whenever a rule in is binarized differently.


A possible reason that the random synchronous binarization method can outperform the baseline method lies in that compared with binarizing SCFG in a fixed way, the random synchronous binarization tends to give a more even distribution of rules among buckets, which alleviates the problem of edge competition.

since the high-frequency source sub-sequences still have high probabilities to be generated in the binarization and lead to the excess competing edges, it just achieves a very small improvement.


We would expect that when comparing larger fragments, on average there would be more transformations needed to change one into the other than when comparing small fragments.

in the previous scheme, small fragments would have higher scores than large fragments, since fewer differences would be observed.


We note that in this example, the score of translating "dos" to "make" was higher than the score of translating "dos" to "both".

the higher level target fragment that composed the translation of "dos" together with the translation of "cuestiones" yielded a higher score when composing "both questions" rather than "to make".


This indeed alleviates the vocabulary coverage problem, especially for the so-called "low density" languages.

these approaches still require bitexts where one side contains the original source language.


Moreover, our proposed paradigm can, in principle, achieve large-scale acquisition of paraphrases with high semantic similarity.

using parallel training texts in pivoting techniques offers the potential advantage of implicit translational knowledge, in the form of sentence alignments, while our approach is unguided in this respect.


One advantage of the bitext-dependent pivoting approach is the use of the additional human knowledge that is encapsulated in the parallel sentence alignment.

we argue that the ability to use much larger resources for paraphrasing should trump the human knowledge advantage.


Pivoting techniques (translating back and forth) rely on limited resources (bitexts), and are subject to shifts in meaning due to their inherent double translation step.

large monolingual resources are relatively easy to collect, and our system involves only a single translation/paraphrasing step per target phrase.


Our system has the advantage of always producing an NL sentence given any input MR, even if there exist unseen MR productions in the input MR. We can achieve this by simply skipping those unseen MR productions during the generation process.

in order to make a fair comparison against WASP −1 ++, which can only generate NL sentences for 97% of the input MRs, we also do not generate any NL sentence in the case of observing an unseen MR production.


The GRE models used here do rely on dependency parsing.

they still generalise across formal domains as the relation identification and characterisation systems, developed on news data, achieve comparable performance when applied directly to a relation extraction task in the biomedical domain (see Hachey (2009) for details).


The simplification reduces the number of free parameters.

low values of n impose an artificially local horizon to the language model, and compromise its ability to capture long-range dependencies, such as syntactic relationships, semantic or thematic constraints.


The semantic space discussed thus far is based on word co-occurrence statistics.

the statistics of how words are distributed across the documents also carry useful semantic information.


Due to its generality, LSA has proven a valuable analysis tool with a wide range of applications.

the SVD procedure is somewhat ad-hoc lacking a sound statistical foundation.


The task of automatically characterizing word meaning in text is typically modeled as word sense disambiguation (WSD): given a list of senses for target lemma w, the task is to pick the best-fitting sense for a given occurrence of w. The list of senses is usually taken from an online dictionary or thesaurus.

clear cut sense boundaries are sometimes hard to define, and the meaning of words depends strongly on the context in which they are used (Cruse, 2000;Hanks, 2000).


The second way in which word sense and vector space models have been related is to assign disambiguated feature vectors to Word-Net concepts (Pantel, 2005;Patwardhan and Pedersen, 2006).

those works do not use sense-tagged data and are not aimed at WSD, rather the applications are to insert new concepts into an ontology and to measure the relatedness of concepts.


Previous research has shown the benefit of jointly learning semantic roles of multiple constituents (Toutanova et al., 2008;Koomen et al., 2005).

our joint model makes predictions for a single constituent, but multiple tasks (WSD and SRL) .


In principle, this problem can be mitigated by training the pipeline model on automatically predicted labels using cross-validation, but in our case we found that automatically predicted WSD labels decreased the performance of the pipeline model even more.

the joint model computes the full probability distribution over the semantic roles and preposition senses.


Since 90s machine learning based approaches to WSD using sense marked corpora have gained ground (Eneko Agirre & Philip Edmonds, 2007).

the creation of sense marked corpora has always remained a costly proposition.


One might argue that any word within the synset could serve the purpose of translation.

the exact lexical substitution has to respect native speaker acceptability.


Despite its importance in basic NLP tasks, the problem has been largely overlooked in NLP research, probably due to it presumed simplicity.

as we have shown, simple methods for MWE identification, such as our baselines, do not perform consistently well across MWEs.


The advantage of such methods is that association relations are established at the phrase level instead of the lexical level, so they have the potential to resolve the above-mentioned translation problem.

when applying association-based methods, we have to consider the following complications.


(1996), this two-step approach drastically reduces the search space.

translations of collocated context words in the source word sequence create noisy candidate words, which might cause incorrect extraction of target translations by naive statistical correlation measures, such as the Dice coef-ficient used by Smadja et al.


Hence, w is a subsequence of f. When counting the word frequency, each word in the target corpus normally contributes a frequency count of one.

since we are interested in the word counts that correlate to w, we adopt the concept of the translation model proposed by Brown et al (1993).


When the training corpus contains more than 9 months of corpora, the precision of collocations extracted by the baseline method did not increase anymore.

the precision of collocations extracted by our method kept on increasing.


The recently introduced online confidence-weighted (CW) learning algorithm for binary classification performs well on many binary NLP tasks.

for multi-class problems CW learning updates and inference cannot be computed analytically or solved as convex optimization problems as they are in the binary case.


exact is expensive to compute, and tends to over-fit in practice (Sec.


Similarly, our exact implementation converges after an average of 1.25 iterations, much faster than either of the approximations.

this rapid convergence appears to come at the expense of accuracy.


LambdaSMART is arguably more powerful than LambdaBoost in that it introduces new complex features and thus adjusts not only the parameters but also the structure of the background model 1 .

1 Note that in a sense our proposed LambdaBoost algorithm is the same as LambdaSMART, but using a single feature at each iteration, rather than a tree.


When a pair occurs more than is expected by chance, the MI score is positive.

if a pair occurs together less than is expected by chance, the mutual information score is negative.