To investigate to what extent noisy breaks are impeding the possible improvements from prosodically enriched models, we replaced all 4 and p breaks in the training and evaluation sets that did not align to the correct phrase boundaries as indicated by the treebank with break 1 for both the conversation sides and human-annotated sentences.

we conclude that, although the use of break indexes can indeed significantly improve performance over baseline models without them on conversation sides, tagging accuracy improves more by using smaller segments, for which the impact of the break indexes is marginal.


Based on these studies, we observe that: (1) bidirectional models tend to achieve better accuracy levels than left-toright models, (2) generative models seem to perform somewhat better than discriminative models on this task, and (3) prosody improves tagging performance of models on conversation sides, but has much less impact on smaller segments.

to text, conversational speech represents a significant challenge because the transcripts are not segmented into sentences.


It can be shown that the model selection in our method is equivalent to a MAP inference, conducted under the assumption that the prior probability of a model decreases exponentially with its length (Goldwater, 2006).

similarly, when the corpus is broken down into single-character tokens, merging individual pairs does not produce any increase in compression.


Row "For" shows the conditions for template instantiation, where "s" indicates that the corresponding template is instantiated when the current character starts a new word, and "a" indicates that the corresponding template is instantiated when the current character does not start a new word.

we show that effective decoding can be achieved with standard beam-search, which gives significant speed improvements compared to the decoding algorithm of Z&C08, and achieves accuracies that are competitive with the state-of-the-art.


Rather, when treated as two singlecharacter words, they can make sense in a sentence such as "请 (please) 自 (self) 来 (come) 取 (take)".

when each character is processed, it is combined with existing candidates in all possible ways to generate new candidates, and an agenda is used to keep the N -best candidate outputs from the begining of the sentence to the current character.


This improves the speed by a significant factor.

beam-search is approximate and can run in linear time.


During both training and decoding, English letters and Arabic numbers are segmented using simple rules, treating consecutive English letters or Arabic numbers as a single word.

if any item in the agenda is the same as the corresponding gold-standard, the decoding process moves to the next character, without any change to the parameter values.


In comparison with our tree kernel-based method, it shows that 1) when using golden anaphoric zero anaphors, the feature-based method performs about 11%, 17% and 15 lower in precision, recall and F-measure, respectively; 2) when golden zero anaphors are given and feature-based anaphoricity determination is applied, the featurebased method performs about 5%, 18% and 17 lower in precision, recall and F-measure, respectively; and 3) when tree kernel-based zero anaphor detection and feature-based anaphoricity determination are applied, the feature-based method per-forms about 7%, 8% and 10 lower in precision, recall and F-measure, respectively.

given a zero anaphor candidate, it is critical to construct a proper parse tree structure for tree kernel-based zero anaphor detection.


At the first glance, it seems that a zero anaphor can occur between any two constituents in a parse tree.

a documentlevel parse tree for an entire document is constructed by attaching the parse trees of all its sentences to a new-added upper node, as done in Yang et al.


(2006) employed a convolution tree kernel on anaphora resolution of pronouns.

we just select the one with maximal confidence as the zero anaphor in the position and ignore others, if multiple zero anaphor candidates occur in the same position.


Based on the analysis of comma positions, our method decides whether or not to insert a comma at each bunsetsu 1 boundary in an input sentence by machine learning.

this paper proposes a method for automatically inserting commas into Japanese texts.


In the analysis data, there existed 695 bunsetsus whose rightmost morpheme is a conjunction and which are located at the beginning of a sentence.

if the same types of characters appear sequentially, readers may make a reading mistake or feel difficulty in reading them.


In standard Dutch, the noun 'slachtoffer' goes with the 'het' definite article which is marked in its lexical entry.

it should be noted that since our approach cannot differentiate between incomplete and incorrect entries, no entry in the lexicon is modified.


Further, the method we describe in Cholakov and van Noord (2009) is applied to generate the paradigm(s) of each word in question.

because of data sparseness problems, this method is only able to handle unigrams and bigrams.


The model predicts a large number of lexical types which, in turn, leads to large lexical ambiguity.

it is possible that a word could have a problematic lexical entry even if it only occurs in sentences which are assigned a full-span parse.


adding a new section), minor changes (e.g.

in this paper we introduce a new annotated corpus of temporal expressions that is intended to address this shortfall.


The reliable processing of temporal information is an important step in many NLP applications, such as information extraction, question answering, and document summarisation.

similarly, ago occurred 113 times in ACE, but only twice in WikiWars: once in quoted speech, and once used incorrectly instead of earlier in a context-dependent expression.


However, existing corpora have their limitations.

this process also included the annotation of any temporal expression missed by the automatic tagger, and the removal of spurious matches.


Wikipedia did not contain an article for one war, and we considered two articles as inappropriate for our purposes since they did not describe the course of the wars, but rather some general information about the conflicts.

it can often be inferred from the context that subsequent unadorned time references should inherit the same time zone as an earlier time reference.


-Lundi, la chambre des représentants américaine rejeta le projet de soutient du système financier, auquel elle aurait dû consacrer jusqu'à 700 milliards de dollars (près de 12 bilions de kč).


It is not clear how well τ works as an automatic evaluation metric of translation quality.

the left graph of Figure  2 shows a scatter plot of BP and "normalized average adequacy."


Our meta-evaluation of the NTCIR-7 PATMT JE task data shows that this metric outperforms conventional metrics.

since Kendall's τ is given by: the number of increasing pairs the number of all pairs − 1, In this case, we can obtain spearman's ρ as follows: "John" moved by d 1 = 2 words, "hit" moved by d 2 = 0 words, "Bob" moved by d 3 = 2 words, and "yesterday" moved by τ considers only the direction of the movement, whereas ρ considers the distance of the movement.


Such dictionaries are characterized by a complex structure and an important access time when using a querying system.

this study is based on the specific documents and dictiona-ries for each user level.


The rest of the paper is organized as follows.

for our data, a recall of 10% corresponds to a precision of 46% for the worstperforming model and 78% for the best-performing model.


Then the algorithm chooses the least confident sentence, breaking ties by favoring shorter sentences (assuming they contain relatively more informative words to be labeled than long sentences).

(Scheffer et al., 2001) estimate confidence of single token label in HMM based information extraction system by a method similar to the Delta method we used.


We see that in three of the four datasets CW outperforms CRF and in one dataset CRF performs better.

other methods are providing confidence in the predicted output as an absolute information, that is, the probability of a prediction to be correct.


Formally, we define the set Z = {z (i) : The predictions of algorithms that use the mean weight vectorŷ = arg max z µ • Φ(x, z) are invariant to the value of the input Σ (as noted by ).

as before, the confidence estimation methods (KD-PC, KD-Fixed, KBV, WKBV and CRF) were applied on the entire set of predicted labels.


In this case, we draw K labelings from this distribution.

for NER Dutch dataset CRf and CW have almost same number of erroneous words and almost same number of erroneous words detected, and finally in NER Spanish dataset CRf has fewer erroneous words and it detected less erroneous words.


For instance, in ACE05, Organization and Location entity types were split into four entity types (Organization, Location, Geo-Political Entity and Facility).

rule Cr 4 identifies occurrences that have a contextual clue confirming that the mention was to a location (e.g., 'In' or 'At').


Automatic Rule Refinement Tools automatically suggesting entire customization rules to a complex NERL program in the spirit of (Liu et al., 2010) can further reduce human effort in building NER annotators.

similarly, tools for explaining false positives in the spirit of (Huang et al., 2008), are also conceivable.


While rule-based systems are appealing due to their well-known "explainability," most, if not all, state-of-the-art results for NER tasks are based on machine learning techniques.

the customizations that modify existing instances (C EB ,C AT A ,C G ) require CR and CO rules.


After discussing related work, we present our empirical results and conclude.

we will see how this changes substantially when moving to an out-of-domain scenario.


On analysis, we found that most of these correspond to the ACE event type CONTACT.

the depth of the embedding of the second target creates issues of data sparseness, as the path-enclosed trees become very large and very diverse.


Second, it does not require solving the sentence-level task perfectly, and also does not require precise sentence-level training labels.

the five least subjective sentences are preceded by circles with numbers denoting the subjectivity order (1 being least subjective according to SVM sle f s ).


Embryonic stem cells have the potential to form any cell type in the human body.

that is vastly different from stem cell research and as an ovarian cancer survivor, I am not going to stand in the way of science.


To alleviate this problem, we employ semisupervised learning in order to adapt the model to the target language.

we also run experiments with two different language pairs -English-Chinese and English-French -to see if the distance between language families influences the effectiveness of our method.


We begin with the extraction of bursty topics from the query log.

we define a Query Session containing a bursty topic t, qs(t), as the queries which are issued by the same user after he issued t, within 30 minutes.


To better evaluate this tendency of each source, Table 6 shows the change rate of the highest density to the overall density measured by lin.

the body part has a higher density than the title part.


The principle of statistical relevance also identifies and removes irrelevant factors.

the more complex, lengthtwo rules synthesize facts extracted from multiple pages, and infer results that are not stated anywhere in the corpus.


Although we build patterns and do experiments on Chinese questions, our method does not take advantage of the particularity of Chinese language and thus can be easily implemented on other languages.

fraction lines, college entrance exam 分数，数线，高考，考分，录取，...


We first select top 3000 patterns by information gain, merge these patterns with hand-crafted ones and combine similar patterns.

disagreements show up on eighteen percents of questions.


In this split each blog is represented in both training and test sets.

finally,in the future we plan to extend our model to perform joint modeling and summarization of idealogical discourse.


During inference, we hold the corpus-level count matrices fixed, and keep sampling from the above 2 * K +1-component multinomial while only changing the document-level count matrices: C DK , C DX 1 until convergence.

,in modeling ideology, we aim toward contrasting two or more ideological perspectives each of which is subjective in nature.


For word w n in document d, instead of sampling z n , x n,1 , x n,2 independently, we sample them as a block as follows: The above three equations can be normalized to form a 2 * K + 1 multinomial distribution: one component for generating a word from the ideology topic, K components for generating the word from the ideology-independent portion of topic k = 1, • • • , K, and finally K components for generating the word from the ideology-specific portion of topic k = 1, • • • , K. Each of these 2 * K + 1 cases corresponds to a unique assignment of the variables z n , x n,1 , x n,2 .

a conservative writer might stress issues like pro-life, God and faith.


We start with an overview of relevant research (Section 2) and present the characteristics of the Swiss German dialect area (Section 3).

these normalized weights can be interpreted as conditional probabilities of the corresponding transfer rule: p(r | t), where r is the rule and t is the geographic location (represented as a pair of longitude and latitude coordinates) situated in German-speaking Switzerland.


The Alemannic Wikipedia contains about 6000 articles, among which many are written in a Swiss German dialect.

standard German immer 'always' corresponds to geng, immer, or all, depending on the dialect).


For example, spiele is modelled after Standard German spielen 'to play', although the vowel is a short monophthong in Swiss German and should thus be written spile (ie represents a diphthong in Dieth spelling).

2 all this data is very heterogeneous in terms of the dialects used, spelling conventions and genre.


The scalar product assigns 1 to a pair of identical vectors if and only if P (z i ) = 1 for some i and P (z j ) = 0, ∀j = i.

experimental results on lexical substitution and word similarity show that our algorithm outperforms previously proposed models.


In particular the constituent labels are highly ambiguous, firstly we don't know a priori how many there are, and secondly labels that appear high in a tree (e.g., an S category for a clause) rely on the correct inference of all the latent labels below them.

the probability of a derivation, e, is the product of the probabilities of its component rules, P (e) = c→e∈e P (e|c) .


For each potential new lexicon Λ , NEW-LEX computes the probability p(y * |x i , z i ; θ , Λ ) of the original parse y * under Λ and parameters θ that are the same as θ but have weights for the new lexical items, as described in Section 7.

if the learned lexical items are used in too many incorrect parses, the stochastic gradient updates will down weight them to the point where the lexical induction step can merge or re-split nodes in the trees that contain them.


HDP-DEP outperforms both DMV and PGI across all six languages.

here the selection of π is indexed by the current node's coarse symbol s, the symbol s and subsymbol z of the parent node, and the context c of the current node.


Unlike the universals from Table 1, these rules alone are enough to construct a full dependency tree.

table 4 shows the performance of both our full model (HDP-DEP) and its No-Split version using universal dependency rules across six languages.


This tractable set typically makes stronger independence assumptions between model parameters than the model itself.

variational inference transforms this problem into an optimization problem where we try to find a distribution q(θ, z) from a restricted set Q that minimizes the KL-divergence between q(θ, z) and p(θ, z | x): Rearranging the above yields: where F is defined as F is a lower bound on likelihood.


We notice that the proposed method outperforms all baselines at all operating points.

mining sentiment from text has a wide range of applications from mining product reviews on the Web (morinaga et al., 2002;Turney and Littman, 2003) to analyzing political speeches (Thomas et al., 2006).


For example, both 'Al Gore' and 'William Clinton' are also names of books, but references in text to these are rarely a reference to the written work sense.

for instance, <George Washington, was born in, Virginia> and <George Washington, was born in, the British colony of Virginia> are not in conflict.


Argument Ambiguity: <George Washington, was born in, 'Kortrijk, Belgium'> in addition to his being born in 'Virginia' suggests that 'was born in' is non-functional.

in the future, we could also use a selectional preferences system (Ritter et al., 2010;Kozareva and Hovy, 2010).


These results validate our basic premise that it is important to model the interaction between topical and geographical variation.

an advantage of the logistic normal is that the variational parameters over µ k are available in closed form, where J indicates the number of regions.


This yields reasonable distance errors but poor classification accuracy.

this difficulty is obviated by the priors: given µ and σ 2 , there is only a single η that maximizes P (w|η)P (η|µ, σ 2 ); similarly, only a single µ maximizes P (η|µ)P (µ|a, b 2 ).


In fact, for this setting the algorithm returns the same solution as for K = 5000 on 99.59% of the examples.

first, the variables z * (i, j) may not form a wellformed directed spanning tree.


One major focus of biomedical text mining has been the extraction of named entities, such genes or gene products, and of flat binary relations between such entities, such as protein-protein interactions.

model 2 improves F1 for regulations, while model 3 again increases F1 for both regulations and binding events.


It can be verified that this derivation is a valid member of Y .

x 5 have a translation in the phrase table as "the dog".


Other papers use margin methods such as MIRA (Watanabe et al., 2007;Chiang et al., 2008), updated somewhat to match the MT domain, to perform incremental training of potentially large numbers of features.

the global variable L stores hypotheses combination matrices, one matrix for each range of sentences (s, t) as shown in   where sent.-level costs E s,n := E(r s , e s,n ) output : optimal weight vectorŵ and its loss L 1 begin sort N -best lists by increasing losses: Fig.


Denote the subset of parameters that are unlexicalized by θ ( ) .

our coarse-grained, universal PoS tag set consists of the following 12 tags: NoUN, VERB, ADJ (adjective), ADV (adverb), PRoN (pronoun), DET (determiner), ADP (preposition or postposition), NUM (numeral), CoNJ (conjunction), PRT (particle), PUNC (punctuation mark) and X (a catch-all for other categories such as abbreviations or foreign words).


This nominalization is a relative clause, but does not have a clue about its boundary.

we leave the third-order models (Koo and Collins, 2010) for a future study.


The first constraint function, denoted as F b2 (r s2 : r t2 ), checks if the corresponding words also have a direct dependency relation r t2 .

because the bilingual treebank is generated automatically, using the bilingual constraints alone is not reliable.


Unlike MPP, the MCP can be calculated efficiently using dynamic programming techniques over the parse forest.

extracting this particular set of fragments is not trivial, though: a naive approach that filters a complete table of fragments together with their frequencies fails because that set, in a reasonably sized treebank, is astronomically large.


Hence, generic paraphrase generators, like ours, intended to be useful in many different applications, should be evaluated for many different combinations of the λ i weights.

in a further extension, Model 3, all the candidate phrase pairs e 1 , e 2 are collectively treated as a monolingual parallel corpus.


In the first case, the rank of the first correct item is the minimum rank of any item judged correct by either judge.

the phrase translation probabilities p(e 1 |f ) and p(f |e 2 ) are estimated by maximum likelihood estimation and smoothed using Good-turing smoothing (Foster et al., 2006).


Synonym extraction (Wu and Zhou, 2003), lexical substitution (McCarthy and Navigli, 2007) and paraphrasing (Madnani and Dorr, 2010) are related to collocation correction in the sense that they try to find semantically equivalent words or phrases.

an analysis of a large corpus of annotated learner English confirms this assumption.


Simply considering the weight of the edge w (u, v) gives us no such information.

section 4.2.1 describes our random walk and how we compute the transition probability matrix P and its stationary probability distribution π.


For regularization the parameter γ controls how much emphasis is placed on the agreement between the initial and learned labels.

we can additionally constrainl by requiring that the initial labels cannot be modified, or more generally penalizing the discrepancy betweenl(u) and l(u) for u ∈ L. The methods of (Talukdar and Pereira, 2010) optimize objective functions of this type.


For example, the accuracy of centrality for carnivores is 80% showing that from the top 57 ranked animal instances, 80% belong to carnivores.

the score of node v is given by the number of paths from u to v for all u ∈ V , with longer paths given less weight based on the value of α.


We listed the topics of the hierarchies in Appendix B (submitted as supplementary material).

we adopt Su's method as the baseline here.


Because the algorithm is greedy and the derivatives of the supervised cross-entropy error also modify the matrix W (1) , this objective is not necessarily continuous and a step in the gradient descent direction may not necessarily decrease the objective.

furthermore, we introduce a new dataset that contains distributions over a broad range of human emotions.


Red nodes show the supervised softmax layer for label distribution prediction.

rAAMs learn vector representations only for fixed recursive data structures, whereas our rAE builds this recursive data structure.


Finally, an unsupervised method was adopted to generate, weigh and filter common SSRs without cue phrases.

the result in table 7   filtering were essential in our proposed method.


For example, Continuation was the most common relation in annotated corpus, but the occurrences of Condition relation were rare.

experimental results showed that our methods not only effectively recognized discourse relations but also achieved significant improvement (p<0.01) in sentence level polarity classification.


Comparing to the weights of mismatches for most representative words (−1.0), the proposed method successfully down weighed the words which were not important for discourse identification.

we firstly presented a discourse scheme based on empirical observations.


The overall inter annotator agreement was 86.05% and the Kappa-value was 0.8031.

cur- 1 Defined as ambiguous sentences in this paper rent state-of-the-art methods for sentence-level polarity classification are facing difficulties in ascertaining the polarity of some sentences.


(2008) modeled polarity reversal using HCRFs integrated with inter-sentence discourse structures.

the number of mismatches in match(d i , d j ) could not precisely reflect the confidence of the generated common SSRs.


The derivative of the additional term with respect to w will be: Hence the partial derivative with respect to w x i j will have an additional term λw x i j .

since the learning problem for bag-of-words OLogReg is convex, we will get the global optimum.


(The constraints are similar to the ones in PRank algorithm).

some recent work is trying to identify the degree of sentiment of adjectives and adverbs from text using co-occurrence statistics.


The feature templates we apply to each span can be divided into the following five categories.

a held-out test set of 396 manually aligned sentence pairs was used to evaluate reordering accuracy.


Because each permutation is independent of all the others, reordering inference via a single pass through the tree is optimal.

we impose a maximum terminal length of L, but otherwise allow any binary tree.


We first briefly review issues of string-to-tree rule extraction; then we discuss how to augment the string-to-tree rules to yield fuzzy-tree to exact-tree rules.

the translation model will learn to distinguish good rules from bad ones via the compatibility scores.


Neither of them builds target syntactic trees using target syntax, however.

by contrast, the system with likelihood matching does not perform as well as the other two algorithms, although it also significantly improves the baseline s2t in all tasks.


For our experiments, we set β = 10 −3 and b = 300.

we can replace the lexical word of each node with its categories (i.e.


Examples include models that account for time (Blei and Lafferty, 2006), books (Mimno and McCallum, 2007), and aspect or perspective (Mei and Zhai, 2006;Lin et al., 2008;Paul et al., 2010).

notice that this is a realized discrepancy; it depends on the latent assignments of observed words to topics.


The top left topic represents formulaic language, language that occurs verbatim in many documents.

the second term conditions this distribution on a particular word type w by normalizing the perdocument number of times w appeared in each document (in topic k).


To compute association scores (AS), we need to count the number of times pair of words appear together within a certain window size.

these values of and δ determine the width and depth of the two-dimensional array respectively.


Finally we report variants of our approach using association scores computed on the GWB50 using CU sketch with 2 billion counters.

to answer the point query for a given word pair, we return minimum over all the positions indexed by the k hash functions.


Estan and Varghese introduced the idea of conservative update (Estan and Varghese, 2002) in the context of computer networking.

in some NLP problems, we are not interested in low-frequency items.


Interestingly, no topics were ever considered "good" by one expert and "bad" by the other-when there was disagreement between the experts, one expert always believed the topic to be "intermediate."

samples from this model exhibit the "burstiness" property: the probability of drawing a ball of color w increases each time a ball of that color is drawn.


In such cases, the experts discussed the reasons for their decisions and came to a consensus.

the topicword component of the sampling distribution is no longer a simple predictive distribution-when sampling a new value for z (d) n , the implication of each possible value for subsequent tokens and their topic assignments must be considered.


The query strategy we employed (uncertainty sampling) is a relatively straightforward method which only considers the best estimate for each unlabeled instance, disregarding other estimates that may contain useful information.

aSVM performs nearly as well, with an accuracy of 80% and F-score of .75.


Looking at the CRF-based methods, SSCRF outperforms CRF in particular when 10-25% of the labeled data is used.

the C&C tools (Curran et al., 2007) trained on biomedical literature were employed for POS tagging, lemmatization and parsing.


1 The original AP algorithm considerably improved exemplarbased clustering both in terms of speed and the quality of solutions.

(2009) use a Latent Dirichlet allocation topic model (Blei, Ng, and Jordan, 2003) to find coherent segment boundaries.


For instance, in the example sentence provided at the beginning of this section, the words "arrested" and "killed" probably have a relatively high apriori likelihood of being ca-sually related.

, a n are the arguments associated with e. Examples of predicates could be verbs such as "attacked", "employs", nouns such as "explosion", "protest", etc., and examples of the arguments of "attacked" could be its subject and object nouns.


In a normal regression analysis, one must either assume that participants or the particular choice of items add some randomness to the experiment, and either each participant's responses for all items must be averaged (treating participants as a random factor), or all participant's responses for each item is averaged (treating items as a random factor).

the vocabulary of hidden states is described in Section 2.1 and the emission distribution in Section 2.2 A key feature of the co-reference component of our model (described below) is that syntactic analysis and co-reference resolution happen simultaneously.


Prediction refers to the fact that the human sentence processor is able to anticipate upcoming material, and that processing is facilitated when predictions turn out to be correct (evidenced, e.g., by shorter reading times on the predicted word or phrase).

we did not trim any items due to abnor- 1 We assume that each participant and item bias the reading time of the experiment.


Here the lexicons in source and target languages are extracted from corpora by taking the top 2,000 words in each corpus.

this approach subsumes a range of past tasks, though of course past work has specialized in interesting ways.


In fact, with only one training language, our method performs worse (on average) than the Linguistica baseline.

each such example consists of an input-label pair (x, y), both of which contain complex internal structure: The input x ∈ X consists of a vocabulary list of all words observed in a particular monolingual corpus, and the label y ∈ Y consists of the correct morphological analysis of all the vocabulary items in x.


Many of our precision errors seem to be due to our somewhat limited model of sound change.

intuitively, it is unlikely that a word from one distant branch of tree resembles a word in another branch.


We model the evolution of a word w to its child w as a sequence of unigram edits that include insertions, deletions, and substitutions.


Instead, pieces of the phylogeny are simply "cut" into subtrees whenever an innovation occurs.

homoplasy seems to be relatively uncommon (though not unheard of) in the Oceanic and Formosan families.


For instance, it might be extremely likely that B changes into both A and C, but that A never changes into B or C, and so weighted variants of parsimony might be necessary (Sankoff and Cedergren, 1983).

it cannot model homoplasy, the presence of more than one word in a language for a given gloss.


Using the scoring method, We rank the individual sentences of the general-domain corpus, select only the top N .

many existing domain adaptation methods fall into two broad categories.


Furthermore, this system combination works better than the conventional multi-model approach by up to 0.7 BLEU on both the dev and test sets.

directly concatenating the phrase tables into one larger one isn't strongly motivated; identical phrase pairs within the resulting table can lead to unpredictable behavior during decoding.


Finally, we show that these tiny translation models for model combination can improve system performance even further over the current standard way of producing a domain-adapted MT system.

the exact overlap between the phrase tables was tiny, minimizing this effect.


also for the interest on debt of the state .

note that the values in each column do not add up to 100%, since the candidate translations are often identical, and even a different translation can receive the same human judgment.


consider, for instance, the example in Figure 1.


Rather than postulating a strictly synchronous structure over the source and target sentences, QG identifies a "sloppy" alignment of parse trees assuming that the target tree is in some way "inspired by" the source tree.

it is therefore necessary to impose a global minimum output constraint (1f).


Generated simplification: There was once a sweet little maid.

we refrained from doing so as Zhu et al.


Step 1: Lexical Induction We introduce new lexical items by selecting candidates from the function GEN LEX, following previous work (Zettlemoyer and Collins, 2005; as reviewed in Section 5.3.

for example, the phrase "from" is assigned the category with syntax (N \N )/N P and semantics λy.λf.λx.f (x) ∧ f rom(x, y).


Note that for the global biased matrix, we measure the affinity between local sentences from t and global sentences from other dates.

s 3 : The rig is owned and operated by Transocean, a company hired by BP to carry out the drilling work.


Taking a collection relevant to a news subject as input, the system automatically outputs a timeline with items of component summaries which represent evolutionary trajectories on specific dates.

precision drops due to ex-cessive choice of global timeline-worthy sentences.


Since it is possible that different verbs may be associated with varying number of object arguments, we limit ourselves to verbs that take on at most two objects (or more specifically two noun phrase arguments) as a simplifying assumption: N k = {n 1 , n 2 } where n 2 can be NULL.

the rationale for using a HMM is that we can reuse all previous computation of the probabilities at each level to compute the required probabilities at the current level.


As a baseline, we simply generated T * directly from images without using the corpus.

we show that it is possible to avoid predicting actions directly from images -which is still unreliable -and to use the corpus instead to guide our predictions.


For each such Q, and given a threshold c, the power for the test at Q is the rejection probability Q(S ≥ c).

what can be done in a particular dataset, like the TAC summarization score datsets we consider, to ascertain the approximate validity of theoretically derived large-sample cutoffs for test statistics?


In the paper (Conroy and Dang, 2008) the authors noted that while there is a significant gap in performance between machine systems and human summarizers when measured by average manual metrics, this gap is not present when measured by the averages of the best automatic metric (ROUGE).

in many settings, the null hypothesis comprises many possible probability laws, as here where the null hypothesis is that the underlying probability laws for the score-samples of two separate summarizers are equal, without specifying exactly what that probability distribution is.


We begin with features for each of the quasi-synchronous configurations from Smith and Eisner (2006), adapted to phrase dependency grammars.

we find that we can achieve moderate but consistent improvements over the baseline Moses system, for an average increase of 0.65 BLEU points for ZH-EN and 0.75 for UR-EN.


types for the task of reordering Hindi sentences to be in English word order.

variations on this structure are possible and quite frequent (e.g., clauses with two objects).


It uses a queue D of discovered treelets, and a data-structure C that associate a treelet to those of its subtreelets that have been already computed.

we considered it was good enough for our purpose, since we mainly want to test our algorithm is a practical way.


For each pair of consecutive entities in the corpus, i = 1..I: In the RDM, words from the arguments are informed by the relation through an argument semantic class which is sampled from P (c 1 , c 2 |t r ) = ψ 1,2 t r .

an MRI of the right leg was done which was equivocal for osteomyelitis .


It is not clear why this cluster contains both types of words, but many of the high ranking words beyond the top ten do correspond to treatments, such as "Morphine", "Albumin", "Ativan", and "Tylenol".

the model discovers relation instances and their types by determining which context tokens express the relation.


Each day, NELL has two tasks: (1) to extract additional beliefs from the web to populate its growing knowledge base (KB) with instances of the categories and relations in its ontology, and (2) to learn to perform task 1 better today than it could yesterday.

after some investigation we found that for many relations, the top portion of the result list is more diverse: i.e.


Among all the 24 functional predicates, N-FOIL discovers confident rules for 8 of them (it produces no result for the other 16 predicates).

the two baseline methods are an untrained RWR model and a trained RWR model as described by Lao and Cohen (2010b).


After that, we describe two improvements to the PRA method to make it more suitable for the task of link prediction in knowledge bases.

traditional logical inference methods are too brittle to be used to make complex inferences from automatically-extracted knowledge, and probabilistic inference methods (Richardson and Domingos, 2006) suffer from scalability problems.


The KB we consider is a large triple store, which can be represented as a labeled, directed graph in which each entity a is a node, each binary relation R(a, b) is an edge labeled R between a and b, and unary concepts C(a) are represented as an edge labeled "isa" between the node for the entity a and a node for the concept C. We present a trainable inference method that learns to infer relations by combining the results of different random walks through this graph, and show that the method achieves good scaling properties and robust inference in a KB containing over 500,000 triples extracted from the web by the NELL system (Carlson et al., 2010).

we maximize the objective function where λ 1 controls L 1 -regularization to help structure selection, and λ 2 controls L 2 -regularization to prevent overfitting.


As a basis for LDA modeling, they collect pseudo-documents, i.e.

the vectors incorporate latent semantic information obtained from two variants of LDA topic models.


This suggests that in many cases the distinctions drawn by WordNet are too subtle even for humans to reproduce.

the collection of pseudodocuments used as input to C-LDA is structured in a controled way such that each document conveys semantic information that specifically characterizes the individual categories of interest (attributes, in our case).


Our experimental results show that modeling attributes as latent or explicit topics with C-LDA and L-LDA, respectively, outperforms the purely distributional baseline model DEPVSM and PATTVSM of prior work.

latent topics t in the standard formulation of LDA (2) are constrained to correspond to explicit labels a.


Similarly, let n z be the number of words in the documents assigned to topic z, and n s z be the number of times sense s assigned to topic z.

the co-occurred words may not be semantically related in a manner that is relevant for topic coherence.


The reason may be that words in LDA are independent isolated strings, while in STM0 they are connected by senses.

the window size , does not make a significant difference, yet we note that =10 is still the optimal value, similar to our observation in the text categorization task.


For Technology, we should credit WordNet definitions, since Technology may contain many infrequent technical terms, and STM0 cannot generalize the meaning of words only by distributional information due to their low frequency usage.

the largest gain is observed in Society, Miscellaneous, Culture, Technology.


There are also approaches that use deeper machine learning techniques to train sentiment classifiers on examples that have been labeled for sentiment, either manually or automatically, as described above.

it is expensive and labor intensive and cannot be applied in real time.


In this paper, we promote personalized recommendation as a novel way of helping users to consume large quantities of subjective information.

but this is only useful when users have explicit information needs that can be formulated as queries.


Model fitting for our model is based on the expectation-maximization (EM) algorithm (Dempster et al., 1977).

this suggests that to obtain high precision at the top of a recommended list, comment features are important.


This projection can be thought of as a multivariate linear regression problem with weight matrix G, one weight vector per dimension of u i .

a related line of research looked into predicting helpfulness of reviews in the hope of promoting those with better quality, where helpfulness is usually defined as some function over the percentage of users who found the review to be helpful (Kim et al., 2006;Liu et al., 2007;Danescu-Niculescu-Mizil et al., 2009).


All tuning parameters are determined using the tuning set, and performances are reported over the test set.

learning weights over textual features across all users outperforms learning for each user individually, which holds true even for heavy raters.


If between a pair of users one consistently likes or dislikes the other, then at least for the heavy users, this authorship information alone could be highly informative.

the difficulty in analyzing the textual information in comments can be alleviated by additional contextual information such as author identities.


The uniform distribution is appropriate in our setup, since annotators are not told which system generated each output, and the order of choices is randomized.

to statistical tests such as χ 2 , or the G 2 Log Likelihood Ratio, Fisher's Exact Test produces accurate p-values even when the expected counts are small (as is extremely common in our case).


Consider predicting the number of downloads over g future time steps.

in practice, we multiply the penalty by the number of examples M to facilitate tuning of λ.


the forecast gap is only  Table 4: Classification accuracy (%) for predicting whether ACL papers will be cited within three years. "


We segment each entry noun phrase into words.

the unigram model with random initialization was indifferent from that with default initialization.


consistently segmented) to another stable state (consistently unsegmented).

we evaluated the performance of a model with precision, recall and the F-score, all of which were based on tokens.


We have presented a Bayesian model for syntactic class induction that has two important properties.

we therefore used only two context words for all of our additional test languages (below).


Note that this model with multiple context features is deficient: it can generate data that are inconsistent with any actual corpus, because there is no mechanism to constrain the left context word of token e i to be the same as the right context word of token e i−1 (and similarly with alignment features).

this provides a very simple vector-based clustering baseline.


For example, malaria mosquito, which expresses CAUSE, can be paraphrased using verbs like carry, which do not imply direct causation.

for example, Vanderwende (1994) associated verbs extracted from definitions in an online dictionary with abstract relations.


Example patterns are be filled with (cotton bag, water cup) and use (water sculpture, wood statue).

they learned semantic lexicons using extraction patterns, and then, alternatively, they extracted new patterns using these lexicons.


Second, their approach involves qualitative analysis of the collected data only a posteriori, after manual removal of invalid and trivial generated hypotheses.

eNG: I hear the click of the trigger and the burst of bullets reach me immediately.


Every verb in TroFi has at least one literal usage and one nonliteral usage, so there is no issue with the definition of recall as 100% when there are no literals or no nonliterals.

in the third experiment, we train the algorithm on the twenty-five new verbs that were not used by Birke and Sarkar (2006) and then we test it on the old verbs.


should be trying to capture imagability, not concreteness.

the algorithm is tested with verbs that it has never seen before.


All models use the interpolation method of Eq.

we pose the following question: If we can afford to have only a handful of trees in the model, what would be best approach to construct those trees?


The mixture weights were optimized with respect to TURKDEV using SRILM.

bLOG, USENET and SWITCHbOARD also performed well.


I need to start making a shopping list soon.

this set will be used for initial evaluations and also to tune our models.


For AAC devices this means closely modeling everyday face-to-face communications.

our data does not contain long-term two-sided conversations.


Finally, while the predictive keyboard is a commonly studied interface, it is not appropriate for all AAC users.

in order to build a good long-span language model, we would require millions of such communications.


While it may reduce the diversity of communications, we found that workers were more eager to accept HITs of type 2.

using too large thresholds eventually hurt performance except on SWITCHTEST.


Third, surprisingly with respect to previous work, STK applied to CT 8 provides accuracy lower than BOW, about 8 absolute points.

this stage of the pipeline employs several machine learning models specially trained to handle various types of questions.


In other words, a fragment is any possible tree path, from whose nodes other tree paths can depart.

pASS (Moschitti, 2008), does not work in Jeopardy!.


We found that this problem has been largely resolved in the current release.

multiword nouns allow inflection (singular vs. plural) but no insertion.


With regard to argument extent, we automatically corrected mistakes such as the erroneous inclusion of punctuation marks at the end of clauses/sentences or not including all obligatory complements in a verb phrase argument.

this makes their treatment more challenging.


no such clear majority reading), then M2 normally improves (often substantially) on M1.

similar to lexical patterns, these can capture discourse and non-discourse usage.


Thus, nontensed arguments are most often nominalisations leading to a reason reading, whereas a verb phrase as argument 2 and a sentence as argument 1 often is a result reading.

further adaptations necessary for Arabic are the inclusion of clitics as connectives such as /l/for, /b/by,with and /f/then .


All differences are statistically significant (M1 < M2 < M3 < M4).

sentence-start adverbials consisting of more than one token such as /bydān(6) and /ġyrān(6) were correctly classified as CONTRAST.


The improvements obtained from adding semantic features also suggests that domain-specific semantics can be useful for recognizing some speech acts.

to ensure good precision, only tags that have a confidence value ≥ 1.0 were used.


Table 2 shows the distribution of speech act labels in our data set.

we were very conservative in labelling sentences as Representative speech acts.


In this paper, we present a new ranking scheme, collaborative ranking (CR).

• Redundancy: Collaborators and query may share redundant information.


We now take the average function (Figure 3 (a)) into considerations, as graph clustering algorithm gradually partitions from one cluster (corresponding to threshold 1) to more clusters, the number of query collaborators gradually reduces, meanwhile, the accuracy gradually increases and reaches the highest (73.6%) at threshold of 0.45, which clearly shows that removing noisy collaborators in the query collaboration group can improve the performance.

to traditional non-collaborative ranking scheme which solely relies on the strengths of isolated queries and one stand-alone ranking algorithm, the new scheme integrates the strengths from multiple collaborators of a query and the strengths from multiple ranking algorithms.


Collective disambiguation works very well when a text contains mentions of a sufficiently large number of entities within a thematically homogeneous context.

this constraint may lead to very suboptimal results.


on making tunable vertical cavity surface emitting laser diodes" and laser diodes was one of the non-compositional phrases recognized.

second, we develop a cascaded classification framework for semantic head recognition.


The specific NLP application we evaluate in this paper is information retrieval.

our attempts to use the sim alt recognizer (bold in Table 4) in this way were not successful.


verb constructions are often discontinuous.


Part of the reason may be that by using only named entities in the context rather than all words, we have effectively picked the most useful contextual terms.

also machine learning approach induces challenges of handling heterogenous cases.


This problem is alleviated in our work by exploiting distributional prior knowledge about template slots, which is shown effective when coupled with discriminative features of candidates.

(2) and the proposed Poisson-based prior in Eq.


Their algorithm assigned a candidate from a document to a cluster based on the candidate's feature similarity with candidates from other documents only.

to investigate the possible reasons for this, we examine feature values in the centriods of the two clusters learned by the three models.


Furthermore, by combining this idea with a straightforward machine learning approach, the overall performance of our method is on par with state-of-the-art pattern based methods.

to obtain the final output of our method we assign each unique noun pair the maximum score from all (noun pair, sentence) triples it occurs in, and discard all other sentences for this noun pair.


Candidate Generator This module generates sentences containing candidate word pairs for the target relation from the corpus.

the main contribution of semantic word classes and partial patterns to our method's performance lies not in the final classification step but seems to occur at earlier stages of the process, in the candidate and training data generation steps.


In all cases, the combinations surpass the results of both methods separately.

source word and target word also belong to the same sense.


Those words are delimited by a window (±5 words around w) and punctuation marks.

external knowledge needs to be used in order to guide mapping according to sense.


Cohesion penalty counts the total occurrences of unresolved words for a translation hypothesis, which involve newly encountered unresolved words as well as old unresolved words carried on from the derivation history.

we used a parallel training corpus with 2.1 million Chinese-English sentence pairs, aligned by GIZA++.


The window size restrictions mean that some reference alignments are not reachable from the starting point.

both the training and decoding of full-sentence models have presented difficulties in the past, and approximations are necessary.


source documents similar to the test document and their corresponding target documents) in the training parallel corpus; 3) a topic cache, which stores the target-side topic words related with the test document in the source-side.

if there are translation pairs of sentences/phrases/words in similar bilingual document pairs, this makes the translation much easier.


In this way, we can extract a set of phrase pairs for each bilingual document pairs.

a static cache feature F is designed to capture useful information in the static cache in the same way as the dynamic cache feature, shown in Formula (5).


In principle, the cache-based approach can be well suited for document-level translation.

detailed analysis indicates that the dynamic cache does have negative effect on about one third of documents, largely due to the instability of the dynamic cache at the beginning of translating a document.


Table 1 shows some bilingual phrase pairs extracted from a document pair, which reports a piece of news about "impact on slowdown in US economic growth".

a specific feature is incorporated S to capture useful documentlevel information in the dynamic cache:  4where i e - ¶ is a decay factor to avoid the dependence of the feature's contribution on the cache size.


We cannot compute the sum- requires to know x i .

in future work, we plan to test our method in settings where there are large amounts of monolingual training data (enabling many discriminative features).


We start with 2000 non-zero entries in the covariance matrices and experiment up to 20,000 non-zero entries.

absolute thresholding tends to remove all the less frequent word pairs while leaving the cooccurrences of the frequent words untouched.


In this case, without the constraint of S, the matrix factorization may generate incorrect class alignment, thus lead to almost random classification results (around 50% accuracy), as shown in Figure 1, and further make unreasonable the following evaluation of queries.

when a query q j is a word, P (q j = k) is otherwise, Computational Improvement: It can be computationally intensive if the reconstruction error is computed for all unknown documents and words.


With the study of learning from labeled features, many research efforts on active learning with feature supervision are also reported (Melville et al., 2005;Raghavan et al., 2006).

in our setting of Tri-NMF, orthogonal constraints are enforced on F, G and Euclidean distance is used as the objective function.


Similarly, we can discretize the punctuation variety features.

this criterion evaluates how independently a string is used, and thus how likely it is that the string can be a word.


The combination of the "AV" and "PU" features gives further helps.

if it does not appear in the training data, it is very hard for statistical models to recognize this word.


We further propose three methods for enhancing the LM LL baseline using P x (e, o) to approximate Equation 8.

this method also generates an enhanced probabilistic resolution of logical metonymies.


cLDA's C 2 had an average degradation of only 2.5 points and C 3 was actually improved by 2.1 points.

since a smaller, higher precision list may be useful for many applications we still consider MAP a valid metric and indicate both the highest scoring method and the highest scoring non-threshold method.


Third, analyses will be carried out to uncover the differences between feature combination and integration that have led to different accuracies.

this paper introduces a method of harnessing different knowledge sources under a uniform model for measuring semantic relatedness between words or concepts.


in this method, we follow the work by Rowe and Ciravegna (2010) to set t=2 in order to preserve locally connected nodes.


For example, WordNet and Wikipedia contain features based on synonyms of concepts; while Wikipedia and unstructured documents contain word distributional features.

we search each word in wordNet and extract all possible synsets.


We therefore resort to addressing the issue of information loss on specificity by inheritance.

even methods as simple as "hop counts" between nodes (e.g., that of Rada et al.


such a range might be too narrow to have any real distinguishing power over similarity.


WordNet, or a machine readable thesaurus).

dinu and Lapata's model, which performs similarity calculations in the latent space, is not able to provide accurate word vectors, and thus perform worse at the paraphrase induction task.


We then evaluated our methods on the hierarchical test sets T2 and T3.

2) In order to minimise the problem of polysemy, we assigned each verb to the class which, according to VerbNet, corresponds to its predominant sense in WordNet (Miller, 1995).


The basic idea of our structures is to use (i) one of the three kinds of information above as central node, from which depen- We define three main trees: the PoS-Tag Centered Tree (PCT), e.g.

the worst case complexity of the SPTK is identical to PTK, i.e.


The T → C model for a set of contexts C is: With appropriate priors chosen for the distributions over words and latent variables, P (n|o, C) is a fully generative model of lexical substitution.


The literature on distributional semantics is vast; in this section we focus on outlining the research that is most directly related to capturing effects of context and compositionality.

we follow recent work (Dinu and Lapata, 2010;Ó Séaghdha, 2010;Ritter et al., 2010) in assuming a latent variable model that associates contexts with distributions over a shared set of variables and associates each variable with a distribution over the vocabulary of word types: The set of latent variables Z is typically much smaller than the vocabulary size; this induces a (soft) clustering of the vocabulary.


According to Nissim et al., their definitions are built upon Prince's (1981), and the categorization into old, new, and mediated entities resemble those of Strube (1998) and Eckert and Strube (2001).


Results on 147 Switchboard dialogues demonstrated the effectiveness of these features: we obtained a significant improvement of 8.1% in accuracy over a information-status classifier trained on Nissim's feature set.

there is only one feature that captures the syntactic context of an NP, grammatical role, which is computed based on the parse tree in which the NP resides.


A good metric which covers all POS categories can provide substantial improvement in the classification accuracy.

a user will infer the negative polarity of the first sentence from the negative sense of 'fell' in it while the user will state that the second sentence does not carry any sentiment.


For a WSD engine with a better accuracy, the performance of sensebased SA can be boosted further.

leacock and Chodorow (lCH): To measure similarity between two concepts A and B, leacock and Chodorow (1998) compute the shortest path through hypernymy relation between them under the constraint that there exists such a path.


Likewise, predicates whose THEME contains words with mostly positive (or negative) connotation are likely to be positively (or negatively) connotative predicates.


Notice that the use of Top 6k words from our connotation lexicon along with OpinionFinder lexicon boost the performance up to 78.0%, which is significantly better than than 71.4% using only the conventional OpinionFinder lexicon.

we investigate the use of HITS algorithm (Section 4), and PageRank (Section 5).


A strategy to decide when the reranking model should be applied and when the first hypothesis of the baseline model is more accurate would improve reranking performances.

in order to give an effective representation to SLU hypotheses in SVM, since we are using PTK, we need to represent as trees SLU hypotheses like the one described in section 2.


The output of the algorithm is a word string, W * , such that it is the concatenation of final decoded word strings from each sub lattice.

we will continue to follow the notation introduced in section 3.1.


Table 4 shows the perplexity results by NNLM itself and the interpolated results are shown in Table 5.

the resulting training set for each binary classifier is a stratified sample (Neyman, 1934), and our estimate needs to be calibrated to account for this.


With binary NNLM, we are able to retain all the gain after interpolation with only 20% of the negative examples.

for the standard NNLM, it means only a subset of the data is seen by the model and it does not change through epochs; for binary NNLM, it means the subset of negative examples for each binary classifier does not change.


Previous research has shown the benefit of including various kinds of syntactic and semantic information into the LM (Khudanpur and Wu, 2000).

note that we can achieve 0.5% absolute WER improvement on the test set at only 1% of the standard MELM complexity.


In fact, the short-list approach in (Schwenk, 2007) and the adaptive importance sampling in (Bengio and Senecal, 2008) have exactly this intuition.

in order to train the classifiers, we need to compute all V output nodes and propagate the errors back.


Obviously, with more machines, the estimation can be even faster, it's also reasonable to expect that with more kinds of features, the improvement can be even larger.

taking P (w|h) = P b (w|h) from ML trained binary classifiers gives an LM that meets the MLE constraints for language models.


The variables S jl is a binary indicator of the sentence.

they use the same KL-divergence based sentence selection, but topic model they use are different, and also BL-3 has a sentence ranking process.


We also introduce an aspect distribution θ that controls how often a general or a specific aspect occurs in the collection, where θ is sampled from another Dirichlet prior with parameter α.

we assume that each sentence belongs to both a general aspect and a specific aspect.


The attributes can be derived from the property-level annotation given in the corpus.

this paper is not the first to look at how the algorithmic approach and the alignment approach might be integrated in REG.


1 However, we do include all of these instances in the feature extraction step, on the assumption that they might impact on the content of subsequent references.

contrary to findings from other corpora, in which colour was used much more frequently (Gatt, 2007;Viethen and Dale, 2008), the colour attribute was used in only 26.3% of the referring expressions in our data set.


2 This is precisely the consideration on which the initial work on referring expression generation was based, inspired by Grosz and Sidner's (1986) observations about how the changing attentional structure of a discourse moves different entities in and out of focus.

in addition to the main prediction class content pattern, the split was stratified for Speaker iD and Quartile to ensure that training and test set contained the same proportion of descriptions from each speaker and each quartile of the dialogues.


provide Japan decontamination equipment Note how our system extracts a dative shift rule for to give and a rule that both shifts and substitutes a more appropriate verb for to provide.

source in this war which has carried on for the last 12 days , around 700 palestinians , which include a large number of women and children , have died .


To assess the output quality of the resulting sentence compression system, we compare it to two state-ofthe-art sentence compression systems.

further augmenting the grammar with deletion rules significantly helps retain the core meaning at compression rates this high, however compared to the un-augmented syntactic system grammaticality scores drop.


This column does not apply to implicit determiners, as the subtype information is unavailable.

the term ASt is borrowed from compilers (Aho et al., 1986), where it is used as an intermediate step in the semantic interpretation of programs.


The important feature of this example is that the determiner "any" is syntactically embedded in the restrictor of IMP in the PPT (Figure 4), but it outscopes the implicit determiner in the AST (Figure 3).

the rest of this paper is organized as follows.


This distinction is of key importance in the application of conformance checking, as it helps determine the facts that need to be provided by an organization (de re), and the actions that an organization is required to take (de dicto).

to estimate inter-annotator agreement, a set of 32 sentences was annotated by a second annotator.


(2007) have shown that, after a certain number of initial iterations, Newton's method adds a fixed number of bits to the precision of the approximated solution, even in the above mentioned cases in which the fixed-point iteration method shows exponential time behaviour.

our investigations so far have not found a positive answer to this second question.


This probability is defined as the possibly infinite sum of the probabilities of all strings of the form xwy, for any pair of strings x and y over the alphabet of G. Besides applications in computation of the probability distribution for the next word token and in word error correction, infix probabilities can also be exploited in speech understanding systems to score partial hypotheses in algorithms based on beam search, as discussed by Corazza et al.

the algorithm in (Fred, 2000) does not apply to cases with multiple occurrences of the given infix within a string in the language, which is what was pointed out to be the problematic case.


The intuition is that I w reads a string In more detail, for each i (1 ≤ i ≤ n) and each a ∈ Σ, there is a transition t i−1 a → t j , where j is the length of the longest string that is both a prefix of w and a suffix of a 1 • • • a i−1 a.

the systems of equations for C 1 and C 3 change fundamentally if the infix is extended by one more symbol, which seems to at least make incremental computation very difficult, if not impossible.


From T a set D of dependent nodes are chosen for attachment correction.

one method is to let every other node in T be a candidate governor for d. parser error analysis has shown that errors often occur in local contexts.


For each b 1 ∈ Σ , we define: We then have, for every b ∈ Σ : Furthermore, we have: , and: The rest of the parameter updates can easily be derived using the above updates because of the sumto-1 constraints in Eq.

in standard practice for natural language parsing, transitions are always specified by some finite mean.


Like the brute force and rule-based systems, it is vulnerable  Figure 3: CI probability that the target is category C, given possible categories for result (R) and sister (S).

we trained the initial parser on the CCG-Bank (Hockenmaier and Steedman, 2007;Hockenmaier, 2003) training set ( §02-21), consisting of 39603 sentences of wall Street Journal text (Marcus et al., 1993).


When run incrementally over this toy corpus, both the RB and CI algorithms converge to the target lexicon in an identical sigmoid learning curve (not shown).

for the top-down process, the standard combinators have to be reformulated to take an argument and a result as inputs, rather than two arguments as in the standard bottomup case.


Statistical techniques are vital to many aspects of computational linguistics (Johnson, 2009;Charniak, 1997;Abney, 1996, inter alia).

logarithms of the final counts better fit the observed data.


Our role labeler is fully unsupervised with respect to both tasks -it does not rely on any role annotated data or semantic resources.

our implementation allocates up to N = 21 clusters 5 for each verb, one for each of the 20 most frequent syntactic functions and a default cluster for all other functions.


Example 2: Takes good pictures during the daytime.

a similar approximation method is also used in (Mcdonald and Pereira, 2006) for acyclic dependency graphs.


Based on the definition of opinion unit proposed by Hu and Liu (2004), from the first example, the information we can get is the author's negative opinion about "interior" using an opinion expression "noisy".

the following are our constraints: c1: Constraint on edges in opinion thread (10)-(11).


The tag set is simply {B, I, O} which are signs of begin, inside, outside of a vertex.

through data analysis, we observe that 60.5% of sentences in our corpus do not follow the assumption used by them.


Traditional summarization involves two essential requirements: (1) coverage: the summary should keep alignment with the source collection, which is proved to be significant (Li et al., 2009).

we present an important and novel summarization problem, Interactive Personalized Summarization (IPS), which generates summaries based on human−system interaction for "interests" and personalization.


Note that the inconsistency among evaluators is larger than that between RefSum and supervised summaries, indicating interests take a high proportion in evaluator supervised summaries.

for the readers, the impact of human interests has seldom been considered.


Personalization based on user preference can be captured via various alternative ways, such as eyetracking or mouse-tracking instruments used in (Guo and Agichtein, 2010).

we empirically choose k=3 for the examined context window.


The classic tuning architecture used in the dominant MERT approach (Och, 2003) forms the translation subset and learns weight vector w via Source Sentence candidate generation: choose index pairs (i, j); for each, add j to J (i) optimization: find vector w that minimizes l s (H w , G) 5: return w  Figure 2 shows the pseudocode.

we repeated the scalability study from Section 3, now using our pairwise ranking optimization (hereafter, PRO) approach.


Finally, our approach is robust to local word editing.

we chose to evaluate quality on test corpora to ensure that degradations are not hidden by imperfectly matched web corpora and are consistent with the kind of results often reported for machine translation systems.


AG: K Argument , DP: K P ath , BOW: K BOW , TF k : K T F k .

as pointed out in (Nguyen et al., 2009), such heuristics are tuned on the target relation extraction task and might not be appropriate to compare against the automatic learning approaches.


The core entities in the MUC-6 task (people and organizations) had been worked on previously.

the drugs), but not always feasible; for instance, reports of new research may appear without a name for the drug.


Overall, the system did better on relations having more correct answers.

all of our relations included at least one novel class.


This classifier runs on sentences from a web corpus to extract millions of relational tuples.

we use approximately 22,000 category instances belonging to 122 categories extracted by CPL at the end of its 20 th iteration and the web corpus as input to perform the co-clustering described in Section 3.2 and generate the new relations.


As described in the text, each entry gives the normalized count of the number of known <drug, disease> pairs that occur with both the row context and the column context.

of the 11 million high confident relational tuples extracted by this system only 1 million were concrete facts (Banko, Cararella et al.


Figure 3 shows learning curves, in terms of accuracy vs. annotation time, for each trial in the user study.

we experiment with EM 1 (one-step EM) variants of both MNB/Pool and MNB/Priors, and set α = 50 for the latter (see Section 3.2 for details on tuning this parameter).


Following the same methodology as Section 3.4.2, we evaluated 300 random predictions using the Mechanical Turk service.

to quickly build an English-language filter for twitter, we sampled 150,000 tweets from the twitter Streaming API and asked an annotator spend 10 minutes with DUALISt labeling English and non-English messages and features.


If we think of MaxEnt/GE as a discriminative analog of MNB/Priors+EM, this is consistent with what is known about labeled set size in supervised learning for generative/discriminative pairs (Ng and Jordan, 2002).

perhaps this can be improved by using an ensemble , but that comes at further computational expense.


By releasing DUALIST as an open-source tool, we hope to facilitate language annotation projects and encourage more user experiments in active learning.

our results were achieved in less than 10 minutes of effort each, by labeling an average of 76 sentences and 32 words or phrases per task (compared to the thousands of labeled training sentences used in previous work).


We have presented DUALIST, a new type of dualstrategy annotation interface for semi-supervised active learning.

p (y j |x) ∈ {0, 1} for x ∈ L, and p (y j |x) = p θ (y j |x) for x ∈ U.


Our search algorithm is based on the third-order parsing algorithm, but, the search space is previously shrank by a baseline parser's k-best approximation and a forest pruning algorithm presented in the next section.

(2009) and we apply it to a dependency parser in order to select better candidates with third-order information.


Note that for data point d j i = (x, y), which is the i th training instance of the j th data set, that y does not necessarily have to be a dependency tree.

in Section 3 and 4 we present a set of experiments defining diffent augmented losses covering a task-specific extrinsic loss (MT reordering), a domain adaptation loss, and an alternate intrinsic parser loss.


A parse with perfect scores under ALS and LAS will match the gold-standard training tree.

in this paper, we propose a training algorithm for statistical dependency parsers (Kübler et al., 2009) in which a single model is jointly optimized for a regular supervised training objective over the treebank data as well as a task-specific objective -or more generally an extrinsic objective -on an additional data set.


only a small active set of features (those that have nonzero weights) need to be maintained.


Following previous work (Kwiatkowski et al., 2010), we make use of a higher-order unification learning scheme that defines a space of CCG grammars consistent with the (sentence, logical form) training pairs.

a traditional CCG grammar includes a lexicon Λ with entries like the following: where each lexical item w X : h has words w, a syntactic category X, and a logical form h. For the first example, these are "flights," N, and λ x. f light(x).


We present the complete approach in three parts by describing the factored representation of the lexicon (Section 5), techniques for proposing potential new lexemes and templates (Section 6), and finally a complete learning algorithm (Section 7).

first the NEW-LEX procedure is run on y * as described in Section 6.1 to generate new lexical items.


(2010) apply a minimally supervised approach to extracting entities from text advertisements.

t-SEG models Named Entity Segmentation as a sequence-labeling task using IOB encoding for representing segmentations (each word either begins, is inside, or is outside of a named entity), and uses Conditional Random Fields for learning and inference.


We observe that T-NER again outperforms co-training.

we perform hierarchical clustering using Jcluster (Goodman, 2001) on 52 million tweets; each word is uniquely represented by a bit string based on the path from the root of the resulting hierarchy to the word's leaf.


This results in a set of approximately 1.7 million distinct normalized relation phrases, which are stored in memory at extraction time.

figure 3 shows the precision-recall curves of the systems introduced in this paper.


As Downey's work predicts, precision increased in both systems for extractions found multiple times, compared with extractions found only once.

the syntactic constraint reduces uninformative extractions by capturing relation phrases expressed via LVCs.


We have investigated the use of AL in a real-life annotation experiment with human annotators instead of traditional simulations with gold labels for named entity recognition and sentiment classification.

aL depends on accurate assessments of uncertainty and informativeness and such an accurate assessment is made more difficult if labels are noisy as is the case in crowdsourcing.


∞ uses all words in each context direction.

we can find many previous studies of the topic in the relevant literature.


We will ensemble WTM with other contentbased and collaboration-based methods to build a practical social tag suggestion system.

when given the description of a resource, we can rank tags by computing the scores: in which Pr(w|d) is the trigger power of the word w in the description, which indicates the importance of the word.


The F-measure of EWTM is only F = 0.229 compared with WTM F = 0.267.

the content-based approach plays an important role in social tag suggestion.


OWL-QN is based on the fact that when restricted to a single orthant, the L 1 regularizer is differentiable, and is in fact a linear function of θ.

our approach is based on building different Bayes classifiers as high level features and then learning a linear function of these classifiers for retrieval in the first task and classification in the second.


This motivates us to extract grammar rules from the λ-hybrid trees.

many early works do not rely on probabilistic approaches.


The fourth and fifth rows of Table 4 also show the multilingual results.

'France' has two different IPA sequences in English (Table 2).


We use the phoneme dictionaries in each language to train our approach, as well as the baseline system.

our approach successfully handles the language specific phonemic variation.


Table 2 shows that the proposed methods outperform the baselines both when using Lex S and using Lex L .

(psychosis)"," (neurosis)"), and then finds the correct translation equivalent.


Some of the previous methods tried to alleviate the problem of the limited seed lexicon size (Koehn and Knight, 2002;Morin and Prochasson, 2011;Hazem et al., 2011), while others did not require any seed lexicon (Rapp, 1995;Fung, 1995;Haghighi et al., 2008;Ismail and Manandhar, 2010;Daumé III and Jagarlamudi, 2011).

the first of these types are words whose equivalent does not exist in the English corpus.


There are three context words for the query.

we thank anonymous reviewers of EMNLP-CoNLL 2012 for helpful suggestions and comments on a first version of this paper.


We adopt LP to obtain relations with all bilingual seeds including indirect relations by treating each seed as a label.

the patent translation task is a task that requires bilingual lexicon extraction from non-parallel data.


In addition, for each word in the vocabulary, we apply the presence of the repetition about one particular word.

such targeted analysis is more helpful for medical professionals as they search to develop new therapies and ways to track patient status between visits.


The most informative features are: percentage of negative posts per tread, percentage of negative sentences per post, percentage of positive posts per thread, number of negative posts, and discussion domain.

it has been shown to hold both theoretically (Heider, 1946) and empirically (Leskovec et al., 2010c).


Since Baseline 1 barges-in on users after an information peak, when ID may still be high, it continuously receives a negative reward reflecting the user preference for late barge-ins.

this algorithm takes a set of strings as input and computes a context-free grammar as output by aligning strings based on Minimum Edit Distance.


If we assume a priori that the length of a block is unbounded, then this state space is N K where 0 ∈ N. The probability of transitioning from a state b to another stateb ∈ N K is: where N = kb k , ζ N is the probability that a block has N tokens, 2 and π(b) is the transition distribution given a vector b.

we extend this approach to allow each block of text to be a mixture of multiple classes.


(2011b) experimented with similar tasks using a related HMMbased model called the Structural Topic Model.

we wish to learn arbitrary transitions, both positive and negative, between the latent classes.


In the sports domain, Count Att will add together counts for appearances of a player's height, position, salary, etc..

we will assume that databases are provided in a particular form, called Boyce-Codd Normal Form (BCNF) (Silberschatz et al., 2010).


This metric allows systems to resolve against either reference, Wikipedia or another database, without requiring it to match both if the same entity appears in both references.

the distant supervision model has the ability to place more weight on attributes like director rather than genre, or team rather than position, if those attributes are more discriminative.


This greatly improves the effectiveness of our active learning approach.

the classifier used for selecting balanced samples is the same as the one for supervising uncertainty, which makes the balance control unreliable (the selected uncertainty samples take very low confidences which are unreliable to correctly predict the class label for controlling the balance).


Ding (Ding et al., 2008) proposed a holistic lexicon-based approach of using global information to solve this problem.

that's because two kinds of collocations' polarities cannot be disambiguated.


Therefore, this can significantly increase the recall value, and further produce a better result.

the majority polarity of all the pseudo contexts is chosen as the final polarity.


On the other hand, any word w grammatically related to the PMW via a grammatical relation r provides us with semantic restrictions on the interpretation of the PMW, namely preferred semantic classes A j (we call them abstractions) and a selectional preference score.

this ranking showcases our second major innovation in that the flexibility of our framework allows us to incorporate a wider context than in most prior approaches.


In this paper we generalize the approach of Hsu et al.

where K ijk = K(δ j ) ik are the elements of the tensor K().


More formally, let s(i) be the maximal set of continuous indices j ≥ i including i such that, if j = i, c j = 0.

α and β are hyperparameters to the Dirichlet priors.


Model precision is low in all models, which is a reflection of how challenging the task is on a small corpus laden with proper nouns and low-frequency words.

in contrast with TNG, the word Figure 3: Illustration of the hierarchical Pitman-Yor process for a toy two-word vocabulary V = {honda, civic} and two-topic (T = 2) model with m = 1.


Ergodicity requires that there is non-zero probability that any state in the sampler be reachable from any other state.

the PYP-SCFG columns show results for the 500th sample for both MODEL 1 and HMM initialisations.


This is achieved by obtaining the optimal state value function, V * (s), because if we obtain this, the greedy policy is optimal, which determines the action so as to maximize the state value after the transition occurred.

• The learning of the agent only depends on the reward provided by the environment.


The experimental results demonstrated ASRL tends to converge sub-optimally, and excessively depends on the formulation of features and the score function.

iLP has to take into account the combinations of textual units, whose number increases exponentially.


Fortunately, for Bayesian decipherment, we are able to complete the second step efficiently.

for any word W in a bigram, we need only 2|V | lists ("words to the right of W" and "words to the left of W") instead of |V | 2 lists ("pairs of words that surround W").


Once obtaining a completed hypothesis, the decoder will pass it to the Stanford POS tagger and according to tense verbs to get all tense sequence for this hypothesis.

they are determined by all tense verbs in this sentence according to the strict topdown order.


most of current Statistical Machine Translation (SMT) systems mainly depend on translation model and language model.


This statistics conform to the previous observation (3) very much.

our method will fail when some special Chinese sentences only contain noun predicates.


In the confusion network, the weight of the original Malay word is set to 1, while the weight of a morphological option is one minus the minimum edit distance ratio (Ristad and Yianilos, 1998) between it and the Malay word, multiplied by the highest probability for all pivoting variants for the Malay word.

table 6 shows that CN:pivot,t=0 is better/equal to the original 53%/31% of the time.


We believe this is because lattice encodes many options, but does not use a Malay LM, while 1-best uses a Malay LM, but has to commit to 1-best.

each translation pair is associated with a conditional probability, estimated by pivoting over english: Pr(i|e) and Pr(e|m) are estimated using maximum likelihood from the word alignments.


Training on the adapted large bi-text yields very significant improvements in translation quality compared to both (a) training on the unadapted version, and (b) training on the small bi-text for the resourcepoor language.

cN:pivot has lower unigram precision, which could be due to bad word alignments, as the results for cN:pivot ′ show.


The necessity to search for the whole sentences in the corpus and not only for some smaller context (considering, for example, just left and right neigh-bor), which would lead to lower sparsity, is rationalized by the following example: Their children went to school.

the sampler converges to similar results for both the initializations.


table 3: Labelled precision of the arcs built by each transition of Nivre's arc-eager parser without modification (NE), with a projective buffer transition added (NE+LBA, NE+RBA) and with both projective buffer transitions added (NE+LBA+RBA).


The projective buffer transitions presented here bear some resemblance to the easy-first parser by Goldberg and Elhadad (2010), which allows creation of dependency arcs between any pair of contiguous nodes and is based on the idea of "easy" dependency links being created first.

to test this idea, we also conducted experiments with the alternative policy "use buffer transitions whenever possible, regardless of arc length": as expected, the obtained accuracies were (slightly) worse.


While we have chosen Nivre's well-known arceager parser as our baseline, we have shown that these transitions can be added to any stack-based dependency parser, and we are not aware of any specific property of arc-eager that would make them work better in practice on this parser than on others.

the reader should be aware that the purpose of this table is only to provide a broad idea of how our approach performs with respect to a well-known reference point, and not to make a detailed comparison, since the two parsers have not been tuned in homogeneous conditions: on the one hand, we had access to the CoNLL-X test sets which were unavailable System Arabic Danish Nivre et al.


Here the dashed arrows indicate the modifier to the head and the modifier's modifier, forming a modifier chain of length two.


While pronunciation dictionaries and transcribed audio are available for some languages, these resources are entirely lacking for the vast majority of the world's languages.

these results compare quite favorably to several baselines.


In this section, we describe the set of experiments performed to evaluate the performance of our model.

it appears that whatever information they convey is redundant when considering the text-based feature sets.


Our procedure learned this automatically from a collection of name tokens, without observing any input/output pairs.

the matching experiments were performed on a corpus of 1500 entities (with separate corpora of the same size for training and test.)


The EM procedure repeats the following until convergence: E-step: Given θ, compute the posterior marginal probabilities c xy of all possible phylogeny edges.

our distribution over sequences of entities e is exchangeable.


Our result points a way towards a direction for explaining of this phenomenon by demonstrating that the differences between currenttechnology artificial speech and natural speech can be partially explained through higher-level syntactic features.

w i ; see also (Levy, 2008;Demberg and Keller, 2008).


The other was taken from a 151 million-word (4.3 million fullparagraph) sample of the English Gigaword corpus.

we use word duration estimates from the state-of-the-art open-source text-to-speech system MARY (Schröder et al., 2008, version 4.3.1), with the default voice package included in this version (cmu-slt-hsmm).


this often leads to large stylistic differences between the questions in Yahoo!


We believe that this is mainly because SA@W and SWC are based on semantic and sentiment information at the word level, and these often capture a similar type of information.

table 4 shows the evaluation results of six different systems.


Type constraints are respected (through q-edges): The above is a sophisticated ILP, and most likely NP-hard.

for both datasets, coverage is high as few mappings are missing.


Also, product questions mainly asked for majority opinions, and majority type was thus not considered.

a name entity recognizer for product is learned on these data, with unigrams and POS tags as features.


In order to capture the sequential relations among sentences, we utilize features as the Cartesian product over the terms of N-gram (N=1,2) and POS tags generated from sentences i and j (Lapata et al., 2003).


Since the consumer reviews often include multiple sentences, we can directly use the adjacency of these sentences as training samples.

we employed the product review dataset used in  as corpus.


They argued that product questions were seldom asked for the holders, since the holders (i.e.

for the questions asking for reason and attitude, we generate the answers by summarizing corresponding answer fragments.


In statistical machine translation, minimum error rate training (MERT) is a standard method for tuning a single weight with regard to a given development data.

the idea of ultraconservative update can be formalized as follows: is a distance metric over a pair of weights W and W b .


Experiments on NIST Chinese-to-English translation tasks show that our local training method significantly gains over MERT, with the maximum improvements up to 2.0 BLEU, and its efficiency is comparable to that of the global training method.

it is inefficient since tuning requires iteratively decoding an entire development set, which is impractical for an online service.


Considering the fact that today some corpora for word segmentation are really large (usually tens of thousands of sentences), it is necessary to obtain the latest CTB and investigate whether and how much does annotation transformation bring improvement to a much higher baseline.

the key idea of co-training is utilize the complementarity of different parsing models to mine additional training data from raw text, while iterative training for annotation transformation emphasizes the iterative optimization of the parellelly annotated corpora used  to train the transformation models.


The training procedure learns a discriminative model mapping from the inputs x ∈ X to the outputs y ∈ Y , where X is the set of sentences in the training corpus and Y is the set of corresponding labeled results.

it is valuable to conduct experiments with more source-annotated training data, such as the SIGHAN dataset, to investigate the trend of improvement along with the increment of the additional annotated sentences.


We notice the CG is almost identical for the top combinations.

compared to these methods, S-dict is not competitive in terms of either precision or recall.


Dictionary lookup is a type-based approach to normalisation, i.e., every token instance of a given type will always be normalised in the same way.

as N (the lexicon size cut-off) increases, the performance drops more slowly than the other methods.


They approach lexical variant detection by using a context fitness classifier (Han and Baldwin, 2011) or through dictionary lookup (Gouws et al., 2011).

given that we focus on longer OOVs (specifically those longer than 4 characters), this ultimately isn't a great handicap.


This is because the redundancy in the human generated instructions allows an incorrect semantic parse to be successful, as long as the errors do not affect its ability to guide the system to the correct destination.

the trained semantic parser can parse a novel instruction into a formal plan, which is finally executed for end-to-end evaluation.


By contrast, the previous PCFG approach requires a finite MR language and its grammar grows intractably large for even moderately complex MR languages.

we utilize the hierarchical subgraph relationships between the MRs in the learned semantic lexicon to produce a smaller, more focused set of PCFG rules.


Based on them, a monolingual parsing model and a tree reordering model are trained to pre-reorder source words into the targetlanguage-like order.

all terminal translation states of the sentence pair {f, e} are disjoint but cover f [0,J+1) and e [0,I+1) at the same time, where J = |f | and I = |e|, and all non-terminal translation states correspond to the partial decoding states generated during decoding.


• h r (G), number of translation rules used in G. • h d (G), structure-based score of G. For FDTs generated by phrase-based paradigms, it can be computed by distortion models; while for FDTs generated by syntax-based paradigms, it can be computed by either parsing models or syntactic LMs (Charniak et al., 2003).

from Table 1 we notice that with FDT-SLM integrated, only 0.2 BLEU improvements can be obtained.


As discussed in Section 4.3, this model follows the "traditional" distant supervision heuristic, similarly to (Mintz et al., 2009).

note that this heuristic repeats relation mentions with different labels for the tuples that participate in multiple relations.


(2011a) proposed models where N i for the ith tuple (e 1 , e 2 ) is defined as: {r j | r j (e 1 , e k ) ∈ D, e k = e 2 , r j / ∈ P i }, which is a subset of L \ P i .

z i changes as the algorithm progresses, which may impact the label assigned to the remaining mentions in that group.


The improvement is approximately 1 F1 point throughout the last segment of the curve.

this introduces a challenging learning scenario where the relation expressed by a pair of entities found in a sentence is unknown.


The first of them is a classic readability formula by Kandel and Moles (1958), which is an adaptation of the Flesch (1948) formula for French: where Y is a readability score ranging from 100 (easiest) to 0 (harder); lp is the average number of words per sentence and lm is the average number of syllables per 100 words.

it seemed unlikely that the information collected by means of syntactic parsers, which are still committing a significant number of errors, at least for French, would belie these findings.


However, the assumption of upward movement in the phrase structure tree should rule out movement into the projection interval of a non-ancestor.

m/D If p has multiple children in T and i and j are descended from different children in T , then there is a split point k such that T 's score is T has the maximum score over each of the above cases, for all valid choices of x and k. Note that for subproblems rooted at p, g is the grandparent index, while for subproblems rooted at x, p is the updated grandparent index.


6 The annotation guidelines and the generated corpus are available here.

all verbal mentions are still in singleton clusters after this step.


This is mainly because of a better parsing module as well as joint decoding.

to facilitate decoding, we unify the form of spans so that each span contains exactly one unary rule.


We measure the utility of our proposed LM adaptation approach in two ways: (a) comparing reference translations based perplexity of adapted LMs with the generic LM, and (b) comparing SMT performance of adapted LMs with the generic LM.

then, we select the top-N sentences which are similar to the development set, estimate the bias 4-gram LMs (with n-gram cutoffs tuned as above) with these selected sentences, and interpolate with the generic 4-gram LM as the adapted LMs.


The average number is reduced when unimportant words are eliminated, from 19 to 12 on IWSLT-07 and from 37 to 24 on NIST-06, respectively.

selecting a sentence to maximize P (S|Q) is equivalent to selecting a sentence that maximizes P (Q|S).


A third difference is in the scale of the training -REVERB yields millions of training seeds, where previous systems had orders of magnitude less.

since the relation words in the sentence and seed match, we can learn general pattern templates that may apply to other relations too.


The rest are due to other causes such as OLLIE's ability to handle relationships mediated by nouns and adjectives, or REVERB's shallow syntactic analysis, etc.

these are relation-independent dependency parse-tree patterns that are automatically learned using a novel bootstrapped training set.


The Gibbs sampler iterates over words, doing a blocked sample of (z i,j,l , u i,j,l ).

note, however, that the table indicator (0, j ) for j < j is equivalent to the table indicator (1, j + 1) as far as statistics is concerned.


Figure 6(a) shows this for LDA run on paragraphs of "The Prince".

we need to depart from the earlier HMM style models, see, e.g., (Blei and Moreno, 2001;Gruber et al., 2007).


The rate at which embeddings were learned ranged from 3.4 × 10 −10 to 6.7 × 10 −10 to 10 −9 .

for all of our experiments we employ column vectors from a Cartesian, finitely-dimensional space.


Table 2 shows the 11 contexts (l, v) that appear most frequently in T .

where In our experiments we compose word vectors to create representations for phrase vectors and sentence vectors.


As pointed out in Section 3.5, the majority of sentences require zero or few corrections.

no search or global inference is applied.


the label powerset method of Tsoumakas et al.

we opted for SVM-HMM 2 (Tsochantaridis et al., 2004), which is an implementation of structural support vector machines for sequence tagging.


Both were collections of responses from the AEST, a high-stakes test of English proficiency and had no overlaps.

for each score group, the POS tags were sorted based on the frequencies in training data, and the rank orders were calculated.


The bigram-based features outperformed both unigram-based and trigram-based features.

the results were based on the individual model and we did not combine any models.


Since most sentences in our data are declarative sentences, 'Wh' phrase signals the use of relative clause.

next, responses in the same  Table 4: Pearson correlation coefficients between ngram-based features and expert proficiency scores ** Correlation is significant at the 0.01 level score group were concatenated, and a single vector was generated for each score group.


Finally, we calculated Pearson correlation coefficients between these features and human proficiency scores.


In a related work (Baba and Suzuki, 2012), we collected such keystroke data using Amazon's Mechanical Turk for English and Japanese, and released the error-correction pairs for research purposes.

error-tolerant word completion and next word prediction are also highly desirable features for text input on software (onscreen) keyboards for any language, making the current work relevant beyond Chinese and Japanese.


Its precision is more than 10% lower than PROP cont at the top one million results.

they become almost contradictory when their tendencies get stronger (i.e., emit smells more strongly ⊥ thoroughly reduce smells).


Most methods that exploit monolingual corpora rely on the Distributional Hypothesis (Harris, 1968): expressions that appear in similar contexts are expected to have similar meaning.

for instance, the algorithm explained in Koehn (2009, p.134) extracts both "dass" and ", dass" as counterparts of "that" from the sentence pair.