BPMs try to cancel out overfitting caused by the order of examples, by training several models by shuffling the training examples.

4 it is very time consuming to run the complete training process several times.


We used n = 20 (n of the n-best) for training since we could not use too a large n because it would have slowed down training.

we could examine a larger n during testing, since the testing time did not dominate the time for the experiment.


Although the averaged perceptron outperformed the perceptron, the improvement was slight.

the margin perceptron greatly outperformed compared to the averaged perceptron.


This comparison, of course, is not fair because the setting was different.

we think the results demonstrate a potential of our new algorithm.


Our algorithm could at least improve the accuracy of NER with non-local features and it was indicated that our algorithm was superior to the re-ranking approach in terms of accuracy and training cost.

the achieved accuracy was not better than that of related work (Finkel et al., 2005;Krishnan and Manning, 2006) based on CRFs.


Product features (e.g., "image quality" for digital camera) in a review are good indicators of review quality.

different product features may refer to the same meaning (e.g., "battery life" and "power"), which will bring redundancy in the study.


Typically, the more data is used to estimate the parameters of the translation model, the better it can approximate the true translation probabilities, which will obviously lead to a higher translation performance.


In addition, the numerical information, abbreviations and the documents' style may be very good indicators of their topic.

this information is no longer available after the dimensionality reduction.


Table 2 shows that "General Motors" or "GM" are not repeated in every sentence of the second story.

"GM", "carmaker" and "company" are semantically related.


In contrast, the C&C tagger, which is based on that of Ratnaparkhi (1996), utilizes a wide range of features and a larger contextual window including the previous two tags and the two previous and two following words.

the C&C tagger cannot train on texts which are not fully tagged for POS, so we use the bigram tagger to produce a completely labeled version of the Wycliffe text and train the C&C tagger on this material.


Compare this with the result for the role patient: "hunter" is further away from "lion" and "deer", and will therefore be found to be a rather bad patient of "shoot".

"hunter" is still more plausible as a patient of "shoot" than e.g., "director".


In one, assigning every datapoint into a single cluster, guarantees perfect completeness -all of the data points that are members of the same class are trivially elements of the same cluster.

this cluster is as unhomogeneous as possible, since all classes are included in this single cluster.


In the perfectly homogeneous case, this value, H(C|K), is 0.

in an imperfect situation, the size of this value, in bits, is dependent on the size of the dataset and the distribution of class sizes.


Therefore, in order for the goodness of two clustering solutions to be compared using one these measures, an identical post-processing algorithm must be used.

this problem can be trivially addressed by fixing the classcluster matching function and including it in the definition of the measure as in H. a second and more critical problem is the "problem of matching" (Meila, 2007).


Transformations like those applied by the adjusted Rand Index and a minor adjustment to the Mirkin measure (see Section 4) can address this problem.

pair matching measures also suffer from distributional problems.


V I depends on the relative sizes of the partitions of C and K, not on the number of points in these partitions.

v I is bounded by the maximum number of clusters in C or K, k * .


Normalizing, V I by log n or 1/2 log k * guarantee this range.

meila (2007) raises two potential problems with this modification.


All evaluated measures satisfy P4 and P7.

rand, Mirkin, Fowlkes-Mallows, Gamma, Jaccard and F-Measure all fail to satisfy P3 and P6 in at least one experimental configuration.


Therefore, model parameter θ learned by PMM can be reasonable over the whole of documents.

if we compute the probability of generating an individual document, a document-specific topic weight bias on mixture ratio is to be considered.


Language models and phrase tables have in common that the probabilities of rare events may be overestimated.

in language modeling probability mass must be redistributed in order to account for the unseen n-grams.


The continuous space model has a much higher complexity than a backoff n-gram.

this can be heavily optimized when rescoring n-best lists, i.e.


The performance of NetSum with third-party features over NetSum without third-party features is statistically significant at 95% confidence.

netSum still outperforms the baseline without thirdparty features, leading us to conclude that Ranknet and simple position and term frequency features contribute the maximum performance gains, but increased ROUGE-1 and ROUGE-2 scores are a clear benefit of third-party features.


Therefore acc(categorizer(D k )) will increase if features that contribute to the categorization appear in D k .

acc(categorizer(D k )) will decrease if no features that contribute to the categorization are in D k .


Moreover, strategies incorporating more selection criteria often require more parameters to be set.

proper parametrization is hard to achieve in real-world applications.


We intentionally avoided using features such as semantic triggers or external dictionary look-ups because they depend a lot on the specific subdomain and entity types being used.

one might add them to fin-tune the final classifier, if needed.


Thus, in our committee we employ ME classifiers to meet requirement 1 (fast selection time cycles).

in the end we want to use the annotated corpora to train a CRF and will thus examine the reusability of such an ME-annotated AL corpus for CRFs (cf.


But even with a selector which has only orthographical features and a tester with many more features -which is actually quite an extreme example and a rather unrealistic scenario for a real-world application -AL is more efficient than random selection.

the limits of reusability are taking shape: on PBvar, the AL selection with sub3 converges with the random selection curve after about 100,000 tokens.


the design of features in both systems using a modular approach, such as (Poesio et al., 2005), where the decision on discourse-newness is taken beforehand, and those that integrate discourse-new classification with the actual resolution of coreferent bridging cases.

to earlier investigations (Markert and Nissim, 2005;Garera and Yarowsky, 2006), we provide a more extensive overview on features and also discuss properties that influence their combinability.


N-gram based language models (LM) estimate the probability of occurrence for a word, given a string of n-1 preceding words.

computers have only recently become powerful enough to estimate probabilities on a reasonable amount of training data.


We show that domain specific language and translation models also benefit statistical machine translation.

there are two problems with using domain specific models.


When the domain D is known, domain specific models can be created and used in the translation decoding process.

in many cases, domain D is unknown or changes dynamically.


When the domain is known in advance, it is usually expressible, for example it could be a topic that matches a human-defined category like "sport".

when the domain is delimited in an unsupervised manner, it is used only as a probabilistic variable and does not need to be expressed.


The points of difference are: • In the cluster language model, clusters are defined by human-defined regular expressions.

with the proposed method, clusters are automatically (without human knowledge) defined and created by the entropy reduction based method.


Furthermore, when P (D|f ) is approximated as P (D) = D λ , and the general translation model P (f |e) is used instead of the domain specific translation model P (f |e, D), this equation represents the process of translation using sentence mixture language models (Iyer et al., 1993) as follows: The points that differ from the proposed method are as follows: • In the sentence mixture model, the mixture weight parameters D λ are constant.

in the proposed method, weight parameters P (D|f ) are estimated separately for each sentence.


phrase-based versus rule-based systems, as shown in (Callison-Burch et al., 2006).

callison-Burch concluded that the Bleu score is reliable for comparing variants of the same machine translation system.


Here, we will use the translation candidate with the maximum Bleu score as pseudo reference to bias the system towards the Bleu score.

as pointed out in (Och, 2003), there is no reason to believe that the resulting parameters are optimal with respect to translation quality measured with the Bleu score.


This seems intuitive as the Bleu score uses n-grams up to length four.

one should be careful here: the differences are rather small, so it might be just statistical noise.


Table 2 demonstrates an example how these rules are applied to translate the foreign sentence "欧元/的/大幅/升值" into the English sentence "the significant appreciation of the Euro".

step Partial derivations Rule  there are always other kinds of bilingual phrases extracted directly from training corpus, such as 欧元, the Euro and 的 大幅 升 值, 's significant appreciation, which can produce different candidate sentence translations.


There are only a few overt pronouns in English, Chinese, and many other languages, and state-of-the-art part-of-speech taggers can successfully recognize most of these overt pronouns.

zero pronouns in Chinese, which are not explicitly marked in a text, are hard to be identified.


Noun phrases provide much fruitful information for anaphoricity identification.

useful information such as gender, number, lexical string, etc, is not available in the case of zero pronouns.


For instance, the overrepresentation of "but" as the sentence-initial CC in the second and third subtree of that figure, is dealt with in (Schmid, 2006) by splitting the CC-category into CC/BUT and CC/AND.

also when a range of such transformations is applied, some subtrees are still greatly overrepresented.


This is the approach we take in this paper, which involves switching from PCFGs to PTSGs.

we cannot simply add overrepresented trees to the treebank PCFG; as is clear from figure 2, many of the overrepresented subtrees are in fact spurious variations of the same constructions (e.g.


Turney's (2006) extensive evaluation of LRA on SAT verbal analogy questions, for example, involves roughly ten thousand relational similarity computations 2 .

our system typically requires millions of relational similarity computations because every pair of extracted word-pairs needs to be compared.


Turney's (2006) Latent Relational Analysis is a corpus-based algorithm that computes the relational similarity between word-pairs with remarkably high accuracy.

lRA is focused solely on the relation-matching problem, and by itself is insufficient for lexical analogy generation.


Latent semantic analysis (LSA) (Landauer et al., 1998) has also been used to measure distributional distance with encouraging results (Rapp, 2003).

it too measures the distance between words and not senses.


Although German is not resourcepoor per se, Gurevych (2005) has observed that the German wordnet GermaNet (Kunze, 2004) (about 60,000 synsets) is less developed than the English WordNet (Fellbaum, 1998) (about 117,000 synsets) with respect to the coverage of lexical items and lexical semantic relations represented therein.

substantial raw corpora are available for the German language.


7 As in the monolingual distributional measures, the distance between two concepts is calculated by first determining their DPs.

in the crosslingual approach, a concept is now glossed by nearsynonymous words in an English thesaurus, whereas its profile is made up of the strengths of association with co-occurring German words.


Recent work has applied random walks to NLP tasks such as PP attachment (Toutanova et al., 2004), word sense disambiguation (Mihalcea, 2005;Tarau et al., 2005), and query expansion (Collins-Thompson and Callan, 2005).

to our knowledge, the literature in NLP has only considered using one stationary distribution per specially-constructed graph as a probability estimator.


The highest probability nodes in the table are common because both model variants share the same initial links.

the orders of the remaining nodes in the stationary distributions are different.


SRL feature extraction has relied on various syntactic representations of input sentences, such as syntactic chunks  and full syntactic parses (Gildea and Jurafsky, 2002).

with features from shallow parsing, previous work (Gildea and Palmer, 2002;Punyakanok et al., 2005b) has shown the necessity of full syntactic parsing for SRL.


Our accuracy is most closely comparable to the 78.63% accuracy achieved on the full task by (Pradhan et al., 2005a).

(Pradhan et al., 2005a) uses some additional information since it deals with incorrect parser output by using multiple parsers.


As stated in their paper, "as a consequence, adjunct semantic roles (ARGM's) are basically absent from our test corpus"; and around 13% complement semantic roles cannot be found in etrees in the gold parses.

we recover all SRLs by exploiting more general paths in the LTAG derivation tree.


When using the parent-child relation, the modifiee can be determined based only on the relation between ID 1 and 5.

when using the ancestor-descendant relation, the modifiee cannot be determined based only on the relation between ID 1 and 5.


These conventional methods select the node given by argmax j P (node j | node i ) as the parent node of node i , setting the beam width to 1.

their processes are essentially the same as the process in PARENT METHOD.


We assume that every bit added as redundancy is correctly transmitted for the above-mentioned discussion.

some of these added bits may be transmitted wrongly in the proposed method.


Figure 6: Pseudocode for FindPCs in the case of conjunction powerset as the non-terminals (adding a few more details like the start symbol) and production rules such as {P2} → P2 {P3, P5}.

for our example the chain of patterns applied P 1 , P 2 , P 3 , P 4 , $ could generate a pattern tree that is incompatible with the original tree.


Figure 3 illustrates the different behaviour of the    Table 6 shows that all error types inserted into Sentence 9 in our test set result in the same evaluation score for the PARSEVAL metric, while the LA metric provides a more discriminative treatment of PP attachment errors, label errors and span errors for the same sentence (Table 6).

the differences in the LA results are only indirectly caused by the different error types.


In traditional approaches for document classification, in many cases, documents are classified independently.

the Wikipedia articles are hypertexts and they have a rich structure that is useful for categorization.


This parameter tying is introduced by Ghamrawi and McCallum (2005).


The results vary from the ordering strategy of local classification.

to iterative classification methods, collective classification methods directly estimate most likely assignments.


Most previous inference algorithms for DP-based models involve sampling (Escobar and West, 1995;Teh et al., 2006).

we chose to use variational inference (Blei and Jordan, 2005), which provides a fast deterministic alternative to sampling, hence avoiding issues of diagnosing convergence and aggregating samples.


Many methods have been proposed for solving this problem by automatically extracting gazetteers from large amounts of texts (Riloff and Jones, 1999;Thelen and Riloff, 2002;Etzioni et al., 2005;Shinzato et al., 2006;Talukdar et al., 2006;Nadeau et al., 2006).

these methods require complicated induction of patterns or statistical methods to extract high-quality gazetteers.


For example, Beckham is redirected to Beckham (disambiguation) in the above example.

it is also possible that Beckham redirects to one of the articles (e.g, David Beckham).


The results indicate that structures in Wikipedia are suited for knowledge extraction.

the results also indicate that there is room for improvement, considering that the effects of gaz c and wp c were similar, while the matching rate was greater for wp c. An issue, which we should treat, is the disambiguation of ambiguous entities.


For example, the paragraph titled "Music of Scotland" (shown below in Wikitext) in the Wikipedia article on Scotland contains an enumeration of entities, which can be la- Lexicosyntactic patterns have been employed successfully in the past (e.g., Hearst, 1992;Roark and Charniak, 1998;Cederberg and Widdows, 2003), and this type of tag extraction is still a promising direction for the future.

the brute force approach we tried -of indiscriminately tagging the entities of enumerations of four or more entities -was found to introduce a large amount of noise into the system in our development experiments.


First, even seemingly good patterns can produce false hits due to metaphor and idiomatic expressions.

by restricting their use to relevant regions of text, we could avoid such false positives.


Existing systems generally require rules/patterns to recognize a context in which a weapon is explicitly linked to an event or its consequences (e.g., "attack with <np>", or "<np> caused damage").

weapons are not always directly linked to an event in text, but they may be inferred through context.


In contrast, if a document is relevant to the IE task, then there must be at least one sentence that contains relevant information.

most documents contain a mix of both relevant and irrelevant sentences.


We will refer to such reliable patterns as Primary Patterns.

patterns that are not necessarily reliable and need to be restricted to relevant regions will be called Secondary Patterns.


This demonstrates that our sentence classifier is having the desired effect.

observe that the precision gain comes with some loss in recall points.


These results are particularly noteworthy because AutoSlog-TS requires a human to manually review the patterns and assign event roles to them.


For example, "John's wife" is enough to determine the relationship b etween "John" and "John's wife" in the sentence "John's wife got a good job… " as shown in Figure 1(a) .

sPT is not enough in the coordinated cases, e.g.


Finally, note that a number of statistical MT systems make use of source language syntax in transducer-style approaches; see (Lin, 2004;Ding and Palmer, 2005;Quirk et al., 2005;Liu et al., 2006;Huang et al., 2006).

to the preprocessing approach, they attempt to incorporate syntax directly into the decoding stage.


Such distortions in the word reordering will be quite difficult for the word or phrase-based alignment model to capture.

with the application of a reordering rule to reposition the child CP after its sibling NP under a parent NP, and the PP VP reordering rule for VP introduced previously, the sentence can be easily transformed into "French delegation participate 8 th handicap people Winter Olympics hold at US Salt Lake City," a sentence whose word order is much closer to that of English.


The idea is to compare the two systems given the same type of input: if the reordered system learned a better phrase table, then it might outperform the baseline system on un-reordered inputs despite the mismatch; on the other hand, if the baseline system learned a better phrase table, then it might outperform the reordered system on reordered inputs despite the mismatch.

the results in Table 6 did not settle our question: the reordered system performed worse than the baseline on unreordered data, while the baseline system performed worse than the reordered system on reordered data, both of which can be explained by the mismatched conditions between training and testing.


In our experiments, the phrase-based MT system uses an un-lexicalized reordering model, which might make the effects of the syntactic reordering method more pronounced.

in an early experiment 4 submitted to the official NIST 2006 MT evaluation, the reordered system also improved the BLEU score substantially (by 1.34 on NIST 2006 data) over a phrase-based MT system with lexicalized reordering models .


We could transform tree (1) directly into tree (4) without bothering to generate tree (3).

skipping tree (3) will create us difficulty in applying the EM algorithm to choose a better binarization for each tree node, since tree (4) can neither be classified as left binarization nor as right binarization of the original tree (1) -it is the result of the composition of two left-binarizations.


They relied on a variant of a voted perceptron, and achieved significant improvements.

their work was limited to reranking, thus the improvement was relative to the performance of the baseline system, whether or not there was a good translation in a list.


2005;Dang and Palmer, 2005), when enough labeled training data is available.

creating a large sense-tagged corpus is very expensive and time-consuming, because these data have to be annotated by human experts.


Because it is difficult to know when the classifier reaches maximum effectiveness, previous work used a simple stopping condition when the training set reaches desirable size.

in fact it is almost impossible to predefine an appropriate size of desirable training data for inducing the most effective classifier.


In recent years, there have been attempts to apply active learning for word sense disambiguation .

to our best knowledge, there has been no such attempt to consider the class imbalance problem in the process of active learning for WSD tasks.


Previous work (Estabrooks et al., 2004) reported that under-sampling of the majority class (predominant sense) has been proposed as a good means of increasing the sensitivity of a classifier to the minority class (infrequent sense).

in our active learning experiments, under-sampling is apparently worse than ordinary, over-sampling and our BootOS.


An approach based on pairwise similarities, which encourage nearby data points to have the same class label, has been proposed as a way of incorporating unlabeled data discriminatively (Zhu et al., 2003;Altun et al., 2005;Brefeld and Scheffer, 2006).

this approach generally requires joint inference over the whole data set for prediction, which is not practical as regards the large data sets used for standard sequence labeling tasks in NLP.


The gradient of Equation (1) can be written as follows: Calculating E p(Y|x,λ) as well as the partition function Z(x) is not always tractable.

for linear-chain CRFs, a dynamic programming algorithm similar in nature to the forward-backward algorithm in HMMs has already been developed for an efficient calculation (Lafferty et al., 2001).


In Sequential Viterbi Models, such as HMMs, MEMMs, and Linear Chain CRFs, the type of patterns over output sequences that can be learned by the model depend directly on the model's structure: any pattern that spans more output tags than are covered by the models' order will be very difficult to learn.

increasing a model's order can lead to an increase in the number of model parameters, making the model more susceptible to sparse data problems.


As a simple illustration of this, when encountering the symbol RMNH in a field book entry, this most likely indicates the start of a new (registration number) segment.

in the database, on which all language models are trained, RMNH never occurs as a symbol in the registration number column; it does occur a few times in the column for special remarks but never at the start of the text.


For example, the comma between Las Claritas and 9-VI-1978 only serves to separate the Place segment from the Collection date segment; the comma is not copied to the database.

commas do occur fieldinternally in the database, especially in longer fields such as SPECIAL REMARKS.


Supervised machine learning approaches have been successfully applied for creating systems capable of performing this task.

the supervised nature of these approaches requires large amounts of annotated training data; the acquisition of which is often a laborious and time-consuming process.


Experiments with this approach pointed out that truly random concatenation of database fields results in weak performance; a rather simple baseline approach, which only matches substrings of a field book entry with the contents of the database, leads to better results.

if a small amount of annotated field book entries is available -in this study, 10 entries turned out to be sufficient-one can estimate field ordering probabilities that can be used to generate more realistic training data from the database.


Most of those approaches rely on the analysis of document structure (reflected in, for example, html tags), from which record templates are derived.

this approach does not apply to unstructured text.


This matches our intuition: without generalizing to semantic language models, higher order language models will be relatively sparse and contain much noise.

when taking into account the semantic features, we found that bigram and trigram semantic language model fea-tures outperformed unigrams.


Out of the models they describe, the HMM models are the most expressive models that can compute posterior probabilities using the forward-backward algorithm.

unlike sequence alignments, there are no ordering constraints in word alignments, and the alignments are many-to-many as opposed to one-to-one.


Blunsom & Cohn (2006) use Viterbi decoding to find an alignment of two sentences given a trained CRF model, a * argmax a P Λ (a|C i , C j ).

the posterior probabilities of the labels at each position can be calculated as well using the forwardbackward algorithm: where α l and β l are the forward and backward vectors that are computed with the forward-backward algorithm (Lafferty et al., 2001).


Other utility functions, such as Dice, Jaccard and Hamming can be used as U set agreement .

only metric-based utility functions will result in a metric-based U AM A utility function.


We tried numerous features that compare MeSH terms based on their distance in the ontology, and other features that indicate whether a word is part of a longer term.

none of these feature were selected for the final system.


For the Viterbi alignments, only three results could be generated (one for each symmetrization method).

since the refined method produced a very similar result to the union, only the union is displayed in the figure.


Our experiments were limited by the size of the labeled data.

the results support the theoretical predictions, and demonstrate the advantage of posterior-decoding over Viterbi decoding.


In principle, the predictive accuracy of the language model can be improved by increasing the order of the n-gram.

doing so further exacerbates the sparse data problem.


We focused on machine translation when describing the queued language model access.

it is general enough that it may also be applicable to speech decoders and optical character recognition systems.


For this data set, we also see an improvement when using a part-of-speech language model -the BLEU score increases from 18.19% to 19.05% -consistent with the results reported in the previous section.

moving from a surface word translation mapping to a lemma/morphology mapping leads to a deterioration of performance to a BLEU score of 14.46%.


In particular, since no model of the training material is being learned, the training corpus needs to be stored in order to be queried.

to k-NN, however, the search for closest neighbors does not require any distance, but instead relies on relational similarities.


The questions involved range from reconstruction of ancient word forms, to the elucidation of phonological drift processes, to the determination of phylogenetic relationships between languages.

this problem has received relatively little attention from the computational community.


Like our approach, they use a probabilistic edit model as a formalization of the phonological process.

they do not consider the question of reconstruction or inference in multi-node phylogenies, nor do they present a learning algorithm for such models.


Incorrect prior splits can needlessly fragment training data and incorrect prior tying can limit the model's expressivity.

correct assumptions can increase the efficiency of the learner.


Despite our structural simplicity, we outperform state-tied triphone systems like Young and Woodland (1994), a standard baseline for this task, by nearly 2% absolute.

we fall short of the best current systems.


If a key function is to serve as a filter, matching names must be members of the same equivalence class.

no single partition can produce equivalence classes that both include all matching pairs and exclude all non-matching pairs.


The methods presented above are mostly efficient and always exact.

for models that take global properties of the tree into account, they cannot be applied.


For all four languages, the same treebanks were used, which allows a comparison of the results.

in some cases the size of the training set changed, and at least one treebank, Turkish, underwent a thorough correction phase.


Last year's test set also had an average sentence length of 5.9.

this year, the average sentence length is 7.5 tokens, which is a significant increase.


In this domain, it seems more feasible to use general language resources than for the chemical domain.

the results prove that the extra effort may be unnecessary.


The choice of the best translation is made based on the combination of the probabilities and feature weights, and much discussion has been made of how to make the estimates of probabilites, how to smooth these estimates, and what features are most useful for discriminating among the translations.

a cursory glance at phrasetables produced often suggests that many of the translations are wrong or will never be used in any translation.


However, a cursory glance at phrasetables produced often suggests that many of the translations are wrong or will never be used in any translation.

most obvious ways of reducing the bulk usually lead to a reduction in translation quality as measured by BLEU score.


Tight restrictions on phrase length curtail the power of phrase-based models.

some promising engineering solutions are emerging.


It is perhaps surprising that such a small sample size works as well as the full data.

recent work by Och (2005) and Federico and Bertoldi (2006) has shown that the statistics used by phrase-based systems are not very precise.


for reasons described in §5.3, our other collocation algorithms depend on sorted sets, so we use a merge algorithm.


Double binary search requires that its input sets be in sorted order.

the suffix array returns matchings in lexicographical order, not numeric order.


They show that this approach leads to some time savings for phrase search, although the gains are relatively modest since the search for contiguous phrases is not very expensive to begin with.

the potential savings in the discontiguous case are much greater.


In our baseline algorithm, we would search for ab and cd, and then perform a computation to see whether these subphrases were collocated within an elastic window.

if we instead use abXc and bXcd as the basis of the computation, we gain two advantages.


In our Python implementation this takes several minutes, which in principle should be amortized over the cost for each sentence.

just as Zens and Ney (2007) do for phrase tables, we could compile our data structures into binary memory-mapped files, which can be read into memory in a matter of seconds.


Our work enables this in hierarchical phrase-based models.

we are interested in additional applications.


Babelfish 2 typically use a bilingual dictionary that is either manually compiled or learned from a par-allel corpus.

such dictionaries often have insufficient coverage of proper names and technical terms, leading to poor translation performance due to out of vocabulary (OOV) problem.


Integrating out θ d 's and z dn 's, the probability p(D|α, β) of the corpus is thus: Unfortunately, it is intractable to directly solve the posterior distribution of the hidden variables given a document, namely p(θ, z|w, α, β).

(Blei et al., 2003) has shown that by introducing a set of variational parameters, γ and φ, a tight lower bound on the log likelihood of the probability can be found using the following optimization procedure: γ is the Dirichlet parameter for φ and the multino- are the free variational parameters.


This probably contributed to its inferior result.

we found that the best result comes from combining all the corpora together with K = 60 and L = 40.


Nicholson and Baldwin (2006) investigated the prediction of the inherent semantic relation of a given compound nominalization using as statistical measure the confidence interval.

looked at MWEs in general investigating the semiautomated detection of MWE candidates in texts using error mining techniques and validating them using a combination of the World Wide Web as a corpus and some statistical measures.


With good statistical measures, we are able to distinguish genuine MWE from non-MWEs among the n-gram candidates.

from the perspective of grammar engineering, even with a good candidate list of MWEs, great effort is still required in order to incorporate such word units into a given grammar automatically and in a precise way.


By acquiring new lexical entries for the MWEs candidates validated by the statistical measures, the grammar coverage was shown to improve significantly.

no further investigation on the parser accuracy was reported there.


The annotation indicates a flat structure, where ev-ery token is headed by "Department".

a similar BIO phrase has a very different structure, pursuant to the BIO guidelines.


Example domains of predictive data mining include earthquake prediction, air temperature prediction, foreign exchange prediction, and energy price predic-tion.

predictive data mining is only feasible when a large amount of structured numerical data (e.g., in a database) is available.


One simple approach could be a system (see NGR system in Section 5) trained by a machine learning technique using n-gram features and classifying a message into multiple classes (e.g., NDP, Liberal, or Progressive).

we develop a more sophisticated algorithm and compare its result with several baselines, including the simple n-gram method 2 .


However, these approaches do not address the task of extracting aspect-of relations and make use of syntactic features only for labeling opinion holders and topics.

as we describe below, we find the significant overlap between aspectevaluation relation extraction and aspect-of relation extraction and apply the same approach to both tasks, gaining the generality of the model.


In the aspect-evaluation relation extraction, we evaluated the results against the human annotated gold-standard in a strict manner.

according to our error analysis, some of the errors can be regarded as correct for some real applications.


In the case of inter-sentential relations, our model tends to rely heavily on the statistical clues, because syntactic pattern features cannot be used.

our current method for estimating co-occurrence distributions is not sophisticated as we discussed above.


2 To be exact, the doubly underlined part is polar clause.

it is called polar sentence because of the consistency with polar sentences extracted by using layout structures.


Our results show that we can determine case with an error rate of 4.2%.

our results would have been impossible without a deeper understanding of the linguistic phenomenon of case and a transformation of the representation oriented towards this phenomenon.


Finally, the dual and masculine sound plural do not express nunation.

the feminine sound plural marks nunation explicitly, and all of its case morphemes are written only as diacritics, e.g., Traditional Arabic grammar makes a distinction between verbal clauses (¨ § ¤ ¦ " ! )


We observed that no matter which value the smoothing parameter takes, there are only about 10,000 non-zero features finally selected by Collins' original method.

the two new methods select substantially more features, as shown in Table 5.


Degradation of accuracy in a new domain can be overcome by developing an annotated corpus for that specific domain, e.g., as in the Biology domain.

this solution is feasible only if there is sufficient interest in the use of NLP technology in that domain, and there are sufficient funding and resources.


However, this solution is feasible only if there is sufficient interest in the use of NLP technology in that domain, and there are sufficient funding and resources.

our approach is to use existing resources, and rapidly develop taggers for new domains without using the time and effort to develop annotated data.


The above method forms the basis for our determination of the set of tags that are to be associated with the domain words.

the actual tag to be assigned for an occurrence in text depends on the context of use.


In the Bin vector associated with "creation", all these four features will get the value one, assuming that the four corresponding words are found in Text-Lex.

assuming "creatory" is not found in Text-Lex, the feature "-ion+ory" would get a zero value.


This method does not use POS tagged corpora (although in the reported experiment the initial "perfect" clusters were obtained from the Brown corpus using the POS tag information).

we use the POS tagged WSJ corpus to assist in the induction of tag information for our lexicon.


Our use of the kNN method to identify tags and their probabilities for words was inspired by this work.

their use of kNN method was in the context of supervised learning.


Since our motivation, on the other hand, is to move to a new domain, we didn't consider detection of similarity on the basis of word contexts.

we have shown that the approach of identifying words on the basis of suffixation patterns and using them as exemplars can be applied effectively even when the domain of application is substantially different from the text (the WSJ corpus) providing the exemplars.


As shown in (Cucerzan and Yarowsky, 2000), such information can provide considerable information to build a lexicon that associates possible tags with words.

we use this information only to provide the initial values.


Time did not allow a full-scale experiment, but for all languages except Catalan and Hungarian, the bidirectional parsing method outperformed the unidirectional methods when trained on a 20,000-word subset.

the gain of using bidirectional parsing may be more obvious when the treebank is small.


We performed a beam search by carrying out a K-best search through the set of possible sequences of actions as proposed by Johansson and Nugues (2006).


The ESDU algorithm also fared better with the SVO languages, except for Italian.

the Greenberg's basic word order typology cannot shed enough light into the performance of the three parsing algorithms.


Due to the limitation of participation time, we only applied the first-order decoding parsing algorithm in CONLL-2007.

our algorithm can be used for the second order parsing.


Our system makes no provisions for non-projective edges.

to previous work, we aim to learn labelled dependency trees at one fell swoop.


It is no longer rare to see dependency relations used as features, in tasks such as machine translation (Ding and Palmer, 2005) and relation extraction (Bunescu and Mooney, 2005).

there is one factor that prevents the use of dependency parsing: sparseness of annotated corpora outside Wall Street Journal.


We therefore defined the positive lower bound (10 -10 ) and the negative upper bound (-10 -10 ) to eliminate values that tend to be zero.

the SVM is a binary classifier which only recognizes true or false.


Therefore the parser develops a bias against assigning these roles in general, and recall suffers.

9 precision is very good, thanks to the rich context in which the roles are assigned.


The parsing formalism we use is related to the tree adjoining grammar (TAG) formalisms described in (Chiang, 2003;Shen and Joshi, 2005).

an important difference of our work from this previous work is that our formalism is defined to be "splittable", allowing use of the efficient parsing algorithms of Eisner (2000).


As one example, the cake spine shown above can be s-adjoined into the VP node of the ate spine, to form the tree shown in figure 1(a).

if we use the r-adjunction operation to adjoin the cake tree into the VP node, we get a different structure, which has an additional VP level created by the r-adjunction operation: the resulting tree is shown in figure 1(b).


Note that the mapping from parse trees to derivations is many-to-one: for example, the example trees in section 2.3 have structures that are as "flat" (have as few levels) as is possible, given the set D that is involved.

other similar trees, but with more VP levels, will give the same set D. this issue appears to be benign in the Penn WSJ treebank.


F-dist ## F k : A set of k-feature-sets ## R o : ν optimal rules (feature-sets) ## R k,ω : ω k-feature-sets for generating candidates ## selectNBest(R, n, S, Wr): n best rules from R ## with gain on {wi,r} m i=1 and training samples S procedure weak-learner(F k , S, W r ) ## ν best feature-sets as rules Figure 3: Find optimal feature-sets with given weights distributes features to subsets of features, called buckets, based on frequencies of features.

we guess training using a subset of features depends on how to distribute features to buckets like online learning algorithms that generally depend on the order of the training examples (Kazama and Torisawa, 2007).


SVM learns a final hypothesis as a linear combination of the training examples using some coefficients.

this boosting-based rule learner learns a final hypothesis that is a subset of candidate rules (Kudo and Matsumoto, 2004).


Texts mixing a high number of topics (e.g., Sentence Salads) are almost as likely as natural texts that address only a few topics.

the former has much higher entropy of the topic distribution due to a large number of topics being active in such texts (see also Figure 1).


When only a few words are changed in a true document, it retrains the properties of a true document (high LLPW and low entropy).

as more number of words are changed in a true document, it starts showing the characteristics of a false document (low LLPW and high entropy).


Indeed, through experimental results, it was shown that entropy of the topic distribution is lower and average LLPW of true documents is higher for true documents and the former measure was found to be more effective.

the poor performance of this method on out-of-domain data suggests that we need to use a much larger training corpus to build a robust fake document detector.


These studies also show that children are generally good at referent selection, given a novel target.

there is not consistent evidence regarding whether children actually learn the novel word from one or a few such exposures (retention).


In all of the reported experimental results, children can readily pick the correct referent for a familiar or a novel target word in such a setting (Golinkoff et al., 1992;Halberda and Goldman, 2008;Halberda, 2006;Horst and Samuelson, 2008).

halberda's eye-tracking experiments on both adults and pre-schoolers suggest that the processes involved for referent selection in the familiar target situation may be different from those in the novel target situation.


As discussed in the previous section, results from the human experiments as well as our computational simulations show that the referent of a novel target word can be selected based on the previous knowledge about the present objects and their names.

the success of a subject in a referent selection task does not necessarily mean that the child/model has learned the meaning of the novel word based on that one trial.


Moreover, our retention experiments show that the model can map a recently heard novel word to its recently seen novel referent (among other novel objects) after only one exposure.

the strength of the association of a novel pair after one exposure shows a notable difference compared to the association between a "typical" familiar word and its meaning.


Venkataraman (2001) notes that considering the generation of the text as a single event is unlikely to be how infants approach the segmentation problem.

mBDP-1 uses an incremental search algorithm to segment one utterance at a time, which is more plausible as a model of infants' word segmentation.


We also tentatively conclude that phonotactic patterns can be learned from unsegmented text.

the phonotactic patterns learned by our model ought to be studied in detail to see how well they match the phonotactic patterns of English.


This process appears quite similar to the decision lists technique in machine learning (Rivest, 1987) which has already been combined with the MDL principle (Pfahringer, 1997).

we are not committed to this formalism: we are more interested in the content of the model rather than its knowledge representation.


Thus, in training the VPosition features become dominant as the SRL learns to recognize more verbs.

the VPosition features are inactive when the Baby SRL encounters the novel-verb test sentences.


The bootstrap model learns much more slowly, which is unsurprising, given that it depends on having some reasonable category knowledge in order to develop its clusters-leading to a chicken-and-egg problem.

once started, its performance improves well beyond the word-based model's plateau.


In the 'unambiguous' and the 'biased' conditions, the head words' lexical biases are too strong for the model to overcome.

the results show a realistic effect of the lexical bias.


The algorithm is currently only implemented as an incremental process.

comparison with a batch version of the algorithm, such as by using a Gibbs sampler (Sanborn et al., 2006), would help us further understand the effect of incrementality on language fidelity.


They are in fact important because they allow to structure information, thus fostering their search and reuse.

it is well known that any knowledge-based system suffers from the so-called knowledge acquisition bottleneck, i.e.


Moreover, we propose an unsupervised methodology for acquiring general-specific noun relationships.

it is clear that deep evaluation is needed.


On the one hand, catchwords represent the mass value orientation for a period.


These two synonyms are treated as one term in manual evaluation that corresponds to promote usage frequency.

relationship between the two synonyms is not concerned in automatic extraction.


However, this rule does not generalize to other verbs, such as "drive," as in the sentence "The boy drives his parents crazy," which also has three core arguments "Arg0: driver," "Arg1: thing in motion," and "Arg2: secondary predication on Arg1."

here the action is figurative, and we would expect a layout rule that puts Arg1 in position C: In addition, while modifier arguments have the same meaning across verbs, their pictorial representation may differ based on context.


Thus native speakers do not get as many syntactic hints.

non-native speakers do not have the same degree of built-in English syntactic knowledge.


The resulting system outperformed the original Arabic system trained on 3.3 million sentence pairs corpora when using monotone decoding.

an improvement in monotone decoding is no guarantee for an improvement over the best baseline achievable with full word forms.


There now exist numerous general-purpose algorithms for attribute selection (e.g., (Dale and Reiter, 1995;Krahmer et al., 2003;Belz and Gatt, 2007;Siddharthan and Copestake, 2004)).

these algorithms by-and-large focus on the algorithmic aspects of referring expression generation rather than on psycholinguistic factors that influence language production.


The most relevant previous research is the work of (Gupta and Stent, 2005), who modified Dale and Reiter's algorithm to model speaker adaptation in dialog.

this corpus does not involve dialog so there are no cross-speaker constraints, only within-speaker constraints (speaker style and priming).


For both domains, for automatic and human attribute selection, the speaker-dependent Template-based approach seems to perform the best, then the speakerindependent Template-based approach, and then the Permute&Rank approach.

we find automatic metrics for evaluating generation quality to be unreliable.


PropBank annotation includes the whole chain of empty categories, as well as the antecedent of the empty category (the filler of the gap).

the CoNLL-2005 version only includes the filler of the gap and if there is no filler, the argument is omitted, e.g., no ARG0 (subject) for leave would be included in I said to leave because the subject of leave is unspecified.


linguistic features can often be detected with greater reliability than deep (semantic) features.

deep features can cover more ground because they regularize across differences in surface strings.


This algorithm can cover 98.5% arguments while reducing about 60% of the training samples, according to our statistics.

this is achieved at the price of including a syntactic parse tree as part of the input for semantic parsing.


It should be cleared that the ordinal numbers are only used to distinguish different meanings of a predicate.

if these numbers are treated as tags for predicates, some statistical properties will be obtained as illustrated in Figure 1.


The main reason why we use a sequence labeling method for predicate identification was to relax the effect of the tagging error of PPOS and PPOSS.

we will show later that this module aggravates the total performance.


For example, voice (active or passive) is essential for verb predicate's argument classification.

presence of a genitive word is useful for noun predicate's argument classification.


The argument identifier regards a predicate and its nearest neighbors -its parent and children -as argument candidates.

if the POS tag of a nearest neighbor is IN, MD, or TO, it will be ignored and the next nearest candidates will be used.


The roleset matching method is devised to select the most appropriate role sequence and to identify the correct roleset.

the current features for roleset matching seem to be not enough and other useful features are expected to be found in the future work.


There is no significant difference in score for left and right dependencies, presumably because of the bi-directional parsing.

the system overpredicts dependencies to the root.


We address each of these subtasks with separate components without backward feedback between sub-tasks.

the use of multiple parsers at the beginning of the process, and re-ranking at the end, contribute beneficial stochastic aspects to the system.


Better models, features, and learning algorithms have allowed systems to perform many of these tasks with 90% accuracy or better.

success in integrated, end-to-end natural language understanding remains elusive.


In our case, I is the set of words that occur at least t times in the unlabelled pool, b i = t, ∀i ∈ I, the multisets are the sentences in that pool and w j = 1, ∀j ∈ J. Multiset multicover is NP-hard.

there is a good greedy approximation algorithm for it.


1b would be represented as follows (details omitted): (1) 1 U pron 0 FRAG 2 heeft verb 0 ROOT 3 volkomen adj 4 mod 4 gelijk noun 0 FRAG In the training phase, the MSTParser tries to maximize the scoring margin between the correct parse and all other valid dependency trees for the sentence.

in the case of fragmented trees, the training example is not strictly speaking correct, in the sense that it does not coincide with the desired parse tree.


We have seen that the graph-and the transitionbased parser react similarly to the various filtering methods.

there are interesting differences in the magnitude of the performance changes.


Negation signals in the BioScope corpus always have one consecutive block of scope tokens, including the signal token itself.

the classifiers only predict the first and last element of the scope.


The evaluation of this ratio is difficult because not all true interactions are known.

the Disorder module does not contribute significantly to the prediction.


The simplest implementation would be to check the most recent model with the previous model and stop if their agreement exceeds the intensity cutoff.

independent of wanting to provide users with a longevity control, this is not an ideal approach because there's a risk that these two models could happen to highly agree but then the next model will not highly agree with them.


Such fully unsupervised methods do not incorporate any language-specific parsers or taggers, so can be successfully applied to diverse languages.

unsupervised pattern-based methods suffer from several weaknesses.


This framework allows a fully unsupervised seed-less discovery of concepts without relying on language-specific tools.

it completely ignores potentially useful syntactic or morphological information.


As discussed in Section 5, windowing is required for successful noise reduction.

due to the increase in pattern quality with parser data, it is likely that less noise will be captured by the discovered patterns.


In the distributed model, x will be considered a member if the majority of its k nearest neighbors are members.

it is im-portant that both models can also be used to represent regions with soft boundaries.


This approach is similar to that of seed words (e.g., (Hearst, 1998)) or hook words (e.g., (Davidov and Rappoport, 2008)) in previous work.

in our case they are fixed and rich in grammatical information in the sense that they have to correspond to subject -object pairs in consecutive clauses.


This procedure can be partially solved by identifying complex verbs such as "take in".

we leave this improvement for future work.


Certainly, we are missing important information by excluding phrases and ignoring modality.

these features can be difficult to capture accurately, and since inaccurate input could degrade the clustering accuracy, in this research we stick with the important and easily-obtainable features.


And most of these constructions do not focus so much on the magnitude, but on the order in which one eventuality (the effect) is a reaction to the other (the cause).

a closer look at our data shows that there are also constructions which indicate this property more precisely.


Instead, it relied on built-in syntactic knowledge and a rich hierarchical representation of semantic knowledge to learn links between sentence structure and meaning.

our previous experimental design has a serious drawback that limits its relevance to the study of how children learn their first language.


In the present paper this error is over-determined, because the classifier learns only an agent/non-agent contrast, and the linguistic constraints forbid duplicate agents in a sentence.

for comparison to the earlier paper we test our system on the 'A and B gorp' sentences as well.


As expected, the NPattern feature makes the same overgeneralization error seen by children and the system in (Connor et al., 2008).

when the VPosition feature is added, different results are obtained for the 'A gorp B' and 'A and B gorp' sentences.


The models appear reasonably good at adapting to the empirical fixation distribution of individual readers.

the models typically tend to look at more words than the readers, as noted above.


Because MERT ignores the denominator in Equation 1, it is invariant with respect to the scale of the weight vector θ -the Moses implementation simply normalises the weight vector it finds by its 1 -norm.

when we use these weights in a true probabilistic model, the scaling factor affects the behaviour of the model since it determines how peaked or flat the distribution is.


The TYPE-AUTO predictions contribute little information, only fragmenting the data and leading to over-training and lower accuracy.

the CONTEXT-AUTO predictions improve accuracy, as these scores provide orthogonal and hence helpful information for the semi-supervised classifier.


When the members of the committee are in complete agreement, AL should be aborted since it no longer contributes to the overall learning process -in this case, AL is but a computationally expensive counterpart of random sampling.


We analyze several approaches for modeling non-local dependencies proposed in the literature and find that none of them clearly outperforms the others across several datasets.

as we show, these contributions are, to a large extent, independent and, as we show, the approaches can be used together to yield better results.


The Viterbi algorithm has the limitation that it does not allow incorporating some of the non-local features which will be discussed later, therefore, we cannot use it in our end system.

it has the appealing quality of finding the most likely assignment to a second-order model, and since the baseline features only have second order dependencies, we have tested it on the baseline configuration.


This separation "breaks" the Viterbi decision process to independent maximization of assignment over short chunks, where the greedy policy performs well.

dependencies between isolated named entity chunks have longer-range dependencies and are not captured by second-order transition features, therefore requiring separate mechanisms, which we discuss in Section 5.


The sample text discussed in the introduction shows one such example, where all occurrences of "blinker" are assigned the PER label.

in general, this is not always the case; for example we might see in the same document the word sequences "Australia" and "The bank of Australia".


Both context aggregation and two-stage prediction aggregation treat all tokens in the text similarly.

we observed that the named entities in the beginning of the documents tended to be more easily identifiable and matched gazetteers more often.


Short constituents are more frequent and are generally more likely to be correct.

the correctness of long constituents is an indication that the parser has a correct interpretation of the tree structure and that it is likely to create a high quality tree.


(2008) are supervised, performing semantic analysis of the parse tree and gold standard-based calssification, respectively.

the SEPA algorithm of Reichart and Rappoport (2007), an algorithm for supervised constituency parsers, can be applied to unsupervised parsers in a way that preserves the unsupervised nature of the selection task.


It could be argued that the total number of 'real' labels in the data is indeed large (e.g., because every verb exhibits its own syntactic patterns) and that a small number of labels is just an arbitrary decision of the corpus annotators.

most linguistic theories agree that there is a prototypical level of generalization that uses concepts such as Noun Phrase and Verb Phrase, a level which consists of at most dozens of labels and is strongly manifested by real language data.


There have been earlier studies of hypernym extraction with either lexical or dependency extraction patterns.

these studies applied the techniques to a variety of different data sets and used different evaluation techniques.


In the example sentence, no lexical pattern will associate city with Liverpool because there are too many words in between.

a dependency pattern will create a link between these two words, via the word as.


Precision will be computed against all chosen candidate hypernyms.

recall will only be computed against the positive noun pairs which occur in the phrases selected by the examined method.


The parser also performs part-of-speech tagging and lemmatization, tasks which are useful for the lexical preprocessing methods.

taking future real-time applications in mind, we did not want the lexical processing to be dependent on the parser.


Dependency patterns remain finding more positive targets and obtaining a larger recall score but their precision score is disappointing.

when we examine the precision-recall plots of the two methods ( Figure 1, obtained by varying the acceptance threshold of the machine learner), they are almost indistinguishable.


The dependency parser ignored punctuation signs and therefore the dependency pattern covers both phrases with and without punctuation.

these phrase variants result in different lexical patterns.


The simplest method to estimate the similarity between two multi-contexts is to compute the inner product of their vector representations in the VSM.

formally, we define a space of dimensionality N in which each dimension is associated with one word from the dictionary, and the multi-context m is represented by a row vector where the function f (t i , m) records whether a particular token t i is used in m. Using this representation we define bag-of-words kernel between multicontexts as the bag-of-words representation does not deal well with lexical variability.


Moreover, further experiments have shown that even a larger number of Wikipedia articles (600,000) does not help.

the latent semantic kernels outperform all the other methods, and their composite (K BOW + K W and K BOW + K N Y T ) perform the best on every configuration, demonstrating the effectiveness of latent semantic kernels in fine-grained classification of named entities.


A named entity is then classified according the similarity between the word-domain lists and the global context in which the entity appears.

the evaluation was performed only on 6 person names using two categories.


For the SRL-only task, participants have been provided will all the data but the PRED and APREDs, which they were supposed to fill in with their correct values.

they did not have to determine which tokens are predicates (or more precisely, which are the argument-bearing tokens), since they were marked by 'Y' in the FILLPRED field.


scores can also be computed for a number of other conditions: • Task: Joint or SRL-only • Challenge: open or closed • Domain: in-domain data (IDD, separated from training corpus) or out-of-domain data (OOD) Joint task participants are also evaluated separately on the syntactic dependency task (labeled attachment score, LAS).


The largest number of systems can be compared in the SRL results table (Table 6), where all the systems have been evaluated solely on the SRL performance regardless whether they participated in the Joint or SRL-only task.

since the results might have been influenced by the supplied parser, separate ranking is provided for both types of the systems.


When labeling the semantic role, we use a similar approach as we did in CoNLL Shared Task 2008.

as the frames information is not supplied for all languages, we do not use it in this task.


More specifically, the lack of coverage of hand-crafted linguistic grammars is a major concern.

the CoNLL task is also a good opportunity for the deep processing community to (re-)evaluate their resources and software.


As indicated above in table 1, only 2% of sentences are unparsable, despite 16% requiring the Swap action.

this argument does not explain why our parser did relatively poorly on German semantic dependencies.


The model of (Che et al., 2008) decided one label for each arc before decoding according to unigram features, which caused lower labeled attachment score (LAS).

keeping all possible labels for each arc made the decoding inefficient.


We can see that along with the development data are added into the training data, the performance on the indomain test data is increased.

it is interesting that the additional data is harmful to the ood test.


Both LEGA and MEGA were used for this task.

training and test for the final submission of Chinese, Czech and English run in MEGA, and the rest in LEGA.


they could show that a system can gain accuracy of about 2-4% which is a lot.


We will refer to predicates such as word/2 as observed because they are known in advance.

role/3 is hidden because we need to infer it at test time.


For five of the languages the difference to the F1 scores of the best system is 3%.

for German it is 6.19% and for Czech 10.76%.


As could be expected the parser performs much better for the languages for which a large training set is provided.

as discussed in the next section, simply adding more training data does not seem to solve the problem.


The improvement in scores are modest for Chinese, Czech, English and Japanese, while Catalan, German and Spanish stand out by vast improvements with additional training data.

the improvement when going from 75% to 100% of the training data is only modest for all languages.


Ideally, we would like to choose the most plausible syntactic-semantic structure among all possible structures in that syntactic dependencies and semantic dependencies are correlated.

solving this problem is too difficult because the search space of the problem is extremely large.


In order to avoid losing the benefits of higher-order parsing, we considered applying pseudo-projective transformation (Nivre and Nilsson, 2005).

growth of the number of dependency labels by pseudo-projective transformation increases the dependency parser training time, so we did not adopt transformations.


As for short sentences, the recall of our approach is in the same order as for the baseline.

precision is increased by a factor of two in comparison to the baseline, which is also similar to short sentences.


Due to space limitations we refer the reader to (Zelle and Mooney, 1996) for a detailed description of the Geoquery domain.

recall that the goal of semantic parsing is to produce the following function (Equation (1)): given that y and z are complex structures it is necessary to decompose the structure into a set of smaller decisions to facilitate efficient inference.


Precision and recall are typically used as evaluation metrics in semantic parsing.

as our model inherently has the ability to map any input sentence into the space of meaning representations the trade off between precision and recall does not exist.


Feedback Recall that our learning framework does not require meaning representation annotations.

we do require a Feedback function that informs the learner whether a predicted meaning representation when executed produces the desired response for a given input sentence.


Here we clearly need to identify sets of strings corresponding to the two parts of this language, but it is easy to see that no one context will suffice.

note that the first part is defined by the two contexts (λ, λ), (a, b) and the second by the two contexts (λ, λ), (a, bb).


In English, the capitalization of the phrase European Court of Auditors helps identify the span as a named entity.

in German, all nouns are capitalized, and capitalization is therefore a less useful cue.


Better POS induction algorithms yield lower perplexity language models.

clark did not study the correlation between the perplexity measure and the gold standard tagging.


Hence, evaluting against the full tag set that includes punctuation artificially increases the quality of the reported results, which is why we report results for the non-punctuation tag set.

to be able to directly compare with previous work, we also report results for the full WSJ POS tag set.


The task of marking up these expressions has usually been approached using straightforward sequence labeling techniques using simple features in a small contextual window (Choi et al., 2006;Breck et al., 2007).

due to the simplicity of the feature sets, this approach fails to take into account the fact that the semantic and pragmatic interpretation of sentences is not only determined by words but also by syntactic and shallow-semantic relations.


Conventionally, when measuring the quality of a system for an information extraction task, a predicted entity is counted as correct if it exactly matches the boundaries of a corresponding entity in the gold standard; there is thus no reward for close matches.

since the boundaries of the spans annotated in the MPQA corpus are not strictly defined in the annotation guidelines , measuring precision and recall using exact boundary scoring will result in figures that are too low to be indicative of the usefulness of the system.


The most visible effect of the reranker is that the recall is greatly improved.

this does not seem to have an adverse effect on the precision until the candidate set size goes above 8 -in fact, the precision actually improves over the baseline for small candidate set sizes.


Clustering evaluation has been extensively investigated (Section 3).

the discussion centers around the monosemous case, where each item belongs to exactly one cluster, although polysemy is the common case in NLP.


Hence, H(K|C) = H(K) and H(C|K) = H(C), and both V and NVI obtain their worst possible values 1 .

the score should surely depend on r (the size of each word's gold entry).


Their approach also allows to take the alignments of neighboring words into account.

to our work, they only have a very crude fertility model and they are considering a substantially different model.


In all cases the run-times are much higher than in the standard GIZA++ training.

we are now getting optimality guarantees where previously one could not even tell how far away one is from the optimum.


Example (3) is a factual statement without explicit opinion.

having a fast connection is a positive thing.


MLE estimators for all-fragment models are zerobiased with zero divergence between the average estimate and the true data distribution.

their variance is unboundedly large, leading to unbounded generalisation error on unseen cases.


Furthermore, other frequently employed priors such as the Dirichlet distribution and the Dirichlet Process promote better generalising rule probability distributions based on externally set hyperparameter values, whose selection is frequently sensitive in terms of language pairs, or even the training corpus itself.

the CV-MLE prior aims for a data-driven Bayesian model, focusing on getting information from the data, instead of imposing external human knowledge on them (see also (Mackay and Petoy, 1995)).


They also propose a technique, based on TF-IDF, to de-emphasize sentences similar to those that have already been selected.

this strategy is bootstrapped by random initial choices that do not necessarily favor sentences that are difficult to translate.


Active learning has been studied extensively in the context of multi-class labeling problems, and theoretically optimal selection strategies have been identified for simple classification tasks with metric features (Freund et al., 1997).

natural language applications such as SMT present a significantly higher level of complexity.


Similar with Co-training, the basic idea of Tritraining (Zhou and Li, 2005) is to iteratively expand the labeled training set for the next-round training based on the decisions of the current classifiers.

tri-training employs three classifiers instead of two.


It demonstrates that semi-supervised algorithms are able to learn more bi-lexical features automatically from the unlabeled data, which may help recognize more translation equivalences.

we also notice that the accuracy drops a little after Tri-training.


The topic-sentiment mixture (TSM) model (Mei et al., 2007) can jointly model sentiment and topics by constructing an extra background component and two additional sentiment subtopics on top of the probabilistic latent semantic indexing (pLSI) (Hofmann, 1999).

tSM may suffer from the problem of overfitting the data which is known as a deficiency of pLSI, and postprocessing is also required in order to calculate the sentiment prediction for a document.


(2009) employed lexical prior knowledge for semi-supervised sentiment classification based on non-negative matrix tri-factorization, where the domain-independent prior knowledge was incorporated in conjunction with domain-dependent unlabelled data and a few labelled documents.

this approach performed worse than the JST model on the movie review data even with 40% labelled documents as will be shown in Section 5.


Gibbs sampling is used to estimate the posterior distribution of LSM, as well as the JST and Reverse-JST models that will be discussed in the following two sections.

to LSM that only models document sentiment, the JST model (Lin and He, 2009) can detect sentiment and topic simultaneously, by modelling each document with S (number of sentiment labels) topic-document distributions.


Another recently proposed non-negative matrix tri-factorization approach (Li et al., 2009) also employed lexical prior knowledge for semi-supervised sentiment classification.

when incorporating 10% of labelled documents for training, the non-negative matrix tri-factorization approach performed much worse than LSM, with only around 60% accuracy achieved for all the datasets.


For this work, a subset of 16 emotional categories from this level has been selected, since the hierarchy proposed in WordNet Affect is considerably broader than those commonly used in sentiment analysis.

the first level of emotional categories may be useful to predict the polarity, but it is clearly not enough to predict the intensity of this polarity.


In this sentence, the concepts risk, death and disease are labeled with an emotional category: in particular, the categories assigned to them are fear, fear and dislike respectively.

while the two firsts are retrieved from the affective lexicon by their own synsets, the last one is labeled through its hypernym: since no matching is found for disease in the lexicon, the analysis over its hypernyms detects the category dislike assigned to the synset of its first hypernym, which contains words such as illness and sickness, and the same emotion (dislike) is assigned to disease.


Even if we expected that using quantifiers would lead to better results, the performance in both datasets decreases in 2 out of the 3 ML algorithms.

combining both features improves the results in both datasets.


Besides, the Most Frequent Sense (MFS) heuristic in WSD is usually regarded as a difficult competitor.

the improvement with respect to the random sense heuristic is quite remarkable.


A test on the news corpus removing those sentences not labeled with any emotional meaning has been performed for the 2-classes classification problem, allowing the method to obtain an accuracy of 81.7%.

to correctly classify these sentences, it would be necessary to have additional information about their contexts (i.e.


Some research has used the text that surrounds an image in a news article as a proxy (Feng and Lapata, 2008;Deschacht and Moens, 2007).

in many cases, the surrounding text or a user-provided caption does not simply describe what is depicted in the image (since this is usually obvious to the human reader for which this text is intended), but provides additional information.


For example, we would like to be able to identify that "woman" and "person" may refer to the same entity, whereas "man" and "woman" typically would not.

we also do not want a type inventory that is too large or too fine-grained.


6 K-SVM performs best with all the data; it uses the most expressive representation, but needs 100K examples to make use of it.

feature augmentation and variance regularization provide diminishing returns as the amount of training data increases.


Therefore, we do not measure the performance of these tasks on the Words category set.

we do use the content feature in labeling the test examples in grammaticality judgment.


(2008) result in a better performance than word types, but they are still too sparse for this task.

the average score gained by part of speech tags is also lower than the one by our categories.


Most work in IE has concentrated on entity extraction alone (Tjong Kim Sang, 2002;Sang and Meulder, 2003) or on relation extraction assuming entities are either given or previously extracted (Bunescu et al., 2005;Zhang et al., 2006;Giuliano et al., 2007;Qian et al., 2008).

these tasks are very closely inter-related.


They thus do a joint syntactic parsing and information extraction using a fixed template.

as designed, such a CFG approach cannot handle the cases when an entity is involved in multiple relations and when the relations crisscross each other in the sentence, as in Figure 1.


(2009) present a method to simultaneously do semi-supervised training of entity and relation classifiers.

their coupling method is meant to take advantage of the available unsupervised data and does not do joint inference.


For example, one would not be able to use kernelbased SVM for relation extraction, which has been very successful at this task, because Markov Logic does not support kernel-based machine learning.

our joint approach is independent of the individual machine learning methods for entity and relation extraction, and hence allows use of the best machine learning methods available for each of them.


MD and that usually have their parents to the left.

in some constructions this is not the case, and the parser has a hard time learning these constructions.


improve the classification accuracy, while using bigrams brings a statistically significant improvement over a simple bag-of-words representation.

medlock (2008) illustrates that "whether a particular term acts as a hedge cue is quite often a rather subtle function of its sense usage, in which case the distinctions may well not be captured by part-of-speech tagging".


The ideal method should recognize only one consecutive block for each hedge cue.


In the corpora provided for this task, scopes are annotated as continuous sequences of tokens that include the cue.

the classifiers only predict the first and last element of the scope.


Additionally, the system has been trained on a corpus that contains abstracts and full text articles, instead of only abstracts.

it is possible to confirm that, even with information on dependency syntax, resolving the scopes of hedge cues in biological texts is not a trivial task.


The GENIA tagger (Tsuruoka et al., 2005) takes an important role in our pre-processing set-up.

maybe somewhat surprisingly, we found that its tokenization rules are not always optimally adapted for the BioScope corpus.


For better normalization, we downcase base forms for all parts of speech except proper nouns.

gENIA does not make a PoS distinction between proper and common nouns, as in the Penn Treebank, and hence we give precedence to TnT outputs for tokens tagged as nominal by both taggers.


This provided us with an additional 855 hedged sentences.

the classifiers did not seem able to benefit from the additional training examples, and across several feature configurations performance was found to be consistently lower (though not significantly so).


Rules for scope detection, based on the grammatical relations of the sentence and the part-ofspeech tag of the cue, were manuallydeveloped.

another supervised CRF classifier was used to refine these predictions.


In (10) below it is uncertain whether fat body disintegration is independent of the AdoR.

it is stated with certainty that fat body disintegration is promoted by action of the hemocytes, yet the latter assertion is included in the scope to keep it continuous.


The corresponding classifier must discriminate positive from negative candidates.

identifying one candidate as positive implies that some other candidates must be negatives.


The epistemic verb indicate has as its scope head the token approximation, due to the existence of a clausal complement dependency (ccomp) between them.

the rightmost token of the sentence, significance, has a prepositional modifier dependency (prep to) with approximation.


In fact, setting the threshold to 3 after the shared task, we were able to obtain overall better results (Precision=83.43, Recall=84.81, F-score=84.12, Rank=8/24).

we explicitly targeted precision, and in that respect, our submission results were not surprising.


(b) CUE: either-or SCOPE: either a seqeuncing error or a pseudogene By handling this class to some extent, we could have increased our recall, and therefore, F-score (65 out of 1,044 cues in the evaluation data for biological text involved this class).

we decided against treating this class, as we believe it requires a slightly different treatment due to its special semantics.


(b) FP: possibly FN: possibly splitting them into different roles Left/right expansion strategies were based on the analysis of training data.

we encountered errors caused by these strategies where we found the annotations contradictory.


The classifier with all syntactic features achieves the best F1-score, 2.19% higher than baseline classifier.

in later experiment on evaluation dataset after the shared task, we observed that dependency features actually harmed the performance for full articles dataset.


It seems that the best labeling result of task 1 can be used directly to be the proper intermediate representation of task 2.

the complexity of scope finding for multi-hedge sentences forces us to modify the intermediate result of task 2 for the sake of handling the sentences with more than one hedge cue correctly.


In the near future, we will improve the hedge cue detection performance by investigating more implicit information of potential keywords.

we will study on how to improve scope finding performance by integrating CRF-based and syntactic pattern-based scope finding systems.


We firstly tried to use the token as the basic unit for hedge cues.

several pieces of evidence suggest it is not appropriate.


Even though in the Wikipedia domain the TK+BF score is less than the baseline score, still the performance of the classifiers do not fall much in any of the in-domain and cross-domain experiments.

bF does not have a good performance in 5 of 6 the experiments.


Also, the highest cue level precision of 54.89% was obtained for L class, whereas it was lowered to 51.13% by the addition of S and W features.

the performance improvement is due to the improved recall, which is as per the expectation that syntactic features would help identify new patterns, which lexical features alone cannot.


Our experiments show that the addition of syntactic features helps in improving recall.

the advantage given by syntactic features were surprisingly marginal.


Therefore, in an unsupervised linguistic setting which is rife with ambiguity, modeling this connection can be particularly beneficial.

existing unsupervised morphological analyzers take little advantage of this linguistic property.


These approaches fit within standard statistical approaches to natural language processing, defining statistical objectives and inference strategies, with the learners trying to optimize some combination of the quality of its lexicon and representations of the corpus.

bootstrapping approaches (Gambell and Yang, 2004;Lignos and Yang, 2010) to word segmentation have focused on simple heuristics for populating a lexicon and strategies for using the contents of the lexicon to segment utterances.


Because of the dramatic performance gains shown by the addition of USC in testing, as well as the poor performance of TP, Gambell and Yang conclude that the USC is required for word segmentation and thus is a likely candidate for inclusion in Universal Grammar (Gambell and Yang, 2006).

as the results in Table 2 show, VE is capable of slightly superior performance on syllable input, without assuming any prior constraints on syllable stress distribution.


In tasks where the learned hypothesis is accurate enough, this has no performance loss and it is computationally efficient as the optimal policy is deterministic.

in event extraction the learned hypothesis is likely to make mistakes, thus the optimal policy does not provide a good approximation for it.


During the first stages of language acquisition children make a lot of errors, and parents are not constantly telling them that their sentences are wrong; rather the important thing is that they can communicate with each other.

it is worth studying whether other sources of negative evidence are provided to children.


Combination of instance and feature feedback has been shown to reduce the total annotation cost for supervised learning.

learning problems may not benefit equally from feature feedback.


Direct feedback on a list of features (Raghavan et al., 2006;Druck et al., 2008) is limited to simple features like unigrams.

unigrams are limited in the linguistic phenomena they can capture.


If a concept can be expressed using a few well-selected features from a large feature space, we stand to benefit from feature feedback as few labeled features can provide this information.

if learning a concept requires all or most of the features in the feature space, there is little knowledge that feature feedback can provide.


(2007) analyze benefit from feature feedback at a fixed training data size of 42 labeled units.

the difference between learning problems may vary with the amount of labeled data.


Some problems may benefit significantly from feature feedback even at relatively larger amount of labeled data.

with very large training set, the benefit from feature feedback can be expected to be small and not significant for all problems and all problems will look similar.


2007evaluate benefit from feature feedback in terms of the gain in learning speed.

the learning rate does not tell us how much improvement we get in performance at a given stage in learning.


We found a significant negative correlation (−0.574) between annotation budget (number of AU s) and improvement in performance with feature feedback.

note that this correlation is not very strong, which supports our belief that factors other than the amount of labeled data affect benefit from feature feedback.


So far only a linear relationship of various measures with benefit from feature feedback has been considered.

some of these relationships may not be linear or a combination of several measures together may be stronger indicators of the benefit from feature feedback.


Both select high quality parses by computing the level of agreement among different parser outputs: wheras the former uses several versions of a constituency parser, each trained on a different sample from the training data, the latter uses the parses produced by different dependency parsing algorithms trained on the same data.

a widely acknowledged problem of both supervised-based and ensemble-based methods is that they are dramatically influenced by a) the selection of the training data and b) the accuracy and the typology of errors of the used parser.


For example, a research paper talking about "machine transliteration" may less or even not mention the phrase "machine translation".

since "machine transliteration" is a sub-field of "machine translation", the phrase "machine translation" is also reasonable to be suggested as a keyphrase to indicate the topics of this paper.


Let us take another example: in a news article talking about "iPad" and "iPhone", the word "Apple" may rarely ever come up.

it is known that both "iPad" and "iPhone" are the products of "Apple", and the word "Apple" may thus be a proper keyphrase of this article.


Compared to TextRank, ExpandRank performs better when facing the vocabulary gap by borrowing the information on document level.

the finding of neighbor documents are usually arbitrary.


WAM assumes each translation pair should be of comparable length.

a document is usually much longer than title.


The simplest way to compute W from W is to let W = W , which boils down to using a dense, complete graph G with the unmodified all-pairs similarity as its edge weights.

it has been observed that a sparse W not only save time needed for classification, but also results in better classification accuracy 1 than the full similarity matrix W (Zhu, 2008).


(2009) reported that b-matching graphs achieve semi-supervised classification accuracy higher than k-NN graphs.

without approximation, building a b-matching graph is prohibitive in terms of computational complexity.


From Table 4, we see that mutual k-NN graphs perform significantly better than k-NN graphs.

theere is no significant difference in the accuracy of the mutual k-NN graphs and b-matching graphs.


Their (non-probabilistic) approach finds dictionaries with a minimal number of entries.

the approach does not include a position model.


For the IBM-1 model the first term alone results in a convex, but not strictly convex minimization problem 2 .

eM-like iterative methods generally do not reach the minimum: they are doing block coordinate descent (Bertsekas, 1999, chap.


The above algorithm is fast and can handle large corpora.

it still gets stuck in local minima, and there is no way of telling how close to the optimum one got.


For deriving cuts we tried all the methods implemented in the COIN Cut Generation Library CGL 3 , based on the solver Clp from the same project line.

either the methods were very slow in producing cuts or they produced very few cuts only.


Clearly, the gaps are reduced significantly compared to the LP-relaxation.

except for the IBM-1 (which is convex for λ = 0) the lower bounds are still quite loose.


For example, if stopwords are removed from the corpus, the resulting factors often, but not necessarily, correspond to topics.

if only stopwords are retained, as is commonly done in authorship attribution studies, the resulting factors lose their interpretability as topics; rather, they can be seen as stylistic markers.


The highest LDAH-M accuracy was obtained with 300 topics (Figure 3).

lDAH-S yielded a much higher accuracy than lDAH-M.


For 5000 prolific users (figure omitted due to space limitations), the methods perform comparably, and KOP outperforms LDAH-S by a small margin.

with all the authors (Figure 4(c)), KOP yields a higher accuracy than both LDA+Hellinger variants.


We released an earlier version of such a network that was limited by the fact that only relationships involving at least one monosemous noun had been included, and it was not evaluated on a WSD task (Szumlanski and Gomez, 2010).

the network we present here has relatedness data for over 4,500 polysemous noun targets and 3,000 monosemous noun targets, each of which are related to an average of 27.5 distinct noun senses.


These unintended cluster memberships are bound to cause minor errors in our disambiguation efforts.

our analysis reveals that we do not find such high entropy among the relatives of a polysemous noun that the semantic clustering effect (which is necessary for the success of the disambiguation algorithms described above in Section 3.3) is diminished.


In order to capture subtle and creative opinions, opinion detection systems generally assume that a large body of opinion-labeled data are available.

collections of opinion-labeled data are often limited, especially at the granularity level of sentences; and manual annotation is tedious, expensive and error-prone.


Overall, SSL for opinion detection on movie reviews shows similar trends to SSL for traditional topical classification (Nigam and Ghani, 2000).

the advantages of SSL were not as significant in other data domains.


enforce long distance regularities for more grammatically correct generation.

optimizing both language-model-based probabilities and parser-based probabilities is intractable.


In the 3rd image, the word "way" is chosen to represent "path" or "street" by the image recognizer.

a different sense of way -"very" -is being used in the final output.


Probabilistic history-based models such as MEMMs should be able to avoid (at least some of) such mistakes by performing a Viterbi search to find the highest probability path of the actions.


In this work, we did not conduct extensive feature engineering for improving the accuracy of individual tasks because our primary goal with this paper is to present the learning framework itself.

one of the major merits of using history-based models is that we are allowed to define arbitrary features on the partially completed structure.


Most work on metric learning learns a Mahalanobis distance, which generalizes the standard squared Euclidean distance by modeling the similarity of elements in different dimensions using a positive semi-definite matrix A.

given two vectors x and y, their squared Mahalanobis distance is: the computational complexity of learning a general Mahalanobis matrix is at least O(n 2 ), where n is the dimensionality of the input vectors.


(2005) used a convolutional network and designed a contrastive loss function for optimizing a Eucliden distance metric.

the network of S2Net is equivalent to a linear projection matrix and has a pairwise loss function.


It comes with various ways of cleanly querying and manipulating the data and allows convenient access to the sense inventory and propbank frame files instead of having to interpret the raw .xml versions.

maintaining format consistency with earlier CoNLL tasks was deemed convenient for sites that already had tools configured to deal with that format.


In the coreference resolution space, several works have shown that applying a list of rules from highest to lowest precision is beneficial for coreference resolution (Baldwin, 1997;Raghunathan el al., 2010).

we believe we are the first to show that this high-recall/high-precision strategy yields competitive results for the complete task of coreference resolution, i.e., including mention detection and both nominal and pronominal coreference.


It is thus ideally suited for experimenting with feature selection and other aspects of optimization.

considering all the parameters, it was unfeasible to run an optimization on the amount of data available on CONLL; we focused therefore on feature selection and the choice between single and split classifiers.


The official data have provided gold and automatic parse trees for each sentences in training and development set.

according to statistics, almost 3% mentions have no corresponding constituents in automatic parse trees.


Under normal strategy, such mention will not be recognized and be absent in the clustering stage.

we find that mention has its constituent N P 0 in packed forest.


Since the requirement of this year's task is to model unrestricted coreference, intuitively, we should not constraint in recognizing only noun phrases but also adjective phrase, verb and so on.

we find that most mentions appeared in corpus are noun phrases, and our experimental results indicate that considering constituents annotated with above proposed POS tags achieve the best performance.


Most often this information is used in the form of relationships between words -for example, how semantically similar two words are, or which nouns are the objects of a verb.

it is likely that humans make use of much higher-level information than the similarity between two concepts when processing language (Abelson, 1981).


In all three cases, errors from the parser (or POS tagger) may lead to the deletion of valid mentions.

we found the impact of this was small and was outweighed by the number of spurious mentions removed.


È refers to the probability of coreference link between two mentions produced by our maximum entropy model, and Ü is a binary variable that is set to 1 if two mentions are coreferent, 0 otherwise.

as Finkel and Manning showed, D&B's coreference-only model without transitivity constraints is not really necessary, because they only select the coreference links with probability È ¼ .


The transitivity constraints are formulated as These constraints ensure that when any two corefrent links (e.g., Ü , Ü ) among three mentions exist, the third one Ü must also be a link.

these constraints also bring huge time and space complexity with Ò ¿ constraints (n is number of candidate mention set M, which is larger than 700 in some documents), and cannot be solved in a restricted time and memory environment.


The improvement is due to gains of 3-5% in precision for MUC and B 3 , which are counteracted by smaller losses in recall.

cEAFE shows a loss in precision and a similar gain in recall, resulting in a minimal increase in F-score.


We report results using this scorer in Section 5.

we used the Reconcile-internal versions of scorers to optimize the threshold.


These results are not directly comparable to ours, as indicated by * and **.

we still see that the performance of our type-level transfer method approaches that of bitext-based methods, which require complex bilingual training for each new language.


At the same time, our model forces the residual matrices to be small and this is probably the reason why it performs competitively with the Bridge-CCA.


(2008) and Daumé III and Jagarlamudi (2011) proposed a generative model based on probabilistic canonical correlation analysis, where words are represented by context features and orthographic features 2 .

their experiments showed that orthographic features to be important for effectiveness, which means low per-formance for language pairs with different character types.


Words such as this occur even if using Lex L , and that number increases when Lex S is used.

the proposed methods are able to utilize all the seeds in order to find equivalents for words such as these.


This is because synonyms tend to be linked in the similarity graph and have similar seed distributions.

in the co-occurrence graph, synonyms tend to be indirectly linked through mutual context words, so the seed distributions of the two could be far away from each other.


HAPPY comes next, then DISGUST and FEAR.

at lower confidence levels, HAPPY has equal number of significant features as ANGER and SAD, which is in line with the result in Table 6.


The sequence of words connecting the two entities is a very good predictor of whether they are related or not.

these paths are completely lexicalized and consequently their performance will be limited by data sparseness.