The main advantage offered by SVMs with respect to methods such as those described above is related to the distance (or similarity) function used.

although (Choi, 2000;Hearst, 1997) employ a distance function (i.e.


An alternative to dealing with high dimension data may be to reduce the dimensionality of the data representation.

linear algebra dimensionality reduction methods like singular value decomposition have been adopted by (Choi et al., 2001;Popescu-Belis et al., 2004) in Latent Semantic Analysis (LSA) for the task of thematic segmentation.


On the other hand, dismissing true properties of the data as sampling bias in the training data will result in low accuracy.

the regularization parameter is used to balance the trade-off between these two competing considerations.


In cases where non-linear hypothesis functions should be optimised, each u i can be mapped into ϕ( u i ) ∈ F , where F is a higher dimensional space usually called feature space, in order to make linear the relation between u i and y i .

the original linear learning machine can be adopted in finding the classification solution in the feature space.


word frequencies have been computed to count the number of times that each term (i.e.


We have a small percentage of positive examples relative to the total number of training examples.

in order to ensure that positive points are not considered as being noisy labels, we change the penalty of the minority (positive) class by setting the parameter C + of this class to: where n + is the number of positive training examples, n is the total number of training examples and λ is the scaling factor.


The range of values we select from is: C ∈ 10 −3 , 10 −2 , 10 −1 , 1, 10, 10 2 , 10 3 and γ ∈ 2 −6 , 2 −5 , 2 −4 , ..., 2 4 , 2 6 and for each possible value we perform five-fold cross validation.

we ran the algorithm five times for the 91 = 7 × 13 parameter settings.


We show that the efficiency of these systems can be improved by exploiting some properties of probable constituents, but also that this reliance on probability is problematic for learning from text.

we propose an extension to strict distributional learning that incorporates more information about constituent boundaries.


We would like as much context as possible, but wordsequence contexts become less probable as their length increases, making learning harder.

simple models of context are preferred, although the precise definition varies between systems.


However, we can reduce this further: for a context-pair to be probable, its components must also be probable.

rather than enumerating all patterns in the corpus, we direct the alignment procedure towards patterns where C lef t and C right are probable.


Second, both NP and VP have minimal unit types: nouns and pronouns for NP; verbs for VP.

these types tend to carry more probability mass, since shorter sequences tend to be more frequent.


More damagingly, PP contexts are dominated by open-class words -the top 74% of PP left-contexts are nouns, verbs and adverbs.

a purely probabilistic distributional approach cannot account for prepositional-phrases, since learning data is too sparse.


Intuitively, 'the' occurs frequently at the start of a sentence, and never at the end.


However, these features can also introduce hidden negative effects taking the form of labelling errors that the model makes at places where a model without the gazetteer features would have labelled correctly.

ensuring optimal usage of gazetteers can be difficult.


Other work has supported the use of gazetteers in general but has found that lists of only moderate size are sufficient to provide most of the benefit (Mikheev et al., 1999).

to date the effective use of gazetteers for information extraction has in general been regarded as a "black art".


In these cases the mislabelling appears to be more attributable to the non-gazetteer features, which are in some sense less expressive after being trained with the gazetteer features.

they are less able to decode words that they could previously label correctly.


One interpretation of the findings of our error analysis above is that the addition of the gazetteer features to the model is having an implicit over-regularising effect on the other features.

is it possible to adjust for this effect through more careful explicit regularisation using a prior?


Context segments are short and typically do not involve recursive structures.

we chose to use 1-reversible automata to represent sets of contexts.


Each of the w l has D parameters or dimensions, one for each feature.

the global weight vector w maintains L × D parameters.


In projective languages, we discovered that action WaitRight is not needed.

for projective languages, we just need 4 actions.


However, for analyzing long sentences, the global features usually include some useless information and will confuse the two-step process.

we do not use this method in this shared task.


The features and the strategies for parsing in early and upper stages should be separated into distinct.

we divide the neighboring dependency attachment (for early stage) and normal dependency attachment (for upper stage), and set the neighboring dependency attachment tagger as a preprocessor.


However, the data size of Czech is much larger than other languages; we couldn't finish the training of Czech using the CPOSTAG of the focused word n, before the deadline for submitting.

we used this procedure only for the experiment of Czech.


Fixing bound B , no left dependency may exist between child x i and parent x j such that j−i > B (similarly for right dependencies and B r ).

edge-factored parsing runtime is reduced from O(n 3 ) to O(n(B 2 + B 2 r )).


Note that many of these features are beyond the scope of the edge based factorizations of the unlabeled parser.

a joint model of parsing and labeling could not easily include them without some form of re-ranking or approximate parameter estimation.


A more fine-grained analysis of the Swedish results reveals a high accuracy for function words, which is compatible with previous studies (Nivre, 2006).

the labeled F-score is 100% for infinitive markers (IM) and subordinating conjunctions (UK), and above 95% for determiners (DT).


Since our aim is to add constraints our variables need to explicitly model label decisions.

we introduce binary variables where n is the number of tokens and the index 0 represents the root token.


Because the too early reduce action makes the following words difficult to find the parents.

during training, if a word which is the child of the top of the stack, it is then assigned to the "sleep" category and pushed into stack, otherwise, the conventional reduce action is applied.


For the other languages, usually the identification rate of the longer distance parsing should be lower than the smaller distance.

a future work to parsing Turkish, should put more emphasis on improving not only the noun POS type, but also the distance=2 parsing.


To parse a given sentence, the models are applied sequentially, each one considering candidate word pairs and adding new links without deleting the existing links or creating conflicts (cycles or crossings) with them.

the algorithm can be considered a bottom-up, multi-pass, deterministic parser.


This will identify the link the-ball, and dropping the modifier again leaves us with "kick ball".

doing three passes over this word sequence will bring all linked words into contact and allow us to use our adjacent word linker.


On the one hand, the clusters and their names are based on candidate answers to question; on the other hand, candidates are dependent on question.


When constructing question vector, we assume that the question is an ideal question that contains all the extracted WWFS words.

the values of the WWFS word features in question vector are 1.


Each of the following rows provides the results from consensus decoding for adding an extra system both with/without MERT.

the final row (TC1) combines transla-tions from 12 systems: None, Es, Fr, Ru, Zh, AC1 with/without MERT.


All the bridge languages yield lower BLEU/higher AER relative to the No-Bridge baseline.

our estimate of the posterior probability (Equation 4) is always worse than the posterior probability obtained using a direct model.


The motivation behind Sets 2 and 3 is to train all bridge language systems on as much bitext as possible.

these systems give better results than the corresponding systems in Table 4.


(2006), improved BLEU and NIST scores for phrase-based SMT, but subsequent detailed empirical studies we have performed since then suggest that single word WSD approaches are less successful when evaluated under all other MT metrics (Carpuat and Wu, 2007).

fully phrasal WSD predictions for longer phrases, as reported in this paper, are particularly important to improve translation quality.


We were able to obtain ratings on the entire Ziff-Davis test set as it has only 32 instances; this was not possible for Broadcast news as the test section consists of 410 instances.

we randomly selected 32 source-target sentences to match the size of the Ziff-Davis test set.


For instance, according to Schriefers and Pechmann (1988), an NP such as the large black triangle reflects a conceptualisation of the referent as a black triangle, of which the SIZE property is predicated.

the TYPE+COLOUR combination is not mentally represented as two separable dimensions.


However, even in the VS condition, referents had different basic-level types.

an author faced with a domain like Table 1 had at least the descriptive options in (3a-d).


Together with the earlier findings on logical form, the data evinces a dual strategy whereby (a) sets are partitioned based on basic-level conceptual category; (b) elements of the partitions are described using the same attributes if they are easily perceived and conceptualised.

of the descriptions in (3) above, it is (3b) that is the norm among authors.


For instance, in (5), e 2 needs to be distinguished from the other sofas in Table 1, while {e 1 , e 5 } need to be distinguished from the desks.

distractors are held in an associative array C, such that for all r ∈ R, C[r] is the set of distractors for that referent at a given stage in the procedure.


On later considering COLOUR : blue , the algorithm adds it to D part , since it is contrastive for {e 2 , e 5 }, but will have failed to propagate COLOUR across fragments.

iA part considers values of an attribute in order of discriminatory power (Dale, 1989), defined in the present context as follows: Discriminatory power depends on the number of referents a property includes in its extension, and the number of distractors (U −R) it removes.


The output of IA bool is shown in (10b).

(10) (a)`fan ∧ green´∨`sofa ∧ blue' the green fan and the big sofa' (b)`sofa ∨ fan´∧ small ∧ front ∧`blue ∨ green' the small, blue and green sofa and fan' of IA bool 's requiring a property or disjunction to be true of the the entire set of referents, COLOUR is not included until disjunctions are considered, while values of SIZE and ORIENTATION are included at first pass.


For both types of error corpus, it is not enough to collect a large set of sentences which are likely to contain an error -it is also necessary to examine each sentence in order to determine whether an error has actually occurred, and, if it has, to note the nature of the error.

like the creation of a treebank, the creation of a corpus of ungrammatical sentences requires time and linguistic knowledge, and is by no means a trivial task.


For each sentence in the original tagged corpus, an attempt is made to automatically produce four ungrammatical sentences, one for each of the four error types.

the output of the error creation procedure is, in fact, four error corpora.


Parametrised methods are first optimised for accuracy and then the other measures are taken.

f-scores below the artificial 2/3 baseline are meaningful.


The standard deviation of results across crossvalidation runs is below 0.006 on all measures, except for Method 4.


This method is sound because the data sets for each language contain approximately the same number of tokens -5,000.

evaluating system performance over the aggregated files can be roughly viewed as measuring system performance through an equally weighted arithmetic mean over the languages.


However, MaltParser tends to perform better on shorter sentences, which require the greedy inference algorithm to make less parsing decisions.

the chance of error propagation is reduced significantly when parsing these sentences.


It is important to relate each system's accuracy to a set of linguistic categories, such as parts of speech and dependency types.

we have made an attempt to distinguish a few broad categories that are cross-linguistically identifiable, based on the available documentation of the treebanks used in the shared task.


Specifically, when h = 0, the marginals are given by and for h > 0, the marginals are given by where δ h,m is the Kronecker delta.

the complexity of evaluating all the relevant marginals is dominated by the matrix inversion, and the total complexity is therefore O(n 3 ).


In addition, the hinge loss is 0 if and only if m i,y (w) ≥ E i,y for all y ∈ T (x i ).

the hinge loss directly penalizes margins m i,y (w) which are less than their corresponding losses E i,y .


Moreover, the average sentence length of our test sets is considerably higher than that of the NE-GRA test set.

a slightly lower performance and/or coverage is anticipated, albeit the type and domain as well as the annotation of both the NE-GRA and the TIGER treebanks are very similar.


The additional versions of the parser are inspired by the hypothesis that inclusions make parsing difficult, and this difficulty arises primarily because the parser cannot detect inclusions properly.

a suitable upper bound is to give the parser perfect tagging information.


Due to these grammatical function tags, the perfect tagging parser may disambiguate more syntactic information than provided with POS tags alone.

to make this model more realistic, the parser is required to guess grammatical functions (allowing it to, for example, mistakenly tag an accusative pronoun as nominative, dative or genitive).


In the partial evaluation condition, all levels achieved a highly significant improvement over the baseline (p < 0.001).

the classifications obtained with any of the feature levels are more useful than the baseline, in the sense that they present more overlap with the Gold Standard.


In contrast, ANOVA across the 6 classes (α = 0.05) yields significant differences for 16 out of the 18 features, which indicates that most features serve to distinguish object adjectives from basic and event adjectives.

of the lack of robust distributional differences between basic and event adjectives, 35 basic or event adjectives were classified as BE when using the all level as feature set.


However, these resources are generally not available for resource-scarce languages.

researchers have attempted to derive morphological information heuristically (e.g., Cucerzan and Yarowsky (2000), Clark (2003), Freitag (2004)).


However, the amount of biomedical literature regarding protein interactions is increasing rapidly and it is difficult for interaction database curators to detect and curate protein interaction information manually.

most of the protein interaction information remains hidden in the text of the papers in the biomedical literature.


These parsing approaches consider only syntactic properties of the sentences and do not take into account semantic properties.

although they are complicated and require many resources, their performance is not satisfactory.


However, it is a negative interaction sentence for the KaiA and SasA protein pair, i.e., it does not describe an interaction between this pair of proteins.

before parsing a sentence, we make multiple copies of it, one for each protein pair.


Note that since edit decisions are based in part on (arbitrary) features of the target sequence, and since generation involves construction of the target sequence, it is not uncommon for a greedy generator to make edit decisions which are locally optimal, but which result several edits later in a partially constructed sequence in which no good edits are available.

the problem of generation does not correspond to a simple recurrence relation like Equation 1.


Thus, the problem of generation does not correspond to a simple recurrence relation like Equation 1.

we experimented with several heuristic approaches to generation and found that a beam search works well.


Specifically, the affinity between two words is the maximum number of consonant selfsubstitutions, with any substitutions involving the first five consonants counting for five normal substitutions.

substituting 'b' for 'b' contributes 5 to the score, substituting 'z' for 'z' contributes 1, while operations other than self-subsitutions, and any operations involving vowels, contribute 0.


However, we found that the efficiency of the phrasebased SMT system described in the previous section would be limited for this task, mainly due to two reasons: the character-based phrase models due to possible unseen phrases in an evaluation corpus, and the character sequence model as all candidate transcriptions confidently belong to the target language.

to make the phrase-based SMT system robust against data sparseness for the ranking task, we also make use of the IBM Model 4 (Brown et al., 1993) in both directions.


However, domain specific ontologies generally contains many "fine-grained" categories (e.g., particular categories of people, such as writers, scientists, and so on) and, as a consequence, supervised methods cannot be used because the annotation costs would become prohibitive.

in the literature, the fine-grained classification task has been approached by adopting weakly supervised (Tanev and Magnini, 2006;Fleischman and Hovy, 2002) or unsupervised methods (Cimiano and Völker, 2005).


A great advantage of this methodology is that it is totally corpus based, as it does not require neither the availability of lexical databases, nor the use of complex preprocessing steps such as parsing or anaphora resolution, allowing us to apply it on different languages and domains once large corpora are available for training.

we exploited exactly the same strategy to implement the IBLE classifier required for our purposes, defining a kernel composed by n simple kernels, each representing a different aspect to be considered when estimating contextual similarity among word occurrences.


As pointed out by (Campbell, 2004), this metric is insensitive to the correct attachment of the EC into the parse tree, and more importantly it is not clear whether it adequately measures performance in predicateargument structure recovery.

we use a predicate-argument based evaluation method instead.


This not only reduces the size of the phrase table, but also improves topic relevancy of the extracted phrase pairs.

s, it improves both the efficiency and the performance of machine translation.


For example, given the following hypothesis path,
 izzat ibrahim

 receives

 an economic official

 in

 baghdad
 We find the source phrase containing words [0,1] is first translated into a target phrase "izzat ibrahim", which is followed by the translation from source word 2 to a single target word "receives", etc.. We identify the word alignment within the phrase translation pairs based on IBM model-1 scores.

we get the following source word translation sequence from the above hypothesis (note: source word 5 is translated as NULL): 0 < 1 < 2 < 4 < 3 < 6 < 7 Such decoding sequence determines the translation order between any source word pairs, e.g., word 4 should be translated before word 3, 6 and 7.


When v 0, f (v) ≈ v − 0.5, so roughly speaking, VB for multinomials involves adding α−0.5 to the expected counts when they are much larger than zero, where α is the Dirichlet prior parameter.

vB can be viewed as a more principled version of the wellknown ad hoc technique for approximating Bayesian estimation with EM that involves adding α−1 to the expected counts.


Case assignment CA k represents correspondences between the input case components and the case slots shown in Figure 4.

the generative probability of a case structure is decomposed as follows:  Figure 4: Example of case assignment.


As per the annotation criteria, each bunsetsu has only one modifying head.

in some cases, even if analysis results are semantically correct, they are judged as incorrect from the viewpoint of the annotation.


Although this algorithm is the starting point for our algorithm, its baseline performance is not outstanding.

we present a margin extension to the Collins' perceptron in Section 3.


Then, feature mapping is written as Here, we define: Ideally, we want to determine the labels using the whole feature set as: Algorithm 4.1: Candidate algorithm (parameters: However, if there are non-local features, it is impossible to find the highest scoring candidate efficiently, since we cannot use the Viterbi algorithm.

we cannot use the perceptron algorithms described in the previous sections.


The automatic classification results were evaluated against human judgement, and the performance was encouraging, with accuracy reaching over 85% in some cases.

while human judgement is not straightforward and it is difficult to create a Pan-Chinese lexicon manually, it is observed that combining simple clustering methods with the appropriate data sources appears to be a promising approach toward its automatic construction.


Furthermore, if this hypothesis holds, one would need to consider separate mechanisms to cluster heterogeneous regionspecific words in the Pan-Chinese context.

in the current study we sampled 30 target words specific to each of Beijing, Hong Kong, Singapore, and Taipei, from the financial domain; and used the cosine similarity function to classify them into one or more of the semantic categories in Cilin.


However, due to the lack of editorial and quality control, reviews on products vary greatly in quality.

it is crucial to have a mechanism capable of assessing the quality of reviews and detecting low-quality/noisy reviews.


At the Amazon site, users could vote for a review with a "helpful" or "unhelpful" label.

for each review there are two numbers indicating the statistics of these two labels, namely the number of "helpful" votes and that of "unhelpful" ones.


This is simply because reviews posted earlier are exposed to users for a longer time.

some high quality reviews may get fewer users' vote because of later publication.


In our study, we focus on differentiating lowquality product reviews from high-quality ones.

we treat the task as a binary classification problem.


With the increasing of training data, the improvement of translation quality will become smaller and smaller.

while keeping collecting more and more parallel corpora, it is also important to seek effective ways of making better use of available parallel training data.


We keep the notion of latent semantic concepts and also try to preserve the specifics of the document collection.

we divide the vocabulary into two sets: nouns and the rest of the vocabulary.


The co-occurrence statistics for PMI are typically computed using a sliding window.

pMI will be large only for words that co-occur within a small context of fixed size.


We could not obtain satisfactory results with this approach.

we used the average similarity by using a history of fixed size n. The semantic cohesion score was computed for the position between two text units, t i and t j as follows: Our approach predicts story boundaries at the minima of the semantic cohesion score.


We require an initial set of alignments, which we obtain from a wordfor-word alignment procedure such as GIZA++ or EMD.

we are not aligning input data, but rather re-aligning it with a syntax model.


While short stories are usually concerned with one topic at a time, long documents such as books often cover more than one topic.

the intuition is that a summary should include content covering the important aspects of all the topics in the document, as opposed to only generic aspects relevant to the document as a whole.


The underlying assumption is that words with similar meanings occur in similar contexts, and will be assigned similar vectors.

the distance between the vectors of two target words, as given by some distance measure (e.g., Cosine or Jaccard), is a measure of their semantic similarity.


With respect to upper bounds, we hold that automatic models of plausibility cannot be expected to surpass the typical agreement on the plausibility judgement task between human participants.


As mentioned in Section 4, the dataset consists of verb-argument pairs drawn from two different corpora.

each verb is combined both with some arguments that are seen in FrameNet, and some that are not.


Notice that the computations of homogeneity, completeness and V-measure are completely independent of the number of classes, the number of clusters, the size of the data set and the clustering algorithm used.

these measures can be applied to and compared across any clustering solution, regardless of the number of data points (n-invariance), the number of classes or the number of clusters.


Without manual modification however, k * = n, where each cluster contains only a single data point.

while technically n-invariant, the possible values of V I are heavily dependent on the number of data points being clustered.


Thus, while technically n-invariant, the possible values of V I are heavily dependent on the number of data points being clustered.

it is difficult to compare V I values across data sets and clustering algorithms without fixing k * , as V I will vary over different ranges.


However, that is computationally impractical because a complicated integral computation is needed.

we estimate an approximate distribution of P (π|w, y, α, θ) using Variational Bayes Method(H. Attias, 1999).


log P (w|y, α, θ) is not relevant to Q(π, z|γ, φ).

q(π, z|γ, φ) that maximizes F [q] minimizes KL(q, P ), and gives a good approximate distribution of P (π, z|w, y, α, θ).


This task provides a very limited amount of resources in comparison to other tasks like the translation of journal texts (NIST evaluations) or of parliament speeches (TC-STAR evaluations).

new techniques must be deployed to take the best advantage of the limited resources.


It can be shown that the outputs of a neural network trained in this manner converge to the posterior probabilities.

the neural network directly minimizes the perplexity on the training data.


This is more difficult to perform when the continuous space translation model is used during decoding.


Typically, news articles are written such that the first sentence summarizes the article.

we include a binary feature F (S i ) that equals 1 if S i is the first sentence of the document: F (S i ) = δ i,1 , where δ is the Kronecker delta function.


The distribution of expressions in a business-oriented conversation will change almost synchronously because the call flow is predefined.

acc(categorizer(D k )) will increase if features that contribute to the categorization appear in D k .


In contrast, acc(categorizer(D k )) will decrease if no features that contribute to the categorization are in D k .

from the transitions of acc(categorizer(D k )), we can identify the segments with increases as triggers where the features that have an effect on the outcome appear.


By looking at some documents containing these expressions, we found customer intention phrases such as "would like to make a reservation", "want to check a rate", etc.

it can be induced that the way a customer starts the call may have an impact on the outcome.


For English, there is the SemCor corpus (Fellbaum, 1998) which is annotated with parse trees and Word-Net senses, but it is fairly small, and does not explicitly include any structural semantic information.

we decided to construct and use a treebank with both syntactic information (e.g.


With informed decisions of human annotators made available, the potential for future disagreement of the classifier committee on conflicting instances should decrease.

each AL iteration selects the b sentences with the highest disagreement to focus on the most controversial decision problems.


It has been shown that Conditional Random Fields (CRF) (Lafferty et al., 2001) achieve higher performance on many NLP tasks, such as NER, but on the other hand they are computionally more complex than an ME classifier making them also impractical for the interactive AL process.

in our committee we employ ME classifiers to meet requirement 1 (fast selection time cycles).


This rate will descend as the classifiers get more and more robust in their predictions on unseen data.

an average disagreement approaching zero can be interpreted as an indication that additional annotations will not render any further improvement.


For example, in 3Peter Lin's approach would yield ↑subj :like for Peter and ↑dobj :like for ice-cream, while Padó and Lapata's approach would yield the contexts like (with a weight of 1.0) and ice-cream (with a weight of 0.5) for Peter.

padó and Lapata's measure is more robust against data sparseness but also finds related non-similar terms (which are ultimately unwanted for coreference resolution).


Most of the time, they can only handle a single switch device.

communicating with an AAC system consists of typing messages by means of a virtual table of symbols (words, letters or icons) where the user successively selects the desired items.


However, it is difficult to make the calculations without an approximation.

in the first step, we find the best candidates for D given the input sentence f .


Equation 4illustrates that a good model will provide high probabilities to P (D|f )P (e|f, D) for bilingual sentence pairs (f, e).

for the same reason, a good domain definition will lead to a higher probability for the term: we define the domain D as that which maximizes P (D|f )P (e|D) (an approximation of P (D|f )P (e|f, D)).


P (e|f, D) can be re-written as the following equation using Bayes Here, f is a given constant and D has already been selected by the domain prediction process.

maximizing P (f |e, D)P (e|D) is equivalent to maximizing the above equation.


For almost f , only one domain D i has a vary large value compared with other domains.

this approximation is confirmed to be reasonable.


Our baseline performance was slightly different from that reported in the RWTH experiments (21.9 BLEU socre for RWTH's system and 21.7 for our system).

their improved baseline is shown for comparison.


The most common approach is to use the maximum aposteriori (MAP) decision rule.

we select the hypothesis which maximizes the posterior probability P r(e I 1 |f J 1 ):êÎ This is equivalent to the MBR decision rule under a 0-1 loss function: Hence, the MAP decision rule is optimal for the sentence or string error rate.


The denominator represents a normalization factor that depends only on the source sentence f J 1 .

we can omit it in case of the MAP decision rule during the search process and obtain: Note that the denominator affects the results of the MBR decision rule and, thus, cannot be omitted in that case.


the N best translation hypotheses of the MT system.

the training criterion for the sentencelevel expected Bleu computation is: An advantage of the sentence-level computation is that it is straightforward to plug in alternative evaluation metrics instead of the Bleu score.


Given two peer phrases, we have two choices to use one of them.

we use maximum entropy (ME) model algorithm to estimate their reordering probabilities separately, where the boundary words of foreign phrases and candidate target translation phrases, POS information and dependencies are integrated as features.


In general, we add virtual nodes for each set of the continuous peer phrases and let them have the same height.

for a n-ary sub-tree, there are where n>2.


Table 4: Performance against baseline system These three systems are the same in that the final sentence translation results are generated by the combination of local phrase translation.

they are capable of local reordering but not global reordering.


The path, and in particular the middle verb, provides a connection between the two words of the word-pair, and hence is a good indication of their semantic relation.

for each word-pair extracted, we also extract the dependency pattern derived from the word-pair's dependency path as a feature for the word-pair.


Further, the dimensionality reduction inherent to LSA has the effect of making the predominant sense more dominant while de-emphasizing the other senses.

an LSA-based approach will also conflate information from the different senses, and even more emphasis will be placed on the predominant senses.


measures can be used to measure distance between a word pair by choosing the distance between their closest senses.

even though 'children's recreation' is the predominant sense of play, the 'drama' sense is much closer to actor and so their distance will be chosen.


We know that words that occur close to a target word tend to be good indicators of its intended sense.

we make a second pass of the corpus, using the base WCCM to roughly disambiguate the words in it.


An evaluation, therefore, requires an L 1 that in actuality has adequate knowledge sources.

we chose German to stand in as the resource-poor language L 1 and English as the resource-rich L 2 ; the monolingual evaluation in German will use GermaNet.


zKL's re-estimate for the zero terms adds little information.


However, tree structures include relations between two nodes other than the parent-child relation.

we use ancestor-descendant relations in addition to parent-child relations, so that the added redundancy helps errors be corrected.


However, those rule-based approaches have problems in coverage and consistency.

a number of statistical techniques using machine learning algorithms have recently been proposed.


A number of conventional models use the pair of 3(pizza-and) and 5(ate) as a negative instance because 3 does not modify 5.

those models cannot learn and use the subcategorization preference of verbs well in the parallel structures.


When the ancestor-descendant relation is used, both of the above two instances for nodes 3 and 4 can be considered as positive instances.

it is expected that the ancestordescendant relation helps the algorithm capture the characteristics that cannot be captured well by the parent-child relation.


In exploiting ancestor-descendant information, it came to us that redundant information is effectively utilized in a coding problem in communications (Mackay, 2003).

we propose a method in which the problem of determining the modifiee of a chunk is regarded as a kind of a coding problem: dependency is expressed as a sequence of values, each of which denotes whether a parent-child relation or an ancestordescendant relation holds between two chunks.


P arent[node i ] indicates the node to which node i is attached, and Dis indicates a distance function.

our method predicts the correct actions by measuring the distance between the codeword string[k] and the predicted binary (later extended to real-valued) sequences string output.


The type C is not available in the proposed method because the proposed method analyzes a sentence backwards unlike Kudo and Matsumoto (2002).

we did not use the type C. We used the type A' and B' which are recursive expansion of type A and B as the dynamic features ( Figure 7).


In (C), any one-bit error can be corrected.

(B) has the higher errorcorrecting ability than (A), and (C) has the higher error-correcting ability than (B).


The receiver infers the parent node from the received sequence (string output) in consideration of the codewords that can be transmitted (string[k]).

error-correcting ability, the ability of correcting the errors in predictions in step 3, is dependent on the distances between the codewords (string [k]).


The codewords in PARENT-ANCESTOR METHOD are the concatenation of the bits based on both parent-child relations and ancestor-descendant relations.

the distances between codewords in PARENT-ANCESTOR METHOD are longer than those in PARENT METHOD or AN-CESTOR METHOD.


The net effects of these edit operation sequences are identical, in that they both skip one word each from the two sequences to be aligned.

there is no use in discriminating between these two, and one of them, namely (Delete, Insert), is prohibited.


In Figure 2, the augmented Outside edges in this unique path are plotted as dotted lines for illustration.

we obtain a complete path which is compatible with Collins's perceptron-based sequence learning method.


Recall that the meaning assigned to a state in pattern grammar in Section 2.2 is the set of patterns matching at that state.

according to that semantics, only the edges displayed bold in Figure 8   Pattern trees are fairly shallow (partly because many patterns cover several layers in a parse tree as can be seen in Figures 1 and 2); therefore, the context associated with a production covers a good part of a pattern tree.


With this in mind, we question the assumption that the PARSEVAL results for NEGRA and TüBa-D/Z reflect a real difference in quality between the parser output for parsers trained on the two different treebanks.

we also question the conclusion that PARSEVAL results for German in the same range as the parsing results for the English Penn-II Treebank prove that German is not harder to parse than the more configurational English.


Both the cost function of Section 2.2 and the feature representation of Section 2.3 are defined over a short sequence of segment boundaries.

using the Markovian property, one can perform the above maximization step efficiently via a dynamic programming algorithm.


For example, one can define a feature j by where each event in the event space is a pair (c , p ) of same-sentence tokens in comparable English text, all pairs being equally likely.

to estimate Pr(• | •), the denominator counts same-sentence token pairs (c , p ) in the comparable English corpus that translate into the types (c, p), and the numerator counts such pairs that are also related by a prep ←− of pobj ←− path.


We already have Z i from running the inside algorithm, and we can find the numerator by running the inside algorithm again with θ scaled by α.

with Rényi entropy, all computations and their gradients are O(n 3 )-even in the nonprojective case.


The main explanation for this improved performance is that on many training examples, 10 in step 1 of the new algorithm a correct parse is found, and the algorithm immediately moves on to the next example.

gENLEX is not required, and in particular parsing the example with the large set of entries generated by gENLEX is not required.


Since treebanks generally consist of coarselylabeled context-free tree structures, the maximumlikelihood treebank grammar is typically a poor model as it makes overly strong independence assumptions.

many generative approaches to parsing construct refinements of the treebank grammar which are more suitable for the modeling task.


Our aim has been to build a named entity recognition and disambiguation system that employs a comprehensive list of entities and a vast amount of world knowledge.

we turned our attention to the Wikipedia collection, the largest organized knowledge repository on the Web (Remy, 2002).


(1997) noted, it is fairly common for one of the mentions of an entity in a document to be a long, typical surface form of that entity (e.g., "George W. Bush"), while the other mentions are shorter surface forms (e.g., "Bush").

before attempting to solve the semantic ambiguity, the system hypothesizes in-document coreferences and maps short surface forms to longer surface forms with the same dominant label (for example, "Brown"/ PERSON can be mapped to "Michael Brown"/ PERSON ).


Note that the performance of the disambiguation component is meaningful only when most named entity mentions are accurately identified in text.

we first measured the performance of the named entity recognition component on the CoNLL 2003 test set and obtained a competitive F-measure of 0.835 (82.2% precision and 84.8% recall).


Furthermore, we wanted to create a classifier that does not depend on manually anno-tated sentence data so that our system can be easily ported across domains.

we devised a method to self-train a classifier using a training set of relevant and irrelevant documents for the domain, and a few seed patterns as input.


could potentially have a high semantic affinity for one of the semantic categories, but they are not likely to be useful if they mainly occur in irrelevant documents.

we also define a lower conditional probability threshold θ l that identifies irrelevant extraction patterns.


To counter this, we apply the Primary patterns to all the sentences.

if we compare each Rel row with the Sel row immediately below it, we see the effect of loosening the reins on the Primary patterns (the Secondary patterns are still restricted to the relevant sentences).


LCPs are similar to PPs and typically map to prepositional phrases in English.

they are handled similarly to PPs, i.e., LCPs in a parent VP are repositioned after the sibling VP.


Third, special constructions that do not occur often in the data are less likely to have a significant impact on the translation performance.

our strategy has been to find reordering rules for syntactic categories that are common in the data and systematically different between the two languages.


Syntax-based translation models (Eisner, 2003;Galley et al., 2006; are usually built directly from Penn Treebank (PTB) (Marcus et al., 1993) style parse trees by composing treebank grammar rules.

often no substructures corresponding to partial PTB constituents are extracted to form translation rules.


(2006a), binary features were trained only on a small development set using a variant of voted perceptron for reranking k-best translations.

the improvement is merely relative to the baseline translation system, namely whether or not there is a good translation in their k-best.


Thus, we cannot always assign scores for each reference translation.

possible oracle translations are maintained according to an objective function, i.e.


Target bigram features account for only the fluency of the target side without considering the source/target correspondence.

the in-clusion of target bigram features clearly overfit to the development data.


Figure 2 demonstrates this with more clarity.

the hypothesis formed in Section 1 is confirmed: there is a strong correlation between the sparseness degree and the categorization error rate.


Table 2 Effectiveness of three stopping conditions As shown in Table 2, the min-error strategy based on feedback of human annotator is very close to the ideal method.

when comparing to ideal stopping condition, min-error strategy is a good choice as stopping condition for active learning.


With this approach, the parameter is estimated to maximize the likelihood of labeled data and the negative conditional entropy of unlabeled data.

the structured predictor is trained to separate unlabeled data well under the entropy criterion by parameter estimation.


Using , we can derive the third line from the second line, where p D i (x; λ i ) γ i for all i are canceled out.

our hybrid model is constructed by combining discriminative models, Hereafter, let us assume that our hybrid model consists of CRFs for discriminative models, p D i , and HMMs for generative models, p G j , shown in Equation (2), since this paper focuses solely on sequence modeling.


According to Bayes' rule, p(y|x; θ) ∝ p(x, y; θ), the discriminant functions of generative classifiers are provided by generative models p(x, y; θ).

we can regard L(θ) as the logarithm of the sum of discriminant functions for all missing variables y of unlabeled data.


Following this view, we can directly incorporate unlabeled data into our hybrid model by maximizing the discriminant functions g of our hybrid model in the same way as for a mixture model as explained above.

we maximize the following objective function for estimating the model parameters Θ for generative models of unlabeled data: where p(Θ) is a prior probability distribution of Θ.


To estimate Γ (t+1) , namely procedure 4.2 in Figure 1, we employ the derivatives with respect to γ i and γ j shown in Equation 6, which are the parameters of the discriminative and generative models, respectively.

we obtain the following derivatives with respect to γ i : The first and second terms are constant during iterative procedure 4 in our optimization algorithm shown in Figure 1.


The only difference is that our method uses marginal probabilities given by R instead of the p(x, y; θ) of standard HMMs.

only a forward-backward algorithm is required for the efficient calculation of our parameter estimation process.


First, (Ando and Zhang, 2005) does not describe the unlabeled data used in their NER experiments in detail, and second, we are not licensed to use the TREC corpus including WSJ unlabeled data that they used for their Chunking experiments (training and test data for Chunking is derived from WSJ).

we simply used the supplied unlabeled data of the CoNLL-2003 shared task for both NER and Chunking.


But first order models are incapable of learning that fact.

in order to improve performance, it is often necessary to include scoring functions that span over larger sequences.


However, it also increases the number of parameters the model must learn, making the model more susceptible to sparse data problems.

the usefulness of a model structure for a given task will depend on the types of constraints that are important for the task itself, and on the size and diversity of the training corpus.


In particular, in cases where the transformed value is not generated by the FST, we can assume that one or more of the transformed tags was chosen incorrectly; and make the minimal set of changes to those tags that results in a string that is generated by the FST.

we can compute the optimal un-transformed output value corresponding to each transformed output using the following procedure: 2.


The next tag, O, is used for all words outside of chunks.

the hill-climbing system found that increasing the window size used for O chunks does not help to learn any useful constraints with neighboring tags.


By dividing the I tag in two based on this feature, the model can use separate distributions for these two cases.

the model avoids conflating two contexts that are significantly different from one another for the task at hand.


This distance can be lengthened by moving to higher order model structures, but only at the expense of an increase in the number of model parameters, along with the data sparsity issues that can arise from that increase.

it makes sense to be more selective about how we extend the model structure.


In this task, it was also shown to outperform tf • idf .

the statistical language modeling approach to retrieval has been shown to be superior to the vector space approach for both these IR tasks.


Moreover, the sequence labeller has to be re-trained for each new domain (e.g., natural history vs. archaeology) and possibly also each sub-domain (e.g., insects vs. mammals) due to the fact that the inventory of fields varies.

fully supervised machine learning is not feasible for this task.


However, in the database, on which all language models are trained, RMNH never occurs as a symbol in the registration number column; it does occur a few times in the column for special remarks but never at the start of the text.

a segmentation model trained on the contents of the database, on encountering the symbol RMNH will always opt for continuing the existing segment as opposed to starting a new one, which is most likely the better choice.


An MCA ∼ defines a partition of the set of all word indices c ik {c i k }, which is of size n |c| = i n i .

the number of distinct MCAs of G is the number of partitions of a set of size n. This number is called the n th Bell number (Rota, 1964)  Asymptotically, B n grows faster than an exponential but slower than a factorial.


However, unlike sequence alignments, there are no ordering constraints in word alignments, and the alignments are many-to-many as opposed to one-to-one.

the SMT HMM models cannot be based on pair-HMMs, which generate two sentences simultaneously.


Since the union of all the pairwise alignments results in only a single test MCA, it is hard to make strong arguments about the performance of the system in general.

we concentrate our discussion on general trends, and do not claim that the specific performance numbers we report here are statistically significant.


If a slight reduction in translation quality is allowed, then the average network latency per batch can be brought down to 7 milliseconds by reducing the number of n-grams requested per sentence to around 10,000.

our system can efficiently use the large distributed language model at decoding time.


Having such information available to the translation model allows the direct modeling of these aspects.

for instance: reordering at the sentence level is mostly driven  we extended the phrase-based approach to statistical translation to tightly integrate additional information.


While this problem does not show up as strongly in English -due to the very limited morphological inflection in English -it does constitute a significant problem for morphologically rich languages such as Arabic, German, Czech, etc.

it may be preferably to model translation between morphologically rich languages on the level of lemmas, and thus pooling the evidence for different word forms that derive from a common lemma.


One important thing to note at this stage, is that an analogical equation may have several solutions, some being legitimate word-forms in a given language, others being not.

it is important to select wisely the generated solutions, therefore Step 3.


Note that in the second example, the reference translation is in the plural form while the French word is not.

we conducted a manual evaluation of the translations produced from L 100 000 by ANA-LOG and BASE2 on the 127 French words of the corpus test-in 8 unknown of L ref .


10 Indeed, they used a rather idealistic set of large, homogeneous bitexts (European parliament debates) to acquire paraphrases from.

we feel our approach is more suited for translating "low density" languages and languages with a rich morphology.


Unfortunately, unlike in the case of HMMs and PCFGs, our model permits no tractable dynamic program to compute these counts exactly.

we resort to a Monte Carlo E-step, where many samples of the edit variables are collected, and counts are computed based on these samples.


In the E-step, we fix the parameters, which renders the word types conditionally independent, just as in an HMM.

we can process each word type in turn without approximation.


The forward score for the merged state s at time t is just the sum of the two split scores: while the backward score is a weighted sum of the split scores:β where p 1 and p 2 are the relative (posterior) frequencies of the states s 1 and s 2 .

the likelihood after merging s 1 and s 2 at time t can be computed from these merged forward and backward scores as: where the second sum is over the other substates of x t , i.e.


This method is much more computationally expensive, requiring a full forward-backward pass through the data for each potential merge, and was not found to produce noticeably better performance.

all experiments use the approximate criterion.


The similarity metric and threshold are implicit in the training data.

under this approach a key function can be learned even if the similarity model is unknown, provided that sufficient equivalence classes are available.


This is because each key function in ETK applies only transformations representing orthographic differences between names in the same equivalence class.

the transformations are very conservative.


Furthermore, there was an inconsistency in how main and auxiliary verbs were annotated for this data set relative to others.


Secondly, two teams submitted two systems each, which are described in a single paper by each team.

the systems called "Nilsson" and "Hall, J."


For Chinese, the same training set was used.

the drop from last year's top score to this year's is surprising.


Comparing the results for different languages, we see a tendency that languages with rich morphology, usually accompanied by flexible word order, get lower scores.

the labeled attachment score is below 80% for Arabic, Basque, Czech, Greek, Hungarian, and Turkish.


For languages with many dependency labels or long sentences, training and parsing becomes impractical for current machines.

a promising line of research is the investigation of methods to efficiently incorporate higher-order relations in discriminative parsing.


4 Although we refer the reader to the source text for a full description of this data structure, its salient characteristic is that it implements priority queue operations insert and next-element in O(log log |T |) time.

the total running time for an algorithm to find all contiguous subpatterns and compute their collocations .


We exploit the fact that our large text is actually a collection of relatively short sentences, and that collocated patterns must occur in the same sentence in order to be considered a rule.

we can use the sentence id of each subpattern occurrence as a kind of hash key.


our solution focuses on computations where one or more of the component patterns is frequent.


We cannot easily obtain a list of these in advance, but we observe that they always consist of a pair u and v of patterns from near the top of the frequency list.

we also precompute the locations uXvXw of patterns in which both u and v are among these super-frequent patterns (all unigrams), treating this as the collocation of the frequent pattern uXv and frequent pattern w. We also compute the analagous case for u and vXw.


They observe that the region of the suffix array containing all suffixes prefixed by ua is a subset of the region containing the suffixes prefixed by u.

if we enumerate the phrases of our sentence in such a way that we always search for u before searching for ua, we can restrict the binary search for ua to the range containing the suffixes prefixed by u.


Each node in the tree repre- (2)'s suffix is represented by node (3).

node (1)'s suffix is represented by the node pointed to by the d-edge from node (3), which is node (4).


To further optimize the word sausages, we replace each system prior weight ω m with the l p -norm over the normalized scalar product between the weight vector ω and the row vector b m : m 1 if and only if system m has the highest consensus among all input systems; otherwise, ω I m 0.

the word sausages are able to emulate the candidate selection rule described in Section 2.1.


However, query expansion is needed for this strategy to be effective.

to find transliterations of a name, a promising approach is to automatically expand the given name into a query with the additional requirement of some morpheme expected to be part of relevant transliterations that might appear on the Web.


Clearly any amount of clustering will only increase WSD performance.

to account for this natural improvement and consider only the effect of our particular clustering, we also calculate the expected score for a random clustering of the same granularity, as follows: Let C represent the set of clusters over the possible N synsets containing a given word; we then calculate the expectation that an incorrectly-chosen sense and the actual correct sense would be clustered together in the random clustering as .


(As a canonical example, the ubiquitous hidden Markov model is a series of mixture models chained together.)

developing a generative model for WSD gives other generative NLP algorithms a natural way to take advantage of the hidden senses of words.


First, it describes a random process for word generation.

it is a distribution over words and thus can be integrated into any generative model of text, such as topic models.


Choose the next node in the walk λi+1 Every element of this process, including the synsets, is hidden except for the words of the documents.

given a collection of documents, our goal is to perform posterior inference, which is the task of determining the conditional distribution of the hidden variables given the observations.


to provide a unified account for the detection and encoding of these distinct but related phenomena is a real challenge for NLP systems.


The order of the trigrams could be very different inside each corpus.

a second test is needed to compare the rankings of the n-grams in each corpus.


Unfortunately disproving the null hypothesis does not give much information about the degree of correlation; it only asserts that it exists.

it could be a very insignificant correlation.


The statistical measures discussed here are all different forms of measuring correlations between the component words of MWEs.

as some types of MWEs may have stronger constraints on word order, we believe that more visible effects can be seen in these measures if we look at their application for individual types of MWEs, which is planned for future work.


Only those predictions that occurred 5 times or more were taken into consideration for the generation of the new lexical entries.


This integration algorithm can be represented as follows: where p is one of parties mentioned in a message, m is the number of sentences that contains party p in a message, and Valence k (p) is the valence of p in the k th sentence that contains p. Given the example in Figure 2, the Liberal party appears twice in sentence S0 and S1 and its total valence score is +2, whereas the NDP party appears once in sentence S1 and its valence sum is -1.

our algorithm picks liberal as the winning party that the message predicts.


We expanded the dictionary by hand with external resources including publicly available ordinal thesauri.


"good" expresses the evaluation for "the weather", but "the weather" is not an aspect of digital cameras.

the weather, good is not an opinion in the digital camera domain.


We evaluated the results by recall R and precision P defined as follows Note that, in aspect-of relations, we permit A,C to be correct when the data includes the chain of aspect-of relations A,B and B,C .

we merged the intra-and inter-sentential results as shown in Table 4.


We also conducted a preliminary test of the opinion-hood determination model using the features used in aspect-evaluation relation extraction.


(2) watashi-wa ebi-ga suki-desu I shrimps like (I like shrimps.)

we need to conduct further investigation in order to resolve this kind of problems.


Nowadays, terabyte is not surprisingly large, and larger corpus would be obtained in the future.

we think this kind of research direction is important.


They learn MSA in school, and have no native-speaker intuition about case.

determining case in MSA is a hard problem for everyone, including treebank annotators.


Furthermore, we suspect that the hand-written rules have very few false positives (i.e., cases in which the treebank has been annotated in error but our rules predict exactly that error).

we believe that our tool can serve an important function in improving the treebank annotation.


can only take one of the three values: +1, -1, or 0.

the training examples can be divided into three subsets with respect to k: The new loss after adding the update parameter δ to the parameter α k is shown below: The best feature/update pair (k * , δ * ) that minimizes Loss(ᾱ, k, δ) is determined using the following formulas: The update formula in Equation 6 is problematic when either W + k * or W − k * is zero.


Inflectional and derivational morphemes don't attach to words of just any POS category; they are particular.

we can propose the possibility of JJ (adjective) to "broad" and VB or VBP to "govern" (based on the fact the derivational morphemes "en" and "ment" attach to them) even though by themselves they don't have any suffix information that might be indicative of JJ and VB or VBP.


For RFreq vector, instead of ones and zeros, we first start with the frequency of occurrences of each word and then normalize so that the sum of all feature values is one.

for example, a word with 4 features having non-zero frequencies of 10, 20, 30 and 40 will have the respective values set to 0.1, 0.2, 0.3 and 0.4.


This was not, however, thought sufficient to resolve the global syntactic problem of root attachment where (wrong) statistical preferences could be so strong that even 20 rounds of penalties could not weaken them sufficient to be ruled out.

root and root attachments supported by the CG trees were fixed in the first pass, without reruns.


We define the loss of a dependency tree as the number of words which have an incorrect parent.

the largest loss a dependency tree can have is the length of the sentence.


Table 5 shows the comparison of the proposed method with the average, and three top systems on the CONLL-2007.

our method yields results above the average score on the CONLL-2007 shared task (Nivre et al., 2007).


Although no edge is present in unlabeled data, the presence of a preposition indicates that this edge between the tokens, if existed, will not be a noun modifier (in English corpus, this label is NMOD).

this induced feature should correlate with the label of an edge candidate.


We then train a new model based on counts from these consistent derivations which are weighted in some manner by our confidence in them, given both the degree of remaining ambiguity and also the ranking and/or derivation probabilities provided by the initial model.

the first and hardest step was to map the DR scheme to our GR scheme.


The performance over our out-of-domain PTB-derived test data remains approximately the same for all these models.

we chose to use Model G for the blind test as it incorporates most information from the in-   For the blind test submission we used Models G and H to parse the PCHEMTB and CHILDES data, respectively.


By contrast, the GR scheme does not output punctuation marks directly but follows Nunberg's (1990) linguistic analysis of punctuation as delimiting and typing text units or adjuncts (at constituent boundaries).

the GR scheme includes text (adjunct) relations and treats punctuation marks as indicators of such relationsfor instance, for the example The subject GRs -ncsubj, xsubj and csubj -all have subtypes., RASP outputs the GR (ta dash GRs and) indicating that the dash-delimited parenthetical is a text adjunct of GRs with head and, while the DR scheme gives (DEP GRs and), and two (P and -) relations corresponding to each dash mark.


This means that the closest matching derivation which is used for training the initial model is on average only around 85% similar even by the unlabelled measure.

the mapping procedure required to relate the annotated data to RASP derivations is introducing considerable noise into the training process.


As observed by (Isozaki et al., 2004), incorporating both top-down (root find) and bottomup (base-NP) can yield better improvement over the Yamada's parsing algorithm.

instead of pre-determining the root and base-phrase structures, the tagging results of the neighbor and root parsers were included as new features to add wider information for the shift-reduce parser.


The 580 newly introduced labels consist of a standard PTB label followed by a set of one or more PropBank semantic role such as PP-AM-TMP or NP-A0-A1.

of lowering the six AM-X semantic role labels, 240 new part-of-speech tags were introduced to partition the original tag set which consisted of 45 tags.


The features could potentially look at any information in the labels, which are of the form POS, A, η h , η m , L , but in our experiments, we map labels to a pair (GRM( POS, A, η h , η m , L ), A).

the label features are sensitive only to the triple of nonterminals corresponding to the grammatical relation involved in an adjunction, and a binary flag specifiying whether the operation is s-adjunction or r-adjunction.


The aim of this paper is to test LDA for establishing the semantic coherence of a document based on the premise that a real (coherent) document should discuss only a few number of topics, a property hardly granted for forged documents which are often made up of random assemblage of words or sentences.

the coherence of a document may reflect in the entropy of its posterior topic distribution or in its perplexity for the model.


To further distinguish VPCs from prepositional verbs and other related constructions we also verify the possibility of the particle to be immediately followed by an indirect prepositional complement (like in The plane took off from London), which is a good indicator/delimiter of a VPC since in non-VPC constructions like prepositional verbs the preposition needs to have an NP complement.

we will assume that a true VPC occurs in the following configurations, according to Villavicencio (2005) and Ramisch et al.


There is no semantic representation of the corresponding scenes available from CHILDES.

we automatically construct a scene representation for each utterance, as a set containing the semantic referents of the words in that utterance.


3, is set to 8500 based on the total number of distinct symbols extracted for the development data.

the default probability of a symbol for a novel word will be 1/8500.


Now consider the condition where a novel target name is used in the presence of a familiar and a previously unseen object: Since this is the first time the model has heard the word dax, both meanings ball and dax are equally likely because p(.|dax ) is uniform.

the meaning probabilities cannot be solely used for selecting the referent of dax, and the model has to perform some kind of induction on the potential referents in the scene based on what it has learned about each of them.


We aim to learn categories of words, and we do this by looking for groups of similar word usages.

rather than categorizing a word alone, we categorize a word token with its context from that usage.


By doing this, we ensure that the model only groups words of different types when there is sufficient evidence for their contextual similarity.

when a cluster has been newly created, we require that all frames put into the cluster share the same head word type.


In this case, there exists an oriented association between fruit and mango (mango → fruit) which indicates that mango attracts more fruit than fruit attracts mango.

fruit is more likely to be a more general term than mango.


by evaluating the attraction of x upon y but also the attraction of y upon x.

the maximum value will decide the direction of the general-specific association i.e.


Figure 1 clearly shows our assumption of generality of terms as the hypernym rate only has incoming edges whereas the hyponym isometry only has outgoing edges.

by applying a graph-based ranking algorithm, we aim at producing an ordered list of words from the most general (with the highest value) to the most specific (with the lowest value).


Indeed, if word w is the hypernym of word x, we may represent this relation by the following constraint y › x, where › is the order operator stating that y is more general than x.

for each set of three synsets (the hypernym synset, the seed synset and the hyponym synset), a list of constraints can be established i.e.


By evidencing the quality of the mapping between three hard clusters generated automatically and the hypernym synset, the seed synset and the hyponym synset, we are able to measure the quality of our ranking.

we propose to (1) perform 3-means clustering over the list of ranked words, (2) classify the clusters by level of generality and (3) measure the precision, recall and f-measure of each cluster sorted by level of generality with the hypernym synset, the seed synset and the hyponym synset.


However, these measures may not be as suited for relating one word to other words which may replace it (target selectors).

our similarity measure examines the use of links in WordNet, and attempts to deal with the uniformity problem by normalizing depths based on average leaf node depth.


the relationship among the starting time T S and the ending time T E is below: As a proper example, the starting point can be 60 days away from the highest point.


Thus the relationship among the starting time T S and the ending time T E is below: As a proper example, the starting point can be 60 days away from the highest point.

the Popular Trend process and the Popular Keeping process both last for nearly 3 months.


It proves that Popular Trend process is just as important as Popular Keeping process.

the best time domain of these two processes are both n / 4.


Table 5 indicates that three factors above are all important for weighing a catchword, while β is a little more important than α and γ.

maximum usage frequency of a catchword is a little more important than two other factors.


In the future, we can unite all synonymous catchwords to a word cluster and calculate the cluster's popular degree value.

we would be able to achieve a better performance for extraction.


We are interested in using pictorial communication as an assistive communication tool.

our system needs to be able to handle general text yet produce easy-to-understand pictures, which is in the middle of the spectrum.


Within each position, icons appear in the word order in the input sentence.

a tag sequence uniquely determines an ABC layout of the picture.


Figure 2 demonstrates that peak per-chunk accuracy (77.6%) and macro-averaged F1 scores are achieved using the most general sequence labeling model.

the user study in the next section is based on layouts predicted by Model 3 with σ 2 = 1.0, trained on all the data.


The most common errors are predicting B for chunks labeled O and confusing tags B and C. Manually inspecting the pictures in our training set shows that annotators often omitted the verb (such as "is" or "has") and left the B position empty, since it could be inferred by the presence of the arrow and the images in A and C. Also, annotators tended to disagree on the location of adverbial expressions, dividing them between positions B and C. Finally, only 3.3% of chunks were incorrectly omitted from the pictures.

we conclude that our CRF models are capable of predicting the ABC layouts.


Our procedure of annotating morphemes also constrains morphemes tagged as "Retained" to be aligned to English words.

if we remove "Deleted" morphemes from the morphology corpus, the reduced corpus and English corpus have the property of one-to-one mapping we prefer for source-target corpora in machine translation.


Note that phrase trees are the means to reorder the source sentence represented as phrase segmentations.

a phrase tree is surely not good if no right order can be generated.


This specification of the domain of locality determines the inductive bias of learning with ISBNs.

we need to choose the set of local (i.e.


The proposed model allows to capture interactions between syntax and semantics not only because the syntactic and semantic scores are combined but also because the semantic scoring function relies on features extracted from the head-modifier-predicate relations.

the semantic scoring function depends on the syntactic dependency being built, and, in reverse, the semantic score can modify the dependency chosen.


Otherwise picking the wrong candidate would increase overall score and no role features can re-ject this decision because the corresponding structural constraints are missing.

more weight is shifted away from false positive candidates, resulting in a higher precision of the hasRole predicate.


Our statistics show that only 7.6% sequences and less than 1% dependencies in the corpus provided for training are non-projective.

we use a simplified strategy to projectivize an input sequence.


When no constraint available, however, all word pairs in the an input sequence must be considered, leading to very poor efficiency in computation for no gain in effectiveness.

the training sample needs to be pruned properly.


Syntactic parsing contributes crucially to the overall performance of the joint parsing by providing a solid basis for further semantic parsing.

there is reason to believe that improvement of syntactic dependency parsing can be more influential than that of semantic parsing to the overall improvement.


Table 3 also shows that the unbalance between precision and recall is higher for dependencies of nouns than for dependencies of verbs, and that both recall and precision are higher for dependencies from verbs.

the system performs better for verbs than for nouns.


The goal of this function is to restrict the call of a function link(i, j) which is a nondeterministic operation that adds the arc i → j, the arc j → i, or does nothing at all.

the smaller the value of d is the fewer links can be drawn.


As mentioned above, we started the development of the system from scratch with a very small team (actually only one programmer).

we wanted to focus on certain aspects, totally abandoning our claims for achieving decent results for the others.


The statistics of the training data show that by picking the most frequent frame, the system can pick the correct frame in a large percent of the cases.

we decided to use the most frequent frame baseline for this stage.


Equivalently, we take the following to be proportional to the joint probability of a particular assignment: To estimate the parameters of the generative model we used the following methodology: For Pr(F |P ) we use the maximum likelihood estimate from the training data.

frames that were never observed in the training data have zero probability.


We were confronted with memory shortage when trying to train a model with the entire CoNLL 2008 training data with 4GB memory.

we rewrote it with C++ which can manage the memory more exactly.


However, predicates are not given in CoNLL 2008 shared task and required to be determined automatically by the system.

the first step of the shared task is to identify the verb and noun predicates in a sentence.


It also shows that the two features about predicate's sibling deteriorate the performance.

we delete these two features from remaining experiments.


Moreover, DSRL uses syntactic dependency parsing tree instead of traditional full syntactic parsing tree.

the traditional features need to be amended accordingly.


Due to the summation over the hidden variable representing the choice of simplification (not observed in the training data), our objective is not convex.

we are not guaranteed to find a global optimum; in practice we have gotten good results using the initialization of setting all weights to 0.


In the labeled training corpus, nouns with no labeled arguments are generally skipped (i.e., not labeled as predicates at all).

we made a strong simplifying assumption: if a predicate (either noun or verb) is labeled by our system as having no arguments, we should not label it as being a predicate.


Finally, we extended this heuristic to label each verb with its most common sense for the subcategorization (i.e., set of roles) actually produced by the labeling system.

if one sense was intransitive while the other was transitive, we would be able to distinguish between them (assuming that our labeling system produced the correct set of arguments).


In samples selected by our methods, the ANSK values are very close to the ANSK values of randomly selected samples.

sentences in TE samples are much more complex than in CBS and CMM samples.


We introduced the parameter based sample selection (PBS) approach and its CMM and CBS algorithms that do not deliberately select difficult sentences.

our intuition was that they should select a sample that leads to an accurate parameter estimation but does not contain a high number of complex structures.


The proportion of usable sentences is reduced to 2.11%.

the vocabulary size diminishes by a factor of 10, and the average sentence length drops considerably from almost 25 to less than 7 words, suggesting that most non-trivial examples are lost.


Specifically, every fragment root specifies as its head an artificial root token w 0 (distinguished from a true root dependency by a special relation FRAG).

sentences with a fragmented parse are still represented as a single sentence, including all words; the difference from a fully parsed sentence is that unconnected substructures are attached directly under w 0 .


5), which were adjusted for noisy training data.

improvements are likely with other settings: Nivre et al.


In preliminary experiments we had made the stop set distinct from the set of unlabeled points made available for querying and we saw performance was qualitatively the same but the AL curve was translated down by a few Fmeasure points.

we allow the points in the stop set to be selected during AL.


However, unsupervised pattern-based methods suffer from several weaknesses.

they are frequently restricted to single-word terms and are unable to discover multiword expressions in efficient and precise manner.


While symmetry-based filtering greatly reduces such noise, the basic problem remains.

incorporating at least some parsing information in a language-independent and efficient manner could be beneficial.


In order to avoid truncation of multiword expressions, we also require the meta pattern to start and end by a HFW or bracket.

our meta-patterns match the following regular expression: where "*" means zero or more times, and "+" means one or more time and B can be "(",")" brackets produced by the parser (in these patterns we do not need to guarantee that brackets match properly).


In doing so, we look to maximize the impact of interactions while minimizing the total number.

we look to select instances for which (1) the current hypothesis indicates the feature space is insufficient and (2) the resulting SRWL feature abstraction will help improve performance.


Unlike active learning, where selection is from an unlabeled data source, the quantity of labeled data is fixed and labeled data is selected during each round.

we use the true margin and not the expected margin.


We observed that basing Q margin on the relation stage performs best, which is not surprising given that this stage makes the most mistakes, benefits the most from semantic information, and also has many features which are similar to features from previous stages.

we adapt the querying function described by (Roth and Small, 2008) for the relation classification stage and define our margin for the purposes of instance selection as whereẏ = argmax y ∈Y\y f y (x), the highest scoring class which is not the true label, and Y + = Y\no relation.


Amongst these works, the only interactive learning protocol is (Raghavan and Allan, 2007) where in-stances are presented to an expert and features are labeled which are then emphasized by the learning algorithm.

in this case, although additional information is provided the feature space itself remains static.


Each eventuality is an event 2 or a state between two participants.

the rela-tion of reciprocity (e o (X, Y), e r (Z, W)) describes a situation where the eventuality e r is performed "in return" for e o .


Thus, the rela-tion of reciprocity (e o (X, Y), e r (Z, W)) describes a situation where the eventuality e r is performed "in return" for e o .

reciprocity can be seen as a special type of causal relation.


There were 1,613 distinct types of patterns which generated 1,866 distinct pattern instances.

we selected the top 15 patterns, after manual validation.


Generalizing over many similar instances, will give us better insight into how people communicate -as reactions (effects) to other people's actions (causes).

in this section we present a model for clustering the eventualities we extract through the process described in the previous sections.


For example, "help" and "give" could both be reciprocated by either "thank" or "like".

given a cluster, not only is there a reciprocal relationship between verbs in the e o group with the verbs in the e r group, but there is often a kind of similarity relationship between the verbs within each e o or e r group.


Thus, we design a bimodal mixture model in which we assume that each pair of eventualities (e o , e r ) belongs to a latent class z, and each class is associated with two distinct multinomial distributions from which the two eventualities are independently drawn.

the probability of generating a particular pair is: Each class can be thought of as a general type of reciprocity, such as an action followed by appreciation, or an attack followed by retaliation.


Since we are interested in analyzing how people interact, we would also like to identify the polarity (affective value) associated with each eventuality.

we automatically identify polarity words in both clauses.


In training, our SRL received gold standard feedback consisting of correctly labeled sentences.

when the SRL made a mistake in identifying the semantic role of any noun in a sentence, it received feedback about the 'true' semantic role of this noun.


This is because the data (language in general, child directed speech in particular here) are naturally distributed such that particular nouns that refer to animates tend to be agents, and tend to appear as first nouns, and those that refer to inanimates tend to be non-agents and second nouns.

a learner representing sentence information in terms of words only succeeds with full-distribution 'A gorps B' test sentences even with the simplest animacy feedback (Feedback 1); the A and B nouns in these test sentences reproduce the learned distribution.


This inference is implemented using a beam search that looks at arguments in a fixed order and roles from A0 up.

in the case of ties there is a preference for first seen solutions, meaning A0A1 in this case.


Most fixations last about 200-300 ms but there is considerable variability, both between and within readers.

some fixations last under 100 ms while others last over 500 ms (Rayner, 1998).


Also, tables 6 and 7 show that as a result, NR-type errors make up the majority of the errors made by the better aligners.

the data indicates that the task of slide-presentation generation is highly dependent on the end purpose the presentation will serve, as well as the target audience and other factors.


For multi-token nouns, we have features for the first and last token in the noun.

we hope to learn that Bob begins masculine nouns while inc. ends neutral ones.


The counts are collected seperately for each position by keeping track of four seperate vectors for positions -2, -1, +1 and +2.

each vector is a sparse vector, having the # of dimensions as the size of source language vocabulary.


For example, if the source spanish word to be translated was "prosperidad", then in the example shown in Figure 2, in case of adjacent context, the context word "económica" will show up in +1 position in Spanish and -1 position in English (as adjectives come before nouns in English) but in case of dependency context, the adjective will be the child of noun and hence will show up in +1 position in both languages.

we do not need to use a bag of word model as in Section 3 in order to avoid learning the explicit mapping that adjectives and nouns in Spanish and English are reversed.


We compute the Top N accuracy in the standard way as the number of Spanish test words whose Top N English translation candidates contain a lexicon translation entry out of the total number of Spanish words that can be mapped correctly using the lexicon entries.

if "crecimiento, growth" is the correct mapping based on the lexicon entries, the translation for "crecimiento" will be counted as correct if "growth" occurs in the Top N English translation candidates for "crecimiento".


For instance, Spanish adjectives often follow nouns whereas in English the the ordering is reversed.

one can either learn position mappings, that is, position +1 for adjectives in Spanish is the same as position -1 in English or just add the the word counts from different positions into one common vector as considered in the bag of words approach.


They also show that while their method lends itself to acceptable estimates of accuracy, it is much harder to estimate the recall of the classifier.

the stopping criteria based on minimal absolute or maximum possible performance are not reliable.


NER system should be robust across multiple domains, as it is expected to be applied on a diverse set of documents: historical texts, news articles, patent applications, webpages etc.

we have considered three datasets: CoNLL03 shared task data, MUC7 data and a set of Webpages we have annotated manually.


Before continuing the discussion, we note that we found that adjacent documents in the CoNLL03 and the MUC7 datasets often discuss the same entities.

we ignore document boundaries and analyze global dependencies in 200 and 1000 token windows.


(Krishnan and Manning, 2006) used the intuition that some instances of a token appear in easily-identifiable contexts.

they apply a baseline NER system, and use the resulting predictions as features in a second level of inference.


We show that PUPA outperforms the leading previous parse assessment algorithm for supervised parsers, as well as a strong unsupervised baseline.

pUpA allows obtaining high quality parses without any human involvement.


The number of constituents appearing more than t elbow times is considerably smaller than the number of constituents appearing t elbow times or less.

the fact that a constituent appears t elbow + S times (for a positive integer S) is not a better indication of its quality than the fact that it appears t elbow times.


For the least homogeneous and complete clustering solution, where knowing the cluster tells nothing about the class and vise versa, V I = H(C) + H(K).

the range of values that VI takes is dataset dependent, and the numbers themselves tell us nothing about the quality of the clustering solution (apart from a score of 0, which is given to the best possible solution).


A bound for VI values is a function of the maximum number of clusters in C or K, denoted by k * .

this is obtained when each cluster contains a single element, and the range of VI values is dataset dependent and unbounded when datasets change.


Since V is defined to be the harmonic mean of h and c, V values lie in [0, 1].

it can be used to compare the performance of clustering algorithms across datasets.


NVI is not symmetric since the term in its denominator is H(C), the entropy of the correct class assignment.


However, taking future real-time applications in mind, we did not want the lexical processing to be dependent on the parser.

we have developed an in-house part-ofspeech tagger and lemmatizer based on the material created in the Corpus Spoken Dutch project (Eynde, 2005).


However, given a BOW representation with a cosine similarity measure, this position would not improve classification performance.

we suggest to associate a mathematical function with each term, thus mapping terms and documents to the L 2 space, and using the inner product of this space to express similarity.


The teams were asked to read the texts and assign to each of them the title of the five Wikipedia articles they thought were the most relevant and the other groups would also select.

the consistency of the annotations was an important measure for this data set.


Using a machine learning system, (Medelyan and Witten, 2008) reported a consistency of 30.5%.

our result of 34.5% is significantly better, despite the fact that our method is unsupervised.


We did not take into account cross-lingual parallelism because it is one single feature whose coverage strongly relies on the parallel corpus available.

it is not possible to test the feature in isolation due to data sparseness.


Despite the provisions for the SRL-only task, we are more interested in the approaches and results of the Joint task.

primary system ranking is provided for the Joint task while additional measures are computed for various combinations of parsers and SRL methods across the tasks and challenges.


It is perhaps not surprising that most of the observations from the 2008 shared task still hold; namely, the best systems overall do not use joint learning or optimization (the best such system was placed third in the Joint task, and there were only four systems where the learning methodology can be considered "joint").

most of the observations and conclusions from 2008 shared task hold as well for the current results.


Although, when parsing with gold standard lemma and POS features, our open system outperforms the closed system on out-domain tests (for English), when parsing with P-columns there is no substantial improvement observed after using the HPSG features.

we did not include it in the final submission.


However, it is still inefficient for a singlestage argument identification/classification classification task.

we introduce an assistant argument label ' NoMoreArgument' to alleviate this difficulty.


The CoNLL Shared Task 2009 is to parse syntactic and semantic dependencies of seven languages.

training and development data in form of annotated corpora for Catalan, Chinese, Czech, English, German, Japanese and Spanish is provided, cf.


The performance with this filtering became a little worse.

we decided not to use it generally, but one exception is for the Czech data.


For example, if a sentence with two predicates has 7 words, we will derive 7*2=14 training examples.

the number of training examples generated in this step will be around L times larger than that obtained in the previous step, where L is the average length of sentences.


The predicate sense classification is performed as the last step.

no features representing the predicate sense are employed during the training.


The score functions score ch ,score cmi and score cmo are the linear functions that build up the previously defined second-order global score, e.g., The two tables C and O maintain the dynamic programming structures.

note that the first steps of the inner loop are applied for all l, the syntactic label, but the semantic score function does not depend on l. the best semantic label can be chosen independently.


The second interesting property is that all semantic dependencies exactly overlap with the syntactic tree.

the semantic dependency between a predicate and an argument always has a matching syntactic dependency between a head and a modifier.


Before participating to this Shared Task, our prototype had only been used to annotate function tags in a French Treebank: these data were drastically smaller, and the task was simpler.

coNLL 2009 ST is the first time the crotal System is run for a quite complex task, with so many data as input, and seven different languages (catalan, Spanish (Taulé et al., 2008), chinese (Palmer and Xue, 2009), czech (Hajič et al., 2006), English (Surdeanu et al., 2008), German (Burchardt et al., 2006) and Japanese (Kawahara et al., 2002)).


Moreover, the size of the training set for all seven languages allows such a clustering, and it would even be difficult to process the whole set of predicates due to time and memory limitations.


It is worth noticing that performance would be better using the "real" columns, but we have followed the instructions given by the organizers.

3 sentences with no predicate are skipped and several trees possibly correspond to the same sentence.


In the CoNLL-2009 shared task, the predicate identification is not required, since the predicates are given for us.

the system we present is only composed of three components: a syntactic dependency parser, a predicate classifier and a semantic parser.


However, growth of the number of dependency labels by pseudo-projective transformation increases the dependency parser training time, so we did not adopt transformations.

the parser ignores the presence of non-projective edges in the training and the testing phases.


Nevertheless, a symmetric construction containing two heads seems to be more appropriate for some languages (e. g. German, see (Lang, 2002)).

the presented algorithm is able to deal with phrases containing any number of compounds.


However, in the task of semantic parsing, this decision relies on identifying a hidden intermediate representation (or an alignment) that captures the way in which fragments of the text correspond to the meaning representation.

we formulate the prediction function as follows: Where Φ is a feature function that describes the relationships between an input sentence x, alignment y and meaning representation z. w is the weight vector which contains the parameters of the model.


DIRECT uses the signal directly to learn a binary classifier capable of replicating the feedback, whereas AGGRESSIVE learns a structured predictor that can repeatedly obtain the logical forms for which positive feedback was received.

although the AGGRESSIVE outperforms the DI-RECT approach the concepts each approach learns may be different.


Regular grammars are not learnable, but deterministic finite automata are learnable under various paradigms (Angluin, 1987).

it is possible to learn by changing to representations that have better properties: in particular DFAs are learnable because they are "objective"; there is a correspondence between the structure of the language, (the residual languages) and the representational primitives of the formalism (the states) which is expressed by the Myhill-Nerode theorem.


For any S ⊆ K, S = S and for any C ⊆ F , C = C .

we can form a concept from any set of strings S ⊆ K by taking S , S ; this is a concept as S = S .


However, note that the first part is defined by the two contexts (λ, λ), (a, b) and the second by the two contexts (λ, λ), (a, bb).

it is sufficient to have a set F that includes these four contexts, as well as similar pairs for the other non-terminals in the grammar, and some contexts to define a and b.


We can observe that we have obtained more balanced values of recall and precision, as well as a better value for the F-measure.

the method proposed in this work, that we expect to refine by assigning different probabilities to separators and sub-separators, depending on the context they appear in, provides a very promising approach.


Crucially, the weakened models will make predictions that are roughly similar to the full models, but systematically worse.

model scores from the weakened models provide enough power for the bilingual view to make accu-  Table 1: Sample features used for named entity recognition for the ORG entity in Figure 1. rate predictions, but ensure that bilingual features will be required to optimize the training objective.


The quality of the tagging induced by such algorithms is thus highly variable, and researchers report average results over several random initializations.

applications are not guaranteed to use an induced tagging of the quality reported for the algorithm.


However, since the boundaries of the spans annotated in the MPQA corpus are not strictly defined in the annotation guidelines , measuring precision and recall using exact boundary scoring will result in figures that are too low to be indicative of the usefulness of the system.

most work using this corpus instead use overlap-based precision and recall measures, where a span is counted as correctly detected if it overlaps with a span in the gold standard (Choi et al., 2006;Breck et al., 2007).


However, the similarity between subjective words, which have multiple senses against other words may negatively impact the system accuracy.

the use of the syntactic/semantic kernels, i.e.


For example, in the utterance P.EH1.N S.AH0.L (pencil), if the learner has P.EH1.N and P.EH1.N S.AH0.L in its lexicon but P.EH1.N is more frequent, it may fail to recall P.EH1.N S.AH0.L when examining the second syllable but succeed in recognizing P.EH1.N in the first.

it will break off P.EH1.N instead of P.EH1.N S.AH0.L.


For each feature vector v in the test set, we compute the Euclidean distance to each of the matching vectors in the extended training set, where matching vectors share at least one pattern feature with v. Let t i , i = 1..k be the k vectors with lowest Euclidean distance to v 6 .

then v is classified with a label l as follows: j Count(label(t j )) the score is a weighted average of the k closest training set vectors.


that can be learned from manual translations of a selected batch.

our active learning strategy attempts, at each iteration, to select a batch of mutually diverse source sentences, which, while introducing new concepts, shares at least some commonality with the target domain.


A logistic regression model is used to directly estimate the posterior probability of the binary word label.

our confidence score is essentially the probability of the word being "incorrect".


Intuitively speaking, if this conditional independence assumption holds, the most confident instance of one classifier will act as a random instance for the other classifier.

it can be safely used to expand the training set of the other classifier.


That makes a large set of millions of instances.

we develop a modified version of Tri-training algorithm using sampling techniques, which can work well with such large scale data.


However, these different errors may have different effects on the machine translation quality.

improving alignment quality according to AER or F 1 may not directly lead to an increase of BLEU scores.


While this is a reasonable design choice, one may argue that the reverse is also true that sentiments may vary according to topics.

in this paper, we studied the reverse dependence of the JST model called Reverse-JST, in which sentiments are generated dependent on topic distributions in the modelling process.


Most recent approaches do not only deal with the 2-classes classification problem, but also introduce a new class representing neutrality.

the aim of these works is to classify the text into positive, negative or neutral.