Hence, we decided to generate two different models: the author level model, in which the factuality of events is assessed relative to the author of the text (i.e., at the level of source s 0 ), and the top source level model, in which event factuality is assigned according to the source with a higher level of nesting in the set of relevant sources for that event (e.g., s m s j s 0 ).

features 16-19 were added to convey information on the top-level sources as well.


The high error rate in this area seemed to suggest that the approach assumed in De Facto for these constructions (following Geurts [1998] andGlanzberg [2003]; see Section 3.4.4) was not completely adequate.

we experimented running De Facto without the part of the algorithm dealing with them (Algorithm 2, lines 1-9).


The Uu category is used much more often in FactBank than by Turkers, and the dominant alternative choice for the Turkers was CT+.

the low score for Uu also effectively drops the score for CT+.


Of the remaining 45 non-parenthetical examples, 42 (93%) of those are tagged CT+ or CT−.

both parenthetical and non-parenthetical verbs are about equally likely to lead to a CT tag.


Combining all of the rows where two categories received at least 3 votes, we get 162 examples, which is 25% of the total data set.

a non-negligible subset of our sentences seem to involve examples where readers' responses are divided, suggesting that there is no unique correct label for them.


In order to align with our theoretical conception of the problem as probabilistic, we treat each annotator judgment as a training item.

each sentence appears 10 times in our training data.


The exception to that is conditional antecedents.

we do well with cases like Example (23a), where the event is in a conditional; the classifier assigns 70% of the probability to Uu and 0.15 to PS+.


The original task definition draws on "common-sense" understanding of language (Chapman 2005), and focuses on how people interpret utterances naturalistically.

these entailments are not calculated over just the information contained in the sentence pairs, as a more classical logical approach would have it, but rather over the full utterance meaning.


Thus, these entailments are not calculated over just the information contained in the sentence pairs, as a more classical logical approach would have it, but rather over the full utterance meaning.

they are imbued with all the uncertainty of utterance meanings (Zaenen, Karttunen, and Crouch 2005;Crouch, Karttunen, and Zaenen 2006;Manning 2006).


Instead, we want to be able to draw on information from the more frequently occurring cues also when classifying or disambiguating the less frequent ones.

we will still train a single global classifier.


In a strictly rule-based system, these features would act as hard constraints and yield superior results in these cases.

these instances seem a prime source of inspiration for further improvements to the ranker in future work.


Although both the constituent ranker and dependency rules perform well in isolation, they do not necessarily perform well on the same test items.

we investigated the effects of combining their predictions.


2007), a state-ofthe-art phrase-based MT system that by default does not incorporate any linguistic information (e.g., syntax or morphology or transliteration knowledge).

words that were not directly observed in the bilingual training data were untranslatable.


For example, if one managed to do something, one tried to do it and one probably wanted to do it.

annotators were provided a specificity-ordered modality list as in Figure 5, and were asked to choose the first applicable modality.


The distinction between "Negation" and "NOT" corresponds to the difference between negation that is inherently expressed in the triggering lexical item and negation that is expressed explicitly as a separate lexical item.

i achieved my goal is tagged "Succeed" and i did not achieve my goal is tagged as "NOTSucceed," Example translation rules with tags for modality, negation, and entities combined with syntactic categories.


He had been impressed by the work of Bloomfield and Fries, and had become convinced that linguistics could provide stable and repeatable methods akin to the procedures of cosmic ray physics.

over the following years, he appointed a number of linguists to the RLE staff, starting with Noam Chomsky in 1955, in order to undertake basic research.


The aim was to set up mutually exclusive word-classes (i.e., one class for the noun function, another for the verb function, and so on) The approach was to set up substitution frames to isolate contexts.

walk may occur in the frame 1The was enjoyable or in the frame (2) They home every day.


These words form a word-class and their frames form context classes.

each sentence sequence of word-classes would determine a unique sequence of context classes.


Attention, therefore, focused on the transition stage where the meaning or message of the input language would be expressed and where choices would be made for producing output sentence structure.

the MIT group saw the need for a three-stage model: syntactic analysis, structure transfer, and synthesis-a model for many transfer-based MT systems in subsequent years.


They decided that what was needed was a programming system that accepted notations of a kind familiar to linguists, rather than systems such as FORTRAN and COBOL designed for mathematicians and business applications.

in collaboration with the MIT Computation Center, they developed the first programming language designed specifically for string-handling and pattern-matching, COMIT (Yngve 1961a(Yngve , 1967.


Also, B is the constant from Section 3.1.

we have: From the construction of our proper approximations (Proposition 3), we know that only derivations of length log 2 n or greater can be in Z ,n .


Therefore, we have: From the construction of our proper approximations (Proposition 3), we know that only derivations of length log 2 n or greater can be in Z ,n .

0 using the same proof technique we used here, while relying on the fact that C n ( f * n ) ∈ F n , and therefore C n ( f * n )(z) ≤ sN|z| log n. Equipped with the framework of proper approximations as described previously, we now give our main sample complexity results for probabilistic grammars.


This inequality indeed holds whenever γ < 1 20m .


In addition, we have α 1 + α 2 − 2 ≥ 0 for small enough (can be shown by taking the derivative, with respect to of α 1 + α 2 − 2, which is always positive for small enough , and in addition, noticing that the value of α 1 + α 2 − 2 is 0 when = 0.)


From the requirement of p, we know that for any x we have a z such that yield(z) = x and |z| ≤ α|x|.

if we let X(m) = {x | |x| ≤ log 2 m/α}, then we have for any In addition, from the requirements on p and the definition of K m we have Table D.1 gives a table of notation for symbols used throughout this article.


It was observed that in 26% of the instances, the text conveyed only a small portion of the graphic's message and in 35% of the instances, the text didn't capture the graphic's message at all.

graphics, together with the textual segments, contribute to the overall purpose of a document (Grosz and Sidner 1986) and cannot be ignored.


The system adopts the AutoBrief (Kerpedjiev and Roth 2000) hypothesis that the graphic designer chooses the best design to facilitate the perceptual and cognitive tasks that a viewer will need to perform on the graphic.

the relative effort for different perceptual tasks serves as a communicative signal about what message the graphic designer intended to convey (Elzer et al.


We had to ensure that the propositions presented to the participants (PROPALL) actually covered all information that is important enough to include in the summary of a bar chart.

for each graphic, we also asked participants if there was anything else they felt should be included in the brief summary of the graphic.


2001]), however, a structure such as RST is not helpful here.

the top-down planning approach does not appear to fit.


We also hypothesize that it is appropriate to close the textual summary by bringing the whole graphic back into the user's focus of attention (Grosz and Sidner 1986) (via the propositions in the computational class).

we define an ordering of the proposition classes (creating a partial ordering over the propositions) and present first the message-related propositions, then the specific propositions, and finally the computational propositions.


That particular entity forms the main entity of the Which predicate.

tree A will be an immediate child of the node with the Which predicate and the node with the Which predicate will be an immediate child of node N in tree B.


For instance, nm 1 (sentence(A)) is the normalized score that the forest A receives with respect to the sentence criteria and is equal to sentence(A)/max(sentence(all forests)).

the normalized score that a forest receives for each criteria is always between 0 and 1 and therefore each criteria has an equal impact on the overall score of a forest.


For all message categories, the number of sentences in a proposition class would be limited (less than five) even if all of the highly rated propositions identified for that message category are selected for inclusion.

a straightforward "generate and test" strategy is appropriate for ordering sentences in our case.


Here, however, we treat this particular aspect as a novel text-to-text generation methodology which is combined with other data-to-text approaches in a complete NLG system.

our focus in this section is to highlight a new way of using the text associated with images which has been earlier exploited by various NLP tasks such as indexing and retrieval of images (Pastra, Saggion, and Wilks 2003).


For example, WordNet could not identify Jacques Chirac, a former president of France, whereas OpenCyc ontology contains this information.

we use OpenCyc ontology version v0.7.8b to identify the ontological categories of bar labels in our work.


If our system were to enumerate all entities involved in a comparison message, the realization of the inferred message might be lengthy and the enumeration of little utility to the user.

we set a cut-off C, such that if the number of entities involved in a Maximum Bar or Rank Bar message exceeds C, they are not enumerated but rather we use the generated referring expression for all bars.


The majority of information graphics from popular media are intended to convey a message that is often not captured by the text of the document.

graphics, along with the textual segments, contribute to the overall purpose of a multimodal document and cannot be ignored.


Our event-related class includes not only past and present participles, but other types of deverbal adjectives.

most of the classes used in this article are to some extent backed up by the organization of adjectives in WordNet.


Conversely, most adjectives in the hybrid clusters are problematic.

hybrid clusters are useful to signal problems in the proposed classification.


Conversely, militar ('military') is mostly used as a relational adjective, and is consistently assigned to one of the relational clusters in all solutions.

although polysemous adjectives on average do show a mixed behavior, each lexical item tends to pattern with one of the basic classes.


The lemmata were randomly sampled in a stratified fashion, balancing three factors of variability: frequency, morphological type, and suffix.

the gold standard contains an equal number of adjectives from three frequency bands (low, medium, high), from the four derivational types, and from a series of suffixes within each type.


9 Note that, for each adjective, only 10 different full classification proposals are obtained in each feature set, because each adjective is only used once per run for testing.

while the per-class accuracy for each feature set is assessed from 100 estimates (obtained via 10x10 cv), the accuracy of the different feature sets for full classification is assessed comparing 10 accuracies.


In fact, the class distribution varies with the suffix (see Table 13): Some types, such as -or and the participle, show a clear predominance of the event class (see dark-gray shaded cells); other types, such as -ble, -iu, or -nt, are more spread in their distribution (see light-gray shaded cells).

the suffix seems to influence the resulting readings, with some active-like suffixes building a much more transparent relation to the event (creador 'creating', exportador 'exporting', recomanat 'recommended'), and some passive-like or stative suffixes being more prone to creating a stative meaning (contingent 'contingent', formidable 'formidable | terrific', significatiu 'significant').


The procedure treats the decision on each of the basic classes as if they were all independent.

the probability for an adjective being polysemous amounts to the product of the probabilities of the adjective belonging to each of the basic classes, as expressed in Equation (4) for two arbitrary given classes, A and B. shows that the distribution of polysemous items predicted by Equation (4) is more similar to the distribution obtained with the best machine learning classifier Table 14 Distribution of polysemous items and absolute numbers, according to the prediction (Equation (4); first column), in the machine learning (ML) results shown in Table 12 (second column), and in the gold standard (GS; third column).


Finally, the classifier produces more ER adjectives than observed and also than predicted by Equation (4), but in this case the numbers are so small that no clear tendencies can be observed.

the procedure followed can be said to cause the overgeneration of items for the QE polysemy type, but it does not account for the other two polysemous classes.


By their very nature, polysemous words will show only some evidence for each of the classes, as their occurrences (and thus their properties) will be distributed across the two classes.

they will be untypical members of at least one of the intervening classes.


In contrast, all lexical nodes in the tree t 1 have depth 3.


To save some space, in the following we write right spinal trees horizontally and from left to right, as already done in Figure 3.

the grammar G 2 can alternatively be written as follows: For a node u in a right spinal tree derived by G 2 , we define We exploit this function to compute the excess of left parentheses over right parentheses in a sequence of nodes, and write: Let t be some right spinal tree in T(G 2 ), and let v be some node in t. Assume that u 1 , .


Finally, the inequality in Equation 17is strict unlessp ∈ M(p).


), it also holds that, in the LCFRS derivation tree, each node has as its children rules which expand the set of nonterminals appearing in the parent's r.h.s.

the LCFRS tree constitutes a valid derivation.


Our LCFRS combination functions always concatenate the first elements of the input tuple in order, adding any terminals present in the portion of the input tree matched by the MBOT's l.h.s.

the combination functions maintain the property that the first element in the resulting tuple, t 0 , contains the yield of the subtree of the input tree where the corresponding MBOT rule applied.


From the point of view of string yield, both of the transformations are equivalent to an STSG rule that simply copies the three variables with no re-ordering.

these counterexamples are not sufficient to show that the yield of the composition of two STSGs is not the yield of an STSG.


If we write the jth rule in this sequence as A j → ν j , there must exist a synchronous rule in G of the form A j → ν j , ν j that rewrites the same nonterminal.

g licenses a synchronous derivation (A 1 , A 1 ) ⇒ * (βA 1 δ, β A 1 δ ) for some β and δ .


Similarly, the case where the kth aligned run lies between β and δ in the target leads to a contradiction.

the assumption that j = i must be false.


r There is no procedural way to learn how language is structured; the linguist's job is to figure out what rules reside in the minds of speakers.


When working on meaning, linguists often used prefix notation, allowing the ordered list of symbols following the name of the predicate to stand for the "-arity"the number of arguments-of the particular predicate.

p(a) could represent an adjective like hungry or a verb like vanish; p(a,b), relating two things to each other, could stand for an adjective like different or a verb like love; and p(a,b,c) with three arguments could stand for an adjective like intermediate or a verb like give, show, or tell.


For example, in Figure 2, the cell associated with the two-word substring real-estate market can be closed to both complete and incomplete edges, since real-estate ∈ B, and any complete edge built from entries in that cell would also have to start with the same word and hence would be discarded.


The quadratic bound does not include any potential reduction of work for the remaining open cells, however.

the amount of work to parse the sentence will be less than the worst-case quadratic bound because of this reduction in processing.


8 This leads to the conclusion that when optimizing parameters for high-precision constraints, we can better tune the overall pipeline by choosing an operating point based on the corpus-level tagger performance as opposed to tuning for sentence-specific precision goals.

other than Table 4, we only apply GHP constraints on test set results in the remainder of this section.


It is not until we parse longer sentences that the trends start to differ and exhibit characteristics of the complexity bounds.

by combining high- precision and complexity constraints, we attain the typical-case efficiency benefits of high-precision constraints with worst-case complexity bounds.


With the cross-product approach, fewer active entries in either child cell leads to fewer grammar access operations.

pruning constituents in smallerspan cells directly affects the overall efficiency of parsing.


On the other hand, with the grammar loop method there is a constant number of grammar access operations (i.e., the number of grammar productions) and the number of active entries in each child cell has little impact on efficiency.

with the grammar loop implementation of the CYK algorithm, pruning techniques such as unary constraints will have very little impact on the final run-time efficiency of the parser unless the list of grammar productions is modified given the constraints.


The bracketed text in Example (17) is a placeholder for the verb phrase kicked the ball in Example (16), which has been elided (i.e., left out).

in Example 17, Bill can be thought of as an implicit argument to some kicking event that is not mentioned.


We extended this PMI score to semantic arguments instead of syntactic dependencies.

the value for this feature is computed as follows: We computed Equation (2) by applying verbal SRL (Punyakanok, Roth, and Yih 2008), nominal SRL (Gerber, Chai, and Meyers 2009), and coreference identification (OpenNLP) to the Gigaword corpus (Graff 2003); because these tools are not fast enough to process all 1,000,000 documents in the corpus, however, we selected subsets of the corpus for each p 1 /p 2 combination observed in our implicit argument data.


In order to penalize low-frequency observations with artificially high scores, we used the simple discounting method described by Pantel and Ravichandran (2004) resulting in the following modification of Equation 3: Thus, if two argument positions are rarely observed as coreferent, the value x x+1 will be small, reducing the PMI score.

the denominator in Equation 2is computed with a similar discount factor: if either of the argument positions is rarely observed as coreferent with other argument positions, the value min(x 1 ,x 2 ) min(x 1 ,x 2 )+1 will be small, making the denominator of Equation (2) large, reducing the PMI score.


In our 2010 work, we showed that removing discourse relations from the model did not have a statistically significant effect on performance.

this information should be removed in practical applications of the model, at least until better uses for it can be identified.


Our previous study used fixed partitions of training, development, and testing data.

feature and model parameter selections overfit the development data; we observed a 23-point difference in F 1 between the development (65%) and testing (42%) partitions.


The ADN-Classifier is a tool that takes into account different levels of linguistic information depending on its availability, such as senses, lemmas, or syntactic and semantic information coded in the verbal and nominal lexicons (AnCora-Verb [Aparicio, Taulé, and Martí 2008] and AnCora-Nom [Peris and Taulé 2011]) or in the AnCora-ES corpus.

this article contributes to the semantic analysis of texts focusing on Spanish deverbal nominalizations, although the proposal presented could be extended to other Romance languages.


To analyze this criterion, we observed those nominalized NPs in which the internal argument was explicit and the type of argument that realized it.

we observed that the majority of event nominalizations are complemented by an internal argument (98%).


The percentage decreases considerably in result nominalizations (34%), however.

the data seem to confirm Picallo's and Grimshaw's hypothesis.


The annotators could use these two tests to decide the denotation type.

they had two extra criteria that the data did not provide.


Furthermore, it is worth noting that when the unit of learning and classification used are the examples from the AnCora-ES corpus and not the senses from the lexicon, the baseline also increases.

it seems that proportionally result nominalizations are more highly represented in the corpus than event and underspecified nominalizations.


This is probably due to the fact that most nominal lexicalized nominalizations are of the result type.

the key failure of the ADN-Classifier is basically in distinguishing between the different types of lexicalized constructions.


In each criterion we find features for supporting result nominalizations but not event nominalizations.

the ADN-Classifier uses more features for detecting result than event nominalizations, and therefore achieves a greater degree of accuracy on the former than in the latter.


For each rule justifying a local mother-daughters configuration, all occurrences of the ↑ symbol (called a metavariable) in the functional annotations of the daughters are replaced by the mother node, and for each of the daughter categories, all occurrences of the ↓ metavariable in its annotations are replaced by the corresponding daughter node.

3 the ↓ of the annotations on a daughter category of a rule and the ↑ of the annotations of the rule that further expands that category are always instantiated with the same node.


By substituting all mother-definable node constants by their defining terms we can then produce from the original f-description the equivalent description in (16).

notice that for each acyclic f-structure F the maximal length of the σ in the defining terms is bounded by the depth of F. we see that a mother-definable constant can be eliminated in favor of a constant corresponding to a higher node and a sequence of attributes leading down through the f-structure.


Given LFG's instantiation procedure, as formalized in Definition 2, two distinct node constants can be related in a single equation only if the nodes stand in a motherdaughter relationship.

a daughter and a node external to the mother cannot be related directly by instantiation but only as a consequence of a deduction involving at least one instantiated annotation of some other licensing rule.


The constant/complex clash condition (vii) of Definition 4 prevents atomic values from being substituted for proper subterms and the constant/constant clash condition (vi) prevents them from being equated to nodes.

such a deduction can only involve equations relating a daughter to its mother (or a node to itself).


Because F is isomorphic to M|Σ for any minimal model M of FD, and M|Σ and M share the same universe, we can use F's (finite) universe to define the constants that we require.

for each element a of the universe of F that is not denoted by an atomic value we introduce a constant a a .


This is why the term set must contain the entire set of constants C F (and the terms containing them) and not just the ones for the f-structure roots.

t F contains sufficiently many constant symbols and defining terms for the reducing substitutions to make all the distinctions that could arise from any c-structure and f-description for the given F. It is defined formally in the following way.


So far we have considered only the individual instantiated rules that we obtain from the licensing rules of a derivation for F by replacing the node constants as described by terms of T F .

of Corollary 1, we also observe that our reducing substitutions never replace undefinable daughters of two distinct node-instantiated licensing rules by one and the same constant.


We first define for each node n of c the set N n consisting of all nodes higher than n, all nodes of the same depth as n but preceding (on the left), and n. Now for each N n with |N n | = i let the function ψ i be the restriction of ψ to N n (ψ|N n ).

then we can show by induction for each the equivalence is established by constructing a minimal model M i also on the universe U of M. the isomorphism between M|Σ and M i |Σ is the identity function.


Thus ψ is a reducing substitution for c and ρ. Lemma 3 ensures that there exists a reducing substitution for every derivation of an acyclic f-structure F in G. By Lemma 2 we know that such a substitution preserves equivalence.

14 the collections of instantiated rules that result from the derivations for F and their reducing substitutions must belong to the set consisting of all possible collections of appropriately instantiated and pair-wise compatible rules that together provide descriptions of F. If we extend the Inst function in the obvious way to sets of instantiated rules IR then this set is defined as follows.


The component IR contains all instantiated rules of G that are required to license the subderivation in G that corresponds (under the Cat projection) to the subderivation from n in G F , except that the licensed nodes are replaced in the instantiated rules by their ψ values.

17 the additional components of the root label S:root:IR root record 15 The set of terminals T F is constructed from the full term set T F instead of just ⊥ to allow for the possibility of ↓ appearing in lexical entries (e.g., Zaenen and Kaplan 1995).


18 Now, ψ must be a reducing substitution for the simulated derivation, because all instantiated rules in IR root are appropriately instantiated and pair-wise compatible and because condition (iic) of Definition 16 ensures that rules that are not self-compatible can only be used once for licensing the Cat projection.

because of Lemma 2, the derivation in G F simulates a derivation of an f-description in G that ψ reduces to the equivalent description provided by IR root .


By induction on the depth of the subtrees we first define for each nonterminal n a function ρ n from all nonterminal nodes dominated by n into R such that (a) ρ n licenses the subtree of c with root n, Suppose n with dts(n) = n 1 ..n m is expanded by ρ n = A:t:IR → X 1 :t 1 :IR 1 ..X m :t m :IR m in c , then there is a rule r ∈ R satisfying the conditions of Definition 16(ii).

r ex- IR j , and (r, (n, dts(n))[ψ]) = (r, (t, t 1 ..t m )) by definition of ψ.


These are instantiated by replacing the instantiating nodes of the LFG derivation by their ψ values.

the instantiated description provided by the rule component of the start rule is equivalent to the original f-description and hence the context-free derivation tree at the bottom of Figure 6 is licensed completely by the rules of the constructed grammar.


We observed in conjunction with Lemma 3 that the f-description of every derivation for F can be reduced to an equivalent description that is satisfied in the canonical modelF expanded by some interpretation of root.

the generation grammar can be constructed by considering only the set IRDF containing those elements of IRD F whose instantiated descriptions are modeled by some root expansion ofF.


If an input f-structure F is not complete and coherent, the LFG derivation relation ∆ G does not associate it with any strings, and the set Gen G (F) is empty.

when we determine by inspection that an input f-structure fails to satisfy these conditions, we maintain the context-free result by assigning it a trivial grammar that generates the empty context-free language.


Recall that the constructions that we used in the proof of our main theorem yield in both proof directions FD[ψ] = Inst(IR root ).

the constraining statements in Inst(IR root ) are exactly the ones that result from those in FD by substitution with ψ.


We now see more clearly why this is the case: If there is a context-free derivation for a given string that involves a nonbranching dominance cycle, we know that there is another derivation for that same string that has no such cycle.

the generated language is the same whether or not derivations with nonbranching dominance cycles are allowed.


It differs by postponing the attachment of right-dependents until the complete subtree under the dependent itself has been built.

the dependency tree is built strictly bottom-up, which means that attachment to a dummy root node will always happen at the end, regardless of whether the dummy root node is positioned at the beginning or at the end of the sentence.


Nevertheless, it is worth noting that, for individual languages, differences in scores can be quite substantial.

for Dutch, the First condition outperforms the None/Last condition by 0.80/0.78 in UAS and 0.40/0.42 in LAS.


The first is that, for certain parsing models, the existence and placement of the dummy root node is in fact a parameter worth tuning for best performance.

for the deterministic arc-eager parser, it seems that we can obtain higher parsing accuracy by placing the dummy root node at the end of the sentence (or omitting it completely) instead of placing it at the beginning in the sentence, as is currently the norm in data-driven dependency parsing.


The effect should be strongest in the GOLD-M models because the morphological information is correctly disambiguated.

we expect the same results for the PRED-M models blurred by additional errors introduced by an imperfect morphological prediction.


This nicely confirms the correlation between RM and configurationality.

dutch is positioned between English and German on the scale.


At the same time, Modern Standard Arabic speakers use SVO order much more frequently and consistently than is the case in Classical Arabic.

classical Arabic is roughly at the same position as German on the scale whereas spoken Modern Standard Arabic may be more comparable to Dutch.


It seems that the additional information supplied by the agreement features is not useful enough to outweigh sparse data problems caused by the more fine-grained label set.

we only use case, but designing a smoothing procedure allowing us to use number, gender, and degree is interesting future work.


Adjuncts often differ with respect to their preferred attachment sites.

we annotate PPs and adverbials (AVP, ADV, ADJD) with one of the features N, V, or 0 which indicate a nominal parent (NP or PP), a verbal parent (VP, S), or anything else, respectively.


Years such as 1998 can appear in places where other numbers cannot.

pOS tags of numbers between 1900 and 2019 are marked with year.


VPs that are headed by finite verbs, infinitives, past participles, imperatives, and zu infinitives are all used in different contexts.

we mark object VPs (VP-OC) with a corresponding feature.


When grammatical functions are included in the evaluation, the performance improvement more than doubles on Europarl data.

we decided to use the finite-state morphology in the rest of the experiments we conducted.


Following Charniak and Johnson (2005) we sum up these local feature values in the parse tree.

our versions count the number of times that a particular pattern occurs in the entire parse tree.


German, for instance, has a rich inflectional system and allows for a free word order, as we have already seen in Example (1): Arguments can be scrambled, and topicalizations and extrapositions underlie few restrictions.

discontinuous constituents occur frequently.


In order to extract a PCFG from a treebank, the trees need to be interpretable as CFG derivations.

most work has excluded non-local dependencies; either (in PTB-like treebanks) by discarding labeling conventions such as the co-indexation of the trace nodes in the PTB, or (in NeGra/TIGER-like treebanks) by applying tree transformations, which resolve the crossing branches (e.g., Kübler 2005;Boyd 2007).


The last rule tells us that, given a pair in the yield of A, we can obtain an element in the yield of S by concatenating the two components.

the language generated by this grammar is The terms of grammar fan-out and rank and the properties of monotonicity and ε-freeness will be referred to later and are therefore introduced in the following definition.


All new items that we deduce involve at least one of the agenda items as an antecedent item.

whenever an item is the best in the agenda, we can be sure that we will never find an item with a better (i.e., higher) weight.


Therefore, whenever an item is the best in the agenda, we can be sure that we will never find an item with a better (i.e., higher) weight.

we can safely store this item in the chart and, if it is a goal item, we have found the best parse.


Here, we do not need to consider the number of terminals outside the span of C (to the left or right or in the gaps), because they are not relevant for the inside score.


For this, we need a different length vector (without the lengths of the parts in between the components).

for a given range vector ρ = l 1 , r 1 , .


The other two estimates, which are both not monotonic, act as FOMs in a best-first parsing context.

they contribute to speeding up parsing but they decrease the quality of the parsing output.


The VP is discontinuous with two components that wrap around the yield of the VMFIN.

the LCFRS rule is S(XYZ) → VP(X, Z) VMFIN(Y).


As previously mentioned, in contrast to CFG the order of the right-hand side elements of a rule does not matter for the result of an LCFRS derivation.

we can reorder the right-hand side of a rule before binarizing it.


The optimal binarization produces the best results.

we will use it in all further experiments, in spite of its higher parsing time.


To further complicate matters, the definite article ה ('the') is not realized in writing when following the particles ב ('in'), כ ('like'), and ל ('to').

the form בבית can be interpreted as either ב-בית ('in house') or ב-ה-בית ('in the house').


The filter approach differs in that it poses hard constraints and not soft ones, pruning away parts of the search space entirely.

the use of k-best list is merely a technical detail in our implementation-the agreement information is easily decomposable and the hard constraints can be efficiently incorporated into the CKY search procedure.


In PATB-tokenized MSA, the corresponding type of tag set (core POS extended with a complete description of morphology) would contain upwards of 2,000 tags, many of which are extremely rare (in our training corpus of about 300,000 words, we encounter only 430 POS tags with complete morphology).

researchers have proposed tag sets for MSA whose size is similar to that of the English PTB tag set, as this has proven to be a useful size computationally.


Put differently, we are interested in the tradeoff between relevance and accuracy.

we repeated the experiments with POS tags predicted by the MADA toolkit (Habash and Rambow 2005;Habash, Rambow, and Roth 2012) 15 (see Table 2, columns 5-7).


It is likely, however, that from a machine-learning perspective, representing similar categories with the same tag, or  taking into account further-away tokens in the sentence, may be useful for learning.

we next experimented with modifying some inflectional features that proved most useful in predicted input.


More interestingly, when evaluated on predicted feature values, training on predicted feature values yields better parsing scores than when training on gold, and training on g+p yields best scores, in support of our hypothesis.

in the rest of the table (and in the rest of the experiments), we apply the g+p training variant to the best models so far, both in MaltParser and Easy-First Parser.


The error reduction relative to the gold topline is 62% and 76% for nominal agreement and verb agreement, respectively.

we see that our second hypothesis-that the use of morphological features will reduce grammaticality errors in the resulting parse trees with respect to agreement phenomena-is borne out.


They don't show surface agreement with a noun, even though the corresponding number-word token would.

we replaced the digit tokens' NUMBER value with "N," and denoted these experiments with NUMDGT.


Unless horizontal markovization is applied, PCFGs generate words independently.

no phrasal rule parameter in the model differentiates between Examples (10a) and (10b).


TSGs can store lexicalized tree fragments as rules.

if we have seen [ MWV kicked the bucket] several times before, we can store that whole lexicalized fragment in the grammar.


The morphological analyses were obtained with significantly different tools: in Arabic, we had a morphological generator/ranker (MADA), whereas for French we had only a discriminative classifier (Morfette).

french analysis quality was lower (Section 5.3).


The Arabic MWE list consists mainly of named entities and nominal compounds, hence the high concentration of MWN types in the pre-processed ATB (Table 10).

this particular Arabic MWE identification experiment is similar to joint parsing and named entity recognition (NER) (Finkel and Manning 2009).


By examining the NEs initially recognized in aligned sentence pairs, we have the following two observations: (1) Alignment can help fix those NEs that are initially incorrectly recognized when they are not the correct counterparts of each other.

alignment and recognition should be jointly optimized.


(2) Alignment cannot help in determining the appropriate scope when each word within an NE (or within its larger context window covering the NE) is correctly matched to its counterpart.

the information of those initial NEs should be utilized to decide the appropriate NE scope.


Because translation rarely occurs for personal names (Chen, Yang, and Lin 2003), the desired NE type "LOC" should be shared between the English NE Lake Constance and its corresponding Chinese NE "康斯 坦茨湖."

the original incorrect type "PER" of the given English NE is fixed; it thus corroborates the need for using the translation mode ratio and NE type consistency constraint.


By ignoring the initial NEs after their corresponding candidate sets have been generated, we lose the information provided by these initial NEs that is otherwise available to the alignment model.

though the initially detected NEs might be unreliable by themselves, they should act as anchors to provide scope preference information, even after the expanded candidate set has been generated from these NEs.


This equation is thus computationally infeasible due to a large search space.

it is further simplified and derived as follows by first explicitly denoting the link between C a(k) and E k as ) denote one possible alignment between C K 1 and E K 1 .


For example, if the A between "康斯坦茨湖" and Constance Lake is [康斯坦茨, Constance, TL] and [湖, Lake, TS] (N TS = N TL = 1), then its associated translation mode ratio will be 0.5 (i.e., δ = 1/2).

the internal alignment probability P(A|E, T) will be further deduced by introducing the translation mode M n and the translation mode ratio δ as follows: Combining Equations (4) and (5), the NE alignment probability P(A S,k |C a(k) , E k , T k ), which integrates internal component alignment information such as translation mode ratio and NE type constraint, is finally obtained as follows.


Let String [C] denote the associated Chinese string of C, cc l denote the l-th Chinese character within that string, and L denote the total number of Chinese characters within C. Then we will have a range of bigram probabilities for candidates with different lengths.

it is systematically biased 6 (in probability value) towards candidates with shorter lengths.


For example, in the reference { 欧盟执行委员会 ::[European Commission]}, both the Chinese components "盟" (alliance) and "执行" (execution) have no counterparts, and they would have prevented the matched portion { 委员会 ::[Commission]} from being included in the final output.

only { 欧 ::[European]} is eventually extracted.


In this case, every component in either the CNE or the ENE is well matched to its counterpart.

there is no significant difference among the alignment scores of various NE pair candidates with different scopes (such as { 韩 :: The same situation also appears in the transliteration case.


Other references with lower case words could also be found (e.g., { 世贸组织部长会议 ::[WTO ministerial meeting]}), however.

additional features such as their relative positions are also required.


Table 15 shows that an additional four type errors are also corrected apart from the six boundary errors.

in comparison with the original MERT-W, this new version (MERT-W-CB) gains more in type-sensitive F-score (an increase of 1.5%, from 81.5% to 83.0%) than in type-insensitive F-score (an increase of 0.9%, from 87.1% to 88.0%).


In fact, both PER and LOC can be freely exchanged in this situation in either a Chinese or English context.

more complicated syntactic or semantic information is required in some cases involving transliteration.


In addition, the feature LogP(δ A |T) in Equation (6) always prefers PER (which are always completely transliterated in the corpus) when all components in an NE are transliterated (i.e., δ A = 0).

the feature is ranked second (31 out of 37 errors are complete transliterations, although only 10 of them should have been assigned PER).


In addition, the model does not use linguistic rules or tree banks.

although our experiments are conducted on Chinese-English language pairs, it is expected that the proposed approach can be applied to other language pairs with little adaptation effort.


In many NLP applications, the associated process will be considerably simplified if the included NEs can be identified first.

it is important to have a good NE recognizer that has been well trained.


The view postulates that metaphor is not limited to similarity-based meaning extensions of individual words, but rather involves reconceptualization of a whole area of experience in terms of another.

metaphor always involves two concepts or conceptual domains: the target (also called the topic or tenor in the linguistics literature) and the source (also called the vehicle).


(Wilks 1978) The verb drink normally requires a grammatical subject of type ANIMATE and a grammatical object of type LIQUID, as in Example (17a).

drink taking a car as a subject in (17b) is an anomaly, which, according to Wilks, indicates a metaphorical use of drink.


≡ typically worn by women ≈ expected to be worn by women ≈ must be worn by women ≈ must be worn by Muslim women Burqa <= By doing insertions and substitutions, the system arrives from the definition "typically worn by women" to that of "must be worn by Muslim women."

it establishes a link between the concepts of make-up and burqa.


Humans use metaphor attempting to gain a better understanding of an abstract concept by comparing it to their physical experiences.

abstract concepts expose different distributional behavior in a corpus.


The system then (1) creates source domains describing these examples by means of verb clustering (such as the verb cluster in Figure 3); (2) identifies new target domains associated with the same source domain by means of noun clustering (see, e.g., ABSTRACT cluster in Figure 3), and (3) establishes a link between the source and the target clusters based on the seed examples.

the system captures metaphorical associations implicitly.


These cases were excluded because for many such examples it was hard for humans to produce a literal paraphrase realized in the form of the same syntactic construction.

their paraphrasing was deemed to be an unfairly hard task for the system.


The probabilities P(c|v) and P(c) were estimated from the corpus data as follows: where f (v, c) is the number of times the predicate v co-occurs with the argument class c in the relation R, and f (c) is the number of times the argument class occurs in the relation R regardless of the identity of the predicate.

for each verb, its SPS can be calculated for specific grammatical relations.


The metaphor paraphrasing task can be divided into two subtasks: (1) generating paraphrases, that is, other ways of expressing the same meaning in a given context, and (2) discriminating between literal and metaphorical paraphrases.

the proposed approach is theoretically grounded in two ideas underlying each of these subtasks: r The meaning of a word in context emerges through interaction with the meaning of the words surrounding it.


The highest semantic fit then indicates the most literal paraphrase.

the context-based probabilistic model is used for paraphrase generation and the selectional preference model for literalness detection.


The key difference between the two is that whereas WSD makes use of a predefined sense-inventory to characterize the meaning of a word in context, lexical substitution is aimed at automatic induction of meanings.

the goal of lexical substitution is to generate the set of semantically valid substitutes for the word.


Choosing appropriate data structures, the emptiness of shared forests can be decided in linear time and space with respect to the size of the forest.

the computational complexity of universal recognition is upper-bounded by the complexity of constructing the shared forest.


It is then straightforward to compute the string yield of t in time O(|w||G|).

we have a nondeterministic polynomial-time decider for the universal recognition problem.


In particular, the fan-out of f is exactly the block-degree of u.

any bound on the block-degree of the trees in the treebank translates into a bound on the fan-out of the extracted grammar.


Before presenting the actual binarization, we determine the parsing complexity of the binarized grammar.

because the binarization preserves the fan-out of the original grammar, and because in a grammar with fan-out k, for concatenation functions conc : k 1 k 2 we have k 1 + k 2 − 1 ≤ k and for wrapping functions wrap : k 1 k 2 j we have k 1 + k 2 − 2 ≤ k, we can rewrite the general parsing complexities as c(conc : the maximal parsing complexity in the binarized grammar is 2k + 2; this is achieved by wrapping operations.


Each of our dependency grammars is restricted to a finite block-degree.

of this restriction, our dependency grammars are not expressive enough to capture linguistic phenomena that require unlimited degrees of non-projectivity, such as the "scrambling" in German subordinate clauses (Becker, Rambow, and Niv 1992).


It will still be useful to refer to the CSP in order to access the intuition of using declarative constraints.

far, we have focused on DCS trees that only use join relations, which are insufficient for capturing higher-order phenomena in language.


The log-partition function A(θ; •, •) is convex, but O(θ, C) is the difference of two log-partition functions and hence is not concave (nor convex).

we resort to gradient-based optimization.


The book is unusually clear and honest in highlighting limitations in the current understanding of crucially relevant concepts, including existing formalization techniques.

the book is not only a valuable summary of the currently available tools for interpreting motion, but also a useful starting point for further research that aims to fill various gaps identified by Mani and Pustejovsky's exploration of the field.


But approximate equivalence is hard to define.

the phenomenon of paraphrases, as understood in linguistics, is difficult to characterize.


Because the boundaries of the spans annotated in the MPQA corpus are not strictly defined in the annotation guidelines (Wiebe, Wilson, and Cardie 2005), however, measuring precision and recall using exact boundary scoring will result in figures that are too low to be indicative of the usefulness of the system.

most work using this corpus instead use overlap-based precision and recall measures, where a span is counted as correctly detected if it overlaps with a span in the gold standard (Choi, Breck, and Cardie 2006;Breck, Choi, and Cardie 2007).


We want to tap into the annotators' intuitive notions of word meaning.


That is to say that one member entails the absence of the other, and given one member, the identity of the other member is obvious.

night and day are good examples of opposites because night is best paraphrased by not day, rather than the negation of any other term.


For example, daylight and darkness are very different along the dimension of light, and they satisfy the binary incompatibity property to some degree, but not as strongly as day and night.

we will consider both daylight and darkness as well as day and night as semantically contrasting pairs (the former pair less so than the latter), but only day and night as opposites.


r Contrasting word pairs are word pairs that have some non-zero degree of binary incompatibility and/or have some non-zero difference across a dimension of meaning.

all opposites are contrasting, but not all contrasting pairs are opposites.


Murphy and Andrew (1993) claim that this is because together opposites convey contrast well, which is rhetorically useful.

we hypothesize that the higher the degree of contrast between two words, the higher the tendency of people to use them together.


These were often words related to the focus words, but neither contrasting nor synonymous.

even though a high tendency to co-occur is a feature of highly contrasting pairs, it is not a sufficient condition for detecting them.


We used this set while we were developing our lexical contrast algorithm described in Section 4.

we will refer to it as the development set.


This resulted in non-zero counts for only 5 of the 162 instances in the development set (3.09%), and 35 of the 790 instances in the test set (4.43%).

these patterns fail to cover a vast majority of closest-antonyms, and even if the system guesses at random for all other instances, it attains only a modest improvement over the baseline (see row a, under "Related work," in Table 10).


The 13th affix pattern from Table 9 (upX and downX) indicates that the two thesaurus paragraphs have contrasting meaning.

ascent and descent occur in prime contrasting thesaurus paragraphs.


Of course, again the system was not privy to this information, but method (a) marked 58 pairs as opposites and 39 pairs as synonyms.

the system concluded that opposites are more dominant and method (c) marked all previously unmarked pairs as opposites, obtaining an accuracy of 90%.


The precision of our method is very good (>0.95).

future work will be aimed at improving recall.


They rely on it to better understand various concepts and in order to communicate effectively.

we believe that computer algorithms that deal with language can also obtain significant gains through the ability to detect contrast and the ability to distinguish between differing degrees of contrast.


Given an ordered set of meta-rules, however, there is no guarantee that the trees they derive are linguistically appropriate and that the derivation process terminates.

to ensure termination and consistency, Prolo needs to additionally provide rule ordering schemes (expressed as automata).


It is worth stressing that the XMG node identification process is reduced to a constraint-solving problem and so it is not a sequential process.

the criticisms leveled by Wintner (2007, 2009) against non-associative constraints on node unification do not apply.


For instance, for verbs, the class specifying the verbal spine (e.g., ActiveVerbForm, see Figure 10) equates the anchor node of the verbal spine with that of the projection skeleton.

the verb projects its index up to the root.


More recently, Villemonte de la Clergerie (2005,2010) proposed a compiler for FB-LTAG that aims at preserving a high degree of factorization in both the abstract grammar specification and the grammar which is compiled from it.

the MGCOMP system does not compute FB-LTAG elementary trees, but factorized trees.


Finally a traditional case of "erasing," 24 such as the agentless passive (or passive without agent) can be expressed in our language by adding an additional alternative where the by-object or agentive complement is not expressed.

equation 39is an augmentation of (38) where we have added the agentless passive alternative (indicated in boldface).


Continuing with the simplified example presented so far, we can for instance define the tree family for verbs taking a nominal subject, a nominal object, and an indirect nominal object (i.e., ditransitive verbs) as follows: The trees generated for such a family will, among others, handle the following contexts: 25 It is straightforward to extend the grammar with new families.

for instance, Equation (42) shows how to define the transitive family (for verbs taking a nominal subject and a nominal object) and Equation (43), the intransitive one (alternatives of a verb sub-categorizing for a nominal subject).


From an initial given set of root concepts and basic level terms, the authors first use Hearst-like lexico-syntactic patterns iteratively to harvest new terms from the Web.

a set of hyponym-hypernym relations is obtained.


Even though in many cases an initial domain terminology is available, new terms emerge continuously, especially in novel or scientific domains.

in this work we aim at fully automatizing the taxonomy induction process.


Therefore, in this work we aim at fully automatizing the taxonomy induction process.

we start from a text corpus for the domain of interest and extract domain terms from the corpus by means of a terminology extraction algorithm.


For this we use our term extraction tool, TermExtractor, 6 that implements measures of domain consensus and relevance to harvest the most relevant terms for the domain from the input corpus.

7 an initial domain terminology T (0) is produced that includes both single-and multi-word expressions (such as, respectively, graph and flow network).


2003;Fahmi and Bouma 2006;Westerhout 2009; Zhang and Jiang 2009), we covered not only a variety of definition styles in our training set, in addition to the classic X is a Y pattern, but also a variety of domains.

our WCL algorithm requires no re-training when changing the application domain, as experimentally demonstrated by Navigli and Velardi (2010).


, t n , where t i is the i-th token of s, we generalize its words t i to word classes t i as follows: that is, a word t i is left unchanged if it occurs frequently in the training corpus (i.e., t i ∈ F); otherwise it is replaced with its part of speech (POS(t i )).


Finally, the third sentence in Table 4, "In computer science, a pixel is a dot that is part of a computer image," is generalized as "In NN 4 NN 1 , a TARGET is a NN 2 ."

a new node NN 4 is added, corresponding to "computer" and new The Word-Class Lattice construction steps on the first three sentences in Table 4.


Next, we move to iteration i + 1 and repeat the last two steps, namely, we perform definition/hypernym extraction and domain filtering on T (i+1) .

of subsequent iterations, the initially empty graph is increasingly populated with new nodes (i.e., domain terms) and edges (i.e., hypernymy relations).


These three steps are iterated on the newly created components, until no change is made to the graph (line 18).

of our pruning recovery phase we return the enriched backbone taxonomy.


An example starting with three components, including the backbone taxonomy tree on the top and two other trees on the bottom (a).

of pruning recovery, we disconnect ordered structure (a); we connect token sequence to token list by means of a "noisy" edge (b); we connect binary tree to balanced binary tree by lexical inclusion (c); and finally binary tree to binary search tree by means of another "noisy" edge (d).


Concerning this issue, Zornitsa Kozareva comments that: "When we gave sets of terms to annotators and asked them to produce a taxonomy, people struggled with the domain terminology and produced quite messy organization.

we decided to go with WordNet and use it as a gold truth" (personal communication).


For i = 2 the B value is 0, if H 2 is the learned taxonomy, and is not defined, if H 2 is the gold standard.

when computing the cumulative Equation 9, we obtain the desired effect of penalizing less the structured learned taxonomies.


As in the previous example, however, there are category nodes that are almost equally "attractive" (e.g., algorithm and technique), and, furthermore, the taxonomy induction algorithm can only select among the set of hypernyms extracted during the harvesting phase.

when no definition suggests that distance metric learning is a hyponym of machine learning algorithm, or of any other concept connected to machine learning algorithm, there is no way of grouping distance metric learning and belief network learning in the desired way.


On the other hand, this new sentence produces an attachment that, in a sense, recovers the error, because category is a "good" domain concept that eventually ends up in subsequent iterations to the upper node abstraction.

what happens is that the upper taxonomy nodes, with the help of the domain heuristic, mitigate the "semantic drift" caused by out-of-domain ambiguity, recovering the ambiguity errors of the intermediate levels.


As we said earlier, our purpose here is mainly to evaluate the quality of Hearst patterns in more technical domains, and the efficacy of DAP and DAP −1 steps in retrieving domain concepts and relations.

replicating step (3) above is not useful in this case since, rather than adding new nodes, step (3) is aimed, as in our optimal branching and pruning recovery steps, at reorganizing and trimming the final graph.


In the present work we take this preliminary finding to the next level, by studying the impact of several graph-based WSI algorithms on a large scale and by integrating them into a Web search result clustering framework.

we are able not only to perform an end-to-end evaluation of WSI approaches, but also to compare them with traditional search result clustering techniques, which instead lack explicit semantics for the query meanings.


In other words, each vertex set V i is intended to contain only words related to a specific sense of q.

s is the sense inventory for the query q and each s i is a sense cluster.


Then, a maximum spanning tree T q of the graph is calculated starting from vertex q (see the bold edges in Figure 4(e)).

t q will include all the infinite-weight edges from q to its direct descendants, namely, the hubs.


Formally: where sim(b i , S j ) is a generic similarity value between b i and S j (0 denotes that no sense is assigned to result r i ).

of sense assignment for each r i ∈ R, we obtain a clustering C = (C 1 , .


We further calculated the average standard deviation over the two data sets' queries, obtaining 6.5 for AMBIENT and 13.1 for MORESQUE.

we anticipate that the longer the query length, the more unbalanced will be the distribution of its subtopics over the top-ranking results.


Not only have we shown that graph-based WSI, when applied to search result clustering, surpasses its nonsemantic alternatives, but we have also provided an end-toend evaluation framework that enables fair comparison of WSI algorithms.

we are able to overcome many of the issues with the evaluation of clustering algorithms (von Luxburg, Williamson, and Guyon 2012), including the lack of a single unbiased intrinsic measure (Manandhar et al.


Even though (statistical) phrases are more representative units for classes than single words (Caropreso, Matwin, and Sebastiani 2001), they are so sparsely distributed that they have limited impact during the classification process.

it is not surprising that the best scoring multi-class, multi-label 3 classification results for the well-known Reuters-21578 data set have been obtained using a BOW representation (Bekkerman and Allan 2003).


In the multiview classifiers, the unigram and bigram classifiers make similar decisions in assigning labels, although the latter generally yielded lower confidence values.

in the merge the unigram and bigram classifiers affirm each other's decisions, which does not result in an overall improvement in classification accuracy.


Syntactic phrases appear to be even sparser than bigrams.

it is not surprising that most papers concluded that classifiers using only syntactic phrases perform worse than the baseline, except when the BOW baseline is low for that particular classification task (Mitra et al.


that the most meaningful phrases are expressed in local dependencies, that is, adjacent words.

syntactic analysis aimed at discovering meaning expressed by long-distance dependencies can only make a small contribution.


The best explanation for an individual morpheme is that it is intended to convey its corresponding predication.

the syntactic analysis of a clause bottoms out in its logical form.


And we have axioms defining or characterizing the former in terms of the latter.

in the "range" example, range is the predicate corresponding to the morpheme.


Emotions, like cognition more generally, mediate between perception and action.

for a particular emotion, we specify an abstract type of perceived eventualities that cause the emotion, and we specify an abstract class of typical responses.


In practice, however, only the ATTACH-RIGHT(i) EF and ATTACH-LEFT(i) EF transitions such that 1 ≤ i ≤ n − 1 need to be considered when parsing a string of length n: Because the number of nodes in the buffer is bounded by n, transitions with i ≥ n will always be undefined because the buffer will become empty before the first i + 1 elementary transitions can be applied.

to parse strings of length n we only need 2n − 1 transitions.


The reasoning is analogous, but in this case max(i, j) is the node that gets removed from the set Π when the arc (k, l) is added, making it impossible to add the arc (i, j) afterwards.

divisible systems can only generate planar dependency graphs.


Therefore, by the induction hypothesis, we know that all the left endpoints of the arcs in L n+1 are in σ n .

if the left endpoints of the arcs in L n+1 are i 1 , i 2 , .


Completeness is also straightforward, because, as discussed in Proposition 2, applying a LEFT-ARC transition to a configuration of H σ (C) or a RIGHT-ARC transition to a configuration of H β (C) will always generate a graph violating the SINGLE-HEAD constraint.

any graph that meets the SINGLE-HEAD constraint and can be obtained using the S P transition system (which has been proven complete) can also be generated by this one.


Therefore, given a graph G that satisfies the NO-COVERED-ROOTS constraint, we know that the transition sequence built as in that proof will never reduce a root node.

all of its REDUCE transitions will be applied to configurations in H σ (C) and, hence, that same transition sequence will also parse G in this variant of the transition system, which proves completeness.


As explained in Section 3.3.2, however, this can be implemented in constant time if the SINGLE-HEAD constraint is present by keeping track of the undirected connected component of each node in the generated dependency graph.

as explained in Section 2.3, the complexity of the planar parser with beam search is O(n), both for the unrestricted version and for the variant that enforces the SINGLE-HEAD and ACYCLICITY constraints.


As already noted, planarity is a very mild relaxation of the better known projectivity constraint, the only difference being that planarity allows graphs with covered roots (see Definition 2), and studies of natural language treebanks have shown the vast majority of non-projective structures to be non-planar as well (Kuhlmann and Nivre 2006;Havelka 2007).

8 being able to parse planar dependency graphs only provides a modest improvement in practical coverage with respect to projective parsing.


Therefore, by the induction hypothesis, we know that all the left endpoints of the arcs in A b n+1 are in σ b n .

if the left endpoints of the arcs in A 1 n+1 are i 1 , i 2 , .


The 2-planar parser can be restricted to graphs satisfying the SINGLE-HEAD and ACYCLICITY constraints in exactly the same way as the planar parser, and the proofs follow the same line of reasoning.

a version of the 2-planar parser that is sound and complete for the set of 2-planar dependency forests (that is, 2-planar dependency graphs without cycles and with each node having at most one head) can be defined as follows: only if ¬∃k | (k, i) ∈ A (single-head) and ¬i ↔ * j ∈ A (acyclicity) RIGHT-ARC 2P−ShAc = RIGHT-ARC H β (C)∩{(σ 1 |i,σ 2 ,j|β,A)∈C|¬(i↔ * j∈A)} only if ¬∃k|(k, j) ∈ A (single-head) and ¬i ↔ * j ∈ A (acyclicity) Because structures in dependency treebanks are typically restricted to forests, this is the version of the 2-planar parser that we use in the experimental evaluation in Section 5.2.


In general, this exemplifies the tradeoff between approximate inference with richer models (beam search) and exact inference with simpler models (dynamic programming).

although the feature model used by Zhang and Nivre (2011) to achieve state-of-the-art accuracy for English makes dynamic programming very difficult due to the combinatorial effect on parsing complexity of complex valency and label set features, the feature representation of a single configuration can still be computed in constant time, which is all that is required to achieve linear-time parsing with beam search.


Due to the growing popularity of dependency parsing, several proposals have been made that group and study different dependency parsers under common (more or less formal) frameworks.

buchholz and Marsi (2006) observed that almost all of the systems participating in the CoNLL-X shared task could be classified as belonging to one of two approaches, which they called the "all pairs" and the "stepwise" approaches.


On the one hand, this can be used as an analytical tool to characterize existing systems for dependency parsing and prove formal properties related to expressivity and complexity.

we have shown that all divisible systems, including a number of well-known systems from the literature, are sound for planar dependency graphs and can be restricted to satisfy a number of other formal constraints, and we have characterized the subclass of efficient divisible transition systems that give linear parsing complexity when combined with greedy inference or beam search as is customary in transition-based parsing.


Note that the number of partitions (entities) in a document is a priori unknown, but it is at most the number of vertices (mentions) because, in an extreme case, each mention in a document could refer to a different entity.

a vertex with index i can be in the first i partitions (i.e., L i = i).


The calculation of the support S il for a vertex v i and label l is an iteration over the incident edges E(v i ), which is equal to n in an all-connected graph.

the adjacency of the vertices depends on the size of the document.


The MUC measure is link-based and does not take singletons into account, anyway.

it is the only one comparable with the state of the art at this point.


The scores for Spanish and Catalan do not seem as good as for English, because the system was originally designed with the English language in mind.

it does not include languagespecific features for Spanish and Catalan, such as whether a mention is an elliptical subject or not.


The number of really useful examples (i.e., mentions wrongly classified by the mention-pair model but correctly classified by the entity-mention model), however, is low.

the difference in their scores is not significant.


(2007), these results can be roughly generalized to any other system using similar information, and even other languages.

these classes require attention in order to improve global performance, and the fact that lexical, morphological, syntactic, and semantic levels are not very useful to deal with them encourages the research on adding world knowledge to coreference resolution systems.


When searching for antecedents, we sort candidates in the same sentential clause from left to right, and we prefer sentences that are closer to the mention under consideration (see Section 3.2.2 for details).

the sorted list of candidates for John 9 10 is It 7 7 , My favorite 8 8 , My 9 9 , A girl 5 5 , the song 6 6 , He 3 3 , a new song 4 4 , John 1 1 , a musician 2 2 .


Recall that our model is a battery of resolution sieves applied sequentially.

in each given sieve, we have partial mention clusters produced by the previous model.


Lastly, pronominal coreference resolution (Sieve 10) is responsible for approximately 9.5 CoNLL F1 points improvement.

it would be possible to build an even simpler system, with just three sieves, that achieves 97% of the performance of our best model (based on the CoNLL score).


Furthermore, our model considers only noun phrases as antecedent candidates, thus missing all mentions that are verbal phrases.

our system misses most coreference links between event mentions.


Given a suspicious document d q and a collection of potential source documents D, the detector should retrieve: (a) a specific text fragment s q ∈ d q , a potential case of plagiarism; and (b) a specific text fragment s ∈ d, the claimed source for s q .

special versions of precision and recall have been proposed that specially fit in this framework (Potthast et al.


As a result, directly comparing paraphrase annotation and detectors' outcomes is not possible, and organizing the data in a way that makes them comparable is required.

we grouped together cases of plagiarism with similar concentrations of paraphrases or in which a kind or paraphrase clearly stands out from the rest in order to observe how the detectors performed on different profiles of plagiarism.


Because same-polarity substitutions occur so often in many different plagiarism cases (this type represents more than 45% of the paraphrase operations in the P4P corpus and 96% of the plagiarism cases include them), however, they do not represent a good discriminating factor.

this was confirmed by a preliminary experiment carried out considering different values for k. k-means was applied by considering 21 features only.


On the one hand, Nawab, Stevenson, and Clough (2010) apply greedy string tiling, which aims at detecting as long as possible identical fragments.

this approach clearly outperforms the rest of detectors when dealing with cases with a high density of identical fragments (c 5 in Figure 7).


Note that such pairs of syntactic and semantic subsequences will, in general, have different numbers of steps on each side and these numbers of steps are, in general, unbounded.

instead of defining atomic synchronized rules as in synchronous grammars (Wu 1997;Chiang 2005), we resort to parametrized models that exploit the internal structure of the paired subsequences.


Most of these approaches not only learn syntactic and semantic representations independently, but also use pipelines at testing time.

in these methods semantic information does not influence syntactic parsing (Punyakanok, Roth, and Yih 2008;Toutanova, Haghighi, and Manning 2008).


This conversion implies that the span of words covered by the subtree headed by the word receiving the label can often be interpreted as receiving the semantic role label.

for the dependency-based representation, the syntactic and the semantic graphs jointly define the semantic role information.


Despite the shortcomings that will be discussed later, we adopt this method because the amount of non-planarity in syntactic structures is often small: only 0.39% of syntactic dependency arcs in the English training set are non-planar.

choice of the planarization strategy for syntactic dependencies is not likely to seriously affect the performance of our method for English.


We train the ISBN to maximize the fit of the approximate model to the data.

both at parsing time and at training time, the parameters of the model are interpreted according to the feed-forward approximation discussed in Section 4.1.5, and not according to the exact latent variable interpretation of ISBNs.


Generally, translated texts are not as rich and variable as original ones, and, in particular, their type/token ratio is lower.

we expect S → T phrase tables (which are based on a parallel corpus whose source is original texts, and whose target is translationese) to have more unique source phrases and a lower number of translations per source phrase.


In any derivation of some PLTAG grammar G, the nodes that each prediction tree contributes to the derived tree are replaced by the nodes of some (canonical) verification tree.

the resulting derived tree can be built (non-incrementally) from only canonical trees, using substitution and adjunction.


Therefore the resulting derived tree can be built (non-incrementally) from only canonical trees, using substitution and adjunction.

if we build a TAG grammar G from G by removing all prediction trees, G and G describe the same set of derived trees.


This replicates the situation in a non-incremental, top-down TAG derivation, in which the verification tree would have been directly integrated with u predict without the intermediate prediction step.

we factorize the bilexical probability as follows: In order to calculate the probability of a derived tree using this probability model, each node is visited twice during the derivation, once from the left side and once from the right side.


If we try to adjoin β 2 into the left A node, the result is (d); this is a tree whose first two leaves are lexical, but we have processed the second word before the first in the derivation.


A verification step for the marker i will only be successful, however, if the lower half of the last node on the current fringe is marked with i.

the current fringe can at least provide a necessary condition for filtering out obviously useless verification operations.


It is possible, however, that the effect of a single predictor of interest can be explained away by simpler predictors that are not of theoretical relevance, but are highly correlated with reading times.

we fitted full models that included predictors that relate to properties of the word, like word length in characters (WORDLENGTH), word frequency in log scale (WORDFREQUENCY), as well as word position in the sentence (WORDNOINSENTENCE).


It is important to notice, however, that the reference is not a trustworthy gold standard, because there are many possible alternative good translations, and the real quality of long words are likely underestimated.

as can be seen in Table 10, the clear majority of produced compounds in the EPOS systems are judged acceptable, even though the precision of overlap for long words with the reference is at most 47% for the EPOS models.


Note that we used a conservative white list that filters out targets whose morphological variants were not seen either in the lexicon or the training data.

14 when this conservative process of automatic target identification is used, our system loses the capability to predict frames for completely unseen LUs, despite the fact that our powerful frame identification model (Section 5) can accurately label frames for new LUs.


We construct a graph with lexical units as vertices.

each vertex corresponds to a lemmatized word or phrase appended with a coarse POS tag.


Second, during execution of the AD 3 algorithm, we always keep track of a sequence of feasible dual points.

each iteration constructs tighter and tighter upper bounds.


Then, they study different graph-based centrality algorithms for deciding the relevance of the nodes on the subgraph.

every word of the context is attached to the highest ranking concept among its possible senses.


The right side of Figure 2 depicts the PPR w2w method, where the word coach is not activated.

there is no reinforcement between coach senses, and the method would correctly choose coach#n5 as the proper synset.


Given a naive Bayes statistical language model, we construct an NB-R representation that produces |S| boolean features F s (x i ) for each token x i and each possible latent state s ∈ S: For a reasonable choice of S (i.e., |S| |V|), each feature should be observed often in a sufficiently large training data set.

compared with n-GRAM-R, NB-R produces far fewer features.


1.8% of tokens in the biomedical test set had POS tags like 'HYPHENATED', which are not part of the tagset for the training data, and were labeled incorrectly by all systems without access to labeled data from the biomedical domain.

an error rate of 1.8 + 3.9 = 5.7 serves as a reasonable lower bound for a system that has never seen labeled examples from the biomedical domain.


On the other hand, the Moroccan dialect, especially in its spoken form, is quite difficult to understand by a Levantine speaker.

from a scientific point of view, the dialects can be considered separate languages in their own right, much like North Germanic languages (Norwegian/Swedish/Danish) and West Slavic languages (Czech/Slovak/Polish).


Relatively little printed material exists in local dialects, such as folkloric literature and some modern poetry, but the vast majority of published Arabic is in MSA.

mSA's dominance is also apparent in data sets available for linguistic research.


The sentences were randomly grouped into sets of 10 sentences each, and when Workers performed our task, they were shown the 10 sentences of a randomly selected set on a single HTML page.

each screen contained a mix of sentences across the three newspapers presented in random order.


As long as P z (τ(Z) = t) = P t (t) for all trees t, our algorithm will generate trees from the desired distribution.

for any tree t the probability P z (Z[t] = ζ(t)) is fixed to P t (t), but any distribution over the remaining variables not contained in t will still yield the desired distribution over trees.


Thus for any tree t the probability P z (Z[t] = ζ(t)) is fixed to P t (t), but any distribution over the remaining variables not contained in t will still yield the desired distribution over trees.

in designing the sampling algorithm, we may choose any distribution P z (Z[¬t] | Z[t] = ζ(t)).


Detailed balance is the condition that, on average, for each pair of states z and z , the number of transitions between the two states is the same in either direction: where is the transition performed by one step of the sweep: It is important to observe that, because the resampling step only changes the tree structure below the ith node, the ith node in the new tree remains the same node.

that is, after making an update from z to z , v(z, i)  , and, mathematically: the condition in Equation 8is symmetric in z and z , and we define the predicate match(z, z , i) to be equivalent to this condition.


We attach a binary variable to each node in the forest indicating whether the node is a boundary between two TSG rules, or is internal to a single TSG rule.

the complete set of variables used by the sampler, z, now consists of two variables at each node in the forest: one indicating the incoming hyperedge, and one binary boundary variable.


By the definition of union closure, * (T 1 ∪ T 2 ) covers both T 1 and T 2 .

t 1 → n and t 2 → n cannot both be minimal.


is also a valid phrase formed by taking the difference of two overlapping phrases, which makes k 1 a valid binary split point for n. This makes T 1 either an edge formed by the binary split at k 1 , or not a minimal phrase.

whenever we have two minimal edges, both consist of a binary split.


The intuition behind this definition is that it measures the number of free indices into the source language string required during parsing, under the assumption that the terminals provide fixed anchor points into the string.

a rule of scope of k can be parsed in O(n k ).


Suppose that we have a measuring tape available to measure distance, and suppose further that our only method to measure weight is the use of beam balance scales.

let us denote by z the entity consisting of the balanced beam scale, the child, and the counterbalancing measuring weight; then the validity of our measuring result depends on a statement z is an AS in order to check statement (2), we have to presuppose statement (3), which is of the very same form and uses the very same predicate.


The sketched scenarios are inherently circular in the sense of the problem of theoretical terms described previously.

in all cases, we are prevented from making empirical statements.


(2010) use the correct response for a query to a database of geographical facts to select data for structured learning of a semantic parser.

the extrinsic set-up can be seen as a general technique for T-non-theoretical grounding in training as well as in testing scenarios.


This fixes the formal problem, but normally does not improve accuracy because it is usually unlikely that more than one word should attach to the artificial root node.

in the error analysis presented by McDonald and Nivre (2007), MaltParser tends to have very low precision on attachments to the root node.


We implemented Wallach, Mimno and McCallum's method for all the models we considered, but found that it did not improve authorship attribution accuracy in preliminary experiments.

in all our experiments we set the elements of α α α (D) and α α α (A) to min{0.1, 5/T (D) } and min{0.1, 5/T (A) }, respectively, yielding relatively sparse topic distributions, since we expect each document and author to be sufficiently  Three-dimensional Dirichlet probability density, given three prior vectors.


This allows us to encode our prior knowledge that stopword use is indicative of authorship.

for DADT we set where v is a stopword ( can be set to zero to obtain symmetric priors).


Note that when analyzing single-authored documents, the author representations yielded by AT are equivalent to LDA-S's representations.

we do not report results obtained with LDA-S.


This allows AT to adapt itself to each document without changing the model specification.

we consider the two following variants: (1) AT: "Pure" AT, without fictitious authors; and (2) AT-FA: AT, when run with the additional preprocessing step of adding a fictitious author to each document.


First, in DADT, author topics are disjoint from document topics, with different priors for each topic set.

the number of author topics T (A) can be different from the number of document topics T (D) , which enables us to vary the number of author and document topics according to the number of authors and documents in the corpus.


This extension is performed by using a conditional independence assumption between the observed random variables corresponding to object features.

instead of considering as features only the words that express the event mentions (which is the way an observable object is represented in the original HDP model), we devised an HDP extension that is also able to represent features such as location, time, and agent for each event mention.


In our implementation, this statement is equivalent with the process of randomly selecting for the first event mention a number of Poisson(α ′ ) observed feature values.

in the general case, the sampling of the binary variable from the m th Markov chain and associated with the t th event mention depends on the value assigned to the hidden variable in the previous t − 1 step: in our implementation, the observed feature value f m is selected for the t th event mention according to the probabilities presented in Equation (11).


This is because, in spite of the fact that OntoNotes provides coreference annotations for both entity and event mentions, the annotations from this data set do not specify which of the mentions refer to entities and which of them refer to events.

in order to identify only the event mentions from OntoNotes, we first ran our event identifier  and then marked as event mentions only those mentions annotated in this data set that overlap with the mentions extracted by the event identifier.


For a given mention m, the scorer compares the true coreference chain that contains the mention m (T m ) against the system chain that contains the same mention m (S m ).

the recall for m is the ratio of the number of common elements in S m and T m over the number of elements in T m .


A single derivation is used for both strings, which limits the divergence phenomena that can be captured.

researchers have developed synchronous grammars with larger rules that, rule-internally, capture more phenomena, typically at increased computational expense (Shieber and Schabes 1990;Eisner 2003;Gildea 2003;Ding and Palmer 2005).


The productions in this monolingual grammar generate a piece of the translation's tree and align it to a piece of the fixed input tree.

arbitrary non-isomorphic structures are possible between the two trees.


The feature weights θ θ θ for the fine model are learned by solving the following quadratic program: In Equation (39b), the violation in the margin ξ i is scaled by the cost of the translation.

if in addition to y y y * i there are other good solutions in the set, the margin for such solutions will not be tightly enforced.


However, given the current state-of-theart of Natural Language Processing (NLP) technology, NLP techniques are not capable of creating meaningful and natural text from scratch and of hiding messages in it.

most of the existing linguistic stegosystems take already-existing text as the cover text, and linguistic properties of the text are used to modify it and hide information.


Although some computational steganalysis systems have been developed to identify stego text from innocent text using statistical methods, these systems can only be applied to text that undergoes certain linguistic transformations such as translation and lexical substitution, and they are not accurate enough for practical evaluation.

most of the current linguistic stegosystems were evaluated by human judges (Murphy andVogel 2007a, 2007b;Meral et al.


In other words, we do not investigate the document-level coherence of stego text because this requires sophisticated knowledge of natural language semantics and pragmatics which we consider to be outside the scope of this work.

it is possible that a totally natural stego sentence can still raise suspicion if it contains words which stand out as being completely different from the rest of the paragraph.


Firstly, the words in the synsets {complete, finish} are encoded with 0 and 1 with base 2, and the words in the synset {labor, project, task, undertaking} are encoded with 0, 1, 2, and 3 with base 4.

the combinations of the substitutions yield the two-digit mixed-radix numbers from 0 2 0 4 to 1 2 3 4 , which are equal to the decimal numbers 0 to 7.


In this example we can see that project has the highest probability among words in the same synset and is encoded with the shortest code.

it is more likely to match the secret bitstring.


For example, if the sender encodes the four best machine translations of the cover sentence using block coding and sends the translation that represents the secret bits to the receiver, it is unlikely that the receiver can retrieve the four best machine translations without knowing the original cover sentence.


As mentioned previously, disconnected graphs are encoded independently.

it is possible that both wed and join are assigned the same codeword which does not match the secret bits.


In contrast, in the ranking task a system is already given a fixed pool of substitutes and is asked to recover the order of the list.

we use the Generalized Average Precision (GAP) to evaluate the ranked lists rather than the metrics provided in the SemEval-2007 lexical substitution task.


Although the ranking task evaluation gives some indication of how reliable the proposed scoring methods are, for the steganography application we require a system that can correctly distinguish acceptable substitutes from unacceptable ones.

we conduct a classification task evaluation which is more related to the steganography application.


Because imperceptibility is an important issue for steganography, we would prefer a system with a higher precision value.

we adopt the NGM method as the linguistic transformation checker in our lexical substitution-based stegosystem.


99.6% of synsets in WordNet have a size of less than eight, which means that most of the synsets cannot exhaust more than a two-bit coding space (i.e., we can only encode at most two bits using a typical synset).

we restrict the maximum codeword size in our coding method to two bits.


To extract the secret message, the enemy needs to generate the same coded synonym graphs as constructed by the sender.

it is difficult to recover the secret bits without knowing the score threshold and the code randomizer.


In this work, we only evaluated the lexical substitution in terms of the sentencelevel naturalness rather than meaning retention and document-level coherence.

it would be interesting to see to what extent the proposed substitution checkers are useful for the security of linguistic steganography at the document-level.


As mentioned in Section 2.2, there is no research on the practical issue of using different types of cover text for the steganography application.

it would be interesting to see whether some types of cover text are better suited to linguistic steganography than others.


The shared task of CoNLL now includes a POS column in its corpora.

the POS tag is a good distinguishing feature for Arabic NEs; it has been studied separately in the literature to determine its impact on NER.


Pang and Lee, for example, are aware of prior work in the field on fact and event-based text analysis and, within that context, focus consciously on the description of "new challenges 1 raised by sentiment-aware applications" as well as the methods proposed to address them.

the survey proves to be an easy, comfortable (and entertaining) read for those with an NLP-centric ancestry.


The remaining chapters are organized around this definition and the sentiment-based applications that it enables.

there are chapters on document-level sentiment classification and rating prediction (Chapter 3), sentence-level subjectivity and sentiment classification (Chapter 4), and aspect-based sentiment analysis (Chapter 5, roughly one-quarter of the book).


2 Whereas the authors acknowledge that from the NLP perspective, "[the aim] is to select the most useful representation for a particular application" (page 12), no examples are provided that demonstrate the impact of selecting one representation over another.

although the overview is of great value to a reader wishing to familiarize themselves with the topic, the advanced reader will be left with the impression that the authors do not know how to select from among the relation sets either.


Human annotation is not only slow and expensive, but it also introduces errors and inconsistencies because of the difficulty and tiring nature of the task.

1 automating parts of the annotation process aims to leverage effectiveness, producing a larger number of high-quality and consistent analyses in shorter time and using fewer resources.


Like Baldridge and Osborne (2003), we investigate ensemble parsing in the context of HPSG grammars; however, our goal does not involve selecting the most informative sentences to retrain the parser, but rather to select those sentences most reliably parsed, in order to enlarge the treebank automatically.

rather than selecting sentences on which two models disagree, we select those where they agree completely.


The fact that tree entropy yields worse values than the baseline is somehow predictable: Given a sentence with n possible trees (note that n may be in the order of dozens or even hundreds), if a small number m of those analyses (1 < m << n) concentrate a large portion of probability mass but exhibit small differences between them, the sentence will be rejected by the baseline (because there is not enough distance between the first and second analyses) but will be accepted by tree entropy (because entropy will be relatively low, given the large value of n).

tree entropy is a good measure for Baldridge and Osborne (2003), whose purpose is to select sentences where the model is less confident, but our simple baseline seems to be better when the goal is to select sentences where the first parse is the correct one.


Because the derived f-descriptions can never be unsatisfiable (the string grammars do not contain atomic values), the f-structure constraints of the string grammars do not actually filter the language of the context-free grammar.

g and its string grammars must have the same language L(g) = L(String 1 (g)) = L(String 2 (g)).


G then assigns to a terminal string an f-structure with reentrant (root E) and (root E') paths if and only if it has the form s s and s is in If all we do is unification on the top level the f-structures for the strings s 1 s 2 would still record information on the structure of their derivation.

distinct strings in {s s | s ∈ L(G 1 ) ∩ L(G 2 )} would get assigned distinct f-structures.


Obviously, if (root E) = (root E') holds in the unified f-structures of the string grammars, then the unification of the string grammar f-structures and the structure in Figure 2 yields F. Otherwise, their unification results in a structure that only properly subsumes F. This is because neither (root E E) nor (root E' E) exists in the unified f-structures of the two string grammars, and therefore their values in the structure in Figure 2 are not merged when the structures are unified.

g derives with F exactly the set of strings We now give a rigorous statement and proof of our undecidability theorem.


Note first that also G does not contain atomic values.

fD cannot be unsatisfiable and must have an f-structure.


Since the emptiness problem for the intersection of context-free languages is in general undecidable, the generation problem must be undecidable too.

of this theorem we know that there does not exist a general generation algorithm, at least if cyclic input structures are considered as legitimate inputs.


If the word w is positive then the probability that the random walk hits a positive word is higher, and if w is negative then the probability that the random walk hits a negative word is higher.

if the word w is positive then the average time it takes a random walk starting at w to hit a positive node should be much less than the average time it takes a random walk starting at w to hit a negative node.


The true gradient is usually the sum of the gradients from each individual training instance.

batch gradient descent requires the training method to go through the entire training set before updating parameters.


This is because a higher frequency feature is expected to be well optimized with higher confidence.

a higher frequency feature is expected to have a lower learning rate.


Usually the convergence speed based on number of training iterations is moderately faster, but the time cost per iteration is slower.


Even if the γ γ γ is initialized with high values of learning rates, after a number of training passes (denoted as T) γ γ γ T is guaranteed to be small enough so that γ max : Without losing generality, our convergence analysis starts from the pass T and we take γ γ γ T as γ γ γ 0 in the following analysis.

we can show that the ADF update rule with a decaying learning rate vector is a dynamic contraction mapping: Theorem 2 (ADF update rule with decaying learning rates) Let γ γ γ t be a learning rate vector in the ADF learning algorithm, which is decaying over the time t and with different decaying rates based on feature frequency information.


This indicates that the feature frequency information has similar characteristics across many different natural language processing tasks.

we simply use the following setting for all tasks: q = n/10, α = 0.995, and β = 0.6.


2008) Latent-variable CRF 94.3 Our Method CRF + ADF training 94.5 multiple models).

it is impressive that our single model-based system without extra resources achieves good performance.


In other words, for the SGD update rule F SGD , the fixed learning rate γ max is derived from the ADF update rule.

according to Lemma 1, the SGD update rule F SGD is a contraction mapping in Euclidean space with Lipschitz continuity degree 1 − γ max /σ 2 , given the condition that γ max ≤ (||x 2 Combining Equations (6) and (7), it goes to |F(w w w t+1 ) − F(w w w t )| ≤ (1 − γ max /σ 2 )|w w w t+1 − w w w t | according to the definition of dynamic contraction mapping, the aDF update rule is a dynamic contraction mapping in Euclidean space with Lipschitz continuity degree 1 − γ max /σ 2 .


Following Theorem 2, we have the following formula for the ADF training: where γ max is the maximum learning rate at the beginning.

let d 0 := |w w w 2 − w w w 1 | and q := 1 − γ max /σ 2 , then we have: When Hence, we have: M t is upper-bounded.


In probabilistic modeling, latent variables are random variables whose values are not provided by the input data.

their values must be inferred at the same time as the model parameters on the basis of the training data and model structure.


topic model trained on document-word co-occurrences will find topics that reflect broad thematic commonalities, the model trained on syntactic co-occurrences finds semantic classes that capture a much tighter sense of similarity: Words assigned high probability in the same topic tend to refer to entities that have similar properties, that perform similar actions, and have similar actions performed on them.

class 1 is represented by attack, raid, assault, campaign, and so on, forming a coherent semantic grouping.