For example, the predicate break is transitive in sentences (1a) and (1b), and intransitive in sentence (1c).

by taking into account the argument's syntactic position and the predicate's transitivity, we can guess that the semantic role expressed by the subject in sentence (1c) is different from the roles expressed by the subjects in sentences (1a,b).


In particular, if a hypothetical system were to be used for automatically annotating data, low collocation would result in higher annotation effort and low purity would result in lower data quality.

high purity is imperative for an effective system whereas high collocation contributes to efficient data labeling.


Finally, we should point out that Titov and Klementiev (2012a) do not cluster adjunct-like modifier arguments that are already explicitly represented in syntax (e.g., TMP, LOC, DIR).

their Coupled+Mods model is most comparable to ours in terms of the clustering objective as it treats both core and adjunct arguments and does not make use of the Brown clustering.


Data sparsity affects our method, which crucially relies on lexical similarity for determining the role-equivalence of clusters.

we reduced the number of syntactic cues used for cluster initialization in order to avoid creating too many small clusters for which similarities cannot be reliably computed.


Therefore, the absolute value of the bias forĤ(c, k) will exceed the aggregate bias of the estimators of marginal entropy,Ĥ(c) andĤ(k).

the V-measure will be positively biased, and this bias would be especially high for systems predicting a large number of clusters.


This type of longtail distribution can be modeled with Zipf's law.

most of our experiments consider Zipfian distributions.


Because T is a regular language and M is represented by a finite automaton, it follows that T and L can themselves be represented as finite automata.

steps 2 and 3 can be solved using weighted finite-state intersection and single-source shortest path algorithms, respectively (Mohri 2009).


A (restricted) Dyck language consist of "well-formed" or "balanced" strings over a finite number of pairs of parentheses.

the string ( [ ( ) ( ) ] { } [ ] ) ( ) is in the Dyck language over three pairs of parentheses (see Berstel 1979 for a more detailed presentation).


We wish to prove that for any machine N, ShortestPath(M ∩ N ) passes through the equivalent states in M to those passed through in M for ShortestPath(M ∩ N).

determinization of the resulting intersection after -removal yields the same topology as intersection with the equivalent φ machine.


We restrict ourselves in this discussion to the left-categorial semiring, the right-categorial version being equivalently defined.

we define the left-categorial semiring (Σ * , ⊕, ⊗, ∞ s , ) over strings Σ * with and ∞ s as special infinity and null string symbols, respectively (as in the normal string semiring).


The accumulated weight computed by the determinization algorithm through 0-1-3 consists of three components: the lowest weight for fine, the saved residual, and the arc weight for mead from 1-3.

the accumulated weight for 0-1-3 for fine mead is 1, JJ ⊗ 1, JJ\VB ⊗ 7, NN = 9, VB_NN .


Similarly, the determinization algorithm for the input fine me will result in picking the weight 5, VB_PRP .

the determinization algorithm will produce the desired result for both input strings in Figure 4 and this can be shown to be true in general.


Each state is thus associated with a distinct single prefix, and each must be reachable from the same set of previous states as the original state.

for each new state, any arc that already has the original state as its destination state must be copied, and the new arc assigned the new destination state and weight, depending on the prefix associated with the new state (lines 26-30).


In other words, the probability of a stack occurrence given the previous stack is factorized as P .

the probability that inform(pricerange(cheap)) is followed by inform(type(restaurant)) will be high even if inform(type(restaurant)) was not observed, as long as inform(pricerange(cheap)) is frequently followed by the tail symbol inform(type(SOMETHING)) in the training data.


Our system does not consider verbs when creating candidate mentions.

it does not resolve coreferences involving events.


Hence, a specific noun phrase may be annotated as a mention in one document but not in another document due to the absence of coreferring mentions in the latter.

machine learning-based mention detectors are difficult to build in this case.


Our coreference resolution approach is similar to previous structure learning approaches for dependency parsing (McDonald, Crammer, and Pereira 2005;.

the arg max combinatorial problem reduces to a maximum branching problem, which can be efficiently solved using the Chu-Liu-Edmonds algorithm (Chu and Liu 1965;Edmonds 1967).


We use the arcs of all training examples; that is, for each training document, we generate an example for each candidate arc.

the learned decision tree (DT) predicts whether the mentions linked by an arc are coreferring.


Because we aim to generate feature templates-conjunctions of basic features that do not include feature values-we ignore the feature values and the decision variable values in the DT.


This predictor finds a maximum scoring document tree for x x x that follows the correct clustering y y y.

its search is constrained to a subset H(x x x, y y y) contained in H(x x x).


Nevertheless, the optimization algorithm is identical in both cases because the loss function can be factored along arcs.

in the large margin predictor, we simply add the loss function values to the original arc weights.


The resulting sieves can also be directly applied to the Arabic and Chinese data sets provided in the CoNLL-2012 Shared Task.

given a mention pair (i, j), where i appears before j in the text, we create a directed arc e = (i, j) if at least one of the following conditions holds: Distance -the number of mentions between i and j is not greater than a given threshold.


When using negative features, the structured perceptron will most likely learn negative weights for these features.

arcs that activate negative features have a lower chance of inclusion in the solution because these arcs tend to decrease the coreference tree weight.


The Arabic data set does not contain named entity, semantic role labeling, and speaker annotations.

for Arabic, we do not derive the following basic features: Sy9, Se3, Se4, Se5, Se6, Se7, and P3.


Features Sy9, Se5, and Se6 are all derived from the SRL basic features.

when following our template generation strategy, SRL information contributes little to the coreference resolution task.


The main issue is that coreference information is highly faceted, and the value of each facet varies substantially from one application to another.

when reporting and comparing coreference system performances, it is very difficult to define one metric that fits all purposes.


If two coreferent nodes m 1 and m 2 belong to different connected components in G(x x x), the coreference tree predictor is unable to generate a tree on G(x x x) containing the coreferent nodes m 1 and m 2 .

there is a trade-off between the restrictiveness of the sieves and the coreference resolution recall.


Then, we use these static trees as the golden standard for training.

the perceptron algorithm learns to predict chains of coreferring mentions instead of general trees.


The effect of a root loss value larger than one is to reduce the creation of new clusters, stimulating larger clusters.

this parameter can be used to adjust the balance between precision and recall.


Observe that a constrained document tree is computed on the graph generated in the candidate pair generation step, which already includes recall errors.

the frequencies reported here are computed over the remaining errors.


However, as far as we know, there is as yet no structure learning system that considers such features.

we must design new prediction algorithms to cope with this complex context.


Moreover, WordNet mostly provides taxonomic knowledge, while neglecting much syntagmatic relational information between concepts.

over the past few years several automatic techniques have been proposed that enrich WordNet with new relation edges, such as those obtained from disambiguated glosses (Mihalcea and Moldovan 2001), collocation dictionaries (Navigli 2005), topic signatures (Agirre et al.


However, we know that the different senses of a real word are often in a semantic or etymological relationship.

random pseudowords can only model homonymous distinctions (such as the centimeter vs. curium senses of the noun cm), and fall short of modeling systematic polysemy (such as the lack vs. insufficiency senses of the noun deficiency).


For each set of these pseudowords we generated corresponding pseudosenseannotated training and test data sets by randomly sampling distinct sentences from the English Gigaword corpus (Graff and Cieri 2003).

we ended up with four data sets, namely: the Senseval-3 data set of real words, and the three artificially sense-tagged data sets for the similarity-based, TS-based, and random pseudowords.


In fact, topic signatures are based on co-occurrence information from the Web snippets retrieved for each sense of an ambiguous noun.

many of the top-ranking words in each topic signature are syntagmatically related to the given sense.


The overall score of 0.79 shows that a large portion of pseudosenses can be associated with their corresponding real senses only.

we can conclude that the similarity-based pseudowords effectively preserve the distinguishability of senses of their real counterparts.


Experimental results proved that the similarity-based approach is able to provide a good modeling of individual senses of real words while preserving the distinguishability of their senses.

of our evaluations we know that the similarity-based pseudowords are reliable substitutes for real ambiguous words in the disambiguation task.


Secondly, we observed that in practice a large enough portion of pseudowords for each polysemy degree can provide a reliable performance estimation on that polysemy degree.

we selected, for each polysemy degree, the top 300 pseudowords according to the calculated averageRank score (cf.


Whereas supervised WSD exploits a training set to perform sense classification, knowledge-based approaches use lexical knowledge bases instead.

a similar operation to that of providing an increasingly large training set is to enrich basic knowledge bases such as WordNet with additional semantic edges, as has previously been done, among others, by Navigli (2005), Cuadros and Rigau (2008), and Navigli and Lapata (2010).


For each action a, the learning agent learns θ values for each of these variables in the user model (θ a = θ a (1), θ a (2), .., θ a (n)).

for each referent, the agent learns two sets of θ values, one for each action.


The mean pre-task recognition score of the two groups were tested with Mann-Whitney U test for two independent samples and found to be not significantly different from each other (Jargon-adapt = 7.33, LDS = 7.45).


First, it makes no commitment to any kind of higher-level discourse structure over the discourse relations annotated between individual text spans.

while theory-neutral itself with respect to higher-level discourse structure, the PDTB invites experimentation with approaches to high-level topic and functional structuring (Stede 2012;Webber, Egg, and Kordoni 2012) or to hierarchical structuring (Mann and Thompson 1988;Asher and Lascarides 2003), as a resource for research aimed at a "data-driven and emergent theory of discourse structure" (Bunt, Prasad, & Joshi 2012, page 61).


Secondly, the annotation of discourse relations is lexically grounded.

rather than asking annotators to directly classify the sense of relations, which is a difficult task (Stede 2008), annotators were asked to look at lexical items that can signal discourse relations, such as the expression in Example (1).


Annotators were also asked to look at adjacent sentences that lacked one of these explicit signals.

where they inferred a discourse relation, they first labeled it with a lexical item that could serve as its explicit signal (such as in Example (2)), before going on to classify its sense.


For example, whereas 15 tokens of that means were noticed and annotated as AltLex, another 18 in the corpus were not examined, such as the one following and in Example (10).

10"I see a lot of evidence indicating a slower economy, and that means my interest-rate outlook has a downward tilt," said Garnett L. Keith Jr. AltLex expressions cannot be exploited in machine learning-for example, for inducing a model of discourse relation annotation-because no individual AltLex expression can be guaranteed to be fully annotated in the corpus.


Sentences may have complex grammatical structure with multiple embedded clauses.

the first step of our proposed system is to simplify the complex sentences with the intention of generating more accurate questions.


In most cases, these questions were outranked by other important questions, which included a combination of topics and subtopics to show higher topic relevance score measured by ESSK.

they do not have a considerable impact on the evaluation statistics.


Through these relations, the words can naturally form a word graph.

we use the graph-based learning approach to leverage the word distributions in sentiment lexicon learning.


Besides, in the bilingual word graph, the English sentiment seed words can continually provide accurate sentiment information.

we observe the increase in the approaches based on the bilingual word graph in term of both precision and recall (Table 2).


Besides, MAD cannot deal with the antonym relation.

these experiments demonstrate the overall superiority of our approach in cross-lingual sentiment lexicon learning.


Sometimes the word move may be forced to be aligned to in the parallel sentences good luck and best wishes on your career move and .

when building the inter-language relations with word alignment, our approach is likely to learn more sentiment word candidates.


The polarity score of the word can be adjusted by the intra-language relation.

though the inter-language relation brings in certain noisy alignments, the intra-language relation can help to refine the polarity score of the word using its intra-language relation.


A Tree Adjoining Grammar (TAG; Joshi and Schabes 1997) consists of a set of elementary trees and two combining operations, substitution and adjunction.

a TAG derivation can be described by a tree (called a derivation tree) specifying which elementary TAG trees were combined using which operations to yield that derivation.


In TAG, such restrictions can be modeled using selective adjoining constraints to specify which modifier tree may or may not be adjoined at a particular node in a given tree.

it is possible to license (1) and to rule out (2c).


Stochastic lexicalized TAG specifies the probability of an adjunction of a given auxiliary tree at a given node in another elementary tree (Resnik 1992;Schabes 1992).

under the standard notion of derivation, the overall probability of the string roasted red pepper would be determined by the probability of red adjoining to pepper and the probability of roasted adjoining to red.


β 1 , β i , and β n are auxiliary trees that all adjoin at the address g in the elementary tree τ. derivations specify the order in which adjunction takes place (e.g., β 2 adjoins to β 1 and the result to α).

if β 2 adjoins to β 1 , there is only one possible derived tree-namely, a tree where β 2 appears above β 1 .


Because completion operates bottom-up, it follows that Type 5 rules apply before Type 4 rules.

when adjoining an auxiliary tree β 1 to a node η o , the Type 5 completor rules, unifying the bottom feature structure of the foot node of β 1 with the bottom feature structure of the node η o , occurs before the Type 4 completor rules, which reassign the bottom reference variable of η o to the bottom feature structure of the root of β 1 .


b[..η]  η.P B := η r .P B In the Type 4 Completor rule, once a scopal auxiliary tree β with root node η r adjoins at some node η, the bottom component of the node η is marked with True, recording that a scopal adjunction has occurred at node η and that it therefore should not accept any further scopal adjunction.

for instance, the derivation of Syrian orthodox churches will proceed in a similar manner as the derivation of all the meerkats depicted in Appendix Figure A.2, but it will fail to produce the chart items (40, 42, ..., 52) associated with the independent adjunction.


Note that this modification does not block modifier adjunction above a predicative adjunction.

it successfully recognizes the sentence At what time did Brockway say Harrison arrived?, shown in Example (15), where a wh-modifier needs to be adjoined above a predicative adjunction.


It is heavily task dependent, in that a representation that is suitable for machine translation may not be appropriate for textual inference tasks, and one that is appropriate for IR may not be ideal for paraphrase detection.

this aspect of semantic model evaluation ideally should take the form of application-oriented testing.


The intuition behind this data set is similar, in that adjectives provide both additional information for disambiguation of the nouns they apply to, but also additional semantic noise.

a  Table 7 Example entries from the adjective-transitive data set without annotator score, third experiment.


In addition, the original features used in the normal classifier are also used in order to leverage the knowledge from the target annotation of the parallel annotated corpus, and the training procedure of the transfer classifier also learns the relative weights between the guiding features and the original features.

the knowledge from both the source annotation and the target annotation are automatically integrated together, and higher and more stable prediction accuracy can be achieved.


return argmax y∈GEN(x) S(y|M, Φ Φ Φ, x) 11: end function provided by the source classifier.

the performance of annotation transformation is correspondingly determined by the accuracy of the source classifier, and we can generate a more accurate parallel annotated corpus for better annotation adaptation if an improved source classifier can be obtained.


Essentially, annotation adaptation and annotation projection tackle different problems; the former aims to transform the annotations from one guideline to another (of course in the same language), whereas the latter aims to project the annotation (as well as the annotation guideline) from one language to another.

the machine learning methods for annotation adaptation pay attention to automatic transformation of annotations, while for annotation projection, the machine learning methods focus on the bilingual projection across languages.


down' in constructions like den Drink hinunterschüttete -'poured the drink down' cannot be handled by the operation sequence model because it generates the English words in strict left-to-right order.


The same result holds for (the composition-only fragment of) the formalism of Baldridge and Kruijff (2003).

we find that, at least with existing means, the weak equivalence result of Vijay-Shanker and Weir cannot be obtained for lexicalized CCG.


Prefix-closedness predicts different word orders for Swiss German subordinate clauses.

prefix-closedness predicts derivation (6), and along with it the alternative word order This word order is in fact grammatical (Shieber 1985, pp.


This means that the new rule instances can never be used in a complete derivation of G , and therefore do not change the generated language.

we have a construction that turns a TAG into a weakly equivalent prefix-closed VW-CCG.


Because L 1 is the only such sublanguage, this would give us proof that L 1 is context-free; but we know it is not.

we conclude that L 1 is not generated by a prefix-closed VW-CCG without target restrictions.


Illustration of the argument used in the proof of Lemma 11.

the arities of our chosen path are bounded by the maximum of three grammarspecific constants: the maximal arity of a lexical category, the arity of S (which is 0), and the maximal arity of a secondary input category.


Crucially though, by Lemma 11 we know that as long as we are interested only in transformed derivations it is sufficient to use a finite number of rule instances-more specifically those whose input and output categories are included in the set K of arity-bounded categories.

for every instance X/Y Yβ Xβ where all three categories are in K, we construct a production and similarly for backward rules.


In this case, the arguments of the secondary input category become inert.

the inertness status of an argument may change over the course of a derivation-but only from active to inert, not back again.


This projection path is not split; specifically, it starts with a composition that produces a category which acts as the primary input category of an application.

the derivation can be transformed (by our rewriting rule R3) in exactly the same way as instance (5) could be transformed into derivation (6).


Entailment rules between predicates are fundamental to many semantic-inference applications.

learning such rules has been an active field of research in recent years.


The first step of our algorithm takes advantage of graph sparsity: Most predicates in language do not entail one another.

it might be possible to decompose entailment graphs into small components and solve each component separately.


Assuming it does, then the violation is caused by omitting the edges in E cross .

there must be, without loss of generality, nodes i ∈ V 1 and k ∈ V 2 such that for some node j, (i, j) and ( j, k) are in E new , but (i, k) is not.


However, this means either (i, j) or ( j, k) is a crossing edge, which is impossible because we omitted all crossing edges.

e new is a better solution than e opt , contradiction.


The predicate entailment relation, described by our entailment graphs, is typically from a "semantically specific" predicate to a more "general" one.

intuitively, the topology of an entailment graph is expected to be "tree-like."


However, we hypothesize that this scenario is rather uncommon.

a natural variant of the Max-Trans-Graph problem is to restrict the required output graph of the optimization problem (3) to an FRG.


It can be verified that this is equivalent to the following statement: For every triplet of nodes i, j, k, if i ⇒ j and i ⇒ k, then either j ⇒ k or k ⇒ j (or both).

this constraint can be stated in a linear form: adding this new type of constraint to the ILP given in Equation 3results in a formulation for Max-trans-Forest: A natural question is whether there is a simpler (polynomial) solution for Max-trans-Forest that avoids the need for an ILP solver.


Moreover, only a direct child of c can choose v as a parent instead of c (Figure 4b), because for any other descendant of c, v would become a second parent, and G red will no longer be a directed forest (Figure 4b').

this case requires adding in G edges from v to all nodes in c and their ancestors; additionally, for each new child of v, denoted by d ∈ V red , we add edges from all nodes in d and their descendants to v. Crucially, although the number of possible subsets of c's children in G red is exponential, the fact that they are independent trees in G red allows us to go over them one by one, and decide for each one whether it will be a child of v or not, depending on whether S v-in (d) is positive.


Thus, this case requires adding in G edges from v to all nodes in c and their ancestors; additionally, for each new child of v, denoted by d ∈ V red , we add edges from all nodes in d and their descendants to v. Crucially, although the number of possible subsets of c's children in G red is exponential, the fact that they are independent trees in G red allows us to go over them one by one, and decide for each one whether it will be a child of v or not, depending on whether S v-in (d) is positive.

the score (9) in this case is: where child(c) are the children of c. Case 2b: Inserting v as a new root in G red .


However, if we allow re-attachment operations over components in the SCC graph, then we would be able to re-attach the strong connectivity component containing the nodes l, m, and n in a single operation.

the idea of our extended TNF algorithm is to allow re-attachment of both nodes and components.


component is 90% of the total number of graph nodes.

contrary to the typed graphs, untyped graphs that contain ambiguous predicates do not decompose well into small components.


Transitivity and FRG constraints limit the recall we are able to reach, and the graph does not decompose very well into components.

in future work we would like to apply our algorithms over graphs where predicate ambiguity is modeled and so transitivity constraints can be properly applied.


Note that the manually annotated gold-standard edges are only a subset of the full set of correct edges, which is actually much larger.

we cannot automatically identify violations, because the graph is missing many true edges.


Examples 2 and 3 are cases where the problem is not in predicate ambiguity but in the connected component itself.

out of five randomly sampled examples where TNCF deleted an edge that connects two weakly connected components, three can be attributed to predicate ambiguity.


We train the parser directly from examples of sentences annotated only with sentiment polarity labels but without any syntactic annotations or polarity annotations of constituents within sentences.


In the first iteration, the set of combination rules is empty.

we have no information about compositionality to improve dictionary rule learning.


Even though the book states that its focus is "certainly not on robustness and coverage" (p. 6), I am still missing more concrete ideas on how robustness and coverage might be addressed than "by using machine learning techniques" (p. 142) without further elaboration.

it is no coincidence that the implementation remains on the level of toy examples, however illustrating they might be.


, u m } and A can be represented by a matrix A = (a ij ), where a ij is the number of instances of unit u i in the sentence s j .

the j th column of A corresponds to sentence s j in A.


In this section, the results of the experiments described in Section 5 are provided and discussed.

the organization of this section and Section 5 are similar.


Table 6 represents the variation of the size of the solutions as a function of k. This variation is calculated as follows: For a given k number and a given algorithm, the standard deviation of the size of the k-covering computed by that algorithm is divided by the average size of these coverings.

it can be noted that LamSCP offers a stability 4 to 8 times superior to ASA concerning the size of the coverings.


Concerning the length of the selected sentences by both algorithms, it is greater than the one for the 5-covering of 2-phonemes, observed in Experiment 2, and slightly deviates from the average sentence length for the whole corpus.

it turns out that covering longer and generally rarer units involves a selection of longer sentences.


Other studies (Schauer and Hahn 2001;Stede 2004;Taboada 2006;Subba and Di-Eugenio 2009) report even higher figures: About 60% of discourse relations are not explicitly signaled.

rather than relying on hand-coded rules based on discourse cues and surface patterns, recent approaches use machine learning techniques with a large set of informative features.


For example, if repetition and synonymy are weighted equally, and hypernymy is given half as much weight as either of them, the score of bank's two senses are: 1 + 0.5 + 0.5 = 2 for the sense money bank and 1 + 0.5 = 1.5 for the sense river bank.

the selected sense for bank in this context is river bank.


a token near the beginning or end of a sentence is unlikely to be the end of a segment.


Notice that the automatic segmenter made two mistakes: (1) it broke the EDU marked 2-3 (Some people use the purchasers' index as a leading indicator) in the human annotation into two separate EDUs, and (2) it could not identify EDU 5 (But the thing it's supposed to measure) and EDU 6 (-manufacturing strength -) as two separate EDUs.

when we align the two annotations, we obtain seven EDUs in total.


Beside HILDA, we also compare our results with two baseline parsers on RST-DT: (1) CRF-O, which uses a single unified CRF-based parsing model shown in Figure 8 (the one used for multi-sentential parsing) without distinguishing between intra-and multisentential parsing, and (2) CRF-T, which uses two different CRF-based parsing models for intra-and multi-sentential parsing in the two-stage approach 1S-1S, both models having the same structure as in Figure 8.

cRF-T is a variation of TSP 1-1, where the DcRF-based (chain-structured) intra-sentential parsing model is replaced with a simpler cRF-based parsing model.


The results of our k-best document-level parsing suggest that often the best tree is missing in the top k parses.

a reranking of k-best document-level parses may not be a suitable option for further improvement at the document-level.


These results suggest that discourse structure helps to distinguish better translations from worse ones.

it would be interesting to explore whether discourse information can be used to rerank alternative MT hypotheses as a post-processing step for the MT output.


Because it does not feature chance correction, it should be used carefully for the reasons we have just seen.

most popular measures are chance-corrected: S (Bennett, Alpert, and Goldstein 1954) relies on a uniform distribution model of chance, π (Scott 1955) and α (Krippendorff 1980) on the mean distribution, and κ (Cohen 1960) on individual distributions.


First, if we observe the configuration given in Figure 9, annotators 2 and 3 have both annotated part of the NAMED ENTITY that has been annotated by annotator 1.

though they do not overlap, their units refer to the same phenomenon.


However, for γ, we follow Krippendorff, who claims that annotators should be interchangeable, because, as stressed by Krippendorff (2011) and Zwick (1988), Cohen's definition of expected agreement (using individual distributions) numerically rewards annotators for not agreeing on their use of values, that is to say when they have different prevalences of categories, and punishes those that do agree.

expected values of γ are computed on the basis of the average distribution of observed annotations of the several annotators.


The complete data tell more about the mean behavior of annotators, whereas data of a given continuum may depend more on the particularities of its content.

γ provides two ways to compute the expected values: one which considers only the data of the continuum being evaluated, as does every other coefficient; and a second one, which considers the data from all continua of the annotation effort the evaluated continuum comes from.


It is necessary here to choose once and for all what unit to compare to what other, rather than to perform all the comparisons.

but making such a choice precisely consists in making an alignment, which is a fundamental feature of γ. it seems that the alphas would need a structural modification to cope with overlapping.


An important point is that this shuffling error type, which is based only on moving positions, has a paradoxical consequence on category agreement, since units of different categories align when sufficient shifting is applied.

c|u α is not blocked at 1, even though it is designed to focus on categories.


The main reason why research on this topic is sparse, and why it may be difficult to enlarge Krippendorff's coefficients to overlapping units, probably results from the fact that we are facing here a major difficulty: the simultaneous double discrepancy between annotators, with annotations possibly differing both in positioning relevant units anywhere on a continuum, and in categorizing each of these free units.

it is difficult for a method to choose precisely which features to compare between different annotators (unlike pure categorizing, where we know exactly what each annotator says for each predefined element to be categorized); and this problem is exacerbated when overlapping units (within an annotator) occur.


2005, page 121), "a discourse frame is described as the grouping together of a number of propositions which are linked by the fact that they must be interpreted with reference to a specific criterion, realized in a frame-initial introducing expression."

temporal or spatial introducing expressions lead, for example, to temporal or spatial discourse frames in the text continuum (unitizing, sporadicity, categorization).


Although mathematical models of language have been studied in the computational linguistics milieu, via Markov models (Manning and Schuetze 1999), Zipf's law and its modifications (Mandelbrot 1953;Zipf 1965;Bell, Cleary, and Witten 1990), and Pitman-Yor models (Teh 2006) more recently, the true mathematical model of linguistic processes is ultimately unknown.

the convergence of a constancy measure must be examined through empirical verification.


Still, these points are located lower than those for news text.

h 2 has the potential to distinguish genre or maybe writing style.


On the other hand, when considering the different character parts as distinct (RongoB), the location drifts towards the plot for Thai, a phonographic script, in terms of characters.

the Rongorongo script could be considered basically morphemic, with some parts functioning phonographically.


On the other hand, in order for the gold-standard goal edge to be constructed, all gold-standard edges that have been expanded must remain in the chart.

our training algorithm triggers a parameter update whenever a gold-standard edge is removed from the chart, the scores of all chart edges are updated, and the original pruned gold edge is returned to the chart.


The only difference is that, for tree linearization, full unordered dependency trees are given as input.

the output word permutations are more constrained (under the projectivity assumption), and more information is available for search disambiguation.


1995;Ehrlich, Obler, and Clark 1997;Vuorinen, Laine, and Rinne 2000;Creamer and Schmitter-Edgecombe 2010).

it is often not the quantity of correctly recalled information but the quality of that information that reveals the most about a person's diagnostic status.


Note that in the WLM, every word in the source narrative is part of one of the story elements.

when we convert alignments to scores in the way just described, any alignment can generate a story element.


As discussed in Section 5.2, our WLM score extraction method is designed such that element-level scores can be extracted with perfect accuracy from a perfect word alignment.

the goal of seeking perfect or near-perfect word alignment accuracy is worthwhile because it will necessarily result in perfect or near-perfect scoring accuracy, which in turn is likely to yield classification accuracy approaching that of manually assigned scores.


The existence of this association allows us to transfer knowledge and inferences from the domain of mechanisms to that of political systems.

we reason about political systems in terms of mechanisms and discuss them using the mechanism terminology, giving rise to a variety of metaphorical expressions.


The high frequency of metaphorical language in textual data makes accurate metaphor processing desired for a number of NLP applications.

the format of metaphor understanding that metaphor processing systems provide should ideally be informed by the requirements of external NLP and a number of considerations arise in that respect: r Metaphor processing typically involves two tasks: metaphor identification (distinguishing between literal and metaphorical language in text) and metaphor interpretation (identifying the intended meaning of the metaphor).


r To enable wide applicability, the system needs to be open-domain-that is, operate in all domains, genres, and topics.

ideally it should not rely on any domain-specific information or focus on individual types of instances (e.g., a limited set of hand-chosen source-target domain mappings).


The authors evaluated their system on a set of example sentences compiled from the MML, whereby highly conventionalized metaphors (or dead metaphors) are taken to be negative examples, reporting an accuracy of 0.58.

krishnakumaran and Zhu do not deal with literal examples as such: Essentially, the distinction they are making is between the senses included in WordNet, even if they are conventional metaphors, and those not included in WordNet.


For instance, whereas common concepts (that are well represented in the data) can be clustered with a high accuracy, this is not always the case for rare concepts for which feature vectors are sparse.

an additional technique is needed to map new, unseen concepts to the concepts present in the graph.


Metaphor makes our thoughts move vivid and enriches our communication with novel imagery, but most importantly it plays a fundamental structural role in our cognition, helping us organize and project knowledge.

its manifestations are pervasive in language and reasoning, making its computational processing an imperative task within NLP and intelligent systems engineering at large.


Our experiments reveal that previous models of coherence are not able to predict this impact.

we develop a novel coherence model, which learns to accurately predict argument realization based on automatically aligned pairs of implicit and explicit arguments.


We apply the outlined identification and linking approach to all text pairs in our corpus of comparable texts.

we induce a total of 698 implicit argument and antecedent pairs.


As discussed in the paper that introduced the entity grid model (Barzilay and Lapata 2008), most entities are, however, only mentioned a few times in a text.

non-realizations constitute the "most frequent" class, independently of whether they are relevant in a given context or not.


We show that the difficulty of estimating similarity is driven primarily by those strongly associated pairs with a high (association) rating in gold standards such as WS-353 and MEN, but a low similarity rating in SimLex-999.

of including these challenging cases, together with a wider diversity of lexical concepts in general, current models achieve notably lower scores on SimLex-999 than on existing gold standard evaluations, and well below the SimLex-999 inter-human agreement ceiling.


The ratings in WS-Sim are mapped directly from WS-353, so that all concept pairs in WS-Sim that receive a high rating are associated and all pairs that receive a low rating are unassociated.

any model that simply reflects association would score highly on WS-Sim, irrespective of how well it captures similarity.


Even if a formal characterization of similarity existed, the evidence in Section 2 suggests that the instructions would need separate cases to cover different concept types, increasing the difficulty of the rating task.

we preferred to appeal to intuition on similarity, and to verify post hoc that subjects were able to interpret and apply the informal characterization consistently for each concept type.


The pair tokens {(w n , w n+j ) : −t(n) ≤ j ≤ t(n), w i ∈ s} are then appended to the training data.

target/context training pairs are such that (i) only words within a k-window of the target are selected as context words for that target, and (ii) words closer to the target are more likely to be selected than those further away.


However, although words that occur together in text often have associated meanings, these meanings may be very similar or indeed very different.

possibly excepting the population of Argentina, most people would agree that, strictly speaking, Maradona is not synonymous with football (despite their high rating of 8.62 in WordSim-353).


There was no well-accepted annotation or transformed standard.

we referred to a large number of linguistic studies, developed a Chinese syntactic dependency annotation standard, and annotated a 50,000-sentence Chinese syntactic dependency treebank on this basis.


Based on syntactic parsing, we hoped to further explore the semantic structure and the relationship of sentences.

we carried out research on semantic role labeling, and worked on the semantic role labeling methods based on the tree kernel, including the hybrid convolution tree kernel (Che et al.


I hoped that my team and I wouldn't become the "silly bears."

when we decided to develop an Internet MT system, we all agreed on the idea that we needed a hybrid approach, with which we could integrate all translation models and subsystems, on each of which we have all spent great effort.


These incidents told us that solving translation problems for these resource-poor languages is urgent.

we have successively released translation services between Chinese and over 20 foreign languages.


In particular, if we assume that MT is modeled according to the log-linear model we can define softmax loss softmax (•) as follows: From Equation 21we can see that only the oracle translations contribute to the numerator, and all candidates in c (i) contributes to the denominator.

intuitively, the softmax objective prefers parameter settings that assign high scores to the oracle translations, and lower scores to any other members of c (i) that are not oracles.


In the case of MT, as there are usually multiple translations e that minimize error(•), and multiple derivations d that result in the same translation e, o (i) will generally contain multiple members.

we cannot be entirely certain that we will reach a global optimum.


However, as mentioned in Section 2.5, evaluation of MT is an active research field, and there are many alternatives in addition to BLEU.

it is of interest whether changing the measure used in optimization can affect the overall quality of the translations achieved, as measured by human evaluators.


w l is a parser with a scoring function s and a dependency d = (i, r, j), such that i is the position of the governor, j the position of the dependent, and r the dependency label, with 1 ≤ i, j ≤ l. We can define a new scoring function s + d in the following way: When running the parser on sentence S, with the scoring function s + d , one can be sure that dependency d will be part of the solution produced by the parser because the score of all its competing dependencies (i.e., dependencies of the form (i , r , j ) with different governors (i = i) or labels (r = r) and the same dependent (j = j)) are set to −∞.

the overall score of trees that have such competing dependencies cannot be higher than the trees that have our desired dependency.


In addition, whereas Hammarström and Borin focus entirely on unsupervised learning, our work considers a broader range of learning paradigms.

although related, Hammarström and Borin and our current presentation are complementary in that they have different focus areas.


Evidently, the level of detail in the full analysis is substantially higher compared with the segmentation, as it contains lemmatization as well as morphological tagging, whereas the segmentation consists of only segment boundary positions.

because of this simplification, morphological segmentation has been amenable to unsupervised machine learning methodology, beginning with the work of Harris (1955).


In addition, the minimally supervised learning setting explicitly assumes a small amount of available annotated word forms.

in the remainder of this article, all discussion on unsupervised methods refers to unsupervised learning in the non-strict sense.


The LSV method utilizes the insight that the predictability of successive letters should be high within morph segments, and low at the boundaries.

a high variety of letters following a prefix indicates a high probability of a boundary.


Lastly, in their work with the supervised enumeration approach, Eger (2013) assumes a large (on the order of tens of thousands) amount of annotated word forms available for learning.

it is left for future work to determine if the approach could be applied successfully in the minimally supervised learning setting.


This complicates model development because if two model variants perform differently empirically, it is uncertain whether it is because of a truly better model or merely better fit with the utilized parameter estimation method, as discussed also by Goldwater (2006, Section 4.2.2.3).

in contrast, within the adaptor grammar framework (Johnson, Griffiths, and Goldwater 2006;Sirts and Goldwater 2013), the focus has not been on finding a single best model, but rather on finding the posterior distribution over segmentations of the words.


The over-segmentation errors occur when a segment that should remain intact is split.

these errors can be assigned into categories c according to the morph tags PREFIX, STEM, SUFFIX, and UNKNOWN.


However, it was found by Virpioja, Kohonen, and Lagus (2011) that performance does not improve consitently with growing data because the method segments less on average for each added training word form.

we optimize the training data size by including only the most frequent words in the following sizes: 10k, 20k, 30k, 40k, 50k, 100k, 200k, 400k, .


Although optimizing it for segmentation performance during training is nontrivial, one can instead apply the heuristic that the method should neither over-segment nor under-segment.

the implementation adjusts α such that the development set precision and recall become approximately equal.


These unsupervised and supervised learning results utilize the available data only partially.

we next discuss results obtained using semi-supervised learning-that is, when utilizing all available annotated and unannotated word forms.


2003;Cozman and Cohen 2006;Fox-Roberts and Rosten 2014).

it would appear to be advantageous to study weighting schemes in combination with all the discussed generative models.


(2013), unseen events constitute a major cause of translation errors when migrating an existing SMT system to a new domain.

it is interesting to restrict the calculation of the repetition rate to those n-grams that are not contained in the out-of-domain corpus.


On the other hand, it is also worth mentioning that the sufficient statistics for a given sentence pair are nonzero for a small fraction of its components.

the time required to update the parameters of the HMM-based alignment model depends only on the number of nonzero components.


Moreover, a significant drop in the RRR measure with respect to the MRR is observed, indicating that the vast majority of the repeated n-grams in the test corpus has already been seen in the training corpus.

we expect a limited effectiveness of online learning for this task.


Assuming a one-to-one correspondence between source and target phrases, reordering in PSMT means searching through a set of permutations of the source phrases.

two subproblems arise: defining the set of permutations in b allowed during decoding (reordering constraints) and scoring the allowed permutations (reordering models or feature functions).


Given a source sentence, the search for its optimal reordering is generally decomposed into a sequence of local reordering decisions, as is done for the whole translation process.

the basic reordering step corresponds to the relative positioning of the word or phrase being translated, with respect to the word or phrase that was previously translated.


Tuples, however, are typically shorter than phrase pairs and are extracted from a unique, monotonic segmentation of the sentence pair.

the problem of spurious phrase segmentation is avoided but non-local reordering becomes an issue.


This means that, for instance, the reordering pattern learned by our example HSMT grammar ( Figure 5, rule 1) may only be used to reorder the specific verb form jdd (renewed) in subsequent test sentences.

hSMT is likely to work better for languages where the syntactic role of phrases is mostly expressed by separate function words (e.g., Chinese) than for languages where this information is largely conveyed by word inflection (e.g., Russian).


In particular, we find it essential for SMT to distinguish between clause-level and phrase-level differences (CDiff and PDiff) because the former account for most longer-range word movements, and the latter for the shorter.

a language pair with only phrase-level discordant features is likely to be suitable for a PSMT approach, where reordering is managed through local distortion or inside translation units.


17 At the phrase level, differences are found in the location of genitives and adjectives.

english and French have very similar clause-level orders, but reordering is abundant at the local level.


As a result, reordering is overwhelmingly local but few crucial long-range reorderings also regularly occur.

this pair is challenging for PSMT but, at the same time, not well suited for a tree-based approach.


For technical reasons, grid X 1 needs a special treatment: the last edge (i 2m , i 2m+1 ) in γ is used to enter grid X 1 for the first time, and is therefore placed at vertex x 11,1 .

all other edges of γ that are placed at X 1 need to be shifted by one position.


When we move to arc (1, 2) of T UL , we lose the two interconnection edges impinging on vertex g 1,1 of G i , but those edges are replaced by two new internal edges of G i , namely, edges (g 1,2 ).

arc (1, 2) of T UL still routes 4k edges of G i .


We observe that the third segment of an (i, j)-path is the reversal of the first segment of a ( j, i)-path.

we can use our argument for the first segments to show the existence of 4d i edge disjoint third segments ending at a vertex in the subtree below a j .


In parsing problems, the variables consist of indices into the string being parsed, and the constraints require that a nonterminal on the right-hand side of a rule has previously been identified.

the graph has a vertex for each endpoint involved in the rule, and a clique for each nonterminal (including the left-hand side nonterminal) connecting all its endpoints.


Hence, the path between any two occurrences of i in T G contains nodes that also contain i, guaranteeing the running intersection property of T G .

by constructing D from G, finding an optimal tree decomposition of D, and then translating the result into a tree decomposition of G, we have a tree decomposition of G having width SOL where SOL ≤ 2tw(D).


Most of the world languages are resource-poor for statistical machine translation; still, many of them are actually related to some resource-rich language.

we propose three novel, language-independent approaches to source language adaptation for resource-poor statistical machine translation.


This causes further mismatch in domain and genre.

translating from Arabic dialects to Modern Standard Arabic requires, among other things, normalizing informal text to a formal form.


There are also many partial cognates, for example, nanti means both will (future tense marker) and later in Malay but only later in Indonesian.

fluent Malay and fluent Indonesian can differ substantially.


Unfortunately, we do not have parallel Malay-Indonesian text, which complicates the process of learning when to apply these operations.

in the following we focus our attention on the simplest and most common operation of word/phrase substitution only, leaving the other two operations for future work.


Finally, we truncate these alignments keeping them for one copy of the original Indonesian-English bitext only.

we end up with improved word alignments for the Indonesian-English bitext, and ultimately with better estimations for Equation (1).


we also try to model context more directly by generating adaptation options at the phrase level.


This may be because the two mappings are extracted from the word alignments of the same Malay-English and Indonesian-English bitexts by pivoting.

we can expect that the phrase-level mapping already contains most, if not all, of the word-level mapping.


However, doing so also risks proposing worse alternatives, for example, changing beliau ('he', respectful) to ia ('he', casual), which the weights on the confusion network edges and the language model would not always handle properly.

we tried paraphrasing non-Indonesian words only, that is, those not in IN-LM.


Table 7 shows that CN:word,t=0 is better/equal to the original 53%/31% of the time.

it is a very good step in the direction of turning Malay into Indonesian.


Crowdsourcing methods must also address the need for quality control.

in order to find good translations, subsequent post-editing and/or ranking is generally necessary.


#ohayo, the characters mean weekend in both Chinese and Japanese.

if these are the only characters that are aligned during the computation of the translation score, both languages are equally likely to be correct.


This is done by showing that, under Model 1, Viterbi alignments do not need to be recomputed every time the segment spans are changed, and can be obtained by updating previously computed ones.

we propose an iterative approach to compute the Viterbi word alignments for IBM Model 1 using dynamic programming.


All words in the right span must be rechecked, because aligning them to the inserted token at q + 1 may yield a better translation score.

this update requires ) removes the first token of the left span with index p. After removing the token, new alignments must be found for all tokens that were aligned to p. the number of operations for this update is K × (q − p + 1), where K is the number of words that were originally aligned to p. In the best case, no words are aligned to the token in p, so nothing needs to be done.


This would raise the complexity of this operation to O(n).

with the new set of update functions, the optimal order of updates would require O(n 5 ) operations.


For instance, if there are no Han characters, any pair involving Japanese or Chinese can be excluded.

one can efficiently filter out unlikely language pairs without losses.


However, this operation is only expensive if we compute the translation score, as the span and language scores can be more efficiently computed.

we define the incomplete IDA score as: We can trivially show that , as the incomplete score does not include the product of the translation score, in the [0, 1] interval.


If it is not, we can skip the computation of the translation score for this language pair, as it will not be higher than 0.4.

if we find that the highest S inc score among all possible segmentations of p, q, u, v score for a given language pair l 1 , r 1 is lower than score for any other language pair, it follows that l 1 , r 1 will never be the highest scoring language pair.


The fact that we are applying the IDA model to tweets that may not contain translations means that many instances in D s,t will not be parallel.

as the last step, we filter these instances, using a classifier that is trained to predict whether each sample in D s,t is actually parallel.


However, this approach is only effective on finding multilingual tweets with languages that do not share the same unicode range.

to allow the extraction of parallel data for language pairs such as English-Spanish, a more general method is needed.


The probability of a pair of tokens a and b and a pair of respective languages a and b is given by the product of the probabilities Thus, the sum of the probabilities of all pairs of language pairs is equal to 1, and we can efficiently compute the probability that the pair of tokens are in different languages as: where P L ( | x) is once again the probability that token x is in language , according to a character-based model, and L is the set of all languages considered.

given a tweet, our model attempts to find a pair of words where P( a = b | a, b) is higher than 0.95.


Furthermore, we must also consider errors from misalignments of the IDA model and misclassifications of the multilingual message detector.

in order to identify messages that are actually parallel, a final identification step is necessary.


r User features -Although the background of the users that parallel post cannot be determined with absolute certainty, it is safe to assume that they are either bilingual speakers and translate their own messages, or hire translators to translate their posts.

users that do not belong to these categories rarely post parallel messages, since they do not have the means, and likewise, users that are in these categories are likely to post a considerable amount of parallel posts.


These messages tend to be more valuable, however, as they contain artifacts that our translation models cannot translate.

it is desirable to consider the aggregate scores of the user posts as additional information for the classification problem.


To provide an interpretable view of the contents of the parallel data that was crawled, we look at the distribution over topics of the parallel data set inferred using latent Dirichlet allocation (LDA) (Blei, Ng, and Jordan 2003).

we grouped the Weibo filtered tweets by user, and ran LDA over the predicted English segments, using 12 topics.


To increase system coverage, the ARC transitions designed by the THMM as well as our systems do not change the nodes in the stack nor buffer in a configuration: Only the nodes linked to the top of the stack or buffer are modified.

features derived from the configurations before and after an ARC transition are not distinct enough to train a good classifier.


It can be seen that a swap sequence is always followed by a shift sequence, the length of which is no less than the swap sequence, and if the two sequences are of the same length, the next transition cannot be sw. Let #(t) to be the number of transition types t in the sequence, then #(la), #(ra), #(pop), and #(sh) − #(sw) are all finite.

the number of swap sequence is finite, indicating that the transition sequence is finite.


A key observation of a linguistically motivated bilexical graph is that there is usually at most one edge between any two words, therefore an ARC transition is not followed by another ARC.

any ARC with its successive transition modifies a configuration much.


Moreover, the decoding algorithms for linear-chain Markov models are very fast.

the syntax-free hybrid system is more appealing for many NLP applications.


On the other hand, the beam decoder for the discriminative model achieves equivalent tagging accuracies to the Viterbi decoder.

the efficiency of both training and testing can be guaranteed.


As such, we are able to support the findings of Rajasekaran and Yooseph (1998), who showed that tree-adjoining grammar (TAG) recognition can be done in time O(M(n 2 )) = O(n 4.76 ) (TAG can be reduced to LCFRS with d = 2).

combinatory categorial grammars, head grammars, and linear indexed grammars can be recognized in time O(M(n 2 )).


Each cell in the matrix keeps track of the nonterminals that can dominate these spans.

computing the transitive closure of this matrix yields in each matrix cell the set of nonterminals that can dominate the assembled indices' spans for the specific string at hand.


The matrix rows and columns are indexed by the set N(d) defined as: where n denotes the length of the sentence, and the exponent d denotes a repeated Cartesian product.

each element of N(d) is a sequence of indices into the string, where each index is annotated with a bit (an element of the set {0, 1}) indicating whether it is marked or unmarked.


Our normal form for LCFRS rules ensures that the leftmost endpoint of B forms the leftmost endpoint of A.

the leftmost endpoint of B is in B's row address, and B is above the diagonal.


In matrix multiplication, only cells that share a row or a column index actually interact when doing multiplication.

in order to copy a nonterminal from (i, j) to another cell which represents the same set of spans, we have to copy it through cells such as (i, insert(j,x)) that share the index i with (i, j).


The condition min i = min k implies that we are not moving this leftmost index from row to column.

the addresses of the three copy symbols required are all formed by adding or removing x orx to the row and column addresses (i, j); because the leftmost index of i is not modified, the copy symbols that are required are all above the diagonal, and are present in the seed matrix t. A appears in cell (k, ) of t (n+3) .


2010), there is still no consensus on which features are actually the best predictors of readability.

when institutions, companies, or other research disciplines wish to use readability prediction techniques, they still rely on the more outdated superficial characteristics and formulas (see, for example, the recent work by van Boom [2014] on the readability of mortgage terms).


The authors explain their result through the following hypothesis: "the choice of words people make when writing in a second language is strongly influenced by the phonology of their native language."

it seems that character bigrams may capture some phonological preferences.


A variant of this kernel can be obtained if the embedding feature map is modified to associate a vector of dimension |Σ| p containing the presence bits (instead of frequencies) of all its substrings of length p with each string.

the character p-grams presence bits kernel is obtained: where in v (s) is 1 if string v occurs as a substring in s, and 0 otherwise.


The p-spectrum kernel assigns a high score even when the p-gram has a high frequency in only one of the two strings.

the intersection kernel captures something more about the correlation between the p-gram frequencies in the two strings.


From this perspective, the p-gram frequencies could actually contain more noise than useful information.

using the p-grams presence bits kernel could be more suitable for the NLI task.


Even if tokenization is fairly straightforward in languages that use spaces to separate words, there are many edge cases including contractions and hyphenated words that require linguistic knowledge.

analyzing the results of the string kernels approach when tokenization is not being used at all will reveal if the linguistic knowledge helps to improve the performance or not.


For instance, "don't" is divided into two tokens, namely, "do" and "n't".

4 the range of p-grams used for computing the blended string kernels was adjusted based on the development set for the raw text documents.


The accuracy obtained by KRR andk 0/1 5−8 on raw text was 82.6%, which is still better than the accuracy of the very same system when tokenization was included (82.0%).

we can conclude that both hypotheses are responsible for the improved results.


However, a KDA classifier can successfully be trained on 75, 000 samples on the same machine.

the results of KDA presented in this section are obtained by training it on a random sample of 75, 000 samples selected during each fold.


Again, the kernel based on LRD was not included in this experiment because it is more computationally expensive.

only the presence bits kernel and the intersection kernel were evaluated on the TOEFL11-Big corpus.


Furthermore, the LOO procedure can easily be performed on a small data set, such as the subset of 329 samples of the ALC.

the LOO procedure is more suitable for this NLI experiment, because it is straightforward to compare newly developed systems with the previous state-of-theart systems.


Although AMRs are usually visualized as tree structures, they can also be viewed as directed acyclic graphs with a unique root (with vertices denoting variables and edges denoting roles and instances).

aMRs can be transformed into sets of triples, which is convenient for evaluation purposes (Cai and Knight 2013).


Nevertheless, in specific circumstances, females may choose to de-emphasize their identity as females by modulating their language usage to sound more male.

although this exception serves to highlight rather than challenge the commonly accepted symbolic association between gender and language, it nevertheless means that it is less feasible to predict how a woman will sound in a randomly selected context.


Findings are ratified and then accumulated.

theories become storehouses for knowledge obtained through empirical methods.


Thousands of players guessed the age and gender of Twitter users based on their tweets, and the results revealed that many Twitter users do not tweet in a gender-stereotypical way.

language is inherently social and while certain language features are on average used more by men or women, individual speakers may diverge from the stereotypical images that tend to be highlighted by many studies.


Besides linguistic differences, boundaries between language varieties are often influenced by other factors such as political boundaries (Chambers and Trudgill 1998).

deciding on the appropriate labels to describe linguistic communication across different groups of speakers (in terms of language, dialect, minority language, regional variety, etc.)


To become a core member of a community, members adopt such community norms.

often a change in behavior can be observed when someone joins a community.


The rest of this section presents a discussion of data sources for studying multilingual communication on a large scale (Section 5.1).

we discuss research on adapting various NLP tools to process mixed-language texts (Section 5.2).


Most of the current NLP tools, such as parsers, are developed for texts written in a single language.

such tools are not optimized for processing texts containing multiple languages.


The two topics are inter-related because sentiment lexicons are often used to express opinions on aspects or targets.

many researchers have studied the two tasks together.


My own research showed that language users can match the probabilities of linguistic features of the environment and they have powerful predictive capabilities that enable them to anticipate the variable linguistic choices of others.

the working hypothesis I have adopted contrasts strongly with that of the garden: Grammar itself is inherently variable and probabilistic in nature, rather than categorical and algebraic.


: A dog was barking, No dog was barking).

corpus-induced vectors of predicates and their negations are very similar.


even propose a "distributional hypothesis of highly contrasting pairs" stating that highly contrasting pairs occur in similar contexts more often than non-contrasting word pairs).

it is impossible to distinguish them from non-contrasting related words (e.g., synonyms) using standard distributional similarity measures, and ad hoc strategies must be devised.


However, all studies we are aware of base their claims on a small number of hand-picked examples of felicitous or implausible alternatives.

no model has been proposed that, given a negated element and an arbitrary predicate, makes explicit predictions about how plausible it is for the predicate to fall into the alternative set of the negated element.


Indeed, these are better cohyponym pairs than many of those we harvested through WordNet cohyponym links, that were often too distantly related (bottle/bath under the vessel category) or based on unusual senses of words (goat/mug under the victim category).

cohyponyms received, on average, lower ratings compared with hyponyms (again, the THERE context, being more tolerant towards distant relations, affords higher cohyponym ratings).


It seems safe to conclude, then, that the subtle pragmatic and semantic implications carried by the two contexts affected subjects' ratings only very marginally, and we are getting robust alternativehood intuitions across contexts.

we also expect that the same similarity measure might be able to approximate the ratings in either context reasonably well.


In many cases the SVO triple represented a property which, although it could be true of the term, did not distinguish it from other terms, especially from other hyponyms of the same head noun.

player that wins tournament is not a good property for golfer: many golfers do win tournaments, but so do other sports players.


This is not quite the sort of defeasible justifications discussed in Section 2.1, but it is a place to start and it contains information pertinent to justification.

individual word types will be modeled as vectors in a finite dimensional space V, whose dimensions reflect aspects of the context of use.


The closer the distances, the better or more prototypical the predication.

tCL's binary view of semantic well-formedness would morph into a more graduated scale, which might more accurately reflect the intuitions of ordinary speakers (Magidor 2013).


For example, when the APT shown at the top of Figure 3 is applied to the co-occurrence type NSUBJ, we reach the node at which the lexemes we/PRP and he/PRP appear.

this APT can be seen as a characterization of the distributional properties of the verbs that nouns that dry/JJ modifies can take as their direct object.


However, shifting PPMI (k = 10) gave optimal results across all of the word similarity tasks.

we report results here for vanilla PPMI (shift k = 1) and shifted PPMI (shift k = 10).


For similarity measures, one straightforward option would be to create a vector from the APT anchored at the head of the phrase or sentence being considered.

the phrasal vector for a red rose would be created taking the node containing rose as the anchor.


Also, current distributional representations do not support logical inference that captures the semantics of negation, logical connectives, and quantifiers.

distributional models and logical representations of natural language meaning are complementary in their strengths, as has frequently been remarked (Coecke, Sadrzadeh, and Clark 2011;Garrette, Erk, and Mooney 2011;Grefenstette and Sadrzadeh 2011; Baroni, Bernardi, and Zamparelli 2014).


Our aim has been to construct a general-purpose natural language understanding system that provides in-depth representations of sentence meaning amenable to automated inference, but that also allows for flexible and graded inferences involving word meaning.

our approach combines logical and distributional methods.


Uncertainty and gradedness at the lexical and phrasal level should inform inference at all levels, so we rely on probabilistic inference to integrate logical and distributional semantics.

our system has three main components, all of which present interesting challenges.


They are motivated by the observation that semantically similar words occur in similar contexts, so words can be represented as vectors in high dimensional spaces generated from the contexts in which they occur (Lund and Burgess 1996;Landauer and Dumais 1997).

distributional models are relatively easier to build than logical representations, automatically acquire knowledge from "big data," and capture the graded nature of linguistic meaning, but they do not adequately capture logical structure (Grefenstette 2013).


So H should follow from T 2 : All ogres are green but not from T 1 : There is a green ogre.

we introduce a new constant D for the query and assert ogre(D) to test if we can then conclude that green(D).


This parser can be configured to produce multiple ranked parses (Ng and Curran 2012); however, we found that the top parses we get from C&C are usually not diverse enough and map to the same logical form.

in addition to the top C&C parse, we use the top parse from another recent CCG parser, EasyCCG (Lewis and Steedman 2014).


Therefore, in addition to the top C&C parse, we use the top parse from another recent CCG parser, EasyCCG (Lewis and Steedman 2014).

for a natural language text N T and query N H , we obtain two parses each, say S T1 and S T2 for T and S H1 and S H2 for H, which are transformed to logical forms T 1 , T 2 , H 1 , H 2 .


Unfortunately the textual entailment data that we use in this article does not lend itself to contextualization-polysemy just does not play a large role in any of the existing RTE data sets that we have used so far.


In MLNs, the probability of a formula is the sum of the probabilities of the possible worlds that satisfy it.

gogate and Domingos (2011) show that to calculate the probability of a formula H given a probabilistic knowledge base KB, it is enough to compute the partition function Z of KB with and without H added as a hard formula: all we need is an appropriate algorithm to estimate the partition function Z of a Markov network.


In computational linguistics and in particular in natural language parsing, however, there is less of an established tradition of using general graphs than in, say, theoretical computer science (although the central role of feature structures in unification-based grammar formalisms arguably marks an exception to this claim).

we note a lack of consensus on which specific structural properties of graphs are most relevant in terms of linguistic adequacy or formal effects on models and algorithms.


Nodes in EDS are independent of surface lexical units, but for each node there is an explicit, manyto-one mapping onto sub-strings of the underlying linguistic signal.

we include EDS as a middle ground between the node-ordered lexicalized dependency graphs of CCD and SDP and the unordered AMR graphs, which provide no overt links to the surface signal.


Although it appears that this subset of graphs presents structurally mildly less complex and shorter (at an average length of 22 tokens) inputs, we find all general tendencies from Section 3 and relative ordering among representations confirmed.

we conjecture that these general contrasts primarily reflect contentful linguistic differences.


Often multiple paragraph descriptions of a study are given but without enough details, equations, or technical explanations to actually understand the approach.

not much assimilation of the topic is possible at the end of many sections of the book.


On the other hand, many syntactic and linear ordering choices are regulated by soft constraints-that is, yield text of variable acceptability.

although both sentences in Example (3) are grammatical, Example (3a) is arguably better English than Example (3b).


A total of 12,000 relation identifiers were extracted from the 200 ontologies, and 13 syntactic templates were found to be sufficient to verbalize these relation identifiers (see Trevisan [2010] for more details on this evaluation).

in general, the lexicon extraction method proposed by Trevisan (2010) provides a generic procedure for automatically lexicalizing ontological data.


A KB query identifies a set of objects by specifying properties (concepts) of these objects and of other objects these objects are related to.

verbalizations of KB queries are in effect descriptions of objects or sets of objects that involve chaining unary and binary relations to describe the set of objects the user wants to identify.


Within the 1-best sequence configurations, the increased lexical ambiguity comes from the FLS backoff; however, in the 12-best sequence configuration more ambiguity comes from the syntactic labels at each sequence and less from the FLS mechanism.

at n = 1 coverage and time are better for syntactic classes, but when looking at n = 12 for trees we can see slightly better results.


The W prefix indicates a wh-NP extraction.

for instance, W0nx0VVVnx1 indicates a subject relative tree (W0nx0) with three verbs as anchor (VVV) and an NP as complement (nx1).


2 Because of the high frequency and systematicity with which metaphor is used in language, it is naturally and systematically reflected in the distributional space.

of metaphorical cross-domain mappings, the words' context vectors tend to be non-homogeneous in structure and to contain vocabulary from different domains.


This demonstrates how metaphorical usages, abundant in the data, structure the distributional space.

the context vectors of different concepts contain a certain degree of crossdomain overlap, thus implicitly encoding cross-domain mappings.


The results that came out of these studies suggest that in order to reliably capture the patterns of the use of metaphor in the data on a large scale, one needs to address conceptual properties of metaphor, along with the surface ones.

the model would need to make generalizations at the level of metaphorical mappings and coarse-grained classes of concepts, in essence representing different domains (such as politics or machines).


We can achieve some clarity about this objective function by rewriting it using linear algebra.

if we define the normalized indicator vectors h k := (h 1k , ..., h Nk ) T where we set then some straightforward computations reveal that: if we collect the vectors h 1 , ..., h K into a matrix H = (h 1 , ..., h K ), then h T k Lh k = (H T LH) kk , and minimizing Equation (3) is equivalent to the following minimization problem on the graph Laplacian: if we could find the optimal H, it would be straightforward to find the cluster memberships from H, since h ik is nonzero if and only if v i is in cluster A k .


(2002) proceed by dropping the constraint that h ik be either 0 or 1/ v ∈A k d , but keeping the orthonormality constraint.

they seek to solve the following problem: By setting T := D 1/2 H, this can be rewritten as This problem is tractable because it is equivalent to the problem of finding the first K eigenvectors of D −1/2 LD −1/2 .


According to our qualitative analysis, the TT constrained clusters exhibited a higher diversity with respect to the target domains they contained in all languages, leading to the system capturing a higher number of new metaphorical patterns, as compared to the unconstrained clusters.

it discovered a more diverse set of metaphorical expressions given the same seeds.


For instance, in the case of the constraint DIFFICULTY & BARRIER, these two nouns are clustered together.

given the seed "confront problem," the system falsely tags expressions such as approach barrier or face barrier as metaphorical.


The matrix B denotes the n × m adjacency matrix, with b ip being the connection weight between the vertex v i and the cluster u p .

b represents the connections between clusters at an upper and lower level of clustering.


However, the process of manually identifying instances is time-consuming and expensive, limiting the size of corpora and the scope of metaphors that can be analyzed in a given time frame.

it can be difficult to draw broad conclusions from these studies.


Authors do not state the degree of cogency (the probability of their claim, as proposed by Toulmin).

we omitted qualifier from the model because of its absence in the data.


In reality, the authors often attack the presented rebuttals by another counter-rebuttal in order to keep the whole argument's position consistent.

we introduced a new component-refutation-which is used for attacking the rebuttal.


We observed that this standpoint is not always explicitly expressed, but remains implicit and must be inferred by the reader.


However, we observed in our data that some authors elaborate on both sides of the controversy equally and put forward an argument for each side (by argument here we mean the claim and its premises, backings, etc.).

we allow multiple arguments to be annotated in one document.


It usually supports the whole argument and is stated by the author as a certain fact.

we extended the scope of backing as an additional support to the whole argument.


16 In the final phase, we extended the range of annotated registers and added newswire articles from the raw corpus in order to test whether the annotation guidelines (and inherently the model) was general enough.

we selected 96 comments/forum posts, 8 blog posts, and 8 articles for this phase.


This says that some documents are easy to annotate whereas others are harder, but interpretation of such an averaged value has no evidence either in Krippendorff (2004) or other papers based upon it.

we use the other methodology and treat the whole corpus as a single long continuum (which yields in the example of claim = α U 0.541).


The "myside bias," defined as a bias against information supporting another side of an argument (Perkins 1985;Wolfe, Britt, and Butler 2009), can be observed by the presence of rebuttals to the author's claim or by formulating arguments for both sides when the overall stance is neutral.

85% of the documents do not consider any opposing side, and only 8% of the documents present a rebuttal, which is then attacked by refutation in 4% of the documents.


27 However, exploiting the pathos dimension of an argument is prevalent in the Web data, for example, as an appeal to emotions.

we experimented with annotating appeal to emotions as a separate category independent of components in the logos dimension.


To the best of our knowledge, there has been very little research on modeling fallacies similarly to arguments at the discourse level (Pineau 2013).

the question, in which detail and structure fallacies should be annotated, remains open.


Instances that are not inscribed are considered to be invoked (also sometimes called evoked), in which "an evaluative response is projected by reference to events or states which are conventionally prized" (Hunston and Thompson 2000b, page 142).

a bright kid or a vicious kid are inscribed.


3 In this article, we advocate a recursive approach, in which complex parser states are represented as the composition of simpler ones, and these are constructed incrementally as parsing proceeds.

we operate in a paradigm in which the model builder designs recursive architectures that automatically discover effective views of the complete parser's evolving state rather than explicitly engineering which features of the state to focus on.


In addition to adding elements to the end of the sequence, the stack LSTM provides a pop operation that moves the stack pointer to the previous element (i.e., the previous element that was extended, not necessarily the next-right-most element).

the x 1 x 2 A stack LSTM extends a conventional left-to-right LSTM with the addition of a stack pointer (notated as TOP in the figure).


Beam search was determined to have minimal impact on scores (absolute improvements of ≤ 0.3% were observed with small beams).

all results we report used greedy decoding-Chen and Manning (2014) likewise only report results with greedy decoding.


The percentage of transliteration pairs in the NEWS10 data sets is much larger than in other parallel corpora.

we also evaluated our transliteration mining system on new data sets extracted from English/Hindi and English/Arabic parallel corpora that have as few as 2% transliteration pairs.


Specifically, hypernym relationships between synonym sets form a hierarchal tree where the root (Entity) provides the most abstract sense.

the "word specificity" feature is the average depth of nouns in the WordNet (noun) hypernym tree, after recent work (Le 2010).


The nature of the DementiaBank conversations is such that the interviewer is never in a position to exhibit a TIB.

all interviewer data points are removed.


In this domain, we assume that recognizing an object shape (i.e., determining whether a star has seven or eight points) requires more time than recognizing a label type (i.e., determining whether a star contains a letter or number).

we will say that object shapes are comparatively "difficult" to recognize, and that label types are comparatively "easy."


Automatic text understanding requires knowledge and, so far, machines know only what we give or teach them.

most natural language processing (NLP) tasks crucially rely on the existence of linguistic resources that encode information about language, be it of morphological, syntactic, or semantic nature.


These parsers, which have a time complexity of O(n 3 ) for n being the length of the input string, operate by composing adjacent substrings of the input sentence into longer substrings.

the structures they can build directly do not involve any discontinuity.


The "derives" relation ⇒ G and other relevant notation are defined as for LCFRSs.

in particular, the language induced by sDCP G is }, whose elements are now sequences of trees.


As terminal reindexing we use g such that g(3) = 5.

we obtain the reindexed rule: By using the substitutions x 1 → b 4 and x 2 → c 6 for both components we obtain the derivation step: We write ⇒ G for u∈N + ,ρ∈P ⇒ u,ρ G .


For a recursive partitioning of h s g in which the root has children labeled {1, 3} and {2}, Algorithm 2 constructs the LCFRS: Alternatively, if we let Algorithm 2 run on the recursive partitioning of h s g in which the root has children labeled {1, 2} and {3}, then it produces the following LCFRS, which is, in fact, a CFG: Observe that terminals are generated individually by the rules that were constructed from the leaves of the recursive partitioning but not by the structural rules obtained from internal nodes.

the LCFRS that we induce is in general not lexicalized.


With dependency structures, however, we need to allow for the scenario where we have a label J of a node in π and two positions p and p in s with parent(p ) = p, and p ∈ Π(J) whereas p / ∈ Π(J).

we will need to consider a subtree of s rooted in p, with a "gap" for child p .


By Theorem 1, sCFTG and single-synthesized sDCP have the same s-term generating power.

it suffices to show that the full class of sDCP is strictly more powerful than single-synthesized sDCP.


Our study also found, however, that existing treebanks are not optimally suited for extracting such phrase pairs in that some structures are too flat for the purpose of extracting minimal rules.

there is a significant number of rules that are rather long and unlikely to repeat in an actual corpus.


Similarly, the two Chinese function words in Example (2), namely, the negation and the modal , match their English translations in grammatical function.

the words are aligned as indicated by the subscripts.


Whenever a legitimate pair of phrases cannot be aligned, the terminal nodes that they dominate will accumulate until an aligned node pair higher up in the tree is reached.

we will obtain translation rules that contain more lexical items.


21 AMR tries to develop semantic representations for MT, which form an inventory of abstract concepts and relations that can generalize over morphosyntactic variations.

it has no difficulty bridging translation divergences caused by reordering, category mismatch, or the absence of function words on either side of a language pair.


A synset is a set of lexemes with the same meaning in the sense that they are interchangeable for each other in context.

we can also define a lexeme as the conjunction of a word and a synset.


Another premise of the model is that a synset is the sum of its lexemes.


we proposed in the beginning of the article (Equation (8)).

replacing a word's embedding by the sum of the embeddings of its senses could generally improve the quality of embeddings-see Huang et al.


Errors in the word test set are probably due to insufficient word embedding models or errors caused by the linear mapping L. As we already mentioned, our synset embeddings can only be as good as the underlying word embeddings.

both cases, insufficient word embeddings and insufficient linear map, also affect the performance of the synset embeddings.


Moreover, their approach links all argument components in a single tree structure.

it is not capable of separating several arguments and recognizing unlinked components.


However, the corpus does not include non-argumentative text units.

the corpus is only of limited use for training end-to-end argumentation structure parsers.


From a theoretical perspective, divergent structures are equivalent to several arguments (one for each claim) (Freeman 2011, page 16).

of this treatment, a great many of theoretical textbooks neglect divergent structures (Henkemans 2000; Reed and Rowe 2004) and also most computational approaches consider arguments as trees (Cohen 1987;Mochales-Palau and Moens 2009;Peldszus 2014).


Because of this common writing procedure, divergent and circular structures are rather unlikely in persuasive essays.

we assume that modeling the argumentation structure of essays as a tree is a reasonable decision.


Evaluating the agreement at the sentence level is an approximation of the actual agreement since the boundaries of argument components can differ from sentence boundaries and a sentence can include several argument components.

5 for the second evaluation strategy, we use Krippendorff's α U (Krippendorff 2004).


For determining the agreement of the stance attribute, we follow the same methodology as for the sentence-level agreement described above, but we consider each sentence containing a claim as "for" or "against" according to its stance attribute, and all sentences without a claim as "none" (N = 1,441; n = 3; k = 3).

the agreement of claims constitutes the upper bound for the stance attribute.


We obtain an agreement of 88.5% and κ = 0.623, which is slightly below the agreement scores  Table 2).

human annotators can reliably differentiate between supporting and attacking claims.


The number of sentences with several argument components is 583, of which 302 include several components with different types (e.g., a claim followed by premise).

the identification of argument components requires the separation of argumentative from non-argumentative text units and the recognition of component boundaries at the token level.


The proportion of paragraphs with unlinked argument components (e.g., unsupported claims without incoming relations) is 421 (23%).

methods that link all argument components in a paragraph are only of limited use for identifying the argumentation structures in our corpus.


It achieves an F1 score of 0.677 for non-argumentative tokens ("O") and 0.867 for argumentative tokens ("Arg-I").

the heuristic baseline effectively separates argumentative from non-argumentative text units.


The macro F1 score of our model is only 0.019 less.

our model achieves 97.9% of human performance.


The second constraint covers all paths of length greater than 1 in a transitive way.

it states that if there is a path from argument component i to argument component j (b ij = 1) and another path from argument component j to argument component k (b jk = 1) then there is also a path from argument component i to argument component k. it iteratively covers paths of length l + 1 by having covered paths of length l. The third constraint avoids cycles by preventing all directed paths starting and ending with the same argument component.


Regarding relation identification and stance recognition, the macro F1 scores of our model are 0.103 and 0.164 lower than human performance.

our model achieves 95.2% of human performance for component identification, 87.9% for relation identification, and 80.5% for stance recognition.


Furthermore, it was necessary to adapt several features of the base classifiers, since the microtext corpus does not include non-argumentative text units.

we did not consider preceding tokens for lexical, indicator, and embedding features and removed Table 13 Model assessment on microtext corpus from Peldszus and Stede (2015) ( † = significant improvement over baseline heuristic; ‡ = significant improvement over base classifier).


In particular, less accurate argumentative relations have a more detrimental effect on the component types ( Figure 4a) than less accurate component types do on the outcomes of the relation identification (Figure 4b).

it is more reasonable to focus on improving relation identification than component classification in future work.


Our argumentation structure parser is a pipeline consisting of several consecutive steps.

potential errors of the upstream models are propagated and  Influence of improving the base classifiers (x-axis shows the proportion of improved predictions and y-axis the macro F1 score).


However, our identification model yields good accuracy and an α U of 0.958 for identifying argument components.

it is unlikely that identification errors will significantly influence the outcome of the downstream models when applied to persuasive essays.


Previous approaches suffer from several limitations: Existing approaches either focus only on particular subtasks of argumentation structure parsing or rely on manually created rules.

previous approaches are only of limited use for parsing argumentation structures in real application scenarios.


In detail, γ considers that the three annotators agree on the fact there is a first unit on the left of the continuum, but do not fully agree on its category (A, B, and A), that two annotators only consider that there is a unit in the middle of the continuum, of category B, and that only one annotator found a third unit of category C at the end of the continuum.

γ has created three empty units, one because annotator 3 missed the second unit, and two because annotators 1 and 2 missed (compared with 3) the third unit.


Fortunately, many sentences are long and complex enough to present rich discourse structures connecting their basic clauses.

although limited, this setting can demonstrate the potential of discourse-level information for MT evaluation.


As shown in Figure 3(a), DR does not include any lexical item.

the syntactic tree kernel, when applied to this representation of the discourse tree, measures the similarity between two candidate translations in terms of their discourse representations only.


In our case, we work with two different translations (one gold and one MT-generated), which makes their RST trees not directly comparable.

we apply an approximation, and we measure F 1 score over the total number of instances of a specific tag, regardless of their position in the tree.


tree and that has many more inaccuracies at the discourse relation level.

the tree kernel finds fewer subtree matches, and the similarity score becomes lower.


Unfortunately, learning features with preference kernels is computationally very expensive, both at training and at testing time.


We propose an approach to building models that incorporates the best of both modes: training on native texts to facilitate the possibility of training from large amounts of data without the need for annotation, but using the correction mode with a modest amount of annotated ESL data so that the model can adapt to writers' errors.

at evaluation time, these models can make use of the potentially erroneous information provided by the writer.


Whether presenting the top three candidates to the user is reasonable depends on the evaluation of how good the top three candidates are.

we also wish to assess top three candidates directly because the top three F1 measure values presented are computed for a gold standard that allows only a single correct answer, when in fact there may be multiple possible correct answers.


It should be noted, however, that data size for prior estimation depends on error type, or, specifically, the error rate and the size of the confusion set (small for agreement, large for prepositions).

for agreement mistakes, even 40 errors provide a good estimate of error patterns.


This is in contrast to features encoding contextual information, which can be estimated using native English data.

the proposed adaptation approach allows one to combine large amounts of native data (which is cheap and can be used to learn context features) with a small amount of annotated ESL data for estimating error patterns.


On the other hand, accuracy measure is only a single point and in order to get improvement it pushes the system toward low recall.

one important direction for future work is a focus on designing a system that can improve the quality of the text compared with the learner baseline.


Further, because we care about specific first-language backgrounds, CoNLL is not an appropriate data set to use: The FCE corpus is the only one that contains data from learners of multiple language backgrounds and information on the first language of the writer, used in Section 5.

in this work we make use of the FCE data set.


Specifically, when an MT system is trained only on the CoNLL corpus, the performance is quite poor (28.25).

all other MT systems use an additional source of supervision-either the Cambridge Learner corpus (CLC), which is a larger version of FCE, or the publicly available Lang-8 corpus.


WordNet, although arguably the best choice for our purpose, is not entirely reliable as a gold standard resource, with occasional inconsistencies and debatable precision regarding the way lexical relations have been encoded: for example, silly is a hyponym of child according to WordNet.

all sampled pairs were manually checked by the authors plus two native English speakers in several iterations.


Note that the lexical split discards all "cross-set" training-test concept pairs.

the number of instances in each subset is lower than with the random split.


34 Because of a wide variety of models and a large space of results, it is not feasible to present all results at once or provide detailed analyses across all potential dimensions of comparison.

we have decided to make a gradual selection of the most interesting experiments and results, and stress (what we consider to be) the most important aspects of the HyperLex evaluation set and models in our comparisons.


Because of their non-compositionality, MWEs exhibit nonsubstitutability, that is, limited morphosyntactic and semantic variability.

the replacement or modification of individual words of an MWE often results in unpredictable meaning shifts or invalid combinations.


On the other hand, large-scale discovery, although potentially useful, is hard to evaluate.

methods tend to be specialized and will not work so well when ported to other MWE categories and languages.


This is often due to a mismatch between discovered MWEs and test sets, and to the difficulty in integrating such noisy lexicons into applications.

it is rare to observe convincing performance gains in downstream tasks.


take the door) meaning get sacked has a strong morphological constraint: the noun porte (door) must be in the singular; if it is in the plural, the sequence has its literal meaning.

using lemmas to identify variants is a potential source of mistakes.