The number n of outermost arguments of the secondary input category is called the degree of the rule.

2 for n = 0 we obtain the rules of function General form of composition rules (primary input category shaded), where n ≥ 0 and | i ∈ {/, /}.


As mentioned in Section 1, the central feature of this formalism is its ability to impose restrictions on the applicability of combinatory rules.

a restricted rule is a rule annotated with constraints that (a) restrict the target of the primary input category; and/or (b) restrict the secondary input category, either in parts or in its entirety.


Derivations are considered complete only if they satisfy additional, node-specific constraints.

substitution is obligatory at every node where it is possible, and adjunction may be specified as either obligatory (OA, Obligatory Adjunction) or forbidden (NA, Null Adjunction) at a given node.


Note that R1 and R2 produce a new derivation for the original sentence, whereas R3 and R4 produce a derivation that yields a permutation of that sentence: The order of the substrings corresponding to the categories Z and X/Y (in the case of rule R3) or X /Y (in the case of rule R4) is reversed.

r3 captures the relation between the two derivations of Swiss German word orders shown in Figure 11: Applying r3 to derivation (5) gives derivation (6).


To address the problem of ungrammatical word orders in Dutch subordinate clauses, the VW-CCG grammar of Steedman (2000) and the multimodal CCG grammar of Baldridge (2002, Section 5.3.1) resort to combinatorial rules other than composition.

they assume that all complement noun phrases undergo obligatory type-raising, and become primary input categories of application rules.


However, our technical results still serve as important reminders that extra care must be taken to avoid overgeneration when designing a grammar.

it is worth double-checking that an OpenCCG grammar does not generate word orders that the grammar developer did not intend.


Then, we describe global methods that perform inference over a larger set of predicates.

we provide details on the method and optimization problem developed by  for which we propose scalable inference algorithms in this article.


Therefore we can obtain training data easily.

we train a sentiment parser, s.parser, from a large amount of review sentences with users' ratings as rough sentiment polarity labels.


In this article, we propose a statistical parsing framework to directly analyze the structure of a sentence from the perspective of sentiment analysis.

we formulate a Context-Free Grammar (CFG)-based sentiment grammar.


we train a sentiment parser, named s.parser, from a large number of review sentences with users' ratings as rough sentiment polarity labels.


In this work, we build a sentiment parser.

we use a modified version of the CYK algorithm that parses sentences in a bottom-up fashion.


We can only observe the sentence s and its polarity label y in training data.

we train the model directly from the examples of sentences annotated only with sentiment polarity labels but without any syntactic annotations or polarity annotations of the constituents within sentences.


Second, when we apply the combination rules, the polarity strength of subspans needs to exceed a predefined threshold τ (≥ 0.5).

for combination rules X → 1 , we define the second constraint as where K is the number of subspans in the rule, and X k is the corresponding polarity label of non-terminal X k in the right hand side.


First, the sentiment grammar and the polarity model are learned from data.

the rules and the parameters used to compute polarity values are learned.


It keeps the local optimums to reduce the huge search space.

the beam search algorithm generates the K-best trees with the highest score φ(s, t) T ψ for each span.


After replacing the subspans good, as expected, funny, and well done with their polarity label P, we can learn the negation rule N → is not P. We present the combination rule learning approach in Algorithm 4.

the first step is to generate combination rule candidates.


( , 2013. s.parser-LongMatch: The longest matching rules are utilized in the decoding process.

the derivations that contain the fewest rules are used for all text spans.


Figure 10 shows the effect of minimum fragment frequency, and maximum fragment length.

figure 10a indicates that a minimum fragment frequency that is too small will introduce noise, and it is difficult to estimate reliable polarity probabilities for infrequent fragments.


As shown in Figure 13b, the "N → is not P" is a shift negation that reduces a fixed polarity strength to reverse the original polarity.

the is not good is more negative than the is not great, as described in Section 3.4.


Similarly, a high quality speech synthesis system based on the selection of speech units requires a rich corpus in terms of diphones, diphones in context, triphones, and prosodic markers.

bunnell (2010) shows the importance of a good coverage of diphones and triphones for the intelligibility of a voice produced by a unit selection speech synthesis system.


However, the creation cost of such data highly depends on the kind of labels used to adapt the model.

the annotation in syntax trees is really more expensive than in Part-of-Speech (POS) tags.


, x n ) T where x j = 1 if s j belongs to X and x j = 0 otherwise.

a covering is a solution X ∈ {0, 1} n of the following system that is, AX ≥ B where B is called the constraint vector.


At each iteration, the set of the redundant sentences of Y is calculated and the costliest one (according to the cost function C) is removed from Y.

an element s of Y is said to be redundant if for each u i ∈ U, its number of instances into Y, denoted m i (Y ), and into Y \ {s}, denoted s is a redundant element of the covering Y if Y \ {s} is also a covering solution of Equation (1).


Indeed, we can observe in Table 2 that the global statistics of POS tags in Le-Monde are quite different from their phonological counterparts summarized in Table 1.

the density of matrix A is 11.03% for a 1-POS covering and 0.57% for a 2-POS covering.


These trends were also observed in the earlier experiments.

as for the 1-covering of 1-POS, not only does LamSCP provide 10.06% (± 0.00) shorter solutions than ASA, but its solutions are optimal for all 50 instances of this SCP.


A well-written text is not merely a sequence of independent and isolated sentences, but instead a sequence of structured and related sentences, where the meaning of a sentence relates to the previous and the following ones.

a well-written text has a coherence structure (Halliday and Hasan 1976;Hobbs 1979), which logically binds its clauses and sentences together to express a meaning as a whole.


Once the EDUs are identified, the discourse parsing problem is determining which discourse units (EDUs or larger units) to relate (i.e., the structure), and what relations (i.e., the labels) to use in the process of building the DT.

discourse parsing requires: (1) a parsing model to explore the search space of possible structures and labels for their nodes, and (2) a parsing algorithm for selecting the best parse tree(s) among the candidates.


It is straightforward to generalize the above algorithm to produce k most probable DTs.

when filling up the dynamic programming tables, rather than storing a single best parse for each sub-tree, we store and keep track (i.e., using back-pointers) of k-best candidates simultaneously.


In this process, each sentence in a document except the boundary sentences (i.e., the first and the last) will be associated with two DTs: one with the previous sentence (say, DT p ) and one with the next (say, DT n ).

for each nonboundary sentence, we will have two decisions: one from DT p and one from DT n .


Because sentence boundaries are considered to also be the EDU boundaries, we measure segmentation accuracy with respect to the intra-sentential segment boundaries, which is a standard method (Soricut and Marcu 2003;Fisher and Roark 2007).

if a sentence contains n EDUs, which corresponds to n − 1 intra-sentential segment boundaries, we measure the segmenter's ability to correctly identify these n − 1 boundaries.


For SPADE, we applied the same modifications to its default settings as described in Fisher and Roark (2007), which delivers significantly improved performance over its original version.

in our experiments on the RST-DT corpus, we trained SPADE using the human-annotated syntactic trees extracted from the Penn Treebank (Marcus, Marcinkiewicz, and Santorini 1994), and, during testing, we replaced the Charniak parser (Charniak 2000) with a more Table 3 Discourse segmentation results of different models on the two corpora.


15 Our experiments on the instructional corpus consider the same 26 primary relations (e.g., Goal:Act, Cause:Effect) used by Subba and Di-Eugenio (2009) and also treat the reversals of non-commutative relations as separate relations.

goal-Act and Act-goal are considered to be two different coherence relations.


We adapt our parsing algorithm accordingly to produce k most probable DTs for each text, and measure the oracle accuracy based on the F-scores of the Relation metric which gives aggregated evaluation on structure and relation labels (see Table 2).

the oracle accuracy O−score for k-best discourse parsing is measured as follows: where N is the total number of texts (sentences or documents) evaluated, g i is the gold DT annotation for text i, h j i is the j th parse hypothesis generated by the parser for text i, and F-score r (g i , h j i ) is the F-score accuracy of hypothesis h j i on the Relation metric, which essentially measures how similar h j i is to g i in terms of its structure and labels.


Notice that inclusion of every new subset of features appears to improve the performance over the previous set.

for sentence-level parsing, when we add the Organizational features with the Dominance set features, we achieve about 2 percentage points absolute improvements in nuclearity and relations.


This attempt to extend Slot Error Rate to unitizing suffers from severe limitations.

all positioning and categorizing errors have the same penalty, which may be a serious drawback for annotation tasks where some fuzziness in boundary positions is allowed, such as TOPIC SEGMENTATION, TOPIC TRANSITION, or DISCOURSE FRAMING.


As we will see, γ relies on an alignment of units by different annotators.

this alignment indicates for unit u a 1 i of annotator a 1 , to which unit u a 2 j of annotator a 2 it corresponds, in order to compute the associated dissimilarity.


It is important to note that this tool was designed by the authors of γ, for tasks where units cannot be considered as atomizable.

it was conceived so that disagreements concerning small units are as important as those concerning large ones.


However, in order to be compliant with u α, c|u α, and κ d , we limit the configurations here so that the units do not overlap at all.

the reference was built with no overlapping units, and we have used a modified version of the shifting shuffling procedure so that the nonoverlapping constraint is fully satisfied, even at high magnitudes.


consider a sequence X of length N. The maximum matching length L i is defined as: L i is the maximum common subsequence before and after i. IfL is the average length of L i , given bȳ then the method obtains the entropy rate h 1 as Given the true entropy rate h * , convergence has been mathematically proven for a stationary process, such that In this article, we consider this entropy rate h 1 as a constancy measure candidate.


The problem here is that the null hypothesis is always related to the alternative hypothesis to a certain extent, because the difference between convergence and non-convergence is merely a matter of degree.

the notion of convergence for a constancy measure does not conform with the philosophy of statistical testing.


10 Note that Chinese characters have morphological features, and the Arabic and Thai languages also have flexibility in terms of which units are considered words and morphemes.

the plots closer to the random data with a smaller Zipf exponent are for language corpora of morphemic sequences.


In this article we begin by proposing a general system to solve the word ordering problem, which does not rely on constraints (which are typically task-specific).

we treat syntax-based word ordering as a structured prediction problem, for which the input is a multi-set (bag) of words and the output is an ordered sentence, together with its syntactic analysis (either CCG derivation or dependency tree, depending on the grammar formalism being used).


For the timeout case, a default output is produced by greedily combining existing edges in the chart in descending order of size.

edges in the chart are sorted by size, and the largest is taken as the current default output.


We apply the same online large-margin training principle; however, the parameter update has to be adjusted for the scaled linear model.

the new goal is to find a separation betweenf (e + ) andf (e − ) instead of f (e + ) and f (e − ), for which the optimization corresponding to the parameter update becomes: where ⃗ θ 0 and ⃗ θ represent the parameter vectors before and after the update, respectively.


Our learning algorithm is similar to LaSO with best-first inference, but the parameter updates are different.

laSO updates parameters when all correct hypotheses are lost, but our algorithm makes an update as soon as the top item from the agenda is incorrect.


Data are partitioned into the folds in time sequence rather than randomly; then in each validation step, data points within a time distance h of any point in the validation data set are excluded from the training data.

for each validation step k, the validation data is which is disregarded, and the training data .


We used a common task in NLP, text classification, on a relatively recent but widely used data source, Twitter streams, to show that blindly following the same evaluation method for tasks of similar nature could lead to invalid conclusions.

we investigated the most common evaluation method for machine learning applications, standard 10-fold cross-validation, and compared it with other validation methods that take the statistical dependence in the data into account, including time-split, bordersplit, neighbor-split, and a combination of time-and border-split validation.


The features we explore are grounded in the idea that individuals recalling the same narrative are likely to use the same sorts of words and semantic concepts.

a retelling of a narrative will be faithful to the source narrative and similar to other retellings.


The work presented here demonstrates the utility of adapting NLP algorithms to clinically elicited data for diagnostic purposes.

the approach we describe for automatically analyzing clinically elicited language data shows promise as part of a pipeline for a screening tool for mild cognitive impairment.


However, the method used for selection of individual examples may introduce a bias into the evaluation and provide an unfair advantage to the system, unless the test sample was selected randomly from arbitrary text.

this type of evaluation is likely to be less objective than the evaluation on continuous corpus text or a random sample.


In this article, we investigate aspects of sentential meaning that are not expressed in local predicate-argument structures.

we examine instances of semantic arguments that are only inferable from discourse context.


Example 1 illustrated a text fragment, in which argument realization is necessary in the first sentence but redundant in the second.

mentioning Iraq in the second sentence is not necessary (for a human being) to understand the meaning of the text.


For the automatic alignment of predicate-argument structure alignments across texts, we opt for an unsupervised graph-based method.

we represent pairs of documents as bipartite graphs and subsequently aim to separate a graph into subgraphs that represent corresponding predicate-argument structures across texts.


In the second step of our induction framework, we rely on alignments between PAS to detect implicit arguments.

we aim to identify argument instances that are present, or explicit, in one PAS but absent, or implicit, from the aligned PAS.


The high-precision sieves are, however, better suited for the cross-document task as we plan to rely on the resulting coreference chains for identifying potential antecedents of implicit arguments.

we prefer fewer but more reliable chains in order to minimize the impact of possible preprocessing errors.


To assess the impact of training data size, we perform an additional experiment with subsets of automatically induced implicit arguments.

we train different classifiers using the full SemEval training data and varying amounts of our automatically induced training data (random samples of 1%, 10%, and 25%).


We argue that this is actually a desirable property because more complicated alternations could affect coherence by themselves.

resulting interplays would make it difficult to distinguish between the isolated effect of argument realization itself and other effects, triggered for example by sentence order (Gordon, Grosz, and Gilliom 1993).


All models compute local coherence ratings based on entity occurrences, however, and should thus be able to predict which realization type coheres best with the given discourse context.

because the input document pairs are identical except for the affected argument position, the coherence scores assigned by each model to pairs of text only differ with respect to the affected entity realization.


The main difference between this simplified model and the original entity grid model lies in the different use of training data: whereas entity grid models treat all non-realized items equally, our model gets to "see" actual examples of entities that are implicit.

our simplified model takes into account implicit mentions of entities, not only explicit ones.


In this article, we provided several research contributions that form a reliable basis for future work.

we developed a framework for automatically inducing instances of implicit arguments, and we designed a novel coherence model that predicts the effect of argument realizations on perceived textual coherence.


As we show in Section 2, the predominant gold standards for semantic evaluation in NLP do not measure the ability of models to reflect similarity.

in both WS-353 and MEN, pairs of words with associated meaning, such as coffee and cup (rating = 6.810), telephone and communication (7.510), or movie and theater (7.710), receive a high rating regardless of whether or not their constituents are similar.


To estimate similarity, we considered proximity in the WordNet taxonomy (Fellbaum 1998).

we applied the measure of Wu and Palmer (1994) (henceforth WupSim), which approximates similarity on a [0,1] scale reflecting the minimum distance between any two synsets of two given concepts in WordNet.


The resource should cover the full range of concepts that occur in natural language.

it should include cases representing the different ways in which humans represent or process concepts, and cases that are both challenging and straightforward for computational models.


In order for a gold standard to be diagnostic of how well a model can be applied to downstream applications, a clear understanding is needed of what exactly the gold standard measures.

it must clearly distinguish between dissociable semantic relations such as association and similarity.


Importantly, inspecting the pairs revealed that a good range of similarity values are represented.

there were many examples of hypernym/hyponym pairs [body, abdomen], cohyponym pairs [cat, dog], synonyms or near synonyms [deodorant, antiperspirant], and antonym pairs [good, evil].


As shown in Fig 5(a), there were clear differences between these new ratings and the original WS-353 ratings.

a high proportion of pairs was given a lower rating by subjects following the SimLex-999 instructions than those following the WS-353 guidelines: The mean SimLex rating was 4.07 compared with 5.91 for WS-353.


As shown in Figure 5(b-c), the correlation between the SimLex-999-based and WS-353 ratings was notably higher (ρ = 0.73) on the WS-Sim subset than the WS-Rel subset (ρ = 0.38).

the tendency of subjects following the SimLex-999 instructions to assign lower ratings than those following the WS-353 instructions was far more pronounced for pairs in WS-Sim ( Figure 5(b)) than for those in WS-Rel ( Figure 5(c)).


Interestingly, the performance of models on the most abstract and most concrete pairs suggests that the distinction characterized by concreteness is at least partially independent of POS.


This finding supports the growing evidence for systematic differences in representation and/or similarity operations between abstract and concrete concepts (Hill, Kiela, and Korhonen 2013), and suggests that at least part of these concreteness effects are independent of POS.

it appears that models built from underlying vectors of co-occurrence counts, such as VSMs and SVD, are better equipped to capture the semantics of concrete entities, whereas the embeddings learned by NLMs can better capture abstract semantics.


We hope that researchers will adopt the resource as a robust means of answering a diverse range of questions pertinent to similarity modeling, distributional semantics, and representation learning in general.

for models to learn high-quality representations for all linguistic concepts, we believe that future work must uncover ways to explicitly or implicitly infer "deeper," more general, conceptual properties such as intentionality, polarity, subjectivity, or concreteness (Gershman and Dyer 2014).


Intelligence requires being able to understand bigger things from knowing about smaller parts.

for language, understanding novel and complex sentences crucially depends on being able to construct their meaning compositionally from smaller parts-words and multiword expressions-of which they are constituted.


New models and techniques should be applied to solve real-world problems and improve people's daily lives.

since we have moved into the era of the Internet.


Starting with a brief introduction to the fundamentals of SMT systems, we follow by covering a wide variety of optimization algorithms for use in both batch and online optimization.

we discuss losses based on direct error minimization, maximum likelihood, maximum margin, risk minimization, ranking, and more, along with the appropriate methods for minimizing these losses.


Given this overall formulation of MT, the features h( f , e, d) that we choose to use to represent each translation hypothesis are of great importance.

with regard to optimization, there are two important distinctions between types of features: local vs. non-local, and dense vs. sparse.


One final variety of features that has proven useful is syntax-based features Marton and Resnik 2008).

phrase-based and hierarchical phrase-based translations do not directly consider syntax (in the linguistic sense) in the construction of the models, so introducing this information in the form of features has a potential for benefit.


Softmax loss is a loss that is similar to the zero-one loss, but directly defines a probabilistic model and attempts to maximize the probability of the oracle translations (Berger, Della Pietra, and Della Pietra 1996;Och and Ney 2002;Blunsom, Cohn, and Osborne 2008).

if we assume that MT is modeled according to the log-linear model we can define softmax loss softmax (•) as follows: From Equation 21we can see that only the oracle translations contribute to the numerator, and all candidates in c (i) contributes to the denominator.


In contrast to softmax loss, which can be viewed as a probabilistic version of zero-one loss, risk defines a probabilistic version of the translation error (Smith and Eisner 2006;Zens, Hasan, and Ney 2007;Li and Eisner 2009;He and Deng 2012).

risk is based on the expected error incurred by a probabilistic model parameterized by w. This combines the advantages of the probabilistic model in softmax loss with the direct consideration of translation accuracy afforded by using error directly.


Simianer, Riezler, and Dyer (2012) propose another method for mixing parameters that, instead of averaging at each iteration, chooses to preserve only the parameters that have been learned over all shards, and sets all the remaining parameters to zero, allowing for a simple sort of feature selection.

we define a S × M matrix that combines the parametersw (t+1) s at each shard asw (t+1) = w (t+1) S , takes the L 2 norm of each matrix column, and averages the columns with high norm values while setting the rest to zero.


In general, most nonlinear models for machine learning can be applied to MT as well, with one major caveat.

the efficiency of the decoding process largely relies on the feature locality assumption mentioned in Section 2.3.


As a result, the overall score of trees that have such competing dependencies cannot be higher than the trees that have our desired dependency.

the resulting parse tree will contain dependency d. This feature will be used in Section 4 in order to parse a sentence for which a certain number of dependencies are set in advance.


As for language processing, learning segmentation can be advantageous compared with learning full analyses.

learning full analysis in a supervised manner typically requires up to tens of thousands of manually annotated sentences.


For some models of the Morfessor family this is not possible because of the use of a hierarchical lexicon, where morphs can be generated from other morphs as well as from individual letters.

this includes the well-performing Categories-MAP variant .


The log-linear form of p(U, S; θ) enables the approach to use a wide range of overlapping features.

poon, Cherry, and Toutanova (2009) utilize a morph-context feature set with individual features defined for each morph and morph substring contexts.


In addition to unsupervised learning, they present experiments in the semi-supervised setting.

they accomplish this by fixing the segmentations of annotated words in D, according to their gold standard segmentation.


2013) approaches directly estimate a conditional distribution of segmentation given a word form.

whereas generative methods generate both word forms and segmentations, the discriminative methods generate only segmentations given word forms.


On the other hand, the semi-supervised setting also makes it possible to apply discriminative techniques to generative models.

model hyperparameters can be selected to optimize segmentation performance, rather than some generative objective, such as likelihood.


Evidently, utilizing annotated data provides a distinct advantage over learning from unannotated data.

learning the supervised CRFs using 1,000 annotated word forms results in substantially higher segmentation accuracies compared with learning in an unsupervised manner from hundreds of thousands or millions of word forms.


These approaches maximize precision (WORDS) and recall (LETTERS) at the cost of the other.

no model can produce more over-segmentation errors compared with LETTERS, and no model can produce more under-segmentation errors compared with WORDS.


In contrast to the Morfessor and AG frameworks, the error patterns produced by the CRF approach do not directly follow the baseline approaches.

we note that the supervised CRF (SV) approach successfully captures SUFFIX-SUFFIX boundaries and fails to find STEM-STEM boundaries-that is, behaves in an opposite manner compared with the baseline results.


However, by examining the error analyses in Tables 7 and 8, we also observe the occurrence of a second kind of trade-off, in which the semi-supervised Morfessor and AG approaches trade off under-segmentation errors to other under-segmentation errors.

although the STEM-SUFFIX and SUFFIX-SUFFIX boundary recall errors are decreased, one also observes an increase in the errors at STEM-STEM and PREFIX-STEM boundaries.


From the set of all possible sentences of the target language, we are interested in that with the highest probability according to the following equation: {Pr(e I 1 | f J 1 )} Early works on SMT decompose Pr(e I 1 | f J 1 ) applying Bayes' theorem and thus obtaining two new distributions, Pr(e I 1 ) and Pr( f J 1 |e I 1 ), which are approximated by means of parametric statistical models.

pr(e I 1 ) is modeled by means of a language model, and pr( f J 1 |e I 1 ) is modeled by means of a translation model.


This is not an important issue because the transformations that are required in the statistical models for their use at character level are trivial.

the compatibility with the user prefix is verified by comparing characters instead of words.


One important consequence of discarding the training samples after each trial is that the computational complexity of processing a new sample does not depend on the number of samples that has been previously seen.

the computational complexity of processing a new sample is constant.


To solve this problem, we use the direct and inverse HMM-based alignment models that are included in the formulation of the IMT system.

these models are used to obtain word alignments in both translation directions.


Single-word alignment models are based on the concept of alignment between word positions of the source and the target sentences f J 1 and e I 1 .

the alignment is defined as a function a : where a j = i if the jth source position is aligned with the ith target position.


Delaying model updates may cause performance degradation, but it also constitutes one way to reduce the strong time requirements of a sentence-wise updating strategy.

if the time between updates is sufficiently high, the use of batch learning techniques could be appropriate (e.g., the training process can be executed overnight), removing the necessity of implementing online learning.


As it can be seen, the boxplot clearly show the small time cost of the learning process for the two different corpora under consideration.

the learning time was never greater than 1 second, and the median times were 0.03 and 0.16 seconds for the XRCE and the Europarl corpora, respectively.


Other works try to avoid retraining the phrase model by aligning the output of the decoder with the reference given by the user (Blain, Schwenk, and Senellart 2012;Simard and Foster 2013).

blain, Schwenk, and Senellart (2012) propose obtaining word alignments between the source and reference sentences using the system output as pivot.


Additionally, we introduce an alternative update rule using the incremental EM algorithm, which solves one important limitation of the update rule that was originally introduced in Ortiz-Martínez, García-Varea, and Casacuberta (2010).

the new update rule allows us to execute more than one training epoch over the incoming data while maintaining constant time and spatial complexity (see Sections 3 and 4.4 as well as Appendix A for more details).


As we can see, the value of the R parameter had a strong influence in the system performance.

the interlaced system with R = 10 clearly underperformed the results obtained by the online system; increasing the value to R = 100 obtained almost identical results; and R = 1,000 produced slightly better results.


In spite of the fact that this update rule allows us to execute E epochs over the training data, it is still very different from the way in which conventional batch training works.

batch training only starts the next training epoch after having processed the whole training corpus in the previous epoch.


To increase the lapse of time (or the number of trials) until a given sample is reprocessed, the samples are processed in an interlaced way, as it is depicted in Figure A.1.

the figure shows which samples are processed by means of the incremental EM algorithm at t = 10 when we want to execute E = 3 epochs, keeping the last R = 5 samples.


Our survey notably differs from previous work (Costa-jussà and Fonollosa 2009) in that we not only review the existing approaches to word reordering in SMT, but we also question why some approaches are more successful than others in different language pairs.

we argue that understanding the complexity of reordering in a given language pair is key to selecting the right SMT models and to improving them.


Section 4 presents a qualitative analysis of word reordering across language pairs.

detailed word order profiles are provided for a sample of seven widely spoken languages representing structural and geographical diversity: namely, English, German, French, Arabic, Turkish, Japanese, and Chinese.


Because searching over the space of all possible translations is NP-hard (Knight 1999), SMT decoders use heuristic search algorithms to only explore a promising subset of the search space.

limiting the set of explorable input permutations is an essential way to reduce decoding complexity.


ITG constraints only admit permutations that are generated by recursively swapping pairs of adjacent blocks of words.

4 ITG constraints disallow reorderings that generalize the patterns (3 1 4 2) and (2 4 1 3), which are rarely attested in natural languages (Wu 1997).


the most widely used type of phrase-based reordering model and is considered a necessary component of PSMT baselines in any language pair.

two variants of POM deserve further attention because of their notable effect on translation quality: hierarchical POM and sparse phrase orientation features.


We insist, however, on the qualitative nature of our analysis: Numbers are only meaningful in combination with the list of specific discordant features, as these have a different impact on word reordering.

we find it essential for SMT to distinguish between clause-level and phrase-level differences (CDiff and PDiff) because the former account for most longer-range word movements, and the latter for the shorter.


We conclude from our analysis that a few linguistic facts can be very useful to predict the reordering characteristics of a language pair and to select the SMT approach that best suits them.

string-based PSMT is preferable for language pairs with only constituent-level differences, like French-English, as these mostly imply short-or medium-range reordering patterns that can be captured by local distortion.


The complexity of parsing with synchronous context-free grammars is polynomial in the sentence length for a fixed grammar, but the degree of the polynomial depends on the grammar.

the degree depends on the length of rules, the permutations represented by the rules, and the parsing strategy adopted to decompose the recognition of a rule into smaller steps.


We assume without loss of generality that the nonterminals of the Chinese component are indexed sequentially.


We still need to show that M is a cyclic permutation multigraph, that is, all the edges mentioned above form a red Hamiltonian cycle and a green Hamiltonian cycle over the vertices of M. We start by observing that, within each component G i , these two cycles are symmetric, in the sense that the green Hamiltonian cycle can be obtained from the red Hamiltonian cycle by switching the first and the second indices of each vertex in an edge.

within G i the green Hamiltonian cycle can be obtained from the red Hamiltonian cycle by a rotation along the axis from vertex g 4k,4k .


If T M has bounded carving width, the edges routed through arc a saturate the width of this arc, making it impossible for other vertices not from G i to be placed within T i .

there is always some core set of vertices from G i that is placed in some subtree of a tree layout of M, and this subtree must exclude vertices from other grids.


We define the nodes of this tree T to be the set I, and its edges to be the set F. The subset of V associated with node i of T is denoted by X i .

a tree decomposition is therefore defined as a pair ({X i | i ∈ I}, T = (I, F)), where each X i , i ∈ I, is a subset of V, and tree T has the following properties r Vertex cover: The nodes of the tree T cover all the vertices of G: r Edge cover: Each edge in G is included in some node of T. for all r Running intersection: The nodes of T containing a given vertex of G form a connected subtree.


In this way, each nonterminal has two endpoints, and adjacent nonterminals share one endpoint.

the left-hand side nonterminal shares one endpoint with the left-most Dependency graph for the CFG rule S → A B C D (left) and the SCFG rule Gadget grid G i in the construction of cyclic permutation multigraph M, assuming nonterminal in the right-hand side and one endpoint with the right-most nonterminal in the right-hand side.


This subtree has treewidth 4d i − 1, and can be constructed in a linear chain adding vertices from D i one at a time.

we process the nodes in D i in columnmajor order, at each step constructing a new node in the tree decomposition of D by adding one vertex of D i , either red or green, and removing the corresponding vertex from the previous red or green column respectively.


We show that when controlling for polysemy, our indicators of higher clusterability tend to correlate with our two gold standard partitionability estimates.

clusterability tends to correlate positively with higher inter-annotator agreement and negatively with a greater proportion of mid-range judgments on a graded scale of instance similarity.


Thus, we propose three novel, language-independent approaches to source language adaptation for resource-poor statistical machine translation.

we build improved statistical machine translation models from a resource-poor language POOR into a target language TGT by adapting and using a large bitext for a related resource-rich language RICH and the same target language TGT.


Still, this latter model works better, which means that phrase-based SMT systems are robust to noise and prefer more variety rather than better translations in the training bitext.

humans usually like high precision, whereas what the downstream SMT system really needs should be high recall.


2010;Ture and Lin 2012) assume that the source and target documents are separate.

these methods reason about the probability that documents A and B are translations.


The automatic identification and retrieval of translated text (bitext) is a well-studied problem in natural language processing.

there has been a long tradition of exploiting the multilingual Web as a source of bitext (Resnik and Smith 2003;Fukushima, Taura, and Chikayama 2006;Li and Liu 2008;Uszkoreit et al.


We also define an evaluation metric that compares the predictions of our model with those obtained by manual annotation.

we test whether the predicted values of .


We also repeated the experiments including negative observed values, with a similar pattern of results.

we first verify how the observed PMI values of phrases depart from those predicted by addition: how much ∆ c PMI(ab) = 0 in practice.


Here, we experiment with a shifted additive model obtained by subtracting a constant vector from the summed PMI vector.

we obtained shifted vectors by computing, for each dimension, the average deviation from the additive model in the training data.


Derivations under different grammar formalisms allow extraction of various dependency structures.

bilexical deep dependency structures beyond surface tree representation can be derived from linguistic analysis grounded by CCG, LFG, and HPSG.


To tackle this problem, we integrate insightful ideas, especially the ones illustrated in Nivre (2009) and Gómez-Rodríguez and Nivre (2010), developed in the tree spanning scenario, and design two new transition systems, both of which are able to produce arbitrary directed graphs.

we explore two techniques to localize transition actions to maximize the effect of a greedy search procedure.


It is worth noting that our parsers exclude all syntactic and grammatical information.


, w n the following holds: The vertex set V consists of n + 1 nodes, each of which is represented by a single integer.

0 represents a virtual root node w 0 , and all others correspond to words in x.


In particular, 0 represents a virtual root node w 0 , and all others correspond to words in x.

the arc set A represents the labeled dependency relations of the particular analysis G. an arc (i, r, j) ∈ A represents a dependency relation r from head w i to dependent w j .


The complexity of naive spanning is θ(n 2 ), 1 because it does nothing to explore the topological properties of a linguistic structure.

the naive graphspanning idea does not fully take advantages of the greedy search of the transitionbased parsing architecture.


As with many other data-driven dependency parsers, we use a global linear model for disambiguation.

a discriminative classifier is utilized to approximate the oracle function for a transition system S that maps a configuration c to a transition t that is defined on c. More formally, a transitionbased statistical parser tries to find the transition sequence c 0,m that maximizes the following score Following the state-of-the-art discriminative disambiguation technique for data-driven parsing, we define the score function as a linear combination of features defined over a configuration and a transition, as follows: where φ defines a vector for each configuration-transition pair and θ is the weight vector for linear combination.


Such tree backbones can be utilized to train a tree parser which provides pseudo path features.

we introduce an algorithm to associate every graph with a projective dependency tree, which we call weighted conversion.


SRL graphs usually annotate only on verbal predicates and their nominalization, whereas the semantic graphs grounded by CCG and HPSG target all words.

sRL provides partial analysis and semantic dependency parsing provides full analysis.


In case syntactic information is not available, we propose a tree approximation technique to induce tree backbones from deep dependency graphs.

our technique guarantees that the automatically derived trees are projective, which is a necessary condition for a number of effective tree parsing algorithms.


r The first type of parser implements a shift-reduce parsing architecture and also uses beam search for practical decoding.

we compare our parser with the state-of-the-art CCG parser introduced in Xu, Clark, and Two stack Two stack (reverse) Parsing accuracy based on syntactic and pseudo tree features.


In this article, we are also concerned with improving tagging efficiency at test time.

we explore unlabeled data to transfer the predictive power of hybrid models to simple sequence models.


In particular, we explore unlabeled data to transfer the predictive power of hybrid models to simple sequence models.

hybrid systems are utilized to create large-scale pseudo training data for cheap models.


To improve POS tagging efficiency without loss of accuracy, we explore unlabeled data to transfer the predictive power of complex, inefficient models to simple, efficient models.

hybrid systems are utilized to create large-scale pseudo training data for cheap sequence models.


Huang, Harper, and Wang (2007) and Huang, Eidelman, and Harper (2009) present empirical studies of generative Chinese POS tagging.

evaluation of the SR-HMM model obtains state-of-the-art performance.


It is relatively easy to label this word, but its complement could be of a large size.

the usage of a word that is complex from one particular view is not necessarily complex from another.


The new features lead to substantial improvements over the strong supervised baseline.

the word clustering information bridges the gap between the local classifier and structured prediction models much.


First, the base models for combining are separately trained in their solution.

one key difference is whether to allow integration of base models at learning time.


On the other hand, the syntaxbased hybrid model can refine the POS tags returned by the same parser, and therefore improve the final parsing results.

by parsing twice, we can obtain better phrase-structure trees.


That means that all partial tag sequences with the same last two tags can be merged together.

at each decoding step, our decoder first generates all new partial tag sequences by labeling the next word, then the top-b sequences with different last two tags are collected for future prediction while others are thrown away.


We use three base models for combination, which is the minimum for performing voting.

the final tagging is the voting result of these three labels.


If a grammar is not in single-initial form, dual-initial rules can be converted to single-initial form by adding an empty span to B that combines with the first spans of C immediately to its left, as shown in Figure 2.

for each dual-initial rule A → B C, if the first span of C appears between spans i and i + 1 of B, create a new nonterminal B with ϕ(B ) = ϕ(B) + 1, and add a rule B → B, where B produces B along with a span of length zero between spans i and i + 1 of B.


We now proceed to show that upper-triangular matrices are sufficient in terms of the grammar.

we need to show the following lemma: For each application of a single-initial rule A → B C, it is possible to create an entry for A by multiplying two upper-triangular matrices T 1 and T 2 , where T 1 contains an entry for B, and T 2 contains an entry for C. A nonterminal B appears in a cell above the diagonal if its row address is smaller than its column address, which in turn occurs if the leftmost endpoint of B appears in the row address rather than the column address.


With T being the seed matrix, we define where T (i) is defined recursively as: Under Condition 4.1, one can show that given an LCFRS derivation tree t over the input string, each node in t must appear in the transitive closure matrix T + .

for each node in t representing nonterminal A spanning endpoints {( 1 , 2 ), ( 3 , 4 ), .


We can now ask the question: In which case the algorithm in Figure 8 is asymptotically more efficient than standard chart parsing techniques with respect to n?


Note, though, that because the texts in the ALC corpus are not tokenized, the string kernel approach is applied directly to the raw text documents.

the string kernel approach is evaluated on tokenized text documents as long as these documents are included in the corpus.


The regularization parameters of KRR and KDA were tuned such that an accuracy of roughly 98% is obtained on the training set.

the classifiers are allowed to disregard around 2% of the training documents to prevent overfitting.


The corpus has been used in various studies-for example, investigations related to e-mail classification (Klimt and Yang 2004) and structure of communication networks (Diesner and Carley 2005).

in studies on language and social dynamics, the Enron e-mail corpus has featured in analyses of power relationships (Diehl, Namata, and Getoor 2007;Gilbert 2012;Prabhakaran, Rambow, and Diab 2012b;Prabhakaran, Reid, and Rambow 2014), since Enron's organizational structure is known and can be integrated in studies on hierarchical power structures connected with quantitative capacity theories of power.


2013), can add new angles to the investigation of sociolinguistic variation.

the study of the interaction between (transcribed) speech, non-speech (laughter, sighs, etc.


We have aimed to provide a comprehensive overview of studies published within the field of CL that touch upon sociolinguistic themes in order to provide an overview of what has been accomplished so far and where there is room for growth.

we have endeavored to illustrate how the large-scale data-driven methods of our community can complement existing sociolinguistic studies, but also how sociolinguistics can inform and challenge our methods and assumptions.


Chapters 10-13 investigate the emerging themes from recent research and applications (e.g., analysis of debates, intentions, fake opinions, and review quality).

chapter 1 motivates and gives an overview of the whole book.


These two problems are the most practically useful research topics of sentiment analysis.

aspect-level analysis forms the core of applications of sentiment analysis as it aims to identify the atomic unit of information contained in sentiment, opinion, and emotion expressions, which is the pair of sentiment and its target.


present a proposal for combining a formal semantic framework, Type Composition Logic (Asher 2011), with Distributional Semantics.

types are recast in algebraic terms, including the addition of coercive or co-compositional functors that in the theory account for meaning shifts.


If someone states, in the middle of a conversation, that "This is not a dog," the negation strongly suggests a restricted set of alternative predicates that might hold true of the object being talked about.

other canids and middle-sized mammals are plausible alternatives, birds are less likely, skyscrapers and other large buildings virtually impossible.


The assumption of that work is that negated word meanings should be orthogonal, which is to say that they should not share any common feature.

widdows proposes a binary negation operator, NOT(A, B), which projects the vector representing A onto the orthogonal space of the B vector.


Conversational negation is something different from a logical truth-functional operator that flips the values of its argument.

in actual linguistic usage, the negation of a predicate often suggests that one of a set of alternatives might be holding.


The alternative possibilities primed by negation have been said to be "the most likely or relevant members of the complement set" (Oaksford 2002, page 140).

oaksford and Stenning (1992) identify several mechanisms used for constraining contrast classes (viz.


Judging alternatives based on short sentential context only is a challenging task, and we decided to exclude from further analysis those pairs that were not rated consistently by the subjects.

we discarded all pairs whose inter-subject variance was not significantly below chance at α = 0.1, based on simulated chance variance distributions.


In order to produce semantic representations for our target nouns in the relevant sentential contexts, we harness the compositional extension of DS.

we adopt the functional model independently proposed by Coecke, Sadrzadeh, and Clark (2010) and Baroni, Bernardi, and Zamparelli (2014).


To estimate the function parameters (matrix weights), we followed the approach of Guevara (2010), Baroni and Zamparelli (2010), and Dinu, Pham, and Baroni (2013), as implemented in the DISSECT toolkit.

11 we extracted from our corpus distributional vectors for sufficiently frequent phrases matching the target template (e.g., this/it is cat), and estimated the corresponding function by optimizing (in the leastsquares sense) the mapping from the vectors of the nouns in these phrases (cat) onto the corpus-extracted phrase vectors.


We think that, just like in standard approaches to semantics and discourse (Kamp and Reyle 1993;Ginzburg 2012), and as also recently advocated by McNally and Boleda (to appear), meaning in DS must be modeled in dynamic terms, that is, by considering how each utterance affects shared informational content as a coherent discourse unfolds.

we assume that part of the meaning of a linguistic expression is given by a probability distribution over possible linguistic expressions that might follow.


Ideally, specific properties of negation and other operators should be induced from corpus data.

the logical ("complement") constraint on negation might be captured by a non-linear scoring function that assigns very low probabilities to vectors that are too similar to the one of the negated element.


Inquisitive semantics, which ultimately stems from the analysis of questions by Groenendijk and Stokhof (1984), is devised as a logical system able to characterize (some) conversation moves, and it recently received a complete axiomatization (Ciardelli and Roelofsen 2011).

inquisitive semantics interprets a question as a partition of the set of possible worlds {w 1 , w 2 .


It is important to note that the Categorial framework makes no commitment to how the multi-linear algebraic objects are realized, only a commitment to their shape and how they are combined.

the vector spaces corresponding to atomic categories, such as noun and sentence, do not have to be distributional in the classical sense; they could be distributed in the connectionist sense (Smolensky 1990), where the basis vectors themselves are not readily interpretable (the neural embeddings that we use in this article fall into this category).


Based on the CCG category (S\NP)/NP of a transitive verb, this is a third-order tensor, S ⊗ N ⊗ N. The relative pronoun has a CCG type of (NP\NP)/(S\NP) or (NP\NP)/(S/NP), resulting in a fourthorder tensor, N ⊗ N ⊗ S ⊗ N. In order to make the learning of a relative pronoun tensor tractable, we make use of the method introduced by Paperno, Pham, and  and Polajnar, Fagarasan, and Clark (2014) in order to reduce the semantic type of the transitive verb to S ⊗ N. This method models a transitive verb as two second-order tensors (matrices), one governing the interaction of the verb with its subject and the other with its object, and has been shown to reproduce sentence semantics without loss of accuracy.

this "decoupling" approximation for the transitive verb allows the relative pronoun to be modeled as a third-order tensor, N ⊗ N ⊗ S. we adopt the approach of Paperno, Pham, and Baroni (2014).


We specify the types and internal contents .

for nouns and adjectives as shown here: noun types are functions from individual justification rules r ∈ I into a set of justifications that an individual satisfying r is of the type N (or ∅, if there is no such justification).


an adjective meaning takes a noun meaning and returns another noun meaning.


τ encodes the sequence of dependencies that lie along the path in t between the occurrences of w and w in t. In general, a path from w to w in t initially travels up towards the root of t (against the directionality of the dependency edges) until an ancestor of w is reached.


The string τ must, therefore, not only encode the sequence of dependency relations appearing along the path, but also whether each edge is traversed in a forward or backward direction.


However, in line with our decision to include white/JJ, , white/JJ rather than white/JJ, AMOD •AMOD, white/JJ , all co-occurrence types are canonicalized through a dependency cancellation process in which adjacent, complementary dependencies are cancelled out.

all occurrences within the string of either rr or rr for r ∈ R are replaced with , and this process is repeated until no further reductions are possible.


In our tree-based interpretation of APTs, nodes are associated with weighted multisets of lexemes.

w (τ) is thought of as a node that is associated with the weighted lexeme multiset in which the weight of w in the multiset is w (τ, w ).


In practice, the offset APT A δ can be obtained by prepending the inverse of the path offset, δ −1 , to all of the co-occurrence types in A and then repeatedly applying the reduction operator until no further reductions are possible.

if τ addresses a node in A, then τ addresses a node in A δ iff τ =↓ (δ −1 τ) and τ ∈ R * R * .


For each i, for 1 ≤ i ≤ n, the APT for w i when contextualized by its role in the dependency tree t, denoted w i ; t , is the APT that satisfies the equality: Alternatively, this can also be expressed with the equality: Note that w h ; t and t are identical.

we take the representation of the distributional semantics of a dependency tree to be the APT for the lexeme at the root of that tree that has been contextualized by the other lexemes appearing below it in the tree.


We begin with INT , which provides a tight implementation of the mutual compatibility of co-occurrences.

a co-occurrence is only deemed to be compatible with the composed lexemes to the extent that is distributionally compatible with the lexeme that it is least compatible with.


This corresponds to the multiset version of intersection.

for all τ ∈ R * R * and w ∈ V: It is clear that the effectiveness of INT increases as the size of C grows, and that it would particularly benefit from distributional smoothing (Dagan, Pereira, and Lee 1994), which can be used to improve plausible co-occurrence coverage by inferring co-occurrences in the APT for a lexeme w based on the co-occurrences in the APTs of distributionally similar lexemes.


An alternative to INT is UNI , where we determine distributional compatibility of a co-occurrence by aggregating across the distributional compatibility of the cooccurrence for each of the lexemes being composed.

for all τ ∈ (R ∪ R) * and w ∈ V: Although this clearly achieves co-occurrence embellishment, whether co-occurrence filtering is achieved depends on the weighting scheme being used.


Although our model shares with these the use of vector addition as a composition operation, the underlying framework is very different.

the actual vectors added depend not just on the form of the words but also their grammatical relationship within the phrase or sentence.


The distinction between vector spaces based on untyped and typed co-occurrences was formalized by Padó and Lapata (2007) and Baroni and Lenci (2010).

baroni and Lenci showed that typed co-occurrences based on grammatical relations were better than untyped cooccurrences for distinguishing certain semantic relations.


More recently, Thater, Fürstenau, and Pinkal (2011) used a similar approach but considered a broader range of operations for combining two vectors where individual vector components are reweighted.

they found that reweighting vector components based on the distributional similarity score between words defining vector components and the observed context words led to improved performance at ranking paraphrases.


For example, the verb catch will contain counts on the dimension for kick introduced by the direct-object ball and counts on the dimension for contract introduced by the direct-object cold.

as in the Erk and Padó approach, the vector for a verb can be seen as a vector of similar verbs, thus making this notion of second-order dependency compatible with that used in work on word sense discrimination (Schütze 1998) rather than referring to secondorder (or higher-order) grammatical dependencies as in this work.


The APT generated by a dependency tree, however, provides contextualized lexeme representations where the weights have been adjusted by the influence of the contextual lexemes so that the co-occurrences relating to the correct usage have been appropriately upweighted, and the co-occurrences found in other circumstances down-weighted.

aPT structures automatically perform word sense induction on lexemelevel representations and this is demonstrable through the lexeme similarity measure.


Thus, the phrasal vector for a red rose would be created taking the node containing rose as the anchor.

the vector representation of the phrase a red rose will be the same as the contextualized representation of rose.


Therefore, our approach combines logical and distributional methods.

we use first-order logic as a basic representation, providing a sentence representation that can be easily interpreted and manipulated.


All variables occurring in rT or rH are existentially quantified, all constants occurring in rT or rH are un-skolemized to new universally quantified variables, and we infer the negation of rH from rT.


The asymmetric features take inspiration from Mikolov, Yih, and Zweig (2013), who found that differences between distributional vectors often encode certain linguistic regularities, like king − man + woman ≈ queen.

the asymmetric classifier uses two sets of features, < f, g >, where: that is, the vector difference between the LHS and the RHS, and this difference vector squared.


We generally find the phrase features are much more likely to predict neutral, and the alignment-based features are much more likely to predict entailing.

the phrasal vectors seem to be much better at capturing non-entailment based on differences in prepositions (walk inside building walk outside building), additional modifiers on the RHS (man old man, room darkened room), and changing semantic roles (man eats near kitten kitten eats).


(2015) show several experiments suggesting that asymmetric classifiers do not perform better at the task of identifying hypernyms than when the RHS vectors alone are used as features.

they find that the asymmetric classifier and variants frequently learn to identify prototypical hypernyms rather than the hypernymy relation itself.


Still, there are many ways our lexical classifier could be improved, even using all of the features in the system.

it seems to do particularly poorly on antonyms (strike miss), and items that require additional world knowledge (surfer → man).


She is interested in natural language processing and machine learning techniques to understand both how people use language and build language-related technology.

she creates models that infer how documents and conversations are structured, and the impressions they make on the readers and listeners.


A second difference is that whereas, in our approach, syntax and aggregation choices are guided by a hypertagger trained to predict the best sequence of syntactic constructs for a given input, in Lampouras and Androutsopoulos the aim is to systematically minimize the length of the output, that is, to maximize aggregation.

we allow for various ways of aggregating a given content into different sentences and select one based on linguistic and semantic criteria, whereas Lampouras and Androutsopoulos (2013) select the aggregated sentences based solely on sentence length.


This language, called the language of tree-shaped conjunctive queries, is a minimal query language that is shared by most knowledge representation languages and is supported by Description Logic reasoners.

the Query Tool formal framework (Guagliardo 2009) defines a tree-shaped conjunctive query as a labeled tree whose edges are labeled with relations and whose nodes are labeled with a variable and a non-empty set of concept names.


In this way, the hypertagger makes high-level microplanning decisions and the grammar and the lexicon further refine those decisions by enforcing hard constraints such as the possible subcategorization pattern of a given literal (encoded in the lexicon)-for example, sDOTnx0VVVpnx1-or the choice (encoded in the grammar trees) between a comma-conjoined relative clause (COMMAW0nx0VVVpnx1) and a relative clause introduced by and (ANDW0nx0VVVpnx1).

we use the hypertagger to rank the syntactic construct sequences in terms of naturalness while the grammar and the lexicon are used to enforce hard lexical and grammatical constraints such as the fact that a comma-separated clause must be followed by a clause introduced by a coordination word (e.g., and or or).


Nowadays, although supervised methods perform better in general domains, unsupervised and semi-supervised models are receiving increasing attention from the research community, with performances close to the state of the art of supervised systems (Ponzetto and Navigli 2010).

knowledge-based and graph-based algorithms are emerging as promising approaches to solve the disambiguation problem (Sinha and Mihalcea 2007;Agirre et al.


In our case, this idea can preserve the textual coherence-a characteristic that is missing in many state-of-the-art systems.

it is missing in systems in which the words are disambiguated independently.


In this case, we use query expansion techniques in order to find an appropriate substitute (Carpineto and Romano 2012).

we use WordNet to find alternative lexicalizations of a lemma, choosing the one that co-occurs more frequently with the words in its context.


for our experiments, we used eight different measures: the Dice coefficient (dice) (Dice 1945), the modified Dice coefficient (mDice) (Kitamura and Matsumoto 1996), the pointwise mutual information (pmi) (Church and Hanks 1990), the t-score measure (t-score) (Church and Hanks 1990), the z-score measure (z-score) (Burrows 2002), the odds ration (odds-r) (Blaheta and Johnson 2001), the chisquared test (chi-s) (Rao 2002), and the chi-squared correct (chi-s-c) (DeGroot et al.


We used the WordNet version that has been used to label each data set.

the different implementations of the vector construction vary on the way in which the co-occurrence is calculated, the corpus used, and the source of the relations.


The weights in the co-occurrence graph are increased by the mean weight of the graph if a corresponding edge exists in the n-gram graph and does not include a stop-word.

10 8 we used the service provided by the Corpus Linguistics group at FAU Erlangen-Nürnberg, with a collocation span of four words on the left and on the right and collocates with minimum frequency: 100.


we notice that the pair of measures with highest results on both data sets is tfidf-mdice with n = 5.


This is important for metaphor identification, as our aim is to identify clusters of target concepts associated with the same source domain on one hand and to ensure that different metaphorical mappings are separated from each other in the overall partition on the other hand.

we use the NJW spectral clustering algorithm introduced by Ng et al.


The system then proceeds to search the respective corpus for source and target domain terms from the connected clusters within a single grammatical relation.

the system classifies verb-direct object and verb-subject relations in the corpus as metaphorical if the lexical items in the grammatical relation appear in the linked source (verb) and target (noun) clusters.


This is likely to be due to its rich derivational morphology.

in Russian, more lexical items can be used to refer to the same concept than in English or Spanish, highlighting slightly different aspects of meaning.


The results of our HGFC experiments also offer support to the cognitive science findings on the differences in organization of abstract and concrete concepts in the human brain (Crutch and Warrington 2005;Wiemer-Hastings and Xu 2005;Huang, Lee, and Federmeier 2010;Adorni and Proverbio 2012).

our experiments have shown that abstract concepts exhibit both within-domain and cross-domain association patterns (i.e., the literal ones and the metaphorical ones) and that the respective patterns can be successfully learned from linguistic data via the words' distributional properties.


Second, we bridge the gap between argumentation theories and argumentation mining by selecting the argumentation model based on research into argumentation theories and related fields in communication studies or psychology.

we adapt normative models from argumentation theory to perform empirical research in NLP and support our application of argumentation theories with an in-depth reliability study.


In the first study, we tackle the problem of relatively high "noise" in the retrieved data.

not all of the documents are related to the given topics in a way that makes them candidates for further deep analysis of argumentation (this study results into 990 annotated documents).


By contrast, macro-level models (also called dialogical models) and rhetorical models highlight the process of argumentation in a dialogue (Bentahar, Moulin, and Bélanger 2010, page 215).

we examine the structure of a single argument produced by a single author in terms of its components, not the relations that can exist among arguments and their authors in time.


Such devices typically include parenthetical verbs, syntactic constructions, token agreements, hedges, challenge questions, discourse markers, and tag questions, among others (Flores-Ferrán and Lovejoy 2015).

kaltenbök, Mihatsch, and Schneider (2010, page 1) define hedging as "a discourse strategy that reduces the force or truth of an utterance and thus reduces the risk a speaker runs when uttering a strong or firm assertion or other speech act."


We believe that future breakthroughs in natural language processing (NLP) applications will require developing synergies between machine learning techniques and bag-ofwords representations on the one hand, and in-depth theoretical accounts, together with detailed analyses of linguistic phenomena, on the other hand.

we investigate: r The complex lexical semantics of evaluative expressions, including semantic categorization and domain dependency.


It is clear from their results that using adjectives that can be classified as conveying appraisal values is beneficial in sentiment analysis.

and unsurprisingly, adjectives labeled as Appreciation are some of the most useful features for the classifier.


Although it was a breakthrough in terms of using sentiment-specific adjectives, and using linguistic insights (phrases rather than isolated adjectives; intensification and negation), the work of Whitelaw and colleagues did not explicitly utilize the information contained in the Appraisal groups.

whether a text contained more Appreciation than Judgment, for instance, was not part of the classification or the information extracted from the text.


In sentiment analysis, discourse structure provides a crucial link between the local sentence level and the entire document (article, conversation, blog post, tweet, headline) and is needed for a better understanding of the opinions expressed in text.

discourse can help in three main tasks: (1) identifying the subjectivity and polarity orientation of evaluative expressions; (2) furnishing important clues for recognizing implicit opinions; and (3) assessing the overall stance of texts.


(2002) were the first to empirically investigate positional features.

depending on the position at which a token appears (first quarter, last quarter, or middle half of the document), the same unigram is treated as different features.


Polanyi and van den Berg (2011) argue that sentiment is a semantic scope phenomenon.

discourse syntax encodes semantic scope; and because sentiment is a semantic phenomenon, its scope is governed by the discourse structure.


We firmly believe that future developments in sentiment analysis need to be grounded in linguistic knowledge (and also extra-linguistic information).

discourse and pragmatic phenomena play such an important role in the interpretation of evaluative language that they need to be taken into account if our goal is to accurately capture sentiment.


Later formulations of the problem, including Fung and Yee (1998) and Rapp (1999), used small seed dictionaries to project word-based context vectors from the vector space of one language into the vector space of the other language.

each position in contextual vector v corresponds to a word in the source vocabulary, 2 and vectors v are computed for each source word in the test set.


3 Finally, contextual similarities are calculated.

each projected vector is compared, using any vector comparison method, with the context vector of each target word.


Prefix contextual similarity is calculated in the same way as the contextual similarity score, but we use source and target word stems, or word prefixes up to five characters long, instead of full words.

the word prefix contextual similarity score for the word pair (blanco, white) is the same as that of (blanca, white).


Finally, we add a final feature indicating the target translation's monolingual frequency, which serves as a sort of prior probability that the target word is of interest at all.

we define this feature as the inverse of the log of the target word's frequency.


We measure performance using accuracy in the top-k ranked translations.

we define top-k accuracy over some set of ranked lists L as follows: where I lk is an indicator function that is 1 if and only if a correct item is included in the top-k elements of list l. top-k accuracy is the proportion of ranked lists in a set of ranked lists for which a correct item is included anywhere in the highest k ranked elements.


For the experiments in this article, we filter the dictionaries to include only high-quality translations.

we only use translations that have a quality score of at least 0.6 under the worker quality metric given by Pavlick et al.


Because our data are composed of news stories, each document also has an associated time stamp, which we use to define a rough document alignment with English news articles.

we treat the set of all foreign language news stories published on a particular day as roughly comparable to those written in English on the same day.


Intuitively, the signals that we defined in Section 2 seem to be orthogonal.

they provide very different types of information about how words are used in language, and we hypothesize that the lists of ranked candidate translations under each signal are uncorrelated with the exception and (hope!)


In order to empirically measure orthogonality of our signals, we measure pairwise Spearman rank-order correlation coefficients.

we first use each signal separately to rank all translation candidates.


We computed how frequently each signal ranks the correct translation higher than any other signal.

we computed how often each signal is a better predictor of how to translate a given word than all other signals.


We binned the words in each evaluation test set by frequency, and each bin contains 100 source-language words.

the most frequent 100 source-language words were put into the first bin, and the least frequent were put into the last bin.


As we did for the word frequency analysis, we bin the words in each evaluation set by burstiness, with each bin containing source words.

the 100 most bursty source-language words were put in the first bin, and the least bursty were put into the last bin.


We would not want to restrict models of bilingual lexicon induction to choosing only one sense, or one translation, for the French word enceinte.

the polysemy of words varies across languages and it is important to be able to account for this in any model of bilingual lexicon induction.


The most obvious way to integrate lexicon induction output into the SMT pipeline would be to induce translations for out-of-vocabulary and rare words.

if a word in our test set does not have a translation in the phrase table, we could induce one for it.


Concretely, let x t be the tth input vector and h t be the tth state vector.

the incremental calculation is: the vector x t is "read" in and a linear map is applied to it alongside the previous state vector h t−1 , then passed through a nonlinear function (here, a component-wise logistic sigmoid, denoted σ).


Word specificity, for nouns, can be approximated using WordNet (Miller 1995).

hypernym relationships between synonym sets form a hierarchal tree where the root (Entity) provides the most abstract sense.


We add these features because our recent work has found an acoustic component to AD (Fraser, Meltzer, and Rudzicz 2016).

for 3 rd -order recurrence quantification analysis with delay 2, neighborhood 0.5, we compute the mean recurrence rate, determinism, {mean, maximal, and entropy of} diagonal line length, laminarity, trapping time, maximal vertical line length, recurrence time of 1 st and 2 nd types, clustering coefficient, and transitivity over windows of length 1,000 samples (with a 500-sample window shift).


Interestingly, the average WordNet depth and mean noun familiarity are among the top five features, but make no appearance among the top five features selected from either the Control or Dementia sets separately.

the ratio of pronouns to all nouns correlates positively with the presence of TIBs (as one might expect, given the relatively generic nature of the former) with a correlation of 0.43.


For these experiments (and those in Section 6.3, where these clusters are used), we sweep the number of TIB clusters (excluding no-TIB) from 1 to 5.

one class is "no-TIB" and another is all remaining TIBs, combined, after isolating, iteratively, TIBs 7, 5, 1, and 6 in Dementia; TIBs 5, 6, {3, 4, and 7}, and {9 and 10} in Controls, and TIBs 4, 7, 1, and 6 in the combined set.


The results in Table 7 show that the learned POMDPs perform well based on simulation runs (shorter and more task-oriented conversations are preferred) and accuracy.

the results show that the Dementia POMDP performs better than all the baseline policies, in all cases at p < 0.05 given two-tailed t-tests with Bonferroni correction.


We focus on answering science exam questions, where many questions require complex inferences, and where building and evaluating answer justifications is challenging.

we construct justifications by aggregating Table 1 Categories of questions and their relative frequencies as identified by Clark et al.


We propose a method to construct answer justifications through information aggregation (or fusion).

we aggregate multiple sentences into hierarchical graph structures (called text aggregation graphs) that capture both intrasentence syntactic structures and intersentence lexical overlaps.


This introduces an additional complication during training: Although we know which answer candidate is correct, the actual quality of any given TAG is hidden.

many TAGs that connect a correct answer to the question do so in the wrong context, producing a poor justification.


For our reranking task, we found that a conservative interpretation of this at least one idea, where exactly one justification is chosen, performed the best.

9 P returns only the top TAG with the highest score according to the dot product with the current Θ Θ Θ.


First, we used the averaged percetron at inference time (Collins 2002).

instead of using the latest Θ Θ Θ after training, we averaged all parameter vectors produced during training, and used the averaged vector, Θ Θ Θ avg , for inference.


Second, we used a large-margin perceptron, similar to Surdeanu, Ciaramita, and Zaragoza (2011).

we update the model not only when the predicted answer is incorrect (line 5), but also when the current model is not confident enough-that is, when the predicted answer is correct, but the difference in F scores between this answer and the second predicted answer is smaller than a small positive hyper parameter τ.


Graphlet structure: Graphlets segment sentences based on clausal and prepositional boundaries to facilitate evaluating how well two sentences connect using structures larger than a word but more fine-grained than the entire sentence.

graphlets are the key structure in our representation of answer justifications because they drive both intra-and intersentence connections in a TAG.


There are two important aspects of the proposed architecture: a) The latent layer-that is, identifying good justifications to use for a given answer, which we implement by modifying stochastic gradient descent to keep track of the latent aspect.

to maintain the latent variable aspect of our ranking perceptron in our ranking neural network, we adapt Equation (1) to use a feed-forward pass through the network to determine the score of a given TAG (rather than the inner product of the features and the parameter vector), and continue to use the implementation of P that uses only the TAG with highest score.


Interestingly, the Simple Wicktionary contributes the fewest sentences to the good justifications for the CR system (7.5%), but for the TAG system it is the third largest contributor (14.8%).

whereas the CR system is typically unable to find a good justification from the Wiktionary, likely owing to its general nature, the TAG system is able to successfully aggregate these sentences with sentences from other domainspecific sources to build complete justifications.


This suggests that although less lexically connected answer justifications are not necessarily less numerous or of lower quality, they are much more challenging to detect given our current connectivity-centered features.

the current features are very good at Table 12 Proportion of good justifications with a given number of connecting word categories (Q, A, X) for both correct and incorrect answers (Top 6).


The inclusion of more information than required for disambiguation (e.g., to produce descriptions as in "the bird on top of the mailbox" in a context in which there is only one visible bird) is common in language use, and it is known to affect reference resolution in a number of ways.


From a computational perspective, these insights are potentially relevant in a number of ways.

we notice that inspection times may provide a suitable account of salience for hearer-oriented REG.


Parsers more powerful than those for CFGs often incur high time costs.

lCFRS parsers have a time complexity that is polynomial in the sentence length, but with a degree that is determined by properties of the grammar.


Grammars induced in this way can be seen as restricted hybrid grammars, in which no freedom exists in the relation between the string component and the tree component.

the presence of discontinuous structures generally leads to high time complexity of string parsing.


Section 7 reports on experiments that provide proof of concept.

lCFRS/sDCP-hybrid grammars are induced from corpora of dependency structures and phrase structures and employed to predict the syntactic structure of unlabeled sentences.


, t n to t 1 • • • t n -that is, omitting the angle brackets and commas.

for n = 0, the s-term is abbreviated by ε.


For any term t we have t| The label at position p in a term t is denoted by t(p).


The start symbol S has only one argument, which is synthesized-that is, rk N (S) = s-rk(S) = 1 and i-rk(S) = 0.

a rule is of the form: where The set of variables occurring in the lists x every variable from X m occurs exactly once in all these lists together.


the left-branching partitioning has internal node labels that are {1, 2, .


At the same time, the numbers of synthesized arguments in the induced sDCP may increase.

we witness a trade-off between the degree of mild context-sensitivity of the LCFRS and the numbers of arguments of the sDCP.


Each choice of values for the parameters determines an experimental scenario.

direct extraction in combination with strict labeling achieves traditional LCFRS parsing.


The second kind of model involves stack-based transition systems, with an added transition that swaps stack elements.

introduced a deterministic system that uses a classifier to determine the next transition to be applied.


This restriction limits the power of hybrid grammars.

discontinuous structures can now only be produced if the fanout is strictly greater than 1, which also implies the time complexity is more than cubic.


The "fishy/shady/illegal/dirty" component in the lexical meaning of the Chinese noun is missing in the lexical meaning of the English noun story.

strictly speaking, +{ and story have different lexical meanings.


For example, in Figure 1, the reason why we align VPe 3 with VPc 2 but not IP 2 is that we need to include the function word Š in the phrase alignment, because the very reason why the object of the verb ëZ/'call', namely, £/'that', appears before the verb is due to the presence of Š.

there is an important grammatical dependency between £ and Š, and excluding Š outside the phrase alignment that contains £ will sever this dependency.


For instance, the alignment between IP 1 and S 1 do not have translation divergence because both nodes have two immediate daughter nodes, and there is a one-to-one alignment between the two immediate daughter nodes in the same word order.

neither of the two conditions in the definition is met and therefore no translation divergence arises.


The Chinese localizer expresses the same lexical meaning as under in terms of spatial relation, but a localizer like is nominal in nature and therefore differs with under in terms of grammatical function.

the Chinese equivalent of the English under is neither ( nor but the string ( ... if we take both grammatical function and lexical semantics into consideration.


In (III), like is the main verb in English whereas its counterpart is an adverbial gern in German.

the original meaning is structurally paraphrased by the translation.


In these methods, probability distributions, such as the distribution over geographical locations for each linguistic variable, are embedded in a Reproducing Kernel Hilbert Space (RKHS).

we use the Hilbert-Schmidt Independence Criterion (HSIC; Gretton et al.


Another way in which kernel methods can be applied to spatial analysis is in Gaussian Processes, which are often used to represent spatial data (Cressie 1988;Ecker and Gelfand 1997).

we can define a kernel over space, so that a response variable is distributed as a Gaussian with covariance defined by the kernel function.


For the spatial kernel k Y , we use a Gaussian RBF, which is a widely used choice for vector data.

we define k Y (y i , y j ; γ) = exp(−γd 2 i,j ), where d 2 ij is the squared Euclidean distance between y i and y j , and γ is a parameter of the kernel function.


(2005a), we use a low-rank approximation to each Gram matrix, using the incomplete Cholesky decomposition (Bach and Jordan 2002).

we approximate the symmetric matrices K X and K Y as low-rank products, K X ≈ AA T and K Y ≈ BB T , where A ∈ R n×r A and B ∈ R n×r B .


We compare HSIC with specific instantiations of the methods described in Section 2, focusing on previous published applications of these methods to dialect analysis.

we consider the following methods: Moran's I We follow Grieve, Speelman, and Geeraerts (2011), using a binary spatial weighting matrix with a distance threshold τ, usually set to the median of the distances between points in the data set.


For each municipality i, we stochastically determine the number and location of the tweets as follows: Number of data points For each municipality, the number of tweets n i is chosen to be proportional to the population, as estimated by Statistics Netherlands.

we drawñ i ∼ Poisson(µ obs × population i ) and then set n i =ñ i + 1, ensuring that each municipality has at least one data point.


W and S can be interpreted as linear maps and a mapping between them is given by the rank 4 tensor E ∈ R |S|×n×|V|×n .

we can then write Equation 12as a tensor matrix product: and Equation (13) states, that Additionally, there is no interaction between different dimensions, so we are creating the tensor by stacking the diagonal matrices E (i,j) over i and j.


So E (d) ∈ R |S|×|V| should also be normalized over the first dimension.

e (d) should be a column normalized matrix.


Because of the writing guidelines used (Peldszus and Stede 2013, page 197), it also exhibits an unusually high proportion of attack relations.

97 of the 112 arguments (86.6%) include at least one attack relation.


The first approaches to studying argumentation date back to the ancient Greek sophists and evolved in the 6th and 5th centuries BCE (van Eemeren, Grootendorst, and Snoeck Henkemans 1996).

the influential works of Aristotle on traditional logic, rhetoric, and dialectics set an important milestone and are a cornerstone of modern argumentation theory.


This can be attributed to the identification of the correct targets of premises.

we observed that agreement on the targets decreases if a paragraph includes several claims or serial argument structures.


However, the role attribute is a unique feature of their corpus and the arguments in their corpus exhibit an unusually high proportion of attack relations.

86.6% of their arguments include attack relations, whereas the proportion of arguments with attack relations in our corpus amounts to only 12.4%.


This can be attributed to the "claim-centered" weight calculation in our objective function.

the predicted relations in matrix R are the only information about serial arguments, whereas the other two scores (c ij and cr ij ) assign higher weights to relations pointing to claims.


They show that correct predictions of one base classifier are not maintained after applying the ILP model if the other base classifier exhibits less accurate predictions.

less accurate argumentative relations have a more detrimental effect on the component types ( Figure 4a) than less accurate component types do on the outcomes of the relation identification (Figure 4b).


Although humans may be able to deduce the missing parts by interpreting the argument, existing argument mining methods fail on that task and may produce incomplete or even wrong argumentation structures.

the presented approach is not able to recognize gaps in reasoning (i.e., missing premises) or to infer the missing components of implicit arguments.


Recently, Mathet, Widlöcher, and Métivier (2015) introduced the new coefficient γ, which relies on different assumptions from the αs and thus better corresponds to computational linguistics annotation efforts.

whereas αs rely on the number of intersections between any pair of units from different annotators, γ builds and relies on an alignment between units that ultimately says which unit from one annotator corresponds to which unit from another annotator, if any.


It shares the same goal as cu α, the measure belonging to the αs dedicated to categorization of a continuum, but relies on the same assumptions as γ.

it shares the same alignment method, before it does a specific computation focused on categories.


Let D be the random variable defined as the function of dissimilarity between pairs of units from different annotators.

the categorial disorder δ we want to estimate is the average value taken by D for true pairs only, which formally corresponds to the conditional expectation of D given the event truePair, and is given by Equation (2): what we really get from an alignment is a "fuzzy set" of true pairs rather than a classical set, and the best estimate of δ we can get from this data is the weighted (by p i ) average value of {δ i }.


On the other hand, because it relies on PL, as discussed in Section 3, cu α provides an agreement value of 0.715, and confirms that it fails to generalize α, being here more conservative, but possibly less conservative with other data.

its observed disagreement is 0.218 instead of 0.2 for α and γ cat , which confirms a structural difference of how to take missing values into account.


A second point is that cu α does not show the same stability, and is lowered by the positional discrepancies.

from magnitudes 0.1 to 0.6, there is a difference of up to 0.06, which is not negligible.


However, when the agreement is not as good as is wished, the researchers would like to have more details about the discrepancies, in order to better understand the difficulties and thus to enhance the annotation model or the annotation manual.

is a given overall low agreement due to poor specification of categories?


Only Krippendorff proposes solutions, with his coefficients cu α and k α, but with different assumptions from the ones we posit for computational linguistics needs.

relying on intersections rather than on an alignment, these coefficients mostly compare quantities of occupied space rather than genuine units.


We also provide a detailed analysis of the relevance of various discourse elements and relations from the RST parse trees for machine translation evaluation.

we show that (i) all aspects of the RST tree are relevant, (ii) nuclearity is more useful than relation type, and (iii) the similarity of the translation RST tree to the reference RST tree is positively correlated with translation quality.


In this work, we focus on the first of the two previous research hypotheses.

we show that sentence-level discourse information can be used to produce reference-based evaluation measures that perform well on their own, but more importantly, can be used to improve over many existing MT evaluation metrics regarding correlation with human assessments.


Our hypothesis in this article is that the similarity between the discourse trees of an automatic translation and of a reference translation provides additional information that can be valuable for evaluating MT systems.

we believe that better system translations should be more similar to the human translations in their discourse structures than worse ones.


We calculated the metrics in this group using Asiya.

we used the following Asiya versions of TER and ROUGE: TERP-A and ROUGE-W. 8 For each metric in groups II and III, we present the system-level and segment-level results for the original metric as well as for the linear interpolation of that metric with DR and with DR-LEX.


Interestingly, all five variants of our discourse metric received moderately high weights, with the four variants using lexical information (DR-LEX's) being more important.

dR-LEX 1 has the fourth highest absolute weight in the overall combination.


For example, the frequency difference for the Elaboration tag between good and gold translation trees is 58, which is smaller than the difference between bad and gold, 323.

the trees for bad translations exhibit a surplus of Elaboration tags.


(2014) we presented our DiscoTK family of discourse-based metrics.

the DISCOTK party metric (discussed in Section 4.3) combined several variants of a discourse tree representation with other metrics from the ASIYA MT evaluation toolkit, and yielded the best-performing metric in the WMT14 Metrics shared task.


Both methods demonstrated state-of-the-art performance in several text correction competitions.

this article addresses the problem of correcting grammatical context-sensitive mistakes made by English as a Second Language (ESL) writers, a subject that has recently attracted significant attention (Izumi et al.


The matrices also give a sense of the error baseline.

as the numbers show, most words are used correctly, and the error rates are very low (the error rates can be estimated by looking at the matrix diagonals in the tables).


For example, from Table 2, label on corresponds to source on in 85.2% of the cases and corresponds to source in in 7.6% of the cases.

in 7.6% of the cases when the preposition on should have been used, the writer instead used in.


We use three phenomena-articles, prepositions, and verb agreementas these occur in the FCE data from 11 first languages (shown in Table 12), excluding languages that only have a handful of examples, and study these phenomena within the NB adaptation framework, training on the native data from the Web1T corpus and adapting using learner data from different first-language groups.

we denote by target, other, related, and unrelated, error patterns obtained from the learners of the same first-language background, all other first languages (excluding the target), languages related to the target, and those unrelated to the target, respectively.


Is there a benefit in collecting error patterns on this new set of learners for adaptation?

can we do better using error patterns from writers of the target language than when adapting using data from other speakers?


However, some languages do exhibit a consistent small improvement when target-language data is added to other-language data for adaptation.

these are the same languages that also exhibit an improvement when target-language data is used instead of other-language data.


adaptation (these results are in bold in the table), some language groups do better with adaptation based on other languages than on their own target data.

it is interesting to observe that preposition and verb agreement errors behave similarly, in that languages that do not benefit from target priors as much as from priors extracted from other languages are those that have less error evidence for prior estimation: for verb agreement and preposition errors there is a strong correlation between the amount of target error evidence and the ability to improve over priors estimated from other languages.


Depending on the type of training data available, a specific approach should be preferred.

classification systems can be trained from significantly less annotated data and can focus on specific errors, whereas MT-based systems do not have this capability.


Observe that the performance of the MT systems depends heavily on the size of the supervision.

when an MT system is trained only on the CoNLL corpus, the performance is quite poor (28.25).


Similarly to LSTMs, GRU networks introduce a mechanism that facilitates the retention of information over multiple time steps.

the GRU computes the hidden state at current time step h t , as the linear combination of previous activation h t−1 , and a new candidate activationh t : where is elementwise multiplication, and the update gate activation z t determines the amount of new information mixed in the current state: The candidate activation is computed as: The reset gate r t determines how much of the current input x t is mixed in the previous state h t−1 to form the candidate activation: where W, U, W z , U z , W r and U r are learnable parameters.


However, since at least the pioneering experiments on prototypes (Rosch 1973(Rosch , 1975, it has been known that, for a given semantic category, certain member concepts are consistently understood as more central to the category than others (even when controlling for clearly confounding factors such as frequency) (Coleman and Kay 1981;Medin, Altom, and Murphy 1984;Lakoff 1990;Hampton 2007).

wordNet and similar resources fail to capture the fact that category membership is a gradual semantic phenomenon.


Second, the graded LE relation also arises when one asks about the applicability of concepts to objects: The boundaries between a category and its instances are much more often fuzzy and vague than unambiguous and clear-cut (Kamp and Partee 1995).

the graded membership (often termed vagueness) measures the graded applicability of a concept to different instances.


The intuition supporting the latter hints that typical characteristics constituting the intension (i.e., concept) expressed by a hypernym (e.g., move or eat for the concept animal) are semantically more general than the characteristics forming the intension 7 of its hyponyms (e.g., bark or has tail for the concept dog).

superordinate concepts such as animal or appliance are semantically less informative than their hyponyms (Murphy 2003), which is also reflected in less specific contexts for hypernyms.


Therefore, an approximate order-embedding is sought: A mapping that violates the order-embedding condition, imposed as a soft constraint, as little as possible.

the penalty L for an ordered pair ( X, Y) of points/vectors in R N + is defined as: L( X, Y) = 0 implies that X Y, according to the reversed product order.