In these experiments, we see that MD results in joint parsing strategies (MDFirst, ArcGreedy) always improve upon the MD standalone/pipeline results.

all MD results across the joint strategies are very close.


Previous work on unlexicalized transition-based parsing models only focused on projective constituency trees (Dyer et al., 2016;Liu and Zhang, 2017).

cross and Huang (2016b) introduced a system that does not require explicit binarization.


We argue that an important consequence of the simplicity of unlexicalized systems is that their derivations are easier to learn.

mL-GAP derivations have a better incrementality than those of mL-GAP-LEX (Section 6.1) and are more economical in terms of number of GAP actions needed to derive discontinuous trees (Section 6.2).


In this paper, we introduce a synchronous bidirectional-neural machine translation (SB-NMT) that predicts its outputs using left-to-right and right-to-left decoding simultaneously and interactively, in order to leverage both of the history and future information at the same time.

we first propose a new algorithm that enables synchronous bidirectional decoding in a single model.


3) Compared with two-phase decoding scheme in previous work, our decoder is faster and more compact, using one beam search algorithm.

we make the following contributions in this paper: • We propose a synchronous bidirectional NMT model that adopts one decoder to generate outputs with left-to-right and rightto-left directions simultaneously and interactively.


• Extensive experiments on NIST Chinese-English, WMT14 English-German and WMT18 Russian-English translation tasks demonstrate that our SB-NMT model obtains significant improvements over the strong Transformer model by 3.92, 1.49, and 1.04 BLEU points, respectively.

our approach separately establishes the state-ofthe-art BLEU score of 51.11 and 29.21 on Chinese-English and English-German translation tasks.


In this paper, we build our model based on the powerful Transformer (Vaswani et al., 2017) with an encoder-decoder framework, where the encoder network first transforms an input sequence of symbols x = (x 1 , x 2 , ..., x n ) to a sequence of continues representations z = (z 1 , z 2 , ..., z n ), from which the decoder generates an output sequence y = (y 1 , y 2 , ..., y m ) one element at a time.

relying entirely on the multi-head attention mechanism, the Transformer with beam search algorithm achieves the state-of-the-art results for machine translation.


Formally, multi-head attention first obtains h different representations of (Q i , K i , V i ).

for each attention head i, we project the hidden-state matrix into distinct query, key, and value representations i , respectively.


where  Scaled dot-product attention can be described as mapping a query and a set of key-value pairs to an output.

we can then multiply query Q i by key K i to obtain an attention weight matrix, which is then multiplied by value V i for each token to obtain the self-attention token representation.


Gate Mechanism We also propose a gate mechanism to dynamically control the amount of information flow from the forward and backward contexts.

we apply a feed-forward gating layer upon − → H history as well as − → H f uture to enrich the nonlinear expressiveness of our model: where denotes element-wise multiplication.


Extensive experiments demonstrate the effectiveness of our proposed model.

our model respectively establishes state-of-the-art BLEU scores of 51.11 and 29.21 on NIST Chinese-English and WMT14 English-German translation tasks.


@ 1 RUM LSTM (a) (b) Story (abridged) The raccoon that topples your trashcan and pillages your garden may leave more than just a mess.

more likely than not, it also contaminates your yard with parasites -most notably, raccoon roundworms baylisascaris procyonis (...) true in varying degrees throughout North America, where urban raccoons may infect people more than previously assumed.


The proposed embedding form can be seen as a generalization over the input-label embeddings with a bilinear form, because its degenerate form is equivalent to the bilinear form of Equation (5).

this can be simply derived if we set one of the two nonlinear projection functions in the second line of Equation (9) to be the identity function (e.g., g out (•) = I), set all biases to zero, and make the σ(.)


Focusing on tone, we explore this claim from a computational perspective using a precisely defined notion of locality.

we use quantifier-free (QF) first-order (FO) logical transductions, which have been shown (Chandlee and Lindell, in prep.)


We can talk of a set of models of the same signature, where signature refers to a set S = {f 1 , ..., f n , R 1 , ..., R n } of named functions and relations.

we consider models of strings of the signature {p, s, P σ∈Σ }, where p and s are the predecessor and successor functions, respectively, indicating the order of positions in the string, and for every σ ∈ Σ there is a unary relation P σ indicating the labels of each position.


The analysis of emerging topics over the course of a document is related to a large number of research areas.

topic modeling (Blei et al., 2003) and TDT (Jin et al., 1999) focus on representing and extracting the semantic topical content of text.


• Penalizing beams for no − op operators: Another way of biasing the beams toward generating longer sequences, is by penalizing for the number of times a beam takes no − op as the action.

we reduce the beam score by a hyperparameter-controlled logarithmic factor of the number of no − op actions taken till now.


We propose to solve these problems by using basis vectors to produce basis-customized weights, as shown visually in Figure 2.

we use a trainable set of d dim basis vectors B = {b 1 , b 2 , ..., b d }, where dim is the dimension of the original weights.


Based on our analysis of DREAM, we argue that dialogue-based reading comprehension is at least as difficult as existing non-conversational counterparts.

answering 34% of DREAM questions requires unspoken commonsense knowledge, for example, unspoken scene information.


For convenience, we define the following notations, which will be referred in the rest of this paper.

let D s denote the turns spoken by speaker s = * denotes all the speakers.


We propose speaker embedding to better capture dialogue structure.

in the original framework, given an input context (u −k , .


For all the models, we conduct coreference resolution to determine speaker mentions of s Q based on simple heuristics.

we map three most common speaker abbreviations (i.e., ''M''; ''W'' and ''F'') that appear in dialogues to their eight most common corresponding mentions (i.e., ''man,'' ''boy,'' ''he,'' and ''his''; ''woman,'' ''girl,'' ''she,'' and ''her'') in questions.


In Table 10, we summarize the involved types of dialogue structure and general world knowledge in our approaches.

dialogue Structure we observe 1.4% drop in accuracy if we set the target speaker s Q to * for all questions when we apply dSW++.


Our neural sequence-to-sequence models utilize an encoder-decoder setup (Cho et al., 2014;Sutskever et al., 2014) with an attention mechanism (Bahdanau et al., 2015).

we employ the framework NEMATUS (Sennrich et al., 2017).


The extrinsic evaluations on a tweet classification task exhibit the model's ability to capture useful representations for microblog messages.

our model enables an easy combination with existing neural models for end-to-end training, such as convolutional neural networks, which is shown to perform better in classification than the pipeline approach without joint training.


To this end, we distinguish two latent components in the given collection: topics and discourse, each represented by a certain type of word distribution (distributional word cluster).

at the corpus level, we assume that there are K topics, represented by φ T k (k = 1, 2, .


Here we use rectified linear units (Nair and Hinton, 2010) as the activate functions.

the weight matrix of f φ T (•) (after the softmax normalization) is considered as the topic-word distributions φ T .


In topic modeling experiments, we consider the five topic model baselines treating each tweet as a document: LDA (Blei et al., 2003), BTM (Yan et al., 2013), LF-LDA, LF-DMM (Nguyen et al., 2015), and NTM (Miao et al., 2017).

bTM and LF-DMM are the state-of-the-art topic models for short texts.


It can be observed that our models exhibit generally better performance, showing the effectiveness of our framework in inducing interpretable discourse roles.

we observe the best results achieved by our joint model TOPIC+DISC, which is learned to distinguish topicand discourse-words, important in recognizing indicative words to reflect latent discourse.


As can be seen, the curves corresponding to the performance on topics and discourse are not monotonic.

better topic coherence scores are achieved given relatively larger topic numbers for TREC with the best result observed at K = 80.


To further understand why our model learns meaningful representations for topics and discourse, we present a case study based on the example conversation shown in Figure 1.

we visualize the topic words (with p(w | z) > p(w | d)) in red and the rest of the words in blue to indicate discourse.


Note that the additional requirements make the grammar explicitly ''split'' in the sense of Eisner and Satta (1999), Eisner (2000), and Johnson (2007).

the two processes of attaching left and right children, respectively, are independent, with rules (B, C) → a creating ''initial states'' B and C, respectively, for these two processes.


This requirement is encoded in the absence of a rule with right-hand side (S, P ).

(S, P ) cannot be part of a successful derivation, unless the rule (S, S) → (S, P ) S is subsequently used, which then corresponds to giving a i a right child that has a k among its descendants.


By properties (1)-(3), we can from here on ignore non-critical nodes, so that the remaining task is to calculate σ max − |B ncrit |.

in fact, we go further than that and calculate σ max − M , where we take for granted that the score can be at least as much as the number of gold edges within the remaining input, which leaves us with the task of counting the additional gold edges in the optimal computation.


(2017) integrate residual connections (He et al., 2016) into GCNs to help information propagation.

each node is updated according to Equation (1) first and then the resulting representation is combined with the node's representation from the last iteration: GCNs with Layer Aggregations.


Thus we assign different weights to information from different types of edges to integrate such information.

we concatenate the learned representations from all types of edges and perform a linear transformation, mathematically represented as: hidden is the weight matrix and T is the size of the edge type vocabulary and d hidden is the hidden dimension in DCGCN layers as described in Section 2.2. b f ∈ R d hidden is a bias vector.


Following previous work, we evaluate the results in terms of both BLEU (B) scores (Papineni et al., 2002) and sentence-level CHRF++ (C) scores (Popovic, 2017;Beck et al., 2018).

we use case-insensitive BLEU scores for AMR and case sensitive BLEU scores for NMT.


Our single model achieves 27.6 BLEU points, which is the new state-of-the-art result for single models.

our single DCGCN model consistently outperforms Seq2Seq models by a significant margin when trained without external resources.


When the parameter budget becomes larger, we can observe that the performance difference becomes more prominent.

the BLEU margins between DCGCN models and their best GCN models are 2.0, 2.7, 2.7, and 3.4, respectively.


Including an auxiliary ASR task is straightforward with the two-stage model by simply computing the cross-entropy loss with respect to the softmax output of the first stage, and dropping the second stage.

the auxiliary Mt task computes only the second stage, replacing the inputs s src 1:N by states e asr 1:N computed as: instead of computing the secondstage inputs using the first stage, we compute these inputs through a conventional encoder that encodes the reference transcript y transcr 1:N and uses the same embeddings matrix and unidirectional LStM as the first stage decoder.


We refine this hypothesis to propose that systems with many forms per paradigm have an even stricter limit on irregularity per distinct form.

the two dimensions interact: A system cannot be complex along both axes at once.


For example, m.past denotes the past-tense surface form in paradigm m. An inflected lexicon for a language can be regarded as defining a map M from lexemes to their paradigms.

m ( ).σ = w iff the lexicon contains the triple ( , σ, w).


Importantly, we define the size |m| of a paradigm m to be the number of distinct surface forms in the paradigm, rather than the number of slots.

|m| def = |range(m)| rather than |domain(m)|.


We model all of the conditional probability factors in equation 2using a neural sequence-tosequence model with parameters θ.

we follow Kann and Schütze (2016) and use a long short-term memory-based sequence-to-sequence (seq2seq) model (Sutskever et al., 2014) with attention (Bahdanau et al., 2015).


Visually, our low-entropy trade-off conjecture boils down to the claim that languages cannot exist in the upper right-hand corner of the graph, that is, they cannot have both high e-complexity and high i-complexity.

the upper-right hand corner of the graph is ''emptier'' than it would be by chance.


Ideally, we would like to also predict which morphosyntactic bundles are realized as words within a language, and which bundles are syncretic.

what paradigm shapes are plausible or implausible?


(2018) emphasize the role of syntax in argument identification rather than role labeling.

they develop an argument pruning algorithm that operates over dependency structures and selects argument candidates subject to a parameter determining their distance from the predicate.


Link Type Prediction Our aim is to capture how semantic predicates are linked to adjacent words in a sentence.

we are interested in predicting whether they are linked, and, if they are, what type of link they have.


In the present paper, however, we exploit known T values (as observed in the "depunctuated" version of a treebank).

because T is observed, we can jointly train ✓, to maximize just the p syn model that generated T becomes irrelevant, but we still try to predict what surface punctuation will be added to T .


Useful variants arise via well-known methods for weighted derivation forests (Berstel and Reutenauer, 1988;Goodman, 1999;Li and Eisner, 2009;Eisner, 2016).

to modify Algorithm 1 to maximize over T 0 values ( § § 6.2-6.3) instead of summing over them, we switch to the derivation semiring (Goodman, 1999), as follows.


Nunberg's hand-crafted schemata consider 16 punctuation types and generate a total of 192 edit rules, including the specimens in Table 1.

of the 16 2 = 256 possible underlying punctuation bigrams ab, 3 4 are supposed to undergo absorption or transposition.


First, however, we make the following critical point: It is possible for a model trained on (l (i) , q IID from p(l, q, d) to exceed the performance of a single annotator on this measure.

if we have a model p (l|q, d; θ), trained on (l, q, d) triples, which is a good approximation to p (l|q, d), it is then possible to use p (l|q, d; θ) to make predictions that outperform a single random draw from p (l|q, d).


10 This isn't quite accurate as the annotators are sampled without replacement; however, it simplifies the analysis.

11 for an input (q, d), if we define , d), then the Bayes optimal hypothesis is to output l * if P (h β (a, l * ) = 1|γ,γ) ≥ P (h β (a, NULL) = 1|γ,γ), and to output NULL otherwise.


An analogous strategy is used for the short answer model where examples with no entity answers are given a different weight.

13 we have a unique learned 10-dimensional embedding for each position 1 .


Our results confirm that the contextualized word embeddings perform better than the static ones.

we show that modeling context indeed contributes to recognizing meaning shift: On such tasks, contextualized models performed on par with humans.


We expect that improving the ability of such representations to reveal implicit meaning would require more than a language model training objective.

one future direction is a richer training objective that simultaneously models multiple co-occurrences of the constituent words across different texts, as is commonly done in noun compound interpretation (e.g., Ó Séaghdha and Copestake, 2009;Shwartz and Waterson, 2018;Shwartz and Dagan, 2018).


We are interested in a simpler version of the annotations.

we exclude the discontinuous spans (e.g., a span like   We experimented with six common word representations from two different families detailed below.


With respect to specific phrase types, near perfect performance was achieved among the more syntactic categories.

auxiliary (Did they think we were [going to] feel lucky to get any reservation at all?


We set the number of hidden units proportional to the input length (see Section 3.1 for configuration details).

we compute each attention score a att ij as follows: where att ∈ {w, p, m} is an attention type, W att and W att2 are the weights of the first and second network layers, b att and b att2 are their respective biases, s att ij is an intermediary representation, and σ is a sigmoid activation function.


This, however, is trivial for a computer.

a simple QA system easily solves the example in Figure 2 from seeing the reference to ''Un Bel Di''.


As one author explained, ''There were clues which I did not think were stock clues but were later revealed to be.''

the author's question about the Congress of Vienna used a clue about ''Kraków becoming a free city,'' which the model immediately recognized.


Using an NFH identification model that we describe in Section 4.2, we examined the distribution of NFH in different corpora and domains.

we examined monologues (TED talks; Cettolo et al., 2012), Wikipedia (WikiText-2 and WikiText-103;Merity et al., 2016), journalistic text (PTB: Marcus et al., 1993), and product reviews (Amazon reviews 3 ) in which we found that more than 35.5%, 33.2%, 32.9%, 22.2%, and 37.5% of the numbers, respectively, are NFHs.


16 However, the coreference-centric architecture had to be adapted to the particularities of the NFH task.

(a) the NFH resolution does not involve cluster assignments, and (b) it 14 The Reference cases were treated as a single class for computing the agreement score.


15 This is a somewhat simplified version of the full task defined in Section 3.

we do not require specification of the head in case of OTHER, and we require a single head rather than a list of heads.


6 We see a downward trend in the upset rate ( Figure 3).

the first and last quintiles show a statistically significant difference under a paired t-test (p < .001), meaning that a user's final Elo score is not a good measure of skill at earlier times.


We also inspect how well our model generalizes by measuring our model's accuracy on subsets of the test set.

we hope to see if high activity users or the topic of a debate affect our model's performance.


The latter, who used LSTM cells, also showed, qualitatively, that CNLMs are sensitive to hierarchical structure.

they balance parentheses correctly when generating text.


We use here the popular method of ''diagnostic classifiers'' (Hupkes et al., 2018).

we treat the hidden activations produced by a CNLM whose weights were fixed after language model training as input features for a shallow (logistic) classifier of the property of interest (e.g., plural vs. singular).


We also considered a more cogent evaluation regime, in which we split training and test data so that the number of boundary and non-boundary conditions are balanced, and there is no overlap between training and test words.

we randomly selected positions from the test partitions of the Wikipedia corpus, such that half of these were the last character of a token, and the other half were not.


In this work, we propose to use such graph structures for goal-oriented dialogues.

we compute the dependency parse tree for each utterance in the conversation and use a GCN to capture the interactions between words.


To capture interactions with nodes that are multiple hops away, multiple layers of GCNs can be stacked together.

the representation of node v after k th GCN layer can be formulated as: ∀v ∈ V. Here h k u is the representation of the u th node in the (k − 1) th GCN layer and h 1 u = x u .


We also perform experiments on two goal-oriented dialogue datasets that contain conversations between humans wherein the conversations were collected in a Wizard-of-Oz (WOZ) manner.

we use the Cam676 dataset (Wen et al., 2017), which contains 676 KBgrounded dialogues from the restaurant domain  1 2 L Rule-Based (Bordes et al., 2017) 33.3 − − − − − MEMNN (Bordes et al., 2017) 41.1 − − − − − QRN (Seo et al., 2017) 50.7 − − − − − GMEMNN (Liu and Perez, 2017) 48.7 − − − − − Seq2Seq-Attn (Bahdanau et al., 2015) 46.0 57.3 67.2 56.0 64.9 67.1 Seq2Seq-Attn+Copy  47.3 55.4 − − − 71.6 HRED (Serban et al., 2016) 48.9 58.4 67.9 57.6 65.7 75.6 Mem2Seq (Madotto et al., 2018) 45  (Budzianowski et al., 2018) dataset, which contains 10,438 dialogues.


We also performed ablations by removing specific parts of the encoder.

we experiment with (i) query encoder alone, (ii) query + history encoder, and (iii) query + KB encoder.


This situation is improved upon in the Richer Event Descriptions (RED;O'Gorman et al., 2016) and Situation Entities (SitEnt; Friedrich and Palmer, 2014a,b;Friedrich et al., 2015;Friedrich and Pinkal, 2015b,a;Friedrich et al., 2016) frameworks, which annotate both NPs and entire clauses for genericity.

sitEnt, which is used to annotate MAsC (Ide et al., 2010) and Wikipedia, has the nice property that it recognizes the existence of abstract entities and lexical aspectual class of clauses' main verbs, along with habituality and genericity.


It is important to note that these are all prototypical properties of episodic, habitual, or generic statements, in the same way that volitionality is a prototypical property of agents and change-ofstate is a prototypical property of patients.

our framework explicitly allows for bleed between categories because it only commits to the referential properties, not the categories themselves.


In ridit scoring, ordinal labels are mapped to (0, 1) using the empirical cumulative distribution function of the ratings given by each annotator.

for the responses y (a) given by annotator a, ridit y (a) y Ridit scoring has the effect of reweighting the importance of a scale label based on the frequency with which it is used.


In Table 3, we summarize the corpus statistics.

we use the CoNLL-U format 3 for the input files, take column 2 (FORM) as input, and predict columns 3 (LEMMA), 4 (UPOSTAG), and 6 (FEATS).


The sequence decoder generally performs better across different frequency ranges.

results confirm that the sequence decoder can generate some unseen tags correctly while the whole-tag classifier in principle cannot.


We assume the user specifies the domains they are interested in a priori (e.g., LAW, HEA) and the system returns summaries targeting the semantics of these domains.

we introduce DETRANK, an extension of the well-known TEXTRANK algorithm (Mihalcea and Tarau, 2004), which incorporates domain signals acquired by DETNET * .


For each document, TEXTRANK builds a directed graph G = (V, E) with nodes V corresponding to sentences, and undirected edges E whose weights are computed based on sentence similarity.

edge weights are represented with matrix E where each element E i,j corresponds to the transition probability from vertex i to vertex j.


In order to decide which sentence to include in the summary, a node's centrality is measured using a graph-based ranking algorithm (Mihalcea and Tarau, 2004).

we run a Markov chain withẼ on G until it converges to the stationary distribution e * where each element denotes the salience of a sentence.


Regarding contextual representation models, ELMo used the sampled softmax and GPT and BERT resorted to a subword method.

they used WordPiece (Wu et al., 2016) or BPE (Sennrich et al., 2016) to split the words into subwords and the language models were trained to take subwords as input and also predict subwords.


We assume all vectors (c and w) are normalized.

to optimize the inner product between c and w p(w|c)w, we essentially align the direction of context vector c with the expected word vector under context c, w p(w|c)w = E w∼p(w|c) w. given a word embedding matrix W ∈ R V ×d , our approach aligns c with the corresponding row (AW ) c,: in AW .


Therefore, we introduce two additional baseline models (ELMO-A and ELMO-Sub), which differ from ELMO-C in a minimal way.

their sequence encoders and training recipes are kept the same as ELMO-C.


ELMO-Sub provides subword-level contextual representations, which we suspect could be the cause of unstable performance.

to get the representation for a word in evaluation on word-level tasks, we follow Devlin et al.


These judgments agreed with CoLA's ratings on 87% of sentences with an MCC of 0.713.

13% of the labels in CoLA contradict the observed majority judgment.


Third, we transfer features learned from the real/fake task to classifiers trained on CoLA.

we freeze the weights of the sentence encoder portion of the real/fake classifiers, and train new classifiers on CoLA using the sentence embeddings as input.


We also take care to make the test sentences as simple as possible to reduce classification errors unrelated to the target contrast.

we limit noun phrases to 1 or 2 words and use semantically related vocabulary items within examples.


To overcome the intractability of the argmax operation we use an approximation and experimentally demonstrate its efficacy.

we introduce a Gibbs-perturbation model: a model that augments a given machine learning model with an additive or multiplicative Gaussian noise function (Keshet et al., 2011;Hazan et al., 2013).


In such cases it is likely that a diverse list of high-quality solutions will be valuable.

we experiment with the Universal Dependencies (UD) Treebanks (Nivre et al., 2016;.


Instead, we propose to approximate the marginals by sampling K times from the model and computing the marginals using a maximum-likelihood approach on this sample.

in our first-order dependency parsing example we set μ e to be the number of trees in the K-list that contain the edge e. As noted above, the idea of computing an MST over single-edge marginals has been proposed in Kuncoro et al.


Likewise, Figure 1 (bottom) demonstrates that the percentage of trees that fall into higher 10% UAS bins is substantially higher for MLN and ALN compared to K-best inference (the figure considers all the K-lists from the 72 test sets).

the perturbated lists are of higher quality than the K-best lists both when the oracle solution is considered and when the full lists are evaluated.


Our results indicate a clear advantage for the perturbated model.

for all 11 target corpora it is the final solution of this model that scores best.


The result pattern we observe is very similar to the cross-lingual parsing with gold POS tags, although the absolute numbers are lower.

the averaged UAS of the final solution of our model is 29.8, compared to 26.7 for K-best and 28.1 for 1-best.


We encourage exploration by injecting noise during beam-search (Cho, 2016).

we found it effective to keep the dropout on (e.g., dropout = 0.1).


For example, having raters choose among discrete labels and taking a majority vote (Dagan et al., 2006;Bowman et al., 2015;Williams et al., 2018), or having raters use a continuous Likert scale and taking an average (Pavlick and Callison-Burch, 2016a;Zhang et al., 2017).

the prevailing assumption across annotation methods is that there is a single ''true'' inference about h given p that we should train models to predict, and that this label can be approximated by aggregating multiple (possibly noisy) human ratings as is typical in many other labelling tasks (Snow et al., 2008;Callison-Burch and Dredze, 2010).


The goal of this study is to establish whether such disagreements can safely be attributed to ''noise'' in the annotation process (resolvable via aggregation), or rather are a reproducible signal and thus should be treated as part of the NLI label assigned to the p/h pair.

our primary contributions are: • We perform a large-scale study of humans' sentence-level inferences and measure the degree to which observed disagreements persist across samples of annotators.


The task of RTE/NLI is fundamentally concerned with drawing conclusions about the world on the basis of limited information, but specifically in the setting when both the information and the conclusions are expressed in natural language.

given a proposition p, should one infer some other proposition h to be true?


Even if NLI is treated as a gradable phenomenon (as we believe it should be), a world in which all humans share the same notion of uncertainty necessitates very different models, annotation practices, and modes of evaluation than a world in which people may disagree substantially in specific situations, use different heuristics, and/or have different preferences about how to resolve uncertainty.

current practicesin which we aggregate human judgments through majority vote/averaging and evaluate models on their ability to predict this aggregated label-are only appropriate if humans all tend to use the same process for resolving uncertainties in practice.


(2018) and remove workers who demonstrate consistently low correlations with others' judgments.

for each sentence pair s, for each worker w i , we compute the Spearman correlation between w i 's labels and 4 Raters do not see specific numbers on the slider.


For each sentence pair, we randomly split the collected human labels into train and test.

we hold out 10 labels from each pair to use as our test set.


If the data were in fact sampled from an underlying distribution defined by a single Gaussian, we would expect the points to be distributed approximately randomly around the y = x line.

most of the time the GMM would provide no advantage over the single Gaussian.


We consider each of these sentences to be a candidate premise (p), and generate a corresponding hypothesis (h) by replacing a word w 1 from p with a substitute w 2 , where w 2 has a known lexical semantic relationship to w 1 .

we use as set of 300 word pairs: 100 hypernym/hyponym pairs, 100 antonym pairs, and 100 co-hyponym pairs.


Another natural question arising from the analysis presented thus far is whether the phenomenon under investigation even poses a problem for NLP systems at all.

whether or not humans' judgments can be summarized by a single aggregate label or value might be a moot question, since state-of-the-art models do not, in practice, predict a single value but rather a distribution over values.


Thus, in order to interface with existing NLI models, we discretize 10 our collected human judgments by mapping the raw (unnormalized) score (which is between −50 and 50) into K evenly sized bins, where K is equal to the number of classes that were used in the original dataset from which the p/h pair was drawn.

for pairs drawn from datasets which use the three-way ENTAILMENT/ CONTRADICTION/NEUTRAL labels (i.e., SNLI, MNLI, and RTE2), we consider human scores less than −16.7 to be CONTRADICTION, those greater than 16.7 to be ENTAILMENT, and those in between to be NEUTRAL.


We use pretrained BERT (Devlin et al., 2019), 12 fine-tuned on the training splits of the datasets from which our test data was drawn.

we fine-tune BERT five times, once on each dataset, and then test each model on the subset of our re-annotated p/h pairs that were drawn from the dataset on which it was fine-tuned.


We focus only on p/h pairs on which all three label sources (i.e., the original label provided by the official dataset, the new label we produce by taking the majority vote of our newly collected, discretized human judgments, and the model's prediction) agree.

because we want to evaluate whether the model captures the distribution (not just the majority class that it was trained to predict) we want to focus only on cases where it at least gets the majority class right.


(2006), NLI was proposed as an intermediate task for evaluating whether a model will be useful in applications, and currently, NLI is increasingly used as a means for ''probing'' neural models to assess their knowledge of arbitrary linguistic phenomena (Dasgupta et al., 2018;Ettinger et al., 2018;Poliak et al., 2018b;Poliak et al., 2018a;McCoy et al., 2019).

nLI has largely become an evaluation lingua franca through which we diagnose what a semantic representation knows.


Instead, we attempt to design tasks which capture humans' linguistic behavior in as natural a setting as possible (acknowledging that truly natural annotation is difficult) with the hope that models trained to perform such tasks will be the best match for the ''real world'' settings in which we hope to deploy them.

we generally prefer to punt on precise definitions, and instead train our models to ''do what humans do''.


Similar approaches have been applied in parsing (Martínez Alonso et al., 2015) and supersense tagging (Martínez Alonso et al., 2016).

relevant to this work is past discussion of disagreement on semantic annotation tasks, including anaphora resolution (Poesio and Artstein, 2005), coreference (Versley, 2008;Recasens et al., 2011), word sense disambiguation (Erk and McCarthy, 2009;Passonneau et al., 2012;Jurgens, 2013), veridicality (Geis andZwicky, 1971;Karttunen et al., 2014;de Marneffe et al., 2012), semantic frames (Dumitrache et al., 2019), and grounding (Reidsma and op den Akker, 2008).


Our goal is to develop a self-training algorithm that can substantially enhance DNN models in cases where labeled training data are scarce.

we are focusing ( § 5) on the lightly supervised setup where only a small in-domain labeled dataset is available, and on the domain adaptation setup where the labeled dataset may be large but it comes from a different domain than the one to which the model is meant to be applied.


We consider tagging schemes that maintain various aspects of the structural information encoded in the tree.

we start from two tagging schemes that even if fully predicted still leave ambiguity about the actual parse tree: the number of direct dependants each word has and the distance of each word from the root of the tree.


However, its performance substantially degrades in domain adaptation.

in 5 out of 6 OntoNotes setups and in 9 out of 10 UD setups it is the worst performing model.


Clearly, the positive impact is substantially stronger for smaller training sets.

when the parser is trained with 100 sentences (the green bar) the improvement is higher than 5 UAS points in 6 of 7 cases, among which in 2 (nw and wb) it is higher than 8 UAS points.


The resulting R 2 values are well aligned with the relative quality of the DCST models.

dCST-LM, the least efficient method where the tagger is trained as a language model rather than on a syntactic signal, has an R 2 of 0.03.


Our method for extracting sentences for PSTS is inspired by bilingual pivoting (Bannard and Callison-Burch, 2005), which discovers samelanguage paraphrases by ''pivoting'' over bilingual parallel corpora.

if the English phrases coach and trainer are each translated to the same Slovenian phrase trener in some contexts, this is taken as evidence that coach and trainer have approximately similar meaning.


Our primary question is whether automatically extracted PSTS sentences for a paraphrase pair truly reflect the paraphrase meaning.

for sentences like s bug where s bug ∈ S bug,virus , does the meaning of the word bug in s bug actually reflect its shared meaning with virus?


We calculate precision under each model by soliciting human judgments, via the same crowdsourcing interface used to gather sentence annotations in Section 4.1.

for each of 40 hand-picked polysemous target words t (10 each nouns, verbs, adjectives, and adverbs), we select two paraphrases p and ask workers to judge whether t takes on the meaning of p in the top-10 PSTS sentences from S t,p as ranked by REG or C2V.


We begin by extracting an inventory of sentences pertaining to WordNet senses using IMS.

a pre-trained, off-the-shelf version of IMS 7 is used to predict WordNet 3.0 sense labels for instances of the same target nouns present in the PSTS-derived training set.


The model adopted for predicting hypernymy in context is a fine-tuned version of the BERT pre-trained transformer model (Devlin et al., 2019) (Figure 4).

we use BERT in its configuration for sentence pair classification tasks, where the input consists of two tokenized sentences (c t and c w ), preceded by a [CLS] token and separated by a [SEP] token.


But the field of computational linguistics has taken an interesting turn: Linguists and computational linguists who formerly used made-up sentences are now using naturally-and experimentally-generated corpora on which to base and test their theories.

the actual forms of speech and text corpora are shaped by how people balance and trade off these costs in the context of communication.


Matchers' icons got closer to the target over time, but not at a steady rate.

in the following example, A1 displayed a lower FOK than A2: Likewise, non-answers (e.g., I don't know) after a filler or a long latency were produced by speakers who were more likely to recognize the correct answers later on a multiple choice test; those who produced a non-answer immediately did not know the answers.


In order to see the magnitude of the work and di culties involved, we chose a wellcircumscribed eld and collected texts MED-LINE abstracts in the eld to be annotated.

will contribute to resolving e ciency problems.


The classi cation scheme, in turn, re ects the view of the elds that biochemists have.


They also adopted sophisticated plan-based dialogue models as well at the initial stage of the project.

defects in knowledge often cause failures in processing, which result in the fragile behavior of systems.


Speech recognisers has reached the high-street store, and a significant proportion of the developed world's population has now been exposed to the possibility of controlling one's computer, or creating a document, by v oice.

spoken language understanding continues to be an elusive goal, and the prosodic linkage between acoustic and linguistic patterning is still something of a mystery.


The test data was marked by hand tagging the time expressions and assigning value to them where appropriate.

our work in this area is highly preliminary.


More than 76% of the adjective pair types occur only once, and 49% of the adjective types only occur once.

besides information about the morphological form of the adjectives in the pair, we also included the positional probabilities P( a, x |{a, x}) and P( b, x |{b, x}) as real-valued features.


It should be noted that the direct evidence method as employed here is slightly different from Shaw and Hatzivassiloglou's: we simply compare raw token counts and take the larger value, while they applied a significance test to estimate the probability that a difference between counts arose strictly by chance.

i am indebted to Carol Bleyle, John Carroll, Ann Copestake, Guido Minnen, Miles Osborne, audiences at the University of Groningen and the University of Sussex, and three anonymous reviewers for their comments and suggestions.


When training trigram model (3), we treat those baseNP rules in two ways.

they did not consider any lexical information.


To summarize, the evaluation framework just described supports users in performing a realistic task at their own pace by interacting with an IDEA system.

the argumentation strategy has been implemented as part of a complete argument generator.


When applied to an entity, this function returns the value of the entity with respect to that objective.

then, we present an evaluation framework in which the effectiveness of evaluative arguments can be measured with real users.


(o,opop,e, refo) ≡ s-compellingness(o, e, refo)>µ x +kσ x , where − o, e, and refo are defined as in the previous Def; opop is an objective population (e.g., siblings(o)), and opop>2 − p∈ opop; x∈X = s-compellingness(p, e, refo) − µ x is the mean of X, σ x is the standard deviation and k is a user-defined constant Similar measures for the comparison of two entities are defined and extensively discussed in (Klein 1994).

by setting the constant k to different values, it is possible to control in a principled way how many objectives (i.e., pieces of evidence) are included in an argument, thus controlling the degree of conciseness of the generated arguments.


Hmeidi et al (1997) developed an information retrieval system with an index, but does not explain the underlying stemming algorithm.

arabic morphology is excruciatingly complex (the appendix attempts a brief introduction), and root identification on a scale useful for IR remains problematic.


I will also not adopt the Strict Layer Hypothesis (Selkirk, 1984) which holds that elements of a given prosodic category (such as Intonational Phrase) must be exhaustively analysed into a sequence of elements of the next lower category (such as Phonological Phrase).

in the longer term, the intention is to integrate prosodic realisation within the framework of an HPSG-based concept-to-speech system.


Zwicky's constraints on when this combination can occur are as follows: (15) A personal pronoun NP can form a prosodic phrase with a preceding prosodic host only if the following conditions are satisfied: a. the prosodic host and the pronominal NP are sisters; b. the prosodic host is a lexical category; c. the prosodic host is a category that governs case marking.

i call this type full, and it subsumes both Prosodic Words (of type p-wrd) and metrical trees (of type mtr).


The only complex prosodic objects will be metrical trees.

i will briefly sketch the treatment implemented in the ALE grammar, which does not build a representation of daughters.


We aim to derive a probability distribution p(y) on syllables y from a large sample.

results for the best starting values are shown in Figure 4.


Compared to the commercial product, our system is up to 50% lower in error rate at the same memory size, and about 76% better without memory limits at all (Jianfeng etc.

error Rate Perfect Input 6.82% Actual Input 20.84% Table 3.1 system without spelling correction In the actual input data, approximately 4.6% Chinese characters are typed incorrectly.


We show the results in Figure 1, where we used the error model with positional information and with a maximum context window of four, and used the language model to rescore the 5 best word candidates returned by the error model.

p(ent | ant) is highly dependent upon position.


Put another way, only a small number of documents actually contain the answer to a given query, while every document can in principle be summarized with respect to that query.

we know of no such publicly-available collection.


But this strategy suffers from a sparse estimation problem.

these weights were calculated using shrinkage.


Learning parameter values for the proposed model requires a large collection of summarized documents, which we do not have, but as a proxy, we use a collection of FAQ (frequently-asked question) documents.

in the call-center data the document containing F will appear much more relevant than the corpus as a whole.


uation, and plan to do so with our statistical approach as well, once the model is producing summaries that might be competitive with alternative approaches.)

by definition, the translation during summarization is lossy, and consequently, somewhat easier to design and experiment with.


Every solution of a constraint is a solution of one of its irredundant solved forms.

in each application, we can pick one of the disjuncts on the right-hand side.


This information was the used to find violations of Constraint 1 and Rule 1.

(The latest version of the annotation manual is available from the GNOME project's home page.)


F urthermore, by at least one measure, these software systems evaluate student essays with the same degree of accuracy as human experts.

he had frequented the store for many years.


(4) Pronouns Following (Gundel, et al., 1993), we consider that the evoking power of each of these types of REs decreases as we move down the list.

note that in some cases, the stack-based model performs better than VT, in particular for small k. This occurs when VT searches back through n adjacent left satellites, where n > 1, to find a coreference, but a co-referent is found using the stack-based method by searching back m nonleft satellite units, where m < n. This would be the case, if for instance, VT first found a coreferent for Mr. Clinton In text (4) in D4 (2 units away), but the stack-based model found a coreferent in C4 (1 unit away since the left satellites are not on the stack).


If we compute the vein expressions for both representations, we see that the vein for segment C6 in the intentional representation is <A6 B6 C6>, whereas in the rhetorical representation, the vein is <(B6), C6>.

pronouns have very poor evoking power, and therefore a message emitter employs them only when s/he is certain that the structure of the discourse allows for easy recuperation of the antecedent in the message receiver's memory.


Some were filtered out if they either lied outside an empirically defined search space, named a corridor, or were "not in line" with their neighbours.

the size of the classes is given by (maximum distance -minimum distance) / number of classes.


DSG can generate the language count-k for some arbitrary k that is, fa 1 n a 2 n : : : a k n g, which makes it extremely powerful, whereas R-MCTAG can only generate count-4.

the meta-level grammar G 0 is de ned so that it derives trees that look like derivation trees of G:  An auxiliary tree may adjoin anywhere.


4 Against tradition, we forbid root adjunction, because adjunction at the foot ensures that a bottom-up traversal of the derived tree will encounter elementary trees in the same order as they appear in a bottom-up traversal of the derivation tree, simplifying the calculation of derivations.

for a given application of this rule, it is not possible to say which tree is adjoining into which without examining the rest of the derivation.


Whether the improvements in PIQ will lead to the improvement of parsing accuracy or not?

in building a model, the first feature type to be selected is the feature type which has the largest predictive information quantity for the prediction of the derivation rule among all of the feature type candidates, that is, Where Ω is the set of candidate feature types.


z Conclusion Either of the predictive information quantity of the first word and the second word in the objective word sequence is larger than that of the headword of the parent node which has the largest predictive information quantity among all of the history feature type candidates.

according to the above idea, we build the information-theory-based feature type analysis model, which is composed of four concepts: predictive information quantity, predictive information gain, predictive information redundancy and predictive information summation.


The experiment is conducted on the history feature types of the nodes whose structural distance to the current node is within 2.

from the linguistic viewpoint, the physical position information can hardly grasp the relations between the linguistic structures.


Accuracy is somewhat similar to the familiar metric of precision in that it is calculated over cases for which a decision is made, and performance is similar to recall in that it is calculated over all true frame elements.

roughly 5 of the examples were identi ed as passive uses.


Another application is in wordsense disambiguation, where the roles associ-ated with a word can be cues to its sense.

occurrences of each role and each set of conditioning events were counted in a table, and probabilities calculated by dividing the counts for each role by the total number of observations for each conditioning event.


Both components work together in a unified way, and highly improve the productivity of English writing.

we use a wordlist, which consists of 65502 words, in conjunction with an optimization procedure described in [Gao, 2000].


(kono-hon) ", " (sono-hon) ", " (ano-hon) " indicates a book located either in the territory of the locuter, the listener, or outside the both, respectively.


And such a phrase in its turn can semantically modify an autonomous word by way of attaching a connective to it's right, forming a phrase, or inflecting the head word of the modifier.


The establishment of Computer Science departments in Indonesian universities that did not start until the beginning of 1980's may be partially responsible for this 1 .

we need to develop a systematic understanding of the Bahasa Indonesia (the Indonesian national language) to enable us develop the needed computer applications that will help us manage information intelligently.


Unlike English, where the role of suffix dominates the generation of derived-words, Bahasa Indonesia depends on both prefix and suffix to derive new words.

using this approach, we have developed grammar that understands declarative sentences (Shavitri, 1999).


My thanks go to my students for the realization of Vietools and my colleagues for their opinions.

there are many challenging problems for Vietnamese language processing.


Some work has also started on building search engines.

besides the parsers mentioned above, a parsing formalism called UCSG identifies clause boundaries without using sub-categorization information.


links.nectec.or.th/services/parsit), it is a service of English-to-Thai webpage translation.

other approaches such as generate-and-repair [7] and sentence pattern mapping have been also studied [8].


The 3 billion base pair sequence of the human genome is now available, and attention is focusing on annotating it to extract biological meaning.

i will discuss approaches using stochastic grammars analogous to those used in computational linguistics, both for gene finding and protein family classification.


Large vocabulary continuous speech recognition (LVCSR) is a key technology that can be used to enable content-based information access in audio and video documents.

for each speech segment, the word recognizer determines the sequence of words in the segment, associating start and end times and an optional confidence measure with each word.


During the IIS, every sample was tagged using the grammar estimated above, and then the grammatical features were extracted, before combining them with other kinds of features.

iiS is based on the change of the log-likelihood over the training corpus k , when each of the parameters changes from & .


The main obstacle in the WSME training process resides in the calculation of the first sum in (8).

the & represent the contribution of each feature to the distribution.


By training a set of classifiers on a single training corpus and then combining their outputs in classification, it is often possible to achieve a target accuracy with less labeled training data than would be needed if only one cla ssifier was being used.

as can be seen, the greater the classifier agreement, the more likely it is that a test sample has been correctly labeled.


At this point, test set accuracy begins to decline as additional training instances are automatically harvested.

we see that the larger the pool of candidate instances for annotation is, the better the resulting accuracy.


The boosting indicators assign a positive score to an NP, reflecting a positive likelihood that it is the antecedent of the current pronoun.

although for the current experiments we have only included three knowledge-poor anaphora resolvers, it has to be emphasised that the current implementation of the workbench does not restrict in any way the number or the type of the anaphora resolution methods included.


Of course, that grammar uses the TAG formalism.

these savings even allow to consider the parsing of medium size sentences with the English XtAG.


The first one deals with relative efficiency: it assumes that problem $ can be solved more efficiently than problem .

for the tree selection by the words in a sentence, the precision measured on our sam- 7 The time taken by the lexer phase is linear in the length of the input sentences and is negligible.


A novel feature of our approach is the ability to extract multiple kinds of paraphrases: Identification of lexical paraphrases.

training of the classifier demands negative examples as well; in our case it requires pairs of words in aligned sentences which are not paraphrases of each other.


In this section, we briefly recall the definition of the constraint language for ¡ -structures (CLLS).

then we use a saturation calculus to make the description more explicit.


They are based on dominance constraints (Marcus et al., 1983;Rambow et al., 1995) and extend them with parallelism (Erk and Niehren, 2000) and binding constraints.

3  Having introduced -reduction constraints, we now show how to process them.


The evaluations also provide evidence that this method for automated thesis selection in essays is generalizable.

the agreement figures are given using the kappa statistic and the relative precision (P), recall (R), and F-values (F), which reflect the ability of one judge to identify the sentences labeled as thesis statements or summary sentences by the other judge.


A thesis statement is defined as the sentence that explicitly identifies the purpose of the paper or previews its main ideas (see footnote 1).

the conditional probabilities for such relations were higher and provided evidence that certain sentences were thesis statements.


Lexical Choice (LC) Lexical choice happens in two stages.

the functionality of the individual modules is briefly described below.


Some of the complexity in the RICHES system is there to demonstrate the potential for different types of control strategies.

nLG systems, especially end-to-end, applied nLG systems, have many functionalities in common.


If our hypothesis is correct we expect the "grammar higher" group to have more accurate parses than the trigram-higher group as the poor parse would cause poor grammar perplexity for the sentence, which would then be worse than the trigram perplexity.

secondly, as noted in [2], the parser encourages right branching with a "bonus" multiplicative factor of 1.2 for constituents that end at the right boundary of the sentence, and a penalty of 0.8 for those that do not.


We say that an MMTAG G is in regular form if there are functions ρ i from Σ into the domain of some partial ordering such that for each component α of type X, for each adjunction node η ∈ α, if the jth child of η dominates the ith foot node of α (that is, another component's jth spine would adjoin into the ith spine), then ρ i (X) ρ j (label(η)), and ρ i (X) = ρ j (label(η)) only if η is safe in the ith spine.

it also has much in common with dtree substitution grammar (Rambow et al., 1995).


On the one hand, it is difficult to tune a system with the express goal of causing perplexity to improve, rendering perplexity a particularly good objective measurement.

the output of such a classifier could be used as an input feature in building decision trees.


This is why we restrict ourself to the class of PD-grammars which have such a property.

we fix finite alphabets of terminals (words), of nonterminals (syntactic types or classes), and  In distinction to (Dikovsky, 2001), the nonterminals (and even dependency names) can be structured.


D-tree itself does not presume a linear order on words.

the set of lists of pairwise different valencies, and n is a set of reduction rules.


2001), and SETHIVoice (Gauthron and Colineau 1999).

even in cases for which there is no impediment to collecting data, the expense and time required to collect a corpus can be prohibitive.


In a language generation system, a content planner typically uses one or more "plans" to represent the content to be included in the output and the ordering between content elements.

finally, a threshold in relevancy of 0.1 was used in the constraint learning procedure.


As the full curves show, log-likelihood is obviously the best measure .

with respect to the recall curves (Figures 3 and  4), we find: (i) Examination of 50% of the data in the SLs leads to identification of between 75% (AdjN) and 80% (PNV) of the TPs.


Given one or more (in most cases more than one) descriptions for a single input term, the organization module selects appropriate description(s) for each domain related to the term.

since past examinations and answers are open to the public, we can evaluate the performance of our QA system with minimal cost.


However, since Google (i.e., the search engine used in our system) rates the quality of pages based on hyperlink information, and selectively retrieves those with higher quality (Brin and Page, 1998), we tentatively regarded P Q (d) as a constant.

for example, different senses for "pipeline (processing method/transportation pipe)" are associated with the computer and construction domains (fields), respectively.


First, while general language models quantify the extent to which a given word sequence is linguistically acceptable, our model also quantifies the extent to which the input is acceptable as a term description.

while P (t|d) is simply a relative frequency of t in d, we need predefined domains to compute P (t|c) and P (t).


First, we generated an encyclopedia by way of our Web-based method (see Sections 2 and 3), and evaluated the quality of the encyclopedia itself.

coverage/accuracy did not significantly change, because as shown in Figure 3, most of the descriptions were inherently related to the computer domain.


To sum up, in principle we select d's that are strongly associated with a specific domain, and are likely to be descriptions themselves.

words that are representative for a domain tend to be frequently used in compound word entries associated with the domain, and thus our method is a practical approximation.


We also used test questions from the Class II examination, and evaluated the Web-based encyclopedia in terms of question answering.

for the input term, related domains and descriptions are simultaneously selected.


We use the program GIZA++ (Och and Ney, 2000b;Och and Ney, 2000a), which is an extension of the training program available in EGYPT (Al-Onaizan et al., 1999).

(Berger et al., 1996) applies this approach to the so-called IBM Candide system to build context dependent models, compute automatic sentence splitting and to improve word reordering in translation.


We have been able to obtain a significant better test corpus perplexity and also a slight improvement in translation quality.

we use the Viterbi alignment that was produced by Model 5.


It would be interesting to see whether other axiomatic constraint-based treatments of grammar could be use to support both parsing and generation.

in short, the semantic monotonicity constraint makes the generation of collocations and idioms problematic.


In this paper, we do not attempt to characterize well-formed German dependency trees although we recognize that such a characterization is essential if we attempt to describe the acceptable sentences of German.

the relative pronoun takes the first position of the domain, but it can take other elements along (pied-piping) (5).


1 If we observe a new sentence f, then an optimal decoder will search for an e that maximizes P(e¤ f) 1 The symbols in this formula are:   e).

if its fertility is greater than one, we call it very fertile.


Recent work (Van Halteren et al., 1998) has demonstrated improvement in part-of-speech tag-ging when the outputs of multiple taggers are combined.

the verb database contains mostly syntactic in-formation about its entries, much of which applies at the class level within the database.


Any ACG generates two languages, an abstract language and an object language.

we call it the transductive paradigm because it is reminiscent of the mathematical notion of transduction (see Section 4.2).


In fact, it only makes sense relatively to a given lexicon.

the additive connectives offer a good theoretical ground to provide ACG with feature structures.


Let X be a infinite countable set of λ-variables.


It may turn out that they should be given a more general tag which corresponds to several major category types.

the following table shows numbers of sentences successfully parsed with each of the three different inputs: the extremely low success rate of Version 1 is a reflection of the fact that the ANLt lexicon does not contain any specialist lexical items.


While this figure is rather low for a practical application, it is worth reiterating that this still means that nearly one in three sentences are not only correctly parsed but they are also assigned a logical form.

as a final stage, we may find it useful to follow Kasper et al.


Tag morphemes can be attached to inflected as well as to base forms, so the string patients NNS has the following internal structure: In defining the rules for word tag pairs, we were careful to ensure that the resulting category would have exactly the same feature specification as the word itself.

in the second case, the basic category of the unknown word is supplied and this enables parsing to proceed.


The tagger we use, ltpos, has a reported performance comparable to other state-of-the-art taggers.

in fact, of the 200 sentences, 188 contained words that were not in the lexicon, and of the 12 that remained, 4 were successfully parsed.


With a corpus such as OHSUMED where there is no gold-standard tagged or hand-parsed subpart, it is hard to reliably evaluate our system.

the tag morpheme is specified only for basic category features which the word grammar requires to be shared by word and tag.


However, in the majority of phenomena the correct disambiguation requires a much wider context.

moreover, the rule-based system used can examine the whole sentential context, again a difficult thing for a statistical system 3 .


An example is: We find an entailment between producing, or making and selling goods, derived from Word-Net, since synset ' make, produce, create( has the genus manufacture, defined in the gloss of its homomorphic nominalization as "for sale".

feedback loop 3 from figure 1 brings forward additional semantic information.


Recall that BALANCED is derived from SEC-TIONS5+6 by adding positive examples, but no negative examples.

we have explored several possible algorithms with the help of a machine learning system, and we have found that these automatically derived algorithms perform well on crossvalidation tests.


We assume that we are making the VPE decision before realization, and therefore have access only to syntactic and semantic features, but not to surface features.

in this example, the italicized VP could be elided, since it has a nearby antecedent (in bold) with the same meaning.


The abstract model that captures the fundamental properties of syntactic annotation schemes provides a conceptual tool for assessing the coherence and consistency of existing schemes and those being developed.

tools for editing annotations in the abstract format, which automatically generate virtual AML from Data Category and Dialect Specifications, are already under development in the context of work on the terminological Markup Language, and a tool for automatically generating RDF specifications for user-specified data categories has already been developed in the SALt project.


furthermore, one-word NEs are different from other NEs in the following respects.


We do not have to change our knowledge that Nobel is a person's name.

for instance, we get the following rule when a person's name is incorrectly tagged as a location's name by a POS tagger.


When using cascaded rules, a small change in a rule can damage another rule's functionality.

the third rule extracts YOKO-HAMA-GIN-KOU (= Yokohama Bank) as an organization.


Some NE systems use decision tree learning to classify a word.

we can apply decision tree learning to classify these examples correctly.


(2000)): The MCLE of (5) is easily calculated: P 0 should be set the empirical distribution of the training data.

the deleted interpolation parameters were estimated using heldout training data from section Joint SR Conditional SR PCFG Precision 0.666 0.633 0.700 Recall 0.650 0.639 0.657 table 2: Labelled precision and recall results for joint and conditional shift-reduce parsers, and for a PCFG.


Note that n i=1 E θ (f A→α |x i ) is a quantity calculated in the Inside-Outside algorithm (Lari and Young, 1990) and P( y| x) is easily computed as a by-product of the same dynamic programming calculation.

, (y n , x n )), the maximum (joint) likelihood estimate (MLE) of θ is: (1) it turns out there is another maximum likelihood estimation method which maximizes the conditional likelihood or "pseudo-likelihood" of the training data (Besag, 1975).


Concerning speed, as argued in (Tapanainen & Järvinen,94), we found that rule−based systems are not necessarily slow, since the 1 million words are parsed in 3mn 8 seconds.

this has led, as early as 1958 (Joshi & Hopely 97), to the development of shallow−parsers, which aim at identifying as quickly and accurately as possible, main constituents (and possibly syntactic functions) in an input, without dealing with the most difficult problems encountered with "full−parsing".


It is difficult to compare our result to other results, since most Shallow−parsers pursue different tasks, and use different evaluation metrics.

the program takes as input a file containing rules.


List rule states have only one continuation, while trie rule states in the branch-  Figure 13: The average number of traversals for sentences of a given length as observed in practice (left), and as predicted by the models presented in the latter part of the paper (right).

for minimized grammars, it is the number of states with outgoing transitions (non-black states in figure 2).


The chance that some alignment will match is therefore , which, for small`is roughly linear in ¥ .

13 phrasal categories in n do not constrain whether c can match a .


In previous work, we identified new procedures to detect recognition errors.

it also illustrates cases in which aware sites and corrections occur in different turns.


These groups are meaningful, and are able to capture the important concepts within the corpus.

this gives us the suggestion that the activation function for each node in the LPN should be a nondecreasing function of the inputs.


The semantic nodes are then used to infer the topic within an input document.

the training phase is divided to two stages.


In topic spotting, the goal is to achieve both high recall and precision.

in the model, the contents of the topics and sub-topics (shown as circles) are modeled by a set of attributes, which is simply a group of semantically related words (shown as solid elliptical shaped bags or rectangles).


These sub-topics may have some common attributes, such as the word "water", and each subtopic may be further sub-divided into finer subtopics, etc.

in the training stage, the weight of the word " corn" was found to be very small in topic "wheat", and hence it was removed from the semantic group.


There is indication that these results may be applied to any L1 group acquiring English (Dulay and Burt, 1974;Dulay and Burt, 1975), and some research has focused on developing a general account of acquisition across a broad range of morphosyntactic structures (cf.

up until now we have only gathered information on the errors committed by our learner population, and thus we still have no information on a great deal of grammatical constructions.


We started with an automatically generated treebank grammar.

the proportion of the unknown words is depicted in Figure 4.


Figure 2: The novel algorithm that we capitalize on in this paper Our method used for the experiments is based on treebank training as well as bracketed corpora training.

only those analyses are considered that meet the tagged brackets (here syllable brackets).


Filter 4 alone provides a high degree of compression; for example, it reduces a total of 16,000 words in the combined sentences that include Vernon Jordan' s name in the Clinton corpus to 578 words in 12 sentences; sentences up to the target length can be selected from these based on scores from Filter 1 and then Filter 3.

as this is a MDS system, we also evaluate the nonredundancy of the descriptions, since similar information may be repeated across documents.


Suppose that a probabilistic grammar licenses headed tree analyses for a sentence Y , and assigns them probabilistic weightsàT  be the governor labels for word position respectively.

for a given governor tuple is a vector space (with pointwise sums and scalar products), so that expectations and conditional expectations may be defined.


But since there is an interval corresponding to a draining process (# $ ) at the root, the whole VP will still be preferred as constituent due to the other interpretation.

the total space and time complexity of the above algorithm (on a context-free forest) is .


these probabilities, which are called word densities, have been used to detect important descriptions of words in texts (Kurohashi et al., 1997).


1 The results show that, using the first approach, there is a considerable amount of parallelism in the parsing computations.

when all threads are idle simultaneously, the parser is finished.


The positive coefficient of the Econ metric appears to indicate that an explicit confirmation strategy overall leads to greater user satisfaction than an implicit confirmation strategy.

the Evaluation Committee chaired by Walker (Walker, 2000), with representatives from the nine COMMUNICAtOR sites and from NISt, developed the experimental design.


However the model fit decreased and FlightInfoC was not a significant predictor.

in current work we are exploring the use of tree models and boosting for modeling user satisfaction.


An English SVO structure is translated into SOV in Japanese, or into VSO in Arabic.

also, localized errors are good if the TM is used for corpus preparation or filtering.


In this paper, a new language model, the Multi-Class Composite N-gram, is proposed to avoid a data sparseness problem for spoken language in that it is difficult to collect training data.

the class sequence is the preceding class of the last word of the word succession and the pre-preceding class of the second from the last word.


And the number of increased parameters with the introduction of word successions is at most the number of word successions times 2.

better word-clustering is to be considered based on word connectivity by the reflection neighboring characteristics in the corpus.


Parts-ofspeech that have the same connectivity in each Multi-Class are merged.

the number of estimated parameters, i.e., the number of combinations of word transitions, is Î AE in vocabulary Î .


In order to obtain a systematic solution to this problem, we shall first examine a derivation of the Winnow algorithm in (Gentile and Warmuth, 1998), which motivates a more general solution to be presented later.

this modification allows the positive weight Winnow algorithm for the augmented input e to have the effect of both positive and negative weights for the original input .


Structural features include information obtained from an XML-encoded parse tree generated for each question by NLPWin (Heidorn, 1999) -a natural language parser developed by the Natural Language Processing Group at Microsoft Research.

figure 1 shows the predictive performance of the decision trees built for two training sets: All5145 and Good4617.


These are words like "make", "map" and "picture".

the lowest predictive accuracy was obtained for the Information Need, Topic and Restriction of Intersection queries.


16 The closure operation is defined for p ∈ [0, 1) as p * = 1/(1 − p), so cycles with weights in [0, 1) are allowed.

for example, in ordinary HMM training, x i = Σ * and represents a completely hidden state sequence (cf.


We offer a theorem that highlights the broad applicability of these modeling techniques.

multiplication and addition are replaced by binary operations ⊗ and ⊕ on K. ⊗ is used to combine arc weights into a path weight and ⊕ is used to combine the weights of alternative paths.


That experiment showed roughly a factor of 2 speedup.

we assume that we keep a data structure listing, for each training instance x j and each value y, the i such that Now we can describe our variation on GIS.


The first option is inefficient and the second adds considerably to the complexity of the algorithm.

iiS can lead to speedups when f # (x j , y) is substantially less than f # .


the baseline for this task is 0.8566 (assuming "first" as the default value).


Algorithm 2: Calculating the precision of each pattern.

the precision scores are calculated by cross-checking the patterns across various examples of the same type.


So for each path from the root f-structure, a distinct variable is introduced: ¥ , subscripted with the (abbreviated and possibly empty) feature path: Rule augmentation step 1 adds to each category name a concrete f-structure to which the category corresponds.

any relevant metavariable ( , ¤ ) in the grammar must end up instantiated to one of these.


So, in OT-LFG it is not sufficient to apply just the ¡ £ ¢ construction; I use an additional step: prior to application of ¡ £ ¢ , the LFG grammar (depending on the constraint set ¡ ), which is still an LFG grammar but has category symbols which reflect local constraint violations.

a similar construction as the one presented here can be devised for parsing-based optimization (even for an LFG-style grammar that does not obey the offline parsability condition).


GEN can be extended "backwards" to incorporate morphology just as PRON was extended "forwards" to incorporate phonetics.

this revises Ot's "do only when necessary" mantra to "do only when necessary and then as late as possible" (even if delaying 's means suffering more of them later).


The advantages on the other hand are twofold.

since furthermore the tree is syntactically and pragmatically complete, generation stops yielding the NP the bowl on the table.


In practice, these goals contribute to updating a "goal semantics" which the generator seeks to realise by building a phrase structure tree that (i) realises that goal semantics, (ii) is syntactically complete and (iii) is pragmatically appropriate.

suppose for instance, that the initial goal is to describe the discourse old entity w .


In Section 2 we introduce some preliminary definitions, followed in Section 3 by a first algorithm based on CKY parsing.

to give a few examples, in some natural language generation systems (Langkilde, 2000) nonrecursive CFGs are used to encode very large sets of candidate sentences realizing some input conceptual representation (Langkilde calls such grammars forests).


General CFGs have unfavourable computational properties with respect to intersection.

unfolding such a grammar and parsing the generated strings ¡ Secondary affiliation is the German Research Center for Artificial Intelligence (DFKI).


And the models for these complex structures are relatively less trained as compared with models for tagging.

experiments are carried out in the shallow semantic parser of an air travel dialog system.


We define an event as a parse action together with its context.

longer sentences would have higher change of entropy, in other words, larger impact on models.


Intuitively, it seems that more classes should help, by allowing the system to distinguish different types of constituents and constituent contexts.

note that simply directly clustering all sequences into 12 categories produced almost entirely the latter, with clusters representing various distituent types.


We performed most experiments on the 7422 sentences in the Penn treebank Wall Street Journal section which contained no more than 10 words after the removal of punctuation and null elements (WSJ-10).

it seemed to slightly hurt parsing accuracy overall.


Overall F 1 has dropped, but, interestingly, VP and S recall are higher.

1 this appeared unpromising and most recent work has returned to using structure search.


denotes the probabilities in the pruned model, and P(w i-1 w i ) is a smoothed probability estimate in the original model.

we show how an optimal function which defines the optimal settings of the threshold pairs is efficiently established using our techniques.


This seems to be a very tedious task since for each model size, a large number of settings (θ 1 θ 2 ) have to be tried for finding the optimal ones.

the goal of the pruning criterion is to estimate the performance loss due to pruning each bigram individually.


By evaluating pruned models on the held-out data, optimal settings can be found.

the increase in relative entropy from pruning a bigram is computed by where the summation is over all word pairs (w i-1 w i ).


Some sample settings of the parameters (l, t c , t w ) are shown in Figure 2.

many traditional clustering techniques [Brown et al., 1992] attempt to maximize the average mutual information of adjacent clusters where the same clusters are used for both predicted and conditional words.


Unfortunately, the ACM contains four sub-models and this makes it difficult to be pruned to a specific size.

the ACM) rather than techniques for finding clusters (i.e.


In addition, we allow different values of the threshold for different sub-models.

the basic idea underlying the ACM is the use of different clusters for predicted and conditional words respectively.


the ACM) rather than techniques for finding clusters (i.e.

the IBM model is of the form P(w i |PW i )× P(PW i |CW i-2 CW i-1 ).


For example, in the UPenn Treebank setup, we used section 00-20 as training data, section 21-22 as held-out data, and section 23-24 as test data.

the SLM is interpolated with a 3-gram model as shown in the equation:   We should note that the PPL result of the 3-gram model is 166.6.


More complex caching techniques are possible (Goodman, 2001), but are not necessary for this experiment.

we have proposed a fundamental principle of language generation, namely the entropy rate constancy principle.


Most systems extract co-occurrence and syntactic information from the words surrounding the target term, which is then converted into a vector-space representation of the contexts that each target term appears in (Brown et al., 1992;Pereira et al., 1993;Ruge, 1997;Lin, 1998b).

it is also important to consider how information extracted by systems such as thesaurus extraction systems can be incorporated into tasks which use predominantly supervised techniques, e.g.


Secondly, SEXTANT and MINI-PAR have quite similar results overall, but MINIPAR is slightly better across most measures.

in these experiments, we calculate this for n = 1, 5, and 10.


We can generalize well if the sub-hierarchies are in a well-defined semantic relation with their ancestors.

other trees are more heterogeneous, in the sense that the meanings among the nodes are more diverse.


FRAGMENT or SKIMMED parses or SKIMMED+FRAGMENT parses.

furthermore, a held-out data set was created from section 24 of the WSJ treebank for experimental selection of the variance parameter of the prior distribution.


For example, in wsj 1303.mrg Japan's Daiwa Securities Co. named Masahiro Dozen president., the noun phrase Masahiro Dozen is labeled as an NP-SBJ.

the error reduction row lists the reduction in error rate relative to the upper and lower bounds obtained by the stochastic disambiguation model.


Under the current definition of properties, we can introduce such a property by modifying the underlying unification grammar to produce a certain "diacritic" feature in a parse just in case the parse actually contains the appropriate reduced relative construction.

requires statistics that depend on the set of all parses of the strings in the training cor-pus.


i.e., without expanding these into individual parses.

there are several efficient heuristics that are reputed in graphical models community to produce good visitation schedules.


In each case we show that the quantity can be expressed as the value that maximises a product of functions or else as the sum of a product of functions, each of which depends on a small subset of the variables X.

the dynamic programming algorithms presented here require the information encoded in properties to be local with respect to the features F used in the packed parse representation.


The monolingual group made their judgments based only on the translations' readability and fluency.

s3 appears better than s2 which in turn appears better than s1.


Candidate translations longer than their references are already penalized by the modified n-gram precision measure: there is no need to penalize them again.

to this end, we obtained a human translation by someone lacking native proficiency in both the source (Chinese) and the target language (English).


1991;Dagan and Itai 1994;Pedersen and Bruce 1997;Schutze 1998;Kikui 1999;Mihalcea and Moldovan 1999).


The resulting Co-Training curve degrades substantially.

we regard these cases as irrelevant because they do not contribute any knowledge for the classifier.


X X X X 6. ana gram func X X X 7. ana npform na na X 8. ana agree X X X 9. ana semanticc.

with L=1000 and L=2000 Co-Training does show some improvement.


We solve the second problem in a Bayesian probabilistic framework.

we test each classifier on a test corpus of 5000 examples labeled with and 5000 examples labeled with , which ensures that the baseline is the same for all combinations and , namely 50%.


So we can expect the examples we extract to be noisy.

our hope is that simple extraction methods are sufficient for collecting low-noise training corpora.


To build such systems, we train a family of Naive Bayes classifiers on a large set of examples that are generated automatically from two corpora: a corpus of 41,147,805 English sentences that have no annotations, and BLIPP, a corpus of 1,796,386 automatically parsed English sentences (Charniak, 2000), which is available from the Linguistic Data Consortium (www.ldc.upenn.edu).

given as input sentence pairs such as those shown in (1)-(2), we develop techniques and programs that label the relations that hold between these sentence pairs as CONTRAST, CAUSE-EXPLANATION-EVIDENCE, CONDITION, ELABO-RATION or NONE-OF-THE-ABOVE, even when the discourse relations are not explicitly signalled by discourse markers.


Highly mobile MATCH is a working city guide and navigation system that currently enables mobile users to access restaurant and subway information for New York City (NYC).

a speech manager running on the device gathers audio and communicates with a recognition server running either on the device or on the network.


Table 2 gives the correlation results for User Satisfaction for different feature sets.

in DATE, utterance classification is done along three cross-cutting orthogonal dimensions.


However, real customers of a deployed system may not be willing to give detailed feedback.

the binary task Completion decision tree simply checks if an acknowledgement:flight booking has occurred.


Defining the Response Variables: In principle, either low User Satisfaction or failure to complete the task could be used to define problematic dialogues.

of the dialogues that it classes as problematic 66.7% are problematic (Precision).


In the original tree, long dialogues are not penalized if they have achieved a complex task (i.e.

a number of studies have been done on predicting specific errors in a dialogue, using a variety of automatic and handlabelled features, such as ASR confidence and semantic labels (Aberdeen et al., 2001;Hirschberg et al., 2000;Levow, 1998;Litman et al., 1999).


maxlinks Used in Competitive Linking algorithm: Maximum number of words any word can be aligned with.

this one-toone/zero-to-one assumption is relaxed by redefining the notion of a word.


We also assume the availability of the Web, considered to be the largest open domain text corpus containing information about almost all the different areas of the human knowledge.

pointwise Mutual Information and Corrected Conditional probability) 8 ¡ ¦ 9 4 @ ¡ Ä E C F may be ignored.


(1999a) find NP chunks, using Wall Street Journal training material of about 9000 sentences.

this is clearly disadvantageous and specific to this choice of al- gorithm.


The curve illustrates that the attenuated representation performs better than the pure word-based one at all reasonable training set sizes.

to formulate the task as a machine-learnable classification task, we use a representation that encodes the joint task of chunking and function-tagging a sentence in per-word classification instances.


", a NER may mistake "Even News Broadcasting Corp." as an organization name.

each group can be made up of many binary features.


1999;Collins 2000), and also the Conditional Random Field methods of (Lafferty et al.

the maximumentropy tagger we used represents a serious baseline for the task.


it is not equal to the true class for the example), the example is added to the training data for that class as a negative example, and the next ranked class is checked.

section 5 shows experimental results of English POs tagging and Japanese morphological analysis with revision learning.


For any sub-paths from the beginning of the sentence (BOS) in the lattice, its generative probability can be calculated using HMMs (Nagata, 1999).

their computational cost is large and is a weakness of SVMs.


Our first variation, which is called Two Pool Algorithm A, adds new unlabeled examples to the primary pool when the increasing ratio of support vec-tors in the current classifier decreases, because the gain of accuracy is very little once the ratio is down.

it is not clear that the approach is readily applicable to tasks which have more than 100,000 features and more than 100,000 examples.


However in other cases, where the morphology is very irregular the performance will be poor.

many of the models produced merely memorize a pair of strings -this is extremely inefficient.


Both these features generate an implicit yet simple backoff solution for the lexicalized features HEAD WORD (hw) and CONTENT WORD (cw).

predicate-Argument Structures proposition Bank or propBank is a one million word corpus annotated with predicateargument structures.


Only the best performance achieved under parameter is shown in each case.

certain nodes contain DAGs within themselves.


Thus, we define the count of the sequence § in , , as: denotes the number of occurrences of § in the string Ó , i.e., the expected number of occurrences of § given .

we presented several new and efficient algorithms to deal with more general problems related to the construction of language models found in new language processing applications and reported experimental results showing their practicality for constructing very large models.


Our approach to word similarity to overcome data sparseness is perhaps most similar to (Karov and Edelman, 1998).

this approach makes the task comparable to classic word sense disambiguation (WSD), which is also concerned with distinguishing between possible word senses/interpretations.


The literal reading comprises a locative (5) and a political entity interpretation (6).

whereas a classic (supervised) WSD algorithm is trained on a set of labelled instances of one particular word and assigns word senses to new test instances of the same word, (supervised) metonymy recognition can be trained on a set of labelled instances of different words of one semantic class and assign literal readings and metonymic patterns to new test instances of possibly different words of the same semantic class.


Previous works, on the other hand, have used measures based on bag-of-words.

we first evaluated the precision of one-to-one alignments by sorting them in decreasing order of SntScore.


Here, we only discuss the similarities between Japanese and English sentences for alignment.

the similarity between an English article and a (word-based English translation of) Japanese article is measured by BM25 (Robertson and Walker, 1994).


We are distributing the alignment data discussed in this paper so that it can be used for research and educational purposes.

column mean that the difference in averages for AVSIM is statistically significant at 1% and 5% based on a one-sided Welch test.


The number of English articles used was 35,318, which is 68 percent of all of the articles.

we first used a method based on cross-language information retrieval (CLIR) to align the Japanese and English articles and then used a method based on dynamic programming (DP) matching to align the Japanese and English sentences in these articles.


the English articles before mid-July 1996 do not have such tags.


The average English sentence length was 16.

our loosely tree-based alignment techniques allow statistical models of machine translation to make use of syntactic information while retaining the flexibility to handle cases of non-isomorphic source and target trees.


Lexicalization can increase parsing performance dramatically for English (Carroll and Rooth, 1998;Charniak, 1997Charniak, , 2000Collins, 1997), and the lexicalized model proposed by Collins (1997) has been successfully applied to Czech (Collins et al., 1999) and Chinese (Bikel and Chiang, 2000).

lexicalization has been shown to improve parsing performance for the Penn Treebank (e.g., Carroll and Rooth 1998;Charniak 1997Charniak , 2000Collins 1997).


Example are MO (modifier), HD (head), SB (subject), and OC (clausal object).

this comes at the price of an unacceptable reduction in coverage.


In a second series of experiments, we investigated a more general way of dealing with the flatness of C&R Collins Charniak Current Head sister category X X X Head sister head word X X X Head sister head tag X X Prev.

we also observe a dramatic reduction in coverage (to about 65%).


Second, a series of transducers extracts verb-final, verb-first, and verb-second clauses.

specifier (SPR), marker complement (CMP), infinitival zu marker (PM)) are frequent and easy for all approaches.


Such context variables consist of the sequence name , a number ¡ designating the dependent in left-to-right order (e.g.

to Abney, these are full clauses, not just simplex clause chunks, so that again recursion can occur.


The purpose of this experiment is to see whether the context method for synonymous word extraction can be effectively extended to synonymous collocation extraction.

a number of studies investigate the extraction of synonymous words from monolingual corpora (Carolyn et al., 1992;Grefenstatte, 1994;Lin, 1998;Gasperin et al., 2001).


Vectors were normalised, so that the standard (Euclidean) scalar product and cosine similarity coincided.

by altering the query vector itself, vector negation removes not only unwanted strings but unwanted meanings.


As initialization, we use the word graphs of the monotone IBM4 search.

we used a single-word based search with IBM Model 4.


The situation gets even worse when a trigram language model is used.

additionally, edges may be annotated with probabilities of the language or translation model.


Corpus-based work about pronoun resolution in spoken dialogue is almost non-existent.

the results are comparable to any pronoun resolution algorithm dealing with written text.


With this depth of recursion (and a realistic morphological grammar), we get an unmanagable explosion of the number of states in the compiled (intermediate) FSA.

we can exploit more effectively the "mildly context-free" characteristics of morpholog-ical grammars (at least of German) discussed in sec.


We retrieved 300 documents as a relevant document set.

by allowing a GENERIC place-holder to match with anything as long as the context of the pattern is matched, the extraction patterns can extract the entities that are not tagged properly.


The model of our previous work (Sudo et al., 2001) was based on the paths from predicate nodes in dependency trees.

in this paper, we explored alternative models for the automatic acquisition of extraction patterns.


The motivation for using the clarity metric will given in section 6.1.

terms that have the same sense and for the TDT corpus will increase recall and decrease.


We can thus derive the query structures of the form given in Table 2.

c) Output function: In the FSA, certain states are designated as output states, which indicate that a sequence of tokens has been found and are tagged with the appropriate semantic tag.


The overall results indicate that there are fewer errors in short queries as such queries contain only one CSS component.

if we can find a relevant phrase with sim(q i ,c j )>¡ at the higher character level, we will not perform further matching at the lower levels.


However, the speech interface involves two inevitable problems: one is speech recognition errors, and the other is that much information cannot be conveyed at once in speech communications.

this aims to reduce the subject's cognitive load and possible confusion due to switching the systems (Over, 1999).


2 suggests that class model probability and context model probability can be combined through simple multiplication, in practice some weighting is desirable.


Our solution is the extended lexicalization.

unlike English, it may not be desirable to separate the solution to word segmentation from the solutions to the other three problems.


In fact, Zipf's law has been shown to apply to randomly generated artificial texts (Li, 1992).

linguistica is the best method for corpus sizes below 50 000 words, but its performance degrades from the maximum of 39.6% at 10 000 words to 29.8% for the largest data set.


We present a language-independent and unsupervised algorithm for the segmentation of words into morphs.

for languages with agglutinative morphology this may not be enough.


In such a setting it is natural to make a distinction between the first occurrence of a word and subsequent occurrences, which according to Church are influenced by adaptation (Church and Gale, 1995), referring to an increase in a word's chance of re-occurrence after it has been spotted for the first time.

when a small set of parameters controls all properties of the distribution it is important to have enough parameters to model the relevant aspects of one's data.


This paper presents a way of utilizing statistical decision trees to systematically raise the memory capacity of Markov models and effectively to make Markov models be able to accommodate various features.

for example, we often find that in some cases a certain lexical context can improve the performance of an HMM-based POS tagger, but incorporating such additional features is not easy and it may even degrade the overall performance.


generate possible output chunks for all possible input chunks.


In co-training, the two collaborating classifiers are assumed to be based on two different views, namely two different subsets of the entire feature set.

in the algorithm, one classifier always asks the other classifier to label the most uncertain instances for it.


The accuracy of the natural split is higher than that of the random split.

it assumes that the classes of the classifiers in bootstrapping do not overlap.


In some sections, we may write simply instead of L , L ∅ , NL or NL ∅ .

for X and Y ∈ T (P,≤) , X ≤ Y iif this relation is deductible in the following system where p, q ∈ P , n, k ∈ Z and X, Y, Z ∈ T (P,≤) : 1 This implies that the class has infinite elasticity.


The analyses given in Steedman (2000) assume that this can be accounted for by an extended use of composition.

the probability of generating word Û, given that its lexical category is and that ¼ Û ¼ is head of the th argument of Û .


In relative clauses, there is a dependency between the verbs in the relative clause and the head of the noun phrase that is modified by the relative clause: Since the entire relative clause is an adjunct, it is generated after the noun phrase Smith.

this excludes certain kinds of dependencies that were captured by the SD model.


Therefore, they are only conditioned on their lexical categories.

the conditional model used by the CCG parser of Clark et al.


The Hong Kong Newspaper mainly publishes announcements of the local administration and descriptions of municipal events, such as an anniversary of the fire department, or seasonal festivals.

• WEBS (Websumm (Mani and Bloedorn, 2000)): can be used to produce generic and query-based summaries.


Likewise, subsegmentation of the prefix Al into A# l# is filtered out.

we choose a segmentation into multiple prefixes and suffixes.


The subtrees are then combined by a simple unification process and syntactic features are percolated down.

also thanks to Charles Yang for much useful discussion, and valuable comments from the anonymous reviewers.


To allow the finite/non-finite distinction, and other verb type distinctions, SPLIT-VP annotated all VP nodes with their head tag, merging all finite forms to a single tag VBF.

lexicalization is a (radical) method of marking a distinctive aspect of the otherwise hidden internal contents of a node which influence the external distribution.


Even for topics central to the treebank's Wall Street Journal text, such as stocks, many very plausible dependencies occur only once, for example stocks stabilized, while many others occur not at all, for example stocks skyrocketed.

we base all further exploration on the v ≤ 2, h ≤ 2 grammar.


Rather than using a deeper grammar formalism, our system combines a simple unlexicalized PCFG parser with a shallow pre-processor.

the precision, recall, and the combined F-score is presented for each experiment.