Intuitively, we would assume when the auxiliary language has a smaller average distance to all the target languages, the cross-lingual transfer performance would be better.

from the results in Table 6, we do not see such a pattern.


2013, only assumed that each utterance is generated conditioned on the previous and current topic/DA pairs.

our model is able to model the dependencies of all preceding utterances of a conversation, and hence can better capture the effect between DAs and topics.


Summarization is an important component of strategy instruction in reading and writing skills (Graham and Perin, 2007), but is used less than it could be due to the demands of manual grading and feedback.

integration of NLP with rubric-based assessment has received increasing attention.


These studies rely on human annotation of timestamped subtask goals, e.g., timed captions created through crowdsourcing.

humanin-the-loop annotation is infeasible to deploy for popular video sharing platforms like YouTube that receive hundreds of hours of uploads per minute.


We explore neural encoder-decoder models based on Transformer Networks (Vaswani et al., 2017).

to RNNs, Transformers abandon recurrence in favor of a mix of different types of feedforward layers, e.g., in the case of the Transformer decoder, self-attention layers, cross-attention layers (attending to the encoder outputs), and fully connected feed-forward layers.


determine which object is being referred to.

in many scenarios where a grounding system can be deployed, grounding is not an isolated oneoff task.


ACROSS USERS: Similar to the SAME USER dataset, we create a new example each time the annotator refers to an object.

the past expressions come from the other annotator who was displayed the same scene.


We hypothesize that due to pretrained word embeddings (which include embeddings for words describing the unknown categories), the coreference models will be able to successfully ground to new object categories.

the performance of the Vision model will decrease in the HARD split, because the pretrained visual features of the new objects are not well aligned with representations of unseen words.


We gratefully acknowledge funding support in part by the Honda Research Institute and the Toyota Research Institute.

this article solely reflects the opinions and conclusions of its authors.


Here, we should note that it is often difficult to make sense of these attention weights.

we observe that the attention matrix changes very gradually near the completion of the recipe.


BiDAF w/ static memory is an extended version of the BiDAF model which resembles our proposed PRN model in that it includes a memory unit for the entities.

it does not make any updates on the memory cells.


• Myopicity: The oracle policy chooses the set C i j that maximizes its performance.

the success of LTAL depends on the sequence of choices that are made.


One possibility is that no active learning policy is better than random.

lONGEST outperformed BASEORAClE showing that the problem is at least partially related to BASEORAClE itself.


Because of this, there is no way to pre-fetch the next instance in the background and the annotator has to wait for the selection/generation process to finish before the next instance is presented for annotation.

the runtime for pool-based AL methods is increasing with the pool's size.


However, the runtime for pool-based AL methods is increasing with the pool's size.

the generation method presented in this work does not have this limitation.


One of the main drawbacks of our work is its limitation to binary sentence classification.

multi-class classification in an one-vs-rest schema is compatible with our method and worth further exploration.


In some situations, it is more natural to express the constraints as a set of automata.

naively enforcing multiple automata by fully intersecting them is potentially expensive.


The formulation above assumes that the constraints are described using a single automaton.

in some scenarios, it is more natural to impose multiple constraints by representing them as a set of automata.


While the model can potentially "impose" some constraints which are well-represented in the data, there is no guarantee that the output structure will be valid.


(2017) are valuable for the analysis of the fact-checking problem and pro-vide annotations for stance detection.

they contain only several hundreds of validated claims and it is therefore unlikely that deep learning models can generalize to unobserved claims if trained on these datasets.


The corpus therefore allows training deep neural networks for automated fact-checking, which reach higher performance than shallow machine learning techniques.

the corpus is based on synthetic claims derived from Wikipedia sentences rather than natural claims that originate from heterogeneous web sources.


For each claim, the authors extracted about 30 associated documents using the Google search engine, resulting in a collection of 136,085 documents.

since the documents were not annotated by fact-checkers, irrelevant information is present and important information for the claim validation might be missing.


The corpus consists of transcripts of political debates in English and Arabic and provides annotations for two tasks: identification of check-worthy statements (claims) in the transcripts, and validation of 150 statements (claims) from the debates.

as for the corpus PolitiFact17, no evidence for the validation of these claims is available.


The results illustrated in Table 9 show that BertEmb, USE+MLP, BiLSTM, and extendedESIM reach similar performance, with BertEmb being the best.

compared to the FEVER claim validation problem,  where systems reach up to 0.7 F1 macro, the scores are relatively low.


In fact, we have performed additional experiments in which we pre-trained a model on the FEVER corpus and fine-tuned the parameters on our corpus and vice versa.

no significant performance gain could be observed in both experiments Based on our analysis, we conclude that heterogeneous data and FGE from unreliable sources, as found in our corpus and in the real world, make it difficult to correctly classify the claims.


Traditional NER for the news domain focuses on three coarsegrained entity types: person, location, and organization (Tjong Kim Sang and De Meulder, 2003).

as NLP technologies have been applied to a broader set of domains, many other entity types have been targeted.


Specifically, in this example, John is labeled as PER, so PER is the only possible correct tag at that position.

lives, in, and Texas are labeled as O (nonentity), which here means only that they may not be PER-but any of them could be LOC, since locations are not annotated for this sentence.


As a result, we demonstrate the first accurate, robust, and highly efficient system that is actually a viable substitute for standard, more cumbersome twostage retrieval and re-ranking systems.

with existing literature, which reports multiple seconds to resolve a single mention, we can provide strong retrieval performance across all 5.7 million Wikipedia entities in around 3ms per mention.


Evaluating word embeddings with cognitive language processing data has been proposed previously.

the available datasets are not large enough for powerful machine learning models, the recording technologies produce noisy data, and most importantly, only few datasets are publicly available.


For instance, we chose a wide selection of eye-tracking features that cover early and late word processing.

choosing only general eye-tracking features such as total reading time would also be a viable strategy.


However, choosing only general eye-tracking features such as total reading time would also be a viable strategy.

the EEG evaluation could be more coarse-grained, one could also try to predict known ERP effects (e.g.


This example consists of a series of events, i.e., "check in (a flight)", "be cleared (at the security)", "wait for (the plane)", etc., which humans who have traveled by plane are very familiar with.

this event sequence appears infrequently in text.


An existing straightforward approach (Peng et al., 2016;Xiong et al., 2016) involves creating a set of relevant entities using an entity linking system to detect and disambiguate the names of entities in a document.

this approach is problematic because (1) entity linking systems produce disambiguation errors (Cornolti et al., 2013), and (2) entities appearing in a document are not necessarily relevant to the given document (Gamon et al., 2013;Dunietz and Gillick, 2014).


MAJ and PARTYM outperform the proposed models in Setting 2.

pARTYM is based on knowledge of the future and so is impractical, and MAJ has no discriminative power for individual politicians.


(2004), an Ideal Point model was trained over both politicians and bills/issues, and at inference time the similarity between politician and bill was used to determine how likely the politician was to vote for the bill.

most politicians have distinct views on different issues, meaning that the views of one politician cannot be captured in a single dimension.


As demonstrated in this paper, this model works well when a politician's voting track record has already been established.

it fails for politicians not in the training data, such as those who have never been elected to office or voted on bills relevant to the issues in the target bill.


Finding such documents is an easy task since search engines are capable of returning doc-uments conveying related information.

if search engines are effective in retrieving these documents, the task of putting them into a coherent picture remains a challenge (Shahaf et al., 2012).


The dataset SLP outputs more segments than the modality version.

most segments do not match the reference.


Neural encoder-decoder models have provided a viable new approach for jointly extracting relations and entity pairs.

these models either fail to deal with entity overlapping among relational facts, or neglect to produce the whole entity pairs.


repeat the process to extract all triplets.

to extract multiple entity pairs for a relation type, both CopyR and HRL have to repeatedly predict the relation type in multiple passes, which is computationally inefficient.


As shown in Table 1, SKE has a richest vocabulary (170,206 tokens), while WebNLG has a smallest vocabulary (5,051 tokens).

to NYT and SKE which has a medium body of relations (24 and 50, respectively), WebNLG has a significantly big body of relations (246 relations).


Distantly-supervised models are popular for this task.

sentences can be long and two entities can be located far from each other in a sentence.


In previous works, the information of entity types are commonly utilized to benefit event detection.

the sequential features of entity types have not been well utilized yet in the existing ED methods.


Similar to the word sequences, entity type sequences, which consist of entity type annotations for each token in the word sequences, also contain sequential features, because the position of an entity mention's type in the sequences may affect its importance in the ED process.

to the best of our knowledge, there is no study which regards the entity type sequence as an independent sequence to capture the sequential features and discusses what influence the entity types' sequential features would take to the ED task.


To summarize, on the one hand, ETEED significantly improves the recall values compared with the state-of-the-art methods.

eTeeD produces relatively comparable precisions to the existing methods, which ensures the good F1-score.


As methods which use argument role information, ETEED argument significantly outperforms TEACHER, this result further proves the effectiveness of our model.

in the real testing and application scenarios, we can hardly obtain the arguments role information before getting the event types of the candidate triggers.


Chambers and Jurafsky (2011) design a weakly supervised system, which can automatically induce the event templates and extract event information from unlabeled corpus, to alleviate the need of expert knowledge.

the required external resources are not always available for some low-resource languages.


(2017) utilize the entity type embedding directly as local context of the current word, and calculate the attention values between them; others (Liu et al., 2018b(Liu et al., , 2019Nguyen and Grishman, 2018) concatenate the entity type embeddings with the work token embeddings, in order to integrate these two types of features into mixed representations with the help of neural networks.

these existing studies ignore the entity types' sequential features which may benefit the ED task.


Recent developments in Named Entity Recognition (NER) have resulted in better and better models.


We decided not to open the whole data set, because it is the test set and the tuning models on this set would lead to unfair results.

we could not perform the analysis on a validation set because it is rather poor with respect to different kinds of linguistic properties.


Named entity classification as an instance of PU learning was introduced in Grave (2014), which uses constrained optimization with constraints similar to ours.

they only address the problem of named entity classification, in which mentions are given, and the goal is to assign a type to a named-entity (like 'location', 'person', etc.)


One contribution of this work is the inference step (line 6), which we address using a constrained Integer Linear Program (ILP) and describe in this section.

the constraints are based on a value we call the entity ratio.


A similar assumption was made in Elkan and Noto (2008) when determining the c value, and in Grave (2014) in the constraint determining the percentage of OTHER examples.

we also show in Section 4.8 that knowledge of this ratio is not strictly necessary, and a flat value across all datasets produces similar performance.


In all our experiments so far, we have used the gold entity ratio for each language, as shown in Table 1.

exact knowledge of entity ratio is unlikely in the absence of gold data.


Recently, contextualized Bidirectional Encoder Representations from Transformers (BERT) models have established state-of-the-art performance for a variety of NLP tasks.

there has not been much effort in exploring language transfer using BERT for event extraction.


For ZH, we obtain F1-scores of 84.4% and 79.9%, amounting to an increase of 16.2% and 16.9% over the previous state-of-the-art.

although results using Bi-LSTM-Char-CRF lag behind state-of-the-art for EN, incurring a loss of 10.5% over trigger classification, they are competitive for ZH, with scores of 86.6% and 69.5% and gains of 17.9% and 6.2% over Feng's HNN for trigger identification and classification respectively.


(2016) exploit both languagedependent and language-independent features in the form of universal features such as universal dependencies, limited bilingual dictionaries and aligned multilingual word embeddings to train a model with multiple languages.

this work lags behind in terms of the neural approach used and doesn't investigate the effectiveness of leveraging multiple source languages.


More recently, neural network-based methods have been employed for event temporal relation extraction (Tourille et al., 2017a;Cheng and Miyao, 2017;Meng et al., 2017;Han et al., 2019a) which achieved impressive results.

they all treat the task as a pairwise classification problem.


We hypothesize that this might be due to some sport specific documents that make roughly 1/4 of the dataset's mentions.

without spoiling the test-set we cannot know for sure.


We found that with our approach we can learn additional entity knowledge in BERT that helps in entity linking.

we also found that almost none of the downstream tasks really required entity knowledge, which is an interesting observation and an open question for future research.


Since we focus on domain adaptation and used very simple encoders, we do not attempt to achieve state-of-the-art (e.g., Dai and Huang (2018), Bai and Zhao (2018)).

this performance is on-par with many recent work using multi-task or GANs, including Lan et al.


Remarkable success has been achieved in the last few years on some limited machine reading comprehension (MRC) tasks.

it is still difficult to interpret the predictions of existing MRC models.


For extractive MRC tasks, information retrieval techniques can be very strong baselines to extract sentences that contain questions and their answers when questions provide sufficient information, and most questions are factoid and answerable from the content of a single sentence (Lin et al., 2018;Min et al., 2018).

we face unique challenges to extract evidence sentences for multiple-choice MRC tasks.


Several work also investigate content selection at the token level (Yu et al., 2017;Seo et al., 2018), in which some tokens are automatically skipped by neural models.

they do not utilize any linguistic knowledge, and a set of discontinuous tokens has limited explanation capability.


Their ability to understand user spoken commands and identify the user's intent(s) is one of their main benefits.

this is a very challenging task due to the diversity of domains and languages they are required to support.


Existing language modeling (LM) based approaches such as Para2Vec (Le and Mikolov, 2014) rely on deep neural networks to generate vector representations for the paragraph-level.

these approaches are normally trained with an abundance of utterances to achieve the required performance, which is usually not present in the conversational text domain (Boyanov et al., 2017).


A recent approach proposed by (Le and Mikolov, 2014) generates both word-level and paragraph-level representations.

the approach relies on training the network with a large corpus with billions of tokens, which is rarely available in the conversational text domain.


Thus, the resulting vector is closer to neighbors of both points, shrinking the distances between neighbors of both utterances.Moreover, because each feature (dimension) in the space is a representative utterance, after the collapsing of two utterances, we perform PCA to come up with the new dimensions after the user is done with the labeling.

whenever a "Cannot-Link" constraint is provided, the similarity score between the two utterances is set to zero in the corresponding entry in SimMatrix.


(2016b) learn a fixed vector for each person from all conversational texts in the training corpus.

as a global representation, the fixed person vector needs to be trained from largescale dialogue turns for each interlocutor, and it may have a sparsity issue since some interlocutors have very few dialogue turns.


Therefore, it is hard to learn a global vector for each interlocutor from the sparse corpus.

our ICRED performs well on such a sparse corpus (details in Section 4.5).


Memory Graph Networks (MGN) (Section 2.2): Many previous work in QA or MRC systems use memory networks to evaluate multiple answer candidates with transitive reasoning, and typically store all potentially relevant raw sentences or bag-of-symbols as memory slots.

naive increase of memory slot size or retentionbased sequential update of memory slots often increase search space for answer candidates, leading to poor precision especially for the Episodic Memory QA task.


The sentence lacks the semantics needed to fully understand "club and country".

if we follow the URL in the original text, we can get additional information to assist with the understanding.


As the setup of such a text forecasting task is essentially similar to summarizing future text data, we use two popular evaluation metrics from the literature of text summarization, i.e., BLEU (Papineni et al., 2002) and ROUGE (Lin, 2004), where a score is generated by comparing the automatically generated text against some reference text written by humans.

neither BLEU nor ROUGE considers the notion of time, thus we need a time-sensitive customization of both BLEU and ROUGE.


The learning objective of most sequence-to-sequence models is to minimize the negative log likelihood of the generated sequence as shown in following equation, where y * t is the t-th ground-truth summary token.

with this objective, traditional sequence generation models consider only one direction context in the decoding process, which could cause performance degradation since complete context of one token contains preceding and following tokens, thus feeding only preceded decoded words to the decoder so that the model may generate unnatural sequences.


During pre-training, they are fed with complete sequences.

with a left-context-only decoder, these pretrained language models will suffer from incomplete and inconsistent context and thus cannot generate good enough context-aware word representations, especially during the inference process.


7shows, the decoder's learning objective is to minimize negative likelihood of conditional probability, in which y * t is the t-th ground truth word of summary.

a decoder with this structure is not sufficient enough: if we use the BERT network in this decoder, then during training and inference, in-complete context(part of sentence) is fed into the BERT module, and although we can finetune BERT's parameters, the input distribution is quite different from the pre-train process, and thus harms the quality of generated context representations.


Integrating HRED with the latent variable models such as variational autoencoder (VAE) (Kingma and Welling, 2014) extends another line of advancements (Serban et al., 2016c;Zhao et al., 2017;Park et al., 2018;Le et al., 2018b).

these systems are not designed for taskoriented dialogue modeling as goal information is not considered.


Goal-embedded LM is the best on precision and F1 with G-DuHA having com-  parable performance.

even thought LM can better mention the slots in dialogue generation, utterances are often associated with a wrong role.


This implies hierarchical structures capturing longer dependencies can make up the disadvantages of having no goal information for response generation.

as illustrated in Table 12, HRED could still fail to predict the switch of domain contexts, e.g.


The evaluator also employs structure-sensitive rewards based on evaluation measures such as BLEU, GLEU, and ROUGE-L, which are suitable for QG.

most of the previous works only optimize the cross-entropy loss, which can induce inconsistencies between training (objective) and testing (evaluation) measures.


Model BLEU-1 BLEU-2 BLEU-3 BLEU-4 METEOR ROUGE-L L2A (Du et al., 2017) 43  (Paulus et al., 2018) 11.20-3.50-1.21-0.45-6.68-15.25-SUM ROUGE (Paulus et al., 2018) 11.94-3.   shown in Table 3, seven of our eight models outperform the two baselines, with GE DAS+QSS+ANSS being the best model on syntactic correctness and semantic correctness quality metrics, outperforming all the other models by a large margin.

model GE BLEU+QSS+ANSS generates highly relevant questions and is the best model on relevance metrics.


In summarization, one generates and paraphrases sentences that capture salient points of the text.

generating questions additionally involves determining question type such as what, when, etc., being selective on which keywords to copy from the input into the question, leaving remaining keywords for the answer.


It finally outputs the hypothesis with the maximum probability of generation.

we observe a reasonable difference in the formality scores among hypotheses with similar generation probabilities.


Like greedy decoding, beam search is a likelihood-maximizing decoding algorithmgiven the input sequence x, the objective is to find an output sequence y which maximizes P (y|x).

researchers have shown that for openended generation tasks (including storytelling), beam search produces repetitive, generic and degenerate text (Holtzman et al., 2019).


12 This implies that, as with lexical diversity, the models have no difficulty fitting the statistical distribution of human syntax.

under likelihood-maximizing decoding algorithms such as low k, a completely different distribution emerges, in which text contains more verbs and pronouns than human text, and fewer nouns, adjectives and proper nouns.


As shown in the left plot of Figure 4, the gradients of the "x + F" structure are basically the same, indicating that all the residual blocks have similar speed for gradient descent and optimization.

the right plot of Figure 4 reflects that more gradients are allocated to the lower blocks with the help of SAS, and the overall gradient values are greater.


Hence, the task of German NER has benefited from these developments.

with respect to the availability of a variety of resources, there has not been much progress made until now.


The standard task of NER focuses on the former class of proper names.

it is often not easy to differentiate between both classes.


Time In the standard task of NER, temporal information is not captured by the four base entities.

the aspect of time is important for the research on biodiversity which is constantly evolving.


In language modeling however, this approach of overlapped data points has not yet been fully exploited.

extracting frame-based acoustic features such as mel-fequency cepstral coefficients (MFCCs) using overlapping windows is a common technique in speech processing and more specifically in automatic speech recognition (ASR) (Chiu et al., 2018;Kim and Stern, 2016).


We observed that CEs of the obtained ⇡ ✓ 's were about the same for different values of |D| and Training-1 regimes.

there was no systematic improvement in the training speed of one method over the other.


When we increase the frequency threshold further to 25, the performance of the model has dropped compared with Char-BiLSTM-add-Word-LSTM-Word (g = 0.5, n = 1, θ = 15) as more frequent words are discarded.

we found a relatively small frequency threshold θ = 5 works quite effectively.


In fact, most recent endto-end deep learning models have surpassed the performance of careful human feature-engineering based models in a variety of NLP tasks.

deep neural network based models are often brittle to various sources of randomness in the training of the models.


Most of the interpretation based methods involve one of the following ways of interpreting models: a) sample oriented interpretations: where the interpretation is based on changes in the prediction score with either upweighting or perturbing samples Koh and Liang, 2017); b) interpretations based on feature attributions using attention or input perturbation or gradient-based measures; (Ghaeini et al., 2018;Feng et al., 2018;Bach et al., 2015); c) interpretations using surro-gate linear models (Ribeiro et al., 2016) -these methods can provide local interpretations based on input samples or features.

the presence of inherent randomness makes it difficult to accurately interpret deep neural models among other forms of pathologies (Feng et al., 2018).


We observe that the original SGD reaches the desired minima, however, it almost reaches the saddle point and does a course correction and reaches minima.

we observe that SGD with ASWA is very conservative, it repeatedly restarts and reaches the minima without reaching the saddle point.


We note that on an average, the standard deviations are on the lower side.

we observe that the mean standard deviation of the bins close to 0.5 is on the higher side as is expected given the high uncertainty.


However, we observe that the mean standard deviation of the bins close to 0.5 is on the higher side as is expected given the high uncertainty.

both, ASWA and NASWA based models are relatively more stable than the standard CNN based model.


(2017) that the human coders considered sexist terms as offensive rather than hateful.

in terms of our classifier, this may only be due to the implicit nature of most sexist insults and a lack of sexist samples within the OLID dataset.


Also, there are very few datasets that provide a large number of samples that can be taken advantage of by huge neural networks .

we do acknowledge the difficulty in collecting abusive samples as most discourse online is benign.


This is because there is still value in identifying whether a sample is racist / sexist / cyberbullying over just recognising whether the abuse is explicit or not, and in identifying the target of abuse.

the hierarchical model in its current form still cannot differentiate between various subsets of abusive language.


Analysis of the existing language resources in the area of sentiment analysis shows that they largely concern the English language (Dashtipour et al., 2016).

there is a clear growing interest in other languages, often much more complex than English (e.g.


To analyze sentiment in text, researchers have made different assumptions on linguistic behaviors that are leveraged with approaches developed based on the nature of the text, the representation of the related information and the objectives.

majority of the studies are conducted at the population-level which assumes that people follow a common understanding with regard to the use of language.


(2016) have proposed a phased LSTM that utilizes an addi-tional time gate to control the passing of the information.

the time gate is triggered by periodic oscillations while modeling sensory events, which makes such a design less flexible when the time gaps are highly various.


For instance, when the user frequency is around 80, the WE formulation has the best accuracy in both models.

such an observation is also restricted by the number of frequent users in general -with only 372 posts in the test set when the user frequency is at least 100, the performance is highly dependent on the remaining 3 users.


It can be seen that the last user in the figure (the one at the bottom line) has the greatest values for ✏ and , which means that the decay factor has a great impact on the prediction for this user than the others but the influence from the past decays comparably fastthe user is affected a lot by recent events.

among the 10 users, the second last user is the least influenced by the past which is visualized in darker colors.


In prior research, much progress has been made on text classification, including traditional approaches based on human-designed features (Lazaridou et al., 2013;Zhang et al., 2015a) and neural networks based on deep architectures (Lai et al., 2015;Yang et al., 2016).

such methods prefer to deal with documents and paragraphs, and still have limitations for short texts.


Existing clinical decision support systems are heavily reliant on the structured nature of the EHRs.

the valuable patient-specific data contained in unstructured clinical notes are often manually transcribed into EHRs.


(2018) reported a suite of five clinical prediction tasks, including the length-of-stay, mortality, and ICD-9 code group prediction on the MIMIC-III database using deep learning models and benchmarked their performance against the existing state-of-the-art methods and severity scoring systems.

mining and modeling the valuable patient-specific information in unstructured clinical nursing notes for the development of CDSSs remains mostly uncommon.


The feature extractor is trained to fool the discriminator so that the extracted features are language invariant.

its performance is significantly lower than the variant that uses pretrained CLWE.


Our goal is to predict the sentiment polarity of the examples in a target language-domain pair In Section 5 we compare two CLIDSA variants.

cLIDSA f ull exploits unlabeled data from all possible language-domain pairs, i.e., we set P = L × D. since most previous cLSA methods do not use multi-domain or multilingual unlabeled data, we create a variant cLIDSA min that requires minimal resources by setting A natural way to utilize unlabeled data is to perform the language modeling task.


Note that we would also want to evaluate our model on some low resource languages.

since there isn't an public benchmark for such languages, we leave it to future work.


In earlier work with the writing instruction application, "Writer's Workbench," some features associated with style were evaluated, including: average word length, the distribution of sentence lengths, grammatical types of sentences (e g, simple and complex), the percentage of passive voice verbs, and the percentage of nouns that are nominalizations (see MacDonald et al, 1982 for a complete description of the Writer's Workbench).

to a subjective measure such as, repetitive word usage, the stylistic features in the Writer's Workbench are not subjective.


This resulted in a statistically consistent model dubbed ML-DOP.

mL-DOP suffers from overlearning if the subtrees are trained on the same treebank trees as they are derived from.


PCFG-reduction of Bonnema (1999) By using these PCFG-reductions we can thus parse with all subtrees in polynomial time.

as mentioned above, efficient parsing does not necessarily mean efficient disambiguation: the exact computation of the most probable parse remains exponential.


Feature extraction was unchanged (albeit performed on French data) for the models that capture the contexts for the insertion of negation, prepositions, and subordinating conjunctions.

in every one of these cases, the set of values for the target features changed to reflect the language.


The maximum n-gram length of 1 or 2 yields a high correct ratio, and that of 3 or 4 yields a low correct ratio.

the correct ratio of NIST is not influenced by the maximum n-gram length.


In the original BLEU or NIST formulation of the test set unit (or document or system level) evaluation, n-gram matches are computed at the utterance level, but the mean of n-gram matches is computed at the test-set level.

considering the characteristics of the translation paired comparison method, the average of the utterancelevel scores might be more suitable.


In particular, the averaged utterance-level BLEU score shows the highest correlation.

looking at Figure 10, the system's TOEIC score using this measure deviates from that of the original translation paired comparison method.


In Figure 3, "CNF 1 " got the right POS.

the preceding "that" is falsely assigned as a determiner.


The first two expressions are on the borderline between synset and phraset, while the third is on the borderline between phraset and definition.

in most cases phrasets provide a flexible tool to aid lexicographers in the process of choosing the lexical status of multiword expressions.


Below, we outline an algorithm which circumvents the problem of choosing the right parameters.

to pure Markov clustering, we don't try to find a complete clustering of G into senses at once.


Artifact detection in a signal deserves a separate study in its own right.

in SumTimE we are initially using a median filter and an impossible value filter developed in our collaborator project NEONATE.


The differences of the rates in Figure 3 correspond to the difference of these numbers (i.e., 1467 and 543).

it is very important to note that, for both "pseudo-parallel with CUR" and "contextual similarity with CUR: reduced", the rate of containing correct bilingual term pairs tends to decrease as the order of TP(tE ) sorted by corrEJ(TP(tE)) becomes lower.


Also note that the performance of "pseudoparallel with CUR" is affected little by the similarity lower bounds Ld.

for "contextual similarity with CUR: reduced/full", the performance becomes worse as the similarity lower bound Ld becomes smaller and the crosslingually relevant English/Japanese corpus RCE and RCJ becomes noisier.


The table also shows that the clausal relation would benefit from improvement since the clausal relations is frequently omitted from all the Penn Treebank parsers.

in the case of this relation, we have sacrificed recall for precision.


7 The 'algorithms' are only evaluated on pronouns BC BU CH Cl C2  BC  ---60  0  BU  0  --70  0  CH 60  65  -75 75  Cl  -----C2 ---70 -  8 The difference between the 'worst' and the 'best' systems' mean itt performance is about 1%.

the variance o-2 (a measure of robustness of a system) is lowest for Collins' model 1.


The personal pronoun in the English sentence is used as a noun phrase (NP) similar to the pronoun in the Swedish sentence.

putting all the clues together, the aligner comes up with the following alignment without actually having to know the two languages: the corridors korridorerna are jumping myllrar with av them dem looking at the sentence pair again, a second aligner with knowledge of both languages might realize that the verbs myllrar (English: swarm) and jumping do not really correspond to each other in isolation and that the expressions are rather idiomatic in both languages.


Assume that the alignment program found the following alignment clues which are based on string similarity and co-occurrence statistics: 2 The alignment clues contain only three multiword units.

even these few units cause several overlaps.


Looking at the matrix, we can find clear relations between certain words such as [hand,baggage] and handbagaget.

between other word pairs such as is and sedan we find only low associations which conflict with others and therefore, they can be dismissed in the alignment process.


The lower values for precision for the other two approaches could be expected.

the low recall value for the combined approach is a surprise.


Clue patterns can be defined depending on the information which is available (POS tags, phrase types, semantic tags, named entity markup, dictionaries etc.).

clue patterns have to be designed carefully as they can be misleading.


For example, in the case of English and Japanese, some prepositions are not translated into Japanese.

the preposition 'after' may be translated into Japanese as the noun `ato.'


There are five word links between Source and Target 1, so the TCR is 1.0 by Equation (1).

in the case of Target 2, four words are found in the dictionary (Tt = 4), and there are three word links.


As opposed to filtering based on a threshold, all source sentences are used for knowledge construction, so the coverage of MT knowledge can be maintained.

some context/situation-dependent translations remain in the extracted corpus when only one non-literal translation is in the corpus.


In the case of filtering, the coverage of the MT knowledge is decreased by limiting translation to highly literal sentences.

even though they are non-literal, such sentences may contain literal translations at the phrase level.


Thus, the phrase (A-2)N P is generalized, and the long transfer rule (A-1 )S is generated from the non-literal translation.

the TCR of the top phrase (B-1 )S is 1.0, so all phrases in (B) are generalized and totally six rules are generated.


It is difficult to significantly improve the quality by bilingual corpus filtering because it is difficult to both remove insufficiently literal translations and maintain coverage of MT knowledge.

the BLEU score and the subjective quality both improved in the case of split construction, even though the coverage of the exact rules decreased.


The syntax/semantics interface and the use of unification then ensures that these variables get assigned the appropriate values i.e., the values representing their given arguments.

flat semantics are being increasingly used to (i) underspecify the scope of scope bearing operators and (ii) prevent the combinatorial problems raised during generation and machine translation by the recursive structure of lambda term and first order formulae (Bos, 1995;Copestake et al., 2001).


For example, the parity language 0* (10*10*)* contains strings with an even number of occurrences of the factor "1".

intuitively, it seems improbable that similar counting constraints occur in natural language grammars many regular expressions in Voutilainen's ENGFSiG (1994) involve the Kleene star.


The schematic equivalences presented in Sections 3.3 -3.4 can transform 1554 of these into a star-free form.

there still remain 1103 constraints that use the restriction operator To complete the proof of the star-freeness of ENGFSIG, I show that star-free languages are closed under the restriction operation (as in FSIG).


Effectively, this analysis treats the whole data set as unseen.

notice that for each test/train set split we obtain different regression equations since the PCFA yields different factors for different data sets.


In the past we have found that this wordassociation metric provides an excellent basis for  learning single-word translation relationships, and the higher the score, the more likely the association is to be a true translation relation.

with this particular metric there is no obvious way to combine the scores for indvidual word pairs into a composite score for phrase translation candidates; so we use the scores indirectly to estimate some relevant probabilities, which can then be combined to yield a composite score.


In one sense this makes the task more difficult, since source language phrases have to be discovered as well as target language phrases.

coverage claims are often harder to evaluate since, lacking annotated test data, there is no way to tell how many more phrases a better phrase finder would have discovered that would be mistranslated by the translation finder.


The CCG account given by Steedman (2000) for crossing dependencies in Dutch subordinate clauses relies crucially on >B".

steedman must restrict the harmonic rule >B in order to block some ungrammatical orders.


In our opinion, the Catalan training set is not big enough and the errors in the retraining folds degrade the performance of the bootstrapped model.

the crosslinguistic models show a slightly better behavior, achieving a maximum increase of about 0.5 points, getting also somewhat stable beyond iteration 5.


Bootstrapping, therefore, is not very helpful on improving models.

these models seem to have learned a robust concept which overcomes the errors produced when relabelling folds.


While our method used supervised learning, Stevenson and Merlo (1999) found little difference in performance between unsupervised and supervised approaches on the original MS01 task.

since unsupervised methods are more sensitive to irrelevant features, additional care will be required to determine useful subsets of features from our (larger) general feature space.


For instance, case information for nouns or gender information for verb complements are in most cases unwanted.

only very little semantic information, if any, will be found in this way, and it will need to be supplied by other means.


The required knowledge can either be inferred from speech data or extracted from the literature.


The first idea that suggests itself in order to model these substitutions are phoneme/allophone mapping tables that replace particular L2 sounds by similar speech sounds from the Li inventory.

simple context-free phoneme mapping is problematic in at least two respects: First, for many L2 sounds it is not clear what the 'best' Li equivalent is.


They are based on the assumptions that (1) the naturalness of the translations is effective for selecting good translations because they are sensitive to the broken target sentences due to errors in translation processes, and (2) the source and target correspondences from the semantic point of view are maintained in a state-of-the-art translation system.

the second assumption does not necessarily hold.


Using the movement metaphor of syntactic theory as our guide, we would ideally like to identify a complete path downward from the surface syntactic location of the WH phrase to the location of its associated gap.

it is hard to see how such a path could be represented as one of a finite number of classes.


95% of those phrase translation pairs were judged to be correct.

no results where reported if these additional translation correspondences resulted in improved translation quality.


When a PP is attached to the preceding NP or PP (henceforth referred to as noun attachment), such as in the structure ... eats pizza with anchovies, a phrase boundary between pizza and with is usually considered inappropriate.

when a PP is attached to the verb in the clause (verb attachment), as in the structure ... eats pizza with a fork, an intervening phrase boundary between the PP and its preceding NP or PP (between pizza and with) is optional, and when placed, usually judged appropriate (Marsi et al., 1997).


The bias of IB 1 to base classifications on all training examples available, no matter how low-frequent or exceptional, resulted in a markedly higher recall of up to 82 on noun attachment, indicating that there is more reliable information in local matching on lexical features and the cooccurrence feature than RIPPER estimates.

with a larger training corpus, we might not have found these differences in performance between IB 1 and RIPPER.


Simple chunk parsers only determine the extents of chunks, but neither the interrelationships between them nor their internal structure.

they sometimes try to assign to each chunk a lexical head which can then represent the entire chunk.


This approach increased the coverage of the overall system from 12.5% to 22.1% on the NEGRA-corpus but failed to reduce the average number of analyses which even grew from 16.19% to 18.53% per sentence.

due to its inherent robustness properties our parser always returns the single best solution, i.e.


In the experiment for Czech an indirect approach has been pursued, converting the dependency structures of the Prague Tree Bank into Penn-Treebank-like phrase structure trees.

to this approach, WCDG parses dependency structures directly.


In the case where each text only belongs to one author, the performance of an authorship classifier can be naturally measured by its overall accuracy: the number of correctly classified texts divided by the number of texts classified overall.

overall accuracy does not characterize how the classifier performs on each individual category.


Authorship attribution in Chinese normally requires an initial word segmentation phase, followed by a feature extraction process at the word level, as in English.

word segmentation is itself a hard problem in Chinese, and an improper segmentation may cause insurmountable problems for later prediction phases.


A context n that is too small will not capture sufficient information to accurately model character dependencies.

a context n that is too large will create sparse data problems in training Both extremes will result in a larger perplexity than an optimal length.


Text categorization with n-gram models has also been attempted by (Cavnar and Trenkle, 1994).

they use'n-grams as features for traditional feature selection, and then deploy classifiers based on calculating feature-vector similarities.


(Apt et al., 1994) have used word-based language modeling techniques for both English and German.

their techniques do not apply to Asian languages where word segmentation remains a significant problem.


Such focus would be unproblematic if it were clear how these accounts could be extended to cover multiple, interacting types of structural indications.

even for (Steedman, 2000;Hoffman, 1995), which are the formally most detailed, this is by no means obvious.


corn), while see occurs in 42m and book in 29m.

in a collection of 13k documents about digital signal processing Fourier appears 1100 times, so P(d E Dt ) is about 6.5 10 -5 while P(d E tr) is about 5.5 • 10 -3 , two orders of magnitude better.


As the above reasoning makes clear, linearity is to some extent a matter of choice: certainly the underlying bag of words assumption, that the words are chosen independent of one another, is quite dubious.

it is a good first approximation, and one can extend it from Bernoulli (0 order Markov) to first, second, third order, etc.


As with many other fields, most of the available tools are for English, but the CoNLL02 shared task (Tjong Kim Sang, 2002) has shown that it is possible to use machine learning approaches to design named entity recognisers for languages other than English.

these approaches need annotated corpora to learn how to identify the named entity.


For example, FO contour would clearly be useful in predicting backchannel location.

the challenge of extracting appropriate prosodic features from a pitch tracker lay outside the scope of the research effort reported here.


With the availability of rich linguistic resources, we can also minimize the need to perform complex linguistic processing.

this does not mean that NLP is now out of the picture.


One approach is to expand the query by adding the top k words in C , and those in Gq and Sq.

if we simply append all the terms, the resulting expanded query will likely to be too broad and contain too many terms out of context.


The algorithm is as follows, where E p f, is the empirical expected value of J and E pfi is the expected value according to model p: • Set Az " equal to some arbitrary value, say: • Repeat until convergence: where (t) is the iteration index and the constant C is defined as follows: In practice C is maximised over the (x, y) pairs in the training data, although in theory C can be any constant greater than or equal to the figure in (8).

since determines the rate of convergence of the algorithm, it is preferable to keep C as small as possible.


Surprisingly, students had initiative more of the time in the didactic dialogues (21% of the turns) than in the Socratic dialogues (10% of the turns), and there was no direct relationship between student initiative and learning.

socratic dialogues were more interactive than didactic dialogues as measured by percentage of tutor utterances that were questions and percentage of words in the dialogue uttered by the student, and interactivity had a positive correlation with learning.


3 It may be the case that these annotation assumptions fail on selected examples.

in eliminating the assumptions it is likely that we will introduce more errors than we correct.


These results may be good news for system builders; one possible Socratic teaching strategy would be to ask sequences of targeted questions where strong expectations about plausible answers make it easier to interpret student input.

we must be mindful of the fact that, even in Socratic interaction, students sometimes do take initiative rather than simply answering the sequence of questions posed by the tutor.


When the users in the No Help condition produced outof-coverage utterances the system responded only with a text display of the message "not recognized".

when users in the Help condition produced out-of-coverage utterances they received in-depth feedback such as: "The system heard fly between the hospital and the school, unfortunately it doesn't understand fly when used with the words between the hospital and the school.


In both help conditions, performance improved over the course of the experimental session.

the advantage conferred by help merely diminished and did not disappear during the session.


This word should be broken into the two parts Grund and Rechte.

grund translates usually as reason or foundation.


Such models can be represented by static terminologies or ontologies, which are usually constructed manually.

documents frequently contain unknown terms that represent newly identified or created concepts.


The GUI interface layer utilises dynamic recognition of terms and their clusters, as well as an unrestricted set of tags that can be used for querying.

other systems that use GUI-driven query formulation, such as TAMBIS (Baker et al., 1998), usually use a predefmed ontology impose restrictions on query definition.


The main advantage of XML representation is that it can represent nested structures, something not easily done in RDBs.

even when XML is used to encode documents, many applications still use RDBs for storage and manipulation.


RDBs are generally considered more efficient when it comes to retrieving specific types of elements.

xMLnative DBs provide extended querying facilities given by a native query language (e.g.


evaluated in the context of a non-trivial mediumvocabulary command and control system.

to other work described in the literature ((Wang et al., 2002) is a recent example) rules are treated on a basis of strict parity with other data sources, so that the balance between them can be entirely determined by the training data.


There is not a great deal of training data available, and the statistical methods used are simple and unsophisticated.

we still get a significant improvement on rules alone by adding a trainable component.


The two-step approach would be the natural way of extending current PP attachment disambiguation methods to the more specific 4way attachment we propose here.

based on exploratory data analysis and general wisdom in machine learning, there is reason to believe that it is better to solve the 4-way classification problem directly rather than first solving a more general problem and then specialise the classification.


It is theoretically easy to obtain a gold standard for our evaluation, since the purely syntactic relationships that have to be annotated are less ambiguous than the distinction between collocations and noncollocational candidates.

the annotation of tokens rather than types is a prohibitively laborious task.


Hence apparently it would be preferable to assign different tags to a word when it is used in different syntactic distribution.

this introduces a crisis for the whole classification of Chinese syntactic categories because any given word cannot be prescribed a category and any category cannot be adequately described.


The distributional approach was shown to be quite effective for tasks where new words need to be assigned to a limited number of classes (up to 5; e.g., Riloff and Shepherd, 1997;Roark and Charniak, 1998).

its application to numerous classes, as would be the case with a thesaurus of a realistic size, proves to be much more challenging.


Also, new information on POS correspondences and syntactic functions will be put in the dynamic resources.

if a user rejects a proposal this information is stored as negative data in the dynamic resources on all applicable levels.


The apparently high number of queries actually corresponds to a moderate size dataset, given that the space of parameters includes one parameter for each word-category combination.

assuming SVM Ð Ø does not run out of memory, using the entire dataset for training and testing is extremely  ¾ ).


As we have defined it, OP shows a strong similarity with Named Entity Recognition and Classification (NERC).

a major difference is that in NERC each occurrences of a recognized term has to be classified separately, while in OP it is the term, independently of the context in which it appears, that has to be classified.


Building a classic index at word level was not an option, since we have to search for syntactic structures, not words.


(Fleischman, 2001), have similar accuracy.

the presented weakly supervised Class-Example approach requires as a training data only a list of terms for each class under consideration.


Here, we propose an alignment procedure that explicitly models reordering of words in the hypotheses.

to existing approaches, the context of the whole document rather than a single sentence is considered in this iterative, unsupervised procedure, yielding a more reliable alignment.


The training corpus for alignment is created from a test corpus of N sentences (usually a few hundred) translated by all of the involved MT engines.

the effective size of the training corpus is larger than N , since all pairs of different hypotheses have to be aligned.


Nowadays, most current coreference resolution systems for written text include some means for the detection of nonreferential it.

evaluation figures for this task are not always given.


In view of these results, it would be interesting to see similar annotation experiments on written texts.

a study of the types of confusions that occur showed that quite a few of the disagreements arise from confusions of sub-categories belonging to the same super-category, i.e.


To our knowledge, this task has not been tackled before.

the still fairly good results obtained by only using automatically determined features (P:71.9% / R:55.1% / F:62.4%) show that a practically usable filtering component for nonreferential it can be created even with rather simple means.


Remember that these two models have the same vocabulary and are both de- rived from the same GF interpretation grammar.

the flexibility of the SLM gives a relative improvement of 37% over the Nuance grammar.


This implies that the evaluation carried out is not strictly fair considering the possible task improvement.

a fair automatic evaluation of dialogue move error rate will be possible only when we have a way to do semantic decoding that is not entirely dependent on the GF grammar rules.


The SR dialogues received on average slightly higher scores for understandability (question 1), which can be explained by the shorter length of the system turns for that system.

the difference is not statistically significant (p = 0.97 using a twotailed paired t-test).


When we get to PROJECTIVITY, the quadratic coefficient b is so small that the average running time is practically linear for the great majority of sentences.

the complexity is not much worse for the bounded degrees of non-projectivity (d ≤ 1, d ≤ 2).


Past work on tree-structured outputs has used constraints for the k-best scoring tree (Mc-Donald et al., 2005b) or even all possible trees by using factored representations (Taskar et al., 2004;McDonald et al., 2005c).

we have found that a single margin constraint per example leads to much faster training with a negligible degradation in performance.


These approximations work because freer-word order languages we studied are still primarily projective, making the approximate starting point close to the goal parse.

we would like to investigate the benefits for parsing of more principled approaches to approximate learning and inference techniques such as the learning as search optimization framework of (Daumé and Marcu, 2005).


paired data, would therefore be achievable using standard GHA as follows: In the above, c a and c b are left and right singular vectors.

to be able to feed the algorithm with rows of the matrices M M T and M T M , we would need to have the entire training corpus available simultaneously, and square it, which we hoped to avoid.


Sense dominance may be determined by simple counting in sense-tagged data.

dominance varies with domain, and existing sensetagged data is largely insufficient.


If α is close to 0 5, then even if the system correctly identifies the predominant sense, the naive disambiguation system cannot achieve accuracies much higher than 50%.

if α is close to 0 or 1, then the system may achieve accuracies close to 100%.


On the other hand, using most significant sentence cooccurrences results in mostly semantical similarity (Curran, 2003).

whereas various context representations, similarity measures and clustering methods have already been compared against each other (Purandare, 2004), there is no evidence so far, whether the various window sizes or other parameters have influence on the type of ambiguity found, see also (Manning and Schütze, 1999, p. 259).


This threshold and all others to follow were chosen after experiment-ing with the algorithm.

as will be shown in section 4, the exact set-up of these numbers does not matter.


Unfortunately, the usefulness of their beam search solution is limited: potential alignments are constructed explicitly, which prevents a perfect search of alignment space and the use of algorithms like EM.

the cohesion constraint is based on a tree, which should make it amenable to dynamic programming solutions.


They concluded that methods like ITGs, which create a tree during alignment, perform better than methods with a fixed tree established before alignment begins.

the use of a fixed tree is not the only difference between (Yamada and Knight, 2001) and ITGs; the probability models are also very different.


According to the Gold Standard used for evaluation in the ACL2005 shared task, this interpretation was correct, and therefore, for the example in Figure 3, the F-measure for the YAWA alignment was 100%.

romanian is a pro-drop language and although the translation of the English pronoun is not lexicalized in romanian, one could argue that the auxiliary "ve i" should be aligned also to the pronoun "you" as it incorporates the grammatical information carried by the pronoun.


This is because the relative positions of the two words are the same and the POS-affinity of the English personal pronouns and the Romanian auxiliaries is significant.

the SVM-based combiner deleted this link, producing the result shown in Figure 3.


Viewed globally, both words are likely to belong to the long tail of the Zipf distribution, having almost indistinguishable logarithmic IDF.

in the encyclopedia entry describing the city, the city's name is likely to appear in many sentences, while the building name may appear only in the single sentence that refers to it, and thus the latter should be scored higher.


The DRIU (1.3) indicates that V failed to identify U's intended object in utterance (1.1).

(1.3) does not explicitly mention the repair target, i.e., either book or shelf in this case.


If (1.3) is uttered when V is reaching for a book, it would be natural to consider that (1.3) is aimed at repairing V's interpretation of "the book".

if (1.3) is uttered when V is putting the book on a shelf, it would be natural to consider that (1.3) is aimed at repairing V's interpretation of "the shelf to the right".


In Traum's grounding model, the content of a DU is uniformly grounded.

things in the same DU should be more finely grounded at various levels individually.


Although Traum admitted these problems existed in his model, he retained it for the sake of simplicity.

such partial and mid-DU grounding is necessary to identify repair targets.


In brief, when a level 3 evidence is presented by the follower and negative feedback (i.e., DRIUs) is not provided by the commander, only propositions supported by the evidence are considered to be grounded even though the DU has not yet reached state F. In general, past work on discourse has targeted dialogue consisting of only utterances, or has considered actions as subsidiary elements.

this paper targets action control dialogue, where actions are considered to be primary elements of dialogue as well as utterances.


In this case, the repair target of (5.5) X is "the left box", i.e., #Dst1.

5 the pronoun "that" cannot be resolved by anaphora resolution only using textual information.


7 There are two propositions concerned with #Dst1: destination(content(α)) = #Dst1 and referent(#Dst1) = Box1.

if dest(content(α)) = #Dst1 is not correct, this means that V grammatically misinterpreted (8.1).


In a dialogue where participants are paying attention to each other, the lack of negative feedback can be considered as positive evidence (see (9d)).

it is not clear how long the system needs to wait to consider the lack of negative feedback as positive evidence.


This action will present evidence for "who is the intended agent (#Agt)" at the beginning.

the evidence for "where is the intended position (#Dst)" will require the action to be completed.


However, the evidence for "where is the intended position (#Dst)" will require the action to be completed.

if the position intended by the follower is in a completely different direction from the one intended by the commander, his misunderstanding will be evident at a fairly early stage of the action.


Their model could also handle misunderstanding regarding domain level actions.

we think that their model using coherence to detect and resolve misunderstandings cannot handle DRIUs such as (8.5), since both possible repairs for #Obj1 and #Dst1 have the same degree of coherence in their model.


The semantic orientation classification of words has been pursued by several researchers (Hatzivassiloglou and McKeown, 1997;Turney and Littman, 2003;Kamps et al., 2004;.

no computational model for semantically oriented phrases has been proposed to date although research for a similar purpose has been proposed.


With these models, the nouns (e.g., "risk" and "mortality") that become positive by reducing their degree or amount would make a cluster.

the adjectives or verbs (e.g., "reduce" and "decrease") that are related to reduction would also make a cluster.


To work with numerical scales of the rating variable (i.e., the difference between c = −1 and c = 1 should be larger than that of c = −1 and c = 0), Hofmann (2004) used also a Gaussian distribution for P (c|az) in collaborative filtering.

we do not employ a Gaussian, because in our dataset, the number of rating classes is only 3, which is so small that a Gaussian distribution cannot be a good approximation of the actual probability density function.


We test our procedure to assess Web-corpus randomness on corpora built using seeds chosen following different strategies.

the method per se can also be used to assess the randomness of corpora built in other ways; e.g., by crawling the Web.


We are also interested in evaluating the effect that different seed selection (or, more in general, corpus building) strategies have on the nature of the resulting Web corpus.

rather than performing a qualitative investigation, we develop a quantitative measure that could be used to evaluate and compare a large number of different corpus building methods, as it does not require manual intervention.


The bootstrap estimate of δ i , calledδ i is the mean of the B estimates on the individual datasets: Bootstrap estimation can be used to compute the standard error of δ i : Instead of building one matrix of average distances over N trials, we could build N matrices and compute the variance from there rather than with bootstrap methods.

this second methodology produces noisier results.


Also note that, of the 24.3 pairs/seed output, 5.25 are listed in the French-Japanese Scientific Dictionary.

only 3.9 of those pairs are included in M'*.


For the WSJ testing set, the 2 billion word Web Corpus does not achieve the performance of the Gigaword (see Table 4).

the 10 billion word Web Corpus results approach that of the Gigaword.


The InvR values differ by a negligible 0.05 (out of a maximum of 5.92).

on a per word basis one corpus can sigificantly outperform the other.


Therefore, the long jump distance between the sentences is five.

the best Levenshtein path contains one deletion edge, four identity and five consecutive substitution edges; the Levenshtein distance between the two sentences is six.


However, this is counter-intuitive, as replacing a word with another one which has a similar meaning will rarely change the meaning of a sentence significantly.

replacing the same word with a completely different one probably will.


The same holds for different cases, numbers and genders of most nouns and adjectives.

it does not hold if verb prefixes are changed or removed.


On the Chinese-English task, the smoothed BLEU score has a higher sentence-level correlation than WER.

this is not the case for the Arabic- Table 3: Correlation (r) between human evaluation (adequacy + fluency) and automatic evaluation with BLEU, WER, and CDER (NIST 2004 evaluation; sentence level).


For instance a comma and a period may have different functionalities when tagging the dictionary.

when transformations are allowed to make reference to tokens, i.e., when lexicalized transformations are allowed, some relevant information may be lost because of sparsity.


(2004) report an error rate (Pk) of 0.25 on segmenting broadcast news stories using unsupervised lexical cohesion-based approaches.

topic segmentation of multiparty dialogue seems to be a considerably harder task.


(2003) have shown that a model integrating lexical and conversation-based features outperforms one based on solely lexical cohesion information.

the automatic segmentation models in prior work were developed for predicting toplevel topic segments.


This is suggested by the fact that absolute performance on subtopic prediction degrades when any of the interactional features are combined with the lexical cohesion features.

the interactional features slightly improve performance when predicting top-level segments.


predicting from ASR output Features extracted from ASR transcripts are distinct from those extracted from human transcripts in at least three ways: (1) incorrectly recognized words incur erroneous lexical cohesion features (LF), (2) incorrectly recognized words incur erroneous cue phrase features (CUE), and (3) the ASR system recognizes less overlapping speech (OVR).

to the finding that integrating conversational features with lexical cohesion features is useful for prediction from human transcripts, Table 3 shows that when operating on ASR output, neither adding interactional nor cue phrase features improves the performance of the model using only lexical cohesion features.


The intuition here is that if both of those parameters were varying between a corpus of 19 students to 20 students, then we can't assume that our policy is stable, and hence not reliable.

if these parameters converged as more data was added, this would indicate that the MDP is reliable.


Note that these features are meant to capture the same information in both the source and channel models of Knight and Marcu (2000).

here they are merely treated as evidence for the discriminative learner, which will set the weight of each feature relative to the other (possibly overlapping) features to optimize the models accuracy on the observed data.


This may seem problematic since longer compressions might contribute more to the score (since they contain more bigrams) and thus be preferred.

in Section 3.2 we define a rich feature set, including features on words dropped from the compression that will help disfavor compressions that drop very few words since this is rarely seen in the training data.


For instance, dropping verbs is not that uncommon -a relative clause for instance may be dropped during compression.

dropping the main verb in the sentence is uncommon, since that verb and its arguments typically encode most of the information being conveyed.


These parsers have been trained out-of-domain on the Penn WSJ Treebank and as a result contain noise.

we are merely going to use them as an additional source of features.


It is not unique to use soft syntactic features in this way, as it has been done for many problems in language processing.

we stress this aspect of our model due to the history of compression systems using syntax to provide hard structural constraints on the output.


we cannot neces-sarily calculate this feature since the extent of the production might be well beyond the local context of first-order feature factorization.


During system development, we found this measure to be effective because it was sensitive to the number of CFs mentioned in a given sentence as well as to the strength of the evaluation for each CF.

many sentences may have the same CF sum score (especially sentences which contain an evaluation for only one CF).


Finally, some users found the editing/viewing interface to be good despite the fact that several customers really disliked the viewfi nder .


The pCRU choices reflect frequency in the SUMTIME corpus: later (837 in-stances) and by late evening (327 instances) are more common than by midnight (184 instances).

forecast readers dislike this use of later (because later is used to mean something else in a different type of forecast), and also dislike variants of by evening, because they are unsure how to interpret them ; this is why SUMTIME uses by midnight.


Traditionally, these principles have been defined via an interpretation of the Gricean maxims (Dale, 1989;Reiter, 1990;Dale and Reiter, 1995;van Deemter, 2002) 1 .

little attention has been paid to contextual or intentional influences on attribute selection (but cf.


spatial distance, colour, and shape) and then seeking to merge identical groups determined on the basis of these different qualities (see Thorisson (1994)).

the grouping strategy can still return groups which do not conform to human perceptual principles.


In Figure 1, for example, the pairs {e 1 , e 2 } and {e 5 , e 6 } could easily be consecutively ranked, since the distance between e 1 and e 2 is roughly equal to that between e 5 and e 6 .

they would not naturally be clustered together by a human observer, because grouping of objects also needs to take into account the position of the surrounding elements.


There was a significant main effect of domain type (F = 6.399, p = .01), while the main effect of algorithm was marginally significant (F = 3.542, p = .06).

there was a reliable type × algorithm interaction (F = 3.624, p = .05), confirming the finding that the agreement between target and human output differed between domain types.


Some idioms, such as by and large, contain syntactic violations; these are often completely fixed and hence can be listed in a lexicon as "words with spaces" (Sag et al., 2002).

among those idioms that are syntactically well-formed, some exhibit limited morphosyntactic flexibility, while others may be more syntactically flexible.


The main clause continuation is syntactically more likely.

there is a second, semantic clue provided by the high plausibility of deer being shot and the low plausibility of them shooting.


data, so we can assume consistency of the ratings.

in comparison to the McRae data set, the data is impoverished as it lacks ratings for plausible agents (in terms of the example in Table 1, this means there are no ratings for hunter).


Maximising the data likelihood during λ estimation does not approximate our final task well enough: The log likelihood of the test data is duly improved from −797.1 to −772.2 for the PropBank data and from −501.9 to −446.3 for the FrameNet data.

especially for the FrameNet training data, performance on the correlation task diminishes as data probability rises.


Such implementations do not make direct use of any recorded human motions; this means that they generate average behaviours from a range of people, but it is difficult to adapt them to reproduce the behaviour of an individual.

other ECA implementations have selected non-verbal behaviour based directly on motion-capture recordings of humans.


The findings from the corpus analysis generally agree with those of previous studies (e.g., the predicted pitch accent was correlated with nodding and eyebrow raises), and the corpus as it stands has proved useful for the task for which it was created.

to get a more definitive picture of the patterns in the corpus, it should be re-annotated by multiple coders, and the inter-annotator agreement should be assessed.


In (Bangalore Johnston and Banga-lore, 2005), we have shown that such grammars can be compiled into finite-state transducers enabling effective processing of lattice input from speech and gesture recognition and mutual compensation for errors and ambiguities.

like other approaches based on handcrafted grammars, multimodal grammars can be brittle with respect to extra-grammatical, erroneous and disfluent input.


The aspectual marker is present on the verb byHbw in the LA example in Figure 1. lys Construction (LYS): In the MSA data, lys is interchangeably marked as a verb and as a particle.

in the LA data, lys occurs only as a particle.


They should capture more specific phenomena.

they are not always applicable as we never apply a decision tree when there is a time expression between any of the events involved.


c4.5, to some extent, makes provision for this and prunes the decision trees.


Longer paths typically impose stricter constraints on the slot fillers.

they tend to have fewer occurrences, making them more prone to errors arising from data sparseness.


Finally, we note that our particular figures are specific to this dataset and the biological abstracts domain.

the annotation and analysis methodologies are general and are suggested as highly effective tools for further research.


Using (simplified) RMRS representations, this might amount to: (3) l:a:boil v(e), a:ARG1(k), a:ARG2(x), water(x) (4) l:a:boil v(e), a:ARG2(x), water(x) Such an approach was used for a time in the ERG with unaccusatives.

it turns out to be impossible to carry through consistently for causative alternations.


Navigli and Lapata don't report overall results and therefore, we can't directly compare our results with theirs.

we can see that on a PoS-basis evaluation our results are consistently better for nouns and verbs (especially the Ppr w2w method) and rather similar for adjectives.


Thus, it must respect the following properties: As the objective function is linear with respect to X and as the constraints that X must respect are linear equations, we can solve the clustering problem using an integer linear programming solver.


Many previous works exist in NEs recognition and classification.

most of them do not build a NEs resource but exploit external gazetteers (Bunescu and Pasca, 2006), (Cucerzan, 2007).


From a methodological point of view, our proposal is also close to (Ehrmann and Jacquet, 2007) as the latter proposes a system for NEs finegrained annotation, which is also corpus dependent.

in the present paper we use all syntactic relations for measuring the similarity between NEs whereas in the previous mentioned work, only specific syntactic relations were exploited.


These approaches may be more appropriate for users who are MT researchers themselves.

our approach focuses on providing intuitive visualization of a variety of information sources for users who may not be MTsavvy.


Then, the clause is incorporated into the subtask tree.

agent utterances a dialog system starts planning an agent utterance by identifying the subtask to contribute to next, st a i , based on the subtask tree so far ( , as shown in Equation 3 (Table 1) .


This method is more likely to mislabel tree-internal nodes than those immediately above the leaves.

the same non-terminals show up as error-prone for this method as for the others: out-of-domain, checkavailability, order-problem and summary.


2 A key element in these previous attempts at adapting LDA for WSD is the tendency to remain at a high level, document-like, setting.

we make use of much smaller units of text (a few sentences, rather than a full document), and create an individual model for each (ambiguous) word type.


In 70% of all items, the human judges chose the same string as the original author.

the remaining 30% of the time, the human judges picked an alternative as being the  .


On the one hand, although each of the 4 alternatives was chosen at least once from Table 4, there is a clear preference for one string (and this is also the original string from the TIGER Corpus).

there is no clear preference 9 for any one of the alternatives in Table 5, and, in fact, the alternative that was selected most frequently by the participants is not the original string.


Such rules have been formalised and implemented for the 56 productive prefixes of Italian (Iacobini 2004) 1 , with their French translation equivalent.

finding the translation equivalent for each rule requires specific studies a, ad, anti, arci, auto, co, contro, de, dis, ex, extra, in, inter, intra, iper, ipo, macro, maxi, mega, meta, micro, mini, multi, neo, non, oltre, onni, para, pluri, poli, post, pre, pro, retro, ri, s, semi, sopra, sotto, sovra, stra, sub, super, trans, ultra, vice, mono, uni, bi, di, tri, quasi, pseudo.


As we stated, we chose two morphologically related languages on purpose: they present less divergences to deal with and allow concentrating on the method.

the proposed method (especially that contrastive knowledge acquisition) can clearly be ported to another pair of languages (at least inflexional languages).


Similarly, the metrics proposed for text generation by  (simple accuracy, generation accuracy) are based on string-edit distance from an ideal output.

the work of (Wan et al., 2005) and (Mutton et al., 2007) directly sets as a goal the assessment of sentence-level fluency, regardless of content.


We noticed that if constant relevance values are used, the top ranked queries will consist of a rather small set of top ranked n-grams that are paired with each other in all possible combinations.

it is likely that each time an n-gram is used in a query, the need for finding more occurrences of this particular n-gram decreases.


We have tested an approximate solution that allows for fast computing.

the real effect of this addition was insignificant, and a further description is omitted in this paper.


In the above experiments, Good-Turing (GT) smoothing with Katz backoff was used, although modified Kneser-Ney (KN) interpolation has been shown to outperform other smoothing methods (Chen and Goodman, 1999).


A greedy Viterbi training is then applied to improve this initial guess.

our BP/EM training do not need to compute correlation scores and start the training with uniform parameters.


In fact, brief examination shows that less than half of source language terms successfully pass translation and disambiguation stage.

more than 80% of terms which were skipped due to lack of available translations were re-discovered in the target language during the extension stage, along with the discovery of new correct terms not existing in the given source definition.


It shares much of its motivation with co-training (Blum and Mitchell, 1998) in improving initial models by leveraging additional data that is easy to obtain.

as the examples of Section 2.3 illustrate, COREF's interactions with its users offer substantially more information about interpretation than the raw text generally used for co-training.


We could run TiMBL with different values of k, as this should lead to better feature integration.

this is difficult to explore without development data, and initial experiments with higher k values were not promising (see section 4.2).


On the one hand, one can simply additively combine features into a larger vector for training, as described in section 4.2.

one can use one set of features to constrain another set, as described in section 5.


Media also offers the possibility to compare with the state-of-the-art, which our re-rankers seem to improve.

we need to consider that many Media corpus versions exist and this makes such comparisons not completely reliable.


Thus, one essential issue is to identify more complex expressions which, in appropriate contexts, convey the same (or similar) meaning.

more generally, we are also interested in pairs of expressions in which only a uni-directional inference relation holds 1 .


One way to deal with textual inference is through rule representation, for example X wrote Y ≈ X is author of Y.

manually building collections of inference rules is time-consuming and it is unlikely that humans can exhaustively enumerate all the rules encoding the knowledge needed in reasoning with natural language.


If the score is above a threshold the rule is applied.

these lexical-syntactic rules are only used in about 3% of the attempted proofs and in most cases there is no lexical variation.


At a manual analysis, close to 80% of these are correct rules; this is higher than the estimated accuracy of DIRT, probably due to the bias of the data which consists of pairs which are entailment candidates.

given the small number of inference rules identified this way, we performed another analysis.


These annotations are often used as training data for semantic role labeling systems.

the applicability of these systems is limited to those words for which labeled data exists, and their accuracy is strongly correlated with the amount of labeled data available.


To measure the working memory burden of a text, we'd like to capture the number of discourse entities that a reader must keep in mind.

the "unique entities" identified by the named entity recognition tool may not be a perfect representation of this -several unique entities may actually refer to the same real-world entity under discussion.


First, as can be expected the ASR 1-best output is typically error-prone especially when a user query originates from a noisy environment.

aSR word confusion networks which compactly encode multiple word hypotheses with their probabilities have the potential to alleviate the errors in a 1-best output.


Equation 12 shows a method for parsing the 1best ASR output using the FST.

a similar method can be applied for parsing WCNs.


If a word does not appear in the query at all, its weight becomes equal to the usual tf w,s idf w,S : The overall novelty ranking formula is based on the query-dependent PageRank introduced in Equation (2).

since we already incorporate the relatedness to the query in these two settings, we focus only on related sentences and thus may drop the relatedness to the query part from (2): We set λ to the same value as in OTTERBACHER.


We see in Table 4 from setup #5 experiments that training and testing on error-containing utterances led to a dramatic improvement in F 1score.

our results for experiments using setup #6 (where training data was filtered to contain errorful data but test data was fully preserved) are consistently worse than those of either setup #2 (where both train and test data was untouched) or setup #5 (where both train and test data were prefiltered).


Word segmentation and tagging are the necessary initial steps for almost any language processing system, and Chinese parsers are no exception.

automatic Chinese word segmentation and tagging has been recognized as a very difficult task (Sproat and Emerson, 2003), for the following reasons: First, Chinese text provides few cues for word boundaries (Xia, 2000;Wu, 2003) and part-ofspeech (POS) information.


Recall that based on the deterministic segmentation and tagging results produced by PKU's tokenizertagger, our system can only parse 80 out of the 101 sentences, and among the 21 completely failed sentences, 20 sentences failed due to segmentation and tagging mistakes.

after the application of the hand-crafted FST rules for postprocessing, 100 out of the 101 sentences can be parsed.


Following a strategy widely used in Chinese word segmentation, we did this by regarding the problem as a character tagging problem.

since we intended to learn rules that deal with segmentation and POS tagging simultaneously, we could not adopt the BIO-coding approach.


this results in a very skewed class distribution because the next speaker is the intended addressee 41% of the time, and 38% of instances are plural -the L 1 L 2 L 3 35.17% 30.34% 34.49% Table 2: Distribution of addressees for singular you remaining two classes therefore make up a small percentage of the data.


For the generic vs. referential task, the discourse and multimodal classifiers both outperform the majority class baseline (p < .001), achieving accuracy scores of 68.71% and 68.48% respectively.

to when using manual transcriptions and annotations (see Section 6.1), removing forward-looking (FL) information reduces performance (p < .05).


As all possible analyses are computed, any number of best parses can be extracted.

other treebank parsers use sophisticated search strategies to find the most probable analysis without examining the set of all possible analyses (Charniak et al., 1998;Klein and Manning, 2003).


The reason for this high complexity is the problem of unrestricted crossing configurations, appearing when dependency subtrees are allowed to interleave in every possible way.

just as it has been noted that most non-projective structures appearing in practice are only "slightly" nonprojective , we characterise a sense in which the structures appearing in treebanks can be viewed as being only "slightly" ill-nested.


For example, for the imperfective passive, the CC C root pattern appears in the template C1C EC, and the root is what is left if the two vowels in the stem are skipped over.

we want to extract both the derivational pattern and the root, and the problem for finite state methods, as discussed in Section 1.2, is that both are spread throughout the stem.


While significant, the improvement from the Baseline to LexFilter is quite small, which is due to the Baseline's own rather strong illegal analyses filtering heuristic.

unlike the oracle segmentation case, here the semisupervised lexical probabilities (LexProbs) have a major effect on the parser performance (∼ 69 to ∼ 73.5 F-score), an overall improvement of ∼ 6.6 F-points over the Baseline, which is the previous state-of-the art for this joint task.


Moreover, simultaneous interpretation requires a soundproof booth with audio equipment, which adds an overall cost that is unacceptable for all but the most elaborate multilingual events.

a simultaneous translation system also needs time and effort for preparation and adaptation towards the target application, language and domain.


This coefficient shows a global agreement between all the judges, which goes beyond Cohen's Kappa coefficient.

a low coefficient requires a more detailed analysis, for instance, by using Kappa for each pair of judges.


T036 is more fluent due to the less technical nature of the speech and the more general vocabulary used.

the T036-2 and T036-3 excerpts get a lower quality score, due to the description of data collections or institutions, and thus the use of named entities.


Indeed, there is a high lexical repetition, a large number of named entities, and the quality of the excerpt is very training-dependant.

the system runs into trouble to process foreign names, which are very often not understandable.


Although we can not compare the performance of the restricted automatic system to that of the restricted interpreter (since data sets of questions are different), it seems that of the interpreter is better.

the loss due to subjective evaluation seems to be higher for the interpreter than for the automatic system.


This clearly shows the difficulty of the whole task.

the human end-to-end evaluation of the system in which the system is compared with human interpretation shows that the current translation quality allows for understanding of at least half of the content, and therefore, may be already quite helpful for people not understanding the language of the lecturer at all.


In principle, we can use all of the induced affixes as features for training a POS tagger and an NE recognizer.

we choose to use only those features that survive our feature selection process (to be described below), for the follow-ing reasons.


(2007) reduce the number of POS tags from 45 to 5 when training a factorial dynamic CRF on a small dataset (with only 209 sentences) in order to reduce training and inference time.

we propose a relatively simple model for jointly learning Bengali POS tagging and NER, by exploiting the limited dependencies between the two tasks.


Note, however, that this observation does not hold for English, since many prepositions and determiners are part of an NE.

this observation largely holds for Bengali because prepositions and determiners are typically realized as noun suffixes.


To make these unsupervised taggers practical, one could attempt to automatically construct a POS lexicon, a task commonly known as POS induction.

pOS induction is by no means an easy task, and it is not clear how well unsupervised pOS taggers work when used in combination with an automatically constructed pOS lexicon.


Similar performance trends can be observed when Lexicon 2 is used (see Figure 3(b)).

both baselines achieve comparatively lower tagging accuracies, as a result of the higher unseen word rate associated with Lexicon 2.


However, phrase-based models can fail to reorder words or phrases which would seem obvious if it had access to the POS tags of the individual words.

for example, a translation from french to English will usually correctly reorder the french phrase with POS tags NOUN ADJECTIVE if the surface forms exists in the phrase table or language model, e.g., phrase-based models may not reorder even these small two-word phrases if the phrase is not in the training data or involves rare words.


The ATS models do provide an integrated approach, but their lexical translation is limited to the word level.

to prior work, we present a integrated approach that allows POS-based reordering and phrase translation.


The beam search decoding algorithm is unchanged from traditional phrase-based and factored decoding.

the creation of translation options is extended to include the use of factored templates.


The factored template models were retrained with increased maximum phrase length but this made no difference or negatively impacted translation performance, Figure 1.

using larger phrase lengths over 5 words does not increase translation performance, Figure 1: Varying max phrase length as had been expected.


It can be seen from Table 3 that generalizing the reordering model on POS tags (line 2a) improves performance, compared to the non-lexicalized reordering model (line 2).

this performance does not improve over the lexicalized reordering model on surface forms (line 1a).


In fact, WordNet has been used as a de-facto standard repository of meanings.

to our knowledge, the meanings represented by WordNet have been only used for WSD at a very fine-grained sense level or at a very coarse-grained class level.


In this way, Wikipedia provides a new very large source of annotated data, constantly expanded (Mihalcea, 2007).

some research have been focused on using predefined sets of sense-groupings for learning class-based classifiers for WSD (Segond et al., 1997), (Ciaramita and Johnson, 2003), (Villarejo et al., 2005), (Curran, 2005) and (Ciaramita and Altun, 2006).


Most of the later approaches used the original Lexicographical Files of WN (more recently called SuperSenses) as very coarse-grained sense distinctions.

not so much attention has been paid on learning class-based classifiers from other available sense-groupings such as WordNet Domains (Magnini and Cavaglià, 2000), SUMO labels (Niles and Pease, 2001), EuroWordNet Base Concepts , Top Concept Ontology labels (Alvez et al., 2008) or Basic Level Concepts (Izquierdo et al., 2007).


More specifically, it is the observation that coordination involves two or more constituents of the same categories.

there are a significant number of more complex cases of coordination that defy this generalization and that make the parsing task of detecting the right scope of individual conjuncts and correctly delineating the correct scope of the coordinate structure as a whole difficult.


When comparing the results of experiment 1 (nbest parsing) with the present one, it is evident that the F-scores are very similar: 74.53 for the 50-best reranking setting, and 74.46 for the one where we provided the gold scope.

a comparison of precision and recall shows that there are differences: 50-best reranking results in higher recall, providing gold scope for coordinations in higher precision.


If the same parser is used for this step and for the final parse, we can be certain that only scopes are extracted that are compatible with the grammar of the final parser.

parse forests are generally stored in a highly packed format so that an exhaustive search of the structures is very inefficient and proved impossible with present day computing power.


Since our proposed conjuncts cannot cross these boundaries, the correct second conjunct, ausgebildete Industriekauffrau aus Oldenburg, cannot be suggested.

if we remove these chunk boundaries, the number of possible conjuncts increases dramatically, and parsing times become prohibitive.


When identifying sentences from which story highlights are generated, the situation is slightly different, as the number of story highlights is not fixed.

most stories have between three and four highlights, and on average between four and five sentences per story from which the highlights were generated.


Essentially, a system could simply pick the first two sentences of each article and might thus achieve higher precision scores, since it is less likely to return 'wrong' sentences.

if the scores are similar but there is a difference in the number of unique sentences extracted, this means a system has gone beyond the first 4 sentences and extracted others from deeper down inside the text.


Candidacy for either transliteration or translation is not necessarily determined by orthographic features.

to English (and many other languages), proper names in Hebrew are not capitalized.


Precision and recall obtained are 80% and 82%, respectively.

although foreign words are indeed often TTTs, many originally Hebrew words should sometimes be transliterated.


Foreign words, which retain the sound patterns of their original language with no semantic translation involved, are also (back-)transliterated.

names of countries may be subject to translation or transliteration, as demonstrated in the following examples: We use information obtained from POS tagging (Bar-Haim et al., 2008) to address the problem of identifying TTTs.


The method uses only positive examples for learning which words to transliterate and achieves over 38% error rate reduction when compared to the baseline.

to previous stud-ies this method does not use any parallel corpora for learning the features which define the transliterated terms.


This indicates that, in our experimental conditions, optimization efforts can never reach the global maximum, but it also indicates that searching for less expensive solutions nevertheless might lead (at least) to a local maximum.

if it is true that the goal function is not monotonic, there is no guarantee that the optimal solution actually constitutes the local maximum, i.e.


The search stops with the first consistent solution (as we suggest in the present paper).

it is difficult to quantify the number of cascades needed to come to it and moreover, the full ILP machinery is being used (so again, constraints need to be extensionalized).