When the left-hand side of composition is a transducer and the right-hand side is an acceptor, then their composition produces a transducer in which the range of the left-hand side relation is intersected with the set of strings accepted by the righthand side argument.

if A transduces string x to strings {y, z}, and B accepts y then A • B transduces from x to y.


From our perspective, this ambiguity is spurious, so we stipulate that identity elements may only be siblings to (i.e., on the right-hand side of a production with) another  terminal.

an expression like one thousand one hundred can only be parsed as (+ (* 1 1000) (* 1 100)).


If we were to simply use the temporal information at the document level, the timestamps for the search areas of MH370 at different periods will be incorrect.

we propose a method to extract timestamps (i.e., temporal expressions) at the sentence level.


Secondly, if we normalize the score, the information on the number of evidence sentences, which is important for the LSP method to recognize true taxonomic relationships, will be lost.

we propose to use Equation (2), which combines both information on the number of evidence sentences (embedded inside the original Score LSP score) and the normalized "weighted size" of C W eb (t 1 , t 2 ).


Finally, the two annotators discuss to come up with the gold standard taxonomies.

the number of nodes and average depth of the taxonomies are summarized in Table 1.


The reason is that although some older taxonomic relations are mentioned occasionally in the new data, they are still valid.

if we ignore the older data, their taxonomic relations will be lost in the new taxonomy when it is constructed with only the new data.


The ANOVA detected no statistically significant differences between the models: F(3,24) = 1.56, p = .22.

when decoding the most concrete nouns there was no significant difference in accuracy between image-based and any text-based model.


used more images to build noun representations (on average 350 images per noun compared to 20 used here), and nouns in the ImageNet images were segmented according to bounding boxes.

their input may have been less noisy than Google Images (which we used because of its wider coverage).


Of course we expect modal structure in text-based models commensurate with what people are inclined to report in writing; e.g., it is easy to convey in text that both bananas and lemons are yellow and curvy, and lightbulbs and pears have similar shapes.

we would anticipate correspondences in semantic similarities between image and text-based models and for these correspondences to extend to match neural similarities, e.g., as induced by participants viewing pictures of objects (Carlson et al., 2014).


A second limitation of the current approach, as pointed out by a reviewer, is that the Google image search algorithm (the workings of which are unknown to the authors) may not perform as well for abstract words as it does for concrete words.

the visual model may have been handicapped compared to the textual model when decoding neural representations associated with more abstract words.


As we rely on external embeddings, which encode semantic similarities between lexical items, the dot product of this term and (x rubbedx rinsed ) will be high.

7 ϕ(d, h (t) ) is expected to be positive for d = "soap", thus, predicting "soap" as the likely forthcoming DR.


However, recall that independent random vectors in high dimensions are almost orthogonal, and, assuming they are bounded, their dot products are close to zero.

the products of the relevant ("non-random") terms, in our example (x scrubbedx rinsing ) and (x rubbedx rinsed ), are likely to overcome the ("random") noise.


For longer sentences, the predictors yield deeper constituent hierarchies, offering corresponding lookahead features.

compared to the baseline parser, the performance of the parser with lookahead features decreases more slowly as the length of the sentences increases.


• If ψ 1 = φ and ψ 2 = φ, then ψ 1 = ψ 2 .

there are four possible types of transition from a state T = (j, {σ 1 .


input: in addition, define singles(j) to be the set of phrases that translate the single word at position j: singles(j) = {p : s(p) = j and t(p) = j} Next, define h to be the smallest integer such that for all j, |first(j)| ≤ h, |last(j)| ≤ h, and |singles(j)| ≤ h. h is a measure of the maximal ambiguity of any word x j in the input.


1 Our system learns inventories of reusable phonotactically licit structures from existing lexical items, and assembles new lexical items by combining these learned phonotactic patterns using phonologically plausible structurebuilding operations.

instead of modeling phonotactic generalizations in terms of constraints, we treat the problem as a problem of learning language specific inventories of phonological units and language specific biases on how these phones are likely to be combined.


The second case of the definition covers or-nodes in the graph, where ψ s s is the probability associated with choosing outgoing arc s, s from parent ornode s to child node s .

or-nodes define mixture distributions over outgoing arcs.


The log probability of a form under a model corresponds to the description length of that form under the model; if a model assigns a higher log probability to a form, that means the model is capable of compressing the form more than other models.

we compare models on their ability to assign high probabilities to phonological forms, as in Goldsmith and Riggle (2012).


In contrast, when we halve the contribution from the target context, the result completely loses its fluency by repeatedly generating the translation of "chūkǒu" (i.e., "the export of") until the generated translation reaches the maximum length.

this example indicates that source and target contexts in NMT are highly correlated to translation adequacy and fluency, respectively.


As shown in Table 1, NMT can get stuck in an infinite loop repeatedly generating a phrase due to the overwhelming influence of the source context.

generation terminates early because the translation reaches the maximum length allowed by the implementation, even though the decoding procedure is not finished.


For example, there are dozens of syntactic edge types, each corresponding to a Stanford dependency label.

in our experiments we resort to using only the coarse-grained types: word adjacency, syntactic dependency, etc.


The errors identified by curators offer direct supervision to the machine reading system for continuous improvement.

the most important goal is to attain high recall and reasonable precision.


Accordingly, on  Each syntactic pattern inherently encodes the positions in the tree of the cause and effect heads.

matching these patterns is the first step of argument ID, in addition to (tentative) connective discovery.


Further, when we ran the baseline on the PTB subcorpus with and without gold parses, we saw a similar improvement.

although the limited data and the classifier's failure to upweight positives are still the primary handicaps, better parses would be somewhat helpful for at least the syntax-based approach.


Since we exploit character-level similarities between words, we are able to better model infrequent words.

we should also be more robust to the size of the training data that we use.


The treebank of each synthetic language is generated by stochastically permuting the subtrees in a given real treebank to match the word order of other real languages.

we have many synthetic languages that are Object-Verb like Hindi but also Noun-Adjective like French.


It predicts this for each dependency relation type r ∈ R, such as r = dobj.

the output is a vector of predicted probabilitiesp ∈ [0, 1] N .


To train our model, we will try to minimize the evaluation objective (1) averaged over the training lan- 3 In our experiment, we chose w = 8 by cross-validation over w = 2, 4, 8, 16, ∞.

4 the EC heuristic examines the correlation between relations and tags in the training treebanks.


We initialize the model of (6)-(7) so that the estimated directionalityp(→| r, L), regardless of L, is initially a weighted mean of r's directionalities in the training languages, namelȳ where This is done by setting V = 0 and the bias (b V ) r = logp r 1−pr , clipped to the range [−10, 10].

we make sensible initial predictions even for rare relations r, which allows us to converge reasonably quickly even though we do not update the parameters for rare relations as often.


Rather than try to obtain a UD label like r = amod for each edge, we label the edge deterministically with a POS pair such as r = (parent = NOUN, child = ADJ).

we will attempt to predict the directionality of each POS-pair relation type.


Japanese Word segmentation accuracies around 99% have already been reported for newspaper domains in Japanese (Kudo et al., 2004).

we only conducted experiments on segmenting Twitter text.


It mainly targets transfer learning and is segmented according to BCCWJ (Basic Corpus of Contemporary Written Japanese) standards from the National Institute of Japanese Language (Maekawa, 2007).

for the labeled data we used the "core" subset of BCCWJ consisting of about 59K sentences plus 500 random sentences from the Twitter dataset.


Thai Unsegmented languages, such as Thai, Lao, Myanmar, and Kumer, are also prevalent in South East Asia and are becoming increasingly important targets of natural language processing.

we also conducted an experiment on Thai, using the standard InterBEST 2009 dataset (Kosawat, 2009).


In practice, we terminated the EM algorithm in Figure 9 early after a few iterations.

with a more adaptive semi-supervised learning framework, we expect that NPYCRF will achieve higher accuracy than the current performance.


Different Degree of Contribution among Multiple Authors: Academic publications are often written by more than one author.

we need to distinguish the influence of each author on a citation between two publications.


Manually annotating word-level quality labels is time-consuming and expensive; on the other hand, post-editing translated sentences is  commonly part of the workflow of crowd-sourced and professional translation services.

getting quality labels for free from sentences that have already been post-edited is a much more realistic and sustainable process.


From a machine learning pespective, QE is a sequence labeling problem (i.e., whose output sequence has a fixed length and a small number of labels), while APE is a sequence-to-sequence problem (where the output is of variable length and spans a large vocabulary).

we can regard APE-based QE as a "projection" of a more complex and fine-grained output (APE) into a simpler output space (QE).


We first model and characterize features for each argument and then aggregate them by side to predict the relative success of each side.

the feature vectors for a side can be formulated as the summation of feature vectors of its arguments, i.e.


As shown in Table 6, the precision of rules produced was CASI is 68%, compared with AMIE* which achieved 59% on this metric.

cASI could canonicalize five times more schemas with 9% more precision.


Unfortunately, executing LOLS naively in our setting is prohibitive because it would run inference from scratch millions of times under different policies, training examples, and variations of the decision sequence.

this paper presents efficient algorithms for repeated inference, which are applicable to a wide variety of NLP tasks, including parsing, machine translation and sequence tagging.


for j := 1 to minibatch size : for t := 0 to T −1 : Intervene: Evaluate each action at s t forā t ∈ A(s t ) : 11: 12: Improve: Train with dataset aggregation 13: Finalize: Pick the best policy over all iterations encountered when running the new policy π i+1 (or when parsing test sentences).

beyond just regularizing the training objective, we apply dataset aggregation : we take the training set to include not just Q i but also the examples from previous iterations (line 13).


To record the dependence of β ikx on other quantities, our hypergraph includes the If k − i = 1, then values of β ikx are instead computed at line 6, which does not access any other β values or the pruning mask.

our hypergraph includes the hyperedge v ikx ≺(ġ) whenever i = k −1, 0 ≤ i < k ≤ n, and With this setup, the value β ikx is the maximum score of any derivation of vertexβ ikx (a tree rooted aṫ β ikx , representing a subderivation), where the score of a derivation is the product of its leaf values.


Sincep(d) is a product of rule weights and pruning mask bits at d's leaves ( §5.1), each appearing at most once, bothr and Z vary linearly in any one of these inputs provided that all other inputs are held constant.

the exact effect onr or Z of changing an input m ik can be found from the partial derivatives with respect to it.


In practice, retraining the policy to match the oracle will not match it perfectly in either case.

the baseline method has a further disadvantage: when it trains a policy, its training objective weights all gold or all non-gold examples equally, whereas LOLS invests greater effort in matching the oracle on those states where doing so would give greater downstream reward.


This can occur because some basis models may be initialized with parameters that are globally superior.

the "dead" basis models will receive near-zero gradient updates, and therefore can never improve.


Twitter's "following" relation is a relatively low-cost form of social engagement, and it is less public than retweeting or mentioning another user.

it is unsurprising that the follower network is least useful for socially-informed personalization.


11 As shown in Table 7, the datasets have highly skewed class distributions.

we use the average F1 score of positive and negative classes as the evaluation metic.


This paves the way to assessing our model not solely based on the listener's classification accuracy, but also in terms of how qualitative features of the speaker's production compare to that of our human participants.

the current section briefly reviews some novel findings from the human corpus that we use to inform our model assessment.


3 Negatives In our referential contexts, negation is likely to play a role similar to that of comparatives: a phrase like not the red or blue one singles out the third color, and blue but not bright blue achieves a more nuanced kind of comparison.

as with comparatives, we expect negation to be more likely where one or more distractors are close to the target.


We expect speakers to prefer basic color terms wherever they suffice to achieve the communicative goal, since such terms are most likely to succeed with the widest range of listeners.

a speaker might choose blue even for a clear periwinkle color.


However, if the target were the cyan, the speaker would have many good options.

the fact that the speaker chose blue is interpreted as evidence for the true target.


(2015) produces 617 unique affixes when segmenting 10000 English words.

we explicitly encourage the model towards assignments with the least number of affixes.


Each candidate z is a tuple (string, label), where label refers to the label of the potential edge.

the marginal probability is where Σ * is the set of all possible strings.


In contrast, Turkish is highly agglutinative, with minimal (if any) transformations, but each word can have up to hundreds of related forms.

sibling features have different effects on English and Turkish, leading to changes of −0.3% and +2.1% in F1 score respectively.


This makes a naive character-level approach, such as in Luong and Manning (2016), computationally prohibitive.

reducing the length of the source sequence is key to ensuring reasonable speed in both training and decoding.


It is well known that automatic evaluation metrics such as BLEU encourage reference-like translations and do not fully capture true translation quality (Callison-Burch, 2009;Graham et al., 2015).

we also carry out a recently proposed evaluation from Graham et al.


We evaluated the model on three important NLP tasks (i.e., sentence textual similarity, entity linking, and factoid question answering) involving both unsupervised and supervised settings.

we achieved state-of-the-art results on all three of these tasks.


First, we use standard semantic textual similarity datasets to evaluate the quality of the learned text representations of our method in an unsupervised fashion.

our method clearly outperformed the state-of-the-art methods.


We tested our method using two standard datasets (i.e., CoNLL 2003 and TAC 2010) for the EL task and a popular factoid QA dataset based on the quiz bowl quiz game for the factoid QA task.

our method outperformed recent state-of-the-art methods on both the EL and the factoid QA tasks.


We used words that appear five times or more and entities that appear three times or more in the corpus, and simply ignored the other words and entities.

our vocabulary V consisted of 705,168 words and 957,207 entities.


However, because hyper-linking to the web page itself does not make sense, these kinds of mentions cannot be observed as annotations in Wikipedia.

we use the aforementioned heuristic method to address this problem.


We train the model with the hyper-parameters and the Wikipedia corpus explained in Section 2.2.

the skip-gram model is equivalent to the pre-trained representations used in our model.


Our method depends only on the local (or textual) context available in the target document.

the performance can likely be improved further by combining a global model with our local model as frequently observed in past work (Ratinov et al., 2011;Chisholm and Hachey, 2015;Yamada et al., 2016).


We also conducted a brief error analysis using the NTEE model and the test set of the CoNLL dataset by randomly inspecting 200 errors.

22% of the errors were mentions of which the referent entities were not contained in our vocabulary.


For questions referring to the same answer, we sampled 20% of each for the development set and test sets, and the remaining 60% for the training set.

we obtained 1,535 training, 511 development, and 511 test questions for history, and 2,524 training, 840 development, and 840 test questions for literature.


Those peaks occur in different locations for different feature spaces.

different feature representations yield different predictions about listeners' discrimination.


There are three cases: Here, e out = Pt(f ) = 0, and e in ∈ (1, N ).

for all vertices v ∈ (1, N ) there is an edge g in the chain such that v is between the endpoints of g. e will cross such an edge g. To satisfy the 1-EC property, g must share an endpoint with f , which means g is either (1, 3) or (N −2, N ).


If there is no edge l 1 r 1 , where i ≤ l 1 < p < r 1 ≤ j, then p would be a valid split point.


However, that edge must be present in the overall graph, as all B items start with an ij edge (see rules 3 and 5 in Algorithm 1).

if there is no valid split point for a B, the overall graph must contain a Locked-Chain.


(ii) Combining Items: Assume that in isolation, none of the items being combined contain cycles.

a cycle in the combined item must be composed of paths in multiple items.


A path in one item can only continue in another item by passing through a visible vertex.

a cycle would have to be formed by a set of paths between visible vertices.


Since i < k < l < j, all k-(lj] edges will cross all l-[ik) edges.

applying this rule will cross all k-(lj] edges in the rightmost item.


Each p-value is compared to a threshold which is smaller or equal to α and depends on the number of evaluation datasets N. The dependence of the thresholds on N can be intuitively explained as follows: the probability of making one or more erroneous claims may increase with N, as demonstrated in Section 3.2.

in order to bound this probability by a pre-specified level α, the thresholds for p-values should depend on N. It can be shown that the Holm procedure at level α always rejects thek Bonf erroni hypotheses with the smallest p−values, wherek Bonf erroni is the lower bound for k with a confidence level of 1 − α. ,k Bonf erroni corresponding to a confidence level of 1 − α is always smaller or equal to the number of datasets for which the difference between the compared algorithms is significant at level α.


This naive coupling imperatively generates a large number of pairs that are mostly noise.

the challenge here is to effectively estimate the reliability of each pair.


The embeddings were trained on the concatenation of all the general-domain and in-domain monolingual data as presented by Table 2.

for each pair of domain and translation direction, we have four word embedding spaces: those with 300 or 800 dimensions for source and target languages.


2015, our approach includes more features and an additional classification step.

the induction of a phrase table is much slower.


This can be explained by the much higher frequency of "aigu" while the word "glaucome" is very rare, even in the in-domain monolingual data.

"aigu" has an embedding more accurate than the one of "glaucome" which is then much more difficult to project correctly across languages.


Aligments also form the basis for learning multi-lingual word embeddings (Faruqui and Dyer, 2014;Lu et al., 2015) and in the projection of syntactic and semantic annotations from one language to another (Hwa et al., 2005;McDonald et al., 2011).

there is still a prominent role for word alignment in NLP; research into improvements in word alignment is a worthy goal.


The restaurant reviews, however, only have single ratings for an overall impression.

we explore the task of adaptation from each of the five hotel aspects to the restaurant domain.


This is because, in general, aspects have zero correlations to each other in pathology reports.

it is essential for the model to have the capacity of distinguishing across different aspects in order to succeed in this task.


Moreover, our supervision for the relevance scorer is weak and noisy because the aspect keywords are obtained in a semiautomatic way.

it is not surprising that AAN-NR sometimes delivers a better classification  Table 6: Impact of adding the reconstruction component in the model, measured by the average accuracy on each dataset.


However, when the word does appear, everything but log P ( we have taken advantage of the fact that the CorEx topic model binarizes documents to assume by default that a word does not appear in the document, and then correct the contribution to the update if the word does appear.


Similarly, some of the major gains in F 1 for the disease types, and major losses in F 1 for the disaster types, do not come from the most or least frequent document labels.

if using anchoring single topics within CorEx for document classification, it is important to examine how the anchoring affects prediction for individual document labels.


For instance, the instruction "Reach the cell above the westernmost rock" translates into different goal locations in the two environments shown in Figure 1.

to enable generalization to new, unseen worlds, the model must jointly reason over the instruction text and environment configuration.


Note, to classify the relation to the DCT, in most cases, it was not important to know the concrete Document Creation Time.

we did not pass the DCT as a value to the network.


If the event happened on August 5th, 1998, the two annotations before 1998-08-31 and after 1998-08-01 before 1998-08-31 would both be satisfied.

these two different labels would be considered as correct.


They need humans' help, or they would die.

i persuaded the other people who live there to clean the beach immediately.


Our work focuses on automatically evaluating and scoring the proficiency of narrative construction in student essays.

we use a rubric 2 created by education experts and teachers, and presented by Smarter Balanced, an assessment aligned to U.S. State Standards for grades K-12.


At the same time, larger corpora may also tend to be more broad in scope and variable in style and topic, leading to less idiosyncratic patterns in word co-occurrence.

for each corpus, we curate a smaller sub-corpus that contains 20% of the total corpus documents.


Unlike SGNS, the algorithm does not perform model updates while examining the original documents.

we expect GloVe to be sensitive to random initializations but not sensitive to the order of documents.


having no replies or contributed late in the discussion).

we introduce a preprocessing step that identifies comments that are highly likely to be low karma to decrease the computation cost.


In the following two variants, we provide GRU with explicit subtraction operations, which are inspired by the well known phenomenon that minus operation can be applied to the semantics of word embeddings (Mikolov et al., 2013).

2 we subtract the semantics being translated from the untranslated FUTURE contents at each decoding step.


Finally, Subj1 and IObj2 both refer to Stephen, so [Coref(Subj1, IObj2)] returns true.

the above declarative rule dictates that addition should be performed between 5 and 4.


We use a set of rules based on units of numbers, to detect such irrelevant numbers.

we fail to detect numbers which are irrelevant due to other factors, like associated entities, or associated verb.


Given that the variability in text is much larger than the number of declarative rules that governs Math word problems, we believe that this is a good way to introduce Math knowledge to a natural language understanding system.

future work will involve extending our approach to handle a wider range of word problems, possibly by supporting better grounding of implicit variables and including a larger number of math concepts and declarative rules.


The vocabulary then not only consists of the union over all words w i in any caption of the training set, but also of the union of all attributes a i in any video across the dataset.

an embedding matrix E is used to represent words and we denote E w by an embedding vector of a given w. we obtain input word embedding vectors as: In this section, we introduce the core part of our proposed model, the multi-faceted attention mechanism.


Given a previous word, we can calculate the probability distribution of the next word p t+1 using the model described above.

we can generate captions starting from the special symbol BOS with Beam Search.


Each video is annotated with about 20 sentences.

we have 200,000 video-caption pairs in total.


This is mainly because noun phrases are often left children of some other constituent and the right branching model is unable to incorporate them into the syntactic structures of whole sentences.

although the rightbranching model is the best model in terms of PAR-SEVAL scores, it is not helpful in terms of finding noun phrases.


Perhaps the most obvious feature of machine learning that can impact the effectiveness is the random seed.

far the experiments in this paper have used a fixed seed, and like most prior research this was only implied rather than explicitly stated.


In addition, these result populations can be statistically compared for significance, allowing for stronger claims on improvement.

far this paper has presented a number of effects that can affect the results of a neural network.


Note that Kendall's τ is not suitable for the Finance domain, as its known sentiment lexicon is only binary.

our experimental setting and performance measures are all identical to those of Hamilton et al.


Furthermore, the two-phase bootstrapping method is a general framework which can utilize any lexicon-based sentiment classifier t o p roduce p seudo-labeled d ata.

the more sophisticated ProbLex-DCM could also be used instead of pSenti in this framework, which is likely to deliver an even higher performance.


The documents with their probabilities of being positive in such a middle range should be neutral.

the two elbow points in the probability curve would make appropriate thresholds for the identification of neutral sentiment, and they could be found automatically by a simple algorithm using the central difference to approximate the second derivative.


Finally, we identified the selection of relevant document sets as the most promising direction for future research.

far, our datasets center around factoid questions about entities, and as extractive RC datasets, it is assumed that the answer is mentioned verbatim.


The encoder-decoder architecture decouples the source and target language network components and makes the architecture more modular.

we can consider the encoder output (for the source language) to be embedded in a language-neutral, common subspace -a sort of interlingua.


We see that the transliteration performance of proxy languages is correlated to its orthographic similarity to the target language.

it is preferable to choose a proxy with high orthographic similarity to the target language.


the entire text from memory, but often can answer questions about underlying narrative elements of the document: the salient entities, events, places, and the relations between them.

testing understanding requires the creation of questions that examine highlevel abstractions instead of just facts occurring in one sentence at a time.


Rather, these are "ordinary" words, with very little culture-specific weight, that happen to have synonyms in English that may reflect cognates in some L1s, but not all of them.

they are used differently by authors with different linguistic backgrounds, to the extent that the authors' L1s can be identified through their use of the words in the focus set.


Aiming to disentangle the effect of native language we randomly shuffled texts produced by all authors from each country, thereby "blurring out" any topical (i.e., subreddit-specific) or authorial trace.

we assume that the separability of texts by country can be attributed to the only distinguishing linguistic variable left: the dimension of the native language of a speaker.


These drawbacks of ISO-TimeML, especially the lack of compositionality, make the development of machine learning models difficult.

most prior work has taken a rule-based approach, looking up each token of a time expression in a normalization lexicon and then mapping this sequence of lexical entries to the normalized form (Strötgen and Gertz, 2013;Bethard, 2013;Lee et al., 2014;Strötgen and Gertz, 2015).


It is very unlikely that any piece of text will be annotated with more than one non-operator or with more than one operator, 2 though it is common for text to be anno-   tated with one non-operator and one operator (see Figure 1).

we can use two softmaxes, one for non-operators and one for operators, and the 2-Softmax model thus can produce 0, 1, or 2 labels per input.


In this case, the guidelines establish that a YEAR can have a SUB-INTERVAL link to a SEASON-OF-YEAR, a MONTH-OF-YEAR or WEEK-OF-YEAR.

the algorithm creates a SUB-INTERVAL link between 1992 and 12.


These time expressions are quite particular; they occur in isolation and not within the context of a sentence and they always yield a bounded interval.

their identification is a critical factor in an interval based evaluation metric.


Then, we remove the rest of the cases from the test set.

we also remove the predictions given by the systems, both ours and Heidel-Time, for those instances.


Increasing s means that, on average, the magnitude of f (x i ) − f (x j ) increases so that Φ(z) is closer to 0 or 1, and erroneous pairwise labels are less likely.

larger values of s correspond to less observation noise.


This approach is related to the 'numerically-grounded' language model (Spithourakis et al., 2016), except that we scale the politeness label embedding by its corresponding politeness score, rather than concatenating the two as input to the LSTM.

3 the LFT model is able to simultaneously produce polite, neutral and rude responses depending on the prepended label, similar to recent multilabel, multi-space, and zero-shot machine translation work using language identity or style labels (Sennrich et al., 2016a;Johnson et al., 2017;Ghosh et al., 2017).


Data Annotated data is crucial for developing high accuracy for rare citation classes.

we integrate portions of the dataset of Teufel (2010), 6 which has fine-grained citation function labeled for ACL-related documents using the annotation scheme of Teufel et al.


Here the citing sentence appears much like a BACKGROUND citation when read in isolation; however, the preceding sentence reveals that the citing work's data is based on the citation, making its function USES though no explicit cues suggest this in the citing sentence.

our error analysis supports the observation of Abu-Jbara et al.


When c t is aligned to a target word y t (we assume that the aligns are always correct and align error problem is beyond the focus of this work), we expect that a certain portion of c t and the embedding of y t are semantically equivalent (that is how the information of the source side is transformed to the target side).

there should be always a certain relation among attention vectors, which are aligned to the same target word.


While at first PW seems comprehensive and quite large already (covering 40 languages), the majority of the PW languages are similar from both a genealogical perspective (26/40 are Indo-European) and a geographic perspective (28/40 Western European).

they share many patterns and are not a representative sample of the world's languages.


Yet, having a good estimate for the parameters in M w is essential for the final LM performance, as they are directly responsible for the next-word prediction.

our aim is to improve the quality of representations in M w , focusing on infrequent words.


The <unk> vector is part of the vocabulary at training time, but remains untrained and at its random initialization  since it never occurs in the training data.

we sample a random <unk> vector at test time from the same part of the space as the trained vectors, using a normal distribution with the mean and the variance of M w and the same fixed random seed for all models.


The third EP test language, Finnish, has a slightly higher type/token ratio.

we do observe an improvement of 10 points in perplexity.


Empirically we find that it leads to similar results in testing our theory (Section 4) and better results in downstream WSI applications (Section 6).

sIF embeddings are adopted in our experiments.


estimates for v w , but with high probability about f 1 /(f 1 + f 2 ) fraction of the occurrences used sense s 1 and f 2 /(f 1 + f 2 ) used sense s 2 , and the corresponding estimates for those two subpopulations converge to v s 1 and v s 2 respectively.

by construction, the estimate for v w is a linear combination of those for v s 1 and v s 2 .


By (2) the probability of being output by a discourse is determined by the inner product, so one expects that the vector for "clothing" discourse has a high inner product with all of shoe, jacket, tie1, etc., and thus can stand as surrogate for v tie1 in (1)!

it may be sufficient to consider the following global optimization: Given word vectors {v w } in ℜ d and two integers k, m with k < m, find a set of unit vectors where at most k of the coefficients α w,1 , .


In preliminary experiments with the Yelp data set, we found that the generative classifier accuracy of our model is highly correlated with discriminative classfier accuracy (r ≈ 0.95).

by this measure, we anticipate that the model would be useful for generation applications.


While powerful, LSTM language models usually do not explicitly model many commonly-accepted linguistic phenomena.

standard models lack linguistically informed inductive biases, potentially limiting their accuracy, particularly in lowdata scenarios (Adams et al., 2017;Koehn and Knowles, 2017).


2: We then sum the local hidden and cell states: This formulation is powerful, but comes at the cost of sacrificing the probabilistic interpretation of which paths are likely.

even if almost all of the probability mass comes through the "true" segmentation, the hidden state may still be heavily influenced by all of the "bad" segmentations as well.


We can calculate the total probability that we reach some prefix x <j , and we know how much of this probability comes from each of its predecessors in the lattice, so we can construct a probability distribution over predecessors in the lattice: .

8one way to update the LSTM is to sample one predecessor x <i from the distribution M and simply set h j =h i and c j =c i .


We use this composition function on chunks regardless of whether they are assigned noncompositional embeddings or not, as even highfrequency chunks may display compositional properties.

for every chunk, we compute the chunk embedding vector x j i by concatenating the compositional embedding with the non-compositional embedding if it exists, or otherwise with an <UNK> embedding.


However, this corpus is limited by being relatively small, only containing approximately 45,000 sentences, which we found to be insufficient to effectively train lattice language models.

5 we instead used the Billion Word Corpus (Chelba et al., 2014).


Finally, P letter infers that digits of circles are 2, and in addition she infers from circle that the digit on the square is not a 2 as otherwise, P digit would have clicked on it (pragmatics).


Our recurrences bottom out in very simple, literal semantics, (e.g., context-independent meaning of circle), and we rely on the structure of recurrences to endow words with their rich contextdependent meaning.

our model is very parsimonious and only has four (hyper)parameters.


In order to understand the principles behind how humans perform planning, inference, and pragmatics, we aim to develop a parsimonious, interpretable model with few parameters rather than a highly expressive, data-driven model.

following the tradition of Rational Speech Acts (RSA) (Frank and Goodman, 2012; Goodman and Frank, 2016), we will define in this section a mapping from each word to its literal semantics, and rely on the PIP re- currences (which we will describe in Section 4) to provide context-dependence.


For notational simplicity, let us define s −1 (s +1 ) to be a matrix corresponding to the spatial locations of the objects, where an entry is 1 if the corresponding object has the goal letter (digit) and 0 otherwise.

s j also represents the set of goal-consistent objects given the private knowledge of that player.


Expressions of percentage changes are readily available in many natural language text datasets and can be easily extracted.

computers should be able to learn from such expressions how people decide which verbs to use for what kind of percentage changes.


Without loss of generality, in the subsequent discussion, we focus on selecting the verbs of one particular direction; the way to deal with the other direction is exactly the same.

a percentage change is fully specified by its magnitude in one model.


The central hypothesis of ADAN is that an ideal model for CLSC should learn features that both perform well on sentiment classification for the SOURCE, and are invariant with respect to the shift in language.

as shown in Figure 1, ADAN has a joint feature extractor F that aims to learn features that aid prediction of the sentiment classifier P, and hamper the language discriminator Q, whose goal is to identify whether an input text is from SOURCE or TARGET.


Note first that in all of our experiments, traditional features like bag of words cannot be directly used because SOURCE and TARGET have completely different vocabularies.

unless otherwise specified, BWEs are used as the input representation for all systems to map words from both SOURCE and TARGET into the same feature space.


In particular, the English and Chinese reviews are not translations of each other, and in fact may even come from different domains.

the separation could potentially come from two sources: the content divergence between the English and Chinese reviews, and the language divergence of how words are used in the two languages.


This impressive result vindicates the power of adversarial training to reduce the distance between two complex distributions without any direct supervision, which is also observed in other recent works for different tasks (Zhang et al., 2017;Lample et al., 2018  With the introduction of BWEs (columns 2 and 3), the performance of ADAN is further boosted.

it seems the quality of the BWEs plays an important role in CLSC.


The positive effects of data statements that we anticipate (and negative effects we haven't anticipated) cannot be demonstrated and tested a priori, however, as their impact emerges through practice.

we look to value sensitive design, which encourages us to consider what would happen if a proposed technology were to come into widespread use, over longer periods of time, with attention to a wide range of stakeholders, potential benefits, and harms (Friedman et al., 2006(Friedman et al., , 2017.


After a good deal of frustration and investigation into Big U's system, the developers discover that the third-party language ID component they had included was trained on only highly edited US and UK English text.

it tends to misclassify tweets in regional or nonstandard varieties of English as "not English" and therefore not relevant.


The word-level models, however, did benefit from tokenization, but Elephant did not give us an advantage over the Moses tokenizer.

for (a) Standard naming $1 REF @1 $1 male "n.02" @1 $1 Name @1 "tom" $2 REF @2 $2 EQU @2 "now" $2 time "n.08" @2 $0 NOT $3 $3 REF @3 $3 Time @3 @2 $3 Experiencer @3 @1 $3 afraid "a.01" @3 $3 Stimulus @3 @4 $3 REF @4 $3 entity "n.01" @4   Figure 1.


The automatic evaluation metrics provide overall scores but do not capture how the models perform on certain semantic phenomena present in the DRSs.

we manually inspected the test set output of the three parsers for the semantic phenomena listed in Table 2.


For comparison, we show the results of the WSD+SMT system versus the SMT baseline in the lower part of Table 6: The improvement is smaller, at +1.4% for EN/FR and +1.5% for EN/ES.

the ATT ini NMT model brings higher benefits over the NMT baseline than the WSD+SMT factored model, although the NMT baseline is stronger than the SMT one (see Table 5).


UD provides training examples actually drawn from that distribution D over treebanks-but alas, rather few.

to better estimate the expected performance of Θ under D, we follow Wang and Eisner (2017) and augment our training data with GD's synthetic treebanks.


Formally, loss(x, y; u) is given by where a ranges over the arcs of a tree y, and 1 a / ∈y is an indicator that is 1 if a / ∈ y.

this loss function is high if there exists a tree y that has a high score relative to y yet low precision.


We suspect that this is because we tuned the hyperparameters to maximize UAS, not LAS ( §9.2).

while the average benefit across our 15 test languages was fairly large, this sample was not large enough to establish that it was significantly greater from 0, that is, that future test languages would also see an improvement from data augmentation.


And the gold conclusion for this pair is "neutral" (i.e., the hypothesis is possibly true).

matching should be conducted across phrase granularities.


For simplicity, we use the same architecture for the attention source (just introduced) and for the attention focus, t y (i.e., for the attention focus: f mgran (H y ) = H y mgran ; see Figure 4(b)).

the focus of attention will participate in the matching process as well as be reweighted to form an attentive context vector.


Furthermore, annotators often disagree on what constitutes a mistake, and some gold errors missed by a system may be considered as acceptable usage by another rater.

when a system is compared against the gold truth produced by just one annotator, performance is understated.


The main challenge here is identifying the subject of the target verb.

errors on verbs that are located far from the subject head are typically not handled well in both Russian and English; in the Russian corpus, these account for 20% of all missed errors.


In addition, AMRs directly capture entity relations and abstract away inflections and function words.

they can serve as a source of knowledge for machine translation that is orthogonal to the textual input.


At each recurrent step, each node advances its current state by receiving information from the current states of its adjacent nodes.

with increasing numbers of recurrent steps, each word receives information from a larger context.


That is, starting with an initial configuration c 0 = c s (x), we find y = c 0 , ..., c n such that c i+1 = t i+1 (c i ) and c n ∈ C t .

each y depicts a sequence of decisions that constructs a valid analysis for x at the relevant linguistic level.


a few studies report human evaluation on their challenge sets, such as in MT (Isabelle et al., 2017;Burchardt et al., 2017).


The accuracy metric is somewhat misleading, since the fact that the tagger predicts correctly that a token does not have an attribute is considered a correct answer.

if only 5% of tokens bore a specific morphological attribute, a 95% accuracy is a most frequent baseline score.


Compared with gate mechanism, nonlinear interpolation is much simpler and needs less parameters.

we will use nonlinear interpolation with tanh and λ = 0.1 for all experiments thereafter.


We find that Transformer produces translations with good prefixes (red line or dotted line), whereas Transformer (R2L) generates translations with better suffixes (blue line or ::::: wave ::: line).

they are often unable to translate the whole sentence precisely.


In this particular example, LSTM falls into the wellknown trap of repeating information close to the end, whereas RUM avoids it.

rUM can be seen as a more ''well-rounded'' alternative to LSTM.


The novelty of our model is that it exploits the simple and fundamental multiplicative closure of rotations to generate rotational associative memory for RNNs.

an RNN that uses our RUM units yields state-of-the-art performance for synthetic associative recall while using very few parameters.


This is based on the observation that just like the sum of two real numbers is also a real number, the product of two rotational matrices is another rotational matrix.

3 we use a rotation R t as an additional memory state that accumulates phase as follows We make the associative memory from Equation (2) tunable through the parameter λ ∈ {0, 1}, which serves to switch the associative memory off and on.


We found that LSTM and GRU fail for all learning rates, EURNN is unstable for large learning rates, and RUM is stable for all learning rates.

we use 0.001 for all units except for EURNN, for which we use 0.0001.


8 As the full text for research papers is typically only available in PDF format (sometimes also in HTML and/or XML), it is generally hard to convert to text format.

we focus on publications by just a few well-known publishers, which cover a sizable proportion of the research papers discussed in Science Daily, and for which we developed parsers: American Association for the Advancement of Science (AAAS), Elsevier, Public Library of Science (PLOS), Proceedings of the National Academy of Sciences (PNAS), Springer, and Wiley.


Because g depends on the input b j , the summands F (W, b j ) are generally out of phase.

l will not be close to periodic, which reduces the risk of falling into local minima.


Then let τ be de- where True is any unary predicate that is always true for models in S. In this transduction, ϕ P b (x) is only true for input positions labeled b that do not follow another b; conversely, ϕ P c (x) is only true for input positions labeled b that do follow another b.

for example, the output under τ for the string model for abba from Figure 2 is a model for abca as shown in Figure 3.


Similarly, ϕ P a (x) is defined to mirror P a .

the only change is made to bs following other bs, which are written out as cs.


First, lcp(f (H∅ n Σ * )) = ∅ n−1 , because f (H∅ n λ) = ∅ n−1 H∅ but f (H∅ n ∅) = ∅ n H∅; the longest prefix these outputs share is ∅ n−1 .


A (x, y) Informally, the formula in (7) states that positions x and y are associated in the output model if x is associated to the predecessor of y in the input model, or if x is associated to y in the input and y is final (because, as with the string representation, we assume that a final H cannot shift).

the bounded shift pattern in Rimi, which was ISL given a string representation, is also A-ISL.


12A (x, y) Informally, (12) says that positions x and y are associated in the output if either (1) they are associated in the input, or (2) the predecessor of x is associated to y in the input.This allows a H to spread to exactly one following TBU (if it exists).


Although this applies to a sequence of Hs of any length, it is in fact ISL, because each H vowel only needs to ''look'' one vowel to its left in the input.

the map can be defined with the following QF formulas.


Intuitively, this distinction cannot be made only by referring to the predecessor function on the tonal tier: in each of (18), (18b), and (18c), the first H is the predecessor of the second.


We can utilize a generic segmentation data set derived from Wikipedia that includes headings (Koshorek et al., 2018), but there is also a need in IR and QA for supervised structural topic labels (Agarwal and Yu, 2009;MacAvaney et al., 2018), different languages and more specific domains, such as clinical or biomedical research (Tepper et al., 2012;Tsatsaronis et al., 2012), and news-based TDT (Kumaran and Allan, 2004;Leetaru and Schrodt, 2013).

we introduce WIKISECTION, 2 a large novel data set of 38k articles from the English and German Wikipedia labeled with 242k sections, original headings, and normalized topic labels for up to 30 topics from two domains: diseases and cities.


Because we do not know the expected number of sections, we formulate the objective of our model on the sentence level and later segment based on the predictions.

we assign each sentence s k a sentence topic labelȳ k = topic(s k ), where k = 1, .


For example, a section with heading "Treatment | Gene Therapy" might describe genetics as a subtopic of treatment.

we also approach an extended task WIKISECTION-headings to capture ambiguity in a heading, We follow the CBOW approach (Mikolov et al., 2013) and assign all words in the heading z j ⊂ Z as multi-label bag over the original heading vocabulary.


(2016), word order is not critical for document-centric evaluation settings such as our WIKISECTION task.

we mainly focus on unsupervised compositional sentence representations.


We are now interested in the vector space movement of this embedding over the sequence of sentences.

we apply a number of transformations adapted from Laplacian-of-Gaussian edge detection on images (Ziou and Tabbone, 1998) to obtain the magnitude of embedding deviation (emd) per sentence.


In this case, the problem expands to few-shot learning (Li et al., 2006).

weights are hard to optimize using gradient-based techniques (Ravi and Larochelle, 2016).


One main characteristic of the categorical information here is that both user and product can be cold-start entities (Amplayo et al., 2018a).


Multiple clusters of vectors in the vector space are clearly seen when the epoch is 4.

using the basis vectors makes optimization more efficient by following the learning strategy of starting from one cluster and dividing into smaller coherent clusters.


Considering the abovementioned challenges, however, we hypothesize that incorporating general world knowledge and aspects of the dialogue structure would allow a better understanding of the dialogues.

we modify our baseline systems to include (1) general world knowledge in the form of such as ConceptNet relations (Speer et al., 2017) and a pre-trained language model (Radford et al., 2018), and (2) speaker information for each utterance.


Besides, questions from examinations are generally clean and high-quality.

human performance ceiling on this kind of data set is much higher (e.g., 94.5% on RACE [Lai et al., 2017] and 98.6% on DREAM in accuracy) than that of data sets built by crowdsourcing.


Investigating the value of the baseline for 10,000 instances shows that in 37%  of the cases the baseline is 0 (i.e., none of the sampled parses leads to the correct answer).

37% of the time, wrong parses are ignored rather than discouraged.


With this change, the MRT objective always behaves in a bipolar manner, irrespective of the baseline's value.

mRT NEG can significantly outperform mRT by 2.33 points in F1 score (see Table 3).


(2018), we assume that an interpretable latent discourse role should cluster messages labeled with the same dialogue act.

we adopt purity (Manning et al., 2008), homogeneity (Rosenberg and Hirschberg, 2007), and variation of informa tion (VI) (Meila, 2003;Goldwater and Griffiths, 2007) as our automatic evaluation metrics.


Between one configuration and the next, the stack differs at most in the two rightmost elements and the remaining input differs at most in that it loses its leftmost element.

all but a constant number of values of score(i, j, q) and score (i, j) can be reused, to make the time complexity closer to constant time for each calculation of the optimal step.


Interestingly, although deeper GCNs with more layers will be able to capture richer neighborhood information of a graph, empirically it has been observed that the best performance is achieved with a 2-layer model .

recent efforts that leverage recurrencebased graph neural networks have been explored as the alternatives to encode the structural information of graphs.


In our model, we also add forward and backward sequential connections, as illustrated in Figure 5.

the edge type vocabulary for the extended Levi graph of the dependency tree becomes T = {default, reverse, self, global, forward, backward}.


For the AMR graph we have four types of edges while for dependency trees we have six as mentioned in Section 3.2.

considering different type of edges, we modify the convolution computation as: where dir (u, v) selects the weight matrix and bias term associated with the edge type t. For example, in the AMR generation task, there are four edge types: default, reverse, self, and global.


(2017) show that using a mean-based function to aggregate feature information from different nodes may not be satisfactory, since information from different sources should not be treated equally.

we assign different weights to information from different types of edges to integrate such information.


Our study evaluates 36 diverse languages, using collections of paradigms represented orthographically.

we are measuring the complexity of each written language.


This is the expected number of bits needed to represent a paradigm drawn from p, under a code that is optimized for this purpose.

it may be related to the cost of learning paradigms or the cost of storing them in memory, and thus relevant to functional pressures that prevent languages from growing too complex.


We train the parameters θ on all non-syncretic slot pairs in the training set.

a paradigm with n distinct forms contributes n 2 training examples: Each form in the paradigm is predicted from each of the n − 1 other forms, and from the empty form.


However, H(M ) reflects not only the language's morphological complexity, but also its 8 Where the weight of the tree is taken to include the weight of the special edge empty → σ to the root node σ.

for each slot σ, the weight of empty → σ is the cost of selecting σ as the root.


It will also store that extended path's choice of underlying punctemes, in the form of a punctemeannotated version of the subtree of T that is rooted at w. This is a potential subtree of T 0 .

each element of IN(w) has the form (r, D) where r 2 R and D is a tree.


In practice, the D strings that appear within the matrix M of Algorithm 1 will always represent complete punctuated trees.

they can actually be represented in memory as such, and different trees may share subtrees for efficiency (using pointers).


In a real-world setting, we cannot expect the same exact user utterances to come during deployment that the model came across during its training.

for a new dialog, it is not possible to know beforehand whether model (M ) would provide a correct response or not.


The classifier (C) has to learn this through trial and error and generalize.


The reward function helps achieve two of our aforementioned goals: • Maximize task success rate for the user: The reward function encourages the classifier to learn the dialog scenarios in which the model M might fail and choose a human agent instead.

the classifier helps to avoid sending an incorrect response to the user from the model.


Unfortunately, in our composition experiments the BiLinear model fares generally worse than the FullLex model.

a strong argument can be made in favor of a model that learns information about specific words or groups of words.


Simple heuristics are used to filter questions from the query stream.

the questions are ''natural'' in that they represent real queries from people seeking information.


Each item in our data consists of a four-tuple (q, d, l, s) where q is a question, d is a document, l is a long answer, and s is a short answer.

we introduce random variables Q, D, L, and S corresponding to these items.


However, doing so would not allow us to estimate model performance in a real scenario, where text length will rarely follow the distribution seen in training sets.


The only exception are experiments related to S1, for which we could only access the original data set.

we compare our results with respect to those published in De Clercq and Hoste (2016).


Figure 1: Adversarial evaluation in NLP typically focuses on a specific phenomenon (e.g., word replacements) and then generates the corresponding examples (top).

adversarial examples are limited to the diversity of what the underlying generative model or perturbation rule can produce-and also require downstream human evaluation to ensure validity.


Although we believe these cases need to be handled by any NFH resolution system, we do not think systems should be evaluated on them.

we provide these cases as a separate data set.


In our data set, 18.9% of debaters who have engaged in more than 10 debates participate in 66.9% of the debates.

in order to validate that our model does not simply recognize the most active users, we test on a subset of the 2016 data where we cap the number of debates a user can participate in to 10 (142 debates).


in the word encoder yields a set of attention matrices A = {A (k) } r k=1 for each sentence where A (k) ∈ R n×n .

when measuring the contributions from words to sentences (e.g., in terms of domain scores and representations) we can selectively focus on salient words within the set A = {A (k) } r k=1 : where the softmax function outputs the salience distribution over words: and obtain sentence embeddings g = Zα.


By carefully examining the training procedure, we observe that the softmax layer, which predicts a distribution of the target word, often induces significant overhead, especially when the vocabulary size is large.

we revisit the design of the output layer and consider directly predicting the pre-trained embedding of the target word for a given context.


(1) scale linearly to the vocabulary size, V .

when V is large, the softmax layer becomes the speed bottleneck.


Our approach exhibits small GPU memory footprints, due to reductions of the arithmetic complexity (with fewer intermediate results to keep) and trainable parameter size (with fewer parameters to store).

training with continuous outputs is 2 to 4 times more memory-efficient than with the softmax layer (see Section 5.2).


ELMo uses a character-CNN but it is relatively slow.

we use a pre-trained open-vocabulary word embedding as the input layer of the contextual encoder, reducing the time complexity of the input layer to a negligible level.


To optimize the inner product between c and w p(w|c)w, we essentially align the direction of context vector c with the expected word vector under context c, w p(w|c)w = E w∼p(w|c) w. In other words, given a word embedding matrix W ∈ R V ×d , our approach aligns c with the corresponding row (AW ) c,: in AW .

the objective can be viewed as conducting multivariate regression to approximate (AW ) c,: given the context.


In the spirit of PCA (Jolliffe, 2011), to keep the variance of A, we would like to have (AW ) c,: and (AW ) c ′ ,: distant from each other if c and c ′ are very different.

a pre-trained word embedding, which projects words with different meanings into different positions in space, is a natural choice for the projection matrix W and can help preserve much of the variance of A.


Our main goal is to study how different output layers affect the training speed and performance, which cannot be achieved by just comparing ELMO-C and ELMO, due to the aforementioned minor changes to ELMO-C.

we introduce two additional baseline models (ELMO-A and ELMO-Sub), which differ from ELMO-C in a minimal way.


Specifically, their sequence encoders and training recipes are kept the same as ELMO-C.

eLMO-C, eLMO-A, and eLMO-Sub can be directly compared.


ELMO-Sub takes subwords as input and also predicts subwords.

unlike other models, its vocabulary consists of around 30,000 subwords created using BPE (Sennrich et al., 2016).


The relative-position representation can always be mapped back to the absolute position z t i by: One of the biggest advantages for using such vector-based representations is that at each step, updating the relative-position representations is simply extending the relative-position matrix R t with the next predicted relative position, because the (left, middle, right) relations described in  Equation 3stay unchanged once they are created.

we update R t as follows: (5) where we use r t+1 t+1 to represent the relative position at step t + 1.


The major difference is that they directly use absolute positions, whereas ours utilizes relative positions.

their model needs to re-encode the partial sequence at every step, which is computationally more expensive.


There has also been significant empirical evidence supporting the argument that humans' semantic inferences are uncertain and context-sensitive (Poesio and Artstein, 2005;Versley, 2008;Simons et al., 2010;Recasens et al., 2011;de Marneffe et al., 2012;Passonneau et al., 2012;Pavlick and Callison-Burch, 2016a,b;, among others) suggesting computational models would benefit from focusing on ''speaker meaning'' over ''sentence meaning'' when it comes to NLI Westera and Boleda, 2019).

in this paper, we assume that NLP will maintain this hands-off approach to NLI, avoiding definitions of what inferences humans should make or which types of knowledge they should invoke.


In the DNC, all sub-corpora consist of binary labels (ENTAILMENT/NON-ENTAILMENT) but some subcorpora contain finer-grained labels than others (e.g., three-way or five-way labels).

when sampling, we first stratify across sub-corpora 3 and then across the most fine-grained label type available for the given sub-corpus.


One confound that results from collecting annotations on a continuous scale is that each rater may choose to use the scale differently.

we apply z-score normalization to each worker's labels for each assignment, meaning each worker's ratings are rescaled such that the mean across all labels from a single worker within a single batch is 0 and the standard deviation is 1.


If the mixture model frequently choses to use more than one effective component, and if doing so results in a better fit for the held-out data than the unimodal Gaussian, we interpret this as evidence that, for many sentence pairs, human judgments exhibit reproducibly multimodal distributions.

for such sentence pairs, the current practice of aggregating human judgments into a single label would fail to accurately capture the types of semantic inferences that humans might make about the given p/h pair.


NLI is standardly treated as a classification task.

in order to interface with existing NLI models, we discretize 10 our collected human judgments by mapping the raw (unnormalized) score (which is between −50 and 50) into K evenly sized bins, where K is equal to the number of classes that were used in the original dataset from which the p/h pair was drawn.


We have shown that models that are trained to predict an aggregate score do not, by default, model the same type of uncertainty as that which is captured by distributions over many human raters' judgments.

several challenges would need to be overcome to switch to the proposed NLI evaluation.


Such a model should have a set of classes of interaction event classes and a set of classes of entities that participate in events.

the ontology of the domain should exist.


However, since the building of an ultimate ontology is, in a sense, the goal of science, the explicit ontology exists only in a very restricted and partial form.

iE and Ontology building are inevitably intertwined here.


SLUNG exploits the property of HPSG that allows under-speci ed constraints.

in order to obtain widecoverage from the very beginning of grammar development, we only give loose constraints to individual words that may over-generate wrong interpretations but nonetheless guarantee correct ones to be always generated.


The same idea can be used in IE based on sentential parser.

techniques that can extract information from partial parse results will make the system robust.


In case of the approach based on sentential parsing, we treat the ambiguity problem by preference.

an interpretation that indicates relevant pieces of information exist is preferred to other interpretations.


Annotated corpora constitute not only an integral part of a linguistic investigation but also an essential part of the design methodology for an NLP systems.

the design of IE systems requires clear understanding of information formats of the domain, i.e.


The classi cation scheme, in turn, re ects the view of the elds that biochemists have.

semantic tags we use are the class names in an ontology of the eld.


Our representation of times uses the ISO standard CC:YY:MM:DD:HH:XX:SS, with an optional time zone (ISO-8601 1997).

time points are represented in terms of a calendric coordinate system, rather than a real number line.


In addition to features based on words cooccurring with "today" (Said, Will, Even, Most, and Some features below), some other features (DOW and CCYY) were added based on a granularity hypothesis.

it seems possible that "today" meaning the day of the utterance sets a scale of events at a day or a small number of days.


This means that one can fall back on another less accurate but more general method for pairs which did not occur in the training data.

if we randomly assign an order to unseen pairs, we can cut the error rate in half and raise the overall accuracy to 78.28%.


Shaw and Hatzivassiloglou (1999) propose to generalize the direct evidence method so that it can apply to unseen pairs of adjectives by computing the transitive closure of the ordering relation ≺.

if a ≺ c and c ≺ b, we can conclude that a ≺ b.


The information gain of a feature then is simply the difference between the total entropy of the instance base and the entropy of a single feature: The information gain G(D, f ) is the reduction in uncertainty in D we expect to achieve by learning the value of the feature f .

knowing the value of a feature with a higher G gets us closer on average to knowing the class of an instance than knowing the value of a feature with a lower G does.


It would be worthwhile to investigate applying some of the more sophisticated ensemble learning techniques which have been proposed in the literature (Dietterich, 1997).

boosting (Schapire, 1999;Abney et al., 1999) offers the possibility of achieving high accuracy from a collection of classifiers which individually perform quite poorly.


After that, we have carried out a blind evaluation on unrestricted texts.

sUPAR has been run on two different spanish corpora: a) a part of the spanish version of The Blue Book corpus, which contains the handbook of the International Telecommunications Union CCITT, published in English, French and spanish, and automatically tagged by the Xerox tagger, and b) a part of the Lexesp corpus, which contains spanish texts from different genres and authors.


In terms of quality, however, only a small proportion of multi-word clusters were found to contain infix derivations (11.11%, 4.76%, 0.0% 4.35% and 9.09% for each data set respectively), as opposed to other variations.

strings sharing character sequences in middle position cluster together more successfully.


However, since metrical trees are relational (i.e., one node is stronger than the others), it makes no sense to construct a metrical tree if there is only a single daughter.

if the head's COMPS list is empty, then the argument mkMtr is a singleton list containing only the head's PHON value, and this is returned unaltered as the function value.


Zwicky's constraints on when this combination can occur are as follows: (15) A personal pronoun NP can form a prosodic phrase with a preceding prosodic host only if the following conditions are satisfied: a. the prosodic host and the pronominal NP are sisters; b. the prosodic host is a lexical category; c. the prosodic host is a category that governs case marking.

these considerations motivate a third clause to the definition of mkMtr: if the first two elements of the list are a Prosodic Word and a Leaner, then the two of them combine to form a lnr-mtr, followed by any other material on the input list.


Fig 6 gives a typical example of this phenomenon, where the noun possession is followed by a prepositional complement, while Fig 7 represents  The special prosodic status of lexical heads is incorporated in Selkirk's (1981) notion of ϕ-phrase, and subsequent developments thereof, such as (Selkirk, 1986;Nespor and Vogel, 1986  Turning back briefly to the Head-Specifier Rule shown in Fig 5, we can now see that if ϕ 0 is a metrical tree M, then the value of extMtr(ϕ 1 ϕ 0 ) depends on the syntactic information associated with the leftmost element P of that tree.

if P is the phonology of the lexical head of the phrase, then it can be prosodically disjoined from the following material, otherwise the metrical tree M is extended in the standard way.


By contrast, auxiliaries do not conform to the association between headed structures and prosodic structures that we have seen so far.

if auxiliaries are a subtype of complement-taking verbs, as assumed within HPSG, then they depart from the usual pattern in behaving prosodically like specifiers rather than heads.


We view each answer in a FAQ as a summary of the document relative to the question which preceded it.

an FAQ with question/answer pairs comes equipped with different queries and summaries: the answer to the th question is a summary of the document relative to the th question.


Summaries of documents relative to a query can often mislead a reader into overestimating the relevance of an unrelated document.

very long documents are likely (by sheer luck) to contain some portion which appears related to the query.


This, however, will enlarge the node size even further.

tomabechi's and Wroblewski's algorithms are not suited for concurrent unification.


We will adopt the same terminology as Tomabechi in (Tomabechi, 1992).

we will use the term feature-structure sharing when two arcs in one graph converge to the same node in that graph (also refered to as reentrancy) and data-structure sharing when arcs from two different graphs converge to the same node.


Because ϕ is normal, Y has a mother X via a tree edge, and X is on C as well.

x must dominate Y but is properly dominated by Y in any solution of ϕ, so ϕ is unsatisfiable.


That is, X must dominate Y but is properly dominated by Y in any solution of ϕ, so ϕ is unsatisfiable.

if an undirected constraint graph has a simple hypernormal cycle C with only one dominance edge, C is also a directed cycle, so the constraint is unsatisfiable.


The work discussed here, like Tetreault's, is an attempt at using corpora to compare different versions of centering theory, but considering also parameters of centering theory not studied in this earlier work.

we looked at different ways of defining the notion of utterance, we studied the definition of realization, and more generally the role of semantic information.


F urthermore, by at least one measure, these software systems evaluate student essays with the same degree of accuracy as human experts.

computer-generated scores tend to match h uman expert scores as frequently as two h uman scores match each other Burstein et al., 1998.


We focus on Centering Theory's Rough-Shift transition which is the least well studied among the four transition types.

we examine whether the discourse coherence found in an essay, as de ned by a measure of relative proportion of Rough-Shift transitions, might be a signi cant contributor to the accuracy of computer-generated essay scores.


The processing load in the Rough-Shift cases reported here is not increased by the e ort required to resolve anaphoric reference but instead by the e ort required to nd the relevant topic connections in a discourse bombarded with a rapid succession of multiple entities.

rough-Shifts are the result of absent or extremely short-lived Cbs.


In cases where stack-based approaches find a coreferent (although not the most recent antecedent) elsewhere in the stack, it makes sense to compare the effort required by the two models to establish correct co-referential links.

we assume that from a computational perspective (and, presumably a psycholinguistic one as well), the closer an antecedent is to the referential expression to be resolved, the better.


(4) Pronouns Following (Gundel, et al., 1993), we consider that the evoking power of each of these types of REs decreases as we move down the list.

pragmatic references are easily understood without an antecedent; proper nouns and noun phrases less so, and are typically resolved by inference over the context.


If we compute the vein expressions for both representations, we see that the vein for segment C6 in the intentional representation is <A6 B6 C6>, whereas in the rhetorical representation, the vein is <(B6), C6>.

under the constraints imposed by VT, John is not available as a referent for he in C6 in the rhetorical version, although J o h n is clearly the appropriate antecedent.


For many machine translation systems, experienced experts on individual systems compile the bilingual dictionary, because this is a complicated and difficult task.

this task is knowledge-intensive and labor-intensive, and therefore, time-consuming.


1 Our proposal In this paper, we concentrate on lexical transfer, i.e., target word selection.

the mapping of structures between source and target expressions is not dealt with here.


Thus if we consider each derivation step to correspond to a semantic dependency, then derivational generative capacity describes what other elements a semantic element may depend on.

if we i n terpret the derivation trees of TAG as dependency structures and the derived trees as phrase structures, then the derivational generative capacity of TAG limits the possible dependency structures that can be assigned to a given phrase structure.


Removing a site-segment must not deprive a tree of its foot node.

no site-segment h h ; l i may contain a foot node unless l is itself the foot node.


If two tree sets adjoin into the same tree, the two site-segments must be simultaneously removable.

the two sitesegments must be disjoint, or one must contain the other.


But none of these cases really requires special treatment in a PTAG model, because each composition probability involves not only a bilexical dependency but a biarboreal" tree-tree dependency.

pTAG generates an entire elementary tree at once, conditioned on the entire elementary tree being modi ed.


In the syntactic parsing model which uses rulebased grammar, the probability of a parsing tree can be defined as the probability of the derivation which generates the current parsing tree for the given sentence.

denotes a derivation rule sequence, h i denotes the partial parsing tree derived from In order to accurately estimate the parameters, we need to select some feature types , depending on which we can divide the contextual condition According to the equation of (2) and (3), we have the following equation: In this way, we can get a unite expression of probabilistic evaluation model for statistical syntactic parsing.


The distributions calculated were simply the empirical distributions from the training data.

occurrences of each role and each set of conditioning events were counted in a table, and probabilities calculated by dividing the counts for each role by the total number of observations for each conditioning event.


However, few systems apply other NLP techniques.

only four systems model some coreference relations between entities in the query and documents Morton, 1999Breck et al., 1999Oard et al., 1999Humphreys et al., 1999.


Statistical language modelling (SLM) has been successfully applied to many domains such as speech recognition (Jelinek, 1990), information retrieval (Miller et al., 1999), and spoken language understanding (Zue, 1995).

n-gram language model (LM) has been demonstrated to be highly effective for these domains.


But the disadvantage is the temporal term phenomenon.

some bigrams are used frequently during one time period, and then never used again.


My thanks go to my students for the realization of Vietools and my colleagues for their opinions.

i thank Professor Aravind Joshi, University of Pennsylvania, Philadelphia, USA, for his helpful suggestions i am grateful to Christian Boitet, Professor, Joseph Fourier University, GETA, CLiPS, iMAG, France, for his comments on this paper.


I will discuss what we have obtained, and the methods that are being used to analyse biological sequences.

i will discuss approaches using stochastic grammars analogous to those used in computational linguistics, both for gene finding and protein family classification.


CORETEX (http://coretex.itc.it/), aims at improving core speech recognition technologies, which are central to most applications involving voice technology.

the project addresses the development of generic speech recognition technology and methods to rapidly port technology to new domains and languages with limited supervision, and to produce enriched symbolic speech transcriptions.


Orthogonal to this is the question of sensitivity to segment order.

should our match mechanism treat each string as an unorganised multiset of terms (the bag-of-words approach), or attempt to find the match that best preserves the original segment order in the input (the segment order-sensitive approach)?


As a third orthogonal parameter, we consider the effects of segment contiguity.

do matches over contiguous segments provide closer overall translation correspondence than matches over displaced segments?


We thus provide clear evidence that high-performance TR is achievable with naive methods, and moreso that such methods outperform more intricate, expensive methods.

the dumber the retrieval mechanism, the better.


A striking feature of the graph is that it is rightdecreasing, which is essentially an artifact of the inflated length of each TRec (see Section 4.1) and resultant data sparseness.

for smaller datasets, in the bulk of cases, no TRec in the TM is similar enough to the input to warrant consideration as a translation candidate (i.e.


To combine the findings for accuracy and speed, VSM under character-based indexing suggests itself as the pick of the different system configurations, combining both speed and consistent accuracy.

it offers the best overall retrieval performance.


Essentially, the probability of a given segment set permuting in different string contexts diminishes as the number of co-occurring segments decreases.

for a given string pair, the greater the segment overlap between them (relative to the overall string lengths), the lower the probability that those segments are going to occur in different orderings.


In this paper, we present a study of the effects of data size on machine learning for natural language disambiguation.

we study the problem of selection among confusable words, using orders of magnitude more training data than has ever been applied to this problem.


The theoretical study of the range concatenation grammar [RCG] formalism has revealed many attractive properties which may be used in NLP.

range concatenation languages [RCL] can be parsed in polynomial time and many classical grammatical formalisms can be translated into equivalent RCGs without increasing their worst-case parsing time complexity.


An oracle always indicates all the good ways that will eventually lead to success, and those good ways only, while a guide will indicate all the good ways but may also indicate some wrong ways.

an oracle is a perfect guide (Kay, 2000), and the worst guide indicates all possible ways.


However, an occurrence of the terminal y can be instantiated to the range .

in a clause, several occurrences of the same terminal may well be instantiated to different ranges while several occurrences of the same variable can only be instantiated to the same range.


However, even if this cubic time transformation is not performed, we can show that the (theoretical) throughput of the parser for % cannot be less than the throughput of the parser for .

if we consider the parsers for and % and if we recall the end of Section 2, it is easy to show that the degrees, say and g , of their polynomial parse times are such that t ¼ .


Another potential problem may come from the size of the guiding grammar itself.

experiments with regular approximation of CFLs related in (Nederhof, 2000) show that most reported methods are not practical for large CF grammars, because of the high costs of obtaining the minimal DFSA.


5 In Section 3, we have seen that the language defined by a guiding grammar is a CFL (see (Boullier, 2000a)).

in the case of TAGs, our transformation algorithm approximates the initial tree-adjoining language by a CFL, and the steps of CF parsing performed by the guiding parser can well be understood in terms of TAG parsing.


The corpus we use for identification of paraphrases is a collection of multiple English translations from a foreign source text.

we use literary texts written by foreign authors.


The remaining variable, f , can be determined solely from the derivation's presupposition set using the old AI planning heuristic "use existing objects" (Sacerdoti, 1977) to avoid inventing new objects when others are already available.

we can unify alts(f, λx.browser(x) ∧ ¬f (x)) and alts(netscape, λx.browser(x) ∧ ¬f (x)), discovering that f , the figure, is netscape.


This technique suffers from the fact that, in order to be tractable, this set of patterns is limited and important information in the query can be lost.

the Electric Knowledge search engine does not have patterns that attempt to associate alternative phrases with appropriate pieces of boolean query.


The precision for all three is around 20%.

only about one in five of the responses contain an answer to the query.


It is important to note that the scores for the queries without alternative phrases are still computed with respect to the alternative phrase.

documents only about the "Raven" are considered false positives.


Copies of all other variables and literals can be computed in a completely analogous fashion.

copying gives us another redex starting at A D B , and we can continue with the algorithm usb in Figure 6.


The data were annotated with a highly limited set of labels.

the kind of system question and whether the reply of the user gave rise to communication problems or not.


The evaluations also provide evidence that this method for automated thesis selection in essays is generalizable.

once trained on a few human annotated prompts, it can be applied to other prompts given a similar population of writers, in this case, writers at the college freshman level.


Some of the complexity in the RICHES system is there to demonstrate the potential for different types of control strategies.

we do not make use of the possibilities offered by the interleaving of the RE and LC, as the examples we cover are too simple.


Some eye movements occur primarily at the ends of utterances and at grammatical boundaries, and appear to function as synchronization signals.

one may request a response from a listener by looking at the listener, and suppress the listener's response by looking away.


A better explanation for gaze behavior integrates turn taking with the information structure of the propositional content of an utterance [5].

the beginning of themes are frequently accompanied by a look-away from the hearer, and the beginning of rhemes are frequently accompanied by a look-toward the hearer.


This data is from the Penn Wall Street Journal tree-bank [13], but modified to make the text more "speech-like".

: As in previous work, files F0 to F20 are used for training, F21-F22 for development, and F23-F24 for testing.


Now we define strong generative capacity as the relationship between strings and structural descriptions.

1 The strong generative capacity of a grammar G a CFRS F is the relation For example, the strong generative capacity of the grammar of Figure 3 is  in a CFG the n bs and cs must appear later in the derivation than the m as and ds, whereas in our example they appear in parallel.


A little more work is needed to prove that op l is closed on Σ.

with respect to clause 2 of the above definition, it is necessary to prove that op l (a 1 , a 2 ) = ⊥ or for all labels l , |hole l (op l (a 1 , a 2 ))| ≤ 1, but it is straightforward to see this is the case.


Note that the approximation need not incorporate all of the features from the original grammar in order to provide a sound approximation.

in order to derive a finite CF grammar, we will need to consider only those features that have a finite number of possible values, or at least consider only finitely many of the possible values for infinitely valued features.


The LP tree must satisfy: Thus, whether an edge w− − → w is licensed depends both on valency LP (w) and on field ext (w ).

: w must offer field and w must accept it.


In this paper, mainly from an NLP point of view, we explore a method to produce linguistic resources.

we enhance the method proposed by Fujii and Ishikawa (2000), which extracts encyclopedic knowledge (i.e., term descriptions) from the Web.


Motivated by this background, we introduce an organization model to Fujii's method and reformalize the whole framework.

our proposed method is not only extraction but generation of encyclopedic knowledge.


In addition, our system selectively uses term descriptions categorized into domains related to information technology.

the description of "pipeline (transportation pipe)" is irrelevant or misleading to answer questions associated with "pipeline (processing method)."


Although the first group includes the largest number of pages, other groups are also related to a relatively large number of pages.

our method exploited a number of low ranking pages, which are not browsed or utilized by most Web users.


In addition, we analyzed the result on a term-byterm basis, because reading only a couple of descriptions is not crucial.

we evaluated each term (not description), and in the case where at least one correct description categorized into the correct domain was generated for a term in question, we judged it correct.


The ratio of correct terms was 89.4% (76/85), and in the case where we focused solely on the computer domain, the ratio was 84.8% (67/79).

by reading a couple of descriptions (3.8 descriptions per term), human users can obtain knowledge of approximately 90% of input terms.


That is obtained using the Viterbi alignment provided by a translation model as described in (Brown et al., 1993).

we use the Viterbi alignment that was produced by Model 5.


This ensures strong constraint propagation and thereby reduces nondeterminism.

it avoids the combinatorial explosion that would result from first generating the possible conjunctions of lexical descriptions out of the CNF obtained by lexical lookup and second, testing their satisfiability.


Instead of selecting lexical entries on the basis Ú NP: Figure 4: Example grammar of their whole semantics, we select them on the basis of their index.

a lexical entry is selected iff its head unifies with a proposition in the input semantics.


(Kay, 1996)) is to index edges with semantic indices (for instance, the edge with category N/x:dog(x) will be indexed with x) and to restrict edge combination to these edges which have compatible indices.

an active edge with category A(...)/C(c ...) (with c the semantics index of the missing component) is restricted to combine with inactive edges with category C(c ...), and vice versa.


By specifying global properties of the output and letting constraint propagation fill out the consequences of a choice, situations in which no suitable trees can be built can be detected early.

the global constraint stating that the semantics of a solution tree must be identical with the goal semantics rules out the generation of the phrases in (1b).


It would be interesting to see whether other axiomatic constraint-based treatments of grammar could be use to support both parsing and generation.

we intend to investigate whether the dependency grammar presented in (Duchier, 1999), once equipped with a semantics, could be used not only for parsing but also for generating.


Syntactic structures are related to semantic structures, whereas topological phrase structures are related to phonological structures.

our work lies within the scope of the general framework of Meaning-Text-Theory (Mel'cuk 1988), which considers the modeling of a language as a modular (bi-directional) correspondence between meaning and text.


On the contrary, all verbs placed in the same domain put their dependents in a common pot.

there is no scrambling in German, or more precisely, there is no advantage in assuming an operation that derives 'scrambled' sentences from 'nonscrambled' ones.


7 Within the phrasal type system of (Ginzburg and Sag, 2000) root-cl constitutes the 'start' symbol of the grammar.

phrases of this type have as their content an illocutionary operator embedding the appropriate semantic Before we can explain how these representations can feature in dialogue reasoning and the resolution of CE, we need to sketch briefly the approach to dialogue ellipsis that we assume.


First of all, we must explain how ACGs may manipulate strings of symbols.

we must show how to encode strings as linear λterms.


The other connectives of linear logic are natural candidates for extending the formalism.


The exponentials of linear logic are modal operators that may be used to go beyond linearity.


In this example, the italicized VP could be elided, since it has a nearby antecedent (in bold) with the same meaning.

indeed the antecedents in this example is closer than in the following example in which ellipsis does occur: (3) Mr Coxon says businesses are paying out a smaller percentage of their profits and cash flow in the form of dividends than they have VPE historically.


We created a second corpus by extending SECTIONS5+6 with positive examples from other sections of the PTB so that the number of positive examples equals that of the negative examples.

we included all positive examples from Section 8 through 13.


Goldman said Eastern would miss the same mark by at least $120 million.

(6) Mr Coxon says businesses are paying out a smaller percentage of their profits and cash flow in the form of dividends than they have VPE historically.


It is widely recognized that the proliferation of annotation schemes runs counter to the need to re-use language resources, and that standards for linguistic annotation are becoming increasingly mandatory.

there is a need for a general framework for linguistic annotation that is flexible and extensible enough to accommodate different annotation types and different theoretical and practical approaches, while at the same time enabling their representation in a pivot format that can serve as the basis for comparative evaluation of parser output, such as PARSEVAL (Harrison, et al., 1991), as well as the development of reusable editing and processing tools.


By considering these facts, we modify the above rule generation.

we replace every word in an NE and its character type by '*' to get the left-hand side of the corresponding recognition rule except the following cases.


For NOTRANSFORM, no other modification was made.

empty nodes (represented as -NONE-in the treebank) were turned into rules that generated the empty string ( ), and there was no collapsing of categories (such as PRT and ADVP) as is often done in parsing work (Collins, 1997, etc.).


Note that the model, though not perfect, exhibits the qualitative differences between the settings, both in magnitudes and exponents.

18 : The transform primarily changes the saturation over short spans, while the encoding determines the overall magnitudes.


In human-human interaction, dialogue partners apply sophisticated strategies to detect and correct communication failures so that errors of recognition and understanding rarely lead to a complete breakdown of the interaction (Clark and Wilkes-Gibbs, 1986).

various studies have shown that prosody is an important cue in avoiding such breakdown, e.g.


In previous work, we identified new procedures to detect recognition errors.

we found that prosodic features, in combination with other information already available to the recognizer, can distinguish user turns that are misrecognized by the system far better than traditional methods used in ASR rejection .


In the current study, we focus on another turn category that is potentially useful in error handling.

we examine what we term aware sites -turns where a user, while interacting with a machine, first becomes aware that the system has misrecognized a previous user turn.


Subsequent machine learning experiments distinguishing aware sites from corrections and from normal turns show that aware sites can be classified as such automatically, with a considerable degree of accuracy.

in a 2way classification of aware sites vs. all other turns we achieve an estimated success rate of 87.8%.


For each term t j , compute its df (i) (t j ) and cv (i)  , where df (i)  is defined as the fraction of documents in T i that contain t j .

df (i) (t j ) gives the conditional probability of t j appearing in T i .


In topic spotting, the goal is to achieve both high recall and precision.

we want to allow y(x) to be as large (or as small) as possible in cases when there is no error, or when After the training, we choose the node with the biggest w i value as the common attribute node.


In a view where feature structures represent discretely ordered objects in an empirical model, every feature structure must bear one of these species.

each non-maximally-specific type in a description is equivalent to the disjunction of the maximally specific subtypes that it subsumes.


The non-tailored letter was essentially the letter produced by STOP from a blank questionnaire, with some manual post-editing and tidying up.

during the course of developing STOP we created a set of default rules for handling incomplete or inconsistent questionnaires; the non-tailored letter was produced by activating these default rules without any smoker data.


if a smoker receives a letter from his/her doctor about smoking, then the content of the letter is only of secondary importance, the important thing is the fact of having received a communication from his/her doctor encouraging smoking cessation.


The STOP letters did in fact have an effect on some groups (such as heavy or discouraged smokers), but the clinical trial was too small to provide statistically significant evidence of this.

did we fail because (1) what we were attempting could not work; (2) what we were attempting could only work if we had a lot more knowledge available to us; or (3) we built a poor system?


In such applications, the only way to test hypotheses about the effects of systems on human users may be to run a controlled task/effectiveness evaluation.

there's probably no point in conducting a large-scale task/effectiveness evaluation of an NLG system if you're interested in formulating hypotheses instead of testing them, or if you're interested in theoretical instead of applied hypotheses.


Generally speaking, the number of segments obtained by our algorithm is not sensitive to the length of a given text, which is counted in words.

the number of segments is relatively stable with respect to variation in the text length.


In (Blumofe and Leiserson, 1993), it is shown that for any multi-threaded computation with work T 1 and task graph depth T ∞ , and for any number P of processors, a scheduling will achieve T P ≤ T 1 /P + T ∞ if for the scheduling holds that whenever there are more than P tasks ready, all P threads are executing work.

as long as there is work on any queue, no thread should be idle.


Furthermore, we intend to explore methods for deriving additional metrics from dialogue act tags.

it is possible that sequential or structural metrics based on particular sequences or configurations of dialogue acts might capture differences in dialogue strategies.


When we decide which word to insert, no conditioning variable is used.

a function word like ga is just as likely to be inserted in one place as any other.


In addition, we add a margin condition into the "hinge loss".

we seek a linear weight is a given parameter called the regularization parameter.


This difference in performance may be attributed to the fact that the Pre-dictionOnly model is a "smoothed" version of the Mixed model.

the PredictionOnly model uses a consistent version of the target variables (i.e., predicted values) both for training and testing.


16 Our novel weights fall in a novel 14 Formal derivation of (1): xi, yi | π)P (π); now observe that P (xi, yi | π) = 1 or 0 according to whether π ∈ Π.

15 : (K, ⊗) is a monoid (i.e., ⊗ : K × K → K is associative) with identity 1.