Given a mention m, we use several heuristic rules to generate candidate entities similar to (Bunescu and Pasca, 2006;Huang et al., 2014;Sun et al., 2015).

given a mention m, we retrieve an entity as a candidate from KB, if it matches one of the following conditions: (a) the entity title exactly matches the mention, (b) the anchor text of the entity exactly matches the mention, (c) the title of the entity's redirected page exactly matches the mention Additionally, we add a special candidate NIL for each mention, which refers to a new entity out of KB.


We aim to find the final ranking by aggregating multiple rankings.

given a ranking of entities for one semantic matching signal, e 1 e 2 e 3 .


In this section, we describe our experimental results on tweet entity linking.

we investigate the difference between two semantic matching models and the effectiveness of jointly combining these two semantic matching signals.


The goal of retrieval-based dialogue systems is then to learn a predictive distribution p(y|c, r, θ) parameterized by θ.

given a context c and re-sponse r, we would like to infer the probability of r being a response to context c. Recurrent neural networks are one of the most popular classes of models for processing sequences of words W = {w t } T t=1 with arbitrary length T ∈ N, e.g.


We observed that our model's predictions are biased toward high information utterances.

we observed for some examples that the correct response is generic (i.e.


we compute a "global" mention-entity score based on which we produce the final annotations.


The final loss function is now slightly modified.

we enforce the linear separability in two places: in Ψ(e, m) (exactly as before), but also in Φ(e, m), as follows The inference procedure remains unchanged in this case, with the exception that it will only use the Φ(e, m) global score.


[3 and [4 cases illustrate that the gold standard can be problematic.

instead of annotating the whole span Korean War and linking it to the war of 1950, the gold annotation only include Korean and link it to the general entity of Korea_(country).


if o is the vector representation of a given object then we can think of the dot product v f • o as the value of object o for feature f , and in particular, we take to mean that o 2 has the feature f to a greater extent than o 1 .


One problem with accuracy as a scoring function is that these classification problems are often very imbalanced.

for very rare words, a high accuracy might not necessarily imply that the corresponding direction is accurate.


The main idea is to use the BoW representations of the objects as a kind of weak supervision signal: if an object should be ranked highly for a given feature, we would expect the words describing that feature to appear frequently in its description.

for each feature f we determine a total ordering f such that o f o iff the feature f is more prominent in the BoW represention of object o than in the BoW representation of o.


To this end, we represent each object as a bag of clusters (BoC) and then compute PPMI scores over this representation.

for a cluster C = {w 1 , ..., w m }, we define n(C, o) = m i=1 n(w i , o).


In particular, for a cluster C = {w 1 , ..., w m }, we define n(C, o) = m i=1 n(w i , o).

n(C, o) is the total number of occurrences of words from cluster C in BoW representation of o.


To evaluate our method, we consider the problem of learning interpretable classifiers.

we learn decision trees which are limited to depth 1 and 3, which use the rankings induced by the feature directions as input.


Our method is based on the observation that there is a tradeoff between accurately modelling similarity in a vector space, and faithfully modelling features as directions.

we introduced a postprocessing step, modifying the initial semantic space, which allows us to find higher-quality directions.


(2018) suggests to use pre-defined facts about the conversation agents as the context for the dialogue.

conditioning on those facts (called "memories" in their approach), the dialogue becomes "personalized", purposefully coherent and is perceived being more engaging.


In this paper, we investigate a different approach which leverages the following intuition: an engaging dialogue between two agents is a conversation that is focused and intends to discover information with the goal of increased understanding of each other.

discovering implies asking engaging and inquisitive questions that are not meant to be answered with dull responses.


A successful interview implies that the HR representative was able to get as much information about the applicant as possi-ble within a limited amount of time, while dull and repetitive questions are avoided at all cost.

the amount of gathered information can be seen as a proxy measure to the engagingness of the dialogue.


(2016) analyzed how people ask questions to elicit information about the world within a Battleship game (Gureckis and Markant, 2009).

they proposed to evaluate questions based on Expected Information Gain (Oaksford and Chater, 1994), which is built on the similar principles as DISCOVERYSCORE.


The second type of model addresses the aspect of modeling the human.

among the 3 and "I like apples"), the response would be unlikely.


We can expect a similar behavior from a real human, which might reply with "I don't know" or "I don't understand", when the question is irrelevant to them.

we want to avoid simulating dialogues like this -Chatbot: "Hmm...


To improve the cross-domain parsing performance, we propose a data-oriented model to explore the linguistic generality encoded in a hand-crafted, domainindependent, linguistically-precise English grammar, namely English Resource Grammar (ERG;Flickinger, 2000).

we introduce a cost-sensitive training model to learn crossdomain semantic information implicitly encoded in WikiWoods (Flickinger et al., 2010), i.e., a corpus that collects the wikipedia 1 texts as well as their automatic syntactico-semantic annotations produced by ERG.


Previous work showed that the Maximum Subgraph framework is not only elegant in theory but also effective in practice (Kuhlmann and Jonsson, 2015;Cao et al., 2017a,b).

1EC/P2 graphs are an appropriate graph class for modeling semantic dependency structures (Cao et al., 2017a).


We use words as well as POS tags as clues for scoring an individual arc.

we transform all of them into continuous and dense vectors.


In our first order model, the SCORE function evaluates the preference of a semantic dependency graph by considering every bilexical relation in this graph one by one.

the corresponding SCOREPART function assigns a score to a candidate arc between word i and word j using a non-linear transform from the two feature vectors, viz.


When processing an imperfect sentence, we do not take a loss into consideration if the loss of this sentence is too small.

if a loss of a bilexical relation between two tokens is less than 0.05, we would exclude the loss.


We use the "standard" training, validation, and test splits to facilitate comparisons.

the data splitting policy follows the shared task.


For the neural tagging model, in addition to a BiL-STM layer for encoding words, we set a BiLSTM layer for encoding characters, which supports us to derive character-level representations for all words.

vectors from the characterlevel LSTM is concatenated with the pre-trained word embedding before feeding into the other word-level BiLSTM network to capture contextual information.


Unlike the traditional dropout method where neurons are dropped randomly with the same probability, we encode global information into dropout.

we drop words based on their importance which are calculated from training data via a novel Naïve Bayes (NB) weighting technique.


To gain more insight into what information LSTMs base their decisions on, we propose a generalisation of Contextual Decomposition (GCD).

this setup enables us to accurately distil which part of a prediction stems from semantic heuristics, which part truly emanates from syntactic cues and which part arise from the model biases themselves instead.


We demonstrate the power of GCD through the revelation of some important asymmetries in the way that both the singular-plural and the malefemale distinction are handled.

we find evidence for a default reasoning effect, which we believe could also be important for future work on detecting and removing bias: a default category (singular, masculine) appears to be hard-coded in the weights of the language model, number and gender information in the word embeddings themselves mainly plays a role for phrases of the opposite category (plural, feminine).


's results reveal interesting patterns in sentences used in a sentiment classification task, their work illustrates some limitations as well.

the work deals with a classification task with few classes, aggregates relevance per word for each predicted class, but offers little insight in how word meanings interact to build up sentence meaning beyond 'pushing in the right direction' vs. 'pushing in the wrong direction'.


Pruning information In the second type of experiment, we focus on the model's predictions.

we study how the model's predictions change when it is forced to consider only specific parts of the input, by disregarding all information that does not belong to the inside information of that part of the input.


In this section, we therefore address two different questions: 1) Does the model correctly resolve referents?

: When the model generates a male or female pronoun, does it consistently do this based on male and female referents encountered earlier in the sentence, and 2) If the model correctly performs anaphora resolution, what types of interactions and information does it use to do so?


Our method assumes that the local topology among nearest neighbors will be consistent between two NLP tasks (here, language modeling and the target task).

synonyms will have a similar role across NLP tasks.


We show how the proposed BTSR task highlights the merits of different alignment methods.

we find that using context average type-level alignment is effective in transferring monolingual contextualized embeddings cross-lingually especially in non-parallel contexts, and at the same time improves the monolingual space.


In this paper, we introduce a first study that investigates and compares different ways of aligning the pretrained contextualized embeddings.

we make the comparisons focused on the following properties: (1) aligning contextual-ized embeddings at the level of word tokens versus word types; (2) different training signals: static dictionaries, word alignment, or sentence alignment from parallel data; and (3) aligning different model variants: aligning from independently trained models versus aligning embeddings from a multilingual model with shared vocabulary.


Warm-starting each of the branches in the dual-encoder leads to substantial improvements Image to Caption Model R@1 R@5 R@10 R@50 R@100 R@1 R@5 R@10 R@50 R@100  over the baseline, and combining both branches leads to the best overall performance.

we improve R@10 for caption-toimage from the .296 obtained by Chrupała (2019) by 20% absolute to .495, without using multitask training or pretraining g vis on ImageNet (Deng et al., 2009).


As discussed in Section 2.3, under the adaptationas-priming paradigm, we would expect sentences that share the same specific structure to be more similar to each other than lexically matched sentences that do not share the structure.

9 if X 1 and X 2 are non-lexically-overlapping sets of sentences with shared structure S X , and Y 2 is a set of sentences with structure S Y , but is lexically matched with X 2 , then we would expect AE(X 2 | X 1 ) > AE(Y 2 | X 1 ).


If the RNN LMs were keeping track of whether or not a sentence contained a gap, we would expect sentences with different types of RCs to be more similar to each other in the RNN LMs' representation space than lexically matched sentences without a gap.

if RC X and RC Y are two different types of RCs and Coord Y is a sentence with verb coordination lexically matched with RC Y , then we would ex- Consistent with this prediction, the adaptation effect for models adapted to RCs was greater when they were tested on sentences with other types of RCs than when they were tested on sentences with coordination (bottom panel of Figure 2b).


When the adaptation and test sets matched only in passivity, the adaptation effect was slightly (but significantly) greater than when the adaptation and test sets matched only in reduction (Figure 3).

in the LMs' representation space, (12) is more similar to (13) than it is to (14), suggesting that passivity contributes more towards the similarity between sentences than reduction.


We aim to explore an elusive aspect of mastering the subtle contours of a word's meaning that are shaped by its context.

we investigate patterns of acquisition of English indefinite pronouns by L2 speakers.


This prediction is also supported by our annotated data: In cases where the preferred pronoun is some-, learners infelicitously use any-8.4% of the time, but in cases where the preferred pronoun is any-, learners infelicitously use somealmost 23% of the time.

learners have almost three times as many infelicities of using someinstead of anythan the reverse.


A common use of a pre-trained LM -typically trained on an extremely large corpus -is to predict the likelihood of an 'unseen' sample of text: The higher the score (or the lower the perplexity) a text is assigned, the more probable it is, given the model.

a fluent, wellformed text is likely to be scored higher by an LM than a text containing linguistic anomalies.


Systematically higher scores obtained for learner utterances (Table 6), compared to advanced L2s (Table 7), imply that the mild infelicities of the latter pose a higher challenge to automatic tools.

not only do advanced L2s show fewer errors, but their errors are likely more subtle and more difficult to detect.


We also consider the non-uniform difficulty of IPs across various usage cases, predicted by Hypothesis 2 and shown for non-natives (Section 4.2).

to address this question, we test BERt for infelicitous choices compared to annotators' decisions: for each sentence, we compare the pronoun preferred by the model to the gold annotation.


We select pairs of concepts that are likely to be represented in an image recognition model.

we identify adjectives, nouns, and verbs in the English COCO captions dataset (Chen et al., 2015) that are suitable for testing compositional generalization.


The respective FULL performance for this model is 47.0, indicating that the model performs well whether it has seen examples of the evaluation concept pair at training time or not.

the model achieves better compositional generalization than the captioning models.


In this task, given a pair of character embeddings, we would like to classify the type of their relationship on multiple dimensions.

we consider: fine-grained relations, such as sister/father/friend/enemy; coarse-grained relations, such as familial/social/professional; and relation sentiment, i.e., positive, negative or neural.


Research on the bilingual lexicon has uncovered fascinating interactions between the lexicons of the native language and of the second language in bilingual speakers.

it has been found that the lexicon of the underlying native language affects the organisation of the second language.


2 This implies that, functionally, the bilingual lexicon is an integrated system.

it has been proposed that languages do not simply share graphemes or phonemes, but that the lexicon, monolingual, bilingual or multi-lingual, is a space of distributed word representations where word forms from different languages map onto a common abstract conceptual code (Van Hell and de Groot, 1998).


This in turn can be demonstrated by two expected inequalities: false friends are not closer than real translations but false friends are significantly more similar than (matching) uncorrelated words.

given an L1 word w 1 , and its real L2 translation w 2 and the false friend w 2ff in crosslingual space, we expect the similarity score between the pair (w 1 , w 2 ) not to be lower than the similarity score between the pair (w 1 , w 2ff ).


If confirmed, they give us a fairly detailed view of the structure of the lexicon conceived as a multidimensional, integrated multilingual space.

they inform us on the respective importance of formal and meaning properties of words in this cross-lingual similarity space.


Because our predictions are confirmed, they also confirm that the similarity structure defined by current cross-lingual word embedding models is promising as a view of the structure of the lexicon conceived as a multi-dimensional, integrated multilingual space.

our results inform us on the respective importance of formal and meaning properties of words in this crosslingual similarity space.


To overcome this problem, we propose to derive an n-gram LM from a federated RNN LM model and use that n-gram LM for decoding.

the approximation algorithm is based on SampleApprox , which was recently proposed in Suresh et al.


For , be the weight of the transition of x at state q and d q [x] be the destination state.

for a label x and a state q, ϕ is followed if x / ∈ L[q].


Suppose we are interested in the count of the bi-gram New York.

given a bi-gram LM, SampleApprox generates m sentences and computes it finds all sentences that have the word New, observes how frequently York appears subsequently, and computes the conditional probability.


The input to their method consists of a text description D e of each entity e, but they assume no other prior knowledge.

each word w which occurs sufficiently frequently in the document collection D = {D e | e ∈ E} is considered as a candidate feature.


different names of genres) tend to be used in similar contexts, and can thus be expected to have similar word vectors.

our basic approach for identifying facets consists in clustering the word vectors, from some standard pre-trained word embedding model, corresponding to the features in F .


However, for paradigmatically similar words, such as horror and comedy, this should not be the case.

two words should intuitively end up in the same clusters if they have similar word vectors but dissimilar feature directions.


To avoid finding redundant facets, we identify them in an incremental fashion.

from the clusters obtained by the clustering algorithm, we only select the single most important one, i.e.


To find this subspace, we learn new feature directions c f for each f ∈ X i , which still capture these features but lie in a low-dimensional subspace.

we minimize the following objective: where with r the desired number of dimensions of the subspace.


(2018) used low-depth decision trees to evaluate a method for learning feature directions in vector space embeddings.

if F = {f 1 , ..., f m } is the set of features that were identified, then they represent each entity e using the feature vector (d f 1 • e, ..., d fm • e), with d f the direction modelling feature f as before.


To evaluate the quality of the facets independently of the quality of the feature directions, we also consider classifiers which use as input the facet-specific vector representations e i of the entities.

we train a support vector machine (SVM) for each of the facets, leading to the predictions p 1 , ...p k .


As our final classifier, we estimate a Gaussian model from the positive training examples.

we estimate a univariate Gaussian for each dimension and multiply the corresponding probabilities.


Bottom: showing the 10dimensional representations for the facet X i = { campuses, students, offices, centers, facilities, area, hotels, homes, bridges, hospitals, cities, shops, stations} articles: buildings and organisations.

we retrieved all Wikipedia pages whose semantic type on WikiData corresponds to building or organisation.


We considered the problem of decomposing a vector space embedding into facets, which are characterized by a set of semantically related features and a corresponding subspace of the embedding.

we focused on unsupervised methods, considering both approaches that rely on the vector space itself (i.e.


We hypothesize that our combined (lexical and structural) semantic representation is not only linguistically plausible, but also useful in practice, as the annotations on both levels should be informative predictors for each other.

we expect that knowing what semantic role is signaled by an adposition informs the underlying semantic structure of the sentence, and vice versa.


Thus, the temporal word embeddings are created using data from a large temporal corpus.

we leverage the New York Times (NYT) archive.


One can explain semantic changes with words-significant associated words or terms that correlate to the target word's meaning drift.

letting w be the target word, and t be a time point, we are looking for words that became closer to w in vector space during t. we look at the nearest neighbors of w at times t and t − 1, and denote the set of descriptors by D t : For example, using this method for 'Russia' results in Soviet Union and Soviet for 1989, when the Soviet Union was dissolving, and Ukraine, Kyrgyzstan, and latvia for 1990, when these countries attempted to gain independence from the Union.


In this evaluation, the projected embeddings were used to create the training and test sets and for creating the classifier's features, since this configuration was found to result in the best performance (Section 8.4).

the parameters (optimized using grid search) and the AUC are as follows: Logistic regression produced an AUC of 0.75.


In this paper, we extend the inoculation method-ology, analyzing the ability of models to generalize across different data distributions when addressing a specific inference phenomenon.

we suggest varying both training and test distributions along several linguistic dimensions, like syntactic complexity or lexical diversity.


Contextualized word embeddings have been used across a broad range of NLP tasks, outperforming the previous state-of-the-art models.

several works showed that they capture various types of linguistic knowledge, from syntactic to semantic and discourse relations (e.g.


They suggested to fine-tune the NLI model on the specific challenge dataset in order to find out which case is currently observed.

in the case of a data blind spot, the performance on the challenge dataset is expected to improve after fine-tuning (i.e., the model is "inoculated" against the adversarial data).


We generate the entailed hypothesis templates by applying the inference type.

for entailing dative alternation, we either switch to the alternate constructions (row 5 in Table 1) or remove the first argument after the dative verb (row 6).


The main motivation for our data splits is to control the variance between the train and test sets with respect to certain data dimensions.

for both inference types, we create a variance along the syntactic complexity dimension.


We use a standard model based on BERT (Devlin et al., 2019) as our NLI model.

we used the base-uncased pre-trained model from the pytorch-pretrained-bert library 4 , and fine-tuned it for the NLI task on MultiNLI.


7 Both corpora were crawled from the web, using ParaCrawl (Koehn et al., 2018a).

the task is to produce a score for each sentence pair in these noisy corpora, reflecting the quality of that pair.


The absence of data has over the recent years materialized the proxy fallacy.

methods tailored for low-resource languages are typically tested only by proxy, simulating low-data regimes exclusively on resource-rich languages (Agić et al., 2017).


Since entity mentions are short token sequences and not full sentences, this semi-morphological/semantic task requires a model to rely on the subword information of individual tokens in the absence of sentence context.

subwords can provide evidence useful for entity type classification in the absence of context.


For training embeddings in controlled low-resource settings with our 4 "simulation" languages, we sample nine data points to simulate low-resource scenarios with WE data.

we sample 10K, 20K, 50K, 100K, 200K, 500K, 1M, 2M, and 5M tokens of article text for each of the 4 languages.


For MTAG, we evaluate on the multilingual morphological annotations provided by the Universal Dependencies project (Nivre et al., 2016) and adopt the experimental protocol of Cotterell and Heigold (2017).

we cast MTAG as a sequence labeling task by treating the concatenation of all key-value pairs for a given word as the word's label.


The results on the 12 test languages in Table 3-4 suggest that subword-informed models are better than the baselines in most cases: this validates the initial findings from the simulation experiments.

leveraging subword information is important for WE induction as well as for task-specific training.


For this algorithm (Shi et al., 2017), the score of a transition t does not depend on the whole configuration but only on the index on top of the stack and the first index of the buffer.

local scores s only depend on word indexes (i,j,k).


A score is given to each pair consisting of a shift and its corresponding (left or right) reduction.

we exploit the perfect matching induced by a derivation between shift and reduce transitions: if t is a shift transition and t its matching reduction, we consider a score depending on the equivalence classes of c and c +1 .


The derivation/derived pair is a very powerful concept that remains to be fully exploited.

there are two promising avenues for future improvements.


To alleviate this problem, we propose an NMT approach that heightens the adequacy in machine translation by transferring the semantic knowledge learned from bilingual sentence alignment.

we first design a discriminator that learns to estimate sentence aligning score over translation candidates, and then the learned semantic knowledge is transfered to the NMT model under an adversarial learning framework.


In this paper, we address the problem of inadequate translation by introducing a novel sentence alignment constrain under an adversarial training framework (Goodfellow et al., 2014;Lu et al., 2017;Yang et al., 2018).

our approach contains two sub-models: i) a sentence alignment oriented discriminator D learns to estimate the alignment score and sort the translation candidates by mainly considering the weighted alignment pairs (Ma, 2006) at a sentence representation level; and ii) a standard NMT model G aims to produce an appropriate translation with the highest ranking score (assigned by D) in the candidate list.


2: for number of training iterations do 3: Update G with the learning rate η: Teacher-Forcing: update G on (X, Y + ) (see section 3.5) 8: end for ventional encoder-decoder framework (Cho et al., 2014b).

the encoder contains a stack of six identical layers.


In later sections, we will analyze the semantic information learned by the model through some visualization examples, shown in section 5.1, and the experimental results show that it achieves sufficient accuracy for scoring the alignment between source and target sentences.

in our framework, G aims to generate a translation score higher than the golden-standard, under the premise of encoders and the scoring function learned by D. for each sentence pairs (X, Y + ) in training sets, first, G samples transla-tionŶ given X with greedy searching.


For the encoders and decoders, we follow the recently emerged Transformer (Vaswani et al., 2017).

each encoder is composed of a stack of four identical layers, and each layer consists of a multi-head self-attention sub-layer and a fully connected feed-forward sub-layer.


In the second state, we use an annealing schedule to teach the model to produce stable sequences gradually.

after the initial pre-training steps, we continue training the model with future rewarding.


We then apply a standard dictionary-based technique 4 to create multilingual word embeddings Conneau et al., 2018).

suppose that we have a word-translation dictionary from source language s to target language t be matrices whose columns are the anchors in the jth layer for the source and corresponding target words in the dictionary.


For each layer j, find the linear transformation W ⇤(j) such that The linear transformations are then used to map the LM hidden states for the source language to the target LM space.

contextual representations for the source and target languages are computed by i,c respectively.


(2019), who found that multilingual BERT was useful for zero-shot crosslingual syntactic transfer.

we find nearly no performance benefit from cross-script transfer using BERT in a language pair (English-Japanese) for which they reported 12 Available at https://github.com/googleresearch/bert/ 13 AllenNLP version 0.9.0 was used for these experiments.


, E ln } ∪ {E en }, which are pre-trained on large corpora using any standard word embedding toolkit.

e en represents the pre-trained embedding for english, and e l i represents the pre-trained embedding for non-english language l i at the subword level.


This method, denoted by M M , is also a joint mapping without the need for learning the mapping from E l to E bert .

for each w ∈ V l , we obtain its embedding e(w) in the BERT embedding space E bert as follows.


There is no active competition per se between words in the COHORT model.

the strength of activation of a word does not depend on the value of the activation of the other words, but only on how well the internalised word form matches the perceived spoken input.


Validation and test set comprise 5000 images each.

3 Model is evaluated in term of Recall@k (R@k) and median rank r. given a spoken query, which corresponds to a full utterance, we evaluate the model's ability to rank the unique paired image in the top k images.


8 Then, for the 50 closest matching images, we check if at least one of the caption contains either the target word or the competitor.

as the competitor is embedded in the longer word, we expect the network to produce an embedding close to that of the competitor at the beginning and then when the acoustic signal does 7 Phonemic transcription found in CMU Pronouncing Dictionnary was used 8 we compute the cosine distance between the embedding produced at time step t and all the images (5000) of our collection.


In this work, we propose to leverage unannotated sentences from auxiliary languages to help learning language-agnostic representations.

we explore adversarial training for learning contextual encoders that produce invariant representations across languages to facilitate cross-lingual transfer.


1 We train the encoder of a dependency parser in an adversarial fashion to guide it to avoid capturing language-specific information.

we introduce a language identification task where a classifier predicts the language identity (id) of an input sentence from its encoded representation.


The experimental results moderately correlate with our expectation.

the Germanic family benefits the most from employing German (de) as the auxiliary language; similarly Slavic family with Russian (ru) as the auxiliary language (although German as the auxiliary language brings similar improvements).


In this paper, we study learning language invariant contextual encoders for cross-lingual transfer.

we leverage unlabeled sentences from auxiliary languages and adversarial training to induce language-agnostic encoders to improve the performances of the cross-lingual dependency parsing.


Lee and Dernoncourt (2016) tackled DA classification with a model built upon RNNs and CNNs.

their model can leverage the information of preceding texts, which can effectively help improve the DA classification accuracy.


Such a design is based on the rationale that the shared encoder can transfer parameters between two tasks and reduce the risk of overfitting (Ruder, 2017).

the shared utterance encoder is implemented using the bidirectional gated recurrent unit (Cho et al., 2014, BiGRU), which encodes each utter- of a conversation C n as a series of hidden states .


u t can then be represented as an attention vector l t computed as follows In contrast to the traditional attention mechanism (Bahdanau et al., 2015), which only depends on one set of hidden vectors from the Seq2Seq decoder, the DA attention of our model relies on two sets of hidden vectors, i.e., g t−1 of the conversation-level DA tagger and s t−1 of the conversation-level topic tagger, where dual attention mechanism can capture, for utterances, information about both DAs and topics as well as the interaction between them.

the weights for the DA attention are calculated as follows: The topic attention layer has a similar architecture to the DA attention layer, which takes as input both s t−1 and g t−1 .


As can be seen from the confusion matrices shown in Figure 4, DAH-CRF+LDA utt gives improvement over SAH-CRF for all the four DA classes of DyDA.

directives and Commissive achieve higher improvement margin compared to the other two classes, where the improvement are largely attributed to less number of instances of the directives and Commissive classes being mis-classified into Inform and Questions.


The FASC baseline strengthens ASC by removing word types that are less likely to appear in groundtruth captions, e.g., "ah", "he", "hello," or "wish".

we only keep words with high P (w | GT ) P (w | ASR) values, i.e., words that would be indicative of the groundtruth class if we were to build a Naive-Bayes classifier with addone smoothing; probabilities are computed only over the training set to reduce the risk of overfitting.


We measure per-word-type performance using AUC, which is word-frequency independent.

-for each word type w (e.g., w = beer) we measure how well w is predicted by the classifier based on ASR / spoken tokens AUC t,w (e.g., AUC t,beer = 98) and, conversely, how well w is predicted by the visual classifier AUC v,w (AUC v,beer = 68).


When people in the same environment commonly interact with each other (as in a household or workplace), lexical entrainment will likely occur (Brennan and Clark, 1996).

users of the system will come to refer to objects in similar ways to how they have been referred to in the past.


We run multiple experiments to evaluate the proposed grounding models and characterize the advantages of using coreference along with visual features for grounding.


Although the aggregate performance is low, the coreference models perform strongly on examples where the goal object has past expressions (see 1-column).

coref Supervised with InferSent achieves 70.5% on the STANDARD split, which reduces only to 68.6% on the HARD split.


cooking recipes, how-to guides, scientific processes) are very hard for machines as it demands modeling the intrinsic dynamics of the procedures Yagcioglu et al., 2018).

one must be aware of the entities present in the text, infer relations among them and even anticipate changes in the states of the entities after each action.


For instance, it is quite easy to distinguish raw meat from cooked one in visual domain.

we take advantage of recently proposed RecipeQA dataset (Yagcioglu et al., 2018), a dataset for multimodal comprehension of cooking recipes, and ask whether it is possible to have a model which employs dynamic representations of entities in answering questions that require multimodal understanding of procedures.


A denotes an answer that is either a single image or a series of L images depending on the reasoning task.

for the visual cloze and the visual coherence type questions, the answer contains a single image (L = 1) and for the visual ordering task, it includes a sequence.


On top of these layers, we have another layer that encodes the steps of the recipe in an individual manner.

we obtain a step-level con-textual embedding of the input recipe containing T steps as S = (s 1 , s 2 , .


As we will examine in more detail in Section 4, it also greatly improves the interpretability of model outputs.

we set up the memory with a memory matrix E ∈ R d E ×K by extracting K entities (ingredients) from the first step of the recipe 5 .


3, we illustrate how this interaction takes place in our relational memory module by considering a sample cooking recipe and by presenting how the attention matrix changes throughout the recipe.

the attention matrix at a specific time shows the attention flow from one entity (memory cell) to another along with the attention weights to the corresponding recipe step (offset column).


However, it does not make any updates on the memory cells.

it uses the static entity embeeddings initialized with GloVe word vectors.


We propose an oracle setup that can be considered as an upper bound to what can be achieved with a learned policy.

we use an oracle policy that is allowed to always pick a subset of examples that maximizes its target metric on a development set, which has the same distribution as the test set.


(Zalmout and Habash, 2017)); (b) languages are diverse and different languages require different treatments; and (c) small details may matter a lot.

existing tools and resources, either learned or human curated, should not be trusted blindly, but be carefuly adapted for the problem.


This suggests that the information about the gender is deeply embedded in the representation and is not easy to remove in a post-processing phase.

zeroing the projection of a word's vector on the gender direction is not enough in order to remove all gender information from the word's representation.


In Italian gender change got better results.

changing to feminine got much better results than changing to masculine, probably due to less ambiguity when changing to feminine in some very common articles (see full manual mapping in the Appendix), resulting in fewer cases in which the gender signal leaks through the frequencies of the changed words, as explained above.


Moreover, in this paper, we consider the three tasks independent of each other rather than as a pipeline.

we always take the gold standard from the preceding task instead of the output of the preceding model in the pipeline.


We use our frame detection approach to automatically detect frames of news article headlines related to gun violence during the past few years, which enables large scale analysis of framing trends surrounding this issue in the United States.

we focus on the years 2016, 2017, and 2018 because these three years have witnessed a number of high-profile mass shootings, which often reignited national gun debate.


Further, driven by the attention economy, many online media even use news headlines as "clickbait", presenting sensational but misleading information that deviates from the content included in the actual news story (Chen et al., 2015).

a news story may be framed differently in its headline and the rest of the article.


Long Short Term Memory networks (LSTMs) (Hochreiter and Schmidhuber, 1997) are a variation of RNNs that are designed to avoid the vanishing/exploding gradient problem (Bengio et al., 1994).

biLSTMs take as input a sequence of words x = {x k |k ∈ N } and output a sequence of hidden vectors: H = {h k |k ∈ N } biLSTMs combine a left-to-right (forward) and a right-to-left (backward) LSTM to capture both left and right context.


Our goal is to build a unified model to make global predictions.

our model will be jointly trained on multiple partially annotated datasets C and make predictions on the global tag space T C .


This can be done by modifying the CRF's gold energy function, illustrated in the upper part of Figure 2c.

in this example, John is labeled as PER, so PER is the only possible correct tag at that position.


Note that Equation 7 subsumes all models discussed in this section.

when M = 0, M = 1, the model is the Naive Model discussed in Section 4.1; when M = 1, M = 1, the model is the same as Greenberg et al.


In the first evaluation setting, we take advantage of the fact that although there may not be a single dataset annotated with all types of named entities of interest, there exist several datasets that cover types from more than one of the training corpora.

we are able to select test corpora that each cover types of interest from multiple training corpora.


In the second evaluation setting, we create synthetic datasets from the CoNLL 2003 NER dataset to simulate training and global evaluations.

the CoNLL 2003 dataset is annotated with four entity types: location, person, organization, and miscellaneous entities.


To verify this assumption, we expand the annotations of each dataset using heuristics and compute the pairwise mention-level overlap between the datasets.

suppose we are comparing two datasets, A and B.


-Under the no-supervision setting, we heuristically combine all existing CRF's predictions to make global predictions.

we apply two heuristics to resolve conflicts while preserving entity chunk-level consistency.


We use synthetic data from the dataset to simulate local training and global evaluation.

the CoNLL 2003 NER dataset is annotated with four entity types: location, person, organization, and miscellaneous entities.


A shared embedding look-up is used for all text features ( Figure 1b).

we embed all unigrams and bigrams to get 300-dimensional averaged unigram embeddings and 300-dimensional averaged bigram embeddings for each text feature.


Tables 1 and 2 report performance using brute force nearest-neighbor search.

we score each mention against all 5.7M entities to get the top k neighbors.


(2019) recently presented an extensive set of language-brain encoding experiments.

they evaluated the ability of an ELMo language model to predict brain responses of multiple fMRI datasets.


With a proper modeling of events and event causality above, this section explains the proposed KnowSemLM, a method to inject causality knowledge into a semantic LM.

knowSemLM is based on FES-RNNLM (Frame-Entity-Sentiment infused Recurrent Neural Net Language Model) proposed in .


Finally, we filter cases where the direction of the event causality pairs is unclear from a statistical standpoint.

we calculate the ratio of counts of one direction over another, i.e.


Other works aimed at learning a collection of structured events (Chambers, 2013;Cheung et al., 2013;Balasubramanian et al., 2013;Bamman and Smith, 2014;Nguyen et al., 2015;Inoue et al., 2016).

ferraro and Durme (2016) presented a unified probabilistic model of syntactic and semantic frames while also demonstrating improved coherence.


The weights are computed using a novel neural attention mechanism that enables the model to focus on a small subset of the entities that are less ambiguous in meaning and more relevant to the document.

the attention mechanism is designed to compute weights by jointly addressing entity linking and entity salience detection (Ga-mon et al., 2013;Dunietz and Gillick, 2014) tasks.


Here, we use a simple method based on an entity dictionary that maps an entity name (e.g., "Washington") to a set of possible referent entities (e.g., Washington, D.C. and George Washington).

we first take all words and phrases in a document, treat them as entity names if they exist in the dictionary, and detect all possible referent entities for each detected entity name.


To learn embeddings from the KB, we used the method adopted in the open source Wikipedia2Vec tool (Yamada et al., 2016(Yamada et al., , 2018a.

we generated an entity-annotated corpus from Wikipedia by treating entity links in Wikipedia articles as entity annotations, and trained skip-gram embeddings (Mikolov et al., 2013a,b) of 300 dimensions with negative sampling using the generated corpus as inputs.


Our experimental setup described in this section follows that in past work Jin et al., 2016;Yamada et al., 2018b).

we used the 20NG and R8 datasets to train and test the proposed model.


Furthermore, to investigate the attention mechanism in more detail, we computed the top influential entities in the attention mechanism for each class on the 20NG and R8 datasets.

we calculated the number of times each entity obtained the highest attention weight in the test documents in each class and selected the five most frequent ones.


Overall, our models achieved enhanced performance on this task.

the NABoE-full model successfully outperformed all the baseline models, and the NABoE-entity model achieved competitive performance and outperformed all the baseline models in the literature category.


Moreover, compared with HRL, both the baseline model and our proposed models achieve a significant increase in Recall.

mrmep (layer) achieves 11.5% improvement in NYT, 14.1% improvement in WebNLG, and 14.5% improvement in SKE in Recall value.


Similar to the existing works, we regard ED as a classification problem.

in each sample, there is a candidate trigger, and our goal is to classify this candidate trigger into 34 event types (33 event subtypes and a NA type).


The reason for this is that diagnostic examples were selected for a broader range of topics (not only politics or sports).

document context sentences (DCS) contain 364 unique entities of which only 47 appeared in an exact word form in the training data, and only 42 of them have the same entity type (organization, location or person) -the same type as in the CoNLL 2003 Stanford CMU ELMO FLAIR BERT Figure 4: Radar plot with the strong and weak sides of NER models.


Documents or sentences (later simply documents) are, in the absence of a sequence model, treated as bags of words.

methods for comparing documents are invariant to word order.


Then we report results from an Oracle Weighting scheme (Oracle Weighting) that takes partially annotated data and assigns weights with knowledge of the true labels.

mislabeled entities in set N are given weight 0, and all other tokens are given weight 1.0.


More recently, BERT, a deep bidirectional representation which jointly conditions on both left and right context (Devlin et al., 2019), was proposed, which unlike MUSE, provides contextualized word embeddings, and has been shown to achieve state-of-the-art performance on many NLP tasks.

(Yang et al., 2019) propose a method based on BERT for enhancing event trigger and argument extraction by generating more labeled data.


(2016) to obtain character embeddings using bi-LSTM.

we concatenate both character and word-level features and use a bi-LSTM on top of that.


We observe that in general our languageindependent (monolingual) Bi-LSTM-Char-CRF and BERT-CRF methods are on par with or outperform best attainable results.

bERT-CRF trained monolingually using bERT(base) embeddings outperforms other baselines for both EN and ZH, with F1-scores of 79.2 and 75.3 on trigger identification and classification, respectively, amounting to a 3.3% and 1.6% gain for EN.


Although F1-scores for zero-shot transfer learning(train: EN, test: ZH/AR) are not the best among multilingual experiments, they are still promising and exceed prior published work given the fact that no data from the target language was used to fine-tune.

training with EN using BERT-CRF was helpful for ZH with a performance not far from monolingual performance.


In this paper, we propose a novel deep structured learning model to address the shortcomings of the previous methods.

we adapt the structured support vector machine (SSVM) (Finley and Joachims, 2008) to incorporate linguistic constraints and domain knowledge for making joint predictions on events temporal relations.


We first train the local scoring function without feedback from global constraints.

the local neural network model is optimized using only pair-wise relations in the first stage by minimizing crossentropy loss.


(2017), we augment event pairs with flipped-order pairs.

if a pair (i, j) exists, pair (j, i) is also added to our dataset with the opposite label.


Because of small amount of causal pairs, we simply build an independent final linear layer apart from the original linear layer in Figure 2.

there are two final linear layers: only one of them is active when training temporal or causal pairs.


Then, the representations of the two arguments are concatenated to form the final representation, shown in Figure 2.

we encode each word in an argument into its word embeddings, which are fed into a BiLSTM, to get the hidden representations z i using a fully-connected layer W c on top of the concatenated hidden states We then apply an attention mechanism to induce a distribution of weights over all tokens in the argument; the final argument representation Arg is a weighted sum of z i based on the attention weights α i : Classifier The classifier consists of a single fully-connected layer on top of the encoder, finished with a softmax classification layer.


Each instance in DREAM contains a multi-turn multi-party dialogue, and the correct answer option must be inferred from the dialogue context.

a large portion of questions require multi-sentence inference (84%) and/or commonsense knowledge (34%).


Facing this challenge, we present a response generation model which incorporates Interlocutor-aware Contexts into Recurrent Encoder-Decoder frameworks (ICRED) for RGMPC.

we employ interactive representations to capture dialogue contexts for different interlocutors.


To address the aforementioned problems of RGMPC, this paper incorporates Interlocutoraware Contexts into a Recurrent Encoder-Decoder model (ICRED) for RGMPC, which is also an end-to-end framework.

in order to capture interlocutor information, we exploit interactive interlocutor representations learned from current dialog context rather than the fixed person vectors (Li et al., 2016b) obtained from all dialogs in the training corpus.


Finally, both of the interactive interlocutor representation and addressee memory are utilized for decoding response utterances.

the addressee memory is leveraged to capture the addressee information for each generated word dynamically.


We can see that removing any component causes obvious performance degradation.

"w/o Ctx Adr Vec" performs the worst on almost all of the metrics, which demonstrates the importance of contextual information for the target addressee.


It is clearly observed that our model (ICRED) generates more fluent, consistent and knowledgeable (marked as underline) responses compared to baselines.

the response given by I-CRED "if you want a new kernel , you can install the kernel from the kernel repo", not only explains the reason for kernel installation but also suggests a source of the installation.


In this study, we formalize a novel task of Response Generation for Multi-Party Chatbots (RGMPC) and propose an end-to-end model which incorporates Interlocutor-aware Contexts into Recurrent Encoder-Decoder frameworks (I-CRED) for RGMPC.

we employ interactive speaker models to capture contextual interlocutor information.


QA Modules (Section 2.3, 2.4): An estimated answerâ = QA(m, q) is predicted given a query and MGN graph path output from initial memory slots.

the model outputs a module program {u (k) } for several module networks (e.g.


Inspired by the recently introduced graph traversal networks (Moon et al., 2019) which output discrete graph operations given input contexts, we formulate our MGN as follows.

given a set of initial memory slots (m) and a query (q), the MgN model outputs a sequence path of walk steps (p) within Mg to attend to relevant nodes or expand initial memory slots ( Figure 3): we define the attention-based graph decoder model which prunes unattended paths, which effectively reduce the search space for memory expansion.


It can be seen that the proposed Memory QA model outperforms other QA baselines for precision at all ks.

with the MGN walker model, the MemQA model learns to condition its walk path on query contexts and attend and expand memory nodes, thus outperforming the baseline models that simply rely on their initial memory slots, typically large in size to maintain reasonable recall.


For our first approach, we propose a neural network as shown in Figure 3 to predict discourse relations given a sequence of words.

it operates on a window of words, which we build around a connective that we assume to have been identified by a previous module in the pipeline.


In this paper, we propose to incorporate such topicalinfluence into a language model to both improve its accuracy and enable cross-stream analysis of topical influences.

we propose a novel language model called Topical Influence Language Model (TILM), which is a novel extension of a neural language model to capture the influences on the contents in one text stream by the evolving topics in another related (or possibly same) text stream.


In this paper, we propose to incorporate such topical influence into a language model to both improve its accuracy and enable cross-stream analysis of topical influences.

we propose a novel language model called Topical Influence Language Model (TILM), which is a novel extension of a neural language model to capture the influence on the contents in one text stream by the evolving topics in another related (or possibly same) text stream.


Specifically, we propose a novel language model called Topical Influence Language Model (TILM), which is a novel extension of a neural language model to capture the influence on the contents in one text stream by the evolving topics in another related (or possibly same) text stream.

tILM is a recurrent neural network-based deep learning architecture that incorporates topical influence to model the generation of a dynamically evolving text stream.


While external topical influences have been studied in the context of search behavior modeling (Karmaker Santu et al., 2017, similar study has not been pursued yet for text generation modeling.

current text generation processes are static processes with no notion of time and thus, can not model dynamically evolving text stream data corresponding to evolving influences of related text streams, which the proposed TILM can do.


We used back-propagation to learn the weights of the network connection edges of TILM.

we used Adaptive Moment Estimation, which is a popular stochastic gradient descent technique and commonly known as Adam Optimizer, to compute the gradient for minimizing our objective function in Equation 15.


As community, we considered both the core machine learning community, e.g., NIPS, ICML as well as research communities that apply a fair share of machine learning, e.g, KDD, SIGIR.

we considered all the titles of papers published during the years 1996-2015 by the following six conference venues: NIPS, CVPR, ICML, KDD, SIGIR and WWW.


To model long-term dependencies, an attention module is applied to summarize all previous contexts into a global context vector.

a feed-forward network takes the current context RNN's output c t as the query and all previous context outputs from both context RNNs as keys and values to compute attention scores.


However, even thought LM can better mention the slots in dialogue generation, utterances are often associated with a wrong role.

role confusions are commonly seen such as the user makes reservations for the agent as in Table 2.


The inaccuracy of the question is possibly caused by the use of a loss that is agnostic to sequence information.

given its decomposable nature, the cross-entropy loss on the ground-truth question or any of its (syntactically invalid) anagrams will be the same.


To discourage meaningless multiple repetitions of words in the question (as illustrated in row 3 of Table 1), we maintain a word coverage vector (wcv) for the words already predicted as the sum of all the attention distributions ranging over timesteps 0 until t − 1.


We follow this setting and use the feature vectors to calculate cosine similarity.

the encoder output are tree-like semantic features extracted by CNN encoder while the encoder input embedding provides point information about a specific input element before encoding (Gehring et al., 2017).


By pretraining on massive amounts of unlabeled text (often orders of magnitude larger than the the target task's labeled dataset), these models achieve state-ofthe-art performance across a variety of Natural Language Understanding benchmarks.

the OpenAI GPT2 language model (Radford et al., 2019) achieves state-of-the-art performance on several language modeling benchmarks, even in a zero-shot setting.


We compute the word-level perplexity of the finetuned GPT2-117 on the WritingPrompts-1024 dataset.

we normalize the total negative log probability of the target text by the number of word-level (i.e.


Model confidence over time Several researchers have observed that model overconfidence (the model placing high probability on a small range of tokens) can cause poor quality generation (Jiang and de Rijke, 2018;Holtzman et al., 2019).

they show that for likelihood-maximizing decoding algorithms such as beam search, model confidence can increase in a snowball-like effect, getting stuck in a loop of repetitive but increasingly self-confident text.


The effect of massive pretraining In this study, we find that GPT2-117 is a better story generation model than the Fusion Model in several specific ways: it conditions much more strongly on the provided context, is more sensitive to correct ordering of events, and generates text that is more contentful (using more rare words, concrete words, and named entities).

the stronger conditioning result is striking, as the Fusion Model is a complex task-specific architecture designed to increase story-prompt relevance.


Empirical results show that the proposed approach consistently improves the residual-based models and exhibits desirable generalization ability.

by incorporating the proposed approach to the Transformer model, we establish new state-of-thearts on the IWSLT-2015 EN-VI low-resource machine translation dataset.


In the realm of computer vision, PreAct-ResNet (He et al., 2016b) demonstrated that it is helpful to apply batch normalization to x, instead of F(x, W).

the batch normalization acts on the output of the previous block.


This also demonstrates the generalization ability of our method.

when being applied to the recently proposed Transformer model, our approach establishes new state-of-the-arts on the IWSLT EN-VI low resource machine translation task, which further substantiates its efficiency.


Similar to the case of detection, our models performed substantially above the baseline in this task.

both the MLP and CRF-based models yielded higher F1 scores (close to 0.50) when multiple features are taken into account.


We use the same set of hyperparameters as described in the base papers, except for the batch size with Alleviated TOI (P), where we use a prime batch size in order to prevent any repetitions in batches, as described in Section 4.

on the PTB dataset, we use a sequence length of 70 for all the models.


Given a set of questions Q, MRR is computed as follows: where rank i represents the rank position of the first correct candidate answer for the i th question.

mRR is the average of the reciprocal ranks of results for the questions in set Q. mAP is calculated as follows: where q j ∈ Q is a question whose candidate answers are a 1 , ..., a m j and π jk is the rank associated with those candidate answers.


Our model can predict the next word using the embeddings of the words in the current n-gram window, in addition to the hidden state of the LSTM layer.

we also use a gate to control how much word-level information should be taken before injecting it into the softmax function.


2018, we average our weights at each batch update and assign the ASWA parameters to the model at the end of each epoch.

we replace the model's weights for the next epoch with the averaged weights.


Motivated by previous findings, which showed that direct authorship attribution is not feasible in a large scale scenario, we contribute to this field by proposing a two-step approach in this study.

we propose at first to reduce the number of candidate authors while keeping the correct author in the reduced set with reasonable accuracy.


Using the best models found (depending on the number of candidate authors) we evaluated their performance with respect to the reduction rate.

we reduced the number of candidates by the respective percentage and measured-in terms of hit rate-if the correct author is still in the remaining set.


The variational autoencoder offers the flexibility to customize the model structure.

the proposed framework is compatible with other supervised neural networks to boost their performance.


By regarding the aspect sentiment polarity of the unlabeled data as the discrete latent variable, the model implicitly induces the sentiment polarity via the variational inference.

the representation of the lexical context is extracted by the encoder and the aspect-term sentiment polarity is inferred from the specific ATSA classifier.


For the unlabeled data, it optimizes the ELBO of p(x|a), where the y is latent and integrated.

the dependency between variables is illustrated in Fig.


In previous works, the word embedding is shared among all the components.

the word embedding is also tuned in learning to reconstruct the data.


In this paper, we propose an alternative hard-selection approach, which determines the start and end positions of the opinion snippet, and selects the words between these two positions for sentiment prediction.

we learn deep associations between the sentence and aspect, and the long-term dependencies within the sentence by leveraging the pre-trained BERT model.


The main contributions of our paper are as follows: • We propose a hard-selection approach to address the ABSA task.

our method determines two positions in the sentence to detect the opinion snippet towards a particular aspect, and then uses the framed content for sentiment classification.


In this paper, we propose a hard-selection approach to solve the ABSA task.

we first learn to detect the corresponding opinion snippet O = {w l , w l+1 ..., w r }, where 1 ≤ l ≤ r ≤ N , and then use O to predict the sentiment of the given aspect.


After determining the start position, the "state" is updated and then the end is conditioned on the start.

we define a start vector s ∈ R H and an end vector e ∈ R H .


Firstly, we observe that BERT-Original, BERT-Soft, and BERT-Hard outperform all soft attention baselines (in the first part of Table 2), which demonstrates the effectiveness of fine-tuning the pre-trained model on the aspect-based sentiment classification task.

bERT-Original outperforms AF-LSTM(CONV) by 2.63%∼9.57%, bERT-Soft outperforms AF-LSTM(CONV) by 2.01%∼9.60% and bERT-Hard improves AF-LSTM(CONV) by 3.38%∼11.23% in terms of accuracy.


The improvements are 5.07% and 12.83% for the types 2-3 and More respectively, which demonstrates the ability of our model on handling sentences with More aspects.

bERT-Soft has the poorest performance on the subset Diff among the three methods, which proves that soft attention is more likely to cause attention distraction.


In this paper, we look beyond the traditional population-level sentiment modeling and consider the individuality in a person's expressions by discovering both textual and contextual information.

we construct a hierarchical neural network that leverages valuable information from a person's past expressions, and offer a better understanding of the sentiment from the expresser's perspective.


Motivated by the diversity in individuality, we introduce a model that considers both textual and contextual information and applies hierarchical neural networks to facilitate the aspect of personalization in sentiment analysis.

we focus on analyzing the effect of contextual information and discover ways to embed such information in the prediction process.


In this way, different behaviors concerning the information decay can be analyzed.

✏ and are calculated as where E is the user-index encoder.


This paper presents a personalized sentiment model that captures the individualities in expressing sentiment and analyzes the evolvement of sentiment over time.

we categorize the information used for the modeling into textual and contextual information, and evaluate the effectiveness of using the contextual information to boost the performance of the model.


In this paper, we propose a new model called clustergated convolutional neural network (CGCNN), which jointly explores wordlevel clustering and text classification in an end-to-end manner.

the proposed model firstly uses a bi-directional long short-term memory to learn word representations.


And it can also guide the gating mechanism unit to control clusterdependent feature flows.

it firstly uses a bi-directional long short-term memory model (BiLSTM) to learn word representations and capture local context in text.


We concatenate them as word representation, i.e., ℎ = [ , ] .

the input text is represented as a matrix = [ℎ , ℎ , … , ℎ ] .


With the advent of digitization, advancement in technology, need for evidence-based medicine, increased population, and rising rates of chronic diseases, the utilization of ever-increasing heterogeneous medical data to improve the quality of life has become imperative.

iCUs are data-rich environments where several parameters of patients are monitored continuously.


However, unstructured clinical text and images also contain valuable information concerning the state of a patient.

clinical nursing notes maintain objective and subjective assessments of a patient's condition.


In this paper, we show how to automate this analysis by using machine learning in a realistic setting.

we show how to classify trolls according to their political role-left, news feed, right-by using features extracted from social media, i.e., Twitter, in two scenarios: (i) in a traditional supervised learning scenario, where labels for trolls are available, and (ii) in a distant supervision scenario, where labels for trolls are not available, and we rely on more-commonly-available labels for news outlets mentioned by the trolls.


Our goal in the current paper is to automate this process with the help of machine learning (ML).

we focus on the case of the 2016 US Presidential Elections, for which a public dataset from Twitter is available.


We rely on embeddings extracted from social media.

we use a combination of embeddings built on the user-to-user mention graph, the user-to-hashtag mention graph, and the text of the tweets of the troll accounts.


Such a representation is convenient as it lies in the same space as the user representation.

given a medium m ∈ M , we compute its representation R(m) as where R(u) is the representation of user u, i.e., one (or a concatenation) of the embeddings described in Section 3.1.


In the first case, the set of edges among users U in the similarity graph G consists of the logical OR between the 2-hop closure of the U2H and the U2M graph.

for each two users u, v ∈ U , there is an edge in the similarity graph (u, v) ∈ E if u and v share a common hashtag or a common user mention The graph therefore uses the same information that is available to the embeddings.


The probability of a parse tree is computed from the occurrencefrequencies of the subtrees in the treebank.

the probability of a subtree t is taken as the number of occurrences of t in the training set, I t I, divided by the total number of occurrences of all subtrees t' with the same root label as t. Let r(t) return the root label of t: The probability of a derivation ti0...0t n is computed by the product of the probabilities of its subtrees ti: An important feature of DOP1 is that there may be several derivations that generate the same parse tree.


In case the shortest derivation is not unique, Bod (2000b) proposes to back off to a frequency ordering of the subtrees.

all subtrees of each root label are assigned a rank according to their frequency in the treebank: the most frequent subtree (or subtrees) of each root label gets rank 1, the second most frequent subtree gets rank 2, etc.


During the selection phase, we pick a subset of the newly labelled sentences to add to the training sets of both parsers.

a subset of those sentences labelled by the LTAG parser is added to the training set of the Collins PCFG parser, and vice versa.


A second form of evaluation is to use some data that has been manually or semi-automatically annotated with part of speech (POS) tags, and to use some information theoretic measure to look at the correlation between the 'correct' data and the induced POS tags.

one could look at the conditional entropy of the gold standard tags given the induced tags.


However it is not clear that this directly measures the linguistic plausibility of the classification.

many parts of speech (relative pronouns for example) represent long-distance combinatorial properties, and a simple finite-state model with local context (such as a class n-gram model (Brown et al., 1992)) will not measure this.


Looking at Figures 8 and 9, all of the automatic measures correlate highly with both WH and TOEIC score.

the averaged utterance-level BLEU score shows the highest correlation.


From the MLD-M, 236 terms (uniMeSH 48, un-iMLD 188) were identified.

specialized terms contributed about 9 percent of all words.


Splitting source and target language texts into appropriate units for alignment (henceforth: tokenization) is often not possible without considering the translation relations.

initial tokenization borders may change when the translation relations are investigated.


However, we hope to find additional links with sufficient accuracy from these clues.

we expect clues, which have been found via "self-learning" techniques to increase the recall with an acceptable increase of noise.


Parallelism constraints are very expressive -more expressive than is necessary for modeling ellipsis.

overlapping parallel segments seem useless, but are difficult to resolve.


The type in the type class is computed by unifying: (1) all of the type descriptions in the current block, (2) all of the introducers of the feature descriptions in the block, and (3) the meet (greatest lower bound) in each disjunctive description of the type class of each of its disjuncts.

we are extracting as much type information as we deterministically can from all but one class in the current block and applying it first.


First three measures are for each of the bilingual sentences in the corpus and the fourth measure is for the whole corpus: • Literalness: Few omissions or redundant words appear between the source and target sentences.

most words in the source sentence correspond to some words in the target sentence.


The literalness based on the TCR is judged from a tagged result and a translation dictionary.

'deep analyses' such as parsing are not necessary.


Rules generated from non-literal translations are used only when the input word sequence exactly matches the sequence in the rule.

they are hardly used in different contexts.


On the other hand, the BLEU score and the subjective quality both improved in the case of split construction, even though the coverage of the exact rules decreased.

the subjective quality improved by about 4.9%.


The TCR is capable of measuring literalness not only for bilingual sentences but also for phrases.

a bilingual sentence can be divided into literal phrases and other phrases.


Montague's approach gave rise to much further work aiming at determining the correct rules and representations needed to build a representation of natural language meaning.

computational grammars were developed which by and large took on Montague's proposal, building semantic representations in tandem with syntactic structures.


9 -Raising verbs 5.2 Derivation trees with additional links (Kallmeyer, 2002b;Kallmeyer, 2002a) shows that some of the problems just described can be solved once additional links are added to the derivation tree.

given three nodes n i , n,2 , n,3 such times extremely) complex lambda terms as lexical that n i is above n2 and n3 is above n3 , if n3 is a tree adjoined at the root of n2, then an additional link can be established between ni and n3.


This facilitates some of the steps in our (syntactic) proofs, and makes the derivations more explicit.

the following is taken from (Kapur, 1991 when a class has a limit point it is not learnable because the input to the learner can never provide enough information to justify convergence.


This could be computed just in a single pass, but the Viterbi re-estimation ensures that the data we are computing the log-likelihood-ratio scores from is consistent with the resulting scores.

it ensures that we do not count an instance in the data of a particular translation pair, when there is a higher scoring possibility according to the confidence measure we are computing.


In this paper, we explain how two prominent categorial grammar frameworks, namely Combinatory Categorial Grammar (CCG, Steedman (2000)) and Categorial Type Logic (CTL, Morrill (1994); Moortgat (1997); Oehrle (to appear)), can be integrated to create a hybrid categorial framework, Multi-Modal Combinatory Categorial Grammar (Baldridge, 2002), that inherits the attractive properties of both.

multi-modal CCG retains the favorable computional aspects of CCG whilst incorporating the approach to resource-sensitivity taken in CTL.


For example, Steedman's restricted version of >B x for Dutch can only apply when the secondary functor is a category that will ultimately produce a matrix clause (i.e., s_ SUBS), whereas we wish to avoid using such features to enforce combinatory control.

the formal system should not have to refer to grammar specific properties.


Finally, the multi-modal setting also enables us to introduce more powerful combinators into the grammar, possibly taking it beyond mild contextsensitivity.

because of the tight resourcesensitive control over the applicability of combinatory rules, we can avoid a collapse to a situation where "anything goes".


Here, the input sentence is treated as a word sequence and the output tagging codifies the NEs in the sentence.

each word is tagged as either the beginning of a NE (B tag), a word inside a NE (I tag), or a word outside a NE (0 tag).


In the window, each word around w is codified with a set of primitive features, together with its relative position to w. Each primitive feature with each relative position and each possible value forms a final binary feature for the classifier (e.g., "the word_form at position -2 is calle").

the set of primitive features applied to each word in the window is the following: • Lexical Features The word forms.


Our approach is to port a NER system by porting the model features from Spanish to Catalan.

we concentrate on the features which are language dependent, namely, the lexical features (or word forms) and the functional words.


In addition, bootstrapping techniques should be better studied in this domain, in order to take advantage of the large quantities of available unlabelled data.

we think that it is worth investigating the size and selection of the retraining corpora, and the combination of several algorithms or example views like in the co-training algorithms presented in (Collins and Singer, 1999;Abney, 2002).


Further bottom-up pheno-parsing can in principle be interleaved with top-down tecto-prediction in any manner.

in a chart-parser implementation, we require four kinds of edges: • pheno-edges: by parsing right-to-left and interpreting pheno-rules left-to-right, we need only passive (inactive) edges for bottom-up phenoparsing.


Our work aims to extend the applicability of the latter, by developing a general feature space for automatic verb classification.

merlo and Stevenson (2001) showed that verbs could be automatically classified into one of three lexical semantic classes on the basis of five simple statistical features.


To this end, we generalized the features from MS01, and extended them by considering the full set of alternations from Levin, to capture similar distinctions across a wider range of verb class variability.

we drew on Part I of Levin, which elaborates the alternations she uses to characterize verb classes, in contrast to Part II, which enumerates the classes themselves.


Since the grammar rules are infective, we are able to uniquely determine the inputs for any rule application.


Therefore, we treat the search downward through the phrase-structure tree for the location of a gap as a Markov process.

we Figure 1: Simplified tree from the Penn Treebank.


We show that our approach achieves state of the art performance in each of these cases.

we obtain a 18% accuracy improvement over the best published results for a Greek data set, while using a far simpler technique than previous investigations.


A natural choice is to pick the category c that has the largest posterior probability given the text.

In this section, we present experimental results for our approach on three different languages.


We have demonstrated our approach on three different languages and obtained effective performance in each case.

we have obtained a 18% accuracy improvement for one Greek data set (group B), over a state of the art system that uses signifi- We thank E. Stamatatos for supplying us with the Greek data.


Information structure thus constitutes a crucial factor in determining an expression's contextual appropriateness or interpretability.

in applications that involve human-computer interaction, information structure has thus been found to have a great impact on the understandability of computergenerated language, e.g.


The examples in (3) illustrate several possible information structures, given the placement of the pitch accent.

in languages that have a degree of word order freedom inbetween English and Slavic languages like Czech, we can find examples of a strong interaction between word order and tune.


The progress of the love relationship is viewed as a journey, the lovers as people undertaking the journey, and the love relationship as the vehicle in which they are travelling.

the journey is said to map to the relationship's progress, the travellers to the lovers, and the vehicle to the relationship.


The first heuristic is based on the insight that a variation is more likely to be an error than a true ambiguity if it occurs within a long stretch of otherwise identical material.

the longer the variation n-gram, the more likely that the variation is an error.


Problematic cases Of the 2436 erroneous variation nuclei we discussed above, 140 of them deserve special attention here in that it was clear that the variation was incorrect, but it was not possible to decide based on the tagging guide (Santorini, 1990) which tag would be the right one to assign.

9 even without knowing the correct tag, it is clear that the context demands a uniform tag assignment.


This paper investigates the integration of both linguistic knowledge and external resources for TREC-style question answering.

we describe a high performance question answering system called QUALIFIER (QUestion Answering by Lexical Fabric and External Resources) and analyze its effectiveness using the TREC-11 benchmark.


It uses on the original content words in q" to retrieve the top N", documents in the Web using Google and then extracts the terms in those documents that are highly correlated with the original query terms.

for Vqi" Ea it extracts the list of nearby non-trivial words, wi, where ds(wikile) gives the number of instances that wk and q/°) appear together; and ds (w,kVe ) gives the number of instances that either wfic or qi (p) appears.


We demonstrated that a machine learning approach can overcome the problem of sparse data which is closely related to the productivity of compounding.

by exploiting information about frequent compounds or frequent contexts (which can be easily retrieved from the corpus) we can indirectly recreate evidence about the likelihood of two nouns to form a valid com-pound without necessarily relying on parsed text.


the BLAST system (Krauthammer et al., 2000) and FASTR (Jacquemin, 2001)).

a very common term variation phenomenon in some domains is the usage of acronyms.


Feature extraction rules can carry out any type of processing.

they may involve performing speech recognition on speech data, parsing on text data, application of hand-coded rules to the results of parsing, or some combination of these.


% Accuracy (% Error reduction) Task Base All fts  Bst+w  Bst+w+c  2+2  4way 65.3 66.2 70.9(16) 72.7(19) 69.9(13) 73.9(23) 71.1(17) 73.9(23) Table 1: Percent accuracy (percent error reduction) using combination of features in the two different experimental settings Each binary classifier reaches good or even the state of the art accuracy for these tasks.

the classifier disambiguating the noun-verb attachment reaches an accuracy of 80.2% (baseline 71.6%.


The asymmetry between nominalisation and verbalisation in Chinese was observed in Tai (1997).

verbs are more freely deverbalised than nouns denominalised.


Out of these, 672 were found also tagged nouns or nominalised verbs.

about 10% of all verbs in the subcorpus were playing a role somewhere in the verb-noun categorial shift.


For example, whereas the proper noun John is assigned to two distinct categories in the lexicon in Table  2, in the T-CCG lexicon it is assigned to a single non-maximal type 'NP sg ' which subsumes the two maximal types 'NP sg sbj ' and 'NP sg obj '.

the phenomenon of case syncretism in English proper nouns is captured by having a general singular noun phrase type, which subsumes a plurality of case distinctions.


For example, forward application is expressed as follows -for all compex category descriptions Φ and Ψ such that (SLASH b) ∈ Φ, and {φ | (ARG φ) ∈ Φ} ∪ Ψ is compatible, the following is a valid inference: The functionality ratio of the I-CCG lexicon in Table 4 is 14 14 = 1 and the atomicity ratio is 14 14 = 1.

the lexicon is maximally nonredundant, since all the linguistically significant generalisations are encodable within the lexical inheritance hierarchy.


One way to allow such generalisations to be made involves incorporating the * modal iteration operator used in Propositional Dynamic Logic (Harel, 1984) to denote an unbounded number of arc traversals in a Kripke structure.

category description (RES* φ) is satisfied from node n in a model just in case some finite sequence of result arcs leads from n to a node where φ is satisfied.


In addition, the I-CCG formalism satisfies the 'strong competence' requirement of Bresnan (1982), according to which the grammar used by or implicit in the human sentence processor is the competence grammar itself.

although the result of cache-ing out particularly common lexical entries will undoubtedly be part of a statistically optimised parser, it is not essential to the tractability of the formalism.


Then, out of all queries for which the true answer is an out-of-Wikipedia entity, a subset is randomly selected such that, in the end, the number of queries with out-of-Wikipedia true answers is ½¼± of the total number of queries.

the scenario assumes the task is to detect if a name denotes an entity belonging to the People by occu-pation taxonomy and, in the positive cases, to disambiguate between multiple entities under People by occupation that have the same name.


Experimental results show that the Class-Example approach has best performance.

it reached 65% of accuracy, outperforming in our experimental framework the stateof-the-art Class-Word method by 42%.


In Section 2, we will describe the computation of consensus translations with our approach.

we will present details of the enhanced alignment and reordering procedure.


In this paper, we have shown that information presentation in SDS can be improved by an approach that combines a user model with structuring of options through clustering of attributes and successive refinement.

when presented with dialogues generated by a system that combines user modelling with successive refinement (UMSR) and one that uses refinement without reference to a user model (SR), participants reported that the combined system provided them with a better overview of the available options and that they felt more certain to have been presented with all relevant options.


whether they reduce the search space for the parser.

we define a new measure for the degree of non-projectivity in an acyclic dependency graph obeying the single-head constraint.


We have investigated a series of graph-theoretic constraints on dependency structures, aiming to find a better approximation than PROJECTIVITY for the structures found in naturally occurring data, while maintaining good parsing efficiency.

we have defined the degree of nonprojectivity in terms of the maximum number of connected components that occur under a dependency arc without being dominated by the head of that arc.


Sometimes a non-projective representations are preferred, as in the sentence in Figure 2.

1 for freer-word order languages, non-projectivity is a common phenomenon since the relative positional constraints on dependents is much less rigid.


This robustness to approximations comes from the fact that the online framework sets weights with respect to inference.

the learning method sees common errors due to Training data: T = {(xt, yt)} T t=1 1. w (0) = 0; v = 0; i = 0 2. for n : 1..N 3. for t : 1..T min where y = arg max y s(xt, y ; i+1) 6.


We can see that there is clearly an advantage in introducing second-order features.

the complete tree metric is improved considerably.


This measure allows for close matches in segment boundaries.

the boundaries must be within words/sentences, where is set to be half the av- erage segment length in the test data.


We explored the use of an LSA-PLSA model when averaging multiple PLSA models to reduce the effect of poor model initialization.

the adjacent block similarity from multiple models was averaged and used in the dip computations.


A disambiguation in this case is correct, if the sentence like I ate the banana is assigned to sense #1 (banana) instead of #2 (door).

all sentences where one of the two words occurs are viewed as one set and the WSD algorithm is then supposed to sort them correctly apart.


In our experiments, we hold all variables constant except for the alignment space being searched, and in the case of imperfect searches, the search method.

all of the methods we test will use the same objective function to select the "best" alignment from their space.


Logistic regression scales the similarity scores monotonically but non-linearly.

it changes the density of points at different score levels.


The efficacy of TF*IDF ranking is remarkable in the face of previous results.

tF*IDF was not chosen by the feature selection algorithm of Hatzivassiloglou et al.


For the feature set that contains only information about gaze behavior combined with information about the speaker (Gaze+SP), both classifiers perform significantly better when exploiting gaze information of all meeting participants.

when using solely focus of visual attention to identify the addressee of a dialogue act, listeners' focus of attention provides valuable information for addressee prediction.


In this paper, we discuss misunderstanding problem in the dialogue involving participant's actions as well as utterances.

we focus on misunderstanding in action control dialogue.


This flexibility impacts negatively on performance at all levels of system design, from speech recognition to intention interpretation.

clarification questions initiated by the user are difficult to process because they may appear off-topic and can occur at any point.


One is to extend dialogue system design to be able to meet these contingently rele-vant contributions whenever they occur.

we adapt dialogue manager, lexical database etc.


as listed in Table 2 in advance by means of system-initiated assertions.

we can guide, or shape, to use the terminology introduced by Zoltan-Ford (1991), the users' linguistic behaviour.


Whereas the avoidance of clarificatory userinitiated subdialogues is clearly a benefit, we can also use the results of our empirical investigations to motivate improvements in the other areas of interactive work undertaken by speakers.

topic and activity analysis can become problematic when the decompositions adopted by a user are either insufficient to structure dialogue appropriately for interpretation or, worse, are incompatible with the domain models maintained by the artificial agent.


Secondly, the contents of clarification subdialogues have also been identified as particularly dependent on recipient design.

they concern the preconditions for formulating utterances particularly for the respective hearer.


The preferred choice is then to influence the users' concepts of their communication partner and thus their linguistic behaviour by shaping (Zoltan-Ford, 1991).

schegloff's analysis shows in detail the human interlocutors' preference for those location terms that express group membership.


However, positive-negative disagreement is observed for only 0.7% of the data.

this statistics means that the task of extracting neutral examples, which has hardly been explored, is intrinsically difficult.


The bootstrapped error estimate shows that the unbiased Brown samples are significantly more random than the biased samples and, orthogonally, of the BNC and 3esl samples.

medium frequency terms seem to produce the best results, although the difference among the three Brown categories are not significant.


When we calculate the termhood of a string, we subtract from its total frequency its frequency as a substring of longer candidate terms.

a shorter compound that almost always appears nested in a longer compound will have a comparatively smaller C-value, even if its total frequency is higher than that of the longer compound.


Let us not forget that this article describes only a first attempt at compiling French-Japanese terminology, and that various sources of improvement have been left untapped.

our alignment suffers from the fact that we do not discriminate between different candidate translations.


Nevertheless, a higher "amount" of reordering between a candidate translation and a reference translation should still be reflected in a worse evaluation score.

the more blocks there are to be reordered between reference and candidate sentence, the higher we want the measure to evaluate the distance between these sentences.


When extending the metric by block movements, we drop this constraint for the candidate translation.

only the words in the reference sentence have to be covered exactly once, whereas those in the candidate sentence can be covered zero, one, or multiple times.


An interesting topic in MT evaluation research is the question whether a linear combination of two MT evaluation measures can improve the correlation between automatic and human evaluation.

we expected the combination of CDER and PER to have a significantly higher correlation with human evaluation than the measures alone.


Last year's evaluation exercise (Lee and Przybocki, 2005) was startling in that Bleu's rankings of the Arabic-English translation systems failed to fully correspond to the manual evaluation.

the entry that was ranked 1st in the human evaluation was ranked 6th by Bleu.


(2003) for segmenting meeting transcripts into major topical units.

this study uses the following features: (1) lexical cohesion features: the raw lexical cohesion score and probability of topic shift indicated by the sharpness of change in lexical cohesion score, and (2) conversational features: the number of cue phrases in an analysis window of 5 seconds preceding and following the potential boundary, and other interactional features, including similarity of speaker activity (measured as a change in probability distribution of number of words spoken by each speaker) within 5 seconds preceding and following each potential boundary, the amount of overlapping speech within 30 seconds following each potential boundary, and the amount of silence between speaker turns within 30 seconds preceding each potential boundary.


The goal of using MDPs is to determine the best policy ¡ for a certain state and action space.

we wish to find the best combination of states and actions to maximize the reward at the end of the dialogue.


To test this out, we conducted a 20-fold crossaveraging test over our corpus of 20 students.

we made 20 random orderings of our students to prevent any one ordering from giving a false convergence.


As a further test, we also check that the policies generated for each cut tend to stabilize over time.

the differences between a policy at a smaller cut and the final cut converge to zero as more data is added.


The paper is organized as follows: Section 2 discusses previous approaches to sentence compression.

we discuss the advantages and disadvantages of the models of Knight and Marcu (2000).


In this section we described a discriminative online learning approach to sentence compression, the core of which is a decoding algorithm that searches the entire space of compressions.

let the score of a compression y for a sentence x as we are going to factor this score using a first-order Markov assumption on the words in the compressed sentence Finally, we define the score function to be the dot product between a high dimensional feature representation and a corresponding weight vector Note that this factorization will allow us to define features over two adjacent words in the compression as well as the words in-between that were dropped from the original sentence to create the compression.


This decoding algorithm is dynamic with respect to compression rate.

the algorithm will return the highest scoring compression regardless of length.


Lets consider the sentence x = Mary saw Ralph on Tuesday after lunch, with corresponding parses given in Figure 2.

lets consider the feature representation f(x,3,6).


In particular, lets consider the feature representation f(x,3,6).

the feature representation of making Ralph and after adjacent in the compression and dropping the prepositional phrase on Tuesday.


However, we use these syntactic constraints as soft evidence in our model.

they represent just another layer of evidence to be considered during training when setting parameters.


Based on this observation, we argue that any strategy to effectively summarize evaluative text about a single entity should rely on a preliminary phase of information extraction from the target corpus.

the summarizer should at least know for each document: what features of the entity were evaluated, the polarity of the evaluations and their strengths.


Note that this is one complaint absent from the MEAD* summaries.

where the MEAD* summaries lack structure but contain detail, SEA summaries provide a general, structured overview while lacking in specifics.


As part of the project, the SUMTIME team created a corpus of 1045 forecasts from the commercial output of five different forecasters and the input data (numerical predictions of wind, temperature, etc) that the forecasters examined when they wrote the forecasts .

the SUMTIME corpus contains both the inputs (numerical weather predictions) and the outputs (forecast texts) of the forecast-generation process.


The SUMTIME system builders believe deviating from corpus frequency in such cases makes SUMTIME texts better from the reader's perspective, and it does appear to increase human ratings of the system; but deviating from the corpus in such a way decreases the system's score under corpus-similarity metrics.

judging the output of an NLG system by comparing it to corpus texts by a method that rewards corpus similarity will penalise systems which do not base choice on highest frequency of occurrence in the corpus, even if this is motivated by careful studies of what is best for text readers.


pCRU-roulette differs from the other statistical generators because it does not always try to make the most common choice (maximise the likelihood of the corpus), instead it tries to vary choices.

if there are several competing words and phrases with similar prob-abilities, pCRU-roulette will tend to use different words and phrases in different texts, whereas the other statistical generators will stick to those with the highest frequency.


We approximate the posterior probability Owing to the large set of features that are used for predicate identification and argument extraction, we estimate the probabilities using a classification model.

we use the Adaboost classifier (Freund and Schapire, 1996) wherein a highly accurate classifier is build by combining many "weak" or "simple" base classifiers j , each of which may only be moderately accurate.


We would also like to explore the feasibility of inducing the syntactic and morphological transformations automatically.

for the treebank transduction approach, it would be interesting to apply an LA language model for the lexical substitution phase as a means of pruning out implausible word sequences.


An initial step toward unifying the formalisms was taken (Shieber, 2004) in making use of the formallanguage-theoretic device of bimorphisms, previously used to characterize the tree relations definable by tree transducers.

the tree relations definable by synchronous tree-substitution grammars (STSG) were shown to be just those definable by linear complete bimorphisms, thereby providing for the first time a clear relationship between synchronous grammars and tree transducers.


The additional power of embedded transducers is, we will show in this section, exactly what is needed to characterize the additional power that TAGs represent over CFGs in describing tree languages.

we show that the relation between a TAG derivation tree and derived tree is characterized by a deterministic linear complete embedded tree transducer (DL-CETT).


Because of the more severe constraints on TAG, in particular that all combinatorial limitations on putting subtrees together must be manifest in the labels in the trees themselves, the outputs actually contain more structure than the corresponding transducer output.

the state-labeled nodes are merely for bookkeeping.


If we assume that a set of answer candidates is provided to our model by some means (e.g., words comprising documents extracted by an information retrieval engine), we wish to compute the value P (in(w, Γ Q )|w).

the appropriateness of a candidate answer w is proportional to the probability that it will occur in the question contexts Γ Q extracted from the question.