To overcome the limitations, this paper presents SOLAR a new framework of Scalable Online Learning Algorithms for Ranking, to tackle the challenge of scalable learning to rank.

we propose two novel SOLAR algorithms and analyze their IR measure bounds theoretically.


Learning to rank [27,8,29,31,7] aims to learn some ranking model from training data using machine learning methods, which has been actively studied in information retrieval (IR).

consider a document retrieval task, given a query, a ranking model assigns a relevance score to each document in a collection of documents, and then ranks the documents in decreasing order of relevance scores.


To overcome the above limitations, this paper investigates SOLAR -a new framework of Scalable Online Learning Algorithms for Ranking, which aims to learn a ranking model from a sequence of training data in an online learning fashion.

by following the pairwise learning to rank framework, we formally formulate the learning problem, and then present two different SOLAR algorithms to solve the challenging task together with the analysis of their theoretical properties.


Similar to [16], we employ the Kullback-Leibler divergence to measure the distance between the current model w to be optimized and the previous model w t , and the regularization terms include both the loss suffered at current triplet instance and the confidence on current triplet instance.

we formulate the optimization of second-order online learning to rank as: where γ is the trade-off parameter.


In the case of graph-of-words, node labels are unique inside a graph since they represent unique terms of the document and we can therefore omit these functions since they are injective in our case and we can substitute node ids for node labels.

the general problem of subgraph matching, which defines an isomorphism between a graph and a subgraph and is NP-complete (Garey and Johnson, 1990), can be reduced to a polynomial problem when node labels are unique.


For these reasons, we could not capitalize on all standard methods.

we believe new kernels that support a very high number of unique node labels could yield even better performances.


We compare the nearest cross-language neighbors in the various representations in Table 1.

we selected five words from the Spanish test data and searched for its three nearest neighbors in KLEMENTIEV, CHANDAR and INVERTED.


To conquer the problems described above, we propose a multi-task learning framework based on a sequence learning model to conduct machine translation from one source language to multiple target languages, inspired by the recently proposed neural machine translation(NMT) framework proposed by .

we extend the recurrent neural network based encoder-decoder framework to a multi-task learning model that shares an encoder across all language pairs and utilize a different decoder for each target language.


We extend this into multiple languages setting.

suppose we want to translate from English to many different languages, for instance, French(Fr), Dutch(Nl), Spanish(Es).


Recall is defined as the number of correctly segmented words divided by the total number of words in the gold standard segmentation.

r oov reflects the model generalization ability.


The solid nodes indicate the active neurons, while the hollow ones indicate the suppressed neurons.

the links denote the information flow, where the solid edges denote the acceptation of the combinations while the dashed edges means rejection of that.


Then each character c ∈ C is mapped into an ddimensional embedding space as c ∈ For each character c i in a given sentence c 1:n , the context characters c i−w 1 :i+w2 are mapped to their corresponding character embeddings as c i−w 1 :i+w 2 , where w 1 and w 2 are left and right context lengths respectively.

the unknown characters and characters exceeding the sentence boundaries are mapped to special symbols, "unknown", "start" and "end" respectively.


Inspired by the success of the gated recurrent neural network (Cho et al., 2014b;Chung et al., 2014), we propose a gated recursive neural network (GRNN) by introducing two kinds of gates, namely "reset gate" and "update gate".

there are two reset gates, r L and r R , partially reading the information from left child and right child respectively.


The main idea of layer-wise training is to train the network with adding the layers one by one.

we first train the neural network with the first hidden layer only.


Hence, we utilize the dropout strategy in this work.

dropout is to temporarily remove the neuron away with a fixed probability p independently, along with the incoming and outgoing connections of it.


The concept of feature embedding is quite similar to that of character embedding mentioned above.

each context feature is represented as a single vector called feature embedding.


We associate the new topic with the cluster whenever it is close enough.

close documents contribute to a cluster, allowing it to drift towards topic dense regions.


Since these long words are often predictable from some character level information including suffixes or character types, in a Type Feature Character type at time t−i (0 ≤ i ≤ 4) cont # of the same character types before t ch # of times character types changed within 8 characters before t semi-supervised setting we employ a Negative Binomial generalized linear model (GLM) for setting L t adaptively for each character position t in the corpus.

we model the word length ℓ by a Negative Binomial distribution (Cook, 2009): This counts the number of failures of Bernoulli draws with probability (1−p) before r'th success.


Our intuition is that using this proportion, CTB should not be overwhelmed by PD, and both training data can be used up in relatively similar speed.

all training data of CTB can be consumed in about 3 iterations, whereas PD can be consumed in about 14 iterations.


In summary, we can conclude that our coupled model achieves much better performance when the mapping function becomes looser.

this suggests that our coupled model can effectively learn the implicit mapping between heterogeneous annotations, and does not rely on a carefully designed mapping function.


When M ′ becomes smaller, our coupled model becomes inferior to the baseline model.

when M ′ = 1K, the model converges very slowly.


Experimental results demonstrate that our model work better than the baseline and guide-feature based methods on both one-side POS tagging and annotation conversion.

detailed analysis shows several interesting findings.


For example, you can take the vector of king, subtract the vector of man and add the vector of woman to get a vector near queen.

you remove the properties of man and add the properties of woman.


6 states, that Additionally, there is no interaction between different dimensions, so we are creating the tensor by stacking the diagonal matrices E (i,j) over i and j.


So E (d) ∈ R |S|×|W | is also normalized over the first dimension.


Semantic similarity is a field of Natural Language Processing which measures the extent to which two linguistic items are similar.

word similarity is one of the most popular benchmarks for the evaluation of word or sense representations.


Note that the pairs in the two monolingual datasets should be previously aligned.

we refer to each dataset D as {P D , S D }, where P D is the set of pairs and S D is a function mapping each pair in P D to a value on a similarity scale (0-4 for RG-65).


We perform a proof-of-concept experiment where we represent not as a function that is trained to map an adjective to its antonym (good to bad).

by adopting the framework of , we take not to be a matrix that, when multiplied with an adjectiverepresenting vector, returns the vector of an adjective with the opposite meaning.


In this paper, we address semi-supervised sentiment learning via semi-stacking, which integrates two or more semi-supervised learning algorithms from an ensemble learning perspective.

we apply metalearning to predict the unlabeled data given the outputs from the member algorithms and propose N-fold cross validation to guarantee a suitable size of the data for training the meta-classifier.


To better use the labeled samples in the initial labeled data, we employ N-fold cross validation to generate more meta-samples.


Taking sentiment analysis for dialogues as an example, the topic of the document and the author's identity are both valuable for mining user's opinions in the conversation.

each participant in the dialogue usually has specific sentiment polarities towards different topics.


The time series dependence is modeled by a Markov chain.

we assume that there is a labeling matrix L such that and a transition matrix T such that These two equations establish the relation between the hidden state and the labels.


The neural network model takes (H t−1 , s t , u t ) as input, and predict H t as its output.

we use a logistic model to define the probability: The vectors w h , w u , w s are linear combination coefficients to be estimated.


For both datasets, we set the ground truth of sentiment classification of tweets by using human annotation.

we randomly select 1000 conversations from each datasets, and then invite three researchers who work on natural language processing to label sentiment tag of each tweet (i.e., positive, negative or neutral) manually.


Therefore, reviews need to be ranked and recommended to consumers.

review helpfulness plays a critical role in review ranking and recommendation (Ghose and Ipeirotis, 2011;Mudambi and Schuff, 2010;Danescu-Niculescu-Mizil et al., 2009).


Up to this point, we are very interested in first whether a prediction model learned for one category can be generalized to a new category, and second what elements make a review helpful.

we want to know the robustness of our approach and the underlying reasons.


This trains the distributed representation to be useful as an input layer for prediction of words from their neighbors in a Skipgram likelihood.

to maximize summed across all words w st in all sentences w s , where b is the skip-gram window (truncated by the ends of the sentence) and p V (w sj |w st ) is a neural network classifier that takes vector representations for w st and w sj as input (see Section 2).


M-L captures this notion using the difference in the cross-entropies according to each language model (LM).

for a sentence s i , we compute where H tgt is the cross-entropy under the indomain language model and H gen is the crossentropy under the general domain LM.


Here, we weight each sentence pair in the bilingual training corpus proportional to σ i when computing the paraphrase scores.

we weight the counting during the bilingual pivoting so that rather than each occurrence counting as 1, each occurrence counts as the ratio of the sentence's cross-entropies: Htgt .


In the case of relational reference, this means that both target and landmark portions of the description may be left underspecified, and uniqueness will follow from the fact that they mutually disambiguate each other (Teixeira et al., 2014).

example (a) may be considered felicitous even though both 'cone' and 'box' are ambiguous if interpreted independently.


The work in (van Gompel et al., 2014), for instance, points out that when perceptual salience is manipulated so that a high contrast between target and distractors is observed, the size attribute may be preferred to colour.

a highly preferred property may not necessarily match the choices made by human speakers when producing overspecified descriptions.


A graphical representation of one link in the parsing model is shown in Figure 1.

each potential relation r ij between elementary discourse units e i and e j is accompanied by a corresponding latent variable as h ij .


To measure how informative a sequence of relations is, we use density-weighted certainty sampling (DCS).

for a sequence of relations r 1 ...r n taken from a document, d, we use the following formula: In this equation, H(r i ) represented the entropy of the distribution of label predictions for the relation r i generated by the current spectral model, which is a measure of the model's uncertainty for the label of the given relation.


The first line of approaches attempts to generate novel captions directly from images (Farhadi et al., 2010;.

they borrow techniques from computer vision such as object detectors and scene/attribute classifiers, exploit their outputs to extract the visual content of the input image and then generate the caption through surface realization.


Clearly, the image features and the corresponding similarity measures used in retrieval play a significant role here but, as investigated in , what makes this particularly difficult is that while describing an image humans do not explicitly mention every detail.

some parts of an image are more salient than the others.


In our study, we use the recently proposed Caffe deep learning features (Jia et al., 2014), trained on ImageNet, which have been proven to be effective in many computer vision problems.

we use the activations from the seventh hidden layer (fc7), resulting in a 4096-dimensional feature vector.


In our query expansion step, we swap modalities and construct a new query based on the distributed representations of captions.

we expand the original visual query with a new textual query based on the weighted average of the vectors of the retrieved captions as follows: where N and M respectively denote the total number of image-caption pairs in the candidate set N and the number of reference captions associated with each training image, and sim(I q , I i ) refers to the visual similarity score of the image I i to the 1 The adaptive neighborhood parameter was emprically set to 0.15. query image I q 2 which is used to give more importance to the captions of images visually more close to the query image.


(2015), who try different ways of modulating the inclusion of perceptual input in their multi-modal skip-gram model, and find that the entropy of the centroid vector µ works well (where p(µ j ) = µ j || µ|| and m is the vector length): We calculate the directionality of a hyponymhypernym pair with a measure f using the following formula for a word pair (p, q).

since even cohyponyms will not have identical values for f , we introduce a threshold α which sets a minimum difference in generality for hypernym identification: s(p, q) > 0 iff f (q) > f (p) + α, i.e.


The inability to model long-distance dependency has been handicapping SMT for years.

the context independence assumption makes it hard to capture the dependency between translation rules.


(2013) to utilize force decoding to get golden rule derivations.

we define a new forced decoding LM which only accepts two consecutive words (denote as p, q) in the reference translation (y i ): 1 if ∃j, s.t.


See c(a(i, j)) is decomposed as: The three terms in this formula are mutually independent.

any reordering of a(i, p) changes only the first term and the others are unchanged.


Table 4 shows the performance of our method, which indicates that our preordering significantly improved translation accuracy in both RIBES and BLEU scores, from the baseline result attained by Moses without preordering.

the preordering model trained with the Giza data revealed a substantial improvement, while the use of the Nile data further improves accuracy.


We examined how often an intrinsic metric correctly predicts the difference of the subjects' test performance on a problem.

for two translation methods A and B, we say an intrinsic metric M agrees with the CAR by the subjects on problem p i iff metric M scores A's translation of p i (= A(p i )) better than B's translation (= B(p i )) and the CAR is higher on A(p i ) than on B(p i ).


The authors would like to thank Colin Cherry and Roland Kuhn for useful discussions.

evaluating the quality of a translation against human references is a challenging task due to the flexible word ordering of a sentence and the existence of a large number of synonyms for words.


Experiments on the WMT metric task shows that the metric based on the combined representations achieves the best performance, outperforming the state-of-the-art translation metrics by a large margin.

training the distributed representations only needs a reasonable amount of monolingual, unlabeled data that is not necessary drawn from the test domain.


Many machine translation metrics have been proposed in recent years, such as BLEU (Papineni et al., 2002), NIST (Doddington, 2002), TER (Snover et al., 2006), Meteor (Banerjee and Lavie, 2005) and its extensions, and the MEANT family (Lo and Wu, 2011), amongst others.

evaluating translation, however, is not easy.


We propose to evaluate the translations with different word and sentence representations.

we investigate the use of three widely deployed representations: one-hot representations, distributed word representations learned from a neural network model, and distributed sentence representations computed with recursive autoencoder.


Specifically, we investigate the use of three widely deployed representations: one-hot representations, distributed word representations learned from a neural network model, and distributed sentence representations computed with recursive autoencoder.

to leverage the different advantages and focuses, in terms of benefiting evaluation, of various representations, we concatenate the three representations to form one vector representation for each sentence.


This method works under an unsupervised setting.

it does not rely on a parsing tree structure in order to generate sentence level vectors.


Each of the above mentioned representations has a different strength in terms of encoding syntactic and semantic contextual information for a given sentence.

the one-hot representation is able to reflect the particular words that occur in the sentence.


we set n equal to 2 for the one-hot n-gram representation, the vector size of the recursive auto-encoder to 10, and the vector size of word embeddings to 80.


Results in Tables 1 suggest that metrics based on three single representations all obtained comparable or better performance than BLEU, TER and Meteor.

the metric based on recursive auto-encoder outperformed the other testing metrics on system-level.


In contrast, the representation based metric associates hypothesis 2 with a much higher score than that of hypothesis 1, namely 0.865 and 0.555, respectively.

the score for hypothesis 2 is close to one, suggesting that the RAE based metric considers this translation is almost identical to the reference.


(2011) experiment with different thresholds set on word alignment strengths to filter context information.

they discard the (f #e, f ) pairs in which the f #e alignment score is smaller than the threshold.


Using a few different NLP tasks, we then empirically show that our method can indeed learn a better classifier for the target domain than a few baselines.

our method performs consistently better than or competitively with Structural Correspondence Learning (SCL) (Blitzer et al., 2006), a wellknown unsupervised domain adaptation method in NLP.


In EA++, unlabeled target data is used to construct a regularizer that brings the two classifiers of the two domains closer.

the regularizer defines a penalty if the source classifier and the target classifier make different predictions on an unlabeled target instance.


Convolutional neural networks (CNNs), originally invented in computer vision (LeCun et al., 1995), has recently attracted much attention in natural language processing (NLP) on problems such as sequence labeling (Collobert et al., 2011), semantic parsing (Yih et al., 2014), and search query retrieval (Shen et al., 2014).

recent work on CNN-based sentence modeling (Kalchbrenner et al., 2014;Kim, 2014) has achieved excellent, often state-of-the-art, results on various classification tasks such as sentiment, subjectivity, and question-type classification.


Experiments on three classification tasks demonstrate the superior performance of our DCNNs over the baseline sequential CNNs.

our accuracy on the TREC dataset outperforms all previously published results in the literature, including those with heavy hand-engineered features.


The expectation of c i,j can then be written as: The GloVe objective is defined in the following form (Pennington et al., 2014): where β(•) represent a 'weighting function'.

β(•) satisfies the relations 0 ≤ β(x) < ∞, and β(x) = 0 if x = 0.


Future research will be devoted to find models to effectively combine TKs and DNN.

our previous model exploiting Linked Open Data in QA (Tymoshenko et al., 2014) seems very promising to find correct answer to clues.


In a modified version ofWordNet-based vector similarity, Sim WV ), (Li et al., 2006), we first create vectors v n and v m for each narrative n and m, where each element corresponds to a word in the type union of n and m. We assign values to each element e in v n using the following formulation: S(e, n) = 1 if e ∈ n max where LS is Lin's universal similarity (Lin, 1998).

if the element e is present in n, S(e, n) will be 1.


All roles that have a common parent based on the Inheritance and Subframe relations will share a set of features in common.

for each base feature φ which is conjoined with the role r in the baseline model (φ ∧ "role=r"), and for each parent r ′ of r, we add a new copy of the feature that is the base feature conjoined with the parent role, (φ ∧"parent_role=r ′ ").


It turns out that the best olfactory representations are obtained with 30 dimensions.

our findings appear to corroborate recent evidence suggesting that olfactory space (at least when using SAFC as a data source) is best modeled using around 30 dimensions.


In extracting predicate-argument structures, it is not possible to directly extract a coordinated noun phrase "wine and sake" as a direct object of the verb "drank".

we need an implicit interpretation rule in order to extract NP in coordinating construction: head bunsetsu b3 should be divided into a content word and a function word , then the content word should be merged with the dependent bunsetsu b2.


that can be a part of a well-formed parse 2 ), c A is entailed by c (i → j) for every i → j ∈ A.

if each arc in a tree-compatible set is individually reachable from configuration c, then that Shift: Right-Arc: only if k | k → j ∈ A (single-head) and j → * i ∈ A (acyclicity).


First, we note that no node in this graph has in-degree greater than 1.

each node except for the dummy root has exactly one head, either explicit or (if no head has been assigned in A or in the gold tree) the dummy root.


Then W 1 is applied to each possible window in the shortest dependency path to produce a feature map: Next, we adopt the widely-used max-over-time pooling operation (Collobert et al., 2011), which can retain the most important features, to obtain the final representation L from the feature map.

like other relation classification systems, we also incorporate some lexical level features such as named entity tags and WordNet hypernyms, which prove useful to this task.


We propose a new approach to the task of fine grained entity type classifications based on label embeddings that allows for information sharing among related labels.

we learn an embedding for each label and each feature such that labels which frequently co-occur are close in the embedded space.


Intuitively, we try to rank labels in Y higher than labels inȲ.

we use the weighted approximate pairwise (WARP) loss of .


For a disorder mention containing numbers between one to ten, new forms are produced by replacing each number in the mention with other forms of the same number.

we consider the numeral, roman numeral, cardinal, and multiplicative forms of a number for replacement.


The subsequent rows show the results of our approach when our ten sieves are added incrementally.

each row shows the results obtained after adding a sieve to the sieves in the previous rows.


In this case, context can be used to disambiguate the mention.

a coreference resolver can first be used to iden-tify the coreference chain to which the ambiguous mention belongs, and then the ambiguous mention can be normalized by normalizing its coreferent yet unambiguous counterparts instead.


For example, one can exploit the contexts surrounding the mentions in the training set.

given a test disorder mention, one may first identify a disorder mention in the training set that is "sufficiently" similar to it based on context, and then normalize it to the concept that the training disorder mention is normalized to.


Another possibility is to exploit additional knowledge bases such as Wikipedia.

one can query Wikipedia for the test mention's string, then employ the titles of the retrieved pages as alternate mention names.


In this paper, we study Open Information Extraction's (Open IE) output as an additional intermediate structure and find that for tasks such as text comprehension, word similarity and word analogy it can be very effective.

for word analogy, Open IE-based embeddings surpass the state of the art.


A key strength is Open IE's ability to balance lexical proximity with long range dependencies in a single representation.

for the word analogy task, Open IE-based embeddings 6 http://www.cs.bgu.ac.il/˜gabriels Dependency Open IE canine dog feline dog incisor bovine carnassial dentition equine feline parvovirus porcine fang-like dysplasia murine bovine Table 4: Closest words to canine in various word embeddings.


As mentioned above, the CN14 model was trained on overt pronouns, but it is not clear how much this helped its resolution performance.

the contexts in which overt and zero pronouns occur may not statistically resemble each other.


1 In addition, motivated by work on overt pronoun resolution, we hypothesize that AZP resolution can be improved by exploiting discourse information.

we design a model of salience and incorporate salience information into our model as a feature.


Motivated in part by previous work on English overt pronoun resolution (e.g., Cherry and Bergsma (2005) and Charniak and Elsner (2009)), we estimate the model parameters using the Expectation-Maximization algorithm (Dempster et al., 1977).

we use EM to iteratively (1) estimate the model parameters from data in which each ZP is labeled with the probability that it corefers with each of its candidate antecedents, and (2) apply the resulting model to re-label each ZP with the probability that it corefers with each of its candidate antecedents.


f n c and assuming that each f i c is conditionally independent given l, we can approximate Expression (7) as: Furthermore, we assume that given context k, each candidate antecedent of z is generated with equal probability.

given Equations (4), (5), (8) and (9), we can rewrite P (l=1|z, k, c) as: As we can see from Equation 10, our model has one group of parameters, namely P (f i c |l=1).


Given P (l=1|z, k, c), the goal of the M-step is to (re)estimate the model parameters, P (l=1|k c ), using maximum likelihood estimation.

p (l=1|k c ) is estimated as follows: where Count(k c ) is the number of times k c appears in the training data, Count(k c , l=1) is the expected number of times k c is the context surrounding an AZp and its antecedent c, and θ is the Laplace smoothing parameter, which we set to 1.


Given context k ′ c , we compute Count(k ′ c , l=1) as follows: To start the induction process, we initialize all parameters with uniform values.


The only difference is that z's candidate antecedents may now include the NPs to which previous AZPs were resolved.

this incremental resolution procedure may increase the number of candidate antecedents of each ZP z.


We want to integrate these knowledge represented by CTs in different earthquake event articles to form a comprehensive understanding of the category "earthquake" (CTH).

our input consists of a set of Wiki articles A c = {a}, belonging to a Wiki category c. As shown in Fig.1, each article a ∈ A c contains a CT (topic tree hierarchy) where T a is a set of topics, each denoted by a tag g and associated with a text description d g , and The output is an integrated comprehensive CTH H c = {T c , R c } where T c = {t} is a set of topics, each denoted by a set of tags t = {g} and associated by a text description d t aggregated by {d g } g∈t , and We map the problem of learning the output H c from the input {H a }, a ∈ A c , as a structure learning problem.


We first find clusters of similar tags T c (each cluster represents a topic) and then derive hierarchical relations R c among these clusters.

given a category c, we first collect relevant Wiki articles A c = {a}.


Then we combine structural information and textual information by defining the weights w(t i , t j ) of the edges (t i , t j ) as a simple weighted average of w struc (t i , t j ) and w text (t i , t j ).

we define: where λ controls the impacts of text correlation and structure dependency in optimal structure learning.


In this paper, we propose a novel method to model short texts based on semantic clustering and convolutional neural network.

we first discover semantic cliques in embedding spaces by a fast clustering algorithm.


Therefore, the detection for SUs are useful, which capture salient local information, as shown in Figure 2.

to obtain the representations of candidate SUs, multiple windows with variable width over word embeddings are used to perform element-wise additive composition, as follows: where, Ewin ∈ R d×m is a window matrix with all weights equal to one, and is the jth column from the sub-matrix PM win,i , which is windowed on projected matrix PM by Ewin with the ith times sliding.


Our approach aims to extend these strategies and it is based on other research lines involving the extraction of implicit information (Palmer et al., 1986;Whittemore et al., 1991;Tetreault, 2002).

we are inspired by recent works on Implicit Semantic Role Labelling (ISRL) (Gerber and Chai, 2012) and very specially on the work by (Blanco and Moldovan, 2014) who adapted the ideas about ISRL to focus on modifiers, including arguments of time, instead of core arguments or roles.


We also set some constraints to select certain events according to the specification of the SemEval task.

we only return those events that are not negated and are not accompanied by modal verbs except will.


The explicit time anchors provided by the NLP tools presented in Section 4.1 do not cover the full set of events involving a particular entity.

most of the events do not have an explicit time anchor and therefore are not captured as part of the TimeLine of that entity.


This result contributes to the broader line of research on avoiding or reducing annotation cost in information extraction (Section 5).

it suggests the potential utility of the seed-based approach in scenarios where manual annotation per each new event is too costly.


These are candidates for the ASIC we use.

gappy pattern matching (He et al., 2015) maps directly to regular expressions.


Instead, we seek a direct transfer approach ( Figure 1) to cross-lingual NER (also classified as transductive transfer learning (Pan and Yang, 2010) and closely related to domain adaptation).

we only assume the availability of comparable corpora and small-sized bilingual dictionaries, and use the same sequential tagging model trained on the source corpus for tagging the target corpus.


Scenario 2: We have a set of blog articles and we are interested in learning how blogging differs across gender.

we seek to learn the writing styles or other indicative patterns -topics of interest, word choices etc.


In order to create a set of terms related to a topic of interest, we used the Hyper-geometric (HG) test.

given the number of sentences in the union of articles identified for all topics; the number of sentences in the articles identified for a specific topic, i.e., in the topic articles; the total number of sentences that include a particular term, t; and the number of sentences within the topic articles, that include t, denoted x; we use the HG test to assess the probability p, to observe ≥ x occurrences of t within sentences selected at random out of the total population of sentences.


Next, we exploit the relatively large size of TR9856 to demonstrate the potential for using supervised machine learning methods.

we trained a simple linear regression using the baseline methods as features, along with a token   Table 3: Mean results over 10-fold cross validation.


Here, we account for the etymology of the words, and we adopt the following definition: two words form a cognate pair if they share a common ancestor and have the same meaning.

they derive directly from the same word, have a similar meaning and, due to various (possibly language-specific) changes across time, their forms might differ.


Finally, Figure 2 shows how Spearman's ρ is affected along three dimensions of study.

we see that ∆BLEU actually benefits from the references with negative ratings.


We use part-of-speech information and dependency arcs from the gold annotation to extract noun phrases containing adjectives.

we first convert all treebanks to coarse universal part-of-speech tags, using existing conventional mappings from the original tagset to the universal tagset (Petrov et al., 2012).


In this paper, we propose a new model Extended Global Topic Random Field (EGTRF) to model non-linear dependencies between words.

we parse sentences into dependency trees and represent them as a graph, and assume the topic assignment of a word is influenced by its adjacent words and distance-2 words.


4 becomes: 4 Implementation and Dataset We modified the Feature-Augmented RNN toolkit 2 and adapted it to handle tree-structured data.

and instead of being run sequentially on the entire training corpus, the RNN is run on all the word tokens in all unrolls of all the sentences in all the books of the corpus.


In this "ideal", error-free scenario the situation slightly changes (unsurprisingly, the "% wins of colored" slightly increases, especially for long suggestions for which we have the highest number of misclassifications), but the overall conclusions remain the same.

the higher percentage of wins is statistically significant only for medium-length suggestions with HTER>0.1 and, in the best case (HTER≤0.2) it is about 56.0%.


The matching score of a source sentence and a target phrase can be measured as the similarity between their feature vectors.

we use the multi-layer perceptron (MLP), a nonlinear function for similarity, to compute their matching score.


Why did the improvement level off quickly?

why do long distance reordering models have a much less leverage over translation performance than short ones?


To this end, we focus in this work on the simple exploitation of the disagreement between hypotheses ranked best according to the decoder and to our featurerich decoder.

we seek to provide the next-pass decoder with separate translation tables that either contain bi-phrases that are unique to the decoder's one-best or to the reranker's onebest, in the hope that it will tend, in a soft manner, to exploit the preferences expressed by the complex features, and to otherwise explore alternative translation choices.


Pivot translation allows for translation of language pairs with little or no parallel data by introducing a third language for which data exists.

the triangulation method, which translates by combining source-pivot and pivot-target translation models into a source-target model, is known for its high translation accuracy.


These include cascade methods that consecutively translate from source to pivot then pivot to target (de Gispert and Mariño, 2006), synthetic data methods that machinetranslate the training data to generate a pseudoparallel corpus (de Gispert and Mariño, 2006), and triangulation methods that obtain a sourcetarget phrase/rule table by merging source-pivot and pivot-target table entries with identical pivot language phrases (Cohn and Lapata, 2007).

the triangulation method is notable for producing higher quality translation results than other pivot methods (Utiyama and Isahara, 2007), so we use it as a base for our work.


To help reduce this ambiguity, our proposed triangulation method remembers the corresponding pivot phrase as additional information to be utilized for disambiguation.

instead of marginalizing over the pivot phrase p, we create an MSCFG rule for the tuple of the connected sourcetarget-pivot phrases such as: The advantage of translation with these rules is that they allow for incorporation of additional features over the pivot sentence such as a strong pivot LM.


We can see that if a sentence has a larger ratio of its word count to its braille length, then it is more suitable to be selected.

for two sentences with the same word count, the one with a shorter braille length is preferred.


In such cases, an important question to answer is by what means such metrics should be compared.


M 2 was assessed by comparing its output against that of the official Helping Our Own (HOO) scorer (Dale and Kilgarriff, 2011), itself based on the GNU wdiff utility.

2 it was evaluated under the assumption that evaluating GEC can be reduced to checking whether a set of predefined errors have been changed into a set of associated corrections.


We here use CBOW for radical embedding because the two methods exhibit few differences, and CBOW is slightly faster in experiments.

a sequence of Chinese characters is decomposed into a sequence of radicals, to which CBOW is applied.


To capture correlations between topics, Pachinko Allocation Model (PAM) (Li and McCallum, 2006) uses a directed acyclic graph (DAG) with leaves representing individual words in the vocabulary and each interior node representing a correlation among its children.

multi-level PAM is capable of revealing interconnection between sub-level nodes by inferencing corresponding super-level nodes.


result in a range which varies from 0 to 1.

the ego user similarity π u 0 = 1, which would be a natural and intuitive answer.


One alternative is to choose a ratio between the number of words and number of keywords.

we define the number of keywords in a tweet as the ratio between number of words in the tweet and k, which is maximized on the development set.


That is, we define the number of keywords in a tweet as the ratio between number of words in the tweet and k, which is maximized on the development set.

if we set k = 3, then we extract one keyword for every three words.


In (b), irony is due to an implicit opposition between a lexicalized proposition P describing an event or state and a pragmatic context external to the utterance in which P is false or is not likely to happen.

the writer asserts or affirms P while he intends to convey P such that P = N ot(P ) or P = P .


However, a user is more likely to reply to the later and keep the information of the former in mind.

for the goal of our email briefing system, importance decided upon the user's action is insufficient.


According to the principle of statistical sampling, texts with NTAs are gathered from 384 users.

since the ultimate goal is to exploit NTAs to automatically generate labeled data for word segmentation, the main task is to select high-quality NTAs, which largely overlap with gold segmentation.


The approach described above is sufficient to parse most Wiktionary data, but a large percentage of Wiktionary inflection tables do not use the correct tags to distinguish between header and content cells, an important component of the parsing procedure.

table authors frequently use only the content cell tag to mark up all of a table's cells, and create "soft" headers with a distinct visual appearance by changing their styling (as with Czech verbs, such as spadat 'to be included, fall off').


In contrast, the linear-chain models fail to exploit the sequential dependencies between nearby answer labels to improve the results significantly: although the labels from the neighboring answers can affect the label of the current answer, this dependency is too loose to have impact on the selection accuracy.

labels should be used together with answers' content to account for stronger and more effective dependencies.


(i) We compute the maximum a posterior (MAP) or the (jointly) most probable sequence of labels using the Viterbi algorithm.

it computes y * = argmax y 1:T P (y 1:T |x 1:T ), where T is the number of comments in the thread.


For each time step in the BLSTM layer, the hidden vector or the output vector is generated by combining the cell memory vectors from two LSTM of both sides.

all the contextual information across the entire sequence (both question and answer sentences) has been taken into consideration.


We aim to learn a verb tensor V (either in full or in decomposed, low-rank form) that minimizes the mean of the squared residuals between the predicted SVO vectors V (s (i) , o (i) ) and those vectors obtained distributionally from the corpus, t (i) .

we attempt to minimize the following loss function: is given by Eq.


However, disallowing pattern X has led to other issues.

for certain semanticssentence pairs, it is not possible to find relaxed hybrid trees that jointly represent them.


We also followed the standard approach for evaluating the correctness of an output semantic representation from our system.

we used a standard script to construct Prolog queries based on the outputs, and used the queries to retrieve answers from the GeoQuery database.


We note that in our experiments we used a small subset of the features used by our relaxed hybrid tree work.

we did not use any long-distance features, and also did not use any character-level features.


In this paper, we explore the largely understudied problem of rhetorical question identification.

we present a simple n-gram based language model to classify rhetorical questions in the Switchboard Dialogue Act Corpus.


Using these observations, we decided to implement a simple n-gram model incorporating contextual cues to identify rhetorical questions.

we used unigrams, bigrams, POS bigrams, and POS trigrams of a question and its immediately preceding and following context as feature sets.


Our system for sarcasm detection augments the feature vector of a tweet with features based on the two types of incongruity.

we use four kinds of features: (a) Lexical, (b) Pragmatic, (c) Implicit congruity, and (d) Explicit incongruity features.


Such a mixture of language is called codeswitching.

code-switching text is defined as text that contains more than one language (code).


Instead of estimating independent classifiers for each user, we start from a generic model and adapt it toward individual users based on their own opinionated text data.

our key assumption is that the adaptation can be achieved via a set of linear transformations over the generic model's parameters.


To validate the necessity of personalizing sentiment classifica-tion models, we also included the global model's performance in Figure 1.

to understand the longitudinal effect of personalized model adaptation, we only used the heavy users (4,021 users) in this experiment.


This is due to the heavily biased class distribution in our collection: global model puts great emphasis on the positive reviews; while the adaptation methods give equal weights to both positive and negative reviews.

in LinAdapt, scaling and shifting operations lead to satisfactory adaptation performance for the negative class with only 15 reviews; while rotation is essential for recognizing the positive class.


For instance, we found that many sports-related articles invoke the emotion "happy".

45% of all "happy" instances are sports-related, and a sports-related article has a 31% chance of having the emotion tag "happy".


To solve these problems, we translate the whole sentences but with reordering constraints ensuring that the opinionated segments are preserved during translation.

the text between the relevant segment boundaries is not reordered nor mixed with the text outside these boundaries.


Second, whereas pagerank assigns an equal weight to the edges connected between an unseen word and its neighbor nodes, we consider their similarities as weights to construct a weighted graph such that neighbor nodes more similar to the unseen word may contribute more to estimate its VA ratings.

the proposed weighted graph model considers both the relations of multiple nodes and the similarity weights among them.


Both lexicons were used for 5-fold cross-validation.

for each run, 80% of the words in the lexicons were considered as seeds and the remaining 20% were used as unseen words.


LetB ∈ R m×d be the matrix returned by the algorithm in Figure 1.

then with probability at least 0.7, the selected rows are not arbitrarily worse than the best rank-m approximation of A (given by SVD) with high probability.


We improve a probabilistic model that considers a basic set of features for monitoring listeners' movements in a virtual environment.

in complex referential scenes, where more objects next to the target are possible referents, gaze turns out to be beneficial and helps deciphering listeners' intention.


In addition, since the SLU module in the previous dialogue systems has a complicated architecture that is composes of many sub-modules, it is difficult for them to be directly applied into the SLU module of intelligent personal assistant software with those many domains for mobile devices.

building up a complicated architecture for each domain can make a heavy system and this kind of system is not proper to mobile devices.


Our summarization system PriorSum follows the traditional extractive framework (Carbonell and Goldstein, 1998;Li et al., 2007).

the sentence ranking process scores and ranks the sentences from documents, and then the sentence selection process chooses the top ranked sentences to generate the final summary in accordance with the length constraint and redundancy among the selected sentences.


To make full use of the phrase information, we design an improved version of the standard CNNs, which use multiple filters for different window sizes as well as two max-over-time pooling operations to get the final summary prior representation.

let W 1 t , • • • , W m t be m filters for window sizes from 1 to m, and correspondingly we can obtain m feature maps C 1 , • • • , C m .


Since the convolution layer follows a linear regression output, we apply a simple strategy to measure how much the learned document-independent features contribute to the saliency estimation.

for each sentence, we ignore its documentdependent features through setting their values as  The blast killed two assailants, wounded 21 Israelis and prompted Israel to suspend implementation of the peace accord with the Palestinians.


In this paper, we revisit this assumption and evaluate the maximum coverage objective for extractive text summarization with syntactic and semantic concepts.

we replace bigram concepts with new ones based on syntactic dependencies, semantic frames, as well as named entities.


The second type of concept is dependency subtrees.

we extract labeled and unlabeled syntactic dependencies, e.g., DEPENDENCY(walks,John) or SUB-JECT(walks,John), from sentences and represent them by such syntactic concepts.


Given the source language parsing structure as in Figure 1 (left), the set of parameters E word will be different for the target language parser shown in Figure 1 (right) but we hypothesize that E pos , E arc , W 1 and W 2 can be shared as indicated with dashed lines.

we expect this to be the case when languages use the same POS tagset and arc label sets, as we presume herein.


In this paper we demonstrate that the simple "universal" knowledge of Bisk and Hockenmaier (2013) can be easily applied to induced clusters given a small number of words labeled as noun, verb or other, and that this small amount of knowledge is sufficient to produce labeled syntactic structures from raw text, something that has not yet been proposed in the literature.

we will provide a labeled evaluation of induced CCG parsers against the English (Hockenmaier and Steedman, 2007) and Chinese (Tse, 2013) CCGbanks.


We have also marked two classes with * to draw the reader's attention to a fully noun cluster in English and an other cluster in Chinese which are highly ambiguous.

in both of these cases the frequent words also occur frequently as verbs, providing additional motivation for a better soft-clustering algorithm in future work.


The complexity of AMR parsing affects transition-based methods that rely on features to represent structure, since these often cannot capture the information necessary to predict the correct transition according to the gold standard.

the features defined are not sufficient to "explain" why different actions should preferred by the model.


We note that not all grammar induction strategies make sense for all domains.

we only apply ABSWHOLEPHRASES to GEO and OVERNIGHT.


Interestingly, strategies like ABSWHOLE-PHRASES and CONCAT-2 help the model even though the resulting recombinant examples are generally not in the support of the test distribution.

these recombinant examples are on average longer than those in the actual dataset, which makes them harder for the attention-based model.


This process terminates when no more nonterminals are emitted.

a sequence decoder is used to hierarchically generate the tree structure.


Moreover, dropout is used for regularizing the model (Zaremba et al., 2015).

dropout operators are used between different LSTM layers and for the hidden layers before the softmax classifiers.


We implement the hierarchical tree decoder in a batch mode, so that it can fully utilize GPUs.

as shown in Algorithm 1, every time we pop multiple nonterminals from the queue and decode these nonterminals in one batch.


We have developed a simple procedure for argument identification.

we identify entities and numbers in input questions and replace them with their type names and unique IDs.


JOBS This benchmark dataset contains 640 queries to a database of job listings.

questions are paired with Prolog-style queries.


For each slot type we manually select two trigger seeds from the guideline and then use the Paraphrase Database (PPDB) (Ganitkevitch et al., 2013;Pavlick et al., 2015) to expand these seeds.

we select top-20 lexical paraphrases based on similarity scores as our new triggers for each slot type.


Under-translation: some words are mistakenly untranslated.

in the state-of-the-art attention-based NMT model (Bahdanau et al., 2015), generating a target word heavily depends on the relevant parts of the source sentence, and a source word is involved in generation of all target words.


In NMT, each target word y i is generated from all source words with probability α i,j for source word x j .

the source word x j is involved in generating all target words and the probability of generating target word y i at time step i is α i,j .


Although attention based model has the capability of jointly making alignment and translation, it does not take into consideration translation history.

a source word that has significantly contributed to the generation of target words in the past, should be assigned lower alignment probabilities, which may not be the case in attention based NMT.


We tokenize and truecase the training data, and represent rare words via BPE (Sennrich et al., 2016).


We find that our GBMT system is significantly better than PBMT as measured by all three metrics across all test sets.

the improvements are up to +1.5/+0.5 BLEU, +0.3/+0.2 METEOR, and -0.8/-0.4 TER on ZH-EN and DE-EN, respectively.


The best fit hypothesis will be used as the goal state to generate the action sequence.

the "goodness" of describing command v i with hypothesis h k in environment E i is formulated as follows: is a feature vector capturing multiple aspects of relations between h k , v i , E i and H t as shown in Table 1; and W captures the weight associated with each feature.


Among the 991 data instances, 793 were used for incremental learning (i.e., space induction and hypothesis selector learning).

given a command, if the robot correctly predicts an action sequence 2 , this correct prediction is used to update the hypothesis selector.


The hypothesis spaces and regression based selectors acquired at each run were evaluated on the other 20% (198) testing instances.

for each testing instance, the induced space and the hypothesis selector were applied to identify a desired goal state.


similarity between the ground truth action sequence A (g) and the predicted sequence A (p) .

the edit distance d between two action sequences A (g) and A (p) is first calculated.


Because different action sequences could lead to a same goal state, we also use Jaccard Index to check the overlap between the changed states.

executing the ground truth action sequence A (g) in the initial scene E i results in a final environment E i .


The averaged Jaccard Index (SJI) is reported.

figure 5(a) shows the results of configurations comparing different base hypothesis induction heuristics using regression model based hypothesis selection.


This paper presents an incremental learning approach that represents and acquires semantics of action verbs based on state changes of the environment.

we propose a hierarchical hypothesis space, where each node in the space describes a possible effect on the world from the verb.


Our method is agnostic to the creation of this static resource, which can be obtained either by an unsupervised similarity-based approach, or from a lexical resource.

candidates obtained at this stage do not disambiguate possible multiple senses of t, and are filtered and ordered in the ranking stage by a supervised model.


The surrogate model allows us to cheaply approximate the quality of a set of parameters w without running eval(w), and the acquisition function uses this surrogate to choose a new value of w. However, a trade-off must be made: should the acquisition function move w into a region where the surrogate believes an optimal value will be found, or should it explore regions of the space that reveal more about how eval behaves, perhaps discovering even better values?

acquisition functions balance a tradeoff between exploration-by selecting w in the regions where the uncertainty of the surrogate model is high, and exploitation-by querying the regions where the model prediction is high.


Pointing makes it possible to deliver information and to associate context to a particular object without knowing how to call it.

human infants use pointing as a fundamental communication tool (Tomasello et al., 2007).


Similarly, in text summarization, it is natural to use some words in the original text in the summarized text as well.

to predict a target word at each timestep, our model first determines the source of the word generation, that is, whether to take one from a predefined shortlist or to copy one from the context.


Notable works are (Luong et al., 2015) and (Hermann et al., 2015).

applying to machine translation task, (Luong et al., 2015) learns to point some words in source sentence and copy it to the target sentence, similarly to our method.


In the decoder, we usually use gated recurrent unit (GRU) Chung et al., 2014).

at each time-step t, the softalignment mechanism first computes the relevance weight e tj which determines the contribution of annotation vector h j to the t-th target word.


Like our framework, this covers common systems such as arc-eager and arcstandard, as well as easy-first parsing.

arc construction in easy-first parsing can be seen as an action composed of a number of shifts, an arc action, and a number of un-shift actions.


Let O be an ordered set of operative tokens.

o is the set of tokens that 1) have been moved out of U , and 2) are not themselves reduced.


For each t ∈ M(T ), our generalization requires the following control parameters to be set (default in bold): Whether creating an arc also reduces it.

we will have two parameters, B L and B R , which specify whether LEFT/RIGHT-ARC actions are bottom up.


Arc-Shift: Whether creating an arc also results in SHIFT.

we will have two parameters, S L and S R , which specify whether LEFT/RIGHT-ARC actions are joined with a SHIFT.


Obviously, there are many interesting instantiations for the generalized transition system.

it would be interesting to investigate parsing performance of systems with different active token size and arc-distance.


It is worth noting that the representation imposes some limitations on what MWEs can be represented.

we can only represent overlapping MWEs if they are cases of embedding, that is, cases where one MWE is properly contained in the other.


It is a slightly modified version of Goldberg and Nivre (2013)'s algorithm 3, an approach they called learning with exploration.

as our neural network uses a crossentropy loss, and not the perceptron loss used in , updates are performed even when the prediction is correct.


The conditions for constituent reachability for SR-BIN do not hold any longer for SR-TMP.

constituent reachability depends crucially on the distinction between temporary and nontemporary symbols.


We also noticed some trends at the level of individual phrases.

classification performance and vector accuracy tended to be lower for metaphorical phrases whose nouns are distributionally similar to nouns that tend to participate in literal phrases (e.g., reception is similar to foyer and refreshment in our corpus; warm reception is metaphorical while warm foyer is literal).


(2014) as it is the current state of the art, this is not without its own challenges.

we see the choice of these 4 expression as a somewhat random decision as other expressions could also be selected for the evaluation with similar ratios to those described in Table 1.


What we expect from the scoring function is that large scores indicate high levels of compositionality.

when α(p) is close to 1, the compositional embedding is mainly considered, and vice versa.


Characteristics of light verbs The verbs "take", "make", and "have" are known as light verbs 7 , and the scoring function tends to assign low scores to light verbs.

our  method can recognize that the light verbs are frequently used to form idiomatic (i.e.


The embedding-based methods assume that word embeddings are given in advance and as a post-processing step, learn or simply employ composition functions to compute phrase embeddings.

there is no distinction between compositional and noncompositional phrases.


An author was more likely to use target words metaphorically when the target sentence relates more closely to their own experience of disease and treatment.

metaphors were relatively frequent when people shared their own disease experience (Topic 0, Topic 9) or sympa-  get, chemo, if, they, as, out, can, like, now, she, feel, did, up, know, think, been, good, time, or, when I'm scared of chemo and ct scans because it makes cancer come back and you become more resistance to treatment with drugs like these later.


Both types of features showed significant improvement over the state-of-the-art.

our system made significant gains in solving the problem of overclassification in metaphor detection.


Regarding the compression method proposed in this paper, we notice that LSTM-z,wb and LSTM-z,w yield similar performance to LSTM-z.

lSTM-z,w outperforms lSTM-z in all scenarios of different vocabulary sizes.


We introduce a new methodology for intrinsic evaluation of word representations.

we identify four fundamental criteria based on the characteristics of natural language that pose difficulties to NLP systems; and develop tests that directly show whether or not representations contain the subspaces necessary to satisfy these criteria.


Based on Hypothesis 1, we learn sense-based embeddings from a large data set, using the Continuous Skip-gram model (Mikolov et al., 2013).

we first apply WSD to link each word to its sense in WordNet using a state-of-the-art tool (Zhong and Ng, 2010), and map WordNet sense output to OntoNotes senses.


It can be observed that both LEEV and LEEV+R outperforms the state-of-the-art results achieved by LEM on Dataset I.

lEEV improves upon lEM by over 5% in Fmeasure and with regularization, lEEV-R further improves upon lEEV by over 4%.


Recurrent Neural Networks (RNNs) are neural nets whose computation graphs have cycles.

rNN sequence models are rNNs which map a sequence of inputs x 1 , .


We evaluate instead on the task of directly inferring sequences of words.

instead of defining the Narrative Cloze to be the evaluation of predictions of held-out events, we define the task to be the evaluation of predictions of held-out text; in this setup, predictions need not be mediated by noisy, automatically-extracted events.


This evaluation takes some inspiration from the evaluation of neural encoder-decoder translation models (Sutskever et al., 2014;Bahdanau et al., 2015), which use similar architectures for the task of Machine Translation.

the task we present can be thought of as "translating" a sentence into its successor.


First, the Accuracy metric measures the percentage of a system's most confident guesses that are totally correct.

for each held-out event, a system makes its single most confident guess for that event, and we calculate the total percentage of such guesses which are totally correct.


Note, first, that the text-level models outperform other models on BLEU.

the two-step model e 1 e 2 t 2 (and comparable model with attention) which first predicts successor events and then, as a separate step, expands these events into text, performs quite poorly.


These results provide evidence that, if the Narrative Cloze task is defined to evaluate prediction of held-out text from a document, then sentencelevel RNN language models provide superior performance to RNN models operating at the event level.

linguistic pre-processing does not obviously benefit encoder-decoder models trained to predict succeeding text.


In addition, the polarity relation scores of a pair of words in the sentiment graphs of similar domains are also more similar.

they tend to be both positive or negative in these two domains.


(4) value: y is something valuable, since it does not make sense to violate something worthless.

the writer is presupposing a positive value of y as a fact.


Next we present a factor graph model ( Figure 2) of the connotation frames that parameterize the dynamics between typed relations.

for each verb predicate, 5 the factor graph contains 9 nodes representing the different aspects of the connotation frame.


Results As shown in Table 5, aspect-level and frame-level models consistently outperform all three baselines -MAJORITY, 3-NN, GRAPH PROP in the development set across the different types of relationships.

the improved F1 scores show that these models are able to perform better across all three classes of labels even in the most skewed cases.


In BTDNNs, the linear transformation makes the feasibility of transferring between domains, and the linear data reconstruction manner ensures the distribution consistency between the transferred domain and the desirable domain.

our BTDNNs has one common encoder f c , two decoders g s and g t which can map an example to the source domain and the target domain respectively.


Motivated by the successful application in computer vision (Kan et al., 2015), we construct the architecture of BTDNNs with one encoder f e , and two decoders, g s and g t shown in Figure 1, which can transform an input example to the source domain and the target domain respectively.

2 the encoder f c tries to map an input example x into the latent feature representation z, which is common to both the source and target domains as follows: The decoder g s attempts to map the latent representation to the source domain, and the decoder g t attempts to map the latent representation to the target domain as follows: where s e (•) and s d (•) are the element-wise nonlinear activation function, e.g., sigmoid or tanh function, W c and b c are the parameters for encoder f c , W s and b s are the parameters for decoder g s , W t and b t are the parameters for decoder g t .


Balance theory constraints: Social balance theory (Heider, 1946) models the sentiment dynamics in an interpersonal network.

in balanced states, entities on positive terms have similar opinions towards other entities and those on negative terms have opposing opinions.


However, as commented by Mejer and Crammer (2012), the annotation process is complex, slow, and prone to mistakes when FA is required.

annotators waste a lot of effort on labeling trivial dependencies which can be well handled by current statistical models (Flannery and Mori, 2015).


3.3 Batch Dependency-wise PA In the framework of single dependency-wise PA, we assume that the selection and annotation of dependencies in the same sentence are strictly independent.

annotators may be asked to annotate the head of one selected word after reading and understanding a whole (sometimes partial) sentence, and may be asked to annotate another selected word in the same sentence in next AL iteration.


We also thank Junhui Li and Chunyu Kit for reading our paper and giving many good suggestions.

zhenghua is very grateful to many of his students: Fangli Lu, Qiuyi Yan, and Yue zhang build the annotation system; Jiayuan Chao, Wei Chen, ziwei Fan, Die Hu, Qingrong Xia, and Yue zhang participate in data annotation.


Fortunately, we can use information retrieval techniques to generate a query-specific set of topical documents.

we adopt a language modeling approach to do so (Croft and Lafferty, 2003).


When using language models for retrieval, query expansion involves estimating an alternative to p q .

when each expansion term is associated with a weight, we normalize these weights to derive the expansion language model, p q + .


All retrieval experiments were conducted by performing 10-fold cross-validation across queries.

we cross-validate the number of expansion terms, k ∈ {5, 10, 25, 50, 100, 250, 500}, and interpolation weight, λ ∈ [0, 1].


Given a contextarticle pair, we refer to world knowledge for supporting evidence.

we first apply an entity linking system to detect the entity mentions on both sides and ground them into Wikipedia entries, each of which has its own description page.


In this section, we give detailed win-loss analysis for the models trained with NDCG@5 metric under headlines+lead paragraphs setting.

given two systems with different feature configurations, we compare their performance on each test sample.


In order to obtain word representation through its characters, in the simplest strategy, character vectors are integrated into their word representation using a weight matrix W (L) that is shared across all words with the same length L, followed by a non-linear function g(•).

c i (1 ≤ i ≤ L) are d-dimensional character vector representations respectively, the corresponding word vector w will be d-dimensional as well: where W (L) ∈ R d×Ld and g is a non-linear function as mentioned above.


The main idea is to replace manual discrete features with automatic real-valued features, which are derived automatically from distributed character representations using neural networks.

convolution neural network 1 (Zheng et al., 2013), tensor neural network (Pei et al., 2014), recursive neural network (Chen et al., 2015a) and longshort-term-memory (LSTM) (Chen et al., 2015b) have been used to derive neural feature representations from input word sequences, which are fed into a CRF inference layer.


Since it is challenging to integrate word features to the CRF inference framework of the existing step action buffer( character-based methods, we take inspiration from word-based discrete segmentation instead.

we follow Zhang and Clark (2007), using the transition-based framework to decode a sentence from left-to-right incrementally, scoring partially segmented results using both character-level and word-level features.


For calculating r c , we further use a bi-directional extension of the original LSTM structure.

the base LSTM is applied to the input character sequence both from left to right and from right to left, leading to two hidden node sequences For the current character c 0 , h cL 0 and h cR 0 are concatenated to form the final vector r c .


Our model can be extended by integrating the baseline discrete features into the feature layer.

where f sep and f app represent the baseline sparse vector for SEP and APP features, respectively, and ⊕ denotes the vector concatenation operation.


For thorough comparison, we implement discrete, neural and combined character-based models as well.

7 the character-based discrete model is a CRF tagging model using character unigrams, bigrams, trigrams and tag transitions (Tseng et al., 2005), and the character-based neural model exploits a bi-directional LSTM layer to model character sequences 8 and a CRF layer for  201495.2 97.2 Zhang et al.


Their network operates on a hierarchy similar to our own, providing further evidence of the promise of hierarchical perspectives.

the HABCNN processes text at the sentence level and the snippet level, where the latter combines adjacent sentences (as we do through an n-gram input).


Below, we use subscripts to index elements of sequences, like word vectors, and superscripts to indicate whether elements come from the text, question, or answer.

we use the subscripts k, m, n, p to index sequences from the text, question, answer, and hypothesis, respectively, and superscripts t, q, a, h. We depict the model schematically in Figure 1.


However, we handle the linearization in data preprocessing so that the model sees only reordered word-vector inputs.

we run the Stanford Dependency Parser on each text sentence to build a dependency graph.


These were collected on Mechanical Turk.

for each true hypothesis in the training set and for each sentence in the Barron's study guide, we found the top 8 results from Solr and considered these to be candidate entailments.


2) Importance-Weighting (IW): Importance weighting is a common technique in active learning literature (Tong and Koller, 2002;Beygelzimer et al., 2009;Beygelzimer et al., 2010), which mitigates the problem that if we query questions actively instead of selecting them uniformly at random, the training (and test) question sets are no longer independent and identically distributed (i.i.d.).

the training set will have a sample selection bias that can impair prediction performance.


For example, while the task of supporting passage selection for factoid QA may be largely cast as a textual entailment problem, what makes an answer better than another in the real world for non-factoid QA often depends on many factors.

different from many other pairmatching NLP tasks, the linguistic similarities between questions and answers may or may not be indicative for our task.


Prior to the average or mean pooling, each biLSTM output vector is multiplied by a softmax weight, which is determined by the question representation from biL-STM.

given the output vector of biL-STM on the answer side at time step t, h a (t), and the question representation, o q , the updated vector h a (t) for each answer token are formulated below.


4) We also observe that because of the smaller data size, we need a decayed learning rate λ in order to stablize the models' training.

we set the initial λ 0 = 1.1, and decrease it for each epoch T > 1 as λ T = λ 0 /T .


In this paper we explore tables as semistructured knowledge for multiple-choice question (MCQ) answering.

we focus on tables that represent general knowledge facts, with cells that contain free-form text (Secton 3 details the nature and semantics of these tables).


To focus and facilitate our work on QA we select a collection of tables that were specifically designed for the task.


Finding the correct answer amounts to finding a cell in the table that is most relevant to a candidate QA pair.

a relevant cell should confirm the assertion made by a particular QA pair.


To create the training data for sentence extraction, we reverse approximated the gold standard label of each document sentence given the summary based on their semantic correspondence (Woodsend and Lapata, 2010).

we designed a rulebased system that determines whether a document sentence matches a highlight and should be labeled with 1 (must be in the summary), and 0 otherwise.


For out-of-vocabulary (OOV) words, we try to find a semantically equivalent replacement present in the news article.

we check if a neighbor, represented by pre-trained 4 embeddings, is in the original document and therefore constitutes a valid substitution.


As our model enjoys the advantage of generation by extraction, we can force the model to inspect the context surrounding an entity and its relative position in the sentence in order to discover extractive patterns, placing less emphasis on the meaning representation of the entity itself.

we perform named entity recognition with the package provided by Hermann et al.


We address this by reranking the positively labeled sentences with the probability scores obtained from the softmax layer (rather than the label itself).

we are more interested in is the relative ranking of each sentence rather than their exact scores.


The results of our human evaluation study are shown in Table 2.

we show, proportionally, how often our participants ranked each system 1st, 2nd, and so on.


This allowed us to perform Analysis of Variance (ANOVA) which revealed a reliable effect of system type.

post-hoc Tukey tests showed that NN-SE and ILP are significantly (p < 0.01) better than LEAD, NN-WE, and NN-ABS but do not differ significantly from each other or the human goldstandard.


This is because, each dimensionality of this distribution denotes the probability of the concept (topic or category) this sentence is respect to.

the concept distribution is a solid semantic representation of the sentence.


The results show that the fixed effects of global position are significant for both measures in both corpora: Entropy in Switchboard, β = 4.2 × 10 −3 , p < 0.001; normalized entropy in Switchboard, β = 5.9 × 10 −4 , p < 0.001; entropy in BNC, β = 1.5 × 10 −2 , p < 0.001; normalized entropy in BNC, β = 1.4 × 10 −3 , p < 0.001).

since the curves of Switchboard seem flat after a boost in the early phase (between 0 to 5 in global position), we fit extra models to examine whether the entropy increase for global positions larger than 10 is significant.


Our main contribution is that we find new entropy change patterns in dialogues that are different from those in written text.

when distinguishing the speakers' roles by topic initiator vs. responder, we see that the initiator's entropy decreases (or remain steady) whilst the responder's increases within a topic episode, and together they form a convergence pattern.


answers have to be selected based on multiple objective functions since motives of all interlocutors i ∈ I shall be satisfied; f i : Ω → R. For capturing the aspect of equal motive satisfaction, the potential of answers has to be represented absolutely and relatively.

the performance of an answer in satisfying motives of an interlocutor i ∈ I is combined with its performance in satisfying motives of counterparts −i ∈ I; max{f i,-i (x)|x ∈ Ω}.


These "information flows" are real-valued vectors describing how words and phrases interact with each other, for example, a word (or phrase) in one sentence can modify the meaning of a word (or phrase) in another sentence through such "information flows".

given two sentences s 1 and s 2 , for every word x t in s 1 , we introduce a "candidate interaction state" for every word x τ in s 2 .


Second, we propose a new mechanism to model the interactions between s 1 and s 2 by allowing information to flow between them.

word t in s 1 may be potentially influenced by all words in s 2 in some degree.


In this paper, we present and evaluate models of referring expression generation that mimic this human non-determinacy and show that this enables us to generate varied references in texts, which, in terms of coherence and comprehensibility, did not yield significant differences from human-produced references according to human judges.

in this study we focus on the choice of referential form, which is the first decision to be made by referring expression generation models (Reiter and Dale, 2000) and which determines whether a reference takes the form of a proper name, a pronoun, a definite description, etc.


We also re-annoted the GREC-2.0 corpus to make it compatible with the VaREG corpus.

we added features for status and recency to the GREC-2.0 corpus and made the terminology consistent beween the two corpora 1 .


, w N , and w <n represents all the tokens generated before token w n .

w N represents the question mark symbol '?'.


(2015), along with our novel multi-focal attention model to enforce coherence, leading to significant performance improvements on CoNLL 2003 (Hoffart et al., 2011) and TAC KBP 2010-2012 tasks (Ji et al., 2010;Ji et al., 2011;Mayfield et al., 2012).

we achieve a 20% relative reduction in error from Chisholm and Hachey (2015) on CoNLL, and a 22% error reduction from Cucerzan (2012) on TAC 2012.


As in other structured prediction work, we will assume that the scores are functions of the features of the input x and labels.

denote a set of singleton features for mention i and label y i by φ s i (x, y i ) ∈ R ns and a set of pairwise features for mentions i and j and their labels by φ p ij (x, y i , y j ) ∈ R np .


This is done by applying a pooling operation.

it concatenates the results of max-pooling and average-pooling, which we found to be slightly more effective than using either one alone: Rather than training a cluster-ranking model from scratch, we first train a mention-ranking model that assigns each mention its highest scoring candidate antecedent.


We view this procedure as a sequential decision process where at each step the algorithm observes the current state x and performs some action u.

we define a state x = (C, m) to consist of C = {c 1 , c 2 , ...}, the set of existing coreference clusters, and m, the current mention being considered.


Each document has K candidate topics, arranged in the matrix form , referred to as the topic matrix.

we fix t i1 = 0, referring to it as the null topic.


We follow this assumption, and propose to incorporate the topic of w c in a way like a latent word.

in addition to the context words, the corresponding embedding t ik is included as a new log-bilinear term that influences the distribution of w c .


To get around this difficulty, we learn word embeddings and topic embeddings separately.

the learning process is divided into two stages: 1.


Another challenge is related to the language gap issue.

the words of reader comments in different languages are composed of different terms in their corresponding languages.


To capture common semantics of multilingual news reader comments, we design two auxiliary distri- butions η e , with dimension Λ e , and η c , with dimension Λ c , to help the generation of ϕ e k and ϕ c k .

we generate η e and η c from the Dirichlet prior distributions Dir(β •1 |Λ e | ) and Dir(β •1 |Λ c | ) respectively, where 1 D denotes a D-dimensional vector whose components are 1.


We propose a new joint model that makes fuller use of the rich link structure within a document network.

our model embeds the weighted stochastic block model (Aicher et al., 2014, WSBM) to identify blocks in which documents are densely connected.


Thus we propose an LDA with block priors, hence BP-LDA, as shown in Figure 4, which has the following generative process: Unlike conventional LDA, which uses an uninformative topic prior, BP-LDA puts a Dirichlet prior π on each block to capture the block's topic distribution and use it as an informative prior when drawing each document's topic distribution.

a document's topic distributioni.e., what the document is about-is not just informed by the words present in the document but the broader context of its network neighborhood.


Algorithm 1 Sampling Process 1: Set λ = 1 and initialize topic assignments 2: for m = 1 to M do 3: Optimize η, τ , and ρ using L-BFGS 4: end for 13: end for Link set A is primarily used to find blocks, so it treats all links deterministically.

the links observed in the input are considered explicit positive links, while the unobserved links are considered explicit negative links, in contrast to the implicit links in B.


We use an automatic coherence detection method (Lau et al., 2014) to evaluate topic quality.

for each topic, we pick out the top n words and compute the average association score of each pair of words, based on the held-out documents in development and test sets.


We make use of the contextual information hidden between sentences in order to identify the authorship of each sentence in a document.

the authorships of the "previous" and "subsequent" sentences have relationships with the authorship of the current sentence.


Both CCG and MT-inspired approaches assume a fairly strong correspondence between the words in the natural language request and the concepts in the meaning representation.

most words in the description should correspond to some concept in the code, and most concepts in the code should correspond to some word in the description.


Conditioning on previous productions helps predicting the next one because it captures the conditional dependencies between the productions of the derivation tree, an improvement over prior work (Quirk et al., 2015).

we can model which combinations of triggers and actions are more compatible, both function and channel.


The scheme was introduced to prevent reviewers from overlooking annotation issues due to passive approval.

an additional # sign was added at the seventh column of each token's annotation.


#SENT=I am <ns type="IV"><i>writting</i> <c>writing</c></ns>... PRP  3  nsubj  2  am  AUX  VBP  3  aux  3 writting VERB VBG 0 root ...

ill formed adjectives that have a plural suffix receive a standard adjectival POS tag.


An additional factor which negatively affects performance in this regime are systematic differences in the EWT annotation of possessive pronouns, expletives and names compared to the UD guidelines, which are utilized in the TLE.

the EWT annotates possessive pronoun UPOS as PRON rather than DET, which leads the UPOS results in this setup to be lower than the PTB POS results.


This would in turn mean that neuro-images of readers carry information about the grammatical structure of what they read.

neuro-imaging provides a partial, noisy annotation of the data with respect to morphosyntactic category.


In this paper, we propose a sentence rewriting based semantic parsing method, which can effectively resolve the mismatch problem by rewriting a sentence into a new form which has the same structure with its target logical form.

we propose two sentence-rewriting methods for two common types of mismatch: a dictionary-based method for 1-N mismatch and a template-based method for N-1 mismatch.


By contrast, by rewriting "daughter" into "female child", our method can resolve this vocabulary mismatch.

we identify two common types of vocabulary mismatch in semantic parsing: 1.


In this section, we describe our sentence rewriting algorithm for  solving the mismatch problem.

we solve the 1-N mismatch problem by dictionarybased rewriting and solve the N-1 mismatch problem by template-based rewriting.


For the dictionary used in rewriting, this paper uses Wiktionary.

given a word, we use its "Translations" part in the Wiktionary as its explanation.


To resolve the N-1 mismatch problem, we propose a template rewriting algorithm, which can rewrite a complicated expression into its simpler form.

we rewrite sentences based on a set of paraphrase template pairs P = {(t i1 , t i2 )|i = 1, 2, ..., n}, where each template t  is a sentence with an argument slot $y, and t i1 and t i2 are paraphrases.


(2014) to automatically collect paraphrase template pairs.

we use the WikiAnswers paraphrase corpus (Fader et al., 2013), which contains 23 million question-clusters, and all ques-  tions in the same cluster express the same meaning.


However, the reward provides a way to modulate the magnitude of the updates.

after each update, our model results in making the derivation, which has the highest reward, to get a bigger score.


To better understand our system, we conduct error analysis on the parse results.

we randomly choose 100 questions which are not correctly answered by our system.


Second, we train our deep network in a supervised manner.

1 motivated by recent successes of applying the mention-ranking model (Denis and Baldridge, 2008) to entity coreference resolution (e.g., Chang et al.


6 We employ online learning to train the network, with one training example in a mini-batch.

we update the weights after processing each training example based on the correct matching scores of the training example (which is 1 for the correct antecedent and 0 otherwise) and the network's predicted matching scores.


to Λ, we train the model using stochastic gradient descent.

the model parameters Λ are updated according to the following update rule: where α is the learning rate, and Λ t and Λ t−1 are model parameters at the tth iteration and the (t − 1)th iteration respectively.


Each test instance corresponds to an AZP z and four of its candidate antecedents.

the four candidate antecedents with the highest salience scores will be chosen.


We believe that this is the beginning, not the end, of applying deep neural networks for AZP resolution.

there is a lot of room for improvements, which may involve incorporating more sophisticated features and improving the design of the network (e.g., the dimensionality of the intermediate representations, the number of hidden layers, the objective function), for instance.


To investigate the contribution of each of these two feature groups, we conduct ablation experiments.

in each ablation experiment, we retrain the network using only one group of features.


In this paper, we address this problem of prompt adaptation using multi-task learning.

we treat each prompt as a different task and introduce a constrained preference-ranking approach that can learn from multiple tasks even when the scoring scale and marking criteria are different across tasks.


As the weight vector w is optimized for pairwise ranking, a further step is needed to use the ranking model for predicting a score.

for each of the n vectors in our training set, a real-scalar value is assigned according to the dotproduct of the weight vector and the training instance (i.e.


The presence of cohesive devices are used as features.


The prompts elicit responses of different genres and of different lengths.

it is important to note that the prompts have different scoring scales and are associated with different grade levels (7-10).


We use the same features outlined in Section 3.3 to encode feature vectors for our learning approaches.

we study three learning approaches denoted and summarised as follows: TAP: which uses the TAP rank algorithm with input vectors x i of dimensionality F .


We now report the results of an experiment that uses varying amounts of target-task training data.

we use all source tasks and initially a small sample of task-specific data for each task (every 128 th target essay) and measure the performance of Tgt-TAP and the All-* models.


Figure 2 shows the performance of Tgt-TAP and the All-* models as target-task data increases.

figure 2 shows that All-MTL-cTAP consistently outperforms all approaches in terms of agreement (QW-κ) and is particularly superior when there is very little target-task training data.


We  now study the performance of the approaches as the number of source tasks changes.

we limit the number of target task training instances to 25 and cumulatively add entire source task data in the order in which they occur in Table 2, starting with the source task appearing directly after the target task.


However, it remains a challenge to answer factoid questions raised in natural language due to numerous expressions of one question.

we focus on the most common questions-ones that can be answered with a single fact in the knowledge base.


For example, "Harry Potter" in "Who created the character Harry Potter" is more likely than "the character", "character Harry", etc.

our method employs a deep network to identify such focus segments in a question.


Relation network In this work, the probability of relations given a question, p(r|q), is modeled by the following network where the relation scoring function v(r, q) measures the similarity between the question and the E(r) is the trainable embedding of the relation (randomly initialized in this work) and f (q) computes the semantic question embedding.

the question q is represented as a sequence of tokens (potentially with unknown ones).


The pretrained embedding approach utilizes unsupervised method to train entity embedings.

we employ the TransE (Bordes et al., 2013), which trains the embedings of entities and relations by enforcing E(s) + E(r) = E(o) for every observed triple (s, r, o) ∈ K. As there exists other improved variants (Gu et al., 2015), TransE scales the best when KB size grows.


In this work, we utilize the hinge loss with negative samples as the training objective.

the loss function w.r.t θ r has the form where r (j) is one of the M r negative samples (i.e.


Similarly, the loss function w.r.t θ s takes the form Despite the negative sample based approximation, there is another practical difficulty when type vector is used as the subject representation.

computing the value of u(s (j) , r (i) , q (i) ) requires to query the KB for all types of each negative sample s (j) .


Hence, we establish a CRF baseline which uses traditional features as input.

the model is trained with Stanford CRF-NER toolkit 7 on the same reversely linked labeling data (section 6.1).


As AO-selecting verbs can change the meaning of a text in important ways, text understanding systems should be sensitive to them.

classifications of AO-selecting verbs according to semantic criteria are important knowledge sources for a wide range of NLP applications, such as event tagging (Saurí et al., 2005), commited belief tagging (Prabhakaran et al., 2010), reported speech tagging (Krestel et al., 2008), the detection of uncertainty (Szarvas et al., 2012) and future-oriented content (Eckle-Kohler et al., 2008), textual entailment (Saurí and Pustejovsky, 2007;Lotan et al., 2013), or determining the degree of factuality of a given text (Saurí and Pustejovsky, 2012;de Marneffe et al., 2012).


Our first set of experiments aims at refining the initial semantic characterization of the classes by linking them to independently constructed semantic classifications at the word sense level.

we consider three different semantic classifications from computational lexicons, which have been created by linguistic experts: (i) the so-called semantic fields in GermaNet, grouping verb senses into 15 coarse classes, such as perception, emotion, (ii) the verb classes given in VerbNet, and (iii) the Frame-semantic frames in FrameNet.


NMT is an end-to-end approach to data-driven machine translation (Kalchbrenner and Blunsom, 2013;Sutskever et al., 2014;Bahdanau et al., 2015).

the NMT models directly estimate the conditional probability p(y|x) given a large collection of source and target sentence pairs (x, y).


Intuitively, personalized model adaptations can be considered as a set of related tasks in individual users, which contribute to a shared global model adaptation.

we assume the distinct ways in which users express their opinions can be characterized by a linear classifier's parameters, i.e., the weights of textual features.


Various feature grouping methods have been explored in (Wang et al., 2013).

we define g(i) → j as a feature grouping method, which maps feature i in {1, 2, .


To simulate the real-world situation where user reviews arrive sequentially and asynchronously, we ordered all reviews chronologically and accessed them one at a time for online model update.

we utilized stochastic gradient descent for this online optimization (Kiwiel, 2001).


The shift in the shared global model reflects changes in social norms, and the discrepancy between the shared global model and personalized models indicates the variances of individuals' opinions.

we calculated Euclidean distance between global model w s and base model w 0 and that between individualized model w u and shared global model w s during online model updating.


This problem is made more complicated by the need to trace the ideas back to those that need to be learned first (e.g., before you can learn about Markov logic networks, you should understand first-order logic and probability).

a crucial question when learning a new subject is "What do I need to know before I start reading about this?"


For this, we use cross entropy and joint entropy: Cross entropy measures the difference between two distributions.

the cross entropy for the distributions X and Y over a given set is defined as: where H(X) is the entropy of X, and D KL (X||Y ) is the Kullback-Leibler divergence of an estimated distribution Y from true distribution X.


We compare against a method for doing so to see how close identifying hierarchical relations comes to our goal of identifying concept dependency relations.

we perform agglomerative clustering over the topic-topic co-occurrence graph G co with weights defined in Eq.


This consists of articles published in a variety of journals, conferences, and workshops related to computational linguistics.

we use a modified copy of the plain text distributed for the ACL Anthology Network (AAN), release 2013 (Radev et al., 2013), which includes 23,261 documents from 1965 to 2013.


Small.02: A subset of Small.01, constructed by randomly removing problems from Small.01 so that each template contains similar number of problems as in Alg514.

small.02 and Alg514 have similar problem distribution among templates.


The main body of each article forms a context, while the cloze style question is formed from one of short highlight sentences, appearing at the top of each article page.

the question is created by replacing a named entity from the summary sentence (e.g.


However, there are approaches such as discriminative reranking (Collins and Koo, 2005) to rerank the output of an existing system that offer a means for further improving the performance of our SLL+RLL model.

the oracle performance using the top-10 Viterbi paths from our SLL+RLL model has an F-score of 82.11 for opinion expressions, 76.77 for targets and 78.10 for holders.


An important difference is that we design heuristic rules not for fully-connected parse tree but for stack of parse trees derived by left-corner parsing.

the extend version of c-command relation plays an important role on our heuristic rules.


We construct a supervised training criterion by having our network predict sentences occurring next to each other in the training data.

for a pair of sentences (s i , s j ), we define a probability p(s i , s j ) that reflects how likely it is for the sentences to be adjacent to one another in the training data.


We answer the question in the affirmative provided our model family is wellspecified.

we introduce the unanimity principle: only predict when all models consistent with the training data predict the same output.


For each safe input x ∈ F, we haveM (x) = M * (x).


In this paper, we investigate whether and to what extent the lexico-semantic models of the native language (L1) are transferred to the second language (L2).

we focus on the problem of lexical choice and investigate it in the context of three typologically diverse languages: Russian, Spanish and English.


Although the addition of the L1 lexico-semantic features does not have a significant effect on the accuracy and precision, the system achieves an absolute improvement in recall of up to 12.71% (on RU dobj ).

the system that uses L1 lexico-semantic features is able to find more errors in the data originating with a set of different L1s.


We now turn our attention to the relation between the native language (L1) of the user, and that of the learner corpora used for training the system.

we wish to measure the gain, if any, in matching the L1 of the user with the L1 of the learner corpora.


Although DeepWalk is effective on learning embeddings of the topological structure, when dealing with a network classification task, it lacks a mechanism to optimize the objective of the target task and thus often leads to suboptimal embeddings.

for our focus of relational network classification, we would like the embeddings to be both representing the topological structure of the network actors and discriminative in predicting the class labels of actors.


We use the Skip-gram (Mikolov et al., 2013a) strategy in Word2Vec, which uses the central word in a sliding window with radius R to predict other words in the window and make local optimizations.

let ω = rw(α) denote the full walk streams obtained from truncated random walks in Section 3.1.1.


We simultaneously optimize two objectives in Section 3.1 and 3.2.

let where η is a key parameter that balances the weights of the two objectives.


Besides, we examine the neighborhood structure of the vertices.

we check the top-K nearest vertices for each vertex in the embedded social space and calculate how many of the vertex pairs have edges between them in the observed network.


However, the lexical and grammatical variability of the language used in social media poses a key challenge for extracting information (Baldwin et al., 2013;O'Connor et al., 2014).

the frequent use of informal language, non-standard grammar and abbreviation forms, as well as typos in social media messages has to be taken into account by effective information extraction systems.


We propose to learn and exploit the semantic similarity between texts from social media messages and medical concepts using deep neural networks.

we investigate the use of techniques from two families of deep neural networks, i.e.


We tackle the concept normalisation task in a different manner.

we use deep neural networks to capture the similarity and/or dependency between terms and effectively represent a given social media message in a low dimensional vector representation, before mapping it to a medical concept.