While both systems behave similarly in extracting glosses and learning new patterns by exploiting html tags, they are substantially different in how acquired glosses are ranked.

the former exploits the bag-of-words representation of each extracted gloss and its intersection with the domain terminology, while the latter leverages Probabilistic Topic Models (PTM) by estimating the probability of words and term/gloss pairs to be pertinent to the domain.


We exploit the potential of the Conditional Random Fields 5 algorithm (Lafferty et al., 2001) to encode prior and posterior contextual information of a given element in a sequence (in our case, a word in a sentence).


Moreover, we observed a minor improvement after incorporating the label-switching heuristics in both corpora.

for the MSR-NLP corpus the improvement was from the aforementioned F=76.34 to F=77.46, while in the W00 dataset, it improved from F=70.72 to F=71.85.


This intuition can be implemented in a number of ways in practice; in this work, we focus on models that represent word meaning as a point in a metric space (Widdows, 2005;Sahlgren, 2006;Turney and Pantel, 2010;Clark, 2015).

one member of this family that has been particularly influential recently is the skip-gram learning algorithm (Mikolov et al., 2013a), which is derived from the log-bilinear language model by Mnih and Hinton (2007).


In this paper, we enrich the multi-sense skipgram model (Neelakantan et al., 2014) by introducing external signals, which are implemented as additional context features during training.

we use a parallel corpus, where the foreign-language words work as a source of external information that helps the algorithm form more distinct clusters.


It seems straightforward to extend our work to a setting where other types of features are used, and we would like to explore this area further.

we would like to integrate multimodal input , for instance with information extracted from images.


This phenomenon is more abundant in more informal settings -such as in conversational spoken language and in social media text -and of course also more common in areas of the world where people are naturally bi-or multilingual, that is, in regions where languages change over short geospatial distances and people generally have at least a basic knowledge of the neighbouring languages.

india is home to several hundred languages, with language diversity and dialectal changes instigating frequent code-mixing.


As this is pioneering work for this combination of language and genre, we created suitable resources for training and evaluating a system for sentiment analysis of Macedonian tweets.

we developed a corpus of tweets annotated with tweet-level sentiment polarity (positive, negative, and neutral), as well as with phrase-level sentiment, which we made freely available for research purposes.


This mainly affects the construction of the sentiment lexicons and the reason for this is the distribution of emoticons, hashtags and sentiment words.

if we want to use hashstags or emoticons as seeds to construct sentiment lexicons, we find that very few tweet messages have emoticons or hashtags.


As the small amount of available training data limits the development of robust systems to automatically detect comparisons, this paper investigates how to use semi-supervised strategies to expand a small set of labeled sentences.

we use structural alignment, a method that starts out from a seed set of manually annotated data and finds similar unlabeled sentences to which the labels can be projected.


The recall, precision, and F-measure were evaluated based on chunks.

the NEs with a BIE tag are correct only if all the tags from the initial B to the last E in the chunk are correct.


Note that the distribution of the signal is proportional to the logarithm of the weight (and not proportional to the weight itself).

we can describe the algorithm informally as follows: Init: terms T of network are associated with a pair of values (CA, TA), current activation and total activation.


Threshold accuracy sets a similarity threshold and determines how many of the n-highest ranking word pairs in a given measurement are also in the top-n pairs of the gold standard.

this evaluation determines whether the right wordpairs would end up above the threshold of being similar.


Together with the mixed outcome of the threshold-evaluation, this shows that corpus-based approaches have good potential to be used when similarity needs to be detected.

when taxonomy-based approaches run into coverage issues, they may be the preferred choice.


The standard training process is to find the parameter values to maximize the likelihood.

it is to minimize the negative log-likelihood of the observed data as defined bŷ This can be completed by using the expectationmaximization (EM) algorithm.


Consequently, we hold that a large-scale knowledge base should play a key role in the semantic parsing.

it might be quite favorable in training such parser and resolving these syntactic ambiguities.


Combining these two elements allows us to train a more accurate semantic parser.

it also contains a factored CCG lexicon from the closed-domain GeoQuery and ATIS.


Thereby, the parse tree y must agree with a dependency parse of the same sentence x i .

it must guarantee the weak supervision variable µ value to be 1.


The syntactic constraint function penalizes ungrammatical parses by encouraging the semantic parser to produce parse trees that agree with a dependency parse of the same sentence (Krishnamurthy and Mitchell, 2012; Krishnamurthy and Mitchell, 2013;Krishnamurthy and Mitchell, 2015).

the syntactic constraint requires the predicate-argument structure of the CCG parse to agree with the predicate-argument structure of the dependency parse.


Step 2: The second step does a stochastic gradient descent update on the parameter θ used in the parsing model.

this update first computes the weak supervision variable µ i value for each parse tree y through the syntactic constraint function Φ and then judges whether the punishment need to be done.


In addition, Table 1, again extracted from (Bentivogli et al., 2009), suggests that the de-gree of word overlap among positive and negative pairs might be a key to understand the complexity of a text-to-text inference task, and, as a consequence, a key to interpret the system's performance.

we can notice that the word overlap for the "Yes" cases and the "Contradiction" cases in the RTE-4 data-set is very similar, and even higher for the RTE-5 data-set.


We are interested to capture correlations between the use of lexical resources (both single resources and in combination) and the performance of inference algorithms.

the goal is to predict the behavior of an entailment algorithm given the characteristics of both the resource and the data-set.


Semantic related tasks have become a noticed trend in Natural Language Processing (NLP) community.

the Semantic Textual Similarity (STS) task has captured a huge attention in the NLP community despite being recently introduced since SemEval 2012 and continuing in Se-mEval 2013, 2014 and 2015 (Agirre et al., 2012;Agirre et al., 2013;Agirre et al., 2014;Agirre et al., 2015).


We use a linear Support Vector Machine to perform multi-class classification in our experiments.

we use the LIBLINEAR 3 package (Fan et al., 2008) which has been shown to be efficient for text classification problems such as this.


This probability distribution does not model the relationships between combinations of feature functions and classes.

if a combination of two or more feature functions is a good class indicator, the CRFs will not be able to discover the relationship.


(why we tended to prefer simpler models over more complex ones.


Note that we perform the classification at the user level, i.e., based on user activity history, from which we extract statistics summarizing the user activity.

for each user, we count the number of comments posted, the number of days in the forum, the number of days with at least one comment, and the number of publications commented on.


Our experimental results show significant improvements when using multiplicative combinations of several sentence similarity measures.

the results of MULTSUM surpasses that of the original submodular optimization method.


Besides the aforementioned work, interested readers are referred to an extensive survey (Nenkova and McKeown, 2012).

they discuss different approaches to sentence representation, scoring and summary selection and their effects on the performance of a summarization system.


Our experimental evaluation shows that the summaries produced by MULTSUM outperforms state-of-theart systems on standard benchmark datasets.

it beats the original submodublar optimization approach on all three variants of ROUGE scores.


The drawback of this method is that phrases are built in an iterative process starting from single words and joining others to them until the expected size of phrases is reached.

at the end of the first iteration, sequences of two words are built, the following iteration produces phrase of three words and so on until the stop-criteria is reached.


By considering all word sequences as phrases, the dependency inside a phrase, is preserved.

word-based language model is a special case of phrase-based language model if only single word phrases are considered.


In this paper, we explore the task of classifying the attributes present in a natural language query into different SQL clauses in a SQL query.

we investigate the effectiveness of various features and Conditional Random Fields for this task.


Sometimes, we may not have complete attribute information to form a SQL query.

there can be attributes other than explicit attributes and implicit attributes in a SQL query.


Instead of generating a term by document matrix, where each term is tf-idf (product of term frequency and inverse document frequency), we compute a term by document matrix M D ∈ m×l , where each term in the matrix is the product of the projection of a word onto a target VNC and inverse document frequency.

the term frequency (tf) of a word is replaced by the projection (inner product) of the word onto σ vn (1).


When this miscollocation is entered, e.g., into the MUST collocation checker 1 for verification, the program suggests (in this order) pass exam, sit exam, take exam, fail exam, and do exam as possible corrections.

the checker offers all possible <verb> + exam collocations found in a reference corpus or dictionary.


We use the following acoustic model setup:  iVectors are used for speaker adaptation (Miao et al., 2014).

for each speaker model parameters are changed so the better fit is obtained.


The interval of a burst at level j in the hierarchy of bursts is the maximal interval during which the optimal state sequence is in state j or higher, i.e., k > j, thus forming a hierarchical organization of burst intervals.

a word considered bursty on a time interval [a, b] with a burstiness level of i is simultaneously considered as busty at a level i−1 on an interval This hierarchy is illustrated in Figure 4 for one word: The word occurs with a burstiness level of 1 on the first utterances, with an important amount of occurrences at the very beginning yielding a short interval at level 2 included in the interval at level 1.


These additional resources are: • Inferred hypernymy relations • Syntactic relations from the golden corpus • Extended syntactic relations • Domain relations from WordNet The phrase "inferred hypernymy relations" means the transitive closure of the hypernymy relation type.

if A is a hyponym of B and B is a hyponym of C, it is inferred that A is a hyponym of C. This type of inference has been done for all synset IDs that participate in hypernymy relations in the WordNet hierarchy and are found in the Bulgarian dictionary.


This procedure improves the performance of the HTR system by 6.2%.

these results emphasize that the out-of-domain data contains useful information.


One feature, whose use is relatively uncommon in the metrics used to assess readability for other populations, is the occurrence of fewer words before the main verb in a sentence, which has proven effective due to the decreased working memory capacity of people with autism.

the closer that main verbs are to the starts of sentences, the more comprehensible the text is for readers with autism.


In this paper, we focus on the task of Arabic persons" names recognition.

we introduce an approach to extract Arabic persons" names from tweets without employing any morphological analysis or languagedependent features.


With the help of table location and the location of individual tokens in the document, only those tokens which are within the given table location are collected by comparing their corresponding coordinate values.

all the tokens, whose x coordinate is between startX and endX and whose y coordinate is between startY and endY are collected as table text tokens.


To tackle the issues described above, we apply a new round-tripping approach that incorporates dual learning (He et al., 2016) for automatic learning from unlabeled data, but transcends this prior work through effective leveraging of monolingual text.

the round-tripping approach takes advantage of the bootstrapping methods including self-training and co-training.


By utilizing the roundtripping approach, the monolingual data play a similar role to the bilingual data, effectively reducing the requirement for parallel data.

each model provides guidance to the other throughout the learning process.


Scoring of the phrase table generated by Moses is achieved through a LSTM encoder-decoder, and the result is then evaluated in an English-to-Spanish translation task.

the model is trained to learn the translation probabilities between English phrases and their corresponding Spanish ones.


To compose the model, both the encoder and decoder are implemented employing RNNs, i.e., the encoder converts source words into a sequence of vectors, and the decoder generates target words one-by-one based on the conditional probability shown in Equation (3).

the encoder takes a sequence of source words as inputs and returns forward hidden vectors − → h j (1 ≤ j ≤ J) of the forward-RNN: Similarly, backward hidden vectors ← − h j (1 ≤ j ≤ J) of the backward-RNN are obtained, in reverse order.


Using this approach it is possible to show how the similarity between two targets changes over time.

for each time point we build the BoW of each concept and then we compute the similarity between the BoWs.


The scientific community works intensively on data acquisition for the ontology building.

the NeOn project 1 has been set up to provide a methodology for the ontology engineering by integrating preexisting knowledge resources into an ontology building process.


To address the ontology accuracy, completeness, conciseness, efficiency, consistency, and other features (Raad and Cruz, 2015), a combination of methods is needed.

gold standard ontology, specific tasks and corpora may be used for evaluation.


The Slovene output contained many nontranslated words.

this task greatly suffers when using the word frequency regularizer, with an error increase of about 10 TER points from English to Slovene.


In this work, we address the evaluation of distributional semantic models trained on smaller, domain-specific texts, particularly philosophical text.

we inspect the behaviour of models using a pretrained background space in learning.


Although we limited our samples to 10 sentences per term, not every core Quine term is used frequently enough to have 5 samples with the maximum size of 10 sentences.

in the full Quine dataset, 68.6% of terms reach the maximum size, while in the Word & Object dataset, only 32.1% of terms reach it.


In future work, it would be valuable to evaluate our approach on texts from a wider array of domains and with different sources of incoherence.

examining raw texts identified by humans as lacking coherence could be performed, to determine how well our model correlates with human judgment.


The distinction these rare labels offered was not necessary.

the two labels that denoted the author said something positive about a citation/field were merged into the appropriate cited/field evaluation categories.


After collecting the maximum possible number of tweets for each user, we filtered them.

we removed the non-Arabic tweets, retweets, and short tweets that contain less than 3 words.


Error Model Summary (Pt) Summary (En) corpus homem na cerca man by fence small sBase homem idoso elderly man sPers homem na cerca man by fence corpus pessoas pedindo ajuda people asking for help moderate sBase pessoas esperando people waiting sPers pessoas aguardam atendimento people waiting for help corpus menino com um balde de terra boy with a bucket full of soil large sBase crianca com balde child with bucket sPers crianca com balde de terra child with bucket full of soil strategies for text summarization.

we notice that the increasing availability of text corpora labelled with author demographics in general (e.g., gender, age, education information etc.)


Domain adaptation (DA) considers the problem of adapting a machine learning model such as a classifier that is trained using a source domain to a target domain.

in Unsupervised Domain Adaptation (UDA) (Blitzer et al., 2006(Blitzer et al., , 2007Pan et al., 2010) we do not assume the availability of any labelled instances from the target domain but a set of labelled instances from the source domain and unlabelled instances from both source and the target domains.


Self-training provides a direct solution to the lack of labelled data in the target do-main in UDA (McClosky et al., 2006;Reichart and Rappoport, 2007;Drury et al., 2011).

the source domain's labelled instances are used to initialise the self-training process and during subsequent iterations labels are inferred for the target domain's unlabelled instances, which can be used to train a classifier for the task of interest.


An interesting research question we ask and answer positively in this paper is whether can we improve the performance of projection-based methods in UDA using self-training?

recent work on UDA (Morerio et al., 2018) has shown that minimising the entropy of a classifier on its predictions in the source and target domains is equivalent to learning a projection space that maximises the correlation between source and target instances.


• Third, we use the pseudo-labelled target domain instances to learn a projection for the target domain (T prj ) following the same procedure used to learn S prj .

we learn a projected feature space in the target domain where the margin between the opposite pseudo-labelled nearest neighbours is maximised.


Tri-training (Zhou and Li, 2005) relaxes the requirement of co-training for the feature spaces to be sufficient and redundant views.

in tri-training, as the name implies three separate classifiers are trained using bootstrapped subsets of instances sampled from the labelled instances.


Therefore, we can use S L to learn a projection for the source domain S prj where the separation between an instance x ∈ S L and its opposite-labelled nearest neighbours is maximised.

for an instance x we represent the set of k of its nearest neighbours NN(x, D, k) selected from a set D by a vector φ(x, D, k) as the sum of the word embeddings of the neighbours given by (1).


Armed with the pseudo-labelled data generated via Algorithm 1, we can now learn a projection for the target domain, T prj , following the same procedure we proposed for learning S prj in Section 3.1.

t prj is defined by the two target-domain projection matrices B + ∈ R d×d and B − ∈ R d×d that maximises the distance between each pseudo-labelled target instance x and its k opposite labelled nearest neighbours se-lected from positively (t + L ) and negatively (t − L ) pseudo-labelled instances.


We use cross-domain sentiment classification as an evaluation task in this paper because it has been used extensively in prior work on UDA, thereby enabling us to directly compare the performance of our proposed method against previously proposed UDA methods.

we use the Amazon multi-domain sentiment dataset, originally created by Blitzer et al.


Using MAD, in Table 1, we compare the relative effectiveness of these three steps towards the overall performance in UDA using k = 1 for all the steps.


Considering the fact that Self-Adapt is performing pseudo-labelling, similar to other self-training methods, it is interesting to see how well it compares against classical selftraining methods for inducing labels (Yarowsky, 1995;Abney, 2007;Zhou and Li, 2005;Søgaard, 2010) when applied to UDA.

we consider the classical self-training (Yarowsky, 1995;Abney, 2007) (Self), Tri-training (Zhou and Li, 2005) (Tri) and Tri-training with diversification (Søgaard, 2010) (Tri-D).


Although MT-Tri is outperforming Self-Adapt in 8 domainpairs, it is noteworthy that MT-Tri is using a larger feature space than that of Self-Adapt.

mT-Tri is using 5000 dimensional tf-idf weighted unigram and bigram vectors for representing reviews, whereas we Self-Adapt uses a 300 dimensional BonG representation computed using pre-trained GloVe vectors.


Besides, we plan to improve our neural network performance by providing richer feature.

recent embedding techniques, such as BERT or ELMO (Devlin et al., 2018;Peters et al., 2018) may provide more accurate representation of the sentences.


To this end, we propose a Transformer NMT model that incorporates dependency relations into self-attention on both source and target sides.

in training, a part of self-attention is learned with constraints based on dependency relations of source or target sentences to attend to a modifiee for each token, and, in decoding, the proposed model translates a sentence in consideration of dependency relations in both the source and target sides, which are captured by our self-attention mechanisms.


lations into self-attention on both source and target sides, dependency-based self-attention.

it parses the dependency structures of source sentences and target sentences by one attention head of the p e -th encoder layer's multihead self-attention and one of the p d -th decoder layer's multi-head self-attention, respectively, and translates a sentence based on the source-side and target-side dependency structures.


While most previous research has focused almost exclusively on English, here we target Bulgarian.

we built a dataset for our experiments based on the knowledge base of Media Scan, which has catalogued and characterized many of the Bulgarian online media in the past five years.


We are aware of only one piece of previous work for Bulgarian that targets toxicity.

(Karadzhov et al., 2017a) built a fake news and click-bait detector for Bulgarian based on data from a hackaton.


(2019)) into the realm of user-generated content, emails in our use case.

we focused on the surrogate generation step of that task, i.e., substituting named entities bearing privacy-relevant information by synthetic, yet mostly natural surrogates.


TextRank, and a topic distribution based method) to perform lyrics summarization, and show that adding contextual information helps such models to produce better summaries.

we enhance text summarization approaches with a method inspired by audio thumbnailing techniques, that leverages the repetitive structure of song texts to improve summaries.


These models differ from more traditional recurrent neural networks in different aspects.

while recurrent models can process sequences (in NLP, typically word embeddings) in order, transformers use a joint model of the right and left context of each word in order to encode an entire sequence or document.


Assessing the size of the child's lexicon is an important part of longitudinal language acquisition studies, from both theoretical and clinical perspectives.

there is evidence that aspects of grammatical development are tightly correlated with vocabulary size (Bates and Goodman, 1997).


One of these macros, "Lexicosyntactic query", queries the corpus at the word and utterance levels.

"Content Lexicosyntactic queries" can find occurrences of lexical properties and sequences of lexical properties in the corpus.


In general, a common approach to demographic inference is supervised classification, from a training set of labeled users, a model is fit to predict user features from the content of their writings.

inferring user characteristics is framed as a predictive task validated on held-out data.


In deep learning, one of the critical points for success is to learn better representation of data with many layers (Ben-gio et al., 2013) than other machine learning algorithms.

if we make a model to learn better representation of data, the model can show better performance.


Thus, even when the prediction is not exactly correct, the predicted word might not be so bad, if the estimated word is very close to the target word in the embedding space like 'programming' and 'coding'.

to check how wrong the prediction is, the word embedding can be used.


It shows that there is knowledge not only in the target probability corresponding to the target class but also in the other class probabilities in the estimation of the trained model.

the other class probabilities can contain additional information describing the input data samples differently even when the samples are in the same class.


In this paper, we propose a new knowledge distillation method, self-knowledge distillation (SKD) based on the word embedding of the training model itself.

self-knowledge is distilled from the predicted probabilities produced by the training model, expecting the model has more information as it is more trained.


First, we assume that q t should be high, and if w k is close to w t in the embedding space, q k should be also high.

the Euclidean distance between words is used to estimate the soft target probability.


Since SKD distills knowledge from the current training model, at the beginning of the training process, the model does not contain relevant information.

we cannot extract any knowledge from the training model at the beginning.


The overall quality of translation of the SKD model looks better than baseline model's.

when the BLEU scores are similar, the sentences translated by the SKD model look better.


As an example to illustrate the output of the Word2Vec model, a query with the word friendship yields friendlhip, friendihip, friendflip, friend-, affection, friendthip, gratitude, affetion, friendflhip and friendfiip as the most similar words.

in addition to the OCR errors of the word queried for, other correctlyspelled, semantically similar words (friend-, affection and gratitude) and even their erroneous forms (affetion) are returned.


adjudicary, halsbury, presuit, decretal, adj) that are either singleword terms or words of multiword terms are not found in the training data.

these are out-of-vocabulary (OOV) items.


Slavic BERT The Slavic model 4 was built using transfer learning from the Multilingual BERT model to four Slavic languages: Bulgarian, Czech, Polish, and Russian.

the Multilingual BERT model was fine-tuned on a stratified dataset of Russian news and Wikipedia articles for the other languages.


We studied the task of multiple-choice reading comprehension for low-resource languages, using a newly collected Bulgarian corpus with 2,633 questions from matriculation exams for twelfth grade in history and biology, and online exams in history without explanatory contexts.

we designed an end-to-end approach, on top of a multilingual BERT model (Devlin et al., 2019), which we fine-tuned on large-scale English reading comprehension corpora, and open-domain commonsense knowledge sources (Wikipedia).


Globally, except for the NUR task, the metaembedding configurations using pre-trained models are slightly better than the ones using indomain models and enhance the performance of the SentEmb model.

they outperform the SentEmb baseline and are ranked among the three best models for the SemEval tasks.


A straightforward approach is to learn attention signals by self-attention and then apply these signals into inputs before feeding them into LSTM.

these attention signals are applied to all gates of a LSTM cell.


Thus, our goal is to construct a better function p(y i |x, θ) given training data D. Each instance in D is a triple (x, E, y), where E is a matrix to rep-resent the discourse dependency tree of x.

element E k,l equals 1 if the edge from x k to x l exists in the discourse tree; otherwise E k,l = 0.


Once we obtain sentence embeddings h i for each sentence x i , the Sentence reader then reads sentence embeddings h i by another bi-directional LSTM and generates context-aware sentence representation H i for each x i .

two vectors generated by the forward recurrent neural network are concatenated into sentence representation H i for x i : We now obtain the context-aware sentence representations H = {H 1 , ..., H N }.


The attention vector is obtained as a weighted sum of γ d,t : Finally, the output layer receives the concatenated vector of H i and Ω i : The training updates the parameters to maximize both the label probability and 1st-order attention distribution α 1,k,l .

we use the following loss function for optimization: In this equation, E k,i is 1 if the edge from x k to x i exists in the training instance.


We used Amazon Mechanical Turk to conduct human evaluation.

randomly selected 100 documents and their four summaries generated by DIS w/ PAR, Lead-3, no-attn, and SummaRuNNer were shown to the workers.


The idea behind word embedding is that semantically similar words should be close to each other.

in an n-dimensional space, the angle between the similar words should be close to zero.


If for the English language computational models of similarity have been implemented and evaluated, this is not the case for other languages.

for the Estonian language there is no human annotated set that reflects the true similarity.


Section 5 presents and discusses the results.

we answer how similarity is influenced by language and quantify the power of the computer vision models to capture the similarity for concrete concepts.


We use some features that are closely related to the screen and user names of accounts.

we determine which of the 105 Unicode code groups an account uses in the screen and user names.


However, the patterns of the most frequent words and PoS can be more successful with identifying some functions, but not other.

it seems that the functional representations based on the vectorized texts did not quite capture the evaluative, personal and informational FTDs.


We calculate the graded entailment value between the composed sentence representations, and in results report area under ROC curve for comparison with previous literature.

we compare with the best model from Kartsaklis and Sadrzadeh (2016), which uses a metric based on the distributional inclusion hypothesis, together with a tensorbased compositional model.


However, in recent years several studies have used different technologies to improve the detection of mental health issues.

some interesting studies explore the relationship between data from online social networks and users' mental conditions (Rahman et al., 2018).


Different approaches to text and data mining methods can be applicable to social media data and may prove invaluable for health moni-toring and surveillance.

natural Language Processing (nLP), also known as Language Technologies (LT) can be used to generate systems for early anorexia detection.


In addition, this is one of the few papers which center on languages other than English.

the authors work on Spanish and Portuguese tweets and the results indicate that the approach is a feasible option for tracking health information on social networks.


For this reason there are currently several studies of this disease (Moessner et al., 2018;Bermejo et al., 2011;Chancellor et al., 2016b).

for Spanish there is no set of Twitter messages concerning this problem, and for this reason we have compiled our own corpus, SAD (Spanish Anorexia Dataset) in order to accomplish the experiments.


In this section, we describe different experiments we carried out to test the validity of the SAD corpus.

we trained several classifiers based on machine learning.


For this reason, we applied pre-processing techniques in order to prepare the data for the text classification.

we preprocessed the tweets of the SAD Dataset following these steps: The tweets were tokenized using NLTK TweetTokenizer 4 and all hashtags were removed.


Table 2 presents some examples of tweets incorrectly classified by our system and Table 1 shows the corresponding translation.

two of the tweets are false positives and the other two false negatives.


NL functions and function applications have a distinctive form.

any morphosyntactically admissible concatenation c(a, e 1 , ..., e m ), which is parsed as a(e 1 , ..., e m ), is a function or function application term (FFAT) in NL (e 1 , ..., e m may be null).


LT-Intro is formalism-agnostic -the exact mathematical structure used for lumping is irrelevant.

as shown in Supplement A 8 , we can implement lump types for NL as 1) record types, 2) function applications, 3) Cartesian product types, or 4) Π-types.


Another difference is CCG's (by default limited) support for word order.

(CCG does not handle concatenations (of terms of types) c(X/Z, Y, Z) and c(Z, Y, X\Z), where Y is nonempty 13 .)


In discussion we have tried to identify this level as semantic compositionality -an interpretation which, besides being supported by results from language processing (Pulman, 1985;Steedman, 2000;Moseley and Pulvermüller, 2014), has interesting implications on NL structure and modeling.

it may reduce syntax and morphology to word and morpheme orders, respectively (with the syntaxsemantics and phonology-morphology interfaces reducing correspondingly), with NL architecture taking on a rather different look.


Last but not least, the vocabulary trainer will be integrated into the setup of the Revita online system for language learning (Katinskaia et al., 2018) to reach a larger audience.

it can be implemented as a part of a testing mode where crowdsourced questions do not influence the learner's final score.


In this paper we propose a novel methodology to address the aforementioned problems by synthetically creating conversation datasets with peculiar characteristics.

we focus on (i) scalable and portable approaches for generating dialogues from scratch involving complex and structured knowledge, and (ii) strategies for isolating and evaluating specific reasoning capabilities of neural architectures using synthetic challenge sets.


Belinkov and Glass (To appear) survey various methods addressing interpretability.

one line of research deals with challenge sets (Lehmann et al., 1996).


Other recent approaches have focused on Memory Networks, that use an external memory component build into the system to store long term contexts (Weston et al., 2014).

end-to-end Memory Networks (Sukhbaatar et al., 2015), an extension where every component is trained in an end-to-end fashion, showed promising results in task oriented dialogues (Bordes et al., 2017).


Natural data would not be suitable for our purpose since the challenging test phenomena can also be found in the training set.

our dataset is constructed using a dialogue selfplay approach (Shah et al., 2018).


the second turn in 10 where USER changes the amount, the responsibility of the agent is compounded by the fact that in addition to understanding the compressed turn, it has to reason about an unseen context pattern.

the context also contains a form of turn that the agent has never seen during training, which is the first turn compression of the second example in Table 10.


In this paper, we explored some challenges connected to dataset creation for conversational agents and interpretability of neural models.

we propose a methodology to create rich datasets for training end-to-end conversational agents and challenge them on unseen patterns at test time.


The apparently low accuracy levels on unseen challenge cases suggest that the synthetic data and challenge generation for low resource dialogue domains can act as a reasonable approximate to real life challenges in such domains.

the more a dialogue model is able to handle these diverse challenges, the more it will be able to handle the unstructured or less structured dialogues in real human-machine interaction.


The main role of this model is to give the probability of distribution on sets of word sequences.

we are interested in n-class LM by using semantic information for the creation of word classes.


In this paper, we evaluate the effectiveness of emotion and sentiment based representations for the task of building machine learning models for FR detection.

we illustrate that improved data representations can be achieved by leveraging a plurality of emotion and sentiment extraction methods, as well as by estimating emotions and sentiments on a part-by-part basis by segmenting the reviews.


In order to simulate silver level annotations, the corpora were vectorized and annotated using a fiveparts iteration.

the corpus was split into five batches containing 20% of the corpus, a model was trained on the gold values of 80% of the corpus, recreating a new model to annotate the remaining 20%, and a different batch was swapped to be annotated each time.


(2014), dynamic filtering from Bamler and Mandt (2017), and dynamic Bernoulli embeddings from Rudolph and Blei (2018).

we study the performance of different initialisation schemes and emphasise what characteristics of each model are more suitable to data scarcity, relying on the distribution of detected drifts.


We can use high-performance and computationally heavy methods such as neural network based methods for the building of the dictionary.

we can partially use their analysis results without recomputing through the CIS dictionary.


For the purpose of augmenting the corpus, we discarded the N-grams where words at the beginning or ending of N-gram are sensitive to preceding and succeeding strings.

if the beginning or end of the N-gram is a onecharacter word and is not in a predefined white list (punctuations, and a part of particles), the Ngram is discarded by this step.


Given two words w i , w j , the alignment of w i and w j results in two new words w i , w j where |w i | = |w j |.

we say character s k ∈ w i is aligned to character t l ∈ w j if k = l. they occur at the same position.


incom.py allows computing LD(w i , w j ) based on a user-defined cost matrix M, which contains the complete alphabets A(L 1 ), A(L 2 ) of two languages L 1 , L 2 as rows and columns, respectively, as well as the costs for every possible character substitution.

for two characters s ∈ A(L 1 ) and t ∈ A(L 2 ), M(s, t) is the cost of substituting s by t. This user defined cost matrix allows computing linguistically motivated alignments by incorporating a linguistic prior into the computation of the Levenshtein distance.


Still, potential improvements can be derived from the results achieved during the experiments.

we will consider the used of attention-based encoder-decoder networks to improve the fluency of complex sentences.


In this paper, we propose a novel model for Transformer neural machine translation that incorporates syntactic distances between two source words into the relative position representations of a selfattention mechanism.

the proposed model encodes pair-wise relative depths on a source dependency tree, which are the differences between the depths of two source words, in the encoder's selfattention.


In recent years, neural machine translation (NMT) has become dominant.

the Transformer model (Vaswani et al., 2017), which is based solely on attention mechanisms, has advanced the state-of-the-art performance on various translation tasks and has become the focus of many MT researchers nowadays.


(2018), we propose a novel Transformer NMT model that incorporates the relationships between two words on source dependency structures into relative position representations in self-attention.

the proposed model adds a vector that encodes relative positional relationships between words on source dependency structures to a word embedding vector.


Because the information on proximity between tokens is not considered in self-attention itself, the information on a token's position is embedded by using positional encoding (PE).

pE provides a matrix that represents the absolute position information of tokens in a sentence, and Transformer adds pE to the embedding matrix of the input tokens.


What is the difference between the two types of image features that we considered?

we are interested whether the more complex deep learning features give a better performance in comparison to the simple RGB pixel-values features.


This process is illustrated in Figure 1, with actual examples included and word embeddings indicated as vectors with dots.

the method works as shown in Algorithm 1 for n distinct classes, k most frequent terms to be incorporated as cluster and normalized term frequency (i.e., Bag-of-words, B ij ) threshold t. A more detailed description is given underneath the algorithm.


This corpus dependence amounts to conceptual overfitting: while it leads to better fit for the original corpus, models are not transferable to new domains and analysis schemes.


Among the false negatives we analyzed, we found that the model is most likely to miss "tricky" quotations that are unusual in their grammatical structure.

it tends to miss a class of quotations that are expressed as short noun phrases or adjectival phrases embedded within a non-quotation sentence such as According to the PARC guidelines, these are cases of quotations since they are attributable statements, but they are difficult for the model to retrieve since they are hard to distinguish from "non-quotation" nominal phrases -in particular in cases like this one, where there are not even overly realized speakers.


In this work, we analyze the performance of models when the training data is extended with synthetic data using different MT approaches.

we investigate back-translated data generated not only by NMT but also by Statistical Machine Translation (SMT) models and combinations of both.


In such cases, it has become the norm to resort to back-translating freely available monolingual data as proposed in (Sennrich et al., 2016a).

one can translate a set of sentences from language L2 into L1 with an already trained MT system for the language pair L2→L1.


(2018) draws an early-stage empirical roadmap to investigating the effects of BT data.

it looks at how the amount of BT data impacts the performance of the final NMT system.


We demonstrate how incorporating knowledge directly from the FrameNet graph structure improves the performance of a neural network-based frame identification system.

we construct a bidirectional LSTM with a loss function that incorporates various graph-and corpus-based frame embeddings for learning and ultimately achieves strong performance gains with the graphbased embeddings over corpus-based embeddings alone.


This work leverages FrameNet's graph structure to boost the performance of neural architectures for FrameID.

we construct frame embeddings from the FrameNet graph structure and use them as input to a neural network.


Corpus embeddings of frames To compare the graph embeddings with frame embeddings learned from a corpus, we took the freely available embeddings from Sikos and Padó (2018), in which the authors generated frame embeddings using the Word2Vec tool.

the authors took lemmatized sentences from the FrameNet annotated data and replaced each predicate (e.g., say.v) with the frame it evokes: FrameNet sentence Officials say.v he left Frame evoked STATEMENT Embedding Input official STATEMENT he left This resulted in sentences of frames in context, so the example input above would not only produce an embedding for the frame STATEMENT, but also embeddings for the context words such as official.


This has been done in a way that is similar to the one described earlier in relation to graphically connecting the various word senses in a synset.

the word sense  (Agirre et al., 2018) 68  Table 2: The state of the art across WSD systems.


We call these words giveaway words as they alone will be a sufficient source of information for the classifier.

the reasoning text in a large number of cases de facto contains the label.


The results reported in the present paper lies at the intersection of those research goals.

we aim here to gain a better insight into these two sources of lexical information, and the quality of the resulting word embeddings, by assessing how graph-based word embeddings compare to mainstream text-based ones.


For example, 'take + SBJ + OBJ:part + in' is proper, while 'take + SBJ + OBJ:part' is not proper (because not complete), and 'read + SBJ + OBJ' is proper, while 'read + SBJ + OBJ:book' is not proper (because not clean).

free slots in pVCCs are complements (subject included) and fillers in pVCCs are idiomatic, carrying some special meaning.


We introduced a function (we call it f now) on vertices of the corpus lattice: it is essentially the corpus frequency of the VCC represented by the given vertex.

this f shows how many corpus clauses are represented by the given vertex, or how many corpus clauses this VCC fits to.


(Please note, that stay also means advancing between vertices, the term itself refers to the fact that the value of f does not change during this step.)

where one (or only few) arrows originate from a vertex, it tend to be a place of stay, similarly, when we have many arrows from the same vertex, it is usually a place of jump.


But what if we have two (or more) potential additional elements which are not significant separately, but together (their f values added up) they would define a regular stay?

what to do when two (or more) elements seem to be mutually exclusively mandatory at one point?


Note that only a fraction of all tweets have hashtags.

within our collected data, we found that only 10-15% of our tweets with emojis also include hashtags.


We address the problem of cognate identification across vocabularies of any pair of languages.

we focus on the case where the examined languages are low-resource, to the extent that no training data whatsoever in these languages, or even closely related ones, is available for the task.


We call this metric the mean average recall@k: where R@k = #cognates within top-k results #cognates in Y , where Q is the number of queries, and L is the number of language pairs.

for each word query q, we rank the pairs (q, y i )∀i and get the top 100 words y for each q.


Therefore, e 2 = M T (f 1 ) = M T (HT (e 1 )).

e 2 is "twice removed" from e 1 , being translated once by a human and once automatically.


This approach is to our best knowledge novel in the context of neural network tagging.

we train a separate model on only the lexical category and use the coarse-grained output tag as an input into the main model.


NLP pipeline for stopword removal and stemming, resulting text is processed into unigrams) but instead of one-hot vectors, GloVe vector embeddings are used.

the 300dimensional GloVe vectors trained on the large Common Crawl corpus of 840 billion tokens with a vocabulary of 2.2 million words.


It is important to underline that we connected the training set context with the correct target word sense, but for the test set we simply connected the contexts to their instance id without any explicit link to the correct results.

the fake words we inserted for generating the cPVs are the correct WordNet sense id string for the training context and the test instance id string for the test contexts.


Attributes from Text The majority of methods to predict attributes for entities in CCKBs are based on techniques from representation learning.

they us distributed representations (i.e., vectors, also called embeddings) to represent the entities, and sometimes also the attributes.


The model uses an n-dimensional entity embedding as its input which is mapped through a tanh nonlinearity onto an h-dimensional hidden layer, which in turn maps onto an |N |-dimensional output layer (where |N | is the number of the numeric attributes) using a sigmoid nonlinearity.

each unit in the output layer corresponds to one numeric attribute, and the model predicts all numeric attributes simultaneously.


Probably the most surprising outcome of our experiment is the asymmetry that we observe: Joint modeling achieves respective improvements over individual modeling for numeric attributes, but not for categorical attributes.

the prediction of numeric attributes profits from the availability of categorical attributes, but not vice versa.


However, important gaps make progress in the field slow.

there is a relative lack of labelled datasets to study the task over a short time-frame, since most known instances of semantic change took place over centuries or decades.


Therefore, incorporating the temporal dimension of the task in our approach is a major direction for future work.

we plan to incorporate temporal approaches that are well-suited for the task, such as temporal word clustering and change point detection.


The input to our neural network consists of various domain-specific features that have been previously shown to work well for the task of checkworthiness prediction.

from (Hassan et al., 2015b), we adopt TF.IDF-weighted bag of words, part-of-speech tags, the presence of named entities, sentiment scores, and sentence length (in number of tokens).


Thus, we have a ranking task, for which we use suitable information retrieval evaluation measures.

we adopt Mean Average Precision (MAP) as our primary evaluation measure.


To model the degree of annotation artifact existence, a classifier that uses only hypotheses for predicting the entailment classes can be trained.

the classifier completely ignoring the information contained in the premise(s) is used.


We demonstrate the effectiveness of the proposed approach on our pre-trained word embedding models in Vietnamese to select which models are suitable for a named entity recognition (NER) task.

we create a large Vietnamese word analogy list to evaluate and select the pre-trained embedding models for the task.


• Evaluator evaluates the pre-trained embeddings for a downstream task.

given multiple sets of pre-trained embeddings, how do we choose the embeddings which will potentially work best for a specific downstream task (e.g., NER)?


Consequently, Set 1 and 2 constitute all terms in Corpus B, while Set 2 and 3 constitute all terms in Corpus T. In a pre-built TF-IDF model, all the terms in the vocabulary are from the background Corpus B, i.e., the terms used to generate the term vector space are those from Set 1 and 2, while those terms in Set 3 will not appear in the TF-IDF model at all.

the terms in Set 3 are all the Figure 1: Term Sets within a Background Corpus B and a Target Corpus T unknown terms with respect to the TF-IDF model.


Instead of assigning different ranks to the neighbouring terms with the same document frequency, we implement some extra operations to make their ranks the same.

for the terms with the same document frequency, i.e., the terms with indices from i to j, we assign the same average rank i+j 2 to all of these, such that this does not affect the rank of any other term.


In order to achieve more comprehensive results for this evaluation, we implement tests for set variants.

for each set of experiments, we make comparisons not only between the two background corpora being evaluated, but also between each corpus and the union of both corpora; for example for COCA vs. COHA, we not only implement the comparison between COCA and COHA, but also between COCA and COCA + COHA and between COHA and COCA + COHA, where COCA + COHA is the union of COCA and COHA.


Our intention is to focus on supervised learning.

this means working on collections of short texts such as single sentences, tweets or short reviews with labeled sentiments.


This paper, therefore, suggests solutions such as lenses (Section 3) and corpus versioning (Section 4.2) to address this issue.

our main contribution is describing the innovations that can be applied at a corpus and evaluation systems level in order to create more flexible and expressive evaluations.


A recurrent function generates a weight matrix W which is used to form vertex embeddings and in turn adjust edge scores.

we first obtain an initial score matrix S 0 using the biaffine one-step parser (2.1), and initialize a recurrent hidden state h 0 using a linear transformation of f enc 's final hidden state.


Finally, our work is inspired by techniques for improving upon maximum likelihood training through error exploration and dynamic oracles Nivre, 2012, 2013), and related techniques in imitation learning for structured prediction (Daumé III et al., 2009;Ross et al., 2011;He et al., 2012;Goodman et al., 2016).

our formulation is closely related to the framework of (Chang et al., 2015), where our oracle can be seen as an optimal roll-out policy which computes action costs without explicit roll-outs.


based on machine translation, cross-lingual language models and multilingual embeddings.

we are interested in training new neural networks in multi-and cross-lingual term extraction and definition retrieval settings.


The percentage of the correctly labelled tweets was around 80%.

the training dataset contains around 20% noise (i.e., incorrectly labeled tweets).


We use existing methods for aligning comparable news articles (Utiyama and Isahara, 2003;Vu et al., 2009).

we create an aligned corpus by pairing a Finnish article with a Swedish article published within a two-day window and sharing three or more named entities.


In order for this to work correctly, we must partially unpack the parse forest.

pp nodes are unpacked (cloned) to become independent children of any ambiguous parent nonterminal nodes up to and including the enclosing Vp node.


In order to combine logical and probabilistic information, we draw on existing work in Statistical Relational AI (Getoor and Taskar, 2007).

we utilize Markov Logic Networks (MLNs) (Domingos and Lowd, 2009), which employ weighted formulas in first-order logic to compactly encode complex undirected probabilistic graphical models.


To do so, we make use of existing approaches that compute distributional representations for phrases.

we compute the vector for a phrase from the vectors of the words in that phrase, using either vector addition (Landauer and Dumais, 1997) or component-wise multiplication (Mitchell and Lapata, 2008;Mitchell and Lapata, 2010).


They show that the distributional only baselines perform very poorly.

they perform worse than strict  Bos and Markert (2005), a system that uses only logic.


The vast amount of world knowledge available in Wikipedia has been shown to benefit many types of text processing tasks, such as coreference resolution (Ponzetto and Strube, 2006;Haghighi and Klein, 2009;Bryl et al., 2010;Rahman and Ng, 2011), information retrieval (Milne, 2007;Li et al., 2007;Potthast et al., 2008;Cimiano et al., 2009), or question answering (Ahn et al., 2004;Kaisser, 2008;Ferrucci et al., 2010).

the user contributed link structure of Wikipedia has been shown to provide useful supervision for training named entity disambiguation (Bunescu and Pasca, 2006;Cucerzan, 2007) and word sense disambiguation (Mihalcea, 2007;Ponzetto and Navigli, 2010) systems.


The analysis performed in (Agirre and Amigó, In prep) shows that Pearson and averaging across datasets are the best suited combination in general.

pearson is more informative than Spearman, in that Spearman only takes the rank differences into account, while pearson does account for value differences as well.


We propose to use dependency relations between words to derive an alternative structural representation.

dependency relations are used to link words in a way that words are always at the leaf level.


A second function is obtained by applying a Distributional Compositional Semantics operator, in line with the approaches introduced in (Mitchell and Lapata, 2010), and it is adopted to account for semantic composition.

the approach described in (Croce et al., 2012c) has been applied.


We decided to use a training dataset that achieved the best results over datasets radically different from the training material in the STS challenge of Semeval 2012.

for the FNWN and OnWN datasets, we arbitrarily selected the training material achieving best results over the 2012 surprise.OnWN; for the headlines and SMT datasets we maximized performance training over surprise.SMTnews.


It can be seen that the level of overlap is quite low and the various heuristics generate quite different lists of nouns.

the recall is low, in almost all cases less than a third of nouns in one list appear in the other.


While the evaluation measure, Pearson correlation, does not take into account the shape of the output distribution, Figure 1 shows that this information may be a useful indicator of model quality and behaviour.

the role of the regression component in our approach is to learn a transformation from the output distributions of the models to the distribution of the training data gold standard.


We also presented evaluation results, using data from the STS 2012 and STS 2013 shared tasks, that indicate that the system performs competitively relative to other approaches for many tasks.

we observed very good performance on the news headline similarity and MT evaluation subtasks of the STS 2013 shared task.


In our previous participation in SemEval12-STS task (Malandrakis et al., 2012) we employed a modification of the pointwise mutual information based on the maximum sense similarity assumption (Resnik, 1995) and the minimization of the respective error in similarity estimation.

exponential weights α were introduced in order to reduce the overestimation of denominator probabilities.


Firstly, Stanford parser 1 is used for sentence tokenization and parsing.

the tokens n't and 'm are replaced with not and am.


CRM is based on a notion of substitutability.

the more appropriate it is to substitute word w 1 in place of word w 2 in a suitable natural language task, the more semantically similar they are.


To obtain the optimal SVR parameters C, g, and p, we employ grid search with 10-fold cross validation on training data.

if the score returned by the regression model is bigger than 5 or less than 0, we normalize it as 5 or 0, respectively.


One reason of the poor performance of FNWN may be the big length difference of sentence pairs.

sentence from WordNet is short while sentence from FrameNet is quite longer, and some samples even have more than one sentence (e.g.


The correlation was 0.16017 and 0.3113, respectively.

the normalization mainly affects the MT features.


In this way, the meaning of a term across a corpus is thoroughly conveyed by the contexts it appears in, where a context may typically be the set of co-occurring words in a document, in a sentence or in a window of surrounding terms.

we take into account two main classes of models: Simple Distributional Spaces and Structured Semantic Spaces.


Semantic space building involves some parame-ters.

each semantic space needs to set up the dimension k of the space.


We conducted experiments with various machine learning algorithms implemented in the Python library scikit-learn (Pedregosa et al., 2011).

we tested linear regression, regularized linear regression (ridge regression), Bayesian ridge regression, support vector regression and regression trees.


Contrary to the observation we made in training data, the methods that used regression to predict the gold standard performed poorly compared with the text similarity functions proposed in Section 5.

the results in Table 2 overcome those in Tables 3  and 4.


The algorithm looks for the longest similar regions by maximizing the number of matches and minimizing the number of insertion/deletion/substitution operations necessary to align the two sequences.

it finds the longest common sequence while tolerating a small number of differences.


The background corpus used for generating the topic models is similar to the COL-WTMF system (Guo and Diab, 2012) from the STS-2012 task, which outperformed LDA.

we use sense definitions from WordNet, Wiktionary and all sentences from the Brown corpus.


We also employ features produced by the opinionextraction model of Yang and Cardie (Yang and Cardie, 2012), which is better suited to process ex-pressions of arbitrary length.

for each sentence, we extract subjective expressions and generate the following features.


We recognize the necessity of having and using manually-constructed semantic-knowledge sources in order to get precise assessments of the semantic similarity of texts, but, in turn, we also consider that it is possible to obtain good estimations of these similarities using less-expensive, and perhaps broader, information sources.

our proposal is to automatically extract the semantic knowledge from large amounts of raw data samples i.e.


Instead of focusing on a specific task, we aim to provide a flexible resource that could be adapted to many possible tasks.

the contribution of this work is in creating a dataset of syntactic-ngrams which is: • Derived from a very large (345 billion words) corpus spanning a long time period.


The syntactic representation we work with is based on dependencygrammar.

we use labeled dependency trees following the "basic" variant of the Stanforddependencies scheme (de Marneffe and Manning, 2008b;de Marneffe and Manning, 2008a).


Content-words and Functional-markers We distinguish between content-words which are meaning bearing elements and functional-markers, which serve to add polarity, modality or definiteness information to the meaning bearing elements, but do not carry semantic meaning of their own, such as the auxiliary verb "have" in Figure 1.

we treat words with a dependency-label of det, poss, neg, aux, auxpass, ps, mark, complm and prt as functional-markers.


The work we present here is strongly based on approaches towards SAS by Meurers and colleagues (Bailey and Meurers, 2008;Meurers et al., 2011a;Meurers et al., 2011b;.

the sentence alignment model described in Section 3 (and again discussed in Section 4) is modeled after the one used by Meurers et al.


Second, in a small number of cases, both annotators agree on one member of the set of alternatives.

the single sentence in the intersection of the sets of sentences identified by the two annotators is taken as the gold-standard annotation.


AgreeEntropy Here we weight the first feature according to the second, taking the entropy as a confidence score for the binary feature.

we value SourceAgree at 0.5 when the feature is true, −0.5 when false, and multiply this with (1 − entropy).


This section discusses experiments on short answer scoring (binary classification) for German, in the context of reading comprehension for language learning.

we investigate the text-based models described in Section 4.2.


In all cases, features and parameter settings were tuned on a development set which was extracted from the larger CREG corpus.

there is no overlap between test and development data.


We attempt to reduce the erroneous signal of the word count approach while maintaining legitimate signal simply by refining the lexicon.

we would like to move closer to the goal in Figure 1, by eliminating words that often carry erroneous signal such as 'play', and keeping words which often carry the sought-after signal, such as 'cheerful'.


The question the judges were told to answer was "Does the word contribute to the associated psychologicalstate (POSEMO or NEGEMO) within the sentence it appears?".

"would the sentence convey less [positive emotion or negative emotion] without this word?".


Note that our additional training data is automatically labeled with a PropBank/NomBank-style parser.

we need to map our annotations to FrameNet.


The main difference between this simplified model and the original entity grid model lies in the different use of training data: while entity grid models treat all non-realized items equally, our model gets to "see" actual examples of entities that are implicit.

our simplified model takes into account implicit mentions of entities, not only explicit ones.


After experimenting with numerous features for measuring complexity, we have found that a few types of word counts are the most helpful.

the most helpful features include: the number of words between two named entities (for two-entity queries), the number of words before the named entity that was part of the query (for one-entity queries), and the total number of non-named-entity, non-stopword words in the sentence.


These experiments cover existing approaches to related tasks, as well as some new methods.

we present type-constrained extensions of discriminative HMMs and SEARN sequence models with continuous word representations that perform well.


We use a distributional semantic model in order to incorporate distributional information as features in our system.

we use the neural-network based models from (Mikolov et al., 2013), also referred as word embeddings.


We also compare to the performance of existing SST systems.

we use Sense-Learner (Mihalcea and Csomai, 2005) as a baseline, which produces estimates of the WordNet sense for each word.


SenseLearner does not seem to capture the relevant information and does not reach baseline performance.

there is no off-the-shelf tool for supersense tagging of Twitter that does much better than assigning the most frequent sense to predicted nouns and verbs.


One important source of error seems to be unreliable part-of-speech tagging.

we predict the wrong POS for 20-35% of the verbs across the data sets, and for 4-6.5% of the nouns.


Our task is to derive a lexicon Lex that quantifies the emotional valence of words (from the tweets in D) to emotion classes.

the lexicon may be thought of as a 2d-associative array where Lex[w][c] indicates the emotional valence of the word w to the emotion class c. When there is no ambiguity, we will use Lex(i, j) to refer to the emotional valence of word w i to the emotion class C j .


From the results obtained for our methods in Table 3 it suggests that our methods achieve the best Fscores for 4 emotions namely anger, fear, sadness and love out of the 6 emotions.

the EM-class-corpus-lex method obtains the best F-score for 3 emotions namely anger, sadness and love.


Furthermore, let P , indexed by k, be a set of positions, relative to a target word w i , in which a context word w j can co-occur with w i .

this is the signed distance between w j and w i , in number of words.


Most interestingly, value is governed by a "peculiar logic" that provides constraints on which actions are deemed morally acceptable and which are not.

the principal of reciprocity states that the valence and magnitude of reciprocal actions (actions that are done "in return" for something else) should match, i.e., positive valued actions should This work is licenced under a Creative Commons Attribution 4.0 International License.


In our analysis, we compared each email text to the description of its associated category in order to investigate the nature of the inference steps involved.

our analysis aims to give answers to the following questions: What text representation is appropriate for the email categorization task?


These metrics were shown to perform well for single document text summarization, especially for short summaries.

lin and Hovy (2003) showed that ROUGE-1 achieves high correlation with human judgments.


In Jaworski and Przepiórkowski 2014 we propose to define 'semantic roles' on the basis of morphosyntactic information, including morphological cases, following the Slavic linguistic tradition stemming from the work of Roman Jakobson (see, e.g., Jakobson 1971a,b).

since the broader context of the work reported here is the development of a syntactico-semantic LFG (Lexical-Functional Grammar; Bresnan 2001; Dalrymple 2001) parser for Polish, we build on the usual LFG approach of obtaining semantic representations on the basis of f-structures, i.e., non-tree-configurational syntactic representations (as opposed to more surfacy tree-configurational c-structures) containing information about predicates, grammatical functions and morphosyntactic features; this so-called description-by-analysis (DBA) approach has been adopted for German (Frank and Erk, 2004;Frank and Semecký, 2004;Frank, 2004), English (Crouch and King, 2006) and Japanese (Umemoto, 2006).


In Section 4.1, we saw that adding more data annotated through projection did not elevate SRL performance.

the same performance was achieved using only a small amount of data.


Finally, based on the observation that the quality of labelling using manually annotated data is higher than using the automatically projected data, we replicate 1K+5K with the 5K data labelled using the model trained on the training subset of 1K at each cross-validation fold.

we perform a one-round self-training with this model.


Given a tree t, the set S(t) is defined as the set containing all the relevant CSSTs of the tree t. As for the tree kernels (Collins and Duffy, 2002), the set S(t) contains all CSSTs derived from the subtrees of t such that if a node n belongs to a subtree t s , all the siblings of n in t belongs to t s .

productions of the initial subtrees are complete.


For RR models we have that CSTK+BA and CSTK+FA both achieve higher accuracy than ADD and FullAdd, with a statistical significante greater than 93.7%, as computed with the sign test.

we have that CSTK+BA has an average accuracy 7.94% higher than ADD and 5.89% higher than FullADD, while CSTK+FA improves on ADD and FullADD by 8.52% and 6.46%, respectively.


We explore a dynamic interpretation of the lexical semantic model developed in Generative Lexicon Theory (Pustejovsky, 1995;.

we are interested in using model building (Blackburn and Bos, 2008;Konrad, 2004;Gardent and Konrad, 2000) and simulation generation (Coyne and Sproat, 2001;Siskind, 2011) to reveal the conceptual presuppositions inherent in natural language expressions.


However, as (Pustejovsky, 2013a) points out, since most manner motion verbs in language are actually directed processes, simple decompositions into change-of-location are inadequate.

they are guarded transitions where the test is not just non-coreference, but makes reference to values on a scale, C, and ensures that it continues in an order-preserving change through the iterations.


For example, the ball enters the room is satisified when the distinguished location, D, (the room) is successfully tested as the location for the moving object.

the location is tested against the current location for an object ((loc(x) = D)?


: At the termination of the action, object ball has moved relative to a surface in a manner that is [+translate].

the movement is a translocation across absolute space, but other attributes (such as the ball's orientation) do not change.


This model, called coref, implements the first two steps of the above algorithm, but instead of extracting only mental state terms associated with event actors (last step), it considers all mentions appearing anywhere in the coreference neighborhood.

all unique sentences traversed by the relevant coreference chains are first pieced together to define a single neighborhood for a given document; then the relative joint frequencies of n-grams are computed by incrementing f once for each neighborhood that contains all terms with correct POS tags.


Let σ * S ( ) denote the best similarity score achievable when comparing elements from set S to using the similarity function σ.


The analogy is complete when we consider that crimson pencils should perhaps be partially rewarded when matched by cardinal, rose or cerise pencils.

a similarity mea-sure should account for an accumulated mass of synonyms.


The set ∆ can then be derived from Γ using the following natural deduction rules: 4 • Initialize ∆ with lambda terms (sets) that have no outscoped sets in Γ: • Add constraints to appropriate sets in ∆: • Add constraints of supersets as constraints on subsets in ∆: For example, the graph in Figure 2 can be translated into the following lambda calculus expression (including quantifiers over eventualities in the source graph, to eliminate unbound variables): The semantic dependency representation defined in this paper assumes semantic dependencies other than those representing continuations are derived compositionally by a categorial grammar.

this definition assumes a Generalized Categorial Grammar (GCG) (Bach, 1981;Oehrle, 1994), because it can be used to distinguish argument and modifier compositions (from which restrictor and nuclear scope sets are derived in a treestructured continuation graph), and because large GCG-annotated corpora defined with this distinction are readily available (Nguyen et al., 2012).


The context presupposes the acquisition of sensory input from the environment.

it assumes that an agent using such a representation is able to classify the entity in focus x as being of type c and is able to use some height sensor to obtain an estimate of x's height (the value of h is the sensor reading).


Contrary to what has been argued in the literature (Rapp, 2002; Sahlgren, 2006)-that bag-of-words models based on secondorder statistics mainly capture paradigmatic relations and that syntagmatic relations need to be gathered from first-order models-we show that second-order models perform well on both paradigmatic and syntagmatic relations if their parameters are properly tuned.

our results show that size of the context window and dimensionality reduction play a key role in differentiating DSM performance on paradigmatic vs. syntagmatic relations.


The ablation value for a given parameter is the proportion of variance (R 2 ) explained by this parameter together with all its interactions, corresponding to the reduction in adjusted R 2 of the linear model fit if the parameter were left out.

it allows us to find out whether a certain parameter has a substantial effect on model performance (on top of all other parameters).


They have already been applied by Bullinaria and Levy (2012) to a different set of tasks, including the TOEFL multiplechoice synonymy task.

bullinaria and Levy found that discarding the initial SVD dimensions (with highest variance) leads to substantial improvements, especially in the TOEFL task.


We choose distributional semantics (Erk, 2012) as our frame of reference, as it produces continuous similarity scores, in line with the graded nature of the modification effects we are investigating.

we look at the compositional extension of distributional semantics (Baroni, 2013), because we need representations not only for words, but also phrases, and we adopt the asymmetric similarity measures developed in the literature on lexical entailment (Kotlerman et al., 2010;Lenci and Benotto, 2012), because we are interested in an asymmetric relation (to what extent the concept denoted by the phrase is a good instance of the target class, and not vice versa).


4 We then constructed a number of tuples that should not display a strong modification effect.

in order to insure that any atypical rating we obtained on the distorted tuples could not be explained away by characteristics of m or h alone (rather than by their combination), for each distorted tuple we constructed a few more tuples with the same h and c but a different m, that we did not expect to be strongly distorting (e.g., plastic, bottle, drinkware ).


(2010) takes inspiration from formal semantics to characterize composition as function application.

in modifier-head phrases, the modifier is treated as a linear function operating on the head vector.


We use the DISSECT toolkit 7 to estimate the parameters of the composition methods and derive phrase vectors.

dISSECT finds optimal parameter settings by learning to approximate corpus-extracted phrase vector examples with least-squares methods (dinu et al., 2013).


In the interaction between compositional models and directional similarity measures, we have observed that simpler models yield better results.

mult and wadd are economical composition models than can be applied on full-rank spaces, which in turn work best with our similarity measures.


The approach we take here is not fully comparable to any of the former approaches, since we try to derive a semantic classification BV-PP pairs in an unsupervised manner and we only use syntactic features, stemming from corpus instances of both the BVs and the PVs.

we do not attempt to classify PVs, but we try to classify syntactic transfers and, by doing so, we identify syntactic transfer patterns which we hypothesize to have a close relation to semantic PV classes and the semantics of the particles.


and still validated a gold standard whose creation was explicitly based on syntactic subcategorization frames.

: there is an apparent tight interrelation between syntax and semantics for PVs, at least in the sense that semantic distinctions can be used to predict different syntactic behaviour.


As in this paper we discuss PESN in the framework of RNTN, computation outside the nodes N 1 , N 2 , N 3 follows that for the standard three-way tensors in RNTN.

the hidden vector i 3 is computed with the following formula: where, W r ∈ R d×(d+d) and V r ∈ R (d+d)×(d+d)×d are the matrix and tensor of the composition function used in RNTN, respectively, each of which is shared over the whole tree in computing vectors i 1 , i 2 , and i 3 .


In addition, the Stanford Sentiment Treebank contains manually annotated sentiment for each individual phrase in a parse tree, so we use such annotation but not other manual lexicons, by assuming such annotation fits the corpus itself the best.

we use bigram and trigram annotation in the treebank.


the context "X was born in" tends to select a human being, whereas "X was launched" tends to rule it out.

back to the example, one can expect the similarity between the distributional representation of the term "C.E.O" and other terms denoting instances of Person according to K to be relatively low, hindering the plausibility of ψ 1 with regard to K. ψ 1 should stand as an outlier among consequences of K, and therefore is probably undesirable.


There is no reason to expect that sim(CEO, Beijing) > sim(CEO, Thelonious Monk).

it is implausible to assume that elements of inst Γ (¬A) should a priori share similar contexts.


But given the additional assumptions made about K, discarding the former is preferable, in that discarding the latter would also result in the loss of ψ 2 .

some subbases of K (like K \(1) here) are more relevant than others (e.g.


Γ1 and Γ2 are primarily viewed as bases, not as theories.

if Cn(Γ1) = Cn(Γ2), but Γ1 ⊆ Γ2 and Γ2 ⊆ Γ1, then Γ1 and Γ2 are not comparable wrt ≺.


For our experiments, we use a Support Vector Machine (SVM) classifier with a linear kernel.

we use LIBSVM (Chang and Lin, 2001) as our training and testing software.


The most marked improvement is in the WEB models (both CMC and DISPLACE) and the BN model's DISPLACE label classification.

the recall value shows improvement in these classifier models.


Having access to multiple annotations of the same item allows to identify correlations among senses of a word.

we can mine how many times the annotators confused 2 particular senses of a word.


Of the 470 sentences containing negations, 13 (2.8%) failed to parse, breaking the mapping.

our mapping strategy has coverage of about 97.2%.


Similarly, lying, sarcasm, irony, and other cases where the writer's internal belief may differ from the expressed belief are not captured.

we take all expressed beliefs at "face value".


This paper explores the utility of homophily within joint models for document-level semantic classification, focusing specifically on tasks which are not associated with any explicit graph structure.

we examine whether implicit semantic document links can improve the results of a point-wise (content-based) classification approach.


The modification involves changing the document frequency count in idf calculation such that for n-grams with n > 1 document frequency is always considered to be one.

we assume that all multi-word candidates occur in one document only.


In the current work and the Semeval shared task this parameter is 15.

kP-Miner calculates the subsumption count over the top 15 terms whereas we calculate it over the top 100 terms.


Our initial hypothesis for the success of logarithmic decay functions is that both positional and document frequency heuristics are governed by the law of diminishing returns.

the distinguishing power of each heuristic decreases as the inputs increase.


2 In our work, we assume that the translation relations are probabilistic.

we assume that there exist probabilistic 'operators' P k that indicate the probabilities -denoted by P k [α|z] -by which a word z of language L k translates into a word α of language R. Our motivation is to induce M different lexical semantic networks that represent the lexical semantic spaces of the languages L 1 , .


First of all, with the wide availability of annotated corpora, it would be beneficial to have a system for mapping relations across approaches.

for classification tasks such as discourse parsing, access to larger amounts of data is likely to yield better results.


While (1) and(2) can be acquired directly from the corpus, (3) involves optimization, since the matrix (3) is supposed to account for the verb's disambiguating effect on all its subjects.

pLF proposes to learn matrices via regression problems such as the following (Guevara, 2010), where subj(v) comprises the subjects seen with the verb v: the verb's subject matrix is learned as the matrix which, multiplied with a subject noun vector, best predicts the noun-verb phrase vector.


Notably, frequency has a negative effect on rank difference.

more frequent verbs are easier to predict.


We assume the reader is familiar with the knowledge representation literature (for a summary, see Brachman and Levesque 2004), and describe only those parts of the system that differ from the mainstream assumptions.

we collapse attribution, unary predication, and IS A links in a single link type '0' (as in Figs.


Though many early AI systems employed hand-encoded scripts, more recent work has attempted to induce scripts with automatic and scalable techniques.

several related techniques approach the problem of script induction as one of learning narrative chains from text corpora (Chambers and Jurafsky, 2008;Chambers and Jurafsky, 2009;Jans et al., 2012;Pichotta and Mooney, 2014).


We call the collection of indicators of an entity as its entity model.

an entity model consists of multiple entity indicators that capture diverse and orthogonal ways an entity can be expressed in the text.


Consideration of the decoding problem in natural language semantic dependency parsing as finding a maximum spanning DAG of a weighted directed graph carries many complexities.

the computational complexity (and approximability) of the problem has not been addressed in the literature to date.


On the other hand, given an outerplanar graph and some random sentential order, we of course do not necessarily have a projective digraph.

an outerplanar drawing in the plane of a digraph does not necessarily have the specific desired order of vertices on its outer face.


An MW-FAS would contain every negative weighted edge.

so, the difference between a nice MWFAs and an MWFAs is just this one edge, for G. we have given an L-reduction from MWFAs to nice MWFAs, where the optimal solution for MW-FAs is just the nice MWFAs with negative weighted edges removed (so both α and β from Definition 1 are equals to 1).


This example demonstrates that sentences including coordinate structures cannot be represented by any strictly left-branching derivations.

incremental derivation approaches cannot achieve a word-by-word incremental interpretation.


When an auxiliary tree a is adjoined to a partial parse tree at a node η, the label of η must be the same as that of the foot of a.

cat(η) = cat(f oot(a)) and rule(η) = rule(f oot(a)) hold.


• Apply the function t i to the semantic representation s i−1 .

the semantic representation for We call the function t i semantic transition function (or transition function for short).


When nil-adjoining operation is applied to η, the variable z is replaced with the identity function λx.x.

after applying λs.s(λx.x) to the semantic representation s i−1 , the semantic transition function t i is applied.


However, the derivation which has this semantic representation is not a partial structure of incremental derivation shown in Figure 3(b).

the derivation is not consistent with that of "Anna met and might marry Manny."


[2011] and explores issues related to the use of this architecture in the context of the CoNLL 2009 shared task.

we present Daisy, a system that (1) employs features derived from dependency parse as input, (2) assigns semantic roles to both verbal and nominal predicates, and (3) automatically assigns word senses to the predicates as described in the CoNLL 2009 shared task (Hajič et al.


At the first stage, the EFI algorithm relies on a decision tree, generated from the training data, to obtain meaningful feature combinations.

the algorithm extracts all the paths leading from the root of the induced tree to any node.


Their system combines the predictions from seven distinct mention-pair resolvers, each of which focuses on a specific pair of mention types (NP, pronoun, verb).

their verb-pronoun resolver falls within the scope of discourse deixis.


We interpret these subgraphs as follows: • sg 1 : The connection between a sentence and subsequent ones.

at least two entities are mentioned in one sentence and the subsequent ones are about these entities.


This model has been argued to perform either better than or on a par with competing architectures, depending on the task and on hyperparameter settings (Baroni et al., 2014;Levy et al., 2015).


In order to create datasets of pairs such as Instance 1-Instance 2 and Instance 1-Instance 3, we first determine whether WordNet verb instances are metaphorical or literal.

we chose verbs with at least three senses (so that there is a higher chance of at least one sense being metaphorical) and less than ten senses (to avoid highly ambiguous verbs).


Rather than viewing instances as either metaphorical or literal, one may also consider a graded notion of metaphoricity.

on a scale from most literal to most metaphorical, instances may have different degrees of metaphoricity (or literalness).


We use linguistic phrase types as a proxy for syntactic and semantic relatedness.

we identify coordinated groups of verb phrases, noun phrases, adjectival phrases, and prepositional phrases.


In the semantic parsing evaluation, most of the parsing errors arose from a failure in the sentence parser or preprocessing directly preceding the sentence parser.

17 out of the 52 axioms had errors arising from the sentence parser.


The natural tableau system was succesfully applied to the SICK textual entailment problems (Marelli et al., 2014) by Abzianidze (2015a).

the theorem prover for natural language, called LangPro, was implemented that integrates three modules: the parsers for Combinatory Categorial Grammar (CCG) (Steedman, 2000), LLFgen that generates LLFs from the CCG derivation trees, and the natural logic tableau prover (NLogPro) which builds tableau proofs.


At the same time, a positive effect that is meant to be true, is asserted.

greece benefits from the situation described.


According to Table 2, "to criticize" has a negcl role while "to help" has a poscl role.

if A criticizes that B helps C to D (D=survive), then, obviously, A disapproves D. a negcl on a poscl gives disapprove, see rule r3.


In this paper, we collect human judgements of the validity of inferences following from the insertion and deletion of various classes of adjectives and analyze the results.

our findings suggest that, in practice, most sentences involving nonsubsective ANs can be safely generalized to statements about the N. non-subsective adjectives often behave like normal, subsective adjectives.


alleged) behave nearly identically to how we expect regular, subsective adjectives to behave (Figure 2b).

in 80% of cases, deleting the plain non-subsective adjective was judged to produce ENTAILMENT, rather than the expected UNKNOWN.


Rather than looking into status-or network-based notions of power differences, we capitalise on the argumentative character of such discussions to study how argumentative aspects such as agreement and disagreement relate to style accommodation.

we focus on the interplay between alignment of beliefs-interlocutors' (dis)agreement on what is said-and alignment of linguistic styleinterlocutors' coordination or lack thereof on how content is expressed.


Indeed, if relations and entities are orthogonal (∀r, e (r ⊥ e)), then if two entities e, e and two relations r, r are distinct (i.e., ||e − e || 2 ≥ and ||r − r || 2 ≥ ), it follows that ||e + r − e − r || 2 = ||e − e || 2 + |||r − r || 2 ≥ 2 by Pythagorean theorem.

two sentences whose semantic representations involve two distinct entities and/or relations will have different values.


Therefore, the F-scores for the classes 'favor' and 'against' are more balanced over all targets than for just   The Stance Dataset with labels for both stance and sentiment allows us, for the first time, to conduct an experiment to determine the extent to which stance detection can be solved with sentiment analysis alone.

we determine the performance of an oracle system that assigns stance as follows: For each target, select a sentiment-tostance assignment (mapping all positive instances to 'favor' and all negative instances to 'against' OR mapping all positive instances to 'against' and all negative instances to 'favor') that maximizes the F-macroT score.


There is a counterpart to the claim that 'language is speaker-dependent': speakers are language-dependent.

the type of person someone is can be correlated with their linguistic experience.


This paper offers a model of individuals based on (a subset of) their linguistic experience.

we model how, by being associated with particular types of language data, people develop a uniqueness representable as a vector in a semantic space.


We further contend that what we know about a person conditions the type of interaction we have with them: we are more likely to start a conversation about linguistics with someone we see as a linguist, and to talk about the bad behaviour of our dog with a person we have primarily modelled as a dog trainer.

the model we have of people helps us successfully communicate with them.


For instance, by virtue of being a chef, or someone interested in cooking, someone will have many interconnected experiences related to food.

a good part of their linguistic experiences will involve talking, reading and writing about food.


Document/experience vectors: we posit that the (rough) meaning of a document can be expressed as an additive function acting over (some of) the words of that document.

we sum the 10 words that are most characteristic for the document.


However, we do not exclude the use of other similarity measures and may employ them in future work.

we envisage additional transformations of PoP-constructed spaces to induce vectors with Gaussian distributions (see for instance the logbased PPMI transformation used in the next section).


It also differs from open IE in that the set of properties to be lexicalised is predefined whereas, by definition, in open IE, the set of relations to be extracted is unrestricted.

while we aim to find the possible lexicalisations of a given set of relations (here DBPedia properties), open IE seeks to extract an unrestricted set of relations from text.


To learn word embeddings which capture the similarity between a triple and a set of paraphrases (rather than just the similarity between a triple and artificially synthesised sentences), we multitask the training of our model with the task of paraphrase detection.

the weights of the W matrix for words are learnt with the training of the triple/sentence similarity function S t/s and the training of a similarity function S p for paraphrases which uses the same embedding matrix W for words and is trained on P, the paraphrase corpus.


The phrase similarity function S p between two natural language phrases p i and p j is defined as follows: Similarly to (Bordes et al., 2014), we train our model using a margin-based ranking loss function so that scores of positive examples should be larger than those of negative examples by a margin of 1.

for S t/s , we minimize: where (t i , s i ) is a positive triple/sentence example and (t j , s i ) a negative one.


While antonymy applies to expressions that represent contrasting meanings, paraphrases are phrases expressing the same meaning, which usually occur in similar textual contexts (Barzilay and McKeown, 2001) or have common translations in other languages (Bannard and Callison-Burch, 2005).

if two words or phrases are paraphrases, they are unlikely to be antonyms of each other.


Moreover, the expression of negation varies across languages, as demonstrated by works in Linguistic Typology (Dahl, 1979, inter alia).

negation can appear as a bounded morpheme or a free morpheme; it can precede or follow the verb; it can 'agree' or not in polarity with indefinite pronouns; it can alter the expression of verbal categories (e.g.


In fact, we offer a comparison between this and supervised deep learning architectures that achieve state-of-art scores in the Sentiment Analysis task.

we also evaluate a bi-directional LSTM (Li et al., 2015) on the same task.


3 The choice of the models' hyper-parameters was based on two (contrasting) criteria: i) conservativeness with those proposed in the original models and ii) comparability among the models in this work.

we ensured that each model had the same sentence vector dimensionality: 300.


In some languages, the scores are very scattered: this fluctuation might be due to their peculiar morphological properties.

arabic is an introflexive language, Chinese is a radically isolating language, and Turkish an agglutinative language.


We analyse their role through a visualizasion of the activation in the hidden layer of the bi-LSTM.

we approximated the objective function through a linear function, and estimated the contribution of each word to the true class probability by computing the prime derivative of the output scores with respect to the embeddings.


We construct fixed length representations for the contexts c l and c r by running convolutional fil-ters over C l and C r .

we calculate the column-wise maximum, minimum and the mean over the matrices C l and C r , as done by Tang et al.


We transform initial context-agnostic representations for target word types by taking an elementwise product of the word type vectors with vectors representing the context.

for the context representations learned in Section 3.1, we take an elementwise product of the word type vectors ( w * ) with ( c * ,max , c * ,min , c * ,mean ) where * ∈ {l, r}.


In this paper we propose the first deep learning models for token-level identification of MWEs.

we consider a layered feedforward network, a recurrent neural network, and convolutional neural networks.


In this paper we propose the first deep learning models for broad-coverage MWE identification.

we propose and evaluate a layered feedforward network, a recurrent neural network, and two convolutional neural networks.


(2014b) our aim is to create dense input features to allow neural network architectures, as well as other machine learning algorithms, to be trained on them.

we propose three neural network models: a layered feedforward network (LFN), a recurrent neural network (RNN), and a convolutional neural network (CNN).


After pre-processing, distributed representations of all tokens and lemmas were obtained from a skip-gram (Mikolov et al., 2013) model.

the gensim (Řehůřek and Sojka, 2010) implementation of skip-gram was trained on a snapshot of Wikipedia from September 2015 to learn 100 dimensional word embeddings.


We therefore also include a representation for the entire sentence.

we separately average the skip-gram embeddings for the tokens and lemmas in the sentence containing the target word.


The proposed deep learning models can also be extended to predict supersense tags in addition to the MWE tags.

we intend to compare the performance of a single model that predicts the supersense and MWE tags, versus two separate models for each task.


In this paper, we present work on detecting intensities (or degrees) of emotion in tweets.

given a tweet and an emotion X, the goal is to determine the intensity or degree of emotion X felt by the speaker-a real-valued score between 0 and 1.


Some studies have shown that emoticons tend to be redundant in terms of the sentiment (Go et al., 2009;.

if we remove a smiley face, ':)', from a tweet, we find that the rest of the tweet still conveys a positive sentiment.


We use the hamming diversity metric for decoding at each time step, which penalizes the selection of words that have already been chosen in other beams (Algorithm 1).

the weight λ associated with this metric is tuned to aggressively promote diversity between the first tokens of each of the K generated sequences, thereby avoiding similar beams like I don't know and I really don't know.


This method ensures that for each cross-validation fold, none of the lemmas in that fold's validation lemma-synset pairs are seen in the training split.

for each split we randomly select 5% of training pairs for validation and take the remainder of the training set that does not share a lemma with the validation set as that fold's training instances.


PROTO-AGENT rather than AGENT, where distinctions in proto-roles are based on clusterings of logical entailments.

pROTO-AGENTs often have certain properties in common, e.g., manipulating other objects or willingly participating in an action; pROTO-pATIENTs are often changed or affected by some action.