However, it is not trivial to combine the algebraic objects created by distributional semantics to get a sensible distributional representation for more complex expressions, consisting of several words.

the formalism of the λ -calculus provides us with general, advanced and efficient methods for composition that can model meaning composition not only of simple phrases, but also more complex phenomena such as coercion or composition with fine-grained types (Asher, 2011;Luo, 2010;Bassac et al., 2010).


• Previous research has indicated that the lexical function approach is able to achieve better results using a reduced space with svd.

the negative values that result from svd are detrimental for the multiplicative approach.


The authors evaluated their models using a similarity task that is similar to the one used by Mitchell & Lapata.

they use more complex compositional expressions: rather than using compositions of two words (such as a verb and an object), they use simple transitive phrases (subject-verbobject).


Because the number of rule indicators n o is fairly large (approximately 4000 in the Penn Treebank), the multiplication by W in the model is also expensive.

because only a small number of rules can apply to a given span and split point, f o is sparse and we can selectively compute the terms necessary for the final bilinear product.


Given a sentence x, graph-based models formulates the parsing process as a searching problem: where y * (x) is tree with highest score, Y (x) is the set of all trees compatible with x, θ are model parameters and Score(x,ŷ(x); θ) represents how likely that a particular treeŷ(x) is the correct analysis for x.

the size of Y (x) is exponential large, which makes it impractical to solve equation (1) directly.


The simplest subgraph uses a first-order factorization (McDonald et al., 2005) which decomposes a dependency tree into single dependency arcs ( The most common choice for ScoreF (x, c; θ), which is the score function for subgraph c in the tree, is a simple linear function: where f (x, c) is the feature representation of subgraph c and w is the corresponding weight vector.

the effectiveness of this function relies heavily on the design of feature vector f (x, c).


Le and Zuidema (2014) proprosed an infinite-order model based on recursive neural network.

their model can only be used as an reranking model since decoding is intractable.


Furthermore, because the neural network uses a distributed representation, it is able to model lexical, part-of-speech (POS) tag, and arc label similarities in a continuous space.

although their model outperforms its greedy hand-engineered counterparts, it is not competitive with state-of-the-art dependency parsers that are trained for structured search.


As in prior work, we train the neural network to model the probability of individual parse actions.

we do not use these probabilities directly for prediction.


The word embeddings capture similar distributional information as word clusters and give consistent improvements by providing a good initialization and information about words not seen in the treebank data.

obtaining more training data is even more important than a good initialization.


These sentences are of course easier to parse, having an average length of 15 words, compared to 24 words for the tune set overall.

because we only use these sentences to extract individual transition decisions, the shorter length does not seem to hurt their utility.


(1992), which added an external stack memory to an RNN.

our architecture provides an embedding of the complete contents of the stack, whereas theirs made only the top of the stack visible to the RNN.


Here, we learned from observable stack manipulation operations (i.e., supervision from a treebank), and the computed embeddings of final parser states were not used for any further prediction.

this could be reversed, giving a device that learns to construct context-free programs (e.g., expression trees) given only observed outputs; one application would be unsupervised parsing.


However, removing the nsubj edge in For most dependencies, this semantics can be hard-coded with high accuracy.

there are at least two cases where more attention is warranted.


A standard pipeline for statistical relational learning involves two steps: one first constructs the knowledge base (KB) from text, and then performs the learning and reasoning tasks using probabilistic first-order logics.

a key issue is that information extraction (IE) errors from text affect the quality of the KB, and propagate to the reasoning task.


For example, prepositional phrases (PPs) express crucial information that IE methods need to extract.

pps are a major source of syntactic ambiguity.


The Collins baseline achieves 84% accuracy on the benchmark Wall Street Journal PP dataset.

drawing a distinction in the precision of different prepositions provides useful insights on its performance.


If "with" is used in the sense of "contains" , then the PP is a likely noun attachment, as in n1 contains n2 in the quad ate, cookies, with, cranberries.

if "with" is used in the sense of "accompanied by", then the PP is a likely verb attachment, as in the quad visted, P aris, with, Sue.


To estimate these parameters, we could use the labeled data as training data and use standard gradient descent to minimize the logistic regression cost function.


Both support comparisons, and ideally we can detect some level of similarity.

if we consider only sub-trees, the two dependency trees share in common only two fragments: [#camera] and [is].


PT kernel would find that the parse in Figure 2(a) with all its subgraphs can be matched as a whole within the parse in Figure 2(b), identifying a close match.


usually results in an SRL that generates densely labeled sentences, resulting in better recall but poorer precision.

training data that is sparsely labeled results in an SRL that weighs the option of not assigning a label with higher probability, resulting in better precision and poorer recall.


(2013) applied projection learning for opinion mining in Chinese.

this work only addresses agent detection and requires translating the MPQA corpus.


We ran experiments using the same data and MPQA partitions as Johansson and Moschitti (2013).

since our system is designed for predicting opinion, agents and targets together, we removed the documents that were not annotated with targets.


The work in (Gui et al., 2014) follows the same line although it considers knowledge transferring between two languages.

the main focus of our work is to filter out the noisy knowledge having sentiment changes by wrong translations.


It is reasonable that a Chinese classifier performs better on Chinese text than an English classifier performs on the translated English text due to the different language distributions and MT errors.

as shown in Tables 3 and 4, the better performance of our proposed method compared with that of the self-boosting method further suggests the effectiveness of our proposed knowledge validation model.


Directly employing the translated resources for sentiment classification in the target language is simple and could get acceptable results.

the gap between the source language and target language inevitably impacts the performance of sentiment classification.


Specifically, for a given document pair (x E , x C ), we can learn bilingual embeddings to reconstruct The sum of 5 losses is used as the loss function of bilingual embeddings: In the unsupervised phase, we have learned the bilingual embeddings, which could capture the semantic information within and across languages.

the sentiment polarities of text are ignored in the unsupervised phase.


In this paper, we only evaluate BSWE when dimension d varies from 50 to 500.

there is still space for further improvement if d continues to increase.


We can see that the distribution of factoids relating to venues, methodology and applications is similar for the two datasets.

factoids related to definitional sentences are almost completely missing in the citing sentences data.


Scientific summarization: Early work in scientific summarization used abstracts of scientific articles to produce summaries of specific scientific papers (Kupiec et al., 1995).

later work (Elkiss et al., 2008) showed that citation sentences are as important in understanding the main contributions of a paper.


In the basic version described above, the scorer is trained to score full sentence plan trees.

it is also used to score incomplete sentence plans during the decoding.


Extractive models can generate more readable headlines, because the final title is derived by tailoring human-written sentences.

extractive models give less informative titles (Alfonseca et al., 2013), because sentences are very sparse, making high-recall candidate extraction difficult.


They leverage titles of sub-documents for supervised training.

we generate a title for a single document using an unsupervised model.


The proposed techniques in Section 4 and 5 are generic methodologies and not tied to any particular models such as any sequence models and instanced based models.

because of superior performance over CRF, we use a hidden unit CRF (HUCRF) of Maaten et al.


For instance, one could simply use the sparse feature vector representation of each label.

ccA's low-dimensional projection is computationally more convenient and arguably more generalizable.


In particular,  proposed a frame-semantics based framework for automatically inducing semantic slots given raw audios.

these approaches generally do not explicitly learn the latent factor representations to model the measurement errors (Skrondal and Rabe-Hesketh, 2004), nor do they jointly consider the complex lexical, syntactic, and semantic relations among words, slots, and utterances.


Following their success of unsupervised SLU, recent studies have also obtained interesting results on the tasks of relation detection Chen et al., 2014a), entity extraction (Wang et al., 2014), and extending domain coverage (El-Kahky et al., 2014;Chen and Rudnicky, 2014).

most of the studies above do not explicitly learn latent factor representations from the data-while we hypothesize that the better robustness in noisy data can be achieved by explicitly modeling the measurement errors (usually produced by automatic speech recognizers (ASR)) using latent variable models and taking additional local and global semantic constraints into account.


fluency features can help capture long-range dependencies for disfluency detection.

the UT model does not perform as well as BCT.


But this model suffers from the problem that the number of transition actions is not identical for different hypotheses in decoding, leading to the failure of performing optimal search.

our novel right-to-left transition-based joint method caters to the disfluency constraint which can not only overcome the decoding deficiency in previous work but also achieve significantly higher performance on disfluency detection as well as dependency parsing.


On one hand, an entity is modeled to have multiple internal representations, each regarding one or more closely related facts.

an entity query is decomposed into one or more subqueries, each describing a fact about target entities.


Structure penalty The depth of a factoid node indicates the level of information granularity.

we also need to control the depth of the factoid hierarchy.


One possible reason is that Wikipedia articles contain much irrelevant information regarding entities, and these two existing models cannot easily make use of additional information.

with Wikipedia full-text available, both of our proposed models achieve obviously better performances.


Typical models include voting model (Macdonald and Ounis, 2006), graph model (Serdyukov et al., 2008), etc.

it is not easy to generalize these models for open domain entity retrieval.


Our work is also related to the information extraction and knowledge representation field since our framework involves extraction and aggregation of knowledge from free texts.

most existing approaches takes two extreme ways: either extract relations based on pre-defined ontology, such as DBpedia (Lehmann et al., 2014); or cluster relation without referring to some ontology, such as OpenIE (Etzioni et al., 2011).


Another line of work tries to improve the accuracy by introducing ontologies (Fodeh et al., 2011;Kumar and Salim, 2012) and structured knowledge bases such as WordNet (Nastase et al., 2010), which provide semantic information about words such as synonym  and antonym (Sansonnet and Bouchet, 2010).

these methods primarily rely on special resources constructed with supervision or even manually, which are difficult to expand and in turn limit their applications in practice.


As we will see below, S1, S2, and S3 are error metrics, so lower scores on them imply better performance.

p C is a correlation metric, so higher correlation implies better performance.


For example, one of the groups on the "Matrix" disambiguation page has a name "Business and government" and there is no Wikipedia category by that name.

the group names generated by our (and baseline) method are from the Wikipedia categories (which forms our topic DAG).


8 Our method is robust across different relationships in terms of NDCG.

we observe some variation in ERR as this metric is more sensitive to the distribution of relevant items than NDCG-the distribution over relevance grades varies per relationship type.


Our approach consists of the following steps: (1) generating high quality embedding for each training instance; (2) pre-training with the stacked denoising auto-encoder (Bengio et al., 2003) for feature dimension reduction; and (3) supervised fine-tuning to optimize the neural networks towards a similarity measure (e.g., dot product).

morph resolution is significantly different from the traditional entity linking task since the latter mainly focuses on formal and explicit entities (e.g., "薄 熙 来 (Bo Xilai)") which tend to have stable referents in Wikipedia.


We found that (Huang et al., 2013) failed to resolve many unpopular morphs (e.g., "小马 (Little Ma)" is a morph referring to Ma Yingjiu, and it only appeared once in the data), because it heavily relies on aggregating contextual and temporal information from multiple instances of each morph.

our unsupervised resolution approach only leverages the pre-trained word embeddings to capture the semantics of morph mentions and entities.


(2011) recognize the importance of combining both local and global context for robust disambiguation.

their approach is limited to EL and optimization is performed in a discrete setting.


Note that our formulation could easily be extended to using additional coefficients for each ob-jective.

these hyper-parameters would have to be estimated on development data and therefore, this method could hurt generalization.


They should not be applied during disambiguation because these priors can be very strong and are not domain independent.

they provide a good initialization which is important for successful continuous optimization.


On the other hand, our system is slightly worse on the KORE dataset compared to Babelfy (6 errors more in total), which might be due to the strong priors and   the small context.

the dataset is rather small, containing only 50 sentences, and has been artificially tailored to the use of highly ambiguous entity mentions.


Because the corpora contain more text we find significantly higher overlap with the different keyphrase corpora.

this comes at the cost of not being able to isolate the domainspecific keyphrases.


This is an advantage over softmax classifiers.

sampling informative negative classes/examples can have a significant impact in the effectiveness of the learned model.


This effect of WPEs is reported by (Zeng et al., 2014).

when using only the text span between the target nouns, the impact of WPE is much smaller.


follow the supervised DA paradigm and assume some labeled data in the target domains.

plank and Moschitti (2013) and Nguyen and Grishman (2014) work on the unsupervised DA.


(2014; study the factor-based compositional embedding models.

none of this work examines word embeddings for tree kernels as well as domain adaptation as we do.


The research on negation and speculation identification in English has received a noticeable boost.

in contrast to the significant achievements concerning English, the research progress in Chinese language is quite limited.


Currently, mainstream approaches formulated the scope resolution as a chunking problem, which classifies every word of a sentence as being inside or outside the scope of a cue.

unlike in English, we found that plenty of errors occurred in Chinese scope resolution by using words as the basic identifying candidate.


In this paper, we are concerned with manipulation actions, that is actions performed by agents (humans or robots) on objects, resulting in some physical change of the object.

most of the current AI systems require manually defined semantic rules.


In total, we are able to collect 90 testing detections and they serve as the testing corpus.

since many of the objects used in the testing data are not present in the training set, an object model-free approach is adopted and thus "subject" and "patient" fields are filled with segment IDs instead of a specific object name.


This by itself is a very interesting research topic and it is beyond this paper's scope.

by applying a couple of common sense Axioms on the testing data, we can provide some flavor of this idea.


Here we want to mention a caveat: there are definitely other common sense Axioms that we are not able to address in the current implementation.

from the case studies presented, we can see that using the presented formal framework, our system is able to reason about manipulation action goals instead of just observing what is happening visually.


Both in TransE and TransH, the embeddings of entities and relations are in the same space.

entities and relations are different types objects, it is insufficient to model them in the same space.


In both TransH and TransR/CTransR, all types of entities share the same mapping vectors/matrices.

different types of entities have different attributes and functions, it is insufficient to let them share the same transform parameters of a relation.


The most likely reason for this is that, as the correction of these error types typically only involves the addition or removal of an -s suffix, i.e., a minor change in number morphology, there is very little room for annotators to disagree.

the next highest category, article and determiner errors (ArtOrDet), has a slightly larger confusion set, {the, a/an, }, which may account for the slightly lower score.


Traditional approaches to word sense disambiguation (WSD) rest on the assumption that there exists a single, unambiguous communicative intention underlying every word in a document.

writers sometimes intend for a word to be interpreted as simultaneously carrying multiple distinct meanings.


Yokogawa (2002) describes a system for detecting the presence of puns in Japanese text.

this work is concerned only with puns which are both imperfect and ungrammatical, relying on syntactic cues rather than the lexical-semantic information we propose to use.


The Lesk model had an accuracy of 56%, which is lower than that of a naïve polysemy model which simply selects the punchline with the highest mean polysemy (66%) and even of a random-choice baseline (62%).

it should be stressed here that the Lesk model did not directly account for the possibility that any given word might be ambiguous.


Following prior work on domain adaptation (Blitzer et al., 2006), high-frequent features (unigrams/bigrams) common to both domains are referred to as domain-independent features or pivots.

we use non-pivots to refer to features that are specific to a single domain.


For example, in sentiment classification, words such as excellent or terrible would express similar sentiment about a product irrespective of the domain.

if a pivot expresses different semantics in source and the target domains, then it will be surrounded by dissimilar sets of non-pivots, and reflected in the first criteria.


Despite their impressive performance, existing methods for word representation learning do not consider the semantic variation of words across different domains.

as described in Section 1, the meaning of a word vary from one domain to another, and must be considered.


Finally, the learnt projection matrix is used to find the nearest neighbors in the source domain for each target domain-specific features.

unlike our proposed method, their method does not learn domain-specific word representations, but simply uses co-occurrence counting when creating in-domain word representations.


This work demonstrates the importance of considering the domain specificity of word senses.

the focus of their work is not to learn representations for words or their senses in a domain, but to construct glossaries.


In particular, the number of senses per word type is automatically estimated.

their method is limited to a single domain, and does not consider how the representations vary across domains.


1) is inspired by the prior work on word representation learning for a single domain (Collobert et al., 2011).

unlike the multilayer neural network in Collobert et al.


Spectral Feature Alignment (SFA) (Pan et al., 2010) and Structural Correspondence Learning (SCL) (Blitzer et al., 2007) are the state-ofthe-art methods for cross-domain sentiment classification.

those methods do not learn word representations.


The prevailing methods for the computation of a vector space representation are based on distributional semantics (Harris, 1954).

these approaches, whether in their conventional co-occurrence based form (Salton et al., 1975;Turney and Pantel, 2010;Landauer and Dooley, 2002), or in their newer predictive branch (Collobert and Weston, 2008;Mikolov et al., 2013;Baroni et al., 2014), suffer from a major drawback: they are unable to model individual word senses or concepts, as they conflate different meanings of a word into a single vectorial representation.


There have been several efforts to adapt and apply distributional approaches to the representation of word senses (Pantel and Lin, 2002;Brody and Lapata, 2009;Reisinger and Mooney, 2010;Huang et al., 2012).

none of these techniques provides representations that are already linked to a standard sense inventory, and consequently such mapping has to be carried out either manually, or with the help of sense-annotated data.


Babelfy (Moro et al., 2014) is an approach with state-ofthe-art performance that relies on random walks on BabelNet multilingual semantic network (Navigli and Ponzetto, 2012a) and densest subgraph heuristics.

the approach is limited to the WSD and Entity Linking tasks.


The resulting performance drops have often been addressed via various domain adaptation approaches (Blitzer et al., 2006;Daume III and Marcu, 2006;Reichart and Rappoport, 2007;Chen et al., 2009;Daumé et al., 2010;Chen et al., 2011;Plank and Moschitti, 2013;Hovy et al., 2015b, inter alia).

the authors and target demographics of social media differ radically from those in newswire text, and domain might in some case be a secondary effect to demographics.


In a standard "improvement over baseline"-setup, this would be problematic.

the results should not be interpreted with respect to their absolute value on the respective tasks, but with respect to the relative differences.


With the exception of Hotels and Fashion Accessories, the two distributions are almost bimodal opposites.

they are still significantly correlated (Spearman ρ is 0.49 at p < 0.01).


For many user-generated content settings, this is realistic, since demographic information is available.

we only predict the target variable (sentiment, topic, or author attribute).


We test for significance with the standard cutoff of p < 0.05.

even under a bootstrapsampling test, we can only limit the number of likely false positives.


That leaves us with 790,061 data points for further analysis.

in our semantic model, function words are not affected by the ∆ semantic similarity adjustment and are therefore not analyzable for the effect of semantically-weighted trigram predictability.


Latent variable topic models, such as Latent Dirichlet Allocation (Blei et al., 2003), are popular approaches for automatically discovering topics in document collections.

learning models that capture the large numbers of distinct topics expressed in today's corpora is challenging.


A hierarchical model can make fine-grained distinctions where data is plentiful, and back-off to more coarse-grained distinctions where data is sparse.

current hierarchical models are hindered by computational complexity.


The example includes simplifications we also utilize in our experiments, namely that all nodes at a given depth in the tree have the same number of children and the same δ value.

the inference techniques we present in Section 4 are applicable to any tree T and set of coefficients {δ a }.


Both LDA with collapsed sampling and SBTDM share an advantage in space Algorithm 1 Compute the sampling distribution for a product of two multinomials with SBT priors with Q(z) = 1 for all child c non-zero for ar and a l do ic ← INTERSECT(ar.c, a l .c) i.children += ic τ (i) += τ (ic) end for return i end function complexity: the model parameters are specified by a sparse set of non-zero counts denoting how often tokens of each word or document are assigned to each topic.

in general the sampling distribution for SBTDM has non-uniform probabilities for each of L different latent variable values.


However, in general the sampling distribution for SBTDM has non-uniform probabilities for each of L different latent variable values.

thus, even if many parameters are zero, a naive approach that computes the complete sampling distribution will still take time linear in L. in SBts the sampling distribution can be constructed efficiently using a simple recursive algorithm that exploits the structure of the tree.


The average test log-likelihoods per sentence for these two WSME models are −494 and −509 respectively.

the W-ERs from using the trained WSME models in hypothesis re-ranking are not as poor as would be expected from their PPLs.


It can be seen that Gaussian LDA is a clear winner, achieving an average 275% higher score on average.

we are using embeddings trained on Wikipedia corpus itself, and the PMI measure is computed from co-occurrence in the Wikipedia corpus.


We also noticed that there were certain words ('don', 'writes', etc) which often came as a top word in many topics in classic LDA.

our model was not able to capture the 'space' topics which LDA was able to identify.


We used lexical, POS, syntactic and discourse-based information in the form of treelike structures to learn to differentiate better from worse translations.

in that work we used convolution kernels, which is computationally expensive and does not scale well to large datasets and complex structures such as graphs and enriched trees.


This works for closely related languages, e.g., the English word "new" is translated as "neu" in German and "nueva" in Spanish.

this fails when two languages are not closely related, e.g., Chinese/English.


The time complexity of computing the gradient is O(V e V f ).

significant speedups can be achieved by precomputing v e v T f and exploiting GPUs for Matrix operations.


Given the reordering framework described above, we could try to directly predict the executions as Miceli Barone and Attardi 2013attempted with their version of the framework.

the executions of a given sentence can have widely different lengths, which could make incremental inexact decoding such as beam search difficult due to the need to prune over partial hypotheses that have different numbers of emitted words.


At top level, this is defined as the previous RNN.

the x(j) and x o (j) vectors, in addition to the feature vectors described as above now contain also the final states of another recurrent neural network.


Deception detection has been formulated as a supervised binary classification problem on single documents.

in daily life, millions of fraud cases involve detailed conversations between deceivers and victims.


A common assumption in previous research was that a member is more likely to show a positive attitude toward other members in the same group, and a negative attitude toward the opposing groups (Abu-Jbara et al., 2012a).

a deceiver may pretend to be innocent by supporting those truth-tellers and attacking his teammates, whose identities have already been exposed.


Various abstractive approaches have been proposed till date (Nenkova et al., 2011).

these methods suffer from severe deficiencies.


The higher ROUGE scores imply that WikiKreator is generally able to retrieve useful information from the web, synthesize them and present the important information in the article.

it may also be noted that the Extractive system outperforms the Perceptron framework.


Our ILPbased abstractive summarization system fuses and selects content from multiple sentences, thereby aggregating information successfully from multiple sources.

lexRank 'extracts' the top 5 sentences that results in some information loss.


In that approach, users enter natural language queries in the middle of an existing program; this query drives a search for programs that are relevant to the query and fit within the surrounding program.

the function used to score derivations is a simple matching heuristic relying on the overlap between query terms and program identifiers.


The underspecified descriptions challenge assumptions in synchronous grammars: much of the target structure is implied rather than stated.

the classification method performs quite well.


Finally, it ranks the translations according to the weights of word-meaning pairs and the weights of the CCG parse trees.

test sentences may contain words which were not present in the training set.


Most previous work of text normalization on informal text made a strong assumption that the system has already known which tokens are non-standard words (NSW) and thus need normalization.


Short text messages or comments from social media websites such as Facebook and Twitter have become one of the most popular communication forms in recent years.

abbreviations, misspelled words and many other non-standard words are very common in short texts for various reasons (e.g., length limitation, need to convey much information, writing style).


This is also the reason why in Table  3, the performance of 3-way classification is significantly better than that of the two-step method using all the features.

we also find that when we only use lexical features (2∼10), the two methods have similar performance on Test set 2, but the two-step method has much better performance than the 3-way classifier method on Test set 1.


It matches the pair of strings (B): each aligned pair of wildcards is substituted in source and target sentences by the same word and string patterns of (A) can indeed be turned into pairs of substrings of the sentences.

it cannot match the pair of sentences (C) in the original kb-SRK.


For example, it is unlikely that replacing a word in a pattern of a rewriting rule by one of its holonyms will yield a semantically similar rewriting rule, so holonym would not be a good pattern type for most applications.

it can be very useful in a rewriting rule to type a wildcard link with the relation holonym, as this provides constrained semantic roles to the linked wildcards in the rule, thus holonym would be a good variable type.


the pair "Former French president General Charles de Gaulle died in November.


In the Flickr-100M dataset, tags are assigned to images and videos in the form of sets of words, rather than grammatically coherent sentences.

the roles that individual words play are still discernible from their visual context, as manifested by the other words in a given set.


hair or fabric) are emphasized by the visual features.

the model based on visual features alone performs poorly on the dataset of Keller and Lapata (2003).


Taking the argument classes produced by the linguistic model as a basis and then re-ranking them to incorporate visual statistics helps to avoid the above problem for the interpolated models, whose output corresponds to grammatical relations.

static interpolation weights (emphasizing linguistic features over the visual ones for all verbs equally) outperformed the predicate-driven interpolation technique, attaining correlations of r = 0.548 and r = 0.476 respectively.


In the future, it would be interesting to derive the information about predicate-argument relations from low-level visual features directly.

to our knowledge, reliably mapping images to actions (i.e.


Existing methods for Japanese predicate argument structure (PAS) analysis identify case arguments of each predicate without considering interactions between the target PAS and others in a sentence.

the argument structures of the predicates in a sentence are semantically related to each other.


In this respect, C-BOW extends the distributional hypothesis (Harris, 1954) that words with similar context distributions should have similar meanings to longer sequences.

the word combinations of C-BOW are not natural linguistic constituents, but arbitrary n-grams (e.g., sequences of 5 words with a gap in the middle).


(Moschitti, 2006), where p is the largest subsequence of children that we want to consider and ρ is the maximal outdegree observed in the two trees.

the average running time tends to be linear for natural language syntactic trees (Moschitti, 2006).


This kernel works in the space of the union of the sets of all subtrees from the upper and lower trees, e.g.

: such features cannot capture the relations between the constituents (or semantic lexical units) from the two trees.


: However, such features cannot capture the relations between the constituents (or semantic lexical units) from the two trees.

these are essential to learn the relation between the two entire sentences 2 .


Moreover the unavailability of the used resources and the opacity of the used rules have also made such systems very difficult to replicate.

the models we propose enable researchers to: (i) build their system without the use of specific resources.


These systems are also adaptable and easy to replicate, but they are subject to an exponential computational complexity and can thus only be used on very small datasets (e.g., they cannot be applied to the MSR Paraphrase corpus).

the model we proposed in this paper can be used on large datasets, because its kernel complexity is about linear (on average).


To some extent, this can be seen a specific kind of context information.

they ignore the label dependence by directly applying Binary Relevance to overcome the multi-label classification difficulty.


But, in both cases, we can consider them as a user's like.

among the expressions of attitudes where the source is the user, some of them do not refer to a like or dislike.


The disfluent structure of the sentence could thus be integrated to our syntactic and semantic rules.

the automatic detection of disfluencies is still an open challenge, in particular in the case of edit disfluencies where the speaker corrects or alters the utterance or abandons it entirely and starts over (Strassel, 2004).


Our system offers a first step in the integration of the interaction context by considering jointly the user's utterance and the previous agent's one that allow us to correctly analyse a large scale of expressions.

the system and the annotators have to focus on the APs without considering the preceding speech turns, which can cause disagreements not only between the system outputs and the human annotations, but also between the human annotators.


(2010) were able to improve near state-of-the-art systems for several tasks, by simply plugging in the learned word representations as additional features.

because these features are estimated by minimizing the prediction errors made on a generic, unsupervised, task they might be suboptimal for the intended purposes.


• We show that the features derived from crowdsourced transcriptions perform as well as crowd grades in predicting expert grades.

crowd grades add additional predictive value.


Further, among the crowdsourcing approaches, we find that the crowd-grades (Model RR-3) does equivalently well (and sometimes worse) than the model using features derived from the crowdsourced speech (Model RR-4).

when we combine all the features from crowdsourcing including the crowd grades, we find much better prediction accuracy (r = 0.76).


Their results show that the former model can recognize MWEs with F1=71.1%, while the latter can significantly improve parsing accuracy and robustness in general.

the authors admit that "it remains to be seen how much of theoretically possible improvement can be realized when using automatic methods for MWU recognition".


They often are pregrouped as words-with-spaces in many parsing architectures (Sag et al., 2002).

we did not use gold tokenization, unrealistic for ambiguous MWEs (Nivre and Nilsson, 2004 Table 7: MORPH link prediction for de+DET constructions: precision of global majority baseline, precision of individual per-construction baseline, precision of Green et al.


The location of an argument on syntactic tree provides an intermediate tag for improving the performance.

building this syntactic tree also introduces the prediction risk inevitably.


This pipeline addressed the data sparsity by initializing the model with word embeddings which is trained from large unlabeled text corpus.

the convolution layer is not the best way to model long distance dependency since it only includes words within limited context.


Without y (t−1) term, the rnn model returns to the feed forward form.


In all curves, performance degrades with increased sentence length.

the performance gain of our model over the baseline model is larger for longer sentences.


Smaller d s means that it is easy to make prediction that long history is unnecessary.

large d s results in a difficult prediction that long historical information is needed.


On one hand, LSTM network is capable of capturing the long distance dependency especially in its deep form.

the traditional feature templates are only good at describing the properties in neighborhood and a small mistake in syntactic tree will results in large deviation in SRL tagging.


In practice, however, our model does not work well if it is only trained on the manually annotated Treebank data sets.

when pre-trained on a large amount of automatically parsed data and then fine-tuned on the Treebank data sets, our model achieves a fairly large improvement in performance.


Third, we utilize the Dropout strategy to address the overfitting prob-lem.


(2013) built a recursive neural network for constituent parsing.

rather than performing full inference, their model can only score parse candidates generated from another parser.


Our model also requires a parser to generate training samples for pre-training.

our system is different in that, during testing, our model performs full inference with no need of other parsers.


(2012) propose a static postparsing analysis to categorise groups of bracket errors in constituency parsing into higher level error classes such as clause attachment.

this cannot account for cascading changes resulting from repairing errors, or limitations which may prevent the parser from applying a repair.


McDonald and Nivre (2011) perform an indepth comparison of the graph-based MSTparser and transition-based MaltParser.

malt-Parser uses support vector machines to deterministically predict the next transition, rather than storing the most probable options in a beam like ZPar.


PPs and coordination have high effective constraint percentages relative to the other error classes for both parsers.

they are also amongst the most isolated errors, with only 0.3% and 0.4% ∆u for MSTparser and ZPar respectively.


Both parsers make few root attachment errors, though MSTparser is less accurate than ZPar.

root constraints provide the largest UAS improvement per number of constraints for both parsers.


Feature-based discriminative supervised models have achieved much progress in dependency parsing (Nivre, 2004;Yamada and Matsumoto, 2003;McDonald et al., 2005), which typically use millions of discrete binary features generated from a limited size training data.

the ability of these models is restricted by the design of features.


The combination is relative simple and its correctness can be measured with the final representation of the non-terminal node (Socher et al., 2013a).

for dependency parsing, all combinations of the head h and its children c i (0 < i ≤ K) are important to measure the correctness of the subtree.


When k is larger, the number of negative samples also needs to multiply increase for training.

we just can obtain at most k negative samples from the k-best outputs of the base parser.


Specific to the re-ranking model, Le and Zuidema (2014) proposed a generative re-ranking model with Inside-Outside Recursive Neural Network (IORNN), which can process trees both bottom-up and top-down.

iORNN works in generative way and just estimates the probability of a given tree, so iORNN cannot fully utilize the incorrect trees in k-best candidate results.


Previous models rely heavily on richer syntactic information through lexicalizing rules, splitting categories, or memorizing long histories.

enriched models incur numerous parameters and sparsity issues, and are insufficient for capturing various syntactic phenomena.


A popular parsing algorithm is a cubic time chartbased dynamic programming algorithm that uses probabilistic context-free grammars (PCFGs).

pCFGs learned from treebanks are too coarse to represent the syntactic structures of texts.


Inspired by CVG (Socher et al., 2013), we differentiate the matrices for each non-terminal (or POS) label X rather than using shared parameters.

our model differs in that the parameters are untied on the basis of the left hand side of a rule, rather than the right hand side, because our model assigns a score discriminatively for each action with the left hand side label X unlike a generative model derived from PCFGs.


As can be seen, the greater word representation dimensions are generally helpful for both WSJ and CTB on the closed development data (dev), which may match with our intuition that the richer syntactic and semantic knowledge representation for each word is required for parsing.

overfitting was observed when using a 32-dimension hidden vector in both tasks, i.e., drops of performance on the open test data (test) when m = 1024, probably caused by the limited generalization capability in the smaller hidden state size.


If the NP her goals is a likely ARG1 of realized the parser should prefer the main clause structure.

if the NP is a likely ARG0 of an (as yet unseen) embedded verb, then the parser should go for the subordinate clause structure.


Fringes capture the fact that in an incremental derivation, a prefix tree can only be combined with an elementary tree at a limited set of nodes.

for instance, the prefix tree in figure 1 has two substitution nodes, for B and C. only substitution into B leads to a valid new prefix tree; if we substitute into C, we obtain the tree in figure 1b, which is not a valid prefix tree (i.e., it represents a non-incremental derivation).


Volokh and Neumann (2008) use a variant of Nivre's (2007) incremental shift-reduce parser and rely only on the current word and previous content to output partial dependency trees; then they output role labels given the full parser output.

to all the joint approaches, we perform both parsing and semantic role labeling strictly incrementally, without having access to the whole sentence, outputting prefix trees and iSRL triples for every sentence prefix.


A category-based evaluation of discontinuous constituents reveals that EX-TENDED has an advantage over DISCO when considering all constituents.

we can also see that the DISCO features yield better results than EXTENDED particularly on the frequent discontinuous categories (NP, VP, AP, PP), which indicates that the information about gap type and gap length is useful for the recovery of discontinuities.


EXTENDED and DISCO yields an improvement on all constituents.

now not only DISCO, but also EXTENDED lead to improved scores on discontinuous constituents.


The parsing speed on the test set drops to around 39 sentences per second.

we achieve 75.10 F 1 , i.e., a slight improvement over the experiments in Tab.


In comparison to the baseline setting of the shift-reduce parser with beam size 8, the results are around 10 points worse.

rparse reaches an F 1 of 26.61 on discontinuous constituents, which is 5.9 points more than we achieved with the best setting with our parser.


The key difference is that Chen and Manning (2014) is a local classifier that greedily optimizes each action.

zhang and Nivre (2011) leverage a structured-prediction model to optimize whole sequences of actions, which correspond to tree structures.


The model size of ZPar (Zhang and Nivre, 2011) is over 250 MB on disk.

the model size of our structured neural parser is only 25 MB.


Bohnet and Nivre (2012) obtain an accuracy of 93.67%, which is higher than our parser.

their parser is a joint model of parsing and POS-tagging, and they use external data in parsing.


The input embeddings of our parser are also trained over large raw text, and in this perspective our model is correlated with the semi-supervised models.

because we fine-tune the word embeddings in supervised training, the embeddings of in-vocabulary words become systematically different from these of out-of-vocabulary words after training, and the effect of pre-trained out-ofvocabulary embeddings become uncertain.


Using the Viterbi algorithm, they can compute the exponential partition function in linear time without approximation.

with a dynamic programming decoder, their sequence labeling model can only extract local features.


However, with a dynamic programming decoder, their sequence labeling model can only extract local features.

our integrated approximated search and learning framework allows rich global features.


The idea of crosslingual transfer using the parser we examined above is straightforward.

to traditional approaches that have to discard rich lexical features (delexicalizing) when transferring models from one language to another, our model can be transferred using the full model trained on the source language side, i.e.


Contrary to the projection approach, CCA assigns embeddings for every word in the monolingual vocabulary.

one potential limitation is that CCA assumes linear transformation of word embeddings, which is difficult to satisfy.


Otherwise, the translational equivalence will be broken.


This design biases the model to learn relations at a granularity optimized for the machine comprehension task.

to a generic discourse analyzer, our method can also utilize additional information available in the machine comprehension context.


4 Ideally, we would be able to consider all possible pairs of sentences in a given paragraph.

to reduce computation costs in practice, we use a sentence window k and consider only sentences that are at most k away from each other.


On the MC500 test set, we again find that model 3, with a score of 63.75%, provides a gain of 3.5% over SWD and is comparable to the performance of RTE+SWD (63.33%) The importance of utilizing multiple relevant sentences to score answers is evident from the higher scores of models 2 and 3 on multi type questions in both test sets.

model 1, which retrieves only a single relevant sentence for each question, achieves the best scores on the single type questions up to 83.25% on MC160 test.


In MC160, the accuracy is even higher for what-questions (almost 80%).

the model does slightly worse on whyquestions, with only 60% accuracy.


This is likely because these questions often have characteristic words occurring in the sentences (such as here, there, after, before, him, her).

questions asking how, which and why have lower recalls since they often involve reasoning over multiple sentences.


Finally they reported an F-score of 8% for role linking.

being strongly lexicalized, their trained model seems heavily dependent on the training data.


(1) is non-convex due to the terms a and b that interact with each other, so it cannot be solved exactly using a standard optimization technique.

a method based on SVD provides an efficient and exact solution.


But the likelihood function under the Brown model is non-convex, making an MLE estimation of the model parameters difficult.

the hard-clustering assumption (Assumption 4.1) allows for a simple Theorem 4.1.


Though the shortest path can be selected, it ignores other related category nodes and loses rich information.

an ideal scheme should not only mirror the distance in the hierarchy, but also take into account all possible paths in order to capture the full aspects of relatedness.


These models share the property of producing a single general representation for each word, which can be utilized in a variety of tasks, from POS tagging to semantic role labelling.

here we attempt to decompose the representations into separate seman- tic and syntactic components.


that we can approximate smaller − small with bigger − big, or smaller − bigger with small − big.

knowing that opposite sides of the square in Figure 1 are parallel to each other still leaves open the question of what happens at the corners.


's (2013b) method for making the word-analogy predictions in terms of addition and subtraction: smaller ≈ bigger − big + small.

in the case of the CBSOW and CBSOWM models, we use the novel approach described in Section 3.5: v smaller ≈ b small ⊕ s bigger .


There is also no Freebase type representing the concept 'municipalities.'

this word is associated with an entity in Freebase.


They report that 97% of the attributes in Freebase are commonly expressed as noun phrases.

unlike our work, they considered open information extraction and did not ground the extractions in an external KB.


Several semantic parsing methods use a domainindependent meaning representation derived from the combinatory categorial grammar (CCG) parses (e.g., (Cai and Yates, 2013;Kwiatkowski et al., 2013;Reddy et al., 2014)).

our query graph design matches closely the graph knowledge base.


These biases are not codified, which results in an idiosyncratic and mysterious user experience, a major drawback of natural language interfaces (Rangel et al., 2014).

our compact grammar precisely specifies the logical functionality.


Although this is a marked improvement in cost and scalability compared to annotated logical forms, it still requires non-trivial effort: the annotator must (i) understand the question and (ii) figure out the answer, which becomes even harder with compositional utterances.

our main source of supervision is paraphrases, which only requires (i), not (ii).


Table 2 might evoke rule-based systems (Woods et al., 1972;Warren and Pereira, 1982) or controlled natural languages (Schwitter, 2010).

there is an important distinction: the grammar need only connect a logical form to one canonical utterance; it is not used directly for parsing.


The simple RNN has the ability to capture context information.

the length of reachable context is often limited.


By using stock prices from Yahoo Finance, they annotated all the news in a transaction date with going up or down categories.

the weakness of this assumption is that all the news in one day will have the same category.


They were integrated into the regression model.

they concluded that their model does not successfully predict stock returns.


They concluded that the ratio of the emotional tweets significantly negatively correlated with Down Jones, NASDAQ and S&P 500, but positively with VIX.

they did not use their model to predict the stock price values.


Their model achieved around 75% accuracy.

their test period was short, from 8 th to 26 th in September 2012, containing only 14 transaction dates.


A sentiment time series was built based on these topics.

the time period of their whole dataset is rather short, only three months.


MaxEnt-LDA hybrid model can jointly discover both aspects and aspectspecific opinion words on a restaurant review dataset (Zhao et al., 2010), while FACTS, CFACTS, FACTS-R, and CFACTS-R model were proposed for sentiment analysis on a product review data (Lakkaraju et al., 2011).

one of the weaknesses of these methods is that there is only one opinion word distribution corresponding to one topic (aspect).


The results showed that the model using mood information outperformed the model without mood by 3.57%, 3.58%, 14.29% and 12.5% accuracy for XOM, EBAY, IBM and KO stock, respectively.

the performance on DELL stock was not improved.


For example, 'yahoo', 'ko' and 'finance' are highly associated with the distribution defined by hidden sentiment 1 and hidden topic 1.

it is rather difficult to guess which sentiment or topic in this joint distribution actually means.


As mentioned above, tags of parent nodes have impact on composition.

some phrases with the same tag should be composed in different ways.


Mention pair models that predict whether or not two mentions are coreferent have historically been very effective for coreference resolution, but do not make use of entity-level information.

we show that the scores produced by such models can be aggregated to define powerful entity-level features between clusters of mentions.


Even when using DAgger, this problem could exist to a lesser degree if the model heavily overfits to the training data.

the agent has a small number of parameters thanks to our model stacking approach, reducing the risk of this happening.


We assign each action the regret r associated with taking that action under the current policy as a cost: The "rolling out" procedure means we naively have to visit O(t 2 ) states each iteration instead of t, where t is the length of a trajectory.

the highly constrained action space described in section 3.1 combined with the use of memoization allows the algorithm to still run efficiently.


In total, there are only 56 features after the feature conjunction.

these features provide strong signal because they are directly related to the probabilities of mentions being coreferent.


These systems often use some form of pretraining for initialization, often word-embeddings learned from external tasks.

there has been little work of this form for coreference resolution.


While early work focused primarily on English (Soon et al., 2001;Ng and Cardie, 2002), efforts have been made toward multilingual systems, this being addressed in recent shared tasks Pradhan et al., 2012).

the lack of annotated data hinders rapid system deployment for new languages.


Since HIPTM can no longer access the votes in the test data, its performance drops significantly compared with VOTE.

it still quite strongly outperforms the two text-based baselines, showing that jointly modeling the voting behavior improves the text-based elements of the model.


The most Tea Party oriented frame node, M3, focuses on criticizing government overspending, a recurring Tea Party theme.

13 Frame M1, least oriented toward the Tea Party, focuses on the downsides of a government shutdown, highlighting establishment Republican concerns about being held responsible for the political and economic consequences.


Several extensions of LDA have been proposed that assign topics not only to individual words but also to multi-word phrases, which we call topical collocations.

as we will discuss in section 2, most of those extensions either rely on a pre-processing step to identify potential collocations (e.g., bigrams and trigrams) or limit attention to bigram dependencies.


found that this modification allowed TNG to outperform LDACOL on a standard information retrieval task.

both LDACOL and TNG do not require words within a sequence to share the same topic, which can result in semantically incoherent collocations.


For example, if the target answer is a small number (say, 2), it is possible to count the number of rows with some random properties and arrive at the correct answer.

as the system encounters more examples, it can potentially learn to disfavor them by recognizing the characteristics of semantically correct logical forms.


For example, the decomposition grammar for s=3 for AMR #194 in the corpus has over 300 million rules.

many uses of decomposition grammars, such as sampling for grammar induction, can be phrased purely in terms of top-down queries.


Finally, we analyzed the asymptotic runtimes in Table 1 in terms of the maximum number d • s of in-boundary edges.

the top-down parser does not manipulate individual edges, but entire s-components.


These methods can efficiently estimate the co-occurrence statistics to model contextual distributions from very large text corpora and they have been demonstrated to be quite effective in a number of NLP tasks.

they still suffer from some major limitations.


These differences are further exhibited in the confusion matrices shown in Figure 4; when the classifier is trained using only monolingual features, it misclassifies 26% of ¬ pairs as ⌘, whereas the bilingual features make this error only 6% of the time.

the bilingual features completely fail to predict the A class, calling over 80% of such pairs ⌘ or ⇠.


Other approaches eliminate the discontinuities via tree transformations (Boyd, 2007;Kübler et al., 2008), sometimes as a pruning step in a coarse-to-fine parsing approach (van Cranenburgh and Bod, 2013).

reported runtimes are still superior to 10 seconds per sentence, which is not practical.


In existing parsers, features are commonly exploited from the parsing history, such as the top k elements on the stack.

such features are expensive in terms of search efficiency.


In practice, this global model is much stronger than the local MaxEnt model.

training this model without any approximation is hard, and the common practice is to rely on well-known heuristics such as an early update with beam search (Collins and Roark, 2004).


7 Every experiment reported here was performed on hardware Feature We borrow the feature templates from Sagae and Lavie (2006).

we found the full feature templates make training and decoding of the structured perceptron much slower, and instead developed simplified templates by removing some, e.g., that access to the child information on the second top node on the stack.


This is particularly true with DP; it sometimes outperforms Z&C, probably because our simple features facilitate state merging of DP, which expands search space.

our main result that the system with optimal search gets a much higher score (90.7 F1) than beambased systems with a larger beam size (90.2 F1) indicates that ordinary beam-based systems suffer from severe search errors even with the help of DP.


Traditionally, CCG graphs are generated as a by-product by deep parsers with a core grammar (Clark et al., 2002;Clark and Curran, 2007b;Fowler and Penn, 2010).

modeling these dependencies within a CCG parser has been shown very effective to improve the parsing accuracy (Clark and Curran, 2007b;Xu et al., 2014).


This part of information is very similar to Semantic Role Labeling (SRL), whose goal is to find semantic roles for verbal predicates as well as their normalization.

functor-argument analysis grounded in CCG is approximation of underlying logic forms and thus provides bi-lexical relations for almost all words.


We also use the syntactic dependency trees provided by the CCGBank to obtain necessary information for graph parsing.

different from experiments in the CCG parsing literature, we use no grammar information.


The only underlying LSTM structure that has been explored so far is a linear chain.

natural language exhibits syntactic properties that would naturally combine words to phrases.


In bag-of-words models, phrase and sentence representations are independent of word order; for example, they can be generated by averaging constituent word representations (Landauer and Dumais, 1997;Foltz et al., 1998).

sequence models construct sentence representations as an order-sensitive function of the sequence of tokens (Elman, 1990;Mikolov, 2012).


For example, this allows the left hidden state in a binary tree to have either an excitatory or inhibitory effect on the forget gate of the right child.

for large values of N , these additional parameters are impractical and may be tied or fixed to zero.


TIME-FLOW is aware of the sequential direction, inherited from the space-awareness of CNN, but it is not sensitive enough about the prediction task, due to the uniform weights in the convolution.

tIME-ARROW, living in location-dependent parameters of convolution units, acts like an arrow pin-pointing the prediction task.


The work was done when Weiwei Guo was in Columbia University summarization systems adopt the extractionbased approach which selects some original sentences from the source documents to create a short summary (Erkan and Radev, 2004;Wan et al., 2007).

the restriction that the whole sentence should be selected potentially yields some overlapping information in the summary.


These features help them select better category-specific content for the summary.

the usability of such features depends on the availability of predefined categories in the summarization task, as well as the availability of training data with the same predefined categories for estimating feature weights.


Later, unified models are proposed to conduct sentence selection and redundancy control simultaneously (McDonald, 2007;Filatova and Hatzivassiloglou, 2004;Yih et al., 2007;Gillick et al., 2007;Lin and Bilmes, 2010;Lin and Bilmes, 2012;Sipos et al., 2012).

extraction-based approaches are unable to evaluate the salience and control the redundancy on the granularity finer than sentences.


For example, some background articles on Mubarak's step-down will likely explain the reasons behind it.

extracting such causal information can be difficult, as demonstrated by the still low results for discourse relation extraction (Lin et al., 2014;Braud and Denis, 2014).


That process leverages how good this connection is in the sub-graph (G*) which consists of X and its outgoing neighbors.

our proposed model uses the non-normalized value of the influence score to leverage how good this connection is on the entire graph instead of G*.


(2012) use 91 timelines from AFP as ground truth along with the AFP news corpus for feature extraction.

their dataset is not publically available.


7 Overall, the recency salience measure provides a better fit than the frequency salience measure with respect to accuracies, suggesting that recency better captures speakers' representations of discourse salience that influence choices of referring expressions.

the models with frequency discourse salience have higher model log likelihood than the models with recency do.


Experiments with the adult-directed news corpus show a close match between speakers' utterances and model predictions.

experiments with child-directed speech show that the models were more likely to predict proper names where pronouns were used, suggesting that the estimates of discourse salience using simple measures were not sufficient to capture a conversation.


A major focus in computational social science has been the study of interpersonal relations through data.

social interactions are complicated, and we rarely have access all of the data that define the relationship between friends or enemies.


They have been good allies for the better part of the game.

immediately after this exchange, Austria suddenly invades German territory.


Previous studies mainly relied on shallow textual clues in Twitter posts in order to predict the number of flu infections, e.g., the number of occurrences of specific keywords (such as "flu" or "influenza") on Twitter.

such a simple approach can lead to incorrect predictions.


For example, given the sentence "I caught a cold," we would predict that the first person ("I," i.e., the poster) is the subject (carrier) of the cold.

we can ignore the sentence, "The TV presenter caught a cold" only if we predict that the subject of the cold is the third person, who is at a different location from the poster.


In addition to the subject labels, we annotated the text span that indicates a subject.

the subjects of diseases/symptoms are often omitted in tweet texts.


The likelihood of a positive episode is extremely high when the subject label of a disease/symptom is FIRSTPERSON (85.1%) or NEARBYPERSON (76.7%).

fAR-AWAYPERSON, NONHUMAN, and NONE subjects represent negative episodes (less than 5.0%).


Similarly, the subject labels improved the performance for "fever".

the subject labels did not improve the performance for headache and runny nose considerably.


That outcome may be achieved through strategies characterized in terms of conversation acts or language with particular stylistic characteristics.

individual acts by themselves lack the power to achieve a complex outcome.


Theory on coordination in groups and organizations emphasizes role differentiation, division of labor and formal and informal management (Kittur and Kraut, 2010).

identification of roles as such has not had a corresponding strong emphasis in the language technologies community, although there has been work on related notions.


What is similar between stances and personas on the one hand and roles on the other is that the unit of analysis is the person.

they are distinct in that stances (e.g., liberal) and personas (e.g., lurker) are not typically defined in terms of what they are meant to accomplish, although they may be associated with kinds of things they do.


Examples include such things as mixed membership stochastic blockmodels (MMSB) (Airoldi et al., 2008), similar unsupervised matrix factorization methods (Hu and Liu, 2012), or semi-supervised role inference models (Zhao et al., 2013).

these approaches do not standardly utilize an outcome as supervision to guide the clustering.


As in NBOW, each word type has an associated embedding.

the composition function g now depends on a parse tree of the input sequence.


Theoretically, word dropout can also be applied to other neural network-based approaches.

we observe no significant performance differences in preliminary experiments when applying word dropout to leaf nodes in RecNNs for sentiment analysis (dropped leaf representations are set to zero vectors), and it slightly hurts performance on the question answering task.


But if we consider that the classifier can only improve its goodness of fit with more features (the sets of features being nested as the support varies), it is likely that the lowest support will lead to the best test accuracy; assuming subsequent regularization to prevent overfitting.

this will come at the cost of an exponential number of features as observed in practice.


In cross-lingual POS tagging, mostly annotation projection has been explored (Fossum and Abney, 2005;Das and Petrov, 2011), since all features in POS tagging models are typically lexical.

using bilingual word representations was recently explored as an alternative to projectionbased approaches (Gouws and Søgaard, 2015).


(2014), in the context of monolingual dependency parsing, investigate continuous word representation for dependency parsing in a monolingual cross-domain setup and compare them to word clusters.

to make the embeddings work, they had to i) bucket real values and perform hierarchical clustering on them, ending up with word clusters very similar to those of ; ii) use syntactic context to estimate embeddings.


Traditionally, such scenarios call for dynamic programming for exact inference.

preliminary experiments showed that, for our model, a Viterbi search based segmenter, even supported by conditional random field (Lafferty et al., 2001) style training, yields similar results as the greedy search based segmenter in this section.


Recently, neural network models for natural language processing tasks have been increasingly focused on for their ability of alleviating the burden of manual feature engineering.

the previous neural models cannot extract the complicated feature compositions as the traditional methods with discrete features.


GPs usually outperform SVMs by a small margin.

these offer the advantages of not using the validation set and the interpretability properties we highlight in the next section.


doctors and nurses or accountants and assistant accountants.

with very few exceptions, we notice that only adjacent classes get misclassified, suggesting that our model captures the general user skill level.


This is expected because the vast majority of jobs in these classes require a university degree (holds for all of the jobs in classes 2 and 3) or are actually jobs in higher education.

classes 5 to 9 have a similar behaviour, tweeting less on this topic.


Differences arise either due to language use or due to the topics people discuss as parts of various social domains.

a large scale investigation of this hypothesis has never been attempted.


We acknowledge that the derivations of this study, similarly to other studies in the field, are reflecting the Twitter population and may experience a bias introduced by users self-mentioning their occupations.

the magnitude, occupational diversity and face validity of our conclusions suggest that the presented approach is useful for future downstream applications.


By formulating such ideas as search or MDL problems of given coding length 1 , word boundaries are found in an algorithmic fashion (Zhikov et al., 2010;Magistry and Sagot, 2013).

such methods have difficulty incorporating higher-order statistics beyond simple heuristics, such as word transitions, word spelling formation, or word length distribution.


Moreover, they usually depends on tuning parameters like thresholds that cannot be learned without human intervention.

statistical models are ready to incorporate all such phenomena within a consistent statistical generative model of a string, and often prove to work better than heuristic methods (Goldwater et al., 2006;Mochihashi et al., 2009).


(2014) proposed an intermediate model between heuristic and statistical models as a product of character and word HMMs.

these two models do not have information shared between the models, which is not the case with generative models.


For example, in Japanese can be segmented into not only / / / (plum/too/peach/too), but also into / / (plum/peach/peach), which is ungrammatical.

we could exclude the latter case 1 For example, Zhikov et al.


Since segment models like NPYLM have segment lengths as hidden states, they are called semi-Markov models (Murphy, 2002).

our model also has hidden part-of-speech, thus we call it a Pitman-Yor Hidden Semi-Markov model (PYHSMM).


In many cases PYHSMM found more "natural" segmentations, but it does not always conform to the gold annotations.

it often oversegments emotional expressions (sequence of the same character, for example) and this is one of the major sources of errors.


CTB was designed to serve syntactic analysis, whereas PD was developed to support information extraction systems.

the key challenge of exploiting the two resources is that they adopt different sets of POS tags which are impossible to be precisely converted from one to another based on heuristic rules.


Since a looser mapping function leads to a larger number of bundled tags and makes the model slower, we implement a paralleled training procedure based on Algorithm 1, and run each experiment with five threads.

it still takes about 20 hours for one iteration when using the complete mapping function; whereas the other three mapping functions need about 6, 2, and 1 hours respectively.


Particularly, when M ′ = 1K, the model converges very slowly.

from the trend of the curves, we expect that the accuracy gap between our coupled model with M ′ = 5K/20K and the baseline model should be much smaller when reaching convergence.


Examples for word embeddings are SENNA (Collobert and Weston, 2008), the hierarchical log-bilinear model (Mnih and Hinton, 2009), word2vec (Mikolov et al., 2013c) and GloVe (Pennington et al., 2014).

there are many other resources that are undoubtedly useful in NLP, including lexical resources like WordNet and Wiktionary and knowledge bases like Wikipedia and Freebase.


They are rank 4 tensors of size ≈10 15 .

we already discussed that they are very sparse, for two reasons: (i) We make the assumption that there is no interaction between dimensions.


The learning objective becomes: The hidden layer S of the autoencoder gives us synset embeddings.

the lexeme embeddings are defined when transitioning from W to S, or more explicitly by: there is also a second lexeme embedding in AutoExtend when transitioning form S to W : Aligning these two representations seems natural, so we impose the following lexeme constraints: this can also be expressed dimension-wise.


Hassan and Mihalcea (2009) built two sets of cross-lingual datasets by translating the English MC-30 (Miller and Charles, 1991) and the WordSim-353 (Finkelstein et al., 2002) datasets into three languages.

these datasets have several issues due to their construction procedure.


k = 3), i.e., the same number of possible senses.

the number of senses in WordNet (Miller, 1995) varies from 1 such as "ben" to 75 such as "break".


(2) The initial value of sense representation is critical for most statistical clustering based approaches.

previous approaches usually adopted random initialization (Neelakantan et al., 2014) or the mean average of candidate words in a gloss .


One possible reason is that we set the number of context clusters for each word to be the same as the number of its corresponding senses in WordNet.

not all senses appear in the our experimented corpus which could lead to fragmented context clustering results.


(2015), that implement very similar ideas.

one major difference between their work and ours is that their strategy is in the same direction of (Yih et al., 2012), which might result in poor performance on general semantic tasks.


the input vector, not unlike what arithmetic negation would do.

the Skip-grambased not matrix is remarkably identity-like, with large positive values concentrated on the diagonal.


Specifically, each participant in the dialogue usually has specific sentiment polarities towards different topics.

most existing sequential data modeling methods are not capable of incorporating the information from both the topic and the author's identity.


Comparing to traditional HMM-based method, it explores deeply into the structure of sentences, and is more flexible in taking external features into account.

it doesn't suffer from the training difficulties of recurrent neural networks, such as the vanishing gradient problem.


In particular, we assume that there is a labeling matrix L such that and a transition matrix T such that These two equations establish the relation between the hidden state and the labels.

we use a neural network model M to model the relation between the hidden states and the observations.


One naive approach is to use aggregated word vectors across a document (e.g., a document's average word-vector location) as input to a standard classifier (e.g., logistic regression).

a document is actually an ordered path of locations through R K , and simple averaging destroys much of the available information.


Indeed, we find that all methods are often outperformed by phrase-count logistic regression with rare-feature up-weighting and carefully chosen regularization.

the out-of-the-box performance of Word2Vec inversion argues for its consideration as a simple default in document classification.


Choosing the most frequent property p may lead to descriptions that closely resemble those observed in the data.

we predict that the availability of a highly discriminatory property may change this preference.


By augmenting a labeled dataset with unlabeled data, we can use a bootstrapping framework to improve predictive accuracy, and reduce the need for labeled data-which could make it much easier to port discourse parsing algorithms to new domains.

a fully unsupervised parser may not be desirable because in many applications specific discourse relations must be identified, which would be difficult to achieve without the use of labeled examples.


In contrast, the MRNN approach tends to generate such anaphoric relationships correctly.

the D-ME LM maintains an explicit coverage state vector tracking which attributes have already been emitted.


The human evaluation agrees with the CAR slightly better than the automatic metrics.

the agreement rates are still less than 0.7 for all pairs of compared systems.


In this scenario, the metric BLEU assigns the same score of 0.489 for these two translations.

the representation based metric associates hypothesis 2 with a much higher score than that of hypothesis 1, namely 0.865 and 0.555, respectively.


One example is the English word "workflow", which appears in the French post-editions both as is (21 sentences) and translated into "flux de travail" (34 sentences).

in the other language directions all the occurrences of 'workflow" are either translated or kept in English.


The probability for the document V is given by: where F θ (V ) is the "free energy", which can be analytically integrated easily, and Z D is the "partition function" for normalization, only associated with the document length D. As the hidden state and document are conditionally independent, the conditional distributions are derived: where 3is the softmax units describing the multinomial distribution of the words, and Equation (4) serves as an efficient inference from words to semantic meanings, where we adopt the probabilities of each hidden unit "activated" as the topic features.

rSM is naturally learned by minimizing the negative log-likelihood function (ML) as follows: the gradient is intractable for the combinatorial normalization term Z D .


Although both CD and α-NCE run slower when the input dimension increases, CD tends to take much more time due to the multinomial sampling at each iteration, especially when more Gibbs steps are used.

running time stays reasonable in α-NCE even if a larger noise size or a larger dimension is applied.


(2011)'s "full" model 87.44% WRRBM (Dahl et al., 2012) 87.42% RSM:CD 86.22% RSM:α-NCE-5 87.09% RSM:α-NCE-5 (idf) 87.81% Table 2: The performance of sentiment classification accuracy on the IMDB dataset using RSMs compared to other BoW-based approaches.

table 2 shows the performance of RSM in sentiment classification, where model combinations reported in previous efforts are not considered.


Specifically, the regularizer defines a penalty if the source classifier and the target classifier make different predictions on an unlabeled target instance.

with this regularizer, EA++ does not strictly restrict either the source classifier or the target classifier to lie in the target subspace X t .


However, with this regularizer, EA++ does not strictly restrict either the source classifier or the target classifier to lie in the target subspace X t .

as we have pointed out above, when only the induced features are used, our method leverages the unlabeled target instances to force the learned classifier to lie in X t .


Therefore x i,j refers to concatenated word vector from the i-th word to the (i + j)-th word: Sequential word concatenation x i,j works as n-gram models which feeds local information into convolution operations.

this setting can not capture long-distance relationships unless we enlarge the window indefinitely which would inevitably cause the data sparsity problem.


The errors in parse trees inevitably affect the classification accuracy.

the parser works substantially better on the TREC dataset since all questions are in formal written English, and the training set for Stanford parser 5 already includes the QuestionBank (Judge et al., 2006) which includes 2,000 TREC sentences.


Given their large numbers of parameters, often in the millions, one would expect that such models can only be effectively learned on very large datasets.

we show here that a complex deep convolution network can be trained on about a thousand training examples, although careful model design and regularisation is paramount.


(2010) preprocessed the text by stemming, down-  casing, and discarding feature instances that occurred in fewer than five reviews.

we did not perform any processing of the text or feature engineering, apart from tokenization, instead learning this automatically.


This suggests that the tendency by certain sites to review specific movies is in itself indicative of the revenue.

this improvement is more difficult to discern with the ANN Text+Meta+Domain model, possibly due to redundancy with the meta data.


sampling (k) 5 -# of iterations (T ) 5 20 # of threads 56 # of dimensions (D) 300 Table 3: Hyper-parameters in our experiments.

in particular, the original SGNS uses β(c i,j ) = 1 for all (i, j), and logistic loss function: GloVe uses a least squared loss function: Table 2 lists the factors of each configuration used differently in SGNS and GloVe.


Note that HUCRF does not always perform better than CRF when initialized randomly.

hUCRF consistently outperforms CRF with the pre-training methods proposed in this work.


It might seem that a low similarity score between two retellings simply indicates that one retelling includes fewer story elements.

given the equivalent number of story elements recalled by the two groups, we can assume that a low similarity score indicates a difference in the quality rather than the quantity of information in the retellings.


2 Due to the nature of the olfactory data source (see Section 3), it is not possible to build olfactory representations for all concepts in the test sets.

cross-modal mappings yield an additional benefit: since linguistic representations have full coverage over the datasets, we can project from linguistic space to perceptual space to also obtain full coverage for the perceptual modalities.


Delexicalized transfer yields worse results than a supervised lexicalized parser trained on a target language treebank.

for languages with no treebanks available, it may be useful to obtain at least a lower-quality parse tree for tasks such as information retrieval.


Traditionally, transition-based parsers were trained to follow a so-called static oracle, which is only defined on the configurations of a canonical computation that generates the gold tree, returning the next transition in said computation.

dynamic oracles are non-deterministic (not limited to one sequence, but supporting all the possible computations leading to the gold tree), and complete (also defined for configurations where the gold tree is unreachable, choosing the transition(s) that lead to a tree with minimum error).


Thus, the configuration has only one individually unreachable arc (0→2), but its loss is 2.

it is worth noting that non-arcdecomposability in the parser is exclusively due to cycles.


Our proposed method achieved the state-of-the-art F-score and OOV recall on two common corpus PKU and MSR.

note that we only exploited the flat segmented results of internal word structure here.


While it has been observed repeatedly that using multiple source languages improves performance Fossum and Abney, 2005;, most available techniques work best for closely related languages.

this paper presents an effort to learn POS taggers for truly low-resource languages, with minimum assumptions about the available language resources.


Distant supervision is a widely applied approach to automatic training of relation extraction systems and has the advantage that it can generate large amounts of labelled data with minimal effort.

this data may contain errors and consequently systems trained using distant supervision tend not to perform as well as those based on manually labelled data.


The main advantage is its ability to automatically generate large amounts of training data automatically.

this automatically labelled data is noisy and usually generates lower performance than approaches trained using manually labelled data.


The overall ratio of positive to negative sentences in this dataset was 1:5.1.

this changes to 1:2.3 after removing examples identified by PRA.


Most Open IE systems employ syntactic information such as parse trees and part of speech (POS) tags, but ignore lexical information.

previous work suggests that Open IE would benefit from lexical information because the same syntactic structure may correspond to different relations.


(2013) is not helpful, which we attribute to data sparsity.

smoothed word representations do improve F-measure.


As we analyze, shortest dependency paths and subtrees play different roles in relation classification.

we can see that DT-RNN does not distinguish the modeling processes of shortest paths and subtrees.


Unlike in the news domain, in the biomedical domain it is rare for the same word or phrase to refer to multiple different concepts.

different words or phrases often refer to the same concept.


Finally, this work challenges prior claims that spoken language is "more complex" than other genres with regards to referentiality.

: whereas in a spoken discourse the potential addressees are by default the participants, web texts such as the reviews studied here have no such default, and may include complex, creative, and domain-specific deictic reference that can be important for computational systems to address.


Decisions relating to relevance of material to a given topic (MO question) are delegated to experts on the website.

the information seeker (MO user posting the question) remains the ultimate judge of relevance.


Understanding the search intents of queries is essential for satisfying users' information needs and is very important for many search tasks such as personalized search, query suggestion, and search result presentation.

it is not a trivial task because the underlying intents of the same query may be different for different users.


Clearly, it is very challenging to learn a CTH from multiple topic hierarchies in different articles due to the following 3 reasons: 1) A topic can be denoted by a variety of tags in different articles (e.g., "foreign aids" and "aids from other countries"); 2) Structural/hierarchical information can be inconsistent (or even opposite) across different articles (e.g., "response subtopicOf aftermath" and "aftermath subtopicOf response" in different earthquake event articles); 3) Intuitively, text descriptions of the topics in Wiki articles are supposed to be able to help determine subtopic relations between topics.

how can we model the textual correlation?


On the one hand, the results of a corrected run of the WHUNLP system provided by the SemEval organizers.

the results of an out of the competition version of the SPINOZAVU team explained in (Caselli et al., 2015).


As future work, we plan to explore in more detail this research line by applying more sophisticated approaches in the temporal analysis at document level.

this is not the only research line that we want to go in depth.


To maintain comparability, we use the ACE-2005 documents with the same split as in (Ji and Grishman, 2008;Liao and Grishman, 2010b;Li et al., 2013) to 40 test documents and 559 training documents.

some evaluation settings differ: Li et al.


Additionally, name matching has been used as a component in cross language entity linking (McNamee et al., 2011a;McNamee et al., 2011b) and cross lingual entity clustering (Green et al., 2012).

little work has focused on logograms, with the exception of Cheng et al.


String Matching We consider two common string matching algorithms: Levenshtein and Jaro-Winkler.

because of the issues mentioned above we expect these to perform poorly when applied to Chinese strings.


"azul" in ES) and have a high similarity score in our proposed BC mapping, are correctly tagged.

ooV Brand have a very large prediction error rate due to the small training data.


This method for automatic paraphrasing has been discussed previously by Rastogi and Van Durme (2014).

whereas their work only discussed the idea as a hypothetical way of augmenting FN, we apply the method, vet the results, and release it as a public resource.


Third, we use a simple systematic process to ensure that the constructed data is enriched with "related" pairs, beyond what one would expect to obtain by random sampling.

to previous work, our enrichment process does not rely on a particular relatedness algorithm or resource such as Wordnet (Fellbaum, 1998), hence the constructed data is less biased in favor of a specific method.


For examples, datasets constructed to assess lexical entailment (Mirkin et al., 2009) and lexical substitution (McCarthy and Navigli, 2009;Kremer et al., 2014;Biemann, 2013) methods.

the focus of the current work is on the more general notion of term-relatedness, which seems to go beyond these more concrete relations.


In addition, we note that a correlation measure gives equal weight to all pairs in the dataset.

in some NLP applications it is more important to properly distinguish related pairs from unrelated ones.


In some tasks, such as statistical machine translation (Kondrak et al., 2003) and sentence alignment, or when studying the similarity or intelligibility of the languages, cognates are seen as words that have similar spelling and meaning, their etymology being completely disregarded.

in problems of language classification, distinguishing cognates from borrowings is essential.


Note that the dev set is not used in the experiments of this paper, since ∆BLEU and IBM BLEU are metrics that do not require training.

the dev set is released along with a test set in the dataset release accompanying this paper.


This is expected, since BLEU treats all references as equal and has no way of discriminating between them.

correlation coefficients increase for ∆BLEU after adding lower scoring references.


Point-wise MI is often used to find interesting bigrams (collocations).

mI is actually better to think of it as a measure of independence than of dependence (manning et al., 1999).


LR is one of the most stable methods for automatic term extraction so far, and more appropriate for sparse data than other metrics.

lR is still biased to two frequent words that are rarely adjacent, such as the pair (the, the) (Pantel et al., 2001).


Previous research and work in Tibetan word segmentation have made great progresses.

cases with unknown words are not satisfactory.


(2012) designed and implemented a Tibetan word segmentation system named "SegT" which is lexicon-based practical system with a constant lexicon.

it has the difficulty of identifying unknown words in newspaper articles and web documents which are highly changeable texts with time.


In comparison, the advantage of the syntactic information is not at all obvious when they are used in a non-factorized fashion in the DEP model; it out-performs the window-based methods (below the dashed line) on only 3 datasets with limited margins.

the window-based methods consistently outperform the dependencybased methods on the FG dataset, confirming our intuition that window-based methods are better at capturing relatedness than similarity.


This is a favorable situation for implementing a bootstrapping approach in which the results for "good" entries are exploited for improving the results of the other ones.

such idea faces two problems: first, detecting "good" entries; second, learning a model from them for improving the performance of the other entries.


It is possible that this introduces a confounding factor.

since we do not see marked effects for gender or region, and since the English results closely track the German data, this seems unlikely.


Probabilistic topic model such as Latent Dirichlet Allocation(LDA) (Blei et al, 2003) has been widely used for discovering latent topics from document collections by capturing words' cooccuring relation.

the "bag of words" assumption is employed in most existing topic models, it assumes the order of words can be ignored and topic assignment of each word is conditionally independent given the topic mixture of a document.


(2012) developed a spatio-temporal model of conflict events in Afghanistan.

here we deal with temporal text data, and model several correlated outputs rather than their single output.


In terms of performing distributed representation learning for output variables, our proposed model shares similarity with the structured output representation learning approach developed by Srikumar and Manning (2014), which extends the structured support vector machines to simultaneously learn the prediction model and the distributed representations of the output labels.

the approach in (Srikumar and Manning, 2014) assumes the training labels (i.e., output values) are given and performs learning in the standard supervised in-domain setting, while our proposed distributed HMMs address cross-domain learning problems by performing unsupervised representation learning.


(2015) and Zhang (2015) have proposed independently to summary source sentences with convolutional neural networks.

they both extend the neural network joint model (NNJM) of Devlin et al.


From a higher level view, EM-UNRAVEL can be seen as a specialized word based MT decoder that can efficiently generate and organize all possible translations in the E-step, and efficiently retrain the model {p(f |e)} on all these hypotheses in the M-step.

to DET-UNRAVEL, EM-UNRAVEL processes the input corpus sentence by sentence.


Similarly to DET-UNRAVEL, the previously described expansion and pruning step is implemented using two arrays H s and H t .

in EM-UNRAVEL the partial hypotheses in H s and H t use the same data structures since-in contrast to DET-UNRAVEL-recombination of hypotheses is possible.


Although the 1,000-best oracle remains at the same level over the iterations, the 1,000best average score 5 increases by 2 BLEU at the last iteration over the first 1,000-best hypotheses produced by Moses, pointing out a strong improvement of the average quality of the 1,000best hypotheses.

except for the IN configuration on medical En→Fr, multi-pass Moses does not bring improvements by itself over the Rerank baseline.


These methods are useful in capturing semantic information carried by high-level units (such as phrases and beyond) and usually do not rely on word alignments.

they suffer from reduced accuracy for representing rare tokens, whose semantic information may not be well generalized.


This method makes full use of parallel corpora and produces high-quality word alignments.

it is unable to exploit the richer monolingual corpora.


In particular, the triangulation method, which translates by combining source-pivot and pivot-target translation models into a source-target model, is known for its high translation accuracy.

in the conventional triangulation method, information of pivot phrases is forgotten and not used in the translation process.


People with normal vision can read news documents with their eyes conveniently.

according to WHO's statistics, up to October 2013, 285 million people are estimated to be visually impaired worldwide: 39 million are blind and 246 have low vision.


There is some thematically related work, such as automatic filtering of pornographic content (Polpinij et al., 2006;Sood et al., 2012;Xiang et al., 2012;Su et al., 2004), but we believe the nature of the task is significantly different such that a different approach is required.

text or document classification, the general technique employed in this paper, is a very common task (Manning et al., 2008).


determining the ease with which a written text can be understood by a reader, since age is certainly a dimension along which readability varies.

our literature review of this area suggested that the aspects being considered in readability assessment are sufficiently different from the dimensions that seem to be most relevant for media age appropriatness ratings.


Due to the inherent subjectivity and poor definition of the task, mentioned above, it is difficult for annotators to reliably produce these annotations (Bryant and Ng, 2015).

this requirement can be relinquished by treating GEC as a text-to-text rewriting task and borrowing metrics from machine translation, as Park and Levy (2011) did with BLEU (Papineni et al., 2002) and METEOR (Lavie and Agarwal, 2007).


These four radicals altogether convey the meaning that "the moment when sun arises from the grass while the moon wanes away", which is exactly "morning".

it is hard to decipher the semantics of strokes, and radicals are the minimum semantic unit for Chinese.


Some Android applications 3 avoid drunk-texting by blocking outgoing texts at the click of a button.

to the best of our knowledge, these tools require a user command to begin blocking.


The training goal is to minimize average perplexity across X.

a deeper look into perplexity beyond corpus-wide average reveals interesting findings.


The information of background corpus has been incorporated using linear combination (Ponte and Croft, 1998;.

to the simple strategy which smooths all documents with the same background, recently corpus structures have been exploited for more accurate smoothing.


Generally, given a non-smoothed document language model P (w|d), which indicates a word distribution for a term w in document d, we attempt to generate a smoothed language model P (w|d + ) that could better estimate the text contents of a document d as d + to avoid zero probabilities for those words not seen in d. Arbitrary assignment of pseudo word counts such as add-λ to every unseen words once was a major improvement for language model smoothing (Chen and Goodman, 1996).

the purpose of smoothing is to estimate language model more accurately.


To the best of our knowledge, only (Li et al., 2010) used a supervised keyword extraction framework (based on KEA) with additional features, such as POS tags to performed keyword extraction on Facebook posts.

at that time Facebook status updates or posts did not contained either hashtags or user mentions.