We propose a faster inference technique using Cholesky decomposition of covariance matrices which can be applied to both the Gibbs and variational/EM method.

1 https://code.google.com/p/word2vec/ 2.2 Latent Dirichlet Allocation (LDA) LDA (Blei et al., 2003) is a probabilistic topic model of corpora of documents which seeks to represent the underlying thematic structure of the document collection.


Hence, ranking-based evaluations, where judges are asked to rank the output of 2 to 5 systems, have been used in recent years, which has yielded much higher inter-annotator agreement (Callison-Burch et al., 2007).

finally, we have presented evidence suggesting that using the pairwise hidden layers is advantageous over simpler flat models.


These compact representations are in turn based on word and sentence embeddings, which are learned using neural networks.

network parameters: We train our neural network using SGD with adagrad, an initial learning rate of η = 0.01, mini-batches of size 30, and L 2 regularization with a decay parameter λ = 1e −4 .


They obtained promising results using syntactic and discourse-based structures.

datasets: We train our neural models on WMT11 and we evaluate them on WMT12.


, t n ): (a) We only consider source phrases p of length at most 10 (i.e., i − i < 10 for p = [i, i ]).

in this manner, we combine the advantages of the hierarchical phrase-based approach on the source side and the tree-based approach with discontinuiety on the target side.


Syntax-based systems have become widely used because of their ability to handle non-local reordering and other linguistic phenomena better than phrase-based models (Och and Ney, 2004).

next, we lift the notion of consistently aligned phrase pairs to our rule spans.


We use the decoder provided by MBOT-Moses of Braune et al.

such fixed phrases will often be assembled inconsistently by substitution from small fragments.


Our method also trains its parameters without any pre-training or post-training procedure.

experimental results show that with the basic features of a hierarchical phrase-based machine translation system, our method produce translations that are better than a linear model.


Similarly, the probability of a plaintext word e j taking a value l given samples for all other plaintext words, #(e j−1 ) −j + e α e,e j−1 .

we observe consistent improvement throughout the experiment.


As shown in Figure 1, we start a few sampling processes each with a different random sample.

unlike our learning setting, their approach relied on large amounts of translation pairs learned from parallel data to train their linear transformations.


Enabled if there is a parent and we did not just come down from it: • DOW N j : move to the child j of the current vertex.

we represent these problems as sequence prediction machine learning tasks, which we address using recurrent neural networks.


We consider a number of methods to map the natural language description of a problem into its formal program representation.

our levels of agreement are likely to be greater than suggested by measures in the table.


Starting with some initial classifiers and a training set of NL and AST pairs, we search for the most likely derivation.

the function used to score derivations is a simple matching heuristic relying on the overlap between query terms and program identifiers.


Thirdly, we demonstrate experimentally, in Section 5, that multiple many-tomany alignments may be an extremely useful first step in boosting the performance of a G2P model.

clearly, a system with more information should not perform worse than a system with less information (unless the additional information is highly noisy), but it is a priori not clear at all how the extra information can be included, as Bhargava and Kondrak (2012) note: output predictions may be in distinct alphabets and/or follow different conventions, and simple rule-based conversions may even deteriorate a baseline system's performance.


For example, we convert word tomorrow into to-mor-row.

but as the lack of prior knowledge, we cannot judge an optimal λ.


Dictionary lookup of Internet slang 2 is performed to filter those ill-OOV words whose correct forms are not single words.

the impact of normalization or NSW detection on NER has not been well studied in social media domain.


Similarly, during decoding in NSW detection, we need the Basic Features 1.

most of previous work on normalization assumed that they already knew which tokens are NSW that need normalization.


this filtering can be done efficiently and yields a manageable number of quadruples on which to computeK k .


The testing set contains 1517 pairs for 89 questions.

it can be very useful in a rewriting rule to type a wildcard link with the relation holonym, as this provides constrained semantic roles to the linked wildcards in the rule, thus holonym would be a good variable type.


The pairs in both datasets were then rated for their plausibility by 27 human subjects, and their judgements were aggregated into a gold standard.

the dataset consists of 52 verb metaphors and their human-produced literal paraphrases.


In the future, it would be interesting to derive the information about predicate-argument relations from low-level visual features directly.

in addition, the image features tend to be sparse for abstract concepts, reducing both the quality and the coverage of abstract clusters.


For instance, they can be generalised to acquire SPs from unbalanced corpora of different sizes (e.g.

some of them are still figuratively used.


In 1.4 beta, most of adjunct usages of "ni" are mixed up with the argument usages of "ni", making the identification of dative cases seemingly easy.

(2009) is the best for the NOM and ACC cases, and Sasano and Kurohashi (2011) the best for the DAT cases.


Supervision Unlike many recent composition models (Kalchbrenner and Blunsom, 2013;Kalchbrenner et al., 2014;Socher et al., 2012;Socher et al., 2013, among others), the context-prediction objective of C-PHRASE does not require annotated data, and it is meant to provide generalpurpose representations that can serve in different tasks.

we use syntax to determine the target units that we build representations for (in the sense that we jointly learn representations of their constituents).


The most salient of these is semantic role labels, such as the ARG0 and destination arcs in Figure 2.

a standard semantics and annotation guideline for aMR alignment is left for future work; our accuracy should be considered only an informal metric.


We consider two metrics, IED and END, which measure accuracy based on the action sequence and environment, respectively.

the robot is also an object in the environment.


We use planner and a simulator that to-gether specify a deterministic mapping from the current environment e i and a logical form z i to a new environment e i+1 .

to previous work , we use postconditions instead of action sequence for two main reasons.


Since the dependencies between constituents can be an exponential number and representing structures in learning algorithms is rather challenging, automatic feature engineering through kernel methods (Shawe-Taylor and Cristianini, 2004;Moschitti, 2006) can be a promising direction.

we used default parameters both for P T K and SP T K whereas we selected h and D parameters of N SP DK that obtained the best average accuracy using a 5-fold cross validation on the training set.


(specified after the ± sign) shows that in most cases the system differences are significant.

with the previous kernels these similarities are computed intra-pair, e.g., between a 1 and a 2 .


In user-sentiment assumption test, we use absolute rating difference ||rating a − rating b || as the measurement between two reviews a and b.

these models only use semantics of texts, while ignoring users who express the sentiment and products which are evaluated, both of which have great influences on interpreting the sentiment of text.


Furthermore, given that .41, 0, .59 , .40, 0, .60 , and .45, 0, .55 give virtually identical information to a sentiment analyst, it seems unreasonable to expect exactly one to be the correct polarity tag for reliable and the other two incorrect.

over all of our experiments, the resulting systems of constraints can be as small as 2 constraints with 2 variables  Table 5: Micro-WNop -SWD Inconsistencies and as large as 3,330 constraints with 4,946 variables.


This would imply, e.g., that if a sentence in the progressive subreddit conveys an ostensibly positive sentiment about the political commentator 'Ollie', 4 then this sentence is likely to have been intended ironically.

we use the Stanford Sentiment Analysis tool (Socher et al., 2013) to infer sentiment.


Automatically classifying instances with multiple possible categories is sometimes much more difficult than classifying instances with a single label.

empirical evaluation shows that  Baseline DFG-both our DFG approach performs significantly better than the state-of-the-art.


2) Context dependency: Two instances from the same context are more likely to share the same emotion label than those from a random selection.

this approach first utilizes a Bayesian network to infer the relationship among the labels and then employ them in the classifier.


Wan (2009) proposed a co-training approach to address the cross-lingual sentiment classification problem.

we propose a new bootstrapping mechanism, based on a principle called dual-view sentiment consensus.


This parsing phase is grounded on a set of patterns (see Figure 3).

this first and pioneering version of the system shows encouraging results for the different tasks performed by the system that concern the detection of relevant like/dislike expressions (substantial agreement with a Fleiss kappa at 0.61), the categorization of the expressions between like and dislike (almost perfect agreement with a Fleiss kappa at 0.84) -polarity assignment -and the identification of the target type (53% of agreement between the reference and the system output).


This first and pioneering version of the system shows encouraging results for the different tasks performed by the system that concern the detection of relevant like/dislike expressions (substantial agreement with a Fleiss kappa at 0.61), the categorization of the expressions between like and dislike (almost perfect agreement with a Fleiss kappa at 0.84) -polarity assignment -and the identification of the target type (53% of agreement between the reference and the system output).

the system defines relevantAttExpr(usrSentence) == T rue, even if any sentence matching with a relevant pattern has been found in the user's sentence.


These words are not represented in E. One way to deal with this case, is to simply set the embeddings of unknown words to zero.

this was the top system in the 2013 edition of SemEval.


Labeled data is, however, expensive to obtain, while unlabeled data is widely available.

finally, Section 7 draws the conclusions.


The structured skip-gram models the following probability: Here, w i ∈ {1, 0} v×1 is a one-hot representation of w = i.

this simple method brings two funda-mental advantages.


We also include two other approaches that are related to the one here proposed, where a neural network initialized with pre-trained word embeddings is used to learn relevant features.

for a given task, only a subset of all the latent aspects captured by the word embeddings will be useful.


There are 5000, 2000 and 3000 word types.

it is possible to derive word representations by exploiting word co-occurrence patterns in large samples of unlabeled text.


Then, the resulting network is searched to select the best scoring word at each node.

table 4 reports the WER results obtained on tst2013 by ROVER methods fed with: different numbers of hypotheses (from 3 to 8), at different granularity levels (whole utterance vs. segment), ranked with different models (random, RR1, RR2 and MLR) trained with different sets of features 8 http://sourceforge.net/p/lemur/wiki/ RankLib/ (Basic, WordBased, Basic+WordBased).


The decrease of precision is misleading, though, due to the small number of occurrences it has been computed on.

ask the parser to perform tokenization will not always solve the problem.


For a given time step (step 2 as an example), argument and predicate are specified with different color.

smaller d s means that it is easy to make prediction that long history is unnecessary.


The above four features are concatenated to be the input representation at this time step for the following LSTM layers.

the traditional feature templates are only good at describing the properties in neighborhood and a small mistake in syntactic tree will results in large deviation in SRL tagging.


Third, we utilize the Dropout strategy to address the overfitting prob-lem.

for example, in figure 1, for the sentence "the assets are sold", our parser can construct the parse tree by performing the action sequence {sh-DT, sh-NNS, rr-NP, sh-VBP, sh-VBN, ru-VP, rr-VP, rr-S}.


If the feature set is too small, it might underfit the model and leads to low performance.

table 4 presents results of different experimental configurations for English.


In order to jointly assign POS tags and construct a constituent structure for an input sentence, we define the following actions for the action set T , following Wang and Xue (2014): • SHIFT-X (sh-x): remove the first word from β, assign a POS tag X to the word and push it onto the top of σ; • REDUCE-UNARY-X (ru-x): pop the top subtree from σ, construct a new unary node labeled with X for the subtree, then push the new subtree back onto σ.

to achieve competitive performance, they had to combine the learned features with the traditional hand-crafted features.


(2014) describe several uses for arclevel constraints in transition-based parsing.

alternatively, more punctuationspecific features to account for its myriad roles in syntax could serve to improve performance.


Different with standard RNN, there are no nonterminal nodes in dependency tree.

1) They first summed up all child nodes into a dense vector v c and then composed subtree representation from v c and vector parent node.


A set of agenda B = B 0 , B 1 , • • • maintains the k-best states for each step j at B j , which is first initialized by inserting the axiom in B 0 .

enriched models incur numerous parameters and sparsity issues, and are insufficient for capturing various syntactic phenomena.


Parameter estimation is performed in parallel by distributing training instances asynchronously in each shard and by updating locally copied parameters using the sub-gradients computed from the distributed mini-batches (Dean et al., 2012).

our parser differs in that we do not differentiate left or right head words.


We generate the top 10 best candidate parse trees using 10 fold cross validation for each sentence in the training data.

in the future, we would like to extend our technique to other real valued kernels such as the string kernels and tagging kernels.


This is a chain rule, which means that any feature that has f j as its component can also be pruned safely.

the objective function for learning L 1 norm SVMs is: where is the hinge loss function for the i-th sample.


We found that SRL features, both in isolation and together with standard syntactic features, improve parsing performance, both when measured using full-sentence F-score, and in terms of incremental F-score.

central to the discriminative approach is the exploration of features that cannot be straightforwardly embedded into the parser using a dynamic program.


Neural probabilistic parsers are attractive for their capability of automatic feature combination and small data sizes.

our integrated approximated search and learning framework allows rich global features.


Treebanks are key resources for developing accurate statistical parsers.

constituency trees were converted to basic non-collapsed dependency trees using Stanford Dependencies (De Marneffe et al., 2006).


To remedy this problem, we use a classifier (specifically logistic regression) to determine whether a modified tree should be used.

building treebanks is expensive and timeconsuming for humans.


In that way, the dictionary A can be readily obtained either using bilingual lexicon induction approaches (Koehn and Knight, 2002;Mann and Yarowsky, 2001;Haghighi et al., 2008), or from resources like Wiktionary 5 and Panlex.

to verify this, we further conduct experiments under both settings using the PROJ+Cluster model.


(2012), which assigns a target word to the cluster with which it is most often aligned: This method also has the drawback that words that do not occur in the alignment dictionary (OOV) cannot be assigned a cluster.

experiments show that by incorporating lexical features, the performance of cross-lingual dependency parsing can be improved significantly.


That said, these methods have the advantage that they are capable of capturing some language-specific syntactic patterns which our approach cannot.

let .., N S } be the alignment dictionary, where c i,j is the number of times when the i th target word w T i is aligned to the j th source word w S j .


In general, we expect our bilingual word embeddings to preserve translational equivalences.

in order to improve the robustness of projection, we utilize a morphology-inspired mechanism, to propagate embeddings from in-vocabulary words to out-ofvocabulary (OOV) words.


In contrast, we are interested in extracting discourse relations with minimal additional annotation, relying primarily on the available question-answer pairs.

"Sally, Sally, come home", Sally's mom calls out.


The non-generic sentence (1b) roughly speaking provides ABox content for a machine-readable knowledge base, i.e., knowledge about particular instances, e.g, "A is an instance of B / has property X".

the major contributions of this work include the study of genericity both on the NP-and clauselevel, and the study of the interaction of these two levels.


As shown in Theorem 4.1, we can take the element-wise power transformation on counts (such as the power of 1, 2/3, 1/2 in this template) while preserving the representational meaning of word embeddings under the Brown model interpretation.

for our setting, the analogous weighted least squares optimization is: where 2 .


The CCA scaling, combined with the square-root transformation, gives the best overall performance.

table 3 shows the result for both 500 and 1000 dimensions.


In word similarity, spectral methods generally excel, with CCA consistently performing the best.

it has been found by many (including ourselves) that setting β = 1 yields substantially worse representations than setting β ∈ {0, 0.5} (Levy et al., 2015).


Though recent knowledge graph embeddings (Lin et al., 2015;Wang et al., 2014) integrate the relational structure among entities, they primarily target at link prediction and lack an explicit relatedness measure.

we use the wikipedia snapshot from Jan 12nd, 2015 as our training data and KB.


Though the shortest path can be selected, it ignores other related category nodes and loses rich information.

the comparison between Ours and Ours-NoH further reveals the effect of integrating the hierarchy in learning entity vectors.


Finally, we want to incorporate all these elements in a single model, with the morphological and word order elements of the model working in harmony.

in the case of these original models and also the CBOM model, we follow Mikolov et al.


's (2013b) method for making the word-analogy predictions in terms of addition and subtraction: smaller ≈ bigger − big + small.


Our semantic parsing model defines a distribution over logical forms given by the domaingeneral grammar G and additional rules triggered by the input utterance x.

... (1) by builder (∼30 minutes) (2) via domain-general grammar (3) via crowdsourcing (∼5 hours) (4) by training a paraphrasing model Figure 1: Functionality-driven process for building semantic parsers.


All types (e.g., person) have the syntactic category TYPENP, and all entities (e.g., Figure 2: Deriving a logical form z (red) and a canonical utterance c (green) from the grammar G. Each node contains a syntactic category and a logical form, which is generated by applying a rule.

our compact grammar precisely specifies the logical functionality.


Such feature is widely used because it's simple and surprisingly efficient in many tasks.

recurrent models outperform rAE indicates that task-specific composing and representation learning with less syntactic information lead to a better result.


The purpose of this method is to investigate whether there are patterns of the price movement in the history of the stock.

continuous Dirichlet Process Mixture (cDPM) model was used to learn the daily topic set of Twitter messages to predict the stock market (Si et al., 2013).


First, a rule-based algorithm is applied to identify the category of each word in the documents.

one important missing thing is that opinions or sentiments are expressed on topics or aspects of companies.


The value of k depends on the size of the training set and the occurrences of each tag.

for instance, as shown in figure 2, the vector of 'is very interesting' can be composed from the vector of the left node 'is' and that of the right node 'very interesting'.


From now on, y is a vector representing the dependency trees corresponding to the whole corpus.

at each iteration t, the convex function f is approximated by a linear function defined by its gradient at the current point z t .


We propose to use a feature-rich discriminative parser, and to learn the parameters of this parser using a convex quadratic objective function.

we would then have to use a higher-order parser, such as the ones described by McDonald and Pereira (2006) and Koo and Collins (2010).


Overall, our approach outperforms the three baselines, with an absolute improvement of 13 points over the extended valence grammar with posterior sparsity and 8 points over the model with universal syntactic rules.

this might be inefficient since Algorithm 1: Frank-Wolfe algorithm for t ∈ {1, ..., T } do Compute the gradient: Solve the linear program: Take the Frank-Wolfe step: it does not use the structure of the polytope and, in particular, the fact that one can easily minimize a linear function over the tree polytope using the minimum weight spanning tree algorithm.


Figure 1 specifies several dependencies: of is a dependent of director, executive vice president and director are conjuncts and and is the coordinator.

also, barks has a right child loudly; this generates a half bracket before V R .


In general, the lexical head of a derived category is determined by the (primary) functor, so that the lexical head of a category X or X|Z 1 |...|Z n that resulted from combining X|Y and Y or Y|Z 1 |...|Z n is identical to the lexical head of X.

we will focus on the English CCGbank but these details apply with only minor changes to German and Chinese as well.


In contrast, our approach directly computes a cost for actions based on coreference evaluation metrics.

this formulates the agent's task so it only has two actions to chose from instead of a number of actions proportional to the number of clusters squared.


We incorporate two different mention pair models into our system.

for each pair, we make a binary decision on whether or not the clusters containing these pairs should be merged.


In this paper we introduce a novel coreference system that combines the advantages of mention pair and entity-centric systems with model stacking.

training the agent on the gold labels alone would unrealistically teach it to make decisions under the assumption that all previous decisions were correct, potentially causing it to over-rely on information from past actions.


Our methodology, outlined as Algorithm 1, is inspired by the recent work of Ganchev and Das (2013) on cross-lingual learning of sequence models.

the third and eighth rows in table 3 show that this baseline is stronger than the delexicalized baseline, but still 6-8 points away from the supervised systems.


Note that all systems predict the same candidate mentions; however a final post-processing discards all mentions that ended up in singleton entities, for compliance with the official scorer.

we propose cross-lingual coreference resolution as a way of transferring information from a rich-resource language to build coreference resolvers for languages with scarcer resources; as a testbed, we transfer from English to Spanish and to Brazilian Portuguese.


Since HIPTM can no longer access the votes in the test data, its performance drops significantly compared with VOTE.

marginal counts are denoted by • and the superscript −b,m excludes the assignment for token w b,m from the corresponding count.


3 To improve topic interpretability, issue nodes have an informed prior from the Congressional Bills Project {φ k } (Table 1).

h3 captures the more strident Tea Party framing of Obamacare as an unconstitutional government takeover.


In the more subtle task of validating each topic token (Group precision) we see a greater variance among the two labeler groups.

the model learns both the schema for a KB, and a set of facts that are related to that schema, thus combining the processes of KB population and ontology construction.


Our sparse sampler runs even faster, and takes less than a second per iteration.

we have also observed that there is a point at which using additional processors actually slows running time.


We use both lexicalized features, where all possible pairs (f (x), g(z)) form distinct features, and binary unlexicalized features indicating whether f (x) and g(z) have a string match.

example inputs and outputs are shown in Figure 1.


More recent work sacrifices compositionality in favor of using more open-ended knowledge bases such as Freebase (Cai and Yates, 2013;Berant et al., 2013;Fader et al., 2014;Reddy et al., 2014).

we also anchor all numerical values (numbers, dates, percentages, etc.)


As our language for logical forms, we use lambda dependency-based compositional semantics (Liang, 2013), or lambda DCS, which we briefly describe here.

even these broader knowledge sources still define a x1: "Greece held its last Summer Olympics in which year?"


In graph parsing, we already know the identity of all nodes and edges in sub-s-graphs (as nodes and edges in SG), and must thus pay attention that merge operations do not accidentally fuse or duplicate them.

our grammars describe how to build graphs from smaller pieces.


If f : A B and g : A B are partial functions, we let the partial function f ∪ g be defined if for all a ∈ A with both f (a) and g(a) defined, we have f (a) = g(a).

iRTGs extend naturally to a synchronous grammar formalism by adding more homomorphisms and algebras.


2(e) for the blockcutpoint graph of the example.

the runtime for checking (c) amortizes to O(s) per rule.


Here, we apply this constraint only to the representation vectors {a i }.

we consider two alternative transformations.


These methods can efficiently estimate the co-occurrence statistics to model contextual distributions from very large text corpora and they have been demonstrated to be quite effective in a number of NLP tasks.

firstly, many different types of semantic knowledge can all be represented as a number of such ranking inequalities, such as synonymantonym, hyponym-hypernym and etc.


This material is based in part on research sponsored by the NSF under grant IIS-1249516 and DARPA under agreement number FA8750-13-2-0017 (the DEFT program).

our classifier uses extensive feature sets to scale natural logic to the enormous number of phrase pairs in PPDB.


In this paper, we show how a clear concept of semantics can be applied to large-scale paraphrase resources.

the views and conclusions contained in this publication are those of the authors and should not be interpreted as representing official policies or endorsements of DARPA or the U.S. Government.


We have not mentioned the effect of our unary merging (Section 3), but the result indicates it has almost the same effect as the previously proposed padding method (Zhu et al., Shift-  We divide baseline state-of-the-art systems into three categories: shift-reduce systems (Sagae and Lavie, 2005;Sagae and Lavie, 2006;Zhu et al., 2013), other chart-based systems (Petrov and Klein, 2007;Socher et al., 2013), and the systems with external semi supervised features or reranking (Charniak and Johnson, 2005;McClosky et al., 2006;Zhu et al., 2013).

all systems are trained with the structured perceptron.


The key ideas are new feature templates that facilitate state merging of dynamic programming and A* search.

for example, if we lexicalize this model by adding features that depend on the head indices of s 0 and/or s 1 , it increases to O(n 6 • |G| • |N |) since we have to maintain three head indices of A, B, and C. This is why Sagae and Lavie's features are too expensive for our system; they rely on head indices of s 0 , s 1 , s 2 , s 3 , the left and right children of s 0 and s 1 , and so on, leading prohibitively huge complexity.


To this end, we propose tree approximation to obtain CCG sub-graphs under the factorization using tree parsing algorithms.

nevertheless, empirical evaluation indicates that explicitly or implicitly using tree-structured information plays an essential role.


We also use the syntactic dependency trees provided by the CCGBank to obtain necessary information for graph parsing.

we also use the syntactic dependency trees provided by the CCGBank to obtain necessary information for graph parsing.


Given these sentence representations, we predict the similarity scoreŷ using a neural network that considers both the distance and angle between the pair (h L , h R ): where r T = [1 2 .

for example, it is a good choice for dependency trees, where the number of dependents of a head can be highly variable.


This is because 1) the responses retrieved by retrieval-based method are actually written by human, so they do not suffer from grammatical and fluency problems, and 2) the combination of various feature functions potentially makes sure the picked responses are semantically relevant to test posts.

it is a bit surprising that this can be achieved to a reasonable level with a linear transformation in the "space of representation", as validated in Section 5.3, where we show that one post can actually invoke many different responses from NRM.


Each topic also has 10 documents and 4 model summaries.

in addition, we also extract the clauses functioning as subjects of sentences as NPs, such as "that clause".


Recall that we require that one NP and at least one VP compose a sentence.

existing multi-document summarization (MDS) works can be classified into three categories: extraction-based approaches, compression-based approaches, and abstraction-based approaches.


A salience score is calculated for each phrase to indicate its importance.

each VP is represented as a set of its concepts and the index value is calculated for each pair of VPs.


We therefore include it for comparison purposes.

9 in matrix form: We will show that lim Here, Because α < 1, this column sum converges to zero when t → ∞.


We perform a leave-one-out evaluation of each event.

this improvement is statistically significant for all ngram precision, recall, and Fmeasures at the α = .01 level using the Wilcoxon signed-rank test.


The exemplar sentences from the exemplar selection stage are the most salient and representative of the input for the current hour.

in this model, input sentences are apriori equally likely to be exemplars; the salience values are uniformly set as the median value of the input similarity scores, as is commonly used in the AP literature (Frey and Dueck, 2007).


We build on political science and communication theory and use probabilistic topic models combined with time series regression analysis (autoregressive distributed-lag models) to gain insights about the language dynamics in the political processes.

ordinary least square regression finds the coefficients that minimize the mean square error of Y = b + j w T j X j given (X, Y ).


Previous linguistic studies have focused on identifying factors that might influence choices of referring expressions.

we propose a language production model that uses dynamic discourse information to account for speakers' choices of referring expressions.


These models propose deterministic constraints governing when pronouns are preferred in local discourse, but it is not clear how these would account for speakers' choices of referring expressions, nor it is clear why there should be such deterministic constraints.

it is not clear from this previous work how and why these factors result in the observed patterns of referring expressions.


The eventual betrayer is more positive, more polite, but plans less than the victim.

cooperation and betrayal do not happen in a cell cut off from the rest of the world.


The best setting for the model parameters 8 is selected via 5-fold cross validation, ensuring that instances from the same game are never found in both train and validation folds.

imminent betrayal is signaled by sudden changes in the balance of conversational attributes such as positive sentiment, politeness, and structured discourse.


In this case, training with cold instances is naturally more efficient than training with other types of diseases/symptoms.

the task of analyzing the modality lies beyond of scope of this study (Kitagawa et al., ).


Discrepancy: number of occurrences of words, such as should, would, could, etc as defined in LIWC (Tausczik and Pennebaker, 2010).

by applying the maximum weighted matching algorithm on this graph, we can obtain the best role assignment for each team.


What is similar between stances and personas on the one hand and roles on the other is that the unit of analysis is the person.

then we describe a series of role identification models.


By incorporating constraints into the role identification process, we expect to guide the model using human intuition such that the results will be more interpretable, although the prediction error might increase because of the limitation of the search space.

these approaches do not standardly utilize an outcome as supervision to guide the clustering.


Theoretically, word dropout can also be applied to other neural network-based approaches.

we present a simple deep neural network that competes with and, in some cases, outperforms such models on sentiment analysis and factoid question answering tasks while taking only a fraction of the training time.


Table 2: The DAN achieves slightly lower accuracies than the more complex QANTA in much less training time, even at early sentence positions where compositionality plays a bigger role.

there is a tradeoff: syntactic functions require more training time than unordered composition functions and are prohibitively expensive in the case of huge datasets or limited computing resources.


Most of the existing approaches to learning to rank can be generally grouped into three major categories: (i) pointwise approaches, (ii) pairwise approaches, and (iii) listwise approaches.

to overcome the above limitations, this paper investigates SOLAR -a new framework of Scalable Online Learning Algorithms for Ranking, which aims to learn a ranking model from a sequence of training data in an online learning fashion.


To overcome the limitations, this paper presents SOLAR a new framework of Scalable Online Learning Algorithms for Ranking, to tackle the challenge of scalable learning to rank.

for example, [12,19] formulated ranking as a regression problem in diverse forms.


Bold font on Figure 1 indicates that a vertex belongs to the main core of the graph.

then in its graph representation (its "structural formula"), it is crucial to differentiate between the multiple nodes labeled the same (e. g., C or H).


• Amazon: 8,000 product reviews over four different sub-collections (books, DVDs, electronics and kitchen appliances) classified as positive or negative -split into 1,600 for training and 400 for test each (Blitzer et al., 2007).

we believe new kernels that support a very high number of unique node labels could yield even better performances.


Moreover, during training we jointly model the alignment and the translation process simultaneously for different language pairs under the same framework.

experiments that demonstrate the effectiveness of our framework will be described in section 4.


Different from conventional statistical machine translation approaches, neural machine translation approaches aim at learning a radically end-to-end neural network model to optimize translation performance by generalizing machine translation as a sequence learning problem.

in reality, language pairs between English and many other target languages may not be large enough, and pivot-based SMT sometimes fails to handle this problem.


The detailed results are shown in Table 4 (4-th and 6-th rows).

each context feature is represented as a single vector called feature embedding.


doctors and nurses or accountants and assistant accountants.

spectral clustering is based on applying sVD to the graph Laplacian and aims to perform an optimal graph partitioning on the NPMI similarity matrix.


To examine this distinction better, we use the Jensen-Shannon divergence (JSD) to quantify the difference between the topic distributions across every  Figure 2: CDFs for six of the most important topics; the x-axis is on the log-scale for display purposes.

these offer the advantages of not using the validation set and the interpretability properties we highlight in the next section.


LSH based tracking segments the vector-space randomly without consideration of the data's distribution.

the key to scale up documents and topics, lies in reducing the number of necessary comparisons.


By formulating such ideas as search or MDL problems of given coding length 1 , word boundaries are found in an algorithmic fashion (Zhikov et al., 2010;Magistry and Sagot, 2013).

when we reach the unigram G 1 and need to use a base measure G 0 , i.e.


In many cases PYHSMM found more "natural" segmentations, but it does not always conform to the gold annotations.

finally, we note that our proposed model for unsupervised learning is most effective for the language which we do not know its syntactic behavior but only know raw strings as its data.


To automatically recognize such linguistic phenomena beyond small "correct" supervised data, we have to extract linguistic knowledge from the statistics of strings themselves in an unsupervised fashion.

we could exclude the latter case 1 For example, Zhikov et al.


Our coupled model has two major parameters to be decided.

the key idea is to bundle two sets of POS tags together (e.g.


As a widely-used structural classification problem, sequence labeling is prone to suffer from the data sparseness issue.

again, algorithm 1 is used to merge the two data with N ′ = 5K and M ′ = 5K.


Along with this paper, we will publish AutoExtend for extending word embeddings to other data types; the lexeme and synset embeddings used in the experiments; and the code needed to replicate our WSD evaluation 2 .

the dataset is the closest we could find for sense similarity.


Note that the pairs in the two monolingual datasets should be previously aligned.

we then discuss the details of our procedure for the construction of the Spanish and Farsi word similarity datasets in Section 3.


Multi-index techniques allows the very fast computation of top-k queries (Norouzi et al., 2012) on the Hamming space.

we favor the parentship distance by selecting first the signatures where one bit differs from the parent's one.


(2) The initial value of sense representation is critical for most statistical clustering based approaches.

this paradigm usually has two limitations: (1) The performance of these approaches is sensitive to the clustering algorithm which requires the setting of the sense number for each word.


It is also capable of taking the ordering of words into account, and collecting information from arbitrary features associated with the context.

we demonstrate the effectiveness of our model by applying it to a specific application: predicting topics and sentiments in dialogues.


We summarize the experiment results in Table 1.

here, s t represents the sentence at time t, and u t represents additional features.


We have released the human annotation data at https://sites.google.com/ site/forrestbao/acl_data.tar.bz2 .

it is necessary to isolate review helpfulness prediction from its outer layer tasks and formulate it as a new problem.


For example, in task a, p(positive) = q 3 + q 4 + q 5 .

as recommended in the documentation, we apply repeated SGD over 20 reorderings of each corpus (for comparability, this was also done when fitting Word2Vec).


We thank the anonymous reviewers for their detailed and insightful comments on earlier drafts of this paper.

the graph is heterogeneous, with two types of nodes for the news sentences and tweets respectively.


M-L captures this notion using the difference in the cross-entropies according to each language model (LM).

we have demonstrated that extracting paraphrases from subsampled data results in higher precision domain-specific paraphrases.


The main limitation of the aforementioned methods is the dependence on simplified corpora and WordNet.

(2011), which both used simplified corpora.


In addition to that, certain street and landmark names might not be depicted at different zoom levels.

referential attributes of other landmark objects were not modelled due to data sparsity and also to reduce computational costs.


This strategy, however, turns out to be far less common in language use if a more discriminatory property is available, as in the example.

this seems to be the case, for instance, of the colour attribute.


as we will demonstrate in Section 3.5, the generated captions perform significantly better than the nearest neighbor captions in terms of human quality judgements.


We extend this intuition to the unsupervised setting of Google image search results and apply it to the lexical entailment task.

we also find some drop in accuracy on the longest paths (bucket 5), especially for wBLESS and BIBLESS, perhaps because semantic similarity is difficult to detect in these cases.


(2015), who try different ways of modulating the inclusion of perceptual input in their multi-modal skip-gram model, and find that the entropy of the centroid vector µ works well (where p(µ j ) = µ j || µ|| and m is the vector length): We calculate the directionality of a hyponymhypernym pair with a measure f using the following formula for a word pair (p, q).

examples of pairs in the respective datasets can be found in Table 1.


On WBLESS we compare to the reported results of Weeds et al.

since even cohyponyms will not have identical values for f , we introduce a threshold α which sets a minimum difference in generality for hypernym identification: s(p, q) > 0 iff f (q) > f (p) + α, i.e.


We use the word2vec tool (Mikolov et al., 2013) to train monolingual vectors, 6 and the CCA-based tool (Faruqui and Dyer, 2014) for projecting word vectors.

to achieve this goal, we replace transliteration by a new technique that captures more complex morpho-phonological transformations of historically-related words.


Romanian-English systems obtain only small (but significant for 4K and 8K, p < .01) improvement.

the donor language is used as pivot to obtain translations via triangulation of OOV loanwords ( §2.2).


To allow the low-resource system to leverage good translations that are missing in the default phrase inventory, while being stable to noisy translation hypotheses, we integrate the acquired translation candidates as synthetic phrases (Tsvetkov et al., 2013;Chahuneau et al., 2013).

the borrowing system only minimally overgenerates the set of output candidates given an input.


In this paper, we introduce a novel recurrent neural network based rule sequence model to incorporate arbitrary long contextual information during estimating probabilities of rule sequences.

(2011) discussed in Section 1, there are several other works using a rule bigram or trigram model in machine translation, Ding and Palmer (2005) use n-gram rule Markov model in the dependency treelet model, Liu and Gildea (2008) applies the same method in a tree-tostring model.


They first translate Japanese input into head final English texts obtained by the method of Isozaki et al.

any reordering of a(i, p) changes only the first term and the others are unchanged.


This is unfortunate since the CARs directly indicate the comprehensibility of the translated dialogues.

the translations by H S were created by first preparing a file containing all the sentences from the 40 problems in a randomized order and then asking a translator to translate the file sentence-by-sentence, without assuming any specific context.


For a given sentence, the greedy unsupervised RAE greedily searches a pair of words that results in minimal reconstruction error by an autoencoder.

we conducted experiments on the wMT metric task data.


Each of the above mentioned representations has a different strength in terms of encoding syntactic and semantic contextual information for a given sentence.

this is mainly caused by the flexible word ordering and the existence of the large number of synonyms for words.


Consider, for instance, the first reference (denoted as "1 R" in Table 2) and their translations.

to leverage the different advantages and focuses, in terms of benefiting evaluation, of various representations, we concatenate the three representations to form one vector representation for each sentence.


The results show that ISF achieves relatively lowε s and increases the domain separation error.

we use standard features including entity types, entity head words, contextual words and other syntactic features derived from parse trees.


In order to capture enough variations, we randomly initialize the set of filters to detect different structure patterns.

the parser works substantially better on the TREC dataset since all questions are in formal written English, and the training set for Stanford parser 5 already includes the QuestionBank (Judge et al., 2006) which includes 2,000 TREC sentences.


The n-gram convolutions help identify local context and map that to a new higher level feature space.

this gives a high level representation of a movie.


Applying multiple hidden layers in succession can require exponentially less data than mapping through a single hidden layer (Bengio, 2009).

(2010) preprocessed the text by stemming, down-  casing, and discarding feature instances that occurred in fewer than five reviews.


SkipGram can be categorized as one of the simplest neural language models (Mnih and Kavukcuoglu, 2013).

then, by using the unified form, we extract the factors of the configurations that they use differently.


Namely, it first counts all the co-occurrences in D, and then, it leverages the gathered co-occurrence information for estimating (possibly better) parameters.

these results are shown in the first and second rows in table 4.


θ ∈ R d and γ ∈ R d and defines a joint probability of y and z conditioned on x as follows: where Y(x, z) is the set of all possible label sequences for x and z, and HUCRF forces the interaction between the observations and the labels at each position j to go through a latent variable z j : see Figure 1 for illustration.

hUCRF with pre-training with Brown clusters (hUCRF B ) and CCA-based clusters (hUCRF C ) further improves performance to 91.36% and 91.37%, respectively.


This dataset is an interesting * Work done when student at University of Trento resource that we make available to the research community 1 .

such clues are retrieved using a BM25 model on CPDB.


To summarize, our algorithm consists of the following steps: 1.

unlike our work, none of these authors developed automatic methods for studying syntactic properties like word order, nor did they utilize recent advances in the field of word alignment algorithms.


Similar results at the lexical level have been reported using automated annotations (Prud'hommeaux and Rouhizadeh, 2012;Rouhizadeh et al., 2013).

we then calculate two above statistics for each shuffle and count the number of times the observed values exceed the values produced by the 1000 shuffles.


2Ranking Order Determination: given the comparison set and the selected dimension, determine the order in which the involved items are ranked within the comparisons, in an ascending or descending order?

we build a Naïve Bayes model using cooccurrences as a baseline to predict proper dimensions.


Its peak is 8,848 metres (29,029 ft) above sea level.

the probability of ith predicate in Freebase is chosen as the comparison dimension given superlative S and its context C can be written as: where z i is estimated using a sigmoid function: where n is the number of candidate predicates for superlative S, m is the length of concatenated vector V , W m×n is the parameter matrix, b is the bias vector, and σ i is the sigmoid function that applies to the i-th element of argument vector.


A chance baseline is obtained by randomly ranking a concept's nearest neighbors.

this approach allows for zero-shot learning, where the model can predict how an object relates to other concepts just from seeing an image of the object, but without ever having seen the object previously (Lazaridou et al., 2014).


In extracting predicate-argument structures, it is not possible to directly extract a coordinated noun phrase "wine and sake" as a direct object of the verb "drank".

basically, the cases "ga", "o" and "ni" in the corpus correspond to "nsubj", "dobj" and "iobj", respectively, however, we should apply the alternative conversion to passive or causative voice, since the annotation is based on active voice.


Recent work on supertagging using a feedforward neural network achieved significant improvements for CCG supertagging and parsing (Lewis and Steedman, 2014).

the output layer represents probability scores of all possible supertags, with the size of the output layer being equal to the size of the lexical category set.


, w N ), the embedding feature of w t (for 1 ≤ t ≤ N ) is obtained by projecting it onto a n-dimensional vector space through the look-up table L w ∈ R |w|×n , where |w| is the size of the vocabulary.

their attempt to tackle the third problem by pairing a conditional random field with their feed-forward tagger provided little accuracy improvement and vastly increased computational complexity, incurring a large efficiency penalty.


The oracle is very efficient, computing loss in O(n), compared to O(n 8 ) for the only previously known dynamic oracle with support for a subset of non-projective trees (Gómez-Rodríguez et al., 2014).

to do so, we can implement WEAKLYCONNECtED so that the first call computes the connected components of A in linear time (Hopcroft and tarjan, 1973) and subsequent calls use this information to find out if two nodes are weakly connected in constant time.


Performance of the Random-reduced classifier is also better than Unfiltered, with the overall improvement largely resulting from increased recall, but below PRA-reduced.

this assumption is violated when the knowledge base is incomplete which can lead to sentences containing instances of relations being wrongly annotated as negative examples.


The results in the first two rows indicate that adding unsmoothed lexical information to the method of Xu et al.

table 1: Data sets and their size (number of sentences).


Relation Extraction (RE) is the task of recognizing relationships between entities mentioned in text.

this work was funded in part by Alberta Innovates Center for Machine Learning (AICML), Natural Sciences and Engineering Research Council of Canada (NSERC), Alberta Innovates technology Futures (AItF), and National Institute of Informatics (NII) International Internship Program.


One is the 50-d embeddings provided by SENNA (Collobert et al., 2011).

we can distinguish these two paths by virtue of the attached subtrees such as "dobj→commandment" and "dobj→ignition".


We propose a new approach to the task of fine grained entity type classifications based on label embeddings that allows for information sharing among related labels.

for example, if the (ordered) top-k labels are person, artist, and location, we output only person and artist as the predicted labels.


Our approach is evaluated on two datasets, one comprising clinical reports and the other comprising biomedical abstracts, achieving state-of-the-art results.

one can query Wikipedia for the test mention's string, then employ the titles of the retrieved pages as alternate mention names.


For example, Open IE chooses different predicate and argument boundaries and assigns different relations between them.

while our comparisons are valid only for the tested tasks and systems, they do provide valuable evidence for the general question of effective intermediate structures.


But it is hard for machine to capture such long distance information.

it is hard for the classifier to recover "它(it)", e.g., "*pro* 这种？(*pro* that kind?)"


1 In addition, motivated by work on overt pronoun resolution, we hypothesize that AZP resolution can be improved by exploiting discourse information.

early approaches to AZP resolution employed heuristic rules to resolve AZPs in Chinese (e.g., Converse (2006), Yeh and Chen (2007)) and Spanish (e.g., Ferrández and Peral (2000)).


Theorem 1 implies that, at each step k, R k in (9) can grasp the first (2 k − 1)-th terms of S, whereas S k in (4) can grasp only the first k-th terms of S. Thus, given the number of steps K, Co-Simmate is always more accurate than Co-Simrank because R K is exponentially closer to S than S K to S. Convergence Rate.

the experiments show that Co-Simmate can be 10.2x faster than the state-ofthe-art competitors.


For a given set of subtopic candidates with annotated subtopics, {(Sn, yn)} (1≤n≤N), we need to estimate the optimal weight w. Empirically, the optimal weight w should minimize the error between the predicted partition y ' and the true partition y, and it should also have a good generalization capability.

these results confirm that the similarity between two subtopic candidates is affected by many factors and our methods can achieve more desirable query subtopics by learning a similarity measure.


Compared with AC, SC achieves 1.86% precision improvement, 2.76% recall improvement, and 2.32% F-Measure improvement.

it is not a trivial task because the underlying intents of the same query may be different for different users.


During backtracking, pseudo-nodes are expanded into an acyclic directed graph, i.e., our final category topic hierarchy H c .

after replacing the tags in a CT (see Fig.1) with the topics they belong to, we can then get a topic hierarchy H a = {T a , R a } for each article a.


The present work is closely related to previous approaches involved in TempEval campaigns Verhagen et al., 2010;Uz-Zaman et al., 2013;Llorens et al., 2015).

the results of an out of the competition version of the SPINOZAVU team explained in (Caselli et al., 2015).


As a prominent example, we consider Section 5 of the ACE-2005 event annotation guidelines, 2 which provides a description for each event type.

to score the binary label assignment (x i , y i ), we use a small set of features that assess the similarity between x i and t's given seed list.


To assess our approach, we compare it (Section 4) with the common fully-supervised approach, which requires annotated triggers for each target event type.

as we match performance of a state-of-the-art fully-supervised system over the aCE-2005 benchmark (and even surpass it), we offer our approach as an appealing way of reducing annotation effort while preserving result quality.


One core is 2.4 times as fast at language identification and 1.8 to 6 times as fast at part-of-speech language modeling.

we tested the original Python, a Java implementation that "should be faster than anything else out there" (weiss, 2013), a C implementation (Lui, 2014), and our replica in hardware.


For each studied language we use Stanford CoreNLP  for EN and ZH, and TreeTagger (Schmid, 1994) for ES to produce  Table 2: F-score Results the tokens and the POS tags.

instead, we seek a direct transfer approach ( Figure 1) to cross-lingual NER (also classified as transductive transfer learning (Pan and Yang, 2010) and closely related to domain adaptation).


Our aim is to increase the type-level coverage of FN.

despite many years of work, most of the words that one confronts in naturally occurring text do not appear at all in FN.


In total, IWNLP enhances the results of Mate Tools in five of the six test cases.

to reduce stress from the servers and to easily reproduce our parsing results, we parse the latest of the monthly XML dumps 4 from Wiktionary.


For SVM, we employ the radial basis function kernel (RBF) and we use the wrapper provided by Weka for LibSVM (Chang and Lin, 2011).

many words have undergone transformations by the augmentation of language-specific diacritics when entering a new language (Ciobanu and Dinu, 2014a).


(2015), we further appended the original messages (as if parroted back).

to ensure that the denominator is never zero, we assume that, for each i there exists at least one reference r i,j whose weight w i,j is strictly positive.


Although distributional word vector dimensions cannot, in general, be identified with linguistic properties, it has been shown that some vector construction strategies yield dimensions that are relatively more interpretable (Murphy et al., 2012;Fyshe et al., 2014;Fyshe et al., 2015;Faruqui et al., 2015b).

table 1 shows the size of vocabulary and number of features induced from every lexicon.


Every word in the lexicon is associated with these properties.

such analysis is difficult to generalize across models of representation.


It also shows that our late fusion methods are more effective than our early fusion method.

a part of the work for improving such thesauri focuses on the filtering of the components of the distributional contexts of words (Padró et al., 2014;Polajnar and Clark, 2014) or their reweighting, either by turning the weights of these components into ranks (Broda et al., 2009) or by adapting them through a bootstrapping method from the thesaurus to improve (Zhitomirsky-Geffet and Dagan, 2009;Yamamoto and asakura, 2010).


We use part-of-speech information and dependency arcs from the gold annotation to extract noun phrases containing adjectives.

these values are different from the dependency length differences for noun phrases without a right dependent (panel a and b).


In the light of recent research (Volkova et al., 2013;Hovy, 2015;Jørgensen et al., 2015), we explore the hypothesis that these biases transfer to NLP tools induced from these resources.

we then map each review to the NUTS region corresponding to the location.


In all of these cases, vocabulary does not factor into the differences, since we are at the POS level.

these models perform better on texts written by certain people, namely those whose language is closer to the training data.


Probabilistic topic models (Blei et al., 2003) are broadly used to uncover the hidden topics of tweets, since the low-dimensional semantic representation is crucial for many applications, such as product recommendation (Zhao et al., 2014), hashtag recommendation (Ma et al., 2014), user interest tracking (Sasaki et al., 2014), sentiment analysis (Si et al., 2013).

the hyperparameters are set as follows: for all the models, we set α = 50/K, β = 0.01; for twitter-LDA, twitterUB-LDA and twitter-BtM, we set γ = 0.5.


The vital segments for two classes are used to train the supervised classifier which can best classify each sentence to the correct author's class.

the co-authored documents include Web pages, books, academic papers and blog posts.


Most of the models above are limited to model linear topical dependencies between words, word topical dependencies can also be modeled by a non-linear way.

and we define the topic model based on EGRF as Extended Global Topic Random Field(EGTRF).


In practice, because of the large vocabulary size N , designing such a matrix is computationally prohibitive.

we count the term P [w i |s j ] only once when computing the likelihood of the sentence.


Related also is the extensive work done in spatio-temporal modelling of meme spread.

the likelihood (1) is commonly approximated by considering subregions of t and assuming constant intensities in sub-regions of t (Møller and Syversveen, 1998;Vanhatalo et al., 2013) to overcome computational difficulties arising due to integration.


One example is application of Hawkes processes (Yang and Zha, 2013), a probabilistic framework for modelling self-excitatory phenomena.


In terms of performing distributed representation learning for output variables, our proposed model shares similarity with the structured output representation learning approach developed by Srikumar and Manning (2014), which extends the structured support vector machines to simultaneously learn the prediction model and the distributed representations of the output labels.

, s * T } using the standard Viterbi algorithm (Rabiner and Juang, 1986) for each labeled source training sentence and each target test sentence.


Indeed, the marginal productivity gains observed with QE at a global level become statistically significant in specific conditions, depending on the length (between 5 and 20 words) of the source sentences and the quality (0.2<HTER≤0.5) of the proposed MT suggestions (our third contribution).

to check this hypothesis we compared the HtER scores obtained in the two conditions (colored vs. grey flags), assuming that noticeable differences would be evidence of unwanted psychological effects.


In this work, we use ReLu (Dahl et al., 2013) as the activation function; • w (1,j) is the parameters for the jth convolution unit on Layer-1, with matrix •ĉ i (0) is a vector constructed by concatenating word vectors in the k-sized sliding widow i; • b (1,j) is a bias term, with vector To distinguish the phrase pair from its context, we use one additional dimension in word embeddings: 1 for words in the phrase pair and 0 for the others.

they both extend the neural network joint model (NNJM) of Devlin et al.


As more sub-models for longer distance reordering being integrated, the translation performance improved consistently, though the improvement leveled off quickly.

to select appropriate rules, more effective criteria are required.


A translation quality gap between genres has also been observed in past OpenMT evaluation campaigns.

next, while spelling errors are common in all topics, its abundance is most prominent in culture.


The idea is to count those words co-occurring with k as the context of j, where k ∈ V l 2 is the translationally equivalent word of j ∈ V l 1 .

the monolingual objective stems from the distributional hypothesis (Harris, 1954) and optimizes words in similar contexts into similar embeddings.


Year Per-track 69.77% 58.58% Per-album 70.60% 57.15% Table 2: Initial experiment varying class and instance granularity For the follow-up experiment, we focus on the task of classifying at the per-album level of granularity, as ultimately this is the level at which the original annotations are obtained.

an instance is represented by a vector with four real values.


M 2 was assessed by comparing its output against that of the official Helping Our Own (HOO) scorer (Dale and Kilgarriff, 2011), itself based on the GNU wdiff utility.

despite these many proposed metrics, no prior work has attempted to answer these questions by comparing them to human judgments.


As demonstrated in previous work, this numeric representation of words has led to big improvements in many NLP tasks such as machine translation (Sutskever et al., 2014), question answering (Iyyer et al., 2014) and document ranking (Shen et al., 2014).

in between is the pronunciation of the sentence in Chinese, called PinYin, which is a form of Romanian phonetic representation of Chinese, similar to the international Phonetic Alphabet (iPA) for English.


These four radicals altogether convey the meaning that "the moment when sun arises from the grass while the moon wanes away", which is exactly "morning".

web search leverages many kinds of ranking signals, an important one of which is the preference signals extracted from click-through logs.


Using parse tree patterns to judge the grammaticality of a sentence is not new.

fragment was among the 28 error types introduced in the CoNLL-2014 shared task (Ng et al., 2014), but the test set used in the task only contained 16 such errors and is too small for our purpose.


For example, if X is Herman Melville's Moby-Dick, W can be Melville's complete works.

they are not directly used to resolve unigram sparsity.


Function ϕ d measures the textual similarity between document d and the document d 0 to smooth.


The basic idea is to smooth a document language model with the documents similar to the document under consideration through clustering (Liu and Croft, 2004;Tao et al., 2006).

with these works, we have proposed a language model smoothing framework which incorporates social factors as a regularizer.


These models broadly fall into two categories: text-based and network-based methods.

the text-based profile locations are noisy and only 1-3% of tweets are geotagged (Cheng et al., 2010;Morstatter et al., 2013b), meaning that geolocation needs to be inferred from other information sources such as the tweet text and network relationships.


Finally, training a predictor for the number of keywords yields further improvements (row 3) over a simple ratio of the number of input words.

the TF-IDF across different methods remains a strong unsupervised baseline (Hasan and Ng, 2010).


The system extracts a list of candidate keywords from a document and trains a decision tree over a large set of hand engineered features, also including TF-IDF, in order to predict the correct keywords on the training set.

these systems are generally limited as these need supervision and cannot scale to new data or data in other languages.


It has also been used as a feature to detect irony (Reyes et al., 2013).

we chose 184 topics split into 9 categories (politics, sport, etc.).


Table 2 presents the result in terms of precision (P), recall (R), macro-averaged F-score (MAF) and accuracy (A).

having it alone is not enough to find ironic instances.


Direct visual impression is that different users use different typing patterns to input Chinese.

singlesegNum(ssN) stands for the number of the segments whose length equals 1 in a sentence .


Among them,  and  improved tense prediction by using eventuality and modality labels.

着 (zhe), 了 (le), 过 (guo)) and time expressions (e.g., 昨天 (yesterday)).


• Local predictor with basic features (Local(b)) • Local predictor with basic features + dependency parsing features (Local(b+p)) • Local predictor with basic features + dependency parsing features + linguistic knowledge features (Local(b+p+l)) • Local predictor + all features introduced in Section 2.1 (Local(all)) • Conditional Random Fields (CRFs): We model a conversation as a sequence of sentences and predict tense using CRFs (Lafferty et al., 2001).

it is much more challenging to predict tense in Chinese conversations and there has not been an effective set of rules to predict Chinese tense so far due to the complexity of language-specific phenomena.


Since most of the soft headers on Wiktionary have a distinct background color from the rest of their containing tables, we initially added a rule that treated content cells that defined a background color in HTML or inline CSS as header cells.

each content cell is matched with the headers immediately up the column, to the left of the row, and in the "corners" located at the row and column intersection of the previous two types of headers.


This validity still holds when two EDUs are connected with a symmetric relation such as joint.

there is a number of recent studies employing RSt features for passage re-ranking under question answering (Joty and Moschitti, 2014;Surdeanu et al., 2014).


The results clearly show that the thread-level features are important, providing consistent improvement for all our learning models.

in the second example (Q u 4 ), the first two comments are classified as bad when using the basic features.


Our local classifiers are support vector machines (SVM) with C = 1 (Joachims, 1999), logistic regression with a Gaussian prior with variance 10, and logistic ordinal regression (McCullagh, 1980).

we also included another real-valued feature: max(20, i)/20, where i represents the position of the comment in the thread.


The goal of the feed forward processing is to calculate the similarity between a pair of questions (q 1 , q 2 ).

after producing the BOW and CNN representations for the two input questions, the BOW-CNN computes two partial similarity scores s bow (q 1 , q 2 ), for the CNN representations, and s conv (q 1 , q 2 ), for the BOW representations.


1 displays top 10 semantically similar words monolingually, across-languages and combined/multilingually for one ES, IT and NL word.

the results clearly reveal that, although both other BWE models critically rely on parallel Europarl data for training, and Gouws et al.


Exp II: Shuffling and Window Size Since our BWESG model relies on the pre-training random shuffling procedure, we also test whether the shuffling has significant or rather minor impact on the induction of BWEs and final BLI scores.

the architecture of our BWE Skip-Gram model for learning bilingual word embeddings from document-aligned comparable data.


The low-rank tensor models are also at least twice as fast to train as the full tensors: on a single core, training a rank-1 tensor takes about 5 seconds for each verb on average, ranks 5-50 each take between 1 and 2 minutes, and the full tensors each take about 4 minutes.

we filter the SVO triples to a set containing 345 distinct verbs: the verbs from our test datasets, along with some additional high-frequency verbs included to produce more representative sentence spaces.


When pattern X is allowed, both m a and m b are not directly associated with any natural language word, so we are able to further insert arbitrarily many (compatible) semantic units between the two units m a and m b while the resulting relaxed hybrid tree remains valid.

results showed that our system consistently yielded higher results than all the previous systems, including our state-of-the-art relaxed hybrid tree system (the full model, when all the features are used), in terms of both accuracy score and F 1measure.


In early experiments, Naive Bayes performed comparably to or outperformed SVM because the dimensionality of the feature space was relatively low.

overall, we achieve our best results when including both precedent and subsequent context along with the question in our feature space.


In this approach, they also chose to use a compressed feature set for n-grams and POS n-grams.

we used unigrams, bigrams, POS bigrams, and POS trigrams of a question and its immediately preceding and following context as feature sets.


A similar incongruity is observed in the sarcastic 'My tooth hurts!

consider the example "I love this paper so much that I made a doggy bag out of it".


Note that the 'native polarity' need not be correct.

(as in case of the tweet 'so i have to be up at 5am to autograph 7,000 pics of myself?


[E1] expresses the happiness emotion through English, and the anger emotion in [E2] is expressed through both Chinese and English, while the fear emotion in [E3] is expressed through a mixed English-Chinese phrase (holdØ4).

emotions in codeswitching texts differ from monolingual texts in that they can be expressed in either monolingual or bilingual forms.


This result indicates LinAdapt well captures the fact that users express opinions differently even with the same words.

after each evaluation run, we added an extra 1000 reviews and repeated the training and evaluation.


Our method is inspired by a personalized ranking model adaptation method developed by .

high variance indicates the word's sentiment polarity frequently changes across different users.


Failure to rec-ognize this difference across users will inevitably lead to inaccurate understanding of opinions.

for each testing case, we are estimating an independent classification model.


First, "奥巴马 (Obama)" is found in the emotion keyword list and tagged.

we rank the templates according to their frequency, and adopt a dominating set algorithm (Johnson, 1974)  We believe that human perception of an emotion is through recognizing important events or semantic contents.


To solve these problems, we translate the whole sentences but with reordering constraints ensuring that the opinionated segments are preserved during translation.

(2013) compare several of these approaches.


Second, whereas pagerank assigns an equal weight to the edges connected between an unseen word and its neighbor nodes, we consider their similarities as weights to construct a weighted graph such that neighbor nodes more similar to the unseen word may contribute more to estimate its VA ratings.

few studies have sought to predict the VA rating of words using regression-based methods (Wei et al., 2011;Malandrakis et al., 2011).


In this paper we report on the extension of the P obs model to also consider listener's visual behaviour.

both the target intended by the IG (o t ) and the one selected by the IF (o p ) were recorded.


A new tag addition method is proposed to solve this problem.

these semantic grammar approaches carry a high development cost and they can also lead to fragile operations since users do not typically know what grammatical constructions are supported by the system.


Firstly, we tune λ 1 by fixing λ 2 = 0.1.

dependency parsing is a crucial component of many natural language processing systems, for tasks such as text classification (Özgür and Güngör, 2010), statistical machine translation (Xu et al., 2009), relation extraction (Bunescu and Mooney, 2005), and question answering (Cui et al., 2005).


This shows that when trained using a small data set, the source language parser is more accurate than the supervised model.

figure 2 shows the average LAS for all 9 languages (except English) on different data sizes using different values of λ 1 .


In general, AMR substructures are graphs.

in general, AMR substructures are graphs.


From Table 1, we can see that the performance of the baseline transition-based system remains very stable when different dependency parsers used are trained on same data set.

for action that predicts the concept label (NEXT-NODE-l c ), we check whether the candidate concept label l c matches the frameset predicted by the semantic role labeler.


We report final AMR parsing results that show an improvement of 7% absolute in F 1 score over the best previously reported result.

the Charniak parser that is trained on a much larger and more diverse dataset (CHARNIAK (ON)) yields the best overall AMR parsing performance.


tain quantitative correlations between parts of the neural network and linguistic properties, in both speech (Wu and King, 2016;Alishahi et al., 2017;Wang et al., 2017) and language processing models (Köhn, 2015;Qian et al., 2016a;Adi et al., 2016;Linzen et al., 2016;Qian et al., 2016b).

we also find that representations from higher layers are better at capturing semantics, even though these are word-level labels.


We compare our method ALLSmooth (CCWM) with Bahdanau et al.

the existing PBSMt methods that used external resources to translate unknown words for SMt are hard to be directly introduced into NMt, because of NMt's soft-alignment mechanism (Bahdanau et al., 2015).


This dependency parsing result will be transformed in another step for traversing the tree, which will be described in the next section to create a dependency tree.

it has been shown that the combination of words and their dependency information can boost performance.


In this work, we proposed a method in which the Seq2Dep NMT model is trained by utilizing syntactic dependencies to provide the model more abundant information.

to utilize the information of both "head" words and syntactic dependencies between them to produce better output.


They report some improvements in translation quality arguing that the attention model has learned to better align source and target words.

in order to compare attentions of multiple systems as well as to measure the difference between attention and word alignment, we convert the hard word alignments into soft ones and use cross entropy between attention and soft alignment as a loss function.


In word alignment, most target words are aligned to one source word.

to the best of our knowledge there is no study that provides an analysis of what kind of phenomena is being captured by attention.


Directly learning from PA based on a forestbased objective in LLGPar is first proposed by Li et al.

second, the two settings lead to very similar performance on Biaffine, without a clear trend.


Both GN3Par and LTPar suffer from the inexact search problem.

the first approach is based on a forest-based training objective for two CRF parsers, i.e., a biaffine neural network graph-based parser (Biaffine) and a traditional log-linear graph-based parser (LL-GPar).


The second approach is based on the idea of constrained decoding for three parsers, i.e., a traditional linear graphbased parser (LGPar), a globally normalized neural network transition-based parser (GN3Par) and a traditional linear transition-based parser (LTPar).

as our reviewers kindly point out, more extensive experiments and systematic analysis are needed to really understand this interesting issue and provide stronger findings, which we leave for future work.


On top of the neural network, we introduce a probabilistic structured layer, defining a conditional log-linear model over nonprojective trees.

l is not negligible comparing to n. It should be noticed that in our labeled model, for different dependency label l we use the same vector representation ϕ(x i ) for each word x i .


Experimental results using manually annotated datasets for lexical paraphrase showed that the proposed method outperformed bilingual pivoting and distributional similarity in terms of metrics such as MRR, MAP, coverage, and Spearman's correlation.

in this study, we conducted the evaluation by applying Pearson's correlation coefficient with a five-step manual evaluation using five datasets constructed by SemEval (Agirre et al., 2012(Agirre et al., , 2013(Agirre et al., , 2014(Agirre et al., , 2015(Agirre et al., , 2016.


Furthermore, our aim was to calculate not the strength of co-occurrence (relation) between words, but the paraphrasability.

we used this dataset to evaluate the acquired paraphrase pairs by MRR and MAP, following Pavlick et al.


With respect to the conditional paraphrase probability and PMI, it is necessary to consider up to the 400th place to cover all correct paraphrases.

we employed the conditional paraphrase probability of bilingual pivoting given in Equation (1), the symmetric paraphrase score of PPDB given by Equation (3), and distributional similarity as baselines, and compared them with PMI shown in Equation (7) and the MIPA score given in Equation (11).


We approximate this marginalization by constructing a tree where the root is the predicate, p, the branches are likely sequences of arguments, and the leaves are the word and label for which we need to estimate a probability, w:l. Formally, we define this tree of possible sequences as: where w f,0 :l f :0 = p:P RED; k and T are thresholds; and argmax k (q) is the k word:label pairs that have the highest probability of being the next argument given the sequence q according to the PRNSFM.

as shown in Figure 2, we use two different embedding layers, one for word values and one for semantic labels, and the two embedding vectors are concatenated before being passed to the LSTM layer.


Our models do not require any handcrafted iSRL annotations for training, and thus can be applied to all predicates observed in large unannotated data on which they are trained.

for a good reader, a reasonable interpretation of the second loss should be that it receives the same A0 and A1 as the first instance.


It involves more than 100 features but does not include word embeddings, and hence we compare it with the PSL models of the weak base setting.

(3) We implement and discuss the transitivity inter-and intra-layers and conclude the transitivity within the observed layer brings the most performance gain.


We start from describing the features for each lexicon pair.

empirically, those path p whose length exceed 10 are dropped as the inference chain is too long.


False alarms come from pairs that contain various unknown Chinese compound words that E-Hownet does not include, e.g., 分 給(distribute to) is composed of 分(issue) and 給(give).

in our PSL models, all possible feature-layered transitivities between pairs are explored.


For recall, the feature-layer transitivity (PSL(WeakBase TrFeat)) enables the model to reach more words for a better recall, while the enrichment of the prior knowledge in PSL(WeakBase TrObv) helps to eliminate uncertainty but decreases recall.

we define the cohesion score of a semantic type with E-Hownet to model the generality.


They also used word insertions and deletions for forced decoding, but they used a high penalty for all insertions and deletions.

for each stack, a beam of the best n hypotheses is kept to speed up the decoding process.


All the embeddings are fine-tuned during training by back-propagating gradients.

the model  performs very well despite being fully character based.


Using higher-order n-grams requires more data for training.


Instead of using the Max-Margin criterion (Taskar et al., 2005) adopted by previous neural network models for CWS (Zheng et al., 2013;Pei et al., 2014;Chen et al., 2015a,b), we try to directly maximize the log-probability of the correct tag sequence following Lample et al.

cWS accuracies can drop gravely on cross-domain corpora.


Further, we have created multiple documents where each document contains N continuous segments from the original threads.

of this, the proposed IB based approach requires a limited number of involved computations, more precisely, linear in terms of number of text snippets.


Here, the concept representation c Mult is a multinomial distribution over the properties in Q.

we use an extension of LDA (Blei et al., 2003) to implement our hypotheses on the usefulness of overarching structure, both commonalities in selectional constraints across predicates, and cooccurrence of properties across concepts.


In the SimLex-999 dataset which focuses on the word similarity, the cognition (lexical relation) and the sentiment modules turned out to be important.

noting that the different types of words may have different sensitivity toward the modules, we adjusted the relative weights for a particular aspect of interest to be from 0.1 to 3.5 while maintaining others to 1.0.


Moreover, because several senses are assigned to the current word with probabilities, we leverage all the related senses to predict the context words.

adopting an online procedure, FCSE-1 clusters the contexts of one word incrementally.


Some bi-gram probabilities have several peaks, and they vary in accordance with parts of speech.

the inference for NPYLM is introduced first; then, the inference of PYHSMM is explained.


w n represents the n-th word in each sentences.

it might be difficult for the PYHSMM-D to separate these two contexts in the case of completely unsupervised training.


We begin with a tokenized sentence which we then convert to a sentence matrix, the rows of which are word vector representations of each token.

1 For this dataset we report the Area Under Curve (AUC), rather than accuracy, because it is imbalanced.


We fixed the filter region sizes and the number of feature maps as in the baseline configuration, thus changing only the pooling strategy or pooling region size.

before training, we under-sampled negative instances to make classes sizes equal.


Using these scoring functions, we assign scores to all possible pairs of conjuncts.

given a coordinator word (e.g., "and," "or," or "but"), a system must return each conjunct span if the word actually plays the role of a coordinator; otherwise, NONE is output for the absence of coordination.


In this article, we propose to investigate a new problem consisting in turning a distributional thesaurus into dense word vectors.

synonyms that are not part of the injected knowledge.


rank-fitting is significantly the worst method and its association with counterfitting is better than retrofitting for P@1 only, but not significantly.

first, T cnt significantly outperforms GloVe and SGNS for all measures 4 .


Our last choice is based on a less constrained form of matrix factorization where T is decom-posed into two matrices in such a way that U m, n. U and V are obtained by minimizing the following expression: where the first term minimizes the reconstruction error of T by the product U • V while the second term is a regularization term, controlled by the parameter λ for avoiding overfitting.

table 2 also shows that there is still room for improvement for reaching the level of the initial thesaurus t cnt .


Each synonym is given with its [rank] among the neighbors of the entry and its similarity value with the entry gration of external knowledge into the thesaurus before its embedding is clearly effective as illustrated by the significant differences between SGNS+retrofit(K) and svd(T cnt +K)+retrof(K).

the relevance of the semantic neighbors of an entry strongly decreases as their rank increases and the first neighbors are particularly important.


The sentences in all datasets were tokenized, compound-split, and lemmatized, and for each target word we automatically determined the set of possible senses, given its context and inflec-  Table 3: WSD accuracy on baselines, UKB, and the three variants of our model (ρ = 0.9) on all test sets.

if by using the SN we intend to learn clear separations between different senses of a word, it attends to reason to limit its application to those cases, while monosemous words can be sufficiently well trained by the usual corpus-based approach, and act as semantic anchors in the broader vector space.


The extent of the regularizer's influence on the model is adapted by a mix parameter ρ ∈ [0, 1]: the higher the value of ρ, the more influence the SN data has on the model, and vice versa.

we computed the Mean Averaged Precision (MAP) score by macro-averaging the AP scores over the set of frames.


Manually inspecting some of the simplifications made, we find that when it comes to sentence splits, both MT-based simplifiers seem to be able to perform this type of transformation in an accu-  rate way.

as we are going to show in this section, standard end-to-end systems without special adaptation to TS do not succeed in learning alternative formulations of the original text.


Taito Corporation's homepage is http://www.taito.com.

each predicate becomes an STT with two entity types (the type of the subject, which is the instance entity, and the type of the object) in V ; a single relation between the two types (the predicate) in R; and two simple initial templates in L: where (PReDICATe) is replaced with the relevant predicate.


The text is annotated with the r-NE tags.

a: Figure 4: LSTM language model training.


Since available video/text pairs are limited in size, the direct application of end-to-end deep learning is not feasible.

let f .., f i+(l−1) be a substring, of length l, of f that corresponds to a single recipe sentence.


Considering the likelihood of object recognition and the likelihood of a combination of r-NEs included in the sequence, we set the likelihood P (e) that e appears as follows: where P (e) is the average of the probability of the result of object recognition: This value indicates the likelihood of object recognition.

we can say that our method is promising to solve this novel task.


Hereafter, we only focus the frames with the procedure-related objects listed in x-NEs, and describe the sequence of such frames as follows: where f i is the i-th frame and |f | is the length of the sequence.

it is reasonable for the procedural text generation to focus more on object recognition than motion recognition.


The third model proposed by us is Leaf-Composition-Tree LSTM (LCT-LSTM).Different from the LT-LSTM and TC-LSTM, this model not only takes the composition nodes into consideration when building the LSTM layer, but also makes sum of the outputs of two modules mentioned above.

during the forward propagation, we add the output of two modules and take the result as the output of the whole model.


Suppose we have vector representations of an argument a and a Wikipedia article w. The similarity score, sim(a, w) is simply the dot product of the two representations, aw T .

for all methods that use the presence of Wikipedia articles, we use several variations of a corpus to determine how well the methods leverage topic-specific articles, as opposed to randomly selected articles.


We then propose modifications of the supervised models to leverage external data.


We run LSTM1 to get the hidden representation {h f 1 ,h f 2 , ...,h fn } and run LSTM2 to get the hidden representation {h b 1 ,h b 2 , ...,h bn }.

the representation of the words as continuous vectors (word embedding) are proved more powerful than discrete representation (Bengio et al., 2003) (Mikolov et al., 2013b).


Thirdly, to capture document level and cross-document level clues without complicated inference rules.

the wider context that "She wants to cal-l her pregnant daughter Saba in Sweden to see if she has delivered."


Such resources have been largely ignored by the mainstream statistical NLP research, because they were not specifically designed for NLP purpose at the first place and they are often far from complete.

2 Approach Overview A typical supervised name tagger is presented in (Lample et al., 2016), consisted of Bi-directional Long Short-Term Memory networks (Bi-LSTM) and CRFs.


They observed that NMT systems are also more accurate at producing inflected forms, but they perform poorly when translating very long sentences.

if the translator did not apply any changes, the system automatically assigned the highest quality rating -"Unchanged".


To validate, whether the post-edits are of good quality, we performed quality assessment of the post-edits according to the LISA Quality Assurance model 3 .

errors related to accuracy, especially, mistranslation and omission errors, occur more often in NMT outputs.


Actually, there have been some effective methods to deal with the situation of translating language pairs with limited resource under different scenarios (Johnson et al., 2016;Sennrich et al., 2016a;.

our approach can be easily applied to any endto-end attention-based NMT framework.


As shown in Figure 2, when predicting the words y 2 and y 3 , we want to attend more information of corresponding words x 2 and x 3 .

in §3.1, we acquired the partially aligned corpora with the phrase pairs and monolingual sentences.


• Erroneous gold labels: Six instances (25%) were actually correctly labeled as "usage" by the system, whereas the gold label was wrong ("I really love the smell of fresh laundry, and the smell of Downy.").

in addition to opinions on product quality, reviewers often share how, where, or why they use the product.


Additionally, the ASR system can be fine-tuned in parallel with the emotion branch by updating the layers of the ASR network.

to fully understand a current emotion expressed by a person, also knowledge of the context is required, like facial expressions, the semantics of a spoken text, gestures and body language, and cultural peculiarities.


The progressive architecture complements information from the ASR network with SER representations trained end-to-end.

in both experiments, dropout with the rate of 0.25 was applied to averaged representations.


Table 1 summarizes our experiments on our proposed local attention models and compares them to the baseline model using several possible scenarios.

for our decoder target, we re-mapped the original target phoneme set from 61 into 39 phoneme class plus the end of sequence mark (eos).


Finally, the decoder task, which predicts the target sequence probability at time t based on previous output and context information c t can be formulated: For speech recognition task, most common input x is a sequence of feature vectors like Mel-spectral filterbank and/or MFCC.

the local property helps our attention module focus on certain parts from the speech that the decoder wants to transcribe, and the monotonicity property strictly generates alignment left-to-right from beginning to the end of the speech.


It contains 113438 words for training and 12753 for testing (12000 unique words).

x ∈ R S×D where D is the number of features and S is the total frame length for an utterance.


Despite the fact that we reset the memory for the Attentive RNN-LM at each sentence boundary whereas the caches for the baseline systems span sentence boundaries, our best Attentive RNN-LM is within 1 perplexity point of the system of (2017) (which is allowed to cache 2,000 previous hidden states), and has a lower perplexity than all of the other baselines.

the words in the X-axis (horizontal) are the inputs at each timestep and the words in the Y-axis (vertical) are the next (or predicted) words.


An LM provides a probability for a sequence of words in a given language, reflecting fluency and the likelihood of that word sequence occurring in that language.

we see our "attentive" RNN-LM (see §3) as a generalized version of these models as we rely on the encoded information in the hidden state of the RNN-LM to represent previous input words and we use a set of attention weights (instead of a key) to retrieve information from the past inputs.


The purpose of our attention mechanism is to enable an RNN-LM to bridge long distance dependencies in language.


IGC thus falls on a continuum between chit-chat (open-ended) and goal-oriented task-completion dialog systems, where the visual context in IGC naturally serves as a detailed topic for a conversation.

we define idf(h, D) = log |D| |{d∈D:h∈d}| , where we set N =10 to cut short each N-best list.


These studies, however, did not incorporate dynamic representations in the input layer.

the resulting novel model, Dynamic Neural text Model, uses the dynamic word embeddings that are constructed from the context in the output and input layers of an RNLM, as shown in Figures 1 and 2.


In addition, dynamic updates of both the input and output layers (D) further improved the performance from those of either the output (C) or input (B) layer.

our proposal incorporates the copy mechanism through the use of dynamic representations in the output layer, integrating them with dynamic mechanisms in both the input and output layers by applying dynamic entity-wise representation.


Then we convert the parse tree into binary with right branching and we remove the internal nodes that have only one child so that our binary tree-structured models can be applied.

we argue that these tags are worth considering when we compose the semantics in the tree-structured neural networks.


The MSE per dimension is visual-    Table 3: Results of the binary sentiment classification task for each language with each translation matrix.

for this task, we built a regressor for each of the three dimensions whose input is a 300-dimension vector and whose output is a real number from 1 to 9.


We opt to use offline alignment to show that such a low-cost approach does, in fact, capture a significant part of the relationship between words of different languages when it comes to sentiment.

this would serve to demonstrate the (expected) limits of the linear transformation model's in handling polysemy and subsequently its impact on the topological mapping of sentiment in vector space.


For this task, we built a regressor for each of the three dimensions whose input is a 300-dimension vector and whose output is a real number from 1 to 9.

a single transformation (linear in the case of our work) is sufficient to learn a projection which allows one to use labeled English data to aid in sentiment analysis.


first, we compare the effects of different features on target target-dependent sentiment analysis.


We call such lexicon-based features as lexicon embeddings because embeddings are a feature input of deep learning model and describe the properties of a word or a phrase.

the resulting matrix has dimension d × (s + m − 1).


The model with CharAVs and LexW2Vs built on top of GoogleW 2V vectors and Dependen-cyW2Vs is effective in order to increase classification accuracy.

to construct embedding inputs for our model, we use a fixed-sized word vocabulary V word and a fixed-sized character vocabulary V char .


Based on these equations, adding dependencybased word vector T is corresponding to considering a composite input [x i , T ] to the GRNN cell that concatenates the advanced continuous word embeddings and advanced dependency-based word embeddings.

gRUs have fewer parameters (U and W are smaller) and thus may train a bit faster or need less data to generalize.


The first convolution produces a fixed-size character feature vector named mgram features by extracting local features around each character window of the given word and using a max pooling over vertical character windows.

we modify Bi-GRNN of (Chung et al., 2014) into Bi-CGRNN to take word embeddings in order to produce a sentence-wide representation from sentence compositions.


The usage in the first sentence, on the other hand, can be reliably disambiguated due to the post-modifying phase news agency.

we break the performance down by the depth of each post in a given thread, and present the results in Figure 3.