We explore a gated recurrent neural network (GRNN) to address this, which is similar in spirit to LSTM Chung et al., 2015), but empirically runs faster.

the transition function of the GRNN used in the work is calculated as follows where stands for element-wise multiplication, i t and f t represent the reset gate and update gate, respectively.


So we introduce a simple attention mechanism to consider the importance of different state vectors.

for each sentence s i in one document d, which contains the sentences vectors s 1 , s 2 , .., s m , we integrate the weights into bi-directional state vector h i and h i .


Specifically, for each sentence s i in one document d, which contains the sentences vectors s 1 , s 2 , .., s m , we integrate the weights into bi-directional state vector h i and h i .

we use the context vector to measure the importance of the sentences.


We use the document representation as features for identifying deceptive opinion spam.

a linear layer is added to transform the document vector into a real-valued vector, whose length is class number C. A softmax function is added to convert real vector to conditional probability for document classification.


(2014)'s settings, in order to compare the neural model with the state-of-the-art discrete model with SVM.

all results are reported by using ten-fold cross-validation.


Another issue we face is that topics often contain many words that are far away from each other in the embedding space.

the true distributions of topics often form two or more dominant clumps.


Since the given term includes many patterns of segmental context, we constrain the pattern to one "contiguous" segment.

the probabilities of a discontiguous segment, such as p(today|It's, umm), are not included in the mixture.


Even though we select prototypes in the same way as (Guo et al., 2014), we use prototypes in a very different manner: we consider prototypes as the basis for representing labels, whereas prototypes are mainly used as additional features in prior works (Haghighi and Klein, 2006;Guo et al., 2014).

prototypes are previously used on the input side, while we use them on the label side.


Meanwhile, as seen in Section 3.1, Turkish is the least ambiguous of these languages (with a mean of 1.60 parses per word compared to 3.10 for Russian and 9.11 for Arabic), so surface context is not much less informative than disambiguated context.

turkish combines the greatest need for full context with higher relative informativeness of the surface context.


Although AsynGrad does the weight update in parallel from different threads, from the viewpoint of w w w it simply receives a sequence of gradient updates (but with potential errors, which is the major difference from non-parallel SGD learning).

the AsynGrad learning problem can be casted as a traditional SGD learning problem, and the only difference compared with traditional SGD is that the gradients are with errors.


This kind of method has been adopted in many studies (Filippova and Strube, 2007;Barzilay and Lapata, 2008;Burstein et al., 2010).

filatova and Hatzivassiloglou (2004) extends entity grids to model semantical relations between entities, which provides a possible further improvement for our models.


We approach this problem using neural networks.

this work makes two contributions to improve stochastic tree modeling with neural networks.


Thus, we developed analogous techniques to encode supertags based on their underlying tree structure.

to embed a supertag, we embed each node, group the resulting vectors to form a tensor, and then summarize the tensor into a single vector using a series of convolutional neural networks.


We first embed each node independently, then combined the vectors to form a tensor of embeddings.

symbols representing the syntactic category and node roles are treated as distinct vocabulary tokens, mapped to integers, and used to retrieve a vector representation that is learned during training.


1 with Λ representing the supertags, G representing embedding matrices, and C representing the convolutional neural network layers.

g s is the syntactic category embedding matrix and g r is the node role embedding matrix.


The difficulty of selecting orderings with a language model is that the possible linearizations can grow exponentially.

our implementations result in a large amount of insertion trees.


Although the methodology presented in this work is closely related to several recent neural networks models for long-distance relationships, it differs distinctly in its treatment of state and search.

forward-planning in a generation task produces a growing horizon of syntactic choices while shrinking the horizon of semantic goals.


At least intuitively, it seems implausible that N-grams are a natural model for sentences.

humans seem to use strong predictive skills that rely on non-local aspects of pragmatics (i.e., the conversational context), semantics (i.e., meaningful continuations), or syntax (i.e., syntactically correct continuations).


The decoder is often trained to predict the next word y t given the context vector c and all the previously predicted words {y 1 , ..., y t−1 }.

the decoder defines a probability over the translation y by decomposing the joint probability into the ordered conditionals: where y = {y 1 , ..., y t−1 }.


The universal dependencies (UD) initiative aims to remedy this by presenting a harmonized set of treebanks, now 54 and counting, with a unified annotation scheme and multilayered annotation.

uD data distinguishes the input space-delimited tokens from the (morpho)syntactic words that participate in the parse tree (Nivre et al., 2016).


in "English-like" settings, our tagging accuracy (97.44) is as high as state-of-the-art English tagging (Manning, 2011).


For raw data we see a minor,~0.02, drop in F 1 , compared to gold-segmented settings.

our model still retains the competitive MA&D performance, in a single, universal, trainable modelwe attribute this to our joint segmentation and tagging strategy, which overcomes error propagation.


Consequently, those stop or function words that occur frequently in the paragraph may mislead the embedding learning process to produce a misty paragraph representation.

the frequent words or modifiers may overshadow the indicative words, thereby drifting the main theme of the semantic content in the paragraph.


Accordingly, those stop or function words may mislead the representation learning process to produce an ambiguous paragraph representation.

the frequent words or modifiers may overshadow the indicative words, thereby making the learned representation deviate from the main theme of the semantic content expressed in the paragraph.


In this paper, we leverage a density peaks clustering summarization method (Rodriguez and Laio, 2014;Zhang et al., 2015), which can take both relevance and redundancy information into account at the same time.

a concise summary for a given document set can be automatically generated through a one-pass process instead of an iterative process.


The results corroborate that, instead of only considering literal term matching for determining the similarity degree between a pair of sentence and document, incorporating concept (semantic) matching into the similarity measure leads to better performance.

the proposed D-EV model is the most robust among all the methods compared in the paper, which supports the important notion of the proposed "learning to distilling" framework.


a system in which expressiveness is treated as a complete space instead of as discrete entities) fits perfectly.

in real life expressiveness present overlap between them because there are not clear frontiers, so being able to take into account those overlaps in the shape of a dynamic synthesizer would be ideal.


The proposal of Daumé III (2009) is that we train a model f aug : {0, 1} (K+1)d → Ω that instead uses an augmented feature vector x aug ∈ {0, 1} (K+1)d defined as where x k ∈ {0, 1} d is a feature representation of x under the k-th domain.

we expand the feature space by K + 1 times by duplicating the original feature x for K times: then given an input from the k-th domain, we only use the original x and the corresponding domain representation x k .


Ando and Zhang 2005propose learning a shared structure across multiple related classification tasks.

they consider K binary classification tasks each of which has its own linear classifier f k : Here, u k ∈ R d and v k ∈ R m are task-specific parameters but Θ ∈ R d×m is a global parameter shared by all classifiers f 1 .


Note that the proposed extensions of Daumé III (2009) and Ando and Zhang (2005) only differ in how the domain-independent information is derived.

the extension of Ando and Zhang (2005) can be seen as a weaker version of the extension of Daumé III (2009) since the domain-independent information is used as is (i.e., x t ) in the former while first transformed by a domain specific LSTM in the latter (i.e., h t ).


Our parsing model is based on transition-based dependency parsing paradigm (Nivre, 2008).

we use an arc-eager transition system, which is one of the famous transition-based parsing systems.


Nevertheless, we achieved substantial improvements in accuracies in all the three tasks for both Hindi and Urdu.

improvements in Urdu are more prominent.


Our only assumption is that parallel sentences (source-target) are available and that the source part is annotated.

we try to infer annotations in the target language from sentencebased alignments only.


This connection allows SRNN to use at the current time step hidden layer's states of previous time steps.

the hidden layer of SRNN represents all previous history and not just n − 1 previous inputs, thus the model can theoretically represent long context.


), and the mwe relation is not used at all.

the observed differences in MWE patterns certainly reflect differences between languages, but greater than this, they capture differences in the annotation process between different languages.


Then, a series of iterations is performed to merge the clusters.

at each iteration the algorithm analyzes each node s in random order and assigns it to the majority class among those associated with its neighbors.


Specifically, at each iteration the algorithm analyzes each node s in random order and assigns it to the majority class among those associated with its neighbors.

it assigns each node s to the class c that maximizes the sum of the weights of the edges s i , s j incident on s j such that c is the class of s i (Equation 6).


In the training process, we apply the pairwise ranking strategy (Collobert et al., 2011) to tune model parameters.

we calculate the ROUGE-2 scores (Lin, 2004) of all the sentences in the training dataset.


To verify the effectiveness of the joint model, we design a baseline called ISOLATION, which performs saliency ranking and relevance ranking in isolation.

it directly uses the sum pooling over sentence embeddings to represent the document cluster.


Our task in this paper is thus to explore how to use relevant public posts to improve summarization of a single news article.

we use Facebook public posts related to a news article to help summarize a popular topic.


(Charniak and Johnson, 2005), (McDonald et al., 2005), (Martins et al., 2013) and so on, reliable constituent-based parsers and dependency parsers are available.

other than selecting the set of POS tags that are considered as long-distance projecting categories, there are few implementation details for the first and second stages.


To assess the importance of individual features in their contexts, we run ablation tests in which we remove individual atomic features from composites, as summarized in Tables 4 and 5.

removing the frame feature F from argument labeling (K-SRL (−F ) )), which causes all argument labeling predictions to be made without frame-specific contests, leads to the most significant decrease on F 1 scores (↓2.5 pp and ↓1 pp) in our ablation tests.


We intend to focus more specifically on feature engineering for instance-based SRL.

we plan to explore automatic feature selection methods especially in the context of composite features.


We are the first to use keystroke logs as signal to improve NLP models.

the contributions of this paper are the following: i) we present a novel bi-LSTM model that exploits keystroke logs as auxiliary task for syntactic sequence prediction tasks; ii) we show that our model works well for two tasks, syntactic chunking and CCG supertagging, and iii) we make the code available at: https://github.com/bplank/coling2016ks.


In this way, we automatically gather labels from keystrokes representing pause durations.

we use the following discretization, i.e., a label for a token is calculated by:  Table 1: Example keystroke log for user 33 (including typo).


Goodkind and Rosenberg (2015) is the only study prior to us that use keystroke loggings in NLP.

they investigate the relationship between pre-word pauses and multi-word expressions and found within MWE pauses vary depending on cognitive task.


To that end, we propose a simple, supervised approach to compound segmentation that integrates existing morphological analysis software with language modeling and optional linguistic constraints.

the proposed segmenter seeks to identify word boundaries in closed compounds, compounds in which word boundaries go undelimited by spaces, hyphens, or other markers (e.g., bookworm).


For instance, TransE method (Bordes et al., 2013) regards the relation in a triplet as a translation between the embedding of the two entities.

transE learns a preference of h + r = t for each triple, where h, r, and t are the representation vector of head entity, relation, and tail entity respectively.


In addition, there are still some works (Xiao et al., 2016b;Xiao et al., 2016a) follow the principle h + r ≈ t, although they do not share the same form of score function.

(Xiao et al., 2016b) proposes to use a generative model to deal with multiple semantic meanings of a relation.


The above process keeps running until all facts are included in the graph G. Moreover, we use s = (t, k) to represent a subject (i.e., a vertex or an edge) of the knowledge graph G, where t indicates subject type, and k is the index of the corresponding vertex or edge.

we let t = 0 to denote a vertex and let t = 1 to denote an edge.


In this work, we define π(•) as follows: We then introduce how to construct different types of graph context.

to take advantage of the graph structure, given a subject s i , we consider three types of context: neighbor context C N (s i ), path context C P (s i ), and edge context C E (s i ).


We then re-define the translation of a given graph context, taking the embedding vector of each subject with different weights by further considering attention mechanism.

we have To utilize these three types of context, we combine them by jointly maximizing the objective functions: We define λ T , λ P and λ N to represent the prestige of neighbor context, path context and edge context separately.


In addition, the framework of this model is flexible to handle sundry information except the graph context.

we can also build text context by using descriptions of entities or additional text information from other sources like Wikipedia.


Our serendipitous property mining system leverages automatically harvested knowledge on the web.

measuring unexpectedness requires knowing the expectation, which requires worldly knowledge.


We leverage an automatically harvested knowledge base for human-like unexpected and relevant property mining.

we leverage Probase 'is-a' knowledge and its property data to mine probabilities, which is essentially equivalent to deriving probabilities from a huge corpus.


For example, given category MAMMAL, people would usually think of typical mammals such as dog.

probase has diverse types of typicalities including the entity typicality for a category, and property typicality for an entity.


Note that, instead of the values of properties, their typicalities are leveraged to directly capture the human interest on the properties for entities and the category.

high P (height|giraffe) means that 'height' draws human interest so that it is discussed frequently with giraffe on the web.


Unlike its category-agnostic approach using a Markov chain, we leverage the category information to model the prototype with beta distributions.

we define a prototype of category c hypothetically as an ordered set of random variables R c = {X a,c |a is a property} (an example distribution of X a,c is shown as the blue dashed line in Figure 2(b)).


In particular, we define a prototype of category c hypothetically as an ordered set of random variables R c = {X a,c |a is a property} (an example distribution of X a,c is shown as the blue dashed line in Figure 2(b)).

unlike DM using averages to produce an ordered set of expected typicalities, we build probability distributions.


We exploit a Dirichlet distribution of a single dimension, which is a beta distribution, to model P (a|e) since a Dirichlet distribution is a conjugate prior of a multinomial distribution.

each random variable X a,c in prototype R c is modeled as a beta distribution corresponding to a property of entities in category c: X a,c ∼ Beta(x; α a,c , β a,c ).


Now we have to learn the parameters specifying the beta distributions Beta(x; α a,c , β a,c ) of the prototype R c .

we use a set O a,c of property typicality P (a|e) for each entity e in the identified category c as samples together with its occurrence probability P (e|c), that we obtain from Probase.


In particular, we use a set O a,c of property typicality P (a|e) for each entity e in the identified category c as samples together with its occurrence probability P (e|c), that we obtain from Probase.

we give a larger weight to a more typical entity in the model.


Unlike Case A and B modeling relevant properties, the prototype may include a random variable representing a noisy property.

when a noisy property is modeled, it looks like Figure 2(c).


A noisy property does not show a general tendency, and it is rather randomly distributed as often modeled by a uniform distribution.

since a noisy property is modeled like a uniform-esque distribution as shown in Figure 2(c), the most value of P (a|e) is expected to happen, and would not be considered serendipitous.


We build a labeled dataset by annotating the mined property with the serendipitous score on a scale of 0, 1/3, 2/3 and 3/3.

3/3 point is given if the property is relevant, peculiar, and novel; 2/3 point is given if the property is relevant, and peculiar; 1/3 point is given if the property is relevant; and 0 point is given if the property is irrelevant.


where rel j,k is the relevance (1 or 0) of the rank k question for j ∈ J. Measurements of serendipity have been established in the context of evaluating recommendation systems (Ge et al., 2010;Shani and Gunawardana, 2011).

(Ge et al., 2010) evaluates serendipity as the ratio of unexpected but relevant results based on a benchmark model that generates expected recommendations, which are relevance-pursuing results.


Specifically, (Ge et al., 2010) evaluates serendipity as the ratio of unexpected but relevant results based on a benchmark model that generates expected recommendations, which are relevance-pursuing results.

from the result questions of each method, we remove the expected ones that are retrieved by the benchmark model.


Therefore, we use several criteria such as 'more relevant questions rank higher' based on (Bordino et al., 2013) and evaluate individual dimensions.

we generate a task with a photograph and two randomly chosen result trivia quizzes out of those returned by all tested methods.


We can see that our method outperforms the baseline.

our approach retains a high positive correlation with the reference rankings based on user perception.


Noting there could be cases where the number of blog posts published in a specific year is higher compared to that in other years due to certain events (e.g., an Olympiad, or a major weather related event), we draw samples for our dataset from several different time periods.

for each culture, we consider an equal number of instances from four different years (2011-2014).


We model the different usages of the words in our set of 100 words by using topic modeling.

we use Latent Dirichlet Allocation (LDA) 5 (Blei et al., 2003) to find a set of topics for each word, and consequently identify the topics specific to either Australia or United States.


To determine the number of topics that best describe the corpus for each of the 100 words, we use the average corpus likelihood over ten runs (Heinrich, 2005).

we choose that number of topics (>= 2, <= 10) for which the corpus likelihood is maximum.


Subsequently, we measure the overall Spearman correlation coefficient between the dominating topic distributions for all 100 words.

the distribution of topics is compared across cultures for each word.


For additional insight into word usage differences between United States and Australia, we expand our study to develop word models to separate word usages in Australia (or United States) from a mix of ten different cultures.

we conduct a one-versus-all classification using the same process as described in Section 3, but using Australia (United States) against a mix of other cultures to examine any features specific to Australia (United States).


From this, we can infer that that the performance improvement over the baseline for the Australia versus United States task can be majorly attributed to the different word usages in Australia from the remaining countries.

united States is more aligned with the "typical" (as measured over ten different countries) usage of these words than Australia is.


Our work is most closely related to that by Paul and Girju (2009), in which they identify cultural differences in people's experiences in various countries from the perspective of tourists and locals.

they analyzed forums and blogs written by tourists and locals about their experiences in three countries, namely Singapore, India, and United Kingdom, using an extension of LDA.


In this paper, we explored the problem of identifying word usage differences between people belonging to different cultures.

we studied differences between Australia and United States based on the words they used frequently in their online writings.


Total Time (TOTAL) is the total duration that the gaze spends within the area of interest.


They claim the evaluation methods used in studies like (Redington et al., 1998;Monaghan and Christiansen, 2008;Mintz, 2003) could be misleading.

if a word is substituted with another one in its category, the resulting sentences could be erroneous in a way that they are not observed in infants' speech.


Near the sentence boundaries the appropriate terms were truncated in Equation 3.

at the beginning of the sentence shorter n-gram contexts were used and at the end of the sentence terms beyond the end-of-sentence utterance were dropped.


Can we also take the classification label into account when clustering the variables by the frequency proportions of their variants?

how can we group those lemmas together that for a given L1 have similar proportions of subcat variant realizations?


In this section we sketch, how the method can be used to advance the qualitative analysis in the context of the SLA research.

we investigate the hypothesis by Wang (2009), suggesting that learners with L1 Chinese overuse pronoun-subjects over noun-subjects in Chinese-English translations.


To model communication in this sense, in our research we employ features based on the information-theoretic notion of surprisal or information density.

we make use of information theory inspired features on the linguistic levels of lexis, part-of-speech and syntax.


Polysemic words owning the same appearance may mean that they share a common intrinsic quality, and the differences among them are reflected by different contexts.

the different contexts of a word exert different influences, and the given context is a good indication of the direction of the word meaning.


Finally, a last possible scansion would be to have assume a trochaic and a pyrrhic foot followed by a double iamb (Shall we not lie as we have lain).

there exists a set of possible analyses for this line which may all be accepted as correct, or at least reasonable.


• mean of wi+1 − recording min.f0 energy • per f0 This feature-set serves to capture the observed association between prosodic discontinuity and SU boundaries.

speakers tend to pause and lengthen SU-final tokens -hence the pause and phone duration features -and the pitch 'reset' between tokens either side of a boundary is likely to be more pronounced than elsewhere: hence the focus on differences with the following token and the SU minimum (Shriberg et al., 2000).


Our alternative measures -parse likelihood and perplexity -indicate that the hypothesised SUs score similarly across the configurations.

parse scores are slightly down on the gold-standard (these are negative logs, so closer to zero is more probable), whilst perplexity scores are more noticeably down  (the lower the score, the better the LM models the input) compared to gold, and the standard deviations indicate more variance within the output SU perplexities.


It is interesting that for both algorithms, using all features on the original documents offers highest Precision@5 and MAP scores, confirming the results shown in Figure 1.

with this features/dataset combination, with the glm library we see slight rises in Precision@5, F1-Score@15 and MAP from 24% to 24.4%, from 19.22% to 20.30% and from 0.127 o 0.136 respectively; for C5.0, while F1-Score rises from 18.95% to just 19.29%, Precision@5 and MAP shows a more significant improvement from 22.2% to 26.8% and from 0.123 to 0.137 respectively, thus supposedly showing a better ranking of the keyphrases found.


Linguistic theories based on the notion of constructions are a recent development in linguistics, which has revitalized the discussion in fields such as syntax, lexicography, and argument structure theory.

it is useful for describing partially schematic constructions: templatic patterns that exhibit lexical as well as syntactic properties, which are too specific to be referred to general grammar rules, but too general to be attributed to specific lexical units, which is a reason why they historically have been overlooked (Fillmore et al., 1988).


While we are aware of no previous work approaching the general problem of searching for construction occurrences from a retrieval perspective, quantitative and corpus-based methods have been a crucial part of the construction-linguistic toolbox from its early days Stefanowitsch and Gries, 2003;Hilpert, 2013).

a number of automatic methods have been presented that mine corpora for frequent patterns.


rodna av ilska 'blush with anger') to be a hard nut to crack because of its great productivity and high lexical variation, but the distributional similarity performs beyond expectations.

this model succeeds in generalizing beyond seen instances and gives high ranking scores to a wide array of new and creative occurrences of the V av NP construction.


The lexical similarities give a smaller improvement for the QUANTIFYING GENITIVE:SCALE than for the related time construction.

while the frame-based feature worked best for the time construction, it receives the lowest average precision in this case.


We propose a new method of automatically extracting learner errors from parallel English as a Second Language (ESL) sentences in an effort to regularise annotation formats and reduce inconsistencies.

given an original and corrected sentence, our method first uses a linguistically enhanced alignment algorithm to determine the most likely mappings between tokens, and secondly employs a rule-based function to decide which alignments should be merged.


The first attempt was made by Swanson and Yamangil (2012), who built a system to align sentences and then classify the nonmatch tokens type for the purposes of ESL feedback.

they used the well-known Levenshtein distance to align the sentences and then classified any non-matches according to the FCE error scheme (Nicholls, 2003) using a maximum entropy classifier.


The mistakes arguably represent not only the shortcomings of this specific system, but also the characteristics of the Japanese loanword phonology as a whole.

the Japanese SR forms which are mistakenly generated by the loanword adaptation grammar can be categorized into 3 types.


This also means that there is clear limit to the predictability of the phonological system that drives the accommodation of loanwords in Japanese.

after a certain point-corresponding to roughly 90% per type-there is a residue of internal inconsistency which is not capturable by any satisfactory phonological generalization.


• Examples: pocket /p6kIt/ → [pok.ket.to] because all the segments in the source form are transformed into corresponding segments in the adapted form.

the number of segments in the adapted form is no smaller than that in the source form.


Secondly, we will explore the various student-assigned numerical ratings that accompany the text comments analyzed here.

we hope to compare these ratings with our automatically generated satisfaction ratings in order to see the degree to which positive comments correlate strongly with high numerical ratings.


We investigate different neural network (NN) architectures for ADR classification.

we propose two new neural network models, Convolutional Recurrent Neural Network (CRNN) by concatenating convolutional neural networks with recurrent neural networks, and Convolutional Neural Network with Attention (CNNA) by adding attention weights into convolutional neural networks.


This paper has explored different neural network (NN) architectures for ADR classification.

it has proposed two new neural network models, Convolutional Recurrent Neural Network (CRNN) and Convolutional Neural Network with Attention (CNNA).


where f is an activation function such as a sigmoid function or a tanh function, and W and U are parameters to be learned.


However, only one correct preposition is labeled in the HSK corpus.

our models are under-estimated due to the one-answer evaluation.


The information of POS tag generally improves the performance for the cases of longer sentences.

the GRU512 model on word-level with POS tag outperforms other models for the sentences of lengths ≥ 4 words.


• PrStSu2a+SM: in contrast with PrStSu, where the prefix or suffix can simply be empty (but always exists in the derivation), we now allow the derivation to not have a prefix and/or a suffix.

a word is modeled as a stem-suffix sequence or a prefix and stem-suffix sequence, where the stem-suffix is a stem or a stem and a suffix.


The syntax of early modern Swedish is strongly influenced by German.

it is very common for subordinate clauses to have verb-final word order, as in both clauses of the second example above.


An interesting fact about early modern Swedish is that many of its historical features, despite having disappeared completely from the modern form of the language, are still attested in other, closely related contemporary languages.

morphological features similar to those of early forms of Swedish can be found in present-day Icelandic, and modern German still exhibits some of the syntactic patterns that were common in early modern Swedish.


While previous work often assumes keyphrases are a static property of a given documents, in many applications, the appropriate set of keyphrases that should be extracted depends on the set of documents that are being considered together.

good keyphrases should not only accurately describe the content of a document, but also reveal what discriminates it from the other documents.


In this paper, we study this problem of extracting discriminative keyphrases.

ly, we propose to use the hierarchical semantic structure between candidate keyphrases to promote keyphrases that have the right level of specificity to clearly distinguish the target document from others.


Once the DBM model has been trained, we use it to predict the hierarchical relationship between pairs of candidate keyphrases.

given a candidate keyphrase, we form pairs with all the other keyphrases from the considered group of documents.


The difference in behaviour between hScore and nChildren is more pronounced on the bottom and middle level keyphrases.

the nChildren clearly outperforms the hScore for positive α, but the resulting performance is still lower than what hScore achieves on the top level keyphrases.


Then, we embed each word w in t p in a continuous space and sum the embedding vectors to obtain an embedding representation.

the embedding matrix A(of size dim × |V |, where V is the vocabulary and dim is the embedding dimension) is used to look up the vectors for words w, and the representation can be calculated as follows: The embedding o 0 is also treated as an initial input of the user interest encoder.


We focus on content word ED as one of the most challenging tasks in this area, illustrating our claims on adjective-noun (AN) combinations.

we (1) investigate the role of context in accurately capturing semantic anomalies and implement a system based on distributional topic coherence, which achieves state-of-the-art accuracy on a standard test set; (2) thoroughly investigate our system's performance across individual adjective classes, concluding that a classdependent approach is beneficial to the task; (3) discuss the data size bottleneck in this area, and highlight the challenges of automatic error generation for content words.


We follow the semantically motivated approach outlined by Kochmar and Briscoe (2014) (henceforth K&B) for adjective-noun (AN) combinations, 1 building on their work by integrating context information in the classification.

we want to learn that although classical dance is more frequent than classic dance, the latter is correct in a context such as They performed a classic Scottish dance.


Approaches based on this idea have a number of weaknesses.

they rely on the availability of a set of plausible alternatives and are unable to detect errors in the absence of such alternatives, even though a number of studies (Leacock et al., 2009;Andersen et al., 2013) have shown that ED alone (without correction) is useful for language learners.


We implement an oracle system via a voting step based on the COMPDIST and COH adjective-specific performance.

for each test item, we use the prediction of the system which has best accuracy for the particular adjective under consideration.


Faced with this mixed findings in the literature regarding the efficiency of cohesive features for the assessment of text readability, our goal is to further investigate this issue.

we present experiments focusing on cohesive features : anaphora chains, reference chains and lexical chains (evaluating sentence similarity).


An important feature of our work is the use of human linguists who do not speak the language but are familiar with the structure of human languages in general, have some knowledge about the language family in question, and can absorb facts about the language quickly by reading reference grammars and looking at data.

in this project, linguists worked interactively with unsupervised morphology induction, annotated named entities, and identified thresholds for automatic tagging of multi-word named entities.


The second largest category of errors is composed of expressions that can not be considered as entities, e.g.


table 5) are similar to that of the results reported in table 4.

the CharCNN system performs the best for Mayan language family, while the PhoneticCNN system performs the best for the Indo-European language family.


More specifically, we focus on coherence in summary evaluation and statistically prove that this feature, which is very important for human understanding of text, has not only been neglected in the creation, but also in the evaluation process.

we focus on coherence in our evaluation, which has previously been neglected in the creation process.


We first model the nugget identification step as a coding task (i.e., assigning categories to predefined units) by aggregating each annotator's nuggets to the levels of sentences and paragraphs.

a sentence or paragraph is considered important, if it contains at least one nugget.


We automatically generate the training corpus from the collected poems.

given a poem consisting of N lines, we first rank the words in each line according to the TextRank scores computed on the poem corpus.


if fragments A and B contain coreferent entity mentions and fragment C originates from the same sentence as fragment B, then fragments A and C should not be clustered together without fragment B); therefore, the second gold-standard only has an impact on purity, and not collocation.

when computing purity, a gold-standard cluster is the union of the fragments originating from a corpus sentence and all fragments that contain coreferent entity mentions with said sentence.


In the last couple of years, Neural Network (NN) based approaches have gained enormous popularity within statistical NLP generally, with applications to Machine Translation , Conversation Modelling (Vinyals and Le, 2015) and Parsing (Tai et al., 2015), to cite only a few.

architectures based on Recurrent Neural Network (RNN) such as LSTMs (Hochreiter and Schmidhuber, 1997) and GRUs  have been succesfully used in Language Modelling tasks due to their ability to model sequential information with long-range dependencies.


RTE is the task of recognizing an inference relation between two sentences expressing the fact that the meaning of one sentence is entailed by the other (Androutsopoulos and Malakasiotis, 2010;Dagan et al., 2013).

entailment-based minimum vertex cover method (Gupta et al., 2014) is an RTe method for single document summarization using graph-based algorithms.


The main idea of our method is to leverage phrase properties to choose a subset of optimal phrases for generating the final summary.

we exploit two phrase properties, popularity and specificity.


The last column indicates the agreement rate of preference comparisons for different systems.

in our experiments, we treat one pairwise comparison as in agreement if four (out of five) annotators give the same preference decision.


A problem with this approach is that the majority of questions generated from single sentences tend to be too specific and low-level to properly measure learners' understandings of the overall contents of text.

what is assessed by such question generation systems ends up essentially being the ability to compare sentences, just requiring learners to find a single sentence that has almost the same surface form as a given interrogative sentence.


Our experiments showed that questions generated by both methods require more inference steps than questions generated by a traditional single-sentence approach.

the first method outperforms the baseline system in terms of the number of inference steps by 0.60 on average, while ensuring comparable grammatical correctness and answer existence.


In future work, we will build a monolingual parallel corpus from a raw corpus by combining our sentence similarity measure with a readability assessment.

we will divide the raw corpus into complex and simple corpora based on readability, and use the paired corpora to align complex sentences with simple ones.


Learning vector representations of words using neural networks has generated a strong enthusiasm in the NLP research community.

many contributions were proposed after the work of (Mikolov et al., 2013a;Mikolov et al., 2013b;Mikolov et al., 2013c) on training word embeddings.


We augment METEOR in two ways: firstly, we replace the use of lexical resources by the use of word embeddings.

we replace Stem and Synonym modules by our new Vector module.


METEOR DBnary improved the score by 0.7 points from METEOR Baseline.

dBnary seems to match more synonyms than WordNet, despite the fact that WordNet is 3.3 time bigger than dBnary in English.


Businesses, governments, and communities increasingly need real-time information from dynamic, large-volume media data streams, such as blogs, Facebook, and Twitter.

the automatic detection of mentions of people, organizations, locations, and other entities (i.e.


The spatial dimension aims to account for linguistic variation in social media across English-speaking countries.

the discussed entities vary from one region to the next, e.g Justin Trudeau in Canada vs. Theresa May -in the UK.


One of the major and critical reasons of the rejection is the existence of prior art that invalidates the submitted patent.

if this type of rejection is overcome by modifications of the claims, this modified part must contain the keywords of novelties.


(2011), who use cross-lingual information to refine the results of event extraction.

they run several monolingual event extraction systems independently, translate the extracted argument fillers into English, and merge together argument fillers across documents.


From Table 1a, we find that both the combinations of Bi-LSTM with Bi-RNN and Bi-LSTM outperform the combination of Bi-LSTM and MLP without any features.

the combination of Bi-LSTM and Bi-LSTM achieves the best result 83.90% without any feature, and its F1-score is about 1.5% higher than the model of Bi-LSTM+MLP.


Lexical properties of a word (e.g., the identity of a word, its lemma, its morphological features) are important for semantic tasks.

in tasks like relation classification, it is often impossible to determine the relation without the semantic ground of the target words.


Models are trained in a cascaded manner.

a RC model is trained first, and then the parameters (e.g., word embeddings, network weights) are used to initialize the neural network for training SRL in the second stage.


Effect of transfer learning By comparing the cascaded training system (CAS), the multi-task learning system (MTL) with SUP, we can find that the task of RC is significantly helpful for improving SRL models.


Let U be the set of binary indicator variables corresponding to the decision whether w i is a non-argument.

u i = 1 if w i is not an argument and u i = 0 otherwise.


Let V be the set of binary indicator variables corresponding to the decision whether w i is a certain semantic role.

v ij = 1 if w i is assigned label L j and v ij = 0 otherwise.


Note that when we talk about 'predictability' of a derived form, we refer to the ability to model an otherwise already established term, for which a distributional analysis can be performed.

we investigate to which extent a one-off compositional procedure can capture the meaning of a word, as in a situation where a speaker encounters an existing term for the first time.


The work then presents a generative multilingual parsing model in which dependent ordering parameters are conditioned on word order typology, obtained from WALS.

the paper utilises the following word order features (henceforth WALS Basic word Order, WBO): 81A (Subject Verb and Object), 85A (Adposition and Noun), 86A (Genitive and Noun), 87A (Adjective and Noun), 88A (Demonstrative and Noun) and 89A (Numeral and Noun).


GRUs have a linear shortcut through timesteps which bypasses the nonlinearity and thus promotes gradient flow.

a GRU computes the hidden state at current time step, h t , as the linear combination of previous activation h t−1 , and a new candidate activationh t : where is elementwise multiplication, and the update gate activation z t determines the amount of new information mixed in the current state: The candidate activation is computed as: The reset gate r t determines how much of the current input x t is mixed in the previous state h t−1 to form the candidate activation: By applying the gru function repeatedly a GRU layer maps a sequence of inputs to a sequence of states: where X stands for the matrix composed of input column vectors x 1 , .


In this paper, we propose to extract Chinese is-a relations from user generated categories.

the task can be divided into two steps: initial model training and iterative learning.


In this paper, we explore the use of embeddings obtained from different levels of lexical and morpho-syntactic linguistic analysis and show that they improve MT evaluation into an MRL.

we report on Arabic, a language with complex and rich morphology.


In this paper, we investigate how embeddings obtained from different levels of lexical and morphosyntactic linguistic analysis can improve MT evaluation into a MRL.

we report on Arabic, a language with complex and rich morphology paired with a high degree of ambiguity (Habash, 2010).


Evaluating a single translation Most of the MT evaluation metrics are not designed to do pairwise, but absolute evaluation.

we are interested in generating a score for a single translation t 1 given a reference r. To achieve this with our pairwise network, we compute the goodness margin, which tells us how good translation t 1 is better than any other translation t ∅ given the reference r. In this case, t ∅ is the average representation for all translations observed during training.


When multiple preordering candidates are available, we merge them into a single graph structure.

we integrate nodes in two preordering lattices if their nodes have same coverage vectors each other as shown in Figure 1(  end for 10: 14:  Figure 1(c) shows an example of merging 5 preordering candidates in A as a single preordering lattice.


However, to ensure high quality of learned phrases, strict syntactic constraints were enforced at both the phrasal and sentential level.

they only considered event expressions and purpose phrases as verb phrases in two types (Figure 2(a)), the first phrasal type is a verb followed by the head noun of its direct object and the second phrasal type is a verb with an attached prepositional phrase, while reasonably both event expressions and purpose phrases can exist in many other syntactic structures.


In order to alleviate error propagation from word alignments and dependency trees, we apply phrase filtering.

we adopt three strategies to filter out inappropriate phrases.


On the other hand, previous studies on temporal relation extraction focus on mining sentence-level information or enforcing coherence on different temporal relation types among various mentions in the same sentence or neighboring sentences, largely ignoring those discourse-level temporal relations in nonadjacent sentences.

only a few studies apply global inference models to exploit temporal relations in discourse level.


Recently,  enriched the TimeBank-Dense corpus, on the top of TimeBank.

they approximated the completeness by labeling locally complete graphs over neighboring sentences.


To address various problems in existing corpora, as described above, we build an event-driven fullyannotated Chinese temporal relation corpus, on the top of the ACE 2005 Chinese corpus with 8 predefined event types and 33 predefined event subtypes (e.g., Die, Attack, and Transport).

all other event mentions of non-predefined event types are ignored in our corpus.


Our work is a typical practice of such tendency.

163 documents from the ACE 2005 Chinese corpus are selected as our experimental data, which contains 1166 event mentions.


Following Roth and Yih (2004) and Li et al.

(2013) in information extraction, we define the following log costs: ILP (Integer Logical Programming), a global inference is employed to achieve global optimization with the following objective function to maximize over a document as follows: while binary constraint (5) ensures that x<i,j,r> is binary value and equality constraint (6) ensures that exactly only one temporal relation can be assigned to each event mention pair.


In addition, the reflexivity and transitivity constraints, as deployed in previous inference models (Bramsen et al., 2006;Chambers and Jurafsky, 2008;Do et al., 2012), are also applied to our model as follows: Here, reflexivity constraint 7  Different from the TimeBank corpus which only annotates the temporal relations in the same sentence or neighboring sentences, our corpus is event-driven fully-annotated.

besides the temporal relations in the same sentence or neighboring sentences, our corpus also contains those in nonadjacent sentences, which occupy 56.3%.


For example, if there is a causal relation between an Attack and a Die event mention, it is obvious to infer they have the Before temporal relation.

a Die event is always the result of an Attack event.


That is, a Die event is always the result of an Attack event.

we learn event relevance from the training set by counting the occurrence frequency f<i,j,r> for each event type pair (evti, evtj) (e.g., (Attack, Die)) with relation r in the training set.


Given a sentence and its dependency parse tree, 1 we follow our previous work (Xu et al., 2015b) and build DRNNs on the shortest dependency path (SDP), which serves as a backbone.

an RNN picks up information along each sub-path, separated by the common ancestor of marked entities.


In recent years, deep neural network has proven its ability to learn task-specific representation automatically, so that avoiding error propagation suffered by traditional feature-based models.

many neural network approaches have been proposed and shown better performance in relation classification (Zeng et al., 2014;Xu et al., 2015) and relation extraction (Nguyen and Grishman, 2015).


In this paper, we solve this problem by relaxing expressed-at-least-once assumption as: Assumption: A relation holding between two entities can be either expressed explicitly or inferred implicitly from all sentences that mention these two entities.

we relax the expressed-at-least-once assumption by not only allowing making predictions from evidences in each single sentence, but also allowing making predictions by inferring from evidences in all sentences collectively.


Other approaches, such as mean-pooling, can also be applied in this phrase, but we use cross-sentence max-pooling for the following reason: We consider that multiple occurrences of a feature do not supply much extra information in entity-pair-level relation extraction.

a discriminative signal that appears only once can also be sufficient for extracting a relation.


Dropout (Srivastava et al., 2014) is also employed on formula (4) for regularization.

at training time, each element in g is randomly dropped out by multiplying a Bernoulli random variable with probability p of being 0.


For convenience of comparing with baseline methods, our model uses the same parameter settings as (Zeng et al., 2015).

at model input layer, we use a minibatch of 50 entity pairs, set the dimension of word embedding d w = 50 and the dimension of position feature d p = 5.


In terms of goals, while (Cheng et al., 2015) aim to produce entity summaries, our goal is to produce a large set of content units that are varied both in terms of content and in terms of structure.

one important difference is that we produce trees of varying shapes and depths while the graphs produced by (Cheng et al., 2015) are restricted to trees of depth one i.e., set of DBpedia triples whose subject is the entity to be described.


Finally, there has been much work in recent years on using ILP for natural language processing.

(Kuznetsova et al., 2012) proposes an ILP formulation for the generation of natural image descriptions from visual and text data and (Filippova and Strube, 2008) uses ILP to model sentence compression.


Astronaut), for each triple t = (s, p, o) in G e , we introduce a binary variable x p s,o such that: 1 if the triple is preserved 0 otherwise Because we use bigrams to capture local and discourse coherence (properties that often co-occur together), we also have variables y t 1 ,t 2 for bigrams of triples such that: 1 if the pair of triples is preserved 0 otherwise For the S-Model, these binary variables capture pairs of triples which share the same subject.

for each bigram of triples t 1 = (s1, p1, o1) and t2 = (s2, p2, o2) in G e such that s1 = s2, we introduce a binary variable y t 1 ,t 2 .


Similarly, for the C-Model, we introduce a binary variable y t 1 ,t 2 for each pair of triples such that the object of one is the subject of the other.


In sum, by modifying the ILP parameters to select various numbers of triples, we can generate solutions of different sizes whilst the 3 ILP models permit producing solutions with relatively small overlap both within and across models.

our content selection method can be used to automatically create a graduated benchmark for natural language generation in which the inputs are of increasing size and exhibit a good level of semantic variability.


Because they are retrieved from DBpedia, the data units selected by our approach are semantically coherent overall.

the triples that are directly connected to the entity being described are all relevant.


It investigates general problem description languages, and general problem solving algorithms, where a "problem" comes in the form of an "initial state", a "goal", and a set of "actions" that can be applied to change states and thus, eventually, reach the goal.

aI Planning is concerned with models of, and algorithms for, goal reachability testing in large statetransition systems.


Here, the derived tree T for an edge e is a tree combining edges to get from elements of s Ω I to e; and a state s Ω contains T if all edges in T are elements of s Ω .

e 0 is feasible if it forms part of a derived tree for a complete sentence.


Action a is applicable in state s if pre a ⊆ s; in that case, the outcome state is defined as s := (s∪add a ∪ e∈CEff a :cona⊆s add e )\(del a ∪ e∈CEff a :cona⊆s del e ).

s results from s by including the add lists of the action plus those effects whose condition holds in s, and afterwards removes the delete lists of the action and those effects.


Two category spaces are defined as follows: but replacing any category c where #(c ) > k by * .

(ii) The optimistic category space of Ω given k is the pair (C Ω [k], γ +Ω ) where γ +Ω is defined like γ Ω but replacing any category c where #(c ) > k by * ; and including c ∈ γ Ω ( * ) whenever γ Ω (c) = c ; and including c ∈ γ Ω (c 1 , * ) whenever γ Ω (c 1 , c 2 ) = c ; and including c ∈ we cut off the generation of categories once their degree exceeds a user-defined threshold k. In the pessimistic (under-approximating) variant, no further combinations are possible behind * .


According to the Wikipedia featured article criteria, 3 these articles are (i) well-written, (ii) comprehensive, (iii) well-researched, (iv) neutral, and (v) stable.

the lead of a featured article is supposed to summarize the topic (according to the guidelines).


This is different from previous corpora consisting of documents from a single text type (e.g., newswire), where each individual source document already provided most of the relevant information.

this property reduces the redundancy in the source documents which is often used by multi-document summarization systems to detect important information.


In the following, we summarize and analyze the result of our corpus construction effort.

we analyze whether our corpus actually is as heterogeneous as we designed it to be via our corpus construction methodology.


This work extends both aspect-based sentiment analysis that assumes a single entity per document and targeted sentiment analysis that assumes a single sentiment towards a target entity.

we identify the sentiment towards each aspect of one or more entities.


In this work, we investigate how to evaluate the development of ideas and exemplification.

we explore whether development is reflected in the structure of graphs constructed from the discourse proximity of essay concepts.


Other than this filtering, we use all the words in the essay, as is, to construct nodes.

we do not use lemmatization or synonymy to collapse similar words.


Finally, the size of the graph can influence PageRank values.

given two nodes with the same type of linking structure, the node in a larger graph (i.e., a graph with more nodes) will receive a smaller PageRank value.


We investigate how our graph-based features perform in comparison to the eRaterFeat feature set and in combination with it.

for holistic scoring, we hypothesize that graph features will improve over eRaterFeat performance by improving construct coverage.


We performed experiments to show that our approach complements previously established features in essay scoring.

we demonstrated that our feature set significantly improves on the state of the art for holistic essay scoring.


For example, in a window of 5, the first word next to the cue word is weighted by 5 5 , while the fourth word away is of weight 2 5 .

as the distance between two words increases, their weighted association score decreases.


This pre-trained model is then used to extract features from the sarcastic datasets.

we use Semeval 2014 (Rosenthal et al., 2014) Twitter Sentiment Analysis Dataset for the training.


As emotion taxonomy, the authors used six basic emotions, i.e., Anger, Disgust, Surprise, Sadness, Joy and Fear.

the blog posts were split into sentences and each sentence was labeled.


This network is replicated five times, one for each personality trait.

we create a CNN for each personality trait and the aim of each CNN is to classify a sentence into binary classes, i.e., whether it expresses a personality trait or not.


It is counterintuitive, since experimental results prove that both of those features learn almost the same global and contextual features.

baseline network dominates over pre-trained network as the former learns most of the features learned by the latter.


In addition, a bilingual attention network is used to capture informative words from both monolingual and bilingual context.

we first construct a document representation through a neural network model.


We use an attention function to aggregate the representation of the salient words to form the bilingual attention vector v (bi) .

in the above equations, we first feed the bilingual word annotations h (Eq.


In this paper, we classify posts taking into account post authorship, likes, topics, and comments.

users and their "likes" hold strong potential for text mining.


Raw agreement between annotators is 0.91, indicating high agreement.

cohen's Kappa for Neu and not Neu labeling is 0.58 (moderate), and for Sup or Uns labeling is 0.84 (almost perfect).


Colours have been extensively studied from this point of view.

previous research in artificial language evolution has shown how artificial languages may emerge based on specific strategies to distinguish colours.


This gap occurs because in some interactions agents prefer distinct discriminative conceptualisations for certain topics.

different prototypes are triggered as more accurate conceptualisations of the topic in a particular context and therefore agents select different terms to describe the same colour sample.


As a simple example, the content of the file named banana{banana.n.01}.txt (for word: banana, synset: banana.n.01) is given below, showing this synset banana.n.01 (food sense) is linked to the synsets apple.n.01 as well as orange.n.01 and orange.n.02, whereas it is co-linked with the synset banana.n.02 (plant sense).

apple {apple.n.01}++ orange {orange.n.01}++ oranges {orange.n.01}++ banana {banana.n.02}+= the symbol '++' denotes an outbound link, whereas '+=' indicates a bidirectional link.


Although it is not clear whether a similar approach would be effective for potentially complex semantic relationships such as evocation, we, in the present work, incorporated the offset semantic vectors relV ec(s, t) (referred to as semantic relational vectors in this research) as a vectorial feature.

we concatenated the obtained 300-dimensional relational vector with the vector obtained from other features described so far.


As already mentioned in this paper, the central notion behind the asymmetric similarity/relatedness is feature inclusion.

(Kotlerman et al., 2010) and (Gawron, 2014) make use of dependency parses acquired from textual corpora as features.


Second, we should incorporate more relational features.

asymmetric similarity features that can be acquired from corpora (Kotlerman et al., 2010;Gawron, 2014) would be beneficial, as some of the evocation relations are rather direct asymmetric relations, such as hypernymy and meronymy.


This paper puts the analysis of the output into the focus.

we investigate to what extent the mined structure can be leveraged to assess the argumentation quality of persuasive essays.


We ask if, to what extent, and how the output of argument mining can be leveraged to assess the argumentation quality of a text.

we consider these questions for persuasive student essays.


Unlike all mentioned approaches, however, we analyze the output of argument mining.

we use the mined structure to assess argumentation quality.


Finally, we analyze the benefit of mining argumentative structure for assessing argumentation quality.

we evaluate the three presented feature types in argumentation-related essay scoring.


(2010), the obtained results thereby reveal that organization is not only about the ordering of discourse functions, but also about argumentative structure.

the novel features that we proposed capture only such structural aspects.


In this paper, we apply Long Short-Term Memory networks with an attention mechanism, which can select important parts of text for the task of similar question retrieval from community Question Answering (cQA) forums.

we use the attention weights for both selecting entire sentences and their subparts, i.e., word/chunk, from shallow syntactic trees.


If the above steps are carried out with good accuracy, complex semantic models can be applied to the selected text to achieve better performance.

such models have to encode structural relations between the constituents of the two questions, e.g., places are there for tourists and some good places to visit, in the learning algorithm.


Second, TS significantly impacts the performance of TKs by feeding them with better representations.

when using supervised methods, i.e., attention model weights, in sentence selection and tree pruning strategies both provide better representations, and higher results than the models using all sentences or those selected by TF×IDF approaches.


(iii) As soon as the number of sentences increases, the supervised models, i.e., sim(att) and attention, perform better than the unsupervised approaches, i.e., sim(nw) and sim(tf-idf).

sim(att) outperforms the baseline, e.g., 74.30 vs. 73.58 (although only when using 2 or 3 sentences).


We demonstrate that applying dropout to arbitrary vectors in LSTM cells may lead to loss of memory thus hindering the ability of the network to encode long-term information.

our technique allows for adding a strong regularizer on the model weights responsible for learning short and long-term dependencies without affecting the ability to capture long-term relationships, which are especially important to model when dealing with natural language.


As RNNs model sequential data by a fully-connected layer, dropout can be applied by simply dropping the previous hidden state of a network.

we modify Equation 1 in the following way: where d is the dropout function defined as follows: where p is the dropout rate and mask is a vector, sampled from the Bernoulli distribution with success probability 1 − p. Long Short-Term Memory networks (Hochreiter and Schmidhuber, 1997) have introduced the concept of gated inputs in RNNs, which effectively allow the network to preserve its memory over a larger number of time steps during both forward and backward passes, thus alleviating the problem of vanishing gradients (Bengio et al., 1994).


Note that on this task regularizing only the recurrent connections is more beneficial than only the forward ones.

lSTM networks trained with our approach and the approach of (Gal, 2015) yield a lower bit-per-character (bpc) score than those trained with forward dropout onlyWe attribute it to pronounced long term dependencies.


To alleviate the negative effect of wrong parsing results, we can filter the hypotheses only when there is more than one candidate.

the syntactic structure can play the role as soft constraints if we adopt relatively tolerant syntactic tree matching rules.


Through comparing the IPMT process on specific sentences, we find that the decoding of many sentences will fail in the baseline system.

there is no hypothesis that can match the prefix in the stack.


Intuitively, by allowing the topic information of the source input sentence and previous translated words to be provided to the decoder, we can maintain the same topic in the translations during the decoding phase, and consequently better translations can be produced.

when a source word has more than one translation option available (e.g.


Here, we propose to examine the impact of these problems by also using other evaluation protocols, based on other types of reference datasets, and by third-party tasks.

following the work of Claveau and Kijak (2016), we rely on information retrieval (IR) as a realistic evaluation use case.


As a side results, all the experiments presented here confirm the interest of using the IR approaches for building distributional thesaurus (Claveau et al., 2014).

the adjusted Okapi-BM25 model offers significant gains of performance over word2vec for most of the evaluation scenarii considered.


Our approach is based on the GloVe model for word embedding (Pennington et al., 2014).

we also associate two vectors with each word i: the vector w i , which intuitively represents the meaning of word i, and the vectorw j , which intuitively represents how the occurrence of i in the context of another word j affects the meaning of that word.


Rather than using w i for modelling the meaning of word i, we then consider a density which is defined by the residual error model.

the residual error model allows us to naturally associate a univariate density with each context vectorw j , given a target word i.


It also leads to a clearer geometric interpretation.

let us write p ij for the orthogonal projection of w i on the line L j = {p | p = λ •w j , λ ∈ R} (see Figure 1), then w i •w j = w j • p ij .


In addition to smoothing the frequency counts, this will give us a way to estimate variance.

we assume that for each target word i there is a multinomial distribution from which all words that appear in the context of i are drawn.


A standard approach is to assume that the parameters of that multinomial distribution are drawn from a Dirichlet distribution.

let x ij be the number of times word j appears in the context of word i in the considered corpus, as before, and let x i = l x il .


From the point of view of the proposed geometric interpretation, this means that valuable information is ignored.

if we want p ij to reflect how strongly context word j is related to word i, we should require that it is small when x ij = 0.


We adopted a straightforward text preprocessing strategy.

following (Pennington et al., 2014), we have removed punctuations, lower-cased the tokens, removed HTML/XML tags, and conducted sentence segmentation.


Evaluation tasks We have evaluated the models on traditional word analogy and word similarity tasks (Levy et al., 2015).

we have used an existing Google Word analogy dataset which we obtained from the GloVe project 12 .


In Table 4 we present a more detailed analysis of the similarity test sets for which our model performs worse than SG.

the table shows the results of a modified test set that only considers the 30% most frequent terms and a modified test set that only considers the 30% least frequent terms.


Doing so allows us to reflect upon theoretical questions regarding the structure of the mental lexicon, and also puts into perspective a number of assumptions underlying recently proposed distributional text-based models.

we focus on word-embedding models which have been proposed to learn aspects of word meaning in a manner similar to humans.


The resulting co-occurrence frequencies were transformed using the positive point-wise mutual information (PMI + ), given the evidence that this measure performs well in count models (Bullinaria and Levy, 2007;Levy et al., 2015).


To test how well we can predict IS for mentions without using any hand-crafted features, we only use word embeddings and IS labels in our attention-based LSTM model described in Section 3.2.

we use mention embeddings (100 dimensions) as the input of the first LSTM.


Comparing the results of LSTM+PAR+FEAT to LSTM+PAR and LSTM+PAR+FEAT−wordEmb, it seems that word embeddings and the small set of simple features (most of them are capturing the surface and syntactic properties of mentions) are complementary.

mediated/function and mediated/bridging benefit most from word embeddings which provide useful semantic knowledge to capture these two categories.


The goal of this research is to build an end-to-end system to analyse the explicit discourse relations in Chinese texts.

we deal with four tasks together: (1) extraction of explicit discourse connectives, (2) linking resolution between the component candidates, (3) classification of the relation type for each discourse connective, and (4) extraction of the discourse arguments of a connective.


The CoNLL-2015 Shared Task  and the CoNLL-2016 Shared Task (Xue et al., 2016) both focus on shallow discourse parsing.

the CoNLL-2016 Shared Task features Chinese discourse parsing with Chinese Discourse Treebank 0.5 (Zhou and Xue, 2015), a PDTB-style annotated corpus.


Position is used mainly as a tiebreaker.

we accept the left-most candidate first.


We aim at investigating which source of information are relevant for the discourse parsing task.

we draw upon the recent success of deep learning methods and present a novel multiview multi-task hierarchical deep learning model for discourse parsing.


Santa Barbara corpus We use the Santa Barbara corpus of spoken American English (Du Bois, 2000) to get a sequence labeling task corresponding to turns in a conversation, the idea being that rhetorical structure could share similarities with the structure of conversations.

we segment the dialogues by pauses and label the first turn-taking utterance as beginning a new turn.


a discourse unit, a speaking turn, a sentence, depending on the task) goes through the hierarchical bi-LSTM that outputs a sequence of labels for the entire document.

an input sequence is represented as a sequence of word embeddings.


To solve this, instead of reducing the set of available relations, one can increase training data size by using the folding mappings to create new training instances instead of altering existing ones.

given an example A AFTER B one automatically adds an extra B BEFORE A.


Two events are coreferent if they represent the same situation.

the text representation of the event should involve the same action and (some of the) arguments (Yang et al., 2015).


We showed that past work for CDEC have resorted to widely different evaluation approaches, making several implicit assumptions, which simplify the coreference task and lead to overlooking coreference mistakes.

as we showed, excluding singleton mentions from the evaluation does not seem justified.


Similar to TF1, we define POS n-gram features (for n = 1, 2, 3) around a line boundary.

we define 15 POS n-gram features at different positions, as shown in Figure 4.


LSTM controls the learning of the representation of a sequence by gates.

at position t, LSTM controls the information that should be kept from previous states by an input gate i t , and the information that should be forgotten by a forget gate f t .


In this work, we develop a novel joint inference method that labels each utterance in a tutoring dialogue session with a dialogue act and a specific mode from a set of pre-defined dialogue acts and modes, respectively.

we develop our joint model using Markov Logic Networks (MLNs), a framework that combines first-order logic with probabilities, and is thus capable of representing complex, uncertain knowledge.


tutoring systems the induce student learning gains, we first need to understand what effective human tutors do.

we would like to identify specific pedagogical steps that promote effective tutoring.


In this work, we present a novel approach to identify higher-level tutorial constructs called modes in tutor-learner dialogue-based interactions.

dialogue modes are sequences of dialogue acts that map to pedagogical goals such as scaffolding (and sometimes to general dialogue goals such as opening a conversation).


As is often the case in several NLP tasks, a pipeline architecture can be naturally adopted to identify dialogue modes.

we first label acts in each utterance of the dialogue, then, using the labeled acts, we label subacts, and using both the labeled acts and subacts, we finally label the higher level modes.


In this approach, a tutor or student could switch between proposed modes while working on a particular problem.

a particular mode is not associated with one problem solving task but rather with parts of such a problem solving task.


The MLN describes a log-linear model where the probability distribution is defined over the set of possible worlds.

the probability distribution represented by the MLN is given by, where N i (ω) is the number of groundings of the first order formula f i that evaluate to True given a world ω.


Since MLNs are simply a compact representation of PGMs, all inference tasks in PGMs are also applicable to MLNs.

the two main inference tasks for MLNs are, 1) Marginal inference, and 2) MAP inference.


mark is a special symbol in the MLN language that specifies a hard constraint that for every grounding of s and t, there is exactly one act label.

every utterance corresponds to one and only one dialogue act.


asserts that the dialogue subact in the tutorial session for student s at time t is equal to u. Mode(s, t, m!)

asserts that the dialogue mode in the tutorial session for student s at time t is equal to m. Finally, ModeSwitch(s, t) asserts that there was a switch in dialogue mode at time t for the tutoring session associated with student s. the mode in the previous time step was different from the mode in the current time step.


Next, we define hard formulas that encode the rule of mode switching.

we specify that mode-switching causes a shift in the dialogue mode using two implications.


We encode the inter-dependency between modes, acts and subacts with a set of soft formulas.

we model this interaction by encoding a formula that connects two successive time-steps of a dialogue.


With the use of a "+" sign, we can increase the total number of weights in the MLN and thus induce more complex distributions.

we can set a different weight for each partially ground formula obtained by grounding all variables in the formula corresponding to the + symbol.


Feature N denote N features extracted from the utterances (we discuss the actual features in the next section), then, we encode these features using soft formulas of the form, Given the MLN specified in the previous section, the inference task is to jointly compute an assignment to all possible groundings of the query predicate, Mode.

we compute this assignment as a solution to the following optimization problem max ω h∈H where H is the set ground atoms of hidden predicates and Q is the set of ground atoms of query predicates, ω is an assignment on all atoms in Q.


For instance, suppose we have 500 students' dialogues in our dataset, and each dialogue has on average 100 utterances/time-steps, then, the formula, ModeSwitch(s, t) ∧ Mode(s, t − 1, m) ⇒ ¬ Mode(s, t, m) itself has at least 1 million possible groundings.

grounding the entire MLN and then applying MAP inference on the ground MLN quickly becomes infeasible.


Next, we describe weight-learning for the soft formulas in our MLN.

we use gradient ascent to compute weights of the soft formulas that maximize the log-likelihood of our dataset.


We reduce the severity of the problem using random restarts (Selman et al., 1996).

we start gradient ascent from several different initialization points and average all the different weights that gradient ascent converges to when starting from these different initialization points.


However, the number of feasible combinations is much lower.

several combinations never occur in the training dataset and we assume that for all such cases, the weight is 0 (the likelihood that the formula is true/false is the same).


The number of weights that we need to learn corresponding to the feature-based formulas turns out to be extremely large.

grounding the "+" variables, we will have at least O(N * d * |M odes| * |Acts| * |Subacts|), where N is the number of features, d is an upper-bound on the number of possible feature-values for a feature.


Instead, we utilize the flexibility of MLNs to incorporate the feature-based soft formulas implicitly.

we remove all the feature-based formulas from the MLN and learn the MLN weights using only the other formulas.


We train an SVM-based pipeline system to label the acts, subacts, modes and mode-switches in sequence.

we use SV M multiclass to first label the acts.


The SVM-based pipeline system yields confidence values in the form of hyper-plane distances for each dialogue utterance for every mode, act, subact and also whether a mode switch occurred.

given an utterance t for student s, we will have hyper-plane distances for Act(s,t,+a), Subact(s,t,+u), Mode(s,t,+m) and ModeSwitch(s,t) (Note that, these are the atoms in the RHS of the soft formulas specified in 5).


For our joint model, we use Gurobi, a state-of-the-art ILP solver, to solve the MAP inference problem.

we ground the MLN with dialogue data from each student independently and solve the MAP problem for each such partially ground MLN independently using Gurobi.


The average F1-score we obtained using the joint method was nearly 6 points higher than the average F1-score obtained using the pipeline SVM classifier.

both precision and recall of mode identification improved over both metrics.


Next, we evaluated statistical significance of our results.


In our next experiment, we evaluated the performance of our model on hidden predicates.

table 4 shows a comparison of how well the systems perform in terms of labeling the hidden ground atoms (ground atoms of the Act and Subact predicates).


in some cases such as Scaffolding, which is an important step that corrects learners when they are going in the wrong direction, there was nearly a 20 percent increase in recall.


In this paper, we presented a novel joint inference method to detect modes in human-to-human tutoring.

modes are high level abstractions of dialogue speech acts, which give us a much deeper understanding of the underlying process by which natural language tutoring occurs.


In this work, we exploited inter-dependencies between lower-level dialogue acts and the higher-level modes using joint inference.

we developed a Markov Logic Network (MLN) to encode the the joint dependencies between dialogue acts, subacts and modes using weighted first-order logic formulas.


We tuned the word embedding parameters method, corpus and dimensionality, while holding other parameters constant (dropout = 0.5, decayrate = 0.5 and layersize = 2).

we tested the methods Word2vec using the Gensim Word2vec package (Řehůřek and Sojka, 2010) and pretrained Glove word embeddings (Pennington et al., 2014).


We therefore propose to decompose the original problem into two separate simplified problems where each problem focuses on an abstraction.

we train a semantic sequence model to learn semantic patterns, and a syntactic sequence model to learn linguistic patterns.


These abstractions help the model training to focus on learning a specific aspect of the problem.

we train a syntactic sequence model to learn linguistic patterns, and a semantic sequence model to learn semantic patterns.


Each of these simpler problems focuses on an abstraction.

we train a semantic sequence model that learns semantic patterns in conversations, and a syntactic sequence model that learns linguistic patterns in conversations.


Finally, with the input of the question vector q, the slot-value vector d and the embedding vector v of dialogue act type, the decoder generates the answer sequence Y .

the slot-value vector d 1 is a one-hot representation of the slot-value pairs which can regularize the generation process and ensure the information adequacy of answers as suggested by (Wen et al., 2015).


The hyper-parameters of the DNN, including the number and size of the layers, have been set-up empirically with a few trials and errors  on the training and validation corpus.

we have not used any automatic hyper-parameter tuning strategy.


Once the data is prepared, we also need to build a new system (i.e., models) for this specific domain.

slot modeling is one of the most demanding components of the system in terms of costs in annotation and computation.


Suppose we have sequences that are only partially labeled.


A good set of embeddings would achieve a high correlation.

we evaluated the SWordSS embeddings on Luong et al.


To avoid this misrepresentation of rare words, we apply SWordSS embeddings in a language modelling framework.

a log-bilinear language model (LBL) (Mnih and Hinton, 2007).


In this paper, we propose a semi-supervised approach to gender classification by leveraging textual features and a specific kind of indirect links among the users which we call "same-interest" links.

we propose a factor graph, namely Textual and Social Factor Graph (TSFG), to model both the textual and the "same-interest" link information.


The basic motivation of our approach lies in the observation that social link information might be helpful to infer user gender.

we focus on the "following" link and think that two users who follow the same particular user could have the same gender.


For instance, in Figure 1, User b, User c and User d follow the same user named Dior and they are thought to be indirectly linked.

once User b and User c are correctly classified to be female with textual features, User d is more likely to be female since she is indirectly linked to User b and User c.  we propose a factor graph, namely Textual and Social Factor Graph (TSFG), to model both the textual and user social link information.


From this table, we can see that there are many users who are only followed by male users or female users.

in our data set, 143389 users are followed by only male users and 119504 users are followed by only female users.


 is a set of relationships between users.

if a user Suppose that there are N users who have the same interest, the number of the same-interest links is 2 N C .


(2015), we apply maximum entropy (ME) for supervised gender classification.

the ME algorithm is implemented with the Mallet Toolkit 4 .


However, the two text types have always been considered separately, while we argue that there is a shared linguistic content between the two that can be used for knowledge transfer.

the output of learners is a subset of the linguistic input that they are able to understand (Barrot, 2015).


Within this context, the automatic assessment of proficiency levels in learner-produced texts would be a powerful tool for increasing both learners' autonomy and teaching professionals' efficiency.

research Questions this paper aims at answering the following research questions: (i) Can we overcome the lack of a sufficient amount of learner output data by incorporating knowledge from L2 input texts when performing proficiency level classification?


In this paper, we propose a novel approach, namely ensemble LSTM, to user classification by incorporating multiple textual perspectives.

our approach first learns a LSTM representation with a LSTM recurrent neural network and then presents a joint learning method to integrating all naturallydivided textual perspectives.


In this paper, we address the above challenge by proposing a novel approach called ensemble LSTM recurrent neural network.

we first consider the features from each style of text as a separate textual perspective.


(2014) propose a multi-label classification approach to predict both the gender and age of authors from texts.

besides the word features, they also adopt some sentiment and emotion features in their approach.


Even though reviews per author type were collected from a different platform (customer reviews from Amazon, expert reviews from LexisNexis Academic), we argue that the source does not determine or predict the type of author for the following reason: Many of these platforms list both types of reviews side by side, on the same platform.

expert reviews from sources like Rotten Tomatoes are not written by Rotten Tomatoes, but come from the same sources that we used for our study -e.g., major newspapers.


To address this issue, this paper proposes to leverage extra information from knowledge graph, which provides ground truth such as attributes and properties of entities and encodes valuable relations between entities.

we propose a joint model to combine knowledge graph information into the objective function of an event embedding learning model.


One important reason for the problem is the lack of background knowledge in training event embeddings.

if it is known that "Steve Jobs" is the CEO of "Apple", and "John" is likely to be a customer at "Starbucks", the two events can have very different embeddings according to their semantic differences.


2008, we automatically align 1,782 instances of daily trading data with news documents from the previous day.

we use the news information in day t − 1 to predict price movements of stock market in day t. The output classification result [Class +1] represents that the stock closing price will increase compared with the opening price in day t, and  represents that the stock closing price will decrease compared with the opening price in day t. Following Das and Chen (2007) and Xie et al.


Figure 4 shows case studies on events about Google, Apple, Microsoft and Facebook.

we apply two-dimensional PCA projection on the 100-dimensional event embeddings.


While text mining on comment archive may provide important information to the recommendation engine, even for fresh users, the integration of such information adds new complexity and challenges to the system, since the text mining results are not directly useful to the recommendation algorithms.

most of the opinion mining techniques available in the literature (Liu and Zhang, 2012;Melville et al., 2002) do not distinguish attributes on user preference from attributes of items.


We build a regression/classification model on top of the user/item representations.

a prediction model is a parameterized function F (U i , I j ) with inputs of user/item vectors and outputs of score estimation.


Here, we use tanh(a) = (e a − e −a )/(e a + e −a ) for function s since it is fast when training the parameters.

the prediction is then obtained by applying: where W 2 is the weight matrix connecting the hidden layer to the output layer and b 2 is a bias vector for the classifier.


Our work consists of two parts: modeling SPs for verbs and incorporating SPs into an SMT system.

we focus on the verb-object (v, obj) and verb-subject (v, subj) selectional preference instances which can be extracted from our target-side corpus.


Tightness (semi-tightness) follows if C(•) is monotone increasing (respectively monotone non-decreasing) in AVG Co (•) when C o is monotone non-decreasing in operator length.

a metric assigning lower complexity to more factorizable permutations (by average operator length) is tight, i.e., it allows comparing permutations by the smallest complexity assigned to them within this framework.


Figure 4 shows the BLEU scores of generated translations on the test sets with respect to the length of the source sentences.

we test the BLEU scores on sentences longer than {0, 10, 20, 30, 40, 50, 60} in the merged test set of MT03, MT04, MT05 and MT06.


This observation calls into question the usefulness of question word demotion in all situations.

in Beetle, question word demotion can be counterproductive.


On the topics of "Samsung Galaxy" and "Xbox" with the active communication environment, CIM still has a significant improvement concerned with the evaluation metrics on all the three opinion categories.

the improvements compared with the best competitors on the positive opinion prediction and the negative opinion prediction are 17.7%, 21.5% for the topic "Samsung Galaxy" and 11.5%, 20.3% for the topic "Xbox" respectively.


In a causal relation, the cause intuitively precedes its effect.

the tense information could be useful features in the tasks of causal analysis.


In Chinese, however, the temporal information is communicated with aspect particles such as 了 (le) and 着 (zhe) and temporal adverbials such as 現在 ("now") and 明天 ("tomorrow") (Xue et al., 2008;Ge et al., 2015).

it is more challenging to determine the tense in Chinese text.


Deep learning techniques are increasingly popular in the textual entailment task, overcoming the fragility of traditional discrete models with hard alignments and logics.

the recently proposed attention models (Rocktäschel et al., 2015; Wang and Jiang, 2015) achieves state-of-the-art accuracy by computing soft word alignments between the premise and hypothesis sentences.


Instead of discrete features and logics, continuous representation of the sentence is more robust to unseen features without sacrificing performance (Bowman et al., 2015).

the attention model based on LSTM can successfully identify the word-by-word correspondences between the two sentences that lead to entailment or contradiction, which makes the entailment relation inference more focused on local information and less vulnerable to misleading information from other parts of the sentence (Rocktäschel et al., 2015;Wang and Jiang, 2015).


To be specific, during the r + 1 hop of the reasoning operation, the process can be formalized as: u i c i , and the dual-channel memories are updated as follows: where 1 ≤ r ≤ (R − 1).

during the r + 1 hop, the word embedding matrices A r+1 and C r+1 are mutually independent, so as the temporal encoding matrices T r+1 A and T r+1 C .


For word-level memory, we first apply a Bi-directional GRU (BiGRU) to compute the hidden states of all the ordered wordsw = {w t } t=(1,2,...,|t|) in the sentence set X, where |t| is the time steps of the words in the sentences.

for the t-th wordw t , the forward GRU and the backward GRU encode GRU (C Rw t ) respectively, where C R is the word embedding matrix of the last hop on the sentence-level memory, and we set the dimension of ⃗ h t and ← h t equals to the dimension of the word embedding.


(2014), but were unable to replicate the reported results.

recall of the re-implementation was lower than expected.


We employ the local similarities φ local (m i , p ij ) from (Francis-Landau et al., 2016), the state-of-the-art neural network model for EL.

: In this formula, W sparse and W CN N are the weights for the feature vectors F sparse and W CN N respectively.


, on the other hand, involves the cosine similarities between the representation vectors at multiple granularities of m i and p ij .

: The intuition for this computation is that the similarities at different levels of contexts might help to enforce the potential topic compatibility between the contexts of the entity mentions and target candidates for EL (Francis-Landau et al., 2016).


In this section, we evaluate the effectiveness of the global features for EL.

we differentiate two types of global features based on the side of information we expect to enforce the coherence.


The first observation from the table is that the performance of all the compared systems on the target domains is much worse than the corresponding in-domain performance.

the performance gap between the in-domain performance and the the worst out-of-domain performance (on the domain wl) is up to 10%, thus indicating the mismatches between the source and the target domains for EL.


In order to better understand the performance gap in the domain adaptation experiments for EL, we visualize the representation vectors of the entity mentions in different domains.

after Global-RNN is trained, we retrieve the representation vectorsc i for the immediate contexts of the entity mentions in the source and target domains, project them into the 2-dimension space via the t-SNE algorithm and plot them.


While there are several methods to estimate domain similarities (Plank and van Noord, 2011), in this work, we employ the mean of the cosine similarities of every mention pairs in the two domains of interest.

let E and F be the two domains of interest, and E = {e 1 , e 2 , .


Normalization and POS tagging The normalization tackles issues that affect subsequent phases of the induction process.

it improves the accuracy of noun chunking on NUT, avoiding an expensive (and possibly pointless) NUT-specific training.


Larger corpora also have the disadvantages that sufficiently frequent but incorrect tokens can end up being extracted as modifiers.

the increase in recall is not matched by a comparable increase of precision (Figure 4b).


In Figure 1, words from word embedding matrix W, sentences from sentence embedding matrix S, documents from document embedding matrix D and class-labels from class embedding matrix C are used as the context from which to predict the target word.

given a sequence of training words w 1 , w 2 , w 3 , ..., w T that belongs to sentence s t in document d t , which has also a set of classification labels associated c 1 , ...c m .


Our goal is to evaluate the usefulness of types as standalone lexical components in the retrieval of research-level mathematical information needs.

our hypothesis is twofold: (a) types are important discriminators in the mathematical discourse and (b) semantic relationships between types can be used to enrich queries and obtain significant improvements in retrieval efficiency.


Following some previous work (Plank and Moschitti, 2013;Gormley et al., 2015), we consider a domain adaptation setting for coarse-grained relation extraction.

we take the union of bn and nw as the training set, half of bc as the development set, and the remainder (i.e., cts and wl as well as the other half of bc) as the test set.


For the other parameters in Θ and Θ , we adopt the settings reported by Nguyen and Grishman (2015).

the non-linear transformation function g is tanh, the mini-batch size is 50, the dropout rate α equals 0.5, and the hyperparameter for the l 2 norms is set to be 3.


However, from an efficiency's point of view, the deep models are very time-consuming to train.

to achieve good success, the models typically require a large number of parameters and weights, making parameter (and hyper-parameter) tuning an expensive process for large-scale applications.


Each word vector is trained to maximize the log probability of neighboring words in a corpus.


In this paper, we propose a joint inference approach to simultaneously solve these disambiguations.

our method consists of two main steps as outlined in ( §2).


Then we employ two types of relation extractors to predict both KB predicates and textual relations existing between two entities or question word and entities in the question.

we propose a neural network based method to map relational phrases to KB predicates, and apply a paraphrase model to find most likely textual relations that describe the phrases.


Since we represent the meaning of a question using clues from two types of heterogeneous resources, we tackle the QA problem in an IE-based fashion involving entity linking and relation extraction.

we simultaneously map relational phrases to KB predicates and textual relations.


To alleviate this problem, we propose a paraphrase based method that can map relational phrases to proper textual relations.

we first apply an open information extractor (Angeli et al., 2015) on the English Wikipedia to construct a repository of <argument 1 , relation, argument 2 > triples, where the arguments are entity phrases found in the input sentence and the relation represents certain relationship between the arguments.


KB Predicate and Entity's Coherence If the relational phrase rp is grounded to a KB predicate kr, we should examine whether the semantic types of the entities fulfill the expectations of KB predicates.

we first obtain the type of subject entity e, which is collected from the KB's schema, and examine whether there exists another entity with the same type taking the subject position of this predicate in the KB.


We also compared our paraphrase model (RAE) with two baselines: EDIT-based and VECTOR-based paraphrase models.

the former computes the token edit distance between the textual relation tr and relation phrase rp as the similarity score, obtaining 43.6% and 35.4% F 1 on the development set of WEBQUESTIONS and QALD-6, respectively.


We mainly rely on the coherence score between KB predicates and textual relations, i.e., Coh kr,tr , to guide the inference model to find a proper assignments.

if kr and tr have the same argument type expectations, we compute the Coh kr,tr as the probability of mapping tr to kr using the neural network as described in §3.2.


This "contribution" is here defined as the number of correct next hops on path P t that are computed, if the context vector v c was replaced by v t+j .

the number of correct hops on the path that can be predicted, if we use context word w t+j alone instead of the complete set of context words of w t .


In this work, we explore several architectures to build neural word embeddings for MSA and dialectal Arabic.

we perform a comprehensive analysis to train and evaluate word representations using CBOW and Skip-Gram models, with the goal of generating a better quality representations for Arabic sentiment classification.


The max pooling operation is shown in Figure 2.

the output of convolution, the feature map c = [c 1 , c 2 , .


In this work we present a deep neural network architecture that takes advantage of the construction of convolutional neural network (CNN) and recurrent neural network (RNN) and joint them together for sentimental analysis of short texts.

our pooling operation on adjacent words is able to retain the local features and their sequential relations in a sentence.