Topic Difference (TopicDiff): We hypothesize that a metaphorical sentence is more likely to be different from its neighboring sentences, in terms of topic, than a literal sentence.

top-icDiff is a two-dimensional binary feature that indicates whether the topic assigned to the target sentence is different from that of the previous and next sentences.


There are then multiple copies of each topic transition and emotion/cognition feature, all paired with a different target word.

if there are N target words, our feature space becomes N + 1 times larger.


One possible reason is that Topic 9 and Topic 0 tend to frequently co-occur and be metaphorical.

if a target word comes after or before Topic 9 and it is Topic 0, then this word may more likely be metaphorical.


The weights of the output layer are also compressed in a same way as embeddings.

the number of trainable neural parameters is a constant regardless of the vocabulary size if we ignore the biases of words.


Our intuition is that oftentimes a word can be defined by a few other words, and that rare words should be defined by common ones.

it is reasonable to use a few common words' embeddings to represent that of a rare word.


As both word embedding and context vectors capture these co-occurrence statistics (Levy and Goldberg, 2014), we can expect that context vectors share the same internal structure as embeddings.

moreover, for a fine-trained network, given any word w and its context h, the output layer's weight vector corresponding to w should specify a large inner-product score for the context h; thus these context vectors should approximate the weight vector of w. word embeddings and the output weight vectors should share the same internal structures and it is plausible to use a same set of sparse representations for both words and target-word weight vectors.


Extrinsic evaluation is a valid methodology, but it does not allow us to understand the properties of representations without further analysis; e.g., if an evaluation shows that embedding A works better than embedding B on a task, then that is not an analysis of the causes of the improvement.

extrinsic evaluations do not satisfy our goals.


The SVMs trained on PPMI and CBOW representations assigned all four test set words to the negative class; in particular, w 3 and w 4 were incorrectly classified.

the accuracy of classification for these models (50%) was not better than random.


In reality, little is understood regarding what constitutes a high quality seed lexicon, even with "traditional" distributional models (Gaussier et al., 2004;Holmlund et al., 2005;Vulić and Moens, 2013).

in this work we ask whether BWE learning could be improved by making more intelligent choices when deciding over seed lexicon entries.


The only bilingual signal required are document alignments.

our focus is on novel less expensive Type 4 models.


However, for words with multiple senses such as "fire" (shooting or employment termination), similar words may indicate multiple event types.

we propose to apply Word Sense Disambiguation (WSD) and learn a distinct embedding for each sense (Section 2.3).


We observe that, for two triggers t 1 and t 2 , if their arguments have the same type and role, then they are more likely to belong to the same type, and vice versa.

we introduce a constraint function f , to enforce inter-dependent triggers and arguments to have coherent types: where P1 and P2 are triggers.


Events in nearby locations in the visualization space are likely to share similar event elements.

jointly learning the two tasks could potentially bring benefits to each other.


Tweets from different events are mixed together and events are evenly distributed across the whole visualization space.

this visualization does not provide any sensible information about the relationships between tweets and events.


To achieve a better level of abstraction, we disambiguate semantic frames and map them to their PropBank/FrameNet represen-tation.

in Ex.3, the resulting frame chain is "rob.01 -but -arrest.01" ("01" indicates the predicate sense).


For each co-reference chain in a document, we first extract semantic frames corresponding to each co-referent mention, disambiguate them as before, and then determine the discourse markers between these frames.

each unique frame contains both the disambiguated predicate and the argument label of the mention.


Each mention can be matched to an argument inside a semantic frame.

we replace each mention with its argument label inside a semantic frame, and We then add discourse markers exactly in they way we do for frame-chain SemLM, and get the following sequence: The comparison of vocabularies between frame-chain and entity-centered SemLMs is summarized in Table 1.


"eat and drink", "decide to buy", they actually represent a unified semantic meaning.

we construct compound verbs to connect them together.


(2015) also use the NYT portion of the Gigaword corpus, but with Concrete annotations; Pichotta and Mooney (2016) use the English Wikipedia as their data, and Stanford NLP tools for pre-processing while we use the Illinois NLP tools.

the eventual chain statistics are different, which leads to different test instances.


However, they rarely appear in Electronics domain (Bollegala et al., 2011).

a sentiment classifier trained in Electronics domain cannot accurately predict their sentiments in Book domain.


For instance, "the ending of this movie is easy to guess."

the sentiment classifier trained in one domain usually cannot be applied to another domain directly (Pang and Lee, 2008).


Researchers have found that sentiment domain adaptation between similar domains, such as Kitchen and Electronics, is much more effective than that between dissimilar domains, such as Kitchen and Book (Blitzer et al., 2007;.

it is beneficial if we take the similarity between source and target domains into consideration when transferring sentiment knowledge.


Denote t m ∈ R D×1 as the term distribution of domain m, where t m w is the probability of term w appearing in domain m. Then the similarity between domains m and n is formulated as: where t = 1 2 (t m + t n ) is the average distribution, and D KL (•, •) is the Kullback-Leibler divergence: We can verify that D JS (t m , t n ) ∈ [0, 1].

the range of T ermSim(m, n) is also [0, 1].


For example, "this battery runs out fast."

it is more useful to measure domain similarity based on sentiment word distributions.


This is because our approach can exploit more sentiment-related characteristics and knowledge for sentiment domain adaptation, such as the general sentiment knowledge shared by various domains, the sentiment graph based domain similarities, and the word-level sentiment polarity relations.

our approach is more suitable for sentiment domain adaptation than these generalpurpose multiple source domain adaptation methods.


For example, "easy" and "works" are positive words in both Electronics and Kitchen domains, and "return" and "broken" are both negative words in them.

transferring the domainspecific sentiment models from similar source domains to target domain is helpful.


Due to the mismatch among different domains, a sentiment classifier trained in one domain may not work well when directly applied to other domains.

domain adaptation for sentiment classification algorithms are highly desirable to reduce the domain discrepancy and manual labeling costs.


As a result, the transferred source domain is supervised and follows similar distribution as the target domain.

any supervised method can be used on the transferred source domain to train a classifier for sentiment classification in a target domain.


Learning matrices B s and B t takes O(m 2 × n s ) and O(m 2 × n t ) operations per iteration.

in real applications, we have k m. the overall complexity of the algorithm, dominated by computation of matrices B s and B t , is O(m 2 × n) where n = max(n s , n t ).


Quotation Features Quotations often involve subjective opinions towards prominent entities in news articles.

we include document-level features encoding this intuition.


While we study the complete document-level analysis over all entity pairs, the KBP task is formulated as query-focused retrieval of entity sentiment from a large pool of potentially relevant documents.

their annotations focus only on query entities and relatively sparse compared to ours (see Sec.


Combined with the results in Table 2, we may infer that to achieve 77.3% accuracy on CTB -test, AL with FA requires 149, 051 × 6.7 = 998, 641.7 seconds of annotation, whereas AL with batch dependency-wise PA needs 56, 389 × 13.6 = 766, 890.4 seconds.

we may roughly say that AL with PA can reduce annotation time over FA by 998,641.7−766,890.4 998,641.7 = 23.2%.


We note that optimizing objective (1) subject to constraints (2), (3) and (6) amounts to finding a MSA and can be solved combinatorially (McDonald et al., 2005).

since formulation (1)-(6) is based only on arc variables, by relaxing constraints (4) and 5, one obtains a Lagrangian dual objective which is nothing but a MSA problem with reparameterized arc scores.


Indeed, by definition of g, we have [g p ] i ≥ 0 if constraint i is satisfied at iteration p: the projection on R + ensure that [u p+1 ] i stays at 0.

6 we do not need to know constraints that have not been violated yet in order to correctly update the Lagrangian multipliers: this is the main intuition behind the NDRC method.


7 Moreover, the subgradient descent algorithm theoretically converges but we have no guarantee that this will happen in a realistic number of iterations.

in order to retrieve an optimal solution in all cases, we embed the Lagrangian Relaxation of the problem within a Branch-and-Bound procedure (Land and Doig, 1960).


For each language, we decided to use the most constrained combination of bounded block degree constraints and well-nestedness which covers over 99% of the data.

we chose to enforce 2-BBD and well-nestedness for English and Spanish, 3-BBD and well-nestedness for Dutch and Portuguese and 3-BBD only for German.


We will demonstrate that, even with heuristic downsampling of frequent terms in word2vec, these techniques result in inferior performance for specific topics.

far, we have sketched out why using the corpus distribution for a specific topic may result in undesirable outcomes.


Since, the askers are non-experts, such rich concept associations are hard to learn from (Q, Q) training archives even if they are available in large scale.

leveraging (Q, A) training data leads to learning richer concept/term associations in SCQA.


The similarity between the pairs is calculated using the similarity metric learnt during the training.

sCQA outputs a value of distance measure (score) for each pair of questions.


These differences make the collaborative filteringbased methods, which have been widely applied to paper citation recommendation, less available in our scenario.

in this paper we focus on content similarity-based methods to deal with the task of news citation recommendation.


A cited article can either be represented by its head-line+lead paragraph or as a whole.

we extract features under two different settings: (a) headlines and lead paragraphs only; (b) the full articles.


However, feature 2 is much too computation-intensive and feature 7 needs POS-tagging as the preprocessing.

these two are only extracted under setting (a).


Due to the diversity of scripts, phoneme inventories, phonotactic constraints, and spelling conventions among the world's languages, they are typically languagespecific.

while most statistical g2p learning methods are language-agnostic, they are trained on language-specific data-namely, a pronunciation dictionary consisting of word-pronunciation pairs, as in Table 1.


This is because IPA Help tables explain phonemes' relationships to graphemes, rather than vice versa.

the English letter x is omitted, since its composite phonemes are better explained by other letters.


The calculation of margin loss could be regarded as to count the number of incorrectly segmented characters and then multiple it with a fixed discount parameter for smoothing.

the loss is proportional to the number of incorrectly segmented characters.


Our model is able to handle them reasonably well just by stringing important sentences together.

the model imitates reasoning with a heuristic.


In other words, the training set will have a sample selection bias that can impair prediction performance.

to mitigate this, we propose to sample questions from a biased sample distribution D. to achieve D, we introduce the weighted loss q,a) which represents how likely it is to observe (q, a) under D compared to D. In this setting, we can show that the generalization error under D is the same as that under D: given appropriate weights w(q, a), we modify our loss-function in order to compute an unbiased estimator of the generalization error.


For Lucene, the improvements over all but the Waterloo corpus baseline are statistically significant.

fRETS is able to capitalize on data more effectively and rival an unstructured model with access to orders of magnitude more data.


However, short-texts usually do not observe the syntax of a written language, nor do they contain enough signals for statistical inference (e.g., topic model).

neither parsing nor topic modeling works well because there are simply not enough signals in the input, and we must derive more semantic signals from the input, e.g., concepts, which have been demonstrated effective in knowledge representation (Wang et al., 2015c;.


Shot-text conceptualization, is an interesting task to infer the most likely concepts for terms in the short-text, which could help better make sense of text data, and extend the texts with categorical or topical information (Song et al., 2011).

our models utilize shorttext conceptualization algorithm to discriminate concept-level sentence senses and provide a good performance on short-texts.


The inspiration is that, in researches of word embeddings: (i) The word vectors are asked to contribute to a prediction task about the target word or the surrounding words in the context; (ii) The word representation vectors are initialized randomly, however they could finally capture precise semantics as an indirect result.

we will utilize this idea in our sentence vectors in a similar manner: The conceptassociated sentence vectors are also asked to contribute to the prediction task of the target word or surrounding words in given contextual text windows.


Although this strategy may not be the best approach and there exist more elaborate attention models (Bahdanau et al., 2014;Luong et al., 2015), the proposed attention model is a proper balance of computational efficiency and complexity.

besides {W,C,S} in CSE models, D and R are added into parameter set which relates to gradients of the loss function Eq.(1).


Human annotators are asked to freely issue 100 queries to each official account to get XiaoIce response.

we obtain 100 query, XiaoIce response pairs for each official account.


In contrast, from a strategic or communicative perspective, alignment -in which preceding words and concepts are reused -must be balanced against a need to move the conversation forward by introducing new words and concepts.

on a communica-tive account, alignment should be modulated by the speaker's discourse act, reflecting whether the balance of the concern is convergence on a current focus or conveyal of new information.


We hypothesize that alignment will be substantially different depending on the discourse act, as speakers' conversational goals vary.

our second experiment examines how alignment differs depending on discourse act.


Intuitively, longer sentences tend to convey more information than short ones.

the per-word entropy of a sentence should be correlated with the sentence length, i.e., the number of words.


By processing RelevanceSet = {mQ, mR, mIPD, mHCS, mACB, mICR, mI, mSCD, mPB, mEU, mFP, mED, mHLP, mSI, mSP, mHLC} Having identified the RelevanceSet, we now intend to identify a satisficing combination of the involved motives.

the mapper sends the RelevanceSet to the mixed motive model handler for specifying the Satisf icingSet that consists of motives that (1) are sufficiently interesting for all interlocutors (i.e.


Specifically, word t in s 1 may be potentially influenced by all words in s 2 in some degree.

for word t in s 1 , a candidate interaction statec  here, the superscript "i" indicates "interaction".


The average of all hidden states are treated as sentence vectors v cnn s 1 and v cnn s 2 .

sIN-CONV is sIN with word vectors substituted by phrase vectors.


In a dialogue, the DA of a sentence is highly relevant to the content of itself and the previous sentences.

to model the interactions and long-range dependence between sentences in a dialogue is crucial for dialogue act analysis.


This happens very often in SimpleQuestions, since only around 5% of the entities have more than 2 connections in the graph.

by applying TransE directly over this set of triples, we would eventually end up with a layout of entities that does not contain clusters of semantically close concepts.


the overall input representation z t is computed as: 5 Latent Predictor Networks Background In order to decode from x to y, many words must be copied into the code, such as the name of the card, the attack and the cost values.


To find q(v) efficiently, we leverage the fact that , which means that the maximum compression obtainable for v at size max is always lower than that of v .

if we can find av such that , that isv at the current size achieves a better compression rate than v at the maximum length, then it follows that all sequences that contain v can be discarded as candidates.


However, it does not provide an illustration on the degree of closeness to achieving the correct code.

we also test using BLEU-4 (Papineni et al., 2002) at the token level.


Large-scale recognition of objects and their attributes in images is still a non-trivial task.

rEG systems now face the challenge of dealing with semantically inadequate expressions.


For instance, in Figure 1, the system might not precisely distinguish between man or woman and generate an inadequate, confusing RE like man in the middle.

we focus on evaluating our system in an object identification task with users, in contrast to previous approaches to REG on images (Mao et al., 2015).


Given the real-time constraints of situated interaction, a speaker often has to start uttering before she has found the optimal expression, but at the same time, she can tailor, extend, adapt, revise or correct her referring expressions in case the listener signals that he did not understand.

human speakers can flex-ibly split and adapt their REs over several utterances during an interaction, a phenomenon called "reference in installments".


First of all, this suggests that the differences in performance between the systems is much bigger in terms of their communicative effectiveness as in terms of the corpusbased measures (Table 1).

on the one hand, the GoogLeNet features are clearly superior to SA-IAPR, whereas differences between GoogLeNet and GoogLeNet glen are minor.


We have implemented two versions of this general strategy: (i) pattern-based installments that always avoid object names in their initial expression and dynamically extend this if necessary, (ii) contextdependent installments that condition the initial expression on the complexity of the scene and extend the initial expression accordingly, inspired by standard approaches to attribute selection in REG (Krahmer and Van Deemter, 2012).

we do not test initial or reformulated expressions in isolation, but the strategy as a whole, which is similar to (Fang et al., 2014).


This systems seems to strike a much better balance between generating simple expressions that avoid  The finding that installment strategies should be combined with insights from traditional distractororiented REG is further corroborated when we compare the success rates on the different subsets of our test set, see Table 5.

the performance of the context-dependent installment system is much more stable on the different subsets than the pattern-based system which has a clear dip in success rate on Set C, which contains target referents with distractors of the same object type.


4: and maximize f (y i ) to obtain the label.

far we assumed the singleton and pairwise scores were given.


We also note the alias-entity map used in each system, as the corresponding gold recall is an upper bound on accuracy, and alias ambiguity determines the difficulty of the task.

performance is not strictly comparable between maps.


By comparing the evaluation scores, we determine which system performs best, which model suits coreference resolution better, and which feature set is useful for improving the recall or precision of a coreference resolver.

evaluation metrics play an important role in the advancement of the underlying technology.


This is indeed a requirement for coreference resolvers to be used in end-tasks.

averaging individual metrics is nothing but a compromise.


Mention m is an incorrectly resolved mention if m ∈ M k,r and L k (m) ∩ L r (m) = ∅.

m is a coreferent mention that has at least one coreference link in the response entities.


When the number of gold mentions that exist in the response entities is larger, the number of detected non-coreference links will also get larger.

it results in higher values for BLANC recall and precision ignoring whether those gold mentions are resolved.


For each entity, LEA considers how important the entity is and how well it is resolved.

lEA evaluates a set of entities as follows: e i ∈E (importance(e i ) × resolution-score(e i )) e k ∈E importance(e k ) We consider the size of an entity as a measure of importance, i.e.


r j is a partial match for k i , if it contains at least one of the coreference links of k i .

if a response entity only contains one mention of k i , it is not a partial mapping of k i .


LEA also does not rely on non-coreferent links in order to detect entity structures or singletons.

the mention identification effect does not apply to LEA recall and precision.


More splits (or similarly merges) in entity k i result in a smaller .

splitting (merging) of an entity in several entities will be penalized implicitly in resolution-score.


the existence of repeated mentions in different response entities is not troublesome for LEA.


The sets of coreference links in k 1 and k 2 are {ab, ac, bc} and {de, df, dg, ef, eg, f g}, respectively.

link(k 1 ) = 3 and link(k 2 ) = 6. ab is the only common link between k 1 and r 1 .


Similarly, k 2 has one common link with r 3 and it has no common links with r 1 or r 2 .

resolution-score(k 1 ) = 1+0+0 3 and resolution-score(k 2 ) = 0+0+1 .


Therefore, resolution-score(k 1 ) = 1+0+0 3 and resolution-score(k 2 ) = 0+0+1 .

lEA recall is computed as: By changing the roles of key and response entities, lEA precision is computed as: 6 Evaluation on Real Data Table 4 shows the scores of the state-of-the-art coreference resolvers developed by Wiseman et al.


On the other hand, the number of these gold mentions in xu is 757.

these different rankings could be a direct result of the mention identification effect.


However, because photographs may contain overlapping elements-for instance, a photograph in the "bus" category of PAS might also show cars, while a photograph in the "cars" category could also contain a bus-texts in one cluster can also be quite similar to texts from other clusters.

these datasets should not be very tightly clustered.


We hypothesize that if a dataset uses a wide variety of words to express the same ideas, similarity metrics that rely on the surface form of the sentence will be at a disadvantage compared to similarity metrics based in distributional semantics.

word2vec, LSTM autoencoders, and skip-thoughts ought to perform better than the n-gram-based methods and dependency count method when applied to CLUE, but should enjoy no advantage when applied to PAS.


k i is the degree of vertex i, so 2m is the expected number of edges between i and j in a random graph.

modularity is highest when nodes in a cluster are highly interconnected, but sparsely connected to nodes in different clusters.


semantics-based similarities rarely have modularity much higher than zero.

we conclude both that CLUE is more tightly clustered than the other datasets and that surface-form-based measures yield tighter clusters than semantics-based measures.


When j = k, we have where u • V is to multiply each column of V with u element-by-element.

∂r ik We proceed to optimize T i with a gradient descent method: is the learning rate function, L 0 is a pre-specified document length threshold, and λ 0 is the initial learning rate.


Note that the sum of each row is equal to 1.

using the same principle, we can derive ϕ c k which can be formulated as: where η e , η c ∼ Dir(β) the incorporation of η e k and η c k on the topic level encourages the word distribution ϕ e k and ϕ c k to share common semantic components of reader comments in different languages.


Inspired by this intuition and the block assignment we obtain in the previous section, we want to extract some prior knowledge from these blocks.

we propose an LDA with block priors, hence BP-LDA, as shown in Figure 4, which has the following generative process: Unlike conventional LDA, which uses an uninformative topic prior, BP-LDA puts a Dirichlet prior π on each block to capture the block's topic distribution and use it as an informative prior when drawing each document's topic distribution.


In terms of link set B, while it adopts all explicit positive links from the input, it does not deny the existence of unobserved links, or implicit negative links.

b consists of only explicit positive links.


Dialects differ from MSA phonologically, morphologically and syntactically and they do not have standard orthographies (Habash, 2010).

resources built for MSA cannot be adapted to dialects very well.


As such sentiment lexicons that are built from standard dictionaries cannot adequately capture the informal language in social media text.

in this paper we propose to generate Arabic sentiment lexicons that are tweet-specific i.e.


Generating sentiment lexicons for Arabic has gained the interest of the research community lately.

we found several efforts for generating these lexicons.


However, according to (Kiritchenko et al., 2014) other thresholds could give better results.

we experimented with the value of this threshold.


The step (see Sub-section 3.2) for learning of HMM parameters {π π π, B, A} is heavily dependent on the initial values of these parameters (Wu, 1983;Xu and Jordan, 1996;Huda et al., 2006).

a good initial estimation of the HMM parameters can help achieve a higher classification accuracy.


For each segment, create a D-dimensional vector where the i th element in the vector is one (zero) if the i th element in the selected word set does (not) appear in the segment.

s binary D-dimensional vectors are generated, and the set of these vectors is denoted by 4.


We represent each sentence as a binary feature vector using the same feature set used in Step 4.

z binary D -dimensional vectors, denoted by  In this approach, we use add-one smoothing (Martin and Jurafsky, 2000) for avoiding zero probabilities of A and B.


The Viterbi decoding algorithm (Forney Jr, 1973) is used to determine the best sequence of states for generat-ing observation sequence.

by using the Hidden Markov model that is constructed in previous sub-section and the learned HMM parameter values, the Viterbi decoding algorithm is applied to find the best sequence of authors for the given sentences.


In our experiments, all possible multi-author documents of articles of these columnists are created.

this corpus permits us to examine the performance of our approach in documents written by more than two authors.


Due to comparatively shorter lengths of these documents, the number of resulted segments that are used for the unsupervised learning in Sub-section 3.1 is clearly not sufficient.

instead of splitting each document into segments of 30 sentences length each, we split it into segments of 10 sentences length each.


The deep architecture of neural network models, however, makes it rather difficult to identify and extract those properties of text that the network has identified as discriminative.

we also describe a preliminary method for visualizing the information the model is exploiting when assigning a specific score to an input text.


However, the class Other is used to indicate that relation between two nominals dose not belong to any of the 9 directed classes.

the class Other is very noisy since it groups many different types of relations with different directions.


In most cases, only common nouns will result in the 1-N mismatch problem.

in order to control the size of rewritings, this paper only rewrite the common nouns in a sentence by replacing them with their dictionary explanations.


When scaling a semantic parser to open-domain situation or web situation, the mismatch problem will be more common as the ontology and language complexity increases (Kwiatkowski et al., 2013).

we believe the sentence rewriting method proposed in this paper is an important technique for the scalability of semantic parser.


To see the reason, assuming that the vocabulary size is V and the number of neurons in the first hidden layer l 1 is L 1 , the size of the weight matrices W 1 and W ′ 1 is V * L 1 , which in our dataset is around two million while the number of training examples is much smaller.

instead of using one-hot vectors, we employ embedding features.


substantial model retraining is often needed when a new prompt is released.

systems that can adapt to new writing tasks (i.e.


learning a weight vector w such that w(x i − x j ) > δ, when essay i has a higher gold score than essay j, where δ is the one-sided margin 3 (Joachims, 2002;Chapelle and Keerthi, 2010).

instead of directly learning to predict the gold score of an essay vector, the model learns a weight vector w that minimizes the misclassification of difference vectors.


Furthermore, for cTAP rank we train a final linear regression step on each of the task-specific training data separately.

we predict a score y for essay x i for task T j asŷ = β j (w • x i ) + j .


QW-κ) is extremely difficult (if not impossible).

table 4 only shows results for models that make use of target task data.


SPARQL) can be issued to retrieve specified facts from such KBs.

answering factoid questions will be straightforward once they are converted into the corresponding structured form.


The core idea is to learn semantic representations of both the question and the knowledge from observed data, such that the correct supporting evidence will be the nearest neighbor of the question in the learned vector space.

a main difference among several approaches lies in the neural networks proposed to represent questions and KB elements.


The fundamental intuition for pruning is that the subject entity must be mentioned by some textual substring (subject mention) in the question.

the candidate space can be restricted to entities whose name/alias matches an n-gram of the question, as in (Yih et al., 2014;.


Then, the question embedding model f consists of a word embedding layer to transform tokens into distributed representations, a two-layer BiGRU to capture the question semantics, and a linear layer to project the final hidden states of the BiGRU into the same vector space as E(r).

subject network As introduced in section 3, the factor p(s|q, r) models the fitness of a subject s appearing in the question q, given the main topic is about the relation r. two forces a) the raw context expressed by q, and b) the candidate topic described by r, jointly impact the fitness of the subject s. For simplicity, we use two additive terms to model the joint effect where u(s, r, q) is the subject scoring function, g(q) is another semantic question embedding, E(s) is a vector representation of a subject, h(r, s) is the subject-relation score, and α is the weight parameter used to trade off the two sources.


Each dimension of a type vector is either 1 or 0, indicating whether the entity is associated with a specific type or not.

the dimensionality of a type vector is equal to the number of types in KB.


Under this setting, with E(s) being a binary vector, let g(q) be a continuous vector with arbitrary value range can be problematic.

when type vector is used as E(s), we add a sigmoid layer upon the final linear projection of g, squashing each element of g(q) to the range [0, 1].


To illustrate the effectiveness of this idea, we parameterize p κ (w|q) with a general-purpose neural labeling model, which consists of a word embedding layer, two layers of BiGRU, and a linearchain conditional random field (CRF).

given a question q of length T , the score of a sequence label configuration y ∈ R T is where H(q) is the hidden output of the top-layer BiGRU, A is the transition matrix possesed by the CRF, and [•] i,j indicates the matrix element on row i collum j.


6 Given the state-of-the-art performance of sequence labeling systems, this result should not be surprising.

the difference in performances entirely comes from their resulting accuracy.


In contrast, under the multi-candidate case, the system also relies on p θs (s|q, r), which has significantly more parameters to estimate, and thus is less robust.

by only focusing on the most probable sub-string, the proposed focused pruning produces much more single-candidate situations, leading to a better overall accuracy.


(ii) In a second step, we refine and extend the semantic characterization of the verb classes by translating it to English and linking it to existing semantic classes in lexical resources at the word sense level: we consider the coarse semantic fields in the German wordnet GermaNet (Kunze and Lemnitzer, 2002), the verb classes in the English lexicon VerbNet (Kipper et al., 2008), and the semantic frames in the English lexicon FrameNet (Baker et al., 1998).

we obtain a detailed semantic characterization of the verb classes, as well as insights into the validity of Levin's theory across the related languages German and English.


H: Will Grant is researching a conflict with Mexicans.

an insufficient overlap of modality markers in T and H might actually indicate non-entailment, but lead to an incorrect classification as entailment when considering only word overlap.


However, the syntactic properties of the English classes are not fully specified, because English has additional kinds of non-finite arguments, such as ing-forms or bare infinitives.

it might be interesting to address this question in the future and to build a similar semantic classification for English from scratch, in particular in the context of extracting modality classes from corpora.


To capture temporal dynamics in a user's opinions with existing solutions, repeated model reconstruction is unavoidable, albeit it is prohibitively expensive.

personalized sentiment analysis requires effective exploitation of users' own opinionated data and efficient execution of model updates across all users.


Plugging this group-wise model adaptation into the logistic function, we can get a personalized logistic regression model P u (y d = 1|x d ) for user u as follows, where w s is the feature weight vector in the global model f s (x).

personalized model adaptation boils down to identifying the optimal model transformation operation A u for each user based on w s and D u .


Different from the model adaptation based personalized sentiment classification solution proposed in (Al Boni et al., 2015), which treats the global model as fixed, MT-LinAdapt adapts the global model to capture the evolving nature of social norms.

in (Al Boni et al., 2015) the individualized model adaptations are independent from each other; but in MT-LinAdapt, the individual learning tasks are coupled together to enable observation sharing across tasks, i.e., multi-task learning.


In Multi-task SVM, only shifting operation is considered in individual users and the global model is simply estimated from the pooled observations across users.

only linear model adaptation is achieved in Multi-task SVM and it cannot leverage prior knowledge conveyed in a predefined sentiment model.


We included an instance-based transfer learning method (Brighton and Mellish, 2002), which considers the k-nearest neighbors of each testing review document from the isolated training set for personalized model training.

for each testing case, we estimated an independent classification model, which is denoted as ReTrain.


To answer this question, learners typically rely on the guidance of domain experts, who can devise pedagogically valuable reading lists that order doc- uments to progress from prerequisite to target concepts.

it is desirable to have a model where each concept is linked to the prerequisite concepts it depends upon -a concept graph.


2 The co-occurrence frequency of instances of c i and c j is relatively higher than that of a nondependency pair.

to predict the concept dependency relation, we need to examine whether the distribution of c i could well approximate the distribution of c j and the joint distribution of c i and c j .


An n-gram appearing only in the third model will have an approximate probability.

the output depends on the order in which users specify models.


Recent years have seen a dramatic growth in the popularity of word embeddings mainly owing to their ability to capture semantic information from massive amounts of textual content.

many tasks in Natural Language Processing have tried to take advantage of the potential of these distributional models.


The word embedding function e is implemented in a usual way as a look-up table V. V is a matrix whose rows can be indexed by words from the vocabulary, that is e(w) = V w , w ∈ V .

each row of V contains embedding of one word from the vocabulary.


The sigmoid outputs a number between 0 and 1 where 0 implies that the information is completely lost and 1 means that the information is completely retained.

the intermediate cell state C t and previous cell state C t−1 are used to update the new cell state C t .


On the other hand, ATTACH action can handle more than two children.

our parser does not require such kind of tree binarization.


For example, the node NP-SBJ 10 shown in Figure 4 satisfies these constraints (the dotted box represents the first constraint).

our parser coindexes NP-SBJ 10 with -NONE-L 14 .


The main research question we address is whether directly optimizing word embeddings for the task of being averaged to produce sentence embeddings leads to word embeddings that are better suited for this task than word2vec does.

we test the embeddings in an unsupervised learning scenario.


In theory, the summation in the denominator of Equation 1 should range over all possible sentences S, which is not feasible in practice.

we replace the set S with the union of the set S + of sentences that occur next to the sentence s i in the training data, and S − , a set of n randomly chosen sentences that are not observed next to the sentence s i in the training data.


This effect might be explained by the fact that this evaluation set primarily consists of very short sentences-it has the lowest average sentence length of all set: 6.63 with a standard deviation of 1.812.

a 300-dimensional representation appears too large for this dataset; this hypothesis is supported by the fact that 200dimensional embeddings work slightly better for this dataset (see Figure 4).


The setting is that we are given a set of inputs (the matrix S), and we want to (adaptively) choose which inputs (rows of S) to obtain the output (corresponding row of T ) for.

proposition 4 states that under the linear system formulation, the set of safe inputs F LS is exactly the same as the row space of S. if we ask for an input that is already in the row space of S, this will not affect F LS at all.


The outputs for x and x are thus α T and β T , respectively.

we are interested in β's such that α T = β T , or in other words, α − β is in the null space of T .


Every filter is shared across all the three different channels, but both the convolution and maxpooling operations are performed individually for each channel.

the total number of feature values generated in the pooling layer is three times the number of filters.


For example, Russian speakers *do exceptions and offers instead of making them, and *find decisions instead of finding solutions, since in Russian do and make have a single translational equivalent (delat'), and so do decision and solution (resheniye).

nonnative speakers who tend to fall back to their L1 translate phrases word-for-word, violating English lexico-semantic conventions.


The more frequently the candidate co-occurs with either of these words, the more plausible it is expected to appear to a learner.

the candidate with the highest cooccurrence frequency is chosen as the distractor.


It is important to note that the Sordoni dataset offers up to 10 references per message, while the Twitter Persona dataset has only 1 reference per message.

bLEU scores cannot be compared across the two Twitter datasets (bLEU scores on 10 references are generally much higher than with 1 reference).


However, the availability of large-scale, wide-coverage corpora still remains a challenge even in the era of big data: parallel corpora are usually only existent for resourcerich languages and restricted to limited domains such as government documents and news articles.

intensive attention has been drawn to exploiting non-parallel corpora for acquiring translation correspondences.


Such a set of phrases can be constructed by collecting either constituents of parsed sentences or strings with hyperlinks on webpages (e.g., Wikipedia).

we assume the two monolingual corpora are readily available and focus on how to extract D from E and F .


The lexicon is built from the translation table by retaining high-probability word pairs.

our approach is capable of learning both new words and new phrases unseen in the seed lexicon.


Our model uses external memory to increase its memory capacity and therefore can store large useful interactions of subsequences.

we can discover some matching patterns with long dependence.


First and foremost, Czech is a Slavic language with not only rich and complex inflection, but also fusional morphology in which a single morpheme can encode multiple grammatical, syntactic, or semantic meanings.

czech possesses an enormously large vocabulary (about 1.5 to 2 times bigger than that of English according to statistics in Table 1) and is a challenging language to translate into.


But they used a different splitting of the training/dev/test data sets.

their results are not directly comparable with ours.


Each latent topic z is described by a multinomial distribution over words P(w i |z), and each word in a document is attributed to a particular latent topic (Blei et al., 2003).

the adaptation vector f w represents a vector of posterior probabilities over latent topics for word sequence w: LDA was found to perform better for RNNLM adaptation than other representations in (Mikolov and Zweig, 2012;Chen et al., 2015).


This makes it more scalable than the KNN approach because the number of comparisons which need to be made scales only with the number of questions, not the size of the training data.

arbitrarily large data sets can be used to train the model without affecting classification time.


Furthermore, the KNN system performs worse on section E than it does on section C, while RNN1 performance is better on section E by 7-10% than on section C. The LSA topic representation consistently yields much better performance than LDA by 25-50% for both systems.

the LDA representation is not further investigated in any experiments.


However, this technique relies heavily on a strong target language model.

one important extension of our work is to further study the interaction between our model and the underlying language model.


Further, Due to constraint (iii) therefore, this subpath of A[k − 1] cannot intersect with sub A[k ] (Q 2 , Q 3 ), above which lies P [k + 1].

a[k ] must cross [k + 2] in some Q 4 , which is lower down than Q 3 .


3For example, nowadays horrendous images are displayed on the cigarette boxes to illustrate the consequences of smoking [P remise] .

(4) statistics show a slight reduction in the number of smokers, indicating that they realize the effects of the negative habit [P remise] ...


Reconsidering the example in Figure 1, without knowing the content "horrendous images are displayed on the cigarette boxes" in sentence 3, one cannot easily tell that "reduction in the number of smokers" in sentence 4 supports the "pictures can influence" claim in sentence 2.

we expect that such content relatedness can be revealed from a discourse analysis, e.g., the appearance of a discourse connective "".


Because a sentence may have more than one argument component, the relative component positions might provide useful information (Peldszus, 2014).

we also include 8 new component position features: whether the source and target components are the whole sentences or the beginning/end components of the sentences; if the source is before or after the target component; and the absolute difference of their positions.


While there are only four semantic class labels at the class-level which may not cover well different aspects of argumentative relation, subtype-level output is not available given the discourse parser we use.

we use relations at type-level as features.


We hypothesize that after a certain limit, larger context windows will produce more noise than helpful information for the prediction.

we set the window-size to 3 in all of our experiments involving Window-context model (all with a separate test set).


It gives positive reward to subgoals fulfilled and gives negative reward for unbound entities, unmet semantic constraints, sentence length, and ambiguous entities.

the best sentence for a given goal is the shortest unambiguous sentence which fulfills the communicative goal and all semantic constraints.


Reusing the search tree is especially effective given that the tree policy causes us to favor areas of the search space with high value.

when we transition to the state with highest value, it is likely that many useful actions have already been explored.


For even the medium world, a sentence with 4 entities would have to consider 1.2 × 10 10 bindings.

the cumulative experiments start with Prune Grammar and Search Bindings turned on.


All experiments still must have Prune Grammar and Search Bindings turned on in order to terminate.

we take this as a baseline to show that the other changes provide additional benefits.


This has been done in other domains in which UCT is used (Computer Go), to achieve a speedup factor of 14.9 using 16 processor threads (Chaslot et al., 2008b).

we believe these optimizations would result in a constant factor speedup.


TA emphasizes documents that are both informative to the classifier and representative of a topic; if a document is not representative, the surrounding context of a topic will be less useful.


Therefore, the factor θ d,k appears in both.

they require that a topic be chosen first and then the document with maximum preference, U , within that topic can be chosen.


Average purity, NMI and RI were highest with LA for the earliest labeling time intervals.

when time is very  limited, using traditional active learning (LA) is preferable to topic overviews; users need time to explore the topics and a subset of documents within them.


(2013) report the performance with both gold-standard argument candidates and predicted argument candidates.

we compare our results with methods based on gold argument candidates in Table 1 and methods based on predicted argument candidates in Table 2.


Note that the third coefficient 1 − λ 1 − λ 2 must be positive, which is the reason why the curve decreases sharply when λ 2 is fixed and λ 1 > 0.65.

figure 3 illustrates that the robustness of our method is very good, which means if the hyperparameters λ 1 , λ 2 are larger or smaller, it will not affect the result very much.


In practice, the argument probability of "Powerful bomb" and "A waiting shed" are much higher than the other two.

"Powerful bomb", "A waiting shed" and "Davao airport" are the final extraction results.


For a trigger, if no pattern can be matched, the event type cannot be assigned and the arguments cannot be correctly identified and classified.

we develop an event type classifier to assign the event type, using both pattern matching information and other features, which gives our system the capability to deal with failed match cases when using patterns alone.


These are the ones that the model is most certain about, suggesting that these are likely to be less noisy.


The blue solid line shows that positive instances are consumed faster than the negative instances with uncertainty sampling.

a higher percentage of positive instances (that approximately equals the number of negative instances getting added) get added and this helps maintain a balanced class distribution.


However, a segmentation is not always given, and a given segmentation might not be optimal for statistical modeling.

this study also presents a model, BiSTM+TS, that incorporates unsupervised topic segmentation into BiSTM.


To generate a document pair d i , the perdocument topic distribution θ i is first drawn from a Dirichlet distribution with the prior α (Step 7).

aligned documents d e i and d f i share the same topic distribution.


In the generative process for a document pair d i , the per-document topic distribution θ i is first drawn in the same way as in BiLDA (Step 7).

in BiSTM, each document pair shares the same topic distribution.


Note that the English articles were collected from the English Wikipedia database dump (2 June 2015) 7 based on inter-language links, even though the original Kyoto Wiki corpus is a parallel corpus, in which each sentence in the Japanese articles is manually translated into English.

our experimental data is not a parallel corpus.


Note that alignments between sections are not given in our experimental data.

y is inferred in both BiSTM and BiSTM+TS.


In addition, Table  5 reveals that BiSTM+TS increases the gap between languages.

segmentation with a comparable granularity between languages might be favorable for the proposed models.


We use the latter approach, obtaining DCS trees by rule-based conversion from universal dependency (UD) trees (McDonald et al., 2013).

nodes in a DCS tree are content words in a UD tree, which are in the form of lemma-POS pairs (Figure 3).


given the sentence "kids play balls", π −1 SUBJ (π ARG (kid)) and π −1 COMP (π ARG (ball)) both contain a playing event whose SUBJ is a kid and COMP is a ball).

addition of query vectors of two paths approximates their intersection because the shared context y gets reinforced.


We use a random walk algorithm to sample paths such that the expected times a path is sampled equals its weight.

the sampled path lengths range from 1 to 19, average 2.1, with an exponential tail.


However, in the RNN architecture, at each time step a word is added and the hidden state is updated recurrently, so those hidden states near the end of the sentence are expected to capture more information 1 .

after adding the attention information to the time sequence hidden representations, the near-the-end hidden variables will be more attended due to their comparatively abundant semantic accumulation, which may result in a biased attentive weight towards the later coming words in RNN.


But the relevant part may be located anywhere in a answer.

when the sample size is large enough 3 , the attention weight should be unformly distributed.


This disconnect between Internet users and the data practices that affect them has led to the assessment that the notice and choice model is ineffective in the status quo (Reidenberg et al., 2015b;Cate, 2010).

an opening exists for language technologies to help "bridge the gap" between privacy policies in their current form and representations that serve the needs of Internet users.


Since the text spans connect data practices to the policy text, we use their overlaps as evidence that two annotators' data practices refer to the same underlying practice in the text.

a score for two data practices that are associated with roughly the same policy text is relatively high, and a score for two data practices that are associated with different text is low.


However, it can predict what the remaining part should look like by comparing words it knows with the information other embedding sets provide about these words.

1TON + learns a model of the dependencies between the individual embedding sets and can use these dependencies to infer what the embedding of an OOV should look like.


Let f (w) be the concatenation of the two distributional and suffix and shape vectors of word w. Then FLORS represents token v i as follows: where ⊕ is vector concatenation.

token v i is tagged based on a 5-word window.


The commentators may tend to use similar, repeated words describing the same type of key events, which may bring additional challenges to traditional summarization methods that are designed to avoid literal repetitions in nature.

naively treating the task as an ordinary document summarization problem can hardly lead to the construction of reasonable sports news reports.


This is intuitive in a geometric sense since determinants are closely related to volumes; in particular, det(L Y ) is proportional to the volume spanned by the vectors q i φ i for i ∈ Y .

item sets with both high-quality and diverse items will have the highest probability (Figure 3).


It is known that MAP inference for DPPs is NPhard (Gillenwater et al., 2012).

we adopt the greedy approximate inference procedure used by Kulesza and Taskar (2011) which is fast and performs reasonably well in practice.


In live texts, 8 Using cosine similarity for all similarity-dependent methods performs poorly in our experiments.

we will not discuss cosine similarity in more details later.


an important event (such as goals) may be stressed multiple times consecutively by the commentator.

in this study we use local literal similarity as a first attempt.


Generally speaking, while big difference in ROUGE may suggest big difference in overall quality, smaller ROUGE differences may not be that indicative enough.

it is interesting to find alternative automatic metrics in order to better reflect the general quality for this task.


In contrast, the CSRS model is trained in a continuous space and does not split training data, which relieves the sparsity problem of lexical features.

the CSRS model achieved better performance us-ing only lexical features compared to the MERS model.


However, the information contained in Rule4 will be considered as context features for Rule1.

this is no longer an advantage for the CSRS model as long as we use rich enough context features, which could be the reason why using both the CSRS and CSRS-MINI models cannot further improve the translation quality compared to using only the CSRS-MINI model.


We only modify the action type in the revising process.

the modified trees are always structurally different instead of only with different dependency labels compared to the original one, which guarantees structured diversity.


By sharing the same classification parameter, bilingual documents with the same sentiment will fall into similar areas in the embedding space.

introducing the logistic regression classifier in our embedding algorithm not only helps to obtain task-specific embedding but also helps to narrow the language barrier.


In our experiments, we use English as the source language and the rest three as target languages.

we have a total of nine tasks with different combinations of three domains and three target languages.


We know that there are additional phrases that indicate causality within sentences but these phrases are neither found in the set of Explicit connectives nor AltLexes.

we expand our definition of AltLex to include these markers when they occur within a sentence.


We also draw on previous work ) that used the left and right sibling of a phrase.

we use the following rules to label new AltLexes: 1.


As previous work (Marcu, 2001) (Biran and McKeown, 2013) used word pairs successfully, we hypothesize that pairs of higher-level categories will improve classification without being penalized as heavily by the sparsity of dealing with individual words.

we include interaction features between every categorical feature for the pre-AltLex text and every feature for the post-AltLex text.


Alternatively, we considered evaluating on the AltLexes in the PDTB but these examples are only limited to inter-sentence cases, whereas the vast majority of our automatically annotated training data was for the intra-sentence case.

we concluded that any evaluation on the PDTB would require additional annotation.


Ultimately, the focus of this work is to improve detection of causal relations.

we did not evaluate some intermediate steps, such as the quality of the automatically annotated corpus.


We estimate the average number of active paths per node pair as T Ne 2 .

the overall time for this method per training iteration is O 2d(η + 1)E kb T We should note that whether this method or the one of Guu et al.


To create a negative example from τ ∈ S, we randomly swapped one of the components of τ with another tuple τ ∈ S. One third of the time we swapped t 1 in τ for t 1 in τ , one third of the time we swapped their R's, and the remaining third of the time we swapped their t 2 's.

distinguishing positive and negative examples in this task is similar to the objectives optimized during training.


On the other hand, gender is also expressed as an inflectional feature via declension or umlaut, especially for adjectives and verbs.

we can see from Figure 4 that the AE also achieves some good results without using context information.


Besides, the case system of the analytic languages has been largely simplified due to historical change.

all the embeddings perform well on analytic languages.


• Quantification of neuron activation pattern reveals different characteristics of the context-based model and the character-based counterpart.

we think that it is necessary to maximize both the utility of word form and the advantage of the context for a better word representation.


Across datasets, we found Pearson correlations in the range 0.45 < r < 0.8 (all p < 0.05), confirming frequent words tend to be used in a greater diversity of contexts.

of this high correlation, we interpret the effect of this measure only after controlling for frequency (this control is naturally captured in equation 7).


Despite this, though, we would not want to make these techniques scalable at the expense of sense distribution quality.

we would like to understand the tradeoff between the accuracy and computation time of these techniques, and optimise this tradeoff.


Compared to HDP, HCA has been shown to be orders of magnitude faster, with similar memory overhead (Buntine and Mishra, 2014).

as long as the quality of the sense distributions given by HCA-WSI are no worse than those from HDP-WSI, it should be worthwhile switching in terms of scalability.


For CNNDM, models must be able to summarize the articles, in order to make sense of the sentence containing the missing word, whereas in LAMBADA the last sentence is not a summary of the broader passage, but a continuation of the same story.

in order to succeed, models must instead understand what is a plausible development of a narrative fragment or a dialogue.


, N ans }, we have: where a i ∈ R dout corresponds to a learned vector associated with answer i.

these models differ primarily in how they combine the property and document to produce the joint representation.


Let h tagger 1 (j) be the hidden layer activation of the tagger network evaluated at token j.

we define the input X implicit by concatenating these tagger activations according to our feature templates: the feature group X implicit is the rowconcatenation of the hidden layer activations of the tagger, as indexed by the feature templates.


Creating such a large dataset with the ground truth annotations is extremely costly; user sensitive demographics e.g., income, age is not available for the majority of social media including Twitter.

we rely our analysis on a large Twitter dataset annotated with demographics and affects using predictive models that can accurately infer user attributes, emotions and sentiments as discussed in Section 3.


It is far easier for a legislator to adapt existing legislative text than to write a bill from scratch.

state legislatures have an increased willingness to adopt legislation drafted by interest groups or by legislators in other states .


This is indeed em- pirically confirmed by the presence of the shortest path between A and Z.

the longer the longest path and the shorter the shortest path are on average, the bigger empirical evidence is given about the transitivity property.


Aliasing would be more likely if there are only a few distinct texts that are provided to participants in our experiment.

we assign each respondent in each evaluation round a distinct candidate biography.


(2015) have defined a fixed set of 222 actions, which is the total number of possible phrases the parser accepts.

the parser-based text game is reduced to a problem that is well suited to a fixed-  Table 1: Statistics for the games "Saving John" and and "Machine of Death".


A source domain contributes partially in terms of zero or more class-labels (clusters) identified using the Algorithm 1.

we refer to the labeled instances from the selected clusters of a source domain as the partial transferable knowledge from that domain.


induced categories) in the auxiliary training set.

we sequentially evaluate all possible one-to-one mappings between the K categories in the auxiliary training set and target domain to report results for the best performing mapping.


(2015) do not explore the four cases and two numbers in the German adjectival system.

our baseline analogy results are useful as a more comprehensive study of the vector offset method for randomly sampled words.


The risk can be further reduced by concentrating the probability mass on y 1 (column 6).

by minimizing the risk on the training data, we expect to obtain a model that correlates well with the gold-standard.


Reducing α further to 1 × 10 −4 , however, results in lower BLEU scores.

we set α = 5 × 10 −3 in the following experiments.


This problem is however a difficult problem on its own and often requires decades of research (see, e.g., Creutz and Lagus (2005) for Finnish and other morphologically rich languages and Huang and Zhao (2007) for Chinese).

many opt to using either a rule-based tokenization approach or a suboptimal, but still available, learning based segmentation algorithm.


When we encounter a previously unseen target context (=state) during decoding, we store the hashes of extracted features in a cache.

for each context, we only run the expensive feature extraction once.


(2013)) is 16 tokens and the longest span has over 100 tokens.

of the strong Markov assumptions that linear-chain CRFs make to ensure tractability, they cannot capture "global" properties of (almost all) quotations and are unable to make joint decisions about the begin point, end point, and content of quotations.


As the upper part of Table 5 shows, a substantial number of B (begin) features look to the left, and a number of E (end) features look to the right.

these features do not look at the quotation itself, but at its immediate external context.


Together with the mode of progression through the text, e.g., temporal or spatial, SE type distribution is a key factor for a reader or listener's intuitive recognition of the discourse mode of a text passage.

the automatic labeling of clauses with their SE type is a prerequisite for automatically identifying a passage's discourse mode, which in turn has promising applications in many areas of NLP, as the mode of a text passage has implications for the linguistic phenomena to be found in the passage.


As this crowdsourcing platform is operated in Japan, signing up for the service and participating require Japanese proficiency.

1 http://crowdsourcing.yahoo.co.jp 2 http://research-lab.yahoo.co.jp/software/index.html Figure 1: User Interface we assumed that the participants were fluent in Japanese.


Wikipedia) are either knowledge or time intensive, and the results have limited coverage.

automatic induction of taxonomies is drawing increasing attention in both NLP and computer vision.


x zn is the hypernymic category of x n .

the problem of inducing a taxonomy structure is equivalent to inferring the conditional distribution p(z|x) over the set of (latent) indices z = {z 1 , .


This is more in line with the type of questions that VQG captures, however, the majority of the questions in Visual7w are designed to be answerable by only the image, making them unnatural for asking a human.

learning to generate the questions in VQA task is not a useful sub-task, as the intersection between VQG and any VQA questions is by definition minimal.


Hence, we can assume that both the scores for individual sentences and the are learned empirically from data using ML.

we have pushed all estimation steps into supervised ML components, which leaves the subset selection step fully principled.


SBL-R seems less promising than ILP-R because it greedily maximizes a function which ILP-R exactly maximizes.

we continue our experiments in the following sec-tions with ILP-R only.


After this, PTB-II applies to the rest of the sentence, which satisfies (P2).

one would obtain the parse "(S (NP She) (VP discussed (PP (IN about) (NP it)))).)."


We use subjects with zero previous knowledge of German, but we translate portions of the sentence into English.

we can presume that they do already know the English words and do not already know the German words (except from seeing them in earlier trials within our experiment).


Even assessing the learner's reading comprehension would give only weak indirect information about what was understood.

we collect data where a learner explicitly guesses a foreign word's translation when seen in the macaronic context.


The system's probability of sampling a given token is proportional to its unigram type probability in the WMT corpus.

rarer words tend to remain in German for longer, allowing the Turker to attempt more guesses for these difficult words.


Let Obs denote the set of indices i for which the user also observes that E i = e * i , the aligned reference translation, because e * i has already been guessed correctly (green feedback) or shown as a clue.

the user's posterior distribution over E is P θ (E = e | E Obs = e * Obs , f , history), where "history" denotes the user's history of past interactions.


We expect high improvement in the N-N category as noun senses are fairly self-contained and often refer to concrete things.

broader (not only local) context is needed to judge their semantic similarity.


We argue that quality is too complex to be represented by a single number and should be instead decomposed into a set of simpler variables that capture the different linguistic and narrative aspects of online news.

we depart from current literature and propose a multidimensional representation of quality.


While the Switchboard corpus gives us a realistic scenario for a setting with unreliable punctuation, the syntactic complexity of telephone conversations is rather low compared to the Wall Street Journal.

as a controlled experiment for assessing how far syntactic competence alone can take us if we stop trusting punctuation and capitalization entirely, we perform joint sentence boundary detection/parsing on a lower-cased, nopunctuation version of the Wall Street Journal.


LAST(S) then denotes the end of the current sentence and is deeper in the buffer than b 0 .

sHIFT is allowed since b 0 belongs to the current sentence.


In the end, "the instructions described the task only through [a handful of] examples of relatedness" (Marelli et al., 2014).

of this hands-off annotation guideline, the SICK dataset contains glaring inconsistencies in semantic relatedness scores, even for sentence pairs where the only difference between two sentences is due to the same known transformation.


The English UD data has 116 tags composed out of 51 distinct attribute-value pairs.

we can project a reasonable number of morpho-syntactic attributes from English, although the number of attribute combinations that occur in the data is less than for morphologically richer languages.


In a source autoencoder, the source-to-target model P (y|x; − → θ ) serves as an encoder to transform the observed source sentence x into a latent target sentence y (highlighted in grey), from which the target-to-source model P (x |y; ← − θ ) reconstructs a copy of the observed source sentence x from the latent target sentence.

monolingual corpora can be combined with parallel corpora to train bidirectional NMT models in a semi-supervised setting.


As P (y|x; θ) is modeled by a single, large neural network, there does not exist a separate target language model P (y; θ) in NMT.

parallel corpora have been the only resource for parameter estimation in most existing NMT systems.


Unfortunately, even for a handful of resource-rich languages, the available domains are unbalanced and restricted to government documents and news reports.

the availability of large-scale, high-quality, and wide-coverage parallel corpora becomes a major obstacle for NMT.


As shown in Figure 3, similar findings are also observed on the English-to-Chinese validation set.

we set k = 10 in the following experiments.


We also tried to use both Chinese and English monolingual corpora through simply setting all the λ to 0.1 but failed to obtain further significant improvements.

our findings can be summarized as follows: 1.


For the matrix sizes involved (> 500×1, 000), the time complexity of matrix product is linear in each dimension, both on CPU (Intel MKL 5 ) and GPU (CuBLAS), with a 10X speedup for GPU (Nvidia K40) compared to CPU (Intel Xeon E5-2680).

the speed trade-off applies to both CPU and GPU hardware, albeit with a different time scale.


Although quality bitexts have been made available over the years (Tiedemann, 2012), creating parallel corpora is a resource-consuming effort involving professional human translation of large volumes of texts in multiple languages.

there is still a lack of parallel data to properly model translation across languages and domains.


In order to establish a fair comparison between LEXACC and STACC, all shared settings were identical.

lexical translations were based on the same previously described GIZA tables extracted from the JRC corpus, and STACC alignment was performed on the same sets of candidates retrieved from the Lucene searches by LEXACC for each language pair.


Similarly to , they find that SemCor may not be a sufficient resource for supersense tagging adaption to different domains.

in our work, we explore the potential of using an automatically annotated Babelfied Wikipedia corpus (Scozzafava et al., 2015) for this task.


To learn our embeddings, we adapt the freely available sample of 500k articles of Babelfied English Wikipedia (Scozzafava et al., 2015  ..

the quality of these NLP processors is considerably lower than the results of the evaluation in-domain."


Furthermore, note that while we report the performance of our embeddings on the word similarity tasks for completeness, there has been a substantial discussion on seeking alternative ways to quantify embedding quality with the focus on their purpose in downstream applications Faruqui et al., 2016).

in the remainder of this paper we explore the usefulness of supersense embeddings in text classification tasks.


to EM training) have to be implemented and optimized from scratch for each new grammar formalism.

development time is spent on reinventing wheels that are slightly different from previous ones, and the resulting implementations still tend to underperform.


Furthermore, an evaluation with realistic grammars shows that our algorithms improve practical parsing times with IRTG grammars encoding context-free grammars, tree-adjoining grammars, and graph grammars by orders of magnitude (Section 6).

our algorithms make IRTG parsing practically feasible for the first time; for graph parsing, we obtain the fastest reported runtimes.


It also has a single binary operation symbol * , such that * is the concatenation of the strings w 1 and w 2 .

the term * (John, * (walks, * (on, Mars))) in Fig.


, X n ) of M such that X is the i-th child state, and for every j = i, X j must be a state for which we previously called BU(S, j, X j ).

a sibling-finder performs a bottom-up rule lookup, changing its state after each call by caching the state and position.


Because the decomposition automaton represents all the ways in which the input object can be built recursively out of smaller structures, including ones which will later be rejected by the grammar, such automata can be very large in practice.

we would like to work with a lazy representation of M R and avoid iterating over all rules.


As typically in mean-field approximations, inference of Q(x) and Q(y) can't be done efficiently with this exact objective L, because of the nonlinear interdependence between x k and y k in the last term.

we introduce two approximations to L, one for use in inferring Q(x) given Q(y) (forward inference), and one for the reverse inference problem (backward inference).


In the third alternative, the constant 1 was chosen to optimise this match, producing a close match to the Word2Vec training gradient, as shown in Figure 1 (Word2Vec versus Unk dup).

word2Vec can be seen as a good approximation to the third model, and a progressively worse approximation to the second and first models.


The connections are the latent states of two adjacent utterances.

hSSM can be considered as a special Boltzmann machine.


From the perspective of generative model, we can also consider that words in a dialogue are generated by the mixture model of latent states, a global/background topic, and a dialogue specific topic.

there are three kinds of units in the hidden layer of our proposed model, which are displayed in Figure 1. h φ is a softmax unit, which indicates the latent state for a utterance.


A dialogue can be abstractly viewed as a sequence of latent states in a certain reasonable order.

formulating the dependency between latent states is another import issue for dialogue structure analysis.


For the sake of simplicity, assume that the order of words in an utterance is ignored.

all of these softmax units can share the same set of weights that connect them to hidden units, thus the visible bias term C(V) and the sub-energy func- (1) can be redefined as follows: wherev rk = Dr i=1 v rik denotes the count for the k th word in the r th utterance of the dialogue,v k = R r=1v rk is the count for the k th word in whole dialogue session.


Also, code snippets in responses that were not accepted were frequently incorrect or tangential to the question asked.

we extracted only the title from the post and use the code snippet from those accepted answers that contain exactly one code snippet (using <code> tags).


We found this model to generate overly short titles like 'sql server 2008' when a length restriction was not imposed on the output text.

we fix the output length to be the average title length in the training set while decoding.


While it is possible to create video-recordings of a human performing American Sign Language (ASL) for use on websites, updating such material is expensive (i.e., re-recording).

researchers investigate technology to automate the synthesis of animations of a signing virtual human, to make it more cost-effective for organizations to provide sign language content online that is easily updated and maintained.


Given that the centroid and the training data for the latent trace are driven by recordings of a (female) signer and the gold standard is a different (male) signer, there are differences between these facial expressions due to idiosyncratic aspects of individual signers.

the metric evaluation in this section is challenging because it is an inter-signer evaluation.


The period of time when the facial expression occurred was time-aligned with the subset of words (the sequence of signs performed on the hands) for the appropriate syntactic domain; the phrase-beginning and phraseending was aligned with the performance of the facial expression.

the difference in appearance between our animation stimuli was subtle: The only portion of the animations that differed between the three conditions (neutral, centroid, and latent-trace) was the face and the head movements during the span of time when the syntactic facial expression should occur (e.g., during the Wh-question).


Note that the number of edges in a complete directed graph is twice the number in its undirected counterpart.

on average, the number of input edges in the pruned undirected models amounts to 73.98% 2 = 36.49% of the number of edges in the complete directed graphs.


2015, we extract all root-to-leaf paths on conversation trees and utilize the state-of-the-art sequence learning model CRF (Lafferty et al., 2001) to detect the leaders 3 .

the posterior probability of each node being a leader or follower is obtained by averaging the different marginal probabilities of the same node over all the tree paths that contain the node.


We can simplify our case by assuming that followers are topically responsive just up to (hence not further than) their nearest ancestor leaders.

we can dismantle each conversation tree into forest by removing the links between leaders and their parents hence producing a set of subgraphs like [R2]-[R6] and [R7]-[R9] in Figure 1.


Because the content of posts are often incomplete and informal, it is difficult to manually annotate topics in a large scale.


• The topical words generated by SATM were mixed with words relevant to the bus explosion in Guangzhou, since it aggregated messages according to topic affinities based on the topics learned in the previous step.

the posts about bus explosion and MH17 crash, both pertaining to disasters, were aggregated together mistakenly, which generated spurious topic results.


To enrich KBs, many efforts have been invested in automatically finding unknown relational facts.

relation extraction (RE), the process of generating relational data from plain text, is a crucial task in NLP.


KBs due to the lack of human-annotated training data.

(Zeng et al., 2015) incorporates multi-instance learning with neural network model, which can build relation extractor based on distant supervision data.


For the first issue, we investigate whether a frame could be mapped to an event-type based on events expressed by exemplars annotated for that frame.

the key is to detect events from the given exemplar sentences in FN.


We notice that ACE corpus only contains about 6,000 labeled instances, while FN contains more than 150,000 exemplars.

a straightforward solution to alleviate the data sparseness problem is to expand the ACE training data by using events detected from FN.


All exemplars under dead.a, deceased.a and lifeless.a express Die events.

this hypothesis amends the flaws of the former two hypotheses.


In details, we firstly detect events in FN under different configurations of formulas' weights and add them to ACE training dataset, respectively.

we obtain several different expanded training datasets.


That is, if a frame is mismapped, then all sentences under it would be mislabeled as events.

even a single mismapped frame could significantly hurt the performance.


To prepare for automatic evaluations, we respectively add the events detected from FN by each of the aforementioned five systems to ACE training corpus.

we obtain five ex-   Table 6: Automatic evaluations of events from FN.


For formulas that are used for other categories of word math problems (algebraic or arithmetic), such as the unitary method, formulas for ratio, percentage, time-distance and rate of interest, none of them have any slot that accepts sets of variables.

further increase in the space of possible applications will be polynomial.


However, as mentioned in section 3.3, when active, the value of a feature is the number of variables associated with the application.

p(y 2 ;P,θ) p(y 1 ;P,θ) = e 4 e 3 = e. y 2 is more probable than y 1 .


If this nearest opinion word appears in the stance word's immediate or embedded clause, he creates a binary feature by concatenating all the words in the dependency path, POS generalizing all words other than the stance and opinion word, and finally prepending "not" if the stance word is adjacent to a negator in the dependency tree.

given the sentence "I can only say that this statement is completely true."


We therefore abstract a stance in an essay as a dependency path from a subject to an object that passes through the governor.

the stancetaking dependency path we identify from the prompt part shown in Figure 1 could be represented as moneyroot-evil.


Any word like "no", "not", or "none" counts as a negation unless it begins a non-negation phrase like "no doubt" or "not only".

6 the count of negations in the prompt part in Figure 1 is 0.


The total negation count we are looking for is the number of nodes in this list that correspond to negation words (unless the negation word begins a negation phrase).

because the word "not" is the child of "root" in the path "itself-root-of-evil" we identified in Figure 2, we consider this path to have been negated one time.


If both the prompt part and the stancetaking text are negated, both disagree with the opposite of the prompt part's stance.

they agree with each other, and their negation sum is even (2).


7 If we find a sentence that both begins with a neutral phrase and is about the prompt part's subject, we turn the Neutral feature on.

sentences like the following can be captured: "In all probability university students wonder whether or not they spend their time uselessly in studying through four or five years in order to take their degree."


We then trim the list of paths by removing any path whose governor or subject does not have a hedge word as an adverb modifier child in the dependency tree.

8 we are able to determine that the essay containing the sentence "There is nearly no place left for dream and imagination" is likely to belong to one of the Somewhat classes w.r.t.


The proportion of trivial cases is very high, such as '的 ('s)'，'我们 (we)', which results in that the non-trivial cases are relatively despised.

a good measure should have a capability to balance the skewed distribution by weighting the test cases.


Given a committee of K base segmenters, we can get K segmentations for sentence We use a mark m k i ∈ {0, 1} to indicate whether word w i is correctly segmented by the k-th segmenter.

the number of words c k correctly identified by the k-th segmenter is we can calculate the degree of difficulty of each word w i .


Balanced F-score The final balanced F-score is It is infeasible to manually judge the difficulty of each word in a dataset.

an empirical method is needed to rate each word.


These structure usually form a different pattern that is hard for the machine algorithm to understand.

it is reasonable to hypothesize that the an idiom phrase is more likely to be a difficult word for word segmentation task.


Besides, P1 P2 P3 P4 P5 P6 P7 if we compare P1 and P2, we will notice that P2 performs just slightly worse than P1 on easy cases, but much better than P1 on difficult cases.

conventional evaluation metric rank P1 as the top system because the P1 gets a lot of credits from a large portion of easy cases.


In case the important temporal expression, that defines when the event occurred, is more than one sentence away, the TLINK will not be annotated.

retrieving the information when the event occurred is not possible.


We now consider examples of errors that are corrected by the classifier-based CUUI system in these three categories but are missed by the MTbased AMU system (Table 7).

examples (1) and Long-distance dependencies: verb agreement (1) , in the case that when one of the members *happen/happens to feel uncomfortable or abnormal , he or she should be aware that .


Native data, in contrast, is cheap and available in large quantities.

the fact that, unlike with MT, it is possible to build a classifier system without any annotated data, is a clear advantage of classifiers.


All of the non-pivot features are projected into the pivot feature space in the form of the new correspondence features, and the pivot feature space is, by design, the most domain independent part of the feature space.

it seems reasonable to completely replace the nonpivot features with the new pivot-like features.


However, most of these studies require either manual analyses or measurements of human characteristics such as brain activities or reading times for each example.

while these analyses are reliable for the examples they focus on, they cannot be generalized to other examples.


(99.6%) woman-ACC date-DAT ask (φ I ask a woman out on a date.)

in this paper, we assume that there is a relationship between the canonical word order and the proportion of each word order in a large corpus and attempt to evaluate several claims on the canonical word order of Japanese double object constructions on the basis of a large corpus.


On the other hand, Claims B and C argue that the canonical word order varies depending on verb, but they still do not take into account the lexical information of the arguments.

these claims can be verified by investigating the distribution of word orders for each verb.


For example, since Japanese is a head-final language and only the verb bunsetsu can be the head of the most noun bunsetsu in Japanese, basho-ni in Example (8) has only one head candidate oita (put), whereas kagi-wo and kare-ni have two head candidates iwareta (told) and oita (put).

we extract only the example "basho-ni oita" from Example (8).


Unfortunately, however, the cosine distance is not strictly metric but semimetric since the triangle inequality is not satisfied.

we cannot directly use the cosine distance because the triangle inequality is a key element for efficient indexing in a metric space.


Additionally, we force every pair to have at least five judgments from the workers.


The word2vec implementation does not write context (output) vectors but only word (input) vectors to a model file.

we modified the source code to save context vectors, and use them in Equation 5.


These studies did not consider semantic composition of relational patterns.

they might suffer from the data sparseness problem, as shown by No-Comp in Figure 4.


), each of which involves different assumptions about and models of various linguistic phenomena (e.g., vocabulary, syntax, and semantics).

state-of-theart systems yield varying performance across subproblems (Mitkov, 2014;Kummerfeld and Klein, 2013;Björkelund and Nugues, 2011;Recasens and Hovy, 2009;Stoyanov et al., 2009;Bengtson and Roth, 2008;Van Deemter and Kibble, 2000;Ng and Cardie, 2002b;Kameyama, 1997).


This model is embedded in an agglomerative clustering process, after which a coreference decision is made between clusters and the given m-anaphor.

our model treats the grouping of candidate mentions into sibling sets independently from antecedent-anaphor linking.


Moreover, many sentences are not suitable for summarization because of their low relevance with the topic.

we filter out the large number of unrelated sentences and treat the remaining sentences as candidates for summary extraction.


We can see that the word labels are very confusing, and the phrase labels for the two topics are totally overlapping with each other and have no discrimination.

it is hard to understand the two topics by looking at the word or phrase labels.


The feedback from the human judges also reveals the above problem and all the three human judges said they need to take more than five times longer to read the summaries.

we want to find a better way to make use of the summary label in practice.


Instead, in our proposed model, we leverage a Bayesian non-parametric infinite mixture model to handle multiple relation semantics by generating multiple translation components for a relation.

different semantics are characterized by different components in our em-bedding model.


Locally optimized models could not exploit these implicit bi-directional preferences.

we use a joint model to find a globally optimal entity-relation assignment from local predictions.


We use a recurrent neural network (RNN) over characters to capture the meaning of the clause.

we obtain a single vector which represents the antecedent clause.


The gradient for mean cosine error is makes sense with nonnegative weights.

to minimize the loss we use the limitedmemory Broyden-Fletcher-Goldfarb-Shanno algorithm for box constraints (L-BFGS-B) (Byrd et al., 1995), a quasi-Newton method that allows bounded optimization.


Moreover, previous locallevel analyses suggest that systematicity seems to be concentrated in word-beginnings and word-endings.

it may be worthwhile to augment the representation of edit distance in our model by making it context-sensitive.


While semantic taxonomies, like WordNet (Fellbaum, 1998), define hypernymy relations between word types, they are limited in scope and domain.

automated methods have been developed to determine, for a given term-pair (x, y), whether y is an hypernym of x, based on their occurrences in a large corpus.


Applying the vanilla settings of SLQS on our dataset, that contains also rare terms, resulted in low performance.

we received assistance from Enrico Santus, who kindly provided the results of SLQS on our dataset after tuning the system as follows.


We assume that there is a one-to-one mapping between decision sequences d 1:j−1 and states s j : that is, we essentially assume that a state encodes the entire history of decisions.

each state can be reached by a unique decision sequence from s † .


Even algorithms that are considered relatively scalable (e.g., the Lanczos algorithm (Cullum and Willoughby, 2002) for computing eigenvalue decomposition of large sparse matrices) fall apart in this scenario, since the data cannot be stored in the memory of a single machine.


By comparison, in previous work, both Unary-X and Reduce-L/R-X actions produce new labeled nodes (some of which are auxiliary nodes due to binarization).

our paradigm has two advantages: • it dramatically reduces the number of possible actions, from 3X + 1 or more in previous work to 3 + X, where X is the number of nonterminal labels, which we argue would simplify learning; • it does not require binarization (Zhu et al., 2013;Wang and Xue, 2014) or compression of unary chains (Mi and Huang, 2015) There is, however, a more closely-related "shiftproject-attach" paradigm by Henderson (2003).


At the end, the phrase table is re-estimated using the phrase pair segmentations obtained from forced decoding.

forced alignment is a reestimation technique where translation probabilities are calculated based on their frequency in best-scoring hypotheses instead of the frequencies of all possible phrase pairs in the bitext.


Our approach and results are partly independent of this representation, however, in that we take a structuralist approach: we derive new, 'second-order embeddings' by modeling the meaning of words by means of their semantic similarity relations to all other words in the vocabulary (de Saussure, 1916;Rieger, 2003).

future research may in principle substitute the deep-learning architectures for semantics considered here by any other method capable of producing semantic similarity values between lexical units.


In   The values D t 0 as a function of t 0 are averages over all words.

it might be possible that the average word's meaning decays linearly in time, while the semantic behavior, over time, of a large fraction of words follows different trends.


Because table relabeling does not merge tables in the CRP hierarchy, even if it replaces an uncommon word with a more common one, the configurational probability does not change.

the model's sparsity preference cannot encourage such moves.


Depending on a broad textual and real-world context, the translation can differ from the source text at any linguistic level -lexical, syntactic, semantic or even discourse -and still be considered perfectly correct.

using a single translation as a proxy for the original text may be unreliable.


In this paper, we apply two kinds of similarity metrics.

there are two assembling tasks, i.e., (1)assembling knowledge based and corpus based similarities; and (2)assembling words to texts.


We apply threshold algorithm to obtain the top short texts based on each query word.

the top-1 result comes from these two ranked lists based on threshold algorithm.


The maximum depth of T P is D P (or D S of T S ).

p is more similar to Q compared with S. we have The lemma tells us that the similarity ordering between candidates in WordNet depends on the integration of the shortest path and the maximum depth of the taxonomy.


After that, we build a weighted inverted list where each list presents a word with sorted corresponding short texts according to the similarity score.

for a given query text q, each word in q corresponds to a list of short texts.


These results are highly significant with p < 10 −3 according to the Mann-Whitney U test.

even though our method is simple and straightforward to apply, it can still beat all attempts at more complicated structural modifications to the TransE model on this dataset.


This is alleviated by filtering, but is not completely eliminated due to the sparsity of relations in the dataset (which is the reason we require the link prediction task).

we believe the mean rank is a more accurate measure of the performance of a model, particularly on raw data.


For each sentence or batch, we only predict the target words in its sentencelevel or batch-level vocabulary.

we reduce both the computing time and the memory usage.


(2015) alleviated the large vocabulary issue by proposing an approach that partitions the training corpus and defines a subset of the full target vocabulary for each partition.

they only use a subset vocabulary for each partition in the training procedure without increasing computational complexity.


Another motivation of this work is to combine the merits of both traditional SMT and NMT, since training an NMT system usually takes several weeks, while the word alignment and rule extraction for SMT are much faster (can be done in one day).

for each training sentence, we build a separate target vocabulary which is the union of following three parts: • target vocabularies of word and phrase translations that can be applied to the current sentence.


As the word-to-word translation dictionary only focuses on the source words, it can not cover the target unaligned functional or content words, where the traditional phrases are designed for this purpose.

in addition to the word dictionary, given a word aligned training corpus, we also extract phrases P where x 1 ...x i is a consecutive source words, and [y 1 , ..., y j ] is a list of target words 1 .


After tree-based convolution, we obtain a set of feature maps, which are one-one corresponding to original words in the sentence.


The CRF in this version is a log-bilinear CRF, where it treats the embeddings and hidden vectors inputs as variables and modifies them according to the objective function.

it enables propagating the gradients back into the LSTM to adjust the parameters.


As a result, it enables propagating the gradients back into the LSTM to adjust the parameters.

the word segmentation and NER training share all the parameters of the LSTM module.


Joint Training In our integrated model, the LSTM parameters are used for both predicting word segmentations and NER.


where it is crucial to note that there is no E 0 variable, standing for the NULL word, anymore.

jumps to a NULL position do not need to be modelled.


Besides hesitations, we treated words and phrases referring to some kind of uncertainty together with indefinite pronouns as fillers such as ilyen "such", olyan "such", izé "thing, gadget",és aztán "and then", valamilyen "some kind of", valahogy "somehow", valamerre "somewhere" 2 .

mCI patients often seem to substitute content words with fillers or indefinite pronouns, moreover, they also appear to use lots of paraphrases, which also indicate uncertainty just like egy ilyen bagolyszerűség a such owl-likeness "something similar to an owl" or az olyan délelőtt volt that such morning was "that happened some time in the morning".


As MCI is strongly related to memory deficit, we believe that the order of the tasks might also influence performance, hence we opted for processing each transcript separately.

for each person, features to be discussed below were calculated separately for the three transcripts and all of them were exploited in the system.


There is some evidence that abstract concepts are also perceptually grounded (Lakoff and Johnson, 1999), albeit in a more complex way, since abstract concepts will relate more varied situations (Barsalou and Wiemer-Hastings, 2005).

uni-modal visual representations are not powerful enough to capture all the semantic intricacies of such abstract concepts, and the linguistic components are more beneficial in such cases.


This might be because the general writing quality of the comments in CMV is high because of the policy of the forum.

the surface text features we used are not very discriminative for comment ranking.


Unlike in our setup, in theirs, images for the same concept are used both for training and to derive visual attributes (our setup is "zero-shot" at the concept level, i.e., we predict attributes of concepts not seen in training).

despite the fact that they used presumably less accurate pre-CNN visual features, the setup is much easier for them, and we take their performance to be an upper bound on ours.