For instance, the extraction "Germany's DAX index was down 0.2 percent" is undesirable for our task because it refers to a generic event and this event may occur in any year.

it is not able to indicate a certain timestamp and thus propagations from a generic event node are very likely to result in propagation errors.


It was adopted as a trade-off between completeness, and the need to simplify the evaluation process (Verhagen et al., 2009).

one deficiency is that it does not allow us to construct the complete temporal graph we seek.


While we can determine the relationship between A and C in the figure with the use of temporal transitivity rules (Setzer et al., 2003;Verhagen, 2005), we cannot reliably determine the relationship between say A and D. In this work, we seek to overcome this limitation, and study what can enable effective article-wide E-E temporal classification.

we want to be able to determine the temporal relationship between two events located anywhere within an article.


We leave for future work an investigation into whether more fine-grained topic segments can lead to further performance gains.

it will be interesting to study if work on argumentative zoning (Teufel and Kan, 2011) can be applied to newswire articles, and whether the subsequent learnt document structures can be used to delineate topic segments more accurately.


This integrated model allows us to examine how solving the word segmentation problem should affect infants' strategies for learning about phonetic variability and how phonetic learning can allow word segmentation to proceed in ways that mimic the idealized input used in previous models.

although the GGJ model achieves high segmentation accuracy on phonemic (nonvariable) input and makes errors that are qualitatively similar to human learners (tending to undersegment the input), its accuracy drops considerably on phonetically noisy data and it tends to oversegment rather than undersegment.


We design our beam sampler to restrict the set of potential intended characters at each timestep.

given a stream of input characters S = s 1 .


However, experiments on data with variation have called these conclusions into question.

ggJ has previously been shown to oversegment rather than undersegment as the input grows noisier (Fleck, 2008), and our results replicate this finding (oversegmentation for the "segment only" model).


SMC algorithms maintain a set of weighted hypotheses; the weights correspond to probabilities, and in our case, the hypotheses correspond to target language word sequences.

we approximate the conditional probability, where ω k n is the normalized weight of sample k at word n (ω k n is the unnormalized weight), and δ t k 1:n is a delta function centered at t k 1:n .


From the normalization corpus, we learn a normalization model that generalizes the normalization process.

from the data we observe that To DanielVeuleman yea iknw imma work on that is normalized to To Daniel Veuleman: yes, I know.


The first ( §4.1) learns word-word and phrase-phrase mappings.

we wish to find that DanielVeuleman is normalized to Daniel Veuleman, that iknw is normalized to I know and that imma is normalized to I am going.


The advantage of the phrase-based model is that it can make decisions for normalization based on context.

it contains phrasal units, such as, go 4, that determine, when the word 4 should be normalized to the preposition for and when to leave it as a number.


The experimental results showed that the correlation between these two variables were strongly positive.

the correlation coefficient on SO/CPP was 0.8129 and on SO/Math was 0.7412.


Specifically, the correlation coefficient on SO/CPP was 0.8129 and on SO/Math was 0.7412.

when the distance between the difficulty scores of two buckets become larger, the two word distributions in the two buckets become less similar, and vice versa.


Lags correspond to the lengths of sequences of noncue words, which are treated as irrelevant to the inference problem at hand.

a speech is represented as a series alternating between cues signaling ideological positions and uninteresting filler.


Because our interest is in measuring proportions-and, as we will see, in comparing those proportions across speakers and campaign periods-we require a way to allow variation in parameters across different conditions.

we seek to measure differences in time spent in each ideology state.


Stanzas often consist of 2, 3, or 4 lines, but stanzas of more lines are also common.

in hip hop, a single verse often contains many stanzas with different rhyme schemes and meters.


Because an article's prior over topic distributions depends on the topic assignments of the articles that it cites, TIR induces a network of dependencies between the topic assignments of the documents.

if we collapse out Θ, the dependencies between the z's of each document form a Bayesian network whose graph is the citation graph.


We would like to thank the researchers in the group for the helpful discussions and assistance on different aspects of the problem.

we would like to thank Chris Brew, Ron Kaplan, Deepak Ramachandran and Adwait Ratnaparkhi.


This word representation with NLM has been used to good effect, for example in (Turian et al., 2010;Collobert et al., 2011;Huang et al., 2012) where induced word representations are used with sophisticated features to improve performance in various NLP tasks.

we first represent the word sequence as a vector where w i is i th word in the sequence, m is the window size, d(w) is the vector representation of word w (an n-dimensional column vector) and is the concatenation of word vectors as an input of neural network.


Hence, when we observe a relation R(w 0 , w), the projected representation of word w in this context is computed by prpj (R,w 0 ) (w) defined as follows:  In order to model co-compositionality, we apply prototype projection to both the verb and the object.

suppose verb is w v and object is w o , C (VerbOf,wo) is used to project w v and C (ObjOf,wv) is used to project w o .


In this study, we use compositional distributional semantic methods to investigate restrictions in adjective ordering.

we focus on properties distinguishing Adjective-Adjective-Noun phrases in which there is flexibility in the adjective ordering from those bound to a rigid order.


8 The results imply that the degree of modification of the A y on the noun is a significant indicator of the type of ordering present.

rigid order A x A y Ns are heavily modified by A y , distorting the meaning of the head noun in the direction of the closest adjective quite drastically, and only undergoing a slight modification when the A x is added.


In particular, rigid order A x A y Ns are heavily modified by A y , distorting the meaning of the head noun in the direction of the closest adjective quite drastically, and only undergoing a slight modification when the A x is added.

in rigid order phrases, for example rapid social change, the A y N expresses a single concept (probably a "kind", in the terminology of formal semantics), strongly related to social, social change, which is then modified by the A x .


Next, we find that for W.ADD, F.ADD and LFM, the distance from the component A x N is a strong indicator of attested-vs. unattested-order rigid order AANs.

attested-order AANs are further from their A x N than unattested-order AANs.


In this paper, we develop a novel representation learning approach to address domain adaptation for text classification with automatically induced discriminative latent features, which are generalizable across domains while informative to the prediction task.

we propose a hierarchical multinomial Naive Bayes model with latent variables to conduct supervised word clustering on labeled documents from both source and target domains, and then use the produced cluster distribution of each word as its latent feature representation for domain adaptation.


In this work, we present a novel supervised representation learning approach to discover a latent representation of words which is not only generalizable across domains but also informative to the classification task.

we propose a hier-archical multinomial Naive Bayes model with latent word cluster variables to perform supervised word clustering on labeled documents from both domains.


By filtering out unimportant details and noises, we expect the low dimensional mapping can capture the intrinsic structure of the input data that is discriminative for the classification task and generalizable across domains.

we learn such a mapping function by conducting supervised word clustering on the labeled documents using a hierarchical multinomial Naive Bayes model.


We then define a transformation matrix Π ∈ R n×m based on the mapping function φ defined in Eq.

2, such that Π i: = φ(w i ) where w i is the i-th word in the vocabulary V. each row of Π is the induced representation vector for one word.


One possible reason is that the FDLDA model is not specialized for supervised word clustering, and it uses a logistic regression model to predict the labels from the word topics, while the final soft word clustering is computed from the learned distribution p(z) and p(w|z).

in the FDLDA model the labels only influence the word clusterings indirectly and hence its influence can be much smaller than the influence of labels as direct parent variables of the word cluster variables in the SWC model.


4 shows that the prior for π l i,κ ,n i ,p is seeded by pseudo-counts that are proportional to the mapping weights over the phonetic units of l i in a shorter context.

the mapping distribution of l i in a shorter context can be thought of as a back-off distribution of l i 's mapping weights in a longer context.


Although the PMM framework was designed to incorporate and augment expert lexicons, we found that it can be adapted to polish the pronunciation list generated by our model.

the training procedure for PMMs includes three steps.


However, such language-specific knowledge is not available for our training framework; therefore, our strategy is to compile a question list that treats each phonetic unit as a unique linguistic class.

our approach to training a contextdependent acoustic model for the automatically discovered units is to let the decision trees grow fully based on acoustic evidence.


The main difference is that the mixture weights for letters of languages with simple pronunciation rules will be sparser and spikier.

in theory, our model should always perform comparable to, if not better than, grapheme recognizers.


Finally, workers were asked to design their questions and answers such that at least two of the four questions required multiple sentences from the story to answer them.

for those questions it should not be possible to find the answer in any individual sentence.


Models based on so-called back-off smoothing have shown good predictive power (Goodman, 2001).

kneser-Ney (kN) and its variants (kneser and Ney, 1995) are still achieving state-of-the-art results for more than a decade after they were originally proposed.


We thus have a trie structure S on W (see Figure 1(a)) con-straining the number of free parameters.

there is one weight parameter per node in the trie S and the matrix of parameters W is of size |S|.


The former typically leads to a solution that is sparse and easily interpretable, while the latter leads to a non-sparse, generally more stable one.

the squared 2 and 1 penalties were used in the context of log-linear language models (Chen and Rosenfeld, 2000;Goodman, 2004), reporting performances competitive with bi-gram and tri-gram interpolated Kneser-Ney smoothing.


This way, the human translator can ensure a high-quality output while the MT system ensures a significant gain of productivity.

interesting is the interactive machine translation (IMT) approach proposed in (Barrachina et al., 2009).


The second problem is how the system deals with word reordering.

the models used by the system were either mono-tonic by nature or non-monotonic but heuristically defined (not estimated from training data).


Here, we propose a simpler approach which can be seen as a particular case of the previous one.

the proposed approach models the edit distance as a Bernoulli process where each character of the candidate string has a probability p e of being erroneous.


Finally, we compute the error-correction probability between two strings from the total number of edits required to transform the candidate translation into the reference translation.

we define the error-correction distribution in Equation 11as: where k = Lev(p, t a ) is the character-level Levenshtein distance between the user defined prefix p and the prefix t a of the hypothesized translation t defined by alignment a.


Here, reordering is defined at rule level by the one-to-one non-terminal correspondence.

reordering is a property inherent to each rule and it is the individual score of each rule what defines, at each step of the derivation, the importance of reordering.


Given an hypergraph, certain hypernodes define a possible solution to the maximization defined in the two IMT formulations.

only those hypernodes that generate a prefix of a potential translation are to be taken into account 3 .


The EU corpora were extracted from the Bulletin of the European Union, which exists in all official languages of the European Union and is publicly available on the Internet.

the chosen Es-En corpus was part of the evaluation of the TransType2 project (Barrachina et al., 2009).


The Zh-En corpus used in the experiments was part of the MT track in the 2011 evaluation campaign of the workshop on spoken language translation (Federico et al., 2011).

we used the dev2010 partition for development and the test2010 partition for test.


Finally, we compared the estimated human effort required to translate the test partitions of the EU and TED corpora with the best IMT configuration (independent suffix formalization with hierarchical translation model) and a conventional post-editing (PE) CAT system with word-completion.

when the user corrects a character, the PE system automatically proposes a different word that begins with the given word prefix but, obviously, the rest of the sentence is not changed.


• More sophisticated measures to estimate the human effort.

measures that estimate the cognitive load involve in reading, understanding and detecting an error in a translation (Foster et al., 2002), in contrast KSMR simply considers a constant cost.


A feature is a local feature if and only if it can be factored among the translation steps in a derivation.

the value of a local feature for ⟨s, t, d⟩ can be calculated as a sum of local scores in each translation step, and the calculation of each local score only requires to look at the rule used in corresponding step and the input sentence.


This paper focuses on exploiting zero pronouns to improve Chinese coreference resolution.

a simplified semantic role labeling framework is proposed to identify clauses and to detect zero pronouns effectively, and two effective methods (refining syntactic parser and refining learning example generation) are employed to exploit zero pronouns for Chinese coreference resolution.


In this paper, we focus on exploiting one of the key characteristics of Chinese text, zero pronouns (ZPs), to improve Chinese coreference resolution.

a simplified semantic role labeling (SRL) framework is proposed to identify Chinese clauses and to detect zero pronouns effectively, and two effective methods are employed to exploit zero pronouns for Chinese coreference resolution.


In Chinese, a zero pronoun always occurs just before a predicate phrase node (e.g., VP).

if the predicate phrase node occurs in a coordinate structure or is modified by an adverbial node, we only need to consider its parent.


Table 7: Performance of zero pronoun detection on the test set using gold and automatic parse trees find that the performance of our zero pronoun detector drops about 12% in F-measure when using automatic parse trees, compared to using gold parse trees.

the performance of zero pronoun detection also depends on the performance of the syntactic parser.


As described in Subsection 5.2, during training and testing, all links associated with zero pronouns will be considered in our coreference resolution system.

we do not distinguish zero pronoun resolution from traditional coreference resolution, and only view zero pronouns as special pronouns.


The results show that: • Using automatically detected zero pronouns achieves better performance under all experimental settings.

using automatic mentions, performance improves by 3.31%, 1.42%, and 2.34% in F-measure on the MUC, BCUBED, and CEAF evaluation metric, respectively.


In this paper, we focus on exploiting one of the key characteristics of Chinese text, zero pronouns, to improve Chinese coreference resolution.

a simplified semantic role labeling framework is proposed to detect zero pronouns effectively, and two effective methods are employed to incorporate zero pronouns into Chinese coreference resolution.


Table 1 also details the performance of four variants of our system that ablate various components and features.

we consider the following cases: • No NEL Mentions: We discard additional mentions, M N EL , provided by NEL (Sec.


In this paper, we investigate the interplay between syntax and morphology with respect to the task of assigning morphological descriptions (or tags) to each token of a sentence.

we examine the effect of syntactic information when it is integrated into the feature model of a morphological tagger.


All languages show agreement to some extent.

all languages show agreement in number (and person) between the subject and the verb of a clause.


In order to test this we run an additional experiment, where we add some of the syntax features to the feature set of featurama.

we add the static features from Table 2 that do not use lemma or part-of-speech information.


An SRCG-rule operates on vectors of ranges in contrast to the way a CFG-rule operates on single ranges (spans).

a non-terminal symbol in an SRCG (CFG) derivation can dominate a subset (substring) of terminals in an input string.


For each adapted non-terminal A ∈ M , a cache C A is maintained for storing the terminating tree fragments expanded from A earlier in the process, and we denote the fragment corresponding to the i-th expansion of A as z i .

the sequence of indices z i is the assignment of a sequence of expansions of A to particular tree fragments.


We aim to capture the evolution of commitments to certain preferences as the dialogue proceeds so as to predict the agents' bargaining behavior.

we wish to predict which of the 61 possible trade actions is executed at the end of each dialogue.


Offers then have a similar effect on the CP-net as the coherence relation Elaboration presented in (Cadilhac et al., 2011).

to automatically update the CP-net constructed so far with a current EDU that's an Offer, the two step rule for Offer(π 1 , π 2 ) is: 1. to update the speaker's CP-net according to the preferences expressed in π 1 , and 2. if π 2 expresses preferences, to enrich the CPnet with these new preferences so that each variable in π 2 depends on each variable in π 1 .


We assume that both the CAUSE and CONDITION subtypes of the CONTIN-GENCY relation will result in pairs of events that are likely to occur together and in a particular order.

we assume that the subtypes of the PDTB taxonomy of Contingency.Cause.Reason and Contingency.Cause.Result are the most likely to occur together as noted in previous work.


Other related work has made use of discourse connectives or discourse taggers (implicit discourse relations) to provide additional evidence of CONTINGENCY (Do et al., 2011;Gordon et al., 2011;Chiarcos, 2012;Pitler et al., 2009;Lin et al., 2010), but we do not because the results have been mixed.

these discourse taggers are trained on The Wall Street Journal (WSJ) and are unlikely to work well on our data.


Generally, this has the effect that the entities occurring in the translation and their mutual relations will display similar patterns as the entities in the source text.

coreference patterns tend to be very similar in translations of a text, and this fact has been exploited with good results to project coreference annotations from one language into another by using word alignments (Postolache et al., 2006;Rahman and Ng, 2012).


Moreover, we believe that this task can lead to interesting insights about anaphora resolution in a multi-lingual context.

we show in this paper that the pronoun prediction task makes it possible to model the resolution of pronominal anaphora as a latent variable and opens up a way to solve a task relying on anaphora resolution without using any data annotated for anaphora.


When computing derivatives with backpropagation, these shared dependencies must be taken into account.

the outputs y i of the antecedent resolution layer are the result of a softmax applied to functions of some shared variables q: The derivatives of any y i with respect to q, which can be any of the weights in the anaphora resolution subnetwork, have dependencies on the derivatives of the other softmax inputs with respect to q: This makes the implementation of backpropagation for this part of the network somewhat more complicated, but in the case of our networks, it has no major impact on training time.


Including these additional features leads to another slight increase in accuracy for both corpora, with similar or increased classifier F-scores for most classes except elle in the news commentary condition.

we should like to point out the performance of our benchmark classifier for elles, which suffered from extremely low recall in the first classifiers and approaches the performance of the other classes, with nearly balanced precision and recall, in this final system.


We are interested in REG under mismatched perceptual basis between conversation partners, where the agent has imperfect perception and knowledge of the shared environment.

we took a well-studied graph-based approach (Krahmer et al., 2003) and extended it to incorporate group spatial relations and uncertainties associated with automated perception of the environment.


The second heuristic is based on the observation that, given a certain orientation, people tend to use a relatum that is closer to the referent than more distant relata.

it is less likely to refer to an object relative to a distant relatum when there is a closer relatum.


We adopted some commonly used cost functions from previous work together with the cost functions defined here.

we experimented with the following different cost functions: Simple Cost: The costs for all hyperarcs are set to 1.


Especially when the perceived world is full of uncertainties, spatial relations tend to be more reliable.

as shown in Table 1, using hypergraphs enables generating group-based relations and results in significantly better performance (45.2%) compared to regular graphs (36.7%) (p = 0.002).


With this in mind, we re-visited the problem of referring expression generation in the context of mismatched perceptions between humans and agents.

we applied and extended the state of the art graph-based approach (Krahmer et al., 2003) in this new setting.


Note that the number of class labels extracted by the individual run shown in the second column (B) is shown in the fourth column (|B|).

there are around 1.6 million unique "list of .." queries, from which class labels are collected in run R ql .


We encode domain knowledge as First Order Logic (FOL) rules and automatically integrate them with a topic model to produce clusters shaped by the data and the constraints at hand.

we extend the Fold-all (First-Order Logic latent Dirichlet Allocation) framework (Andrzejewski et al., 2011) to the relation extraction task, explain how to incorporate meaningful constraints, and develop a scalable inference technique.


(2011) who develop a series of generative probabilistic models for relation extraction.

we extend their relational LDA model by interfacing it with FOL-rules.


More technically, a multinomial distribution over relations θ d i is drawn from a Dirichlet prior (θ ∼ Dir(α)) at the document level.

relation tuples are generated from a multinomial distribution  each tuple in a document is assigned a hidden relation (z = z 1 ...z N ); each relation is represented by a multinomial distribution over features φ r (Dirichlet prior β).


Our model clusters such tuples into underlying semantic relations (e.g., Located, Family) by incorporating general domain knowledge which we encode as First Order Logic rules.

we combine a topic model developed for the relation extraction task with domain relevant rules, and present an algorithm for estimating the parameters of this model.


Our question taxonomy is derived from the UIUIC dataset (Li and Roth, 2002) which defines 6 coarse and 50 fine grain classes.

our set of question categories is formed by adopting 3 coarse classes: HUM (human), LOC (location), ENTY (entities) and replacing the NUM (numeric) coarse class with 3 fine-grain classes: CUR-RENCY, DATE, QUANTITY 2 .


Hence, we propose to exploit a question focus along with the related named entities (according to the mapping from Table 1) of the answer sentence to establish relational links between the tree fragments.

once the question focus and question category are determined, we link the focus word w f ocus in the question, with all the named entities whose type matches the question class (Table 1).


Different from previous work that casts the answer extraction task as a tagging problem and apply a CRF to learn an answer phrase tagger (Yao et al., 2013), we take on a simpler approach using a kernelbased classifier.

we rely on the shallow tree representation, where text spans identified by a shallow syntactic parser serve as a source of candidate answers.


Algorithm 1 specifies the steps to generate training data for our classifier.

for each example representing a triple a, T q , T s composed of the answer a, the question and the answer sentence trees, we generate a set of training examples E with every candidate chunk marked with an ANS tag (one at a time).


Table 4 compares our kernel-based structural model with the previous state-of-the-art systems for answer sentence selection.

we compare with four most recent state of the art answer sentence reranker models (Wang et al., 2007;Heilman and Smith, 2010;Wang and Manning, 2010;Yao et al., 2013), which report their performance on the same questions and candidate sets from TREC 13 as provided by (Wang et al., 2007).


In this paper, we investigate the possibility of using structured annotation of queries to improve web search ranking.

we propose a greedy algorithm which uses the structured data (named annotated tokens in Figure 1(b)) extracted from the top search results to annotate the latent structured semantics in web queries.


For our experiments, we use the standard TAC data sets 8 , which have been used in the NIST competitions and in other summarization studies.

we used the TAC 2010 data set for creating the guided compression corpus and training the SVR pre-selection model, the TAC 2009 data set as development set for parameter tuning, and the TAC 2008 and 2011 data sets as the test set for reporting the final summarization results.


In Table 3 and Table 4, we include the results when this step is not used ("Our System w/o Pre-selection").

all of the sentences in the documents (excluding those containing less than 5 words) are compressed and used in the ILP-based summary sentence selection module.


While this might sound restrictive, we can usually generate at least some training instances from every sentence and discard the remaining families in the tree.

we do not need to extract training instances for all words in a given sentence since the reordering decisions are made independently for every head word.


The specification decoding scheme reflects the intuition of using hiero rules to translate shorter spans and tree-to-string rules to reorder higher-level sentence structures.

the scheme allows one-way switching from the hiero general nonterminal into the more specific nonterminal of a tree-tostring rule.


While how to most effectively apply  it is another topic, we simply show in this section using just alignment scores in binary prediction problems.

we pick the tasks of recognizing textual entailment (RTE), paraphrase identification (PP), and question answering sentence ranking (QA) described in Heilman and Smith (2010): RTE: predicting whether a hypothesis can be inferred from the premise, with training data from RTE-1/2 and RTE-3 dev, and test from RTE-3 test.


The inference routine then predicts the final clustering -a structured prediction problem -using these pairwise scores.


Our inference algorithm is inspired by the best-leftlink approach.

the score s(C; d, w) is defined so that each mention links to the antecedent mention (to its left) with the highest score (as long as the score is above some threshold, say, 0).


In particular, the score s(C; d, w) is defined so that each mention links to the antecedent mention (to its left) with the highest score (as long as the score is above some threshold, say, 0).

: In order to simplify the notation, we introduce a dummy mention with index 0, which is to the left (i.e.


mention i in document d is given by While SGD has no theoretically convergence guarantee, it works excellently in our experiments.

we observe that SGD achieves similar training performance to CCCP with a speed-up of around 10,000.


The rest of the inference process is the same as in the original best-leftlink inference.

this inference procedure combines the classifier score for mention pair i, j, with the constraints score of all mentions currently co-clustered with j.


However, not all mention types are equally interesting.

clusters which contain at least one proper name or a named entity mention are more important for information extraction tasks like Wikification (Mihalcea and Csomai, 2007;Ratinov et al., 2011), cross-document coreference resolution (Bagga and Baldwin, 1998), and entity linking and knowledge based population (Ji and Grishman, 2011).


We generate the gold annotation and system outputs by using the gold and predicted name entity tag annotations provided by the CoNLL shard task 2012.

if a cluster does not include any name entity mention, then it will be removed from the final clustering.


The performance of all systems degrades when considering only clusters that contain name entities, indicating that ENT-C is actually a harder task than the original coreference resolution problem.

resolving ORG coreferent clusters is hard, because names of organizations are sometimes confused with person names, and they can be referred to using a range of pronouns (including "we" and "it").


For example, L 3 M with γ = 0.2 improves L 3 M with γ = 0 by 0.7 points in ACE 2004.

this shows that considering more than a one left-links is helpful.


Overall, it is shown that each one of the constraints has a contribution, and that using all the constraints improves the performance of the system by 1.29% in the AVG F1 score.

most of this improvement (1.19%) is due to the must-link constraints (the first four constraints in the table).


In this paper, we advocate the use of centering as a means of reducing hubs.

we propose to measure the similarity of objects by the inner product (not distance or cosine) in the centered feature space.


Consider a dataset of m-dimensional feature vectors, with each vector x ∈ R m generated independently from a distribution with a finite mean vector µ.

objects x in this dataset are drawn from a distribution P(x), i.e., x ∼ P(x), where E[•] denotes the expectation of a random variable.


To this end, we consider weighted centering, a variation of centering in which each object is associated with a weight, and the origin is shifted to the weighted mean of the data.

we define the weight of an object as the sum of the similarities (inner products) between the object and all objects, regarding this sum as the index of how likely the object can be a hub.


When hubs exist in a dataset, the distribution of N k is expected to skew to the right, and yields a large S N k (Radovanović et al., 2010a).

similarity measures that yield smaller S N k are more desirable in terms of hub reduction.


After centering (K cent and K weighted ) skewness became markedly smaller than that of the noncentered cosine K. F1 score also improved with the decrease in skewness.

weighted centering (K weighted ) slightly outperformed GAMBL, though the difference was small.


For x, z ∈ Σ and y ∈ Σ + we have that is the weight that the grammarḠ assigns to a string xyz that has a cut or bracketing around y. Technically, it corresponds to the sum of the weights of all derivations that have a constituent spanning y.

we have that If x is a string of length m, and is the substring of x from positions t to t , it also happens that for any t between 1 and m. In this paper we will make frequent use of inside and outside quantities.


A Weighted Context Free Grammar (WCFG) over Σ with n states is a tuple G = α , {β σ }, A with: where the inside function We will define the outside function α G : Σ × Σ → R n as: For x, z ∈ Σ and y ∈ Σ + we have that is the weight that the grammar assigns to the string xyz with a cut around y.

g Let us make clear that a WCFG is the same device in classic or algebraic forms.


Some entries of this matrix will correspond to observable quantities.

for any string x ∈ I 1 for which we know p(x) we can define the following observable constraint: The rest of entries of H correspond to a string with an inside-outside cut, and these are not observable.


Define H to be the set of N -element sets containing the top proposals for each weak ranker (we use N = 2000).

each element of H is a set containing the 2000 highest values for some ϕ L or µ i .


The solid line in Figure 2 shows the recall-at-1220 for a boosted ranker using all monolingual statistics and k bilingual statistics, for increasing k. Bilingual statistics are added according to their individual recall, from best-performing to worst.

the point at k = 0 uses only µ 1 , µ 2 , and µ 3 , the point at k = 1 adds the best individually-performing bilingual statistic (Spanish) as a weak ranker, the next point adds the second-best bilingual statistic (German), etc.


It can be seen from Figure 3 that the performance drops smoothly when the window size is larger than 3.

ly, the Fscore of out-of-vocabulary identification decreases relatively fast beyond window size 5, which shows that the size of window (and the number of parameters) is too large that the trained network has overfitted on training data.


We show that a simple graph-based approach can leverage the information encoded in free association to narrow down the ambiguity of meaning, resulting in precise semantic groups.

we find that the main strongly connected component of the FA graph (the so-called core) is very cyclic in nature and contains a large predominance of short cycles (i.e., co-links and triangles).


This illustrates in a beautiful way the nature of free association as compared to the more neutral dictionary.

the characteristic path length is smaller in the FA graph, because humans use generalized event knowledge (McRae and Matsuki, 2009) in free association, producing semantic shortcuts.


In our formulation of CDP , correctness of an expanded pair is judged according to the pair's SVM score using BASE.

we regard an expanded pair that has an SVM score above some threshold α as a true contradiction.


In the realm of Twitter, we observed that many sarcastic tweets have a common structure that creates a positive/negative contrast between a sentiment and a situation.

sarcastic tweets often express a positive sentiment in reference to a negative activity or state.


In this study, we treat the profile summarization as a supervised learning task.

we model each sentence of the profile as a vector.


Here, the quality is measured by manually checking that whether they are well capable of summarizing their profiles.

they are written carefully, and could give an overview of a person and represent the education and experience information of a person.


ij g y y : For the social correlation factor function, we define it through the pairwise network structure.

if the person of sentence i and the person of sentence j have a social relationship, a factor function for this social connection is defined (Tang et al., 2011a;Tang et al., 2011b), i.e., The person-person social relationships are defined on Section 4, e.g.


We use  (the weight of the social connection factor function ( , ) ij g y y ) as the example to explain how we learn the parameters (the algorithm also applies to tune  by simply replacing  with ).

we first write the gradient of each k  with regard to the objective function (Eq.


From Figure 6, we can see that CoFG model with social connection always performs better than MaxEnt, and the performance of our approach descends slowly when the training dataset becomes small.

the performance of CoFG using only 10% training data achieves better performance than MaxEnt using 100% training data.


Table 4: Performances of different approaches to profile summarization in terms of different measurements Figure 6: The performance of CoFG with different training data size Table 5 shows the contribution of the social edges with CoFG.

coFG is our proposed approach with both education and experience information, coFG-edu means that the coFG model considers the social edges of education field (co_major, co_univ) only, and coFG-exp means that the coFG model considers the social edges of work experience field (co_title, co_corp) only.


We choose the following algorithms as baseline systems.

baseline 2 and 3 are summarization systems which are similar to our storyline generation system.


Thus the ratedistortion function is a trade-off between the compactness of new representation and the expected distortion.

in summarization, the summaries can be seen as the new representationX of original documents X.


• practically, our DP best-first parser is only ∼2 times slower than a pure greedy parser, but is guaranteed to reach optimality.

it is ∼20 times faster than a non-DP best-first parser.


Assume a shifted derivation x of state p is a direct descendant from derivation x of state p , then p ∈ R(p ), and we have: which is proved in Section 5.1.

more formally, we can conclude that where T (p) is the left corner states of shifted state p, defined as which represents the set of all states that have the same reducibility as a shifted state p. Algorithm 2 Lazy Expansion of Algorithm 1.


In this paper we ask whether such knowledge generalizes to the observed reality outside the language domain, where we use well-known image datasets as a proxy for observed reality.

we aim to determine which language model yields knowledge that is most suitable for use in Computer Vision.


In this paper we present, for the first time, a successful approach to jointly resolving grammatical errors.

: • We identify two pairs of interacting phenomena, subject-verb and article-NPhead agreements; we show how to reliably identify these pairs in noisy ESL data, thereby facilitating the joint correction of these phenomena.


In the article-NPhead structures, the interaction is between the head of the noun phrase (NP) and the article that refers to the NP (first example in Table 3).

the model should take into account that the article "a" cannot be combined with a noun in plural form.


The heuristics differ from the joint inference in that they enforce agreement by always changing either the noun (NaïveNoun) or the verb (NaïveVerb), while the joint inference does this using the scores produced by the independent models.

the key is the objective function, while the components of the objective function are the same in the heuristics and the joint inference.


The features used in their model are very simple.

they use use word lengths and frequencies, like us, as well as distances between tokens (important in a transitionbased model), and, finally, the history of previous decisions.


One strand of work in dialog research targets the rapid prototyping of virtual humans capable of conducting a conversation with humans in the context of a virtual world.

question answering (QA) characters can respond to a restricted set of topics after training on a set of dialogs whose utterances are annotated with dialogue acts (Leuski and Traum, 2008).


1 http://www.itl.nist.gov/iad/mig/tests/ace/ Most prior work on time normalization has taken a rule-based, string-to-string translation approach.

each word in a time expression is looked up in a normalization lexicon, and then rules map this sequence of lexical entries directly to the normalized form.


In the remainder of this section, we lay out a research agenda centered around capturing this opportunity.

taking a systemic approach to rule-based IE, one can identify a set of research problems by separating rule development and deployment.


Specifically, taking a systemic approach to rule-based IE, one can identify a set of research problems by separating rule development and deployment.

we believe research should focus on: (a) data models and rule language, (b) systems research in rule evaluation and (c) machine learning research for learning problems in this richer target language.


Drawing inspiration from the success of SQL and the database community, we proposed directions for addressing the disconnect.

we call for the standardization of an IE rule language and outline an ambitious research agenda for NLP researchers who wish to tackle research problems of wide interest and value in the industry.


In contrast to these previous PRA-based approaches where all edge labels are either KB labels or at surface-level, in this paper we explore using latent edge labels in addition to surface-level labels in the graph over which PRA is applied.

we focus on the problem of performing inference over a large KB and learn latent edge labels by mining dependency syntax statistics from a large text corpus.


In order to get the edges in E − E, we first collect a set of Subject-Verb-Object (SVO) triples D = {(s, v, o, c)} from a large dependency parsed text corpus, with c ∈ R + denoting the frequency of this triple in the corpus.

the additional edge set is then defined as We define S = {v | ∃(s, v, o) ∈ E syntactic } and set t = t ∪ S. for each pair of directly connected nodes in the KB graph G, we add an additional edge between those two nodes for each verb which takes the NPs represented by two nodes as subjects and objects (or vice versa) as observed in a text corpus.


For this purpose, we first use a frequency based method to filter meta-rule features.

we first divide all the meta-rules into 100 bins, (|F |, |E|), where |F | is the number of words on the source side, and |E| the target side, 0 < |F |, |E| ≤ 10.


We did not discard any n-grams in training this model.


The interpolation weight was determined by the grid search.

we changed the interpolation weight to 0.1, 0.3, 0.5, 0.7, 0.9 to create an interpolated BNLM.


In addition, the first-pass of CONV42 and CONV746 (32.58 and 33.22) were comparable with those of the reranking results of BNLM42 and BNLM746 (32.44 and 33.36), respectively.

there were no significant differences between these results.


Then it uses a SVM classifier over the augmented feature space to obtain classification result.

our FEDA methods use L 2 -regularized SVM with linear kernel by LIBLINEAR package 1 .


Random restarts, often considered a footnote of experimental design, can indeed be useful on scales beyond that generally used in past work.

we found that the initializations that lead to the local optima with highest likelihoods are sometimes very rare, but finding them can be worthwhile; for the problems we looked at, local optima with high likelihoods also achieved high accuracies.


While previous work has performed paraphrase classification using distance or similarity in the latent space (Guo and Diab, 2012;Socher et al., 2011), more direct supervision can be applied.

we convert the latent representations of a pair of sentences v 1 and v 2 into a sample vector, concatenating the element-wise sum v 1 + v 2 and absolute .


Finally, the accessibility of the subject of the second argument in a relation may play a role in determining whether the relation is explicitly marked.

informal observation of the data suggests that there is a tendency for the second argument of an implicit relation to begin with a longer, contentful noun phrase, rather than a pronoun.


The model also found that the features which indicate whether a relation has shared arguments with either the preceding or following relations in the text (Arg1isPrevArg2, Arg2isNextArg1) are both predictors of an implicit outcome.

if a clause in the text serves as the argument for two adjacent relations, then both of these relations are more likely to be realized implicitly.


Hence, we treat a case frame cf * which is selected by a heuristic method as a correct case frame and remove (cf, a * ) which has other cf .

we make ranking data for the learning for each target predicate p in the following steps.


The zero endophora are given priority to zero exophora.

the A/R mentions are selected to the referents in preference to pseudo entities when there are A/R mentions.


φ discourse is the same as φ A of the baseline model and the difference is explained in section 6.3. φ [author ] and φ [reader ] become active when e is [author]/[reader] or the discourse entity corresponding to the A/R mention.

when e is the discourse entity corresponding to the A/R mention, both φ discourse and φ Because φ [author ] , φ [reader ] , φ [US :person] and φ [US :others] correspond to the pseudo entities, which are not mentioned explicitly, we cannot use word information such as expressions and categories.


Testing Automatic Response Models In testing a retrieval-based system, although we can simply use the original responses associated with the query post as positive and treat all the others as negative, this strategy suffers from the problem of spurious negative examples.

with a reasonably good model, the retrieved responses are often good even if they are not the original ones, which brings significant bias to the evaluation.


This requires the entities in the response to be correctly aligned with those in the post.

if the post is about entity A, while the response is about entity B, they are very likely to be mismatched.


However, the second sentence itself also expresses opinion on various opinion targets.

both subjective and objective sentences can be used as explanations.


Property 3: The positions of explanatory clauses are consecutive.

if clause c k and c k+2 explain clause c j , the clause c k+1 would also be explanatory clause of c j .


In this paper, we focus on the second stage.

by given golden cues, we detect their linguistic scopes.


This paper proposes a new approach for tree kernel-based scope detection by using the structured syntactic parse information.

we have explored the way of selecting compatible features for different part-of-speech cues.


We propose, to the best of our knowledge, the first successful attempt to incorporate searcher examination data into passage retrieval for question answering.

we exploit detailed examination data, such as mouse cursor movements and scrolling, to infer the parts of the document the searcher found interesting, and then incorporate this signal into passage retrieval for QA.


A key point to emphasize is that our approach exploits the natural browsing behavior of the users, not requiring any additional effort from the searchers.

our contributions include: • A novel approach to passage retrieval for question answering, that naturally integrates textual and behavioral evidence.


Our paper uses a subset of the real questions posted by users on Community Question Answering (CQA) sites, and searches and interactions from real users -which makes our task unique and more challenging than the previous settings.

our work builds on the rich history of using eye tracking technology to identify areas of interest and attention, and to study reading behavior.


While eye tracking equipment limits the applicability of these findings to lab studies, these studies served as inspiration to our work to detect the inferred areas of interest.

we use mouse cursor tracking as a natural proxy for user's attention, to replace the requirement for eye tracking equipment.


In summary, there is mounting evidence that the user's attention in web search can be approximated using mouse cursor, scrolling, and other interaction data.

hijikata (2004) proposed a method to extract text passages of Web pages based on the user's mouse activity and found that extracted passages based on mouse ac-tivity such as text tracing, link pointing, link clicking and text selection enable more accurate extraction of key words of interest than using the whole text of the page.


Our goal is to incorporate the searcher behavior (in particular, page examination) into passage retrieval.

by analyzing the searcher behavior data, we aim to identify the parts of the page that contain relevant passages for answering a question.


That is, by analyzing the searcher behavior data, we aim to identify the parts of the page that contain relevant passages for answering a question.

given a question, a set of queries generated by searchers attempting to answer this question, and a set of documents retrieved by a search engine for each of the queries, our goal to retrieve a set of passages that contain correct answers for the question.


Specifically, given a question, a set of queries generated by searchers attempting to answer this question, and a set of documents retrieved by a search engine for each of the queries, our goal to retrieve a set of passages that contain correct answers for the question.

our goal is to identify, from searcher behavior, the passages in the documents most likely to contain correct answers to a question, which could then be incorporated into a fully automated question answering system, or returned to the user directly, for example, by incorporating these passages into the result abstracts or "snippets".


Our approach accomplishes the goal above by incorporating both textual and behavioral evidence.

we combine together traditional textbased passage retrieval features, and the inferred user interest in specific parts of a document based on searcher behavior.


We presented the first successful approach to incorporating naturalistic searcher behavior data into passage retrieval for question answering.

we developed a robust method to infer searcher interest in specific parts of the document, which could then be combined with more traditional textual features used for passage retrieval.


The list of 36 basic tags was compiled following the best practices of Penn tagset design (Marcus et al., 1993), and bearing in mind the specifics of Kazakh grammar.

we broke down the major POS categories in sub-categories, in order to capture semantic distinctions and various usage patterns.


Another difference to Arabic is that the affixes are not the same for all verb paradigms.

the weak verbs form several inflectional classes (which, of course, differ only in affixes).


Therefore we turn to measuring the upper bound on accuracy for the joint approach as a whole.

we would like to find a bound on accuracy given an ideal representation of the source sentence.


In order to bridge the gap, we propose to learn a separate transform from the foreign words to an encoding of the reference target words, thus making the source-side representations look more like the target-side encodings used in the oracle experiment.

we learn a linear transform d θ : x → r mapping directly from a vector encoding of the foreign sentence x to an l-dimensional LSA representation r of the reference sentence.


In this paper, we use MTL to jointly adapt SMT models to multiple domains.

we develop multiple SMT systems based on mixture models, where each system is tailored for one specific domain with an in-domain Translation Model (TM) and an in-domain Language Model (LM).


In the second step, with the selected in-domain training data, we develop SMT systems based on mixture models.

we use the mixture model based approach proposed by Koehn and Schroeder 1 Many web portal sites contain domain information for webpages, such as "www.yahoo.com" in English and "www.sina.com.cn" in Chinese and etc.


From Table 3, the MTL-based approach significantly improve the translation quality over the non-adapted baseline, and also outperforms conventional mixture models based methods.

the "Sports" domain benefits the most from the indomain knowledge, which confirms that domain discrepancy should be addressed and may bring large improvements on certain domains.


We modify our approach to take advantage of the document correspondences within our comparable corpus.

we would like to match the marginals for all document pairs.


We assume that we have enough monolingual NEWdomain data in one language to rank comparable document pairs (here, Wikipedia pages) according to how NEW-domain-like they are.

we estimate the similarity to a source language (here, French) corpus in the NEW domain.


We find it informative to evaluate the models using bilingual lexicon induction metrics before integrating our output into full MT.

we do not compare the full joint distributions, but, rather, for a given French word, how our learned model ranks the word's most probable translation under the gold distribution.


That is, we do not compare the full joint distributions, but, rather, for a given French word, how our learned model ranks the word's most probable translation under the gold distribution.

because we are primarily concerned with learning translations for previously unseen words, we evaluate over OOV French word types.


Third, we rank English words by their document-pair co-occurrence score with each French OOV word.

for all words w, we compute D(w), the vector indicating the document pairs in which w occurs, over the set of 50,000 document-pairs which are most NEWdomain-like.


But standard update is not: it is possible that at the end of search, the best y-good derivation d + |x| (x, y), though pruned earlier in the search, ranks even higher in the current model than anything in the final bin (see Figure 4).

there is no mistake at the final step, while there must be some search error in earlier steps which expels the y-good subderivation.


Below both MERT and PRO tune weights on the dev set, while our method on the training set.

our method only uses the dev set to know when to stop training.


We call this method learning in the similarity space (LSS).

in LSS, each document d is first represented with a document space vector (called a d-vector) based on the document itself as in the traditional classification learning of AA.


In each iteration, we plant one userid of an author in the test set and use the other userid of the same author as the query userid.

in the i th run, the test data consist of the following two components: 1.


 Type II: Type II experiments test the cases when no pair exists.

we do not plant any matching userid for the query userid.


For each test userid id, we build a SVM classifier based on the one vs. all strategy.

for training we use id's queries in T*_Q*S10 as the positive documents, and all queries of the other test userids (e.g., 99 userids if the test data has 100 userids) as the negative documents.


While all the definitions provided above for the SVM features are operational, there is a glaring disconnect between the whitespace-border definition of a word and written conventions in Japanese.

in Japanese words are generally not separated by whitespace.


Words without grounded features are all given the same placeholder feature, also resulting in a wordfeature pair.

5 for the feature norm modality, we generate (word, feature norm) pairs; for the SURF modality, we generate (word, codeword) pairs, etc.


Like Petrov and Klein (2007), we impose that the initial symbol S has only one latent annotation.

among rules with S on the left-hand side, only those of the form With such a grammar G we can define probabilities over trees in the following way.


petrov and Klein (2007) found that normalizing by the forest probability (in other words the inside probability of the root node) give better exper- Figure 2: Variational Inference for pCFG-LA.


This method is suitable for structured prediction in log-linear models where second derivatives are computable.

it can be used for multiclass classification with maximum entropy models (a.k.a., softmax or multinomial logistic regression) and for the sequence models that are ubiquitous in NLP, via linear chain Conditional Random Fields (CRFs).


This suggests estimating R using unlabeled data.

if we have n labeled examples D = {x 1 , x 2 , .


Fortunately, in sequence tagging and other NLP tasks, the majority of features are sparse and they often co-occur.

some of the active features would fire and only fire at the same locations in a given sequence.


However, Wallach's experiments were limited to bigrams, a toy setting for language models, and experiments with higher-order n-grams have not yet been sufficiently studied, which we investigate in this paper.

we point out the two fundamental problems caused when extending Wallach's model to a higher-order: sparseness caused by dividing all n-grams into exclusive topics, and local minima caused by the deep hierarchy of the model.


The bigram topic model of Wallach (2006) simply replaces this unigram word distribution (a multinomial) for each topic with a bigram word distribution 1 .

ordinary LDA generates word conditioning only on the latent topic, whereas the bigram topic model generates conditioning on both the latent topic and the previous word, as in the bigram language model.


We test the effectiveness of presented models and the blocked sampling method on unsupervised language model adaptation settings.

we concentrate on the dynamic adaptation: We update the posterior of language model given previously observed contexts, which might be decoded transcripts at that point in ASR or MT.


In each run one set is used as the labelled data S, and the other nine sets are combined and used as the unlabelled data U with segmentations removed.

10% of the training data is used as segmented data, and 90% are used as unsegmented data in our semi-supervised training.


On the other hand, when we focus on the resulting predicate argument structures, the normalized-case structure is more useful.

since a normalized-case structure represents the same meaning in the same representation, normalized-case analysis is useful for recognizing textual entailment and information retrieval.


Therefore, we need a system that first analyzes surface cases and then alternates the surface cases with normalized-cases.

we focus on the transformation of the passive voice into the active voice in this paper.


(2006)'s method, which is based on SVMs, we added the output of Model 2 as a new feature.

we first tuned the parameter vector x on the training data and acquired the knowledge for case alternation with the tuned parameter.


We basically used the same algorithm as Algorithm 1 for acquiring the knowledge for case alternation, but used different constraints on case alternation patterns because possible case alternation patterns are different from those of active-passive alternation.

we replaced the third and fourth lines of Algorithm 1 with "for each c to ga ∈ {NIL, ni}" and "for each c ga to ∈ {wo, ni}," respectively, based on linguistic analysis of active-causative alternation in Japanese.


The LFT-based scores and features are fed to a machine learning algorithm.

there are 477 features derived from the logic prover: • 9 LFT-based scores (3 × 3; three scores (2 directions and average), three LFT modes) pred.


When training data is available (MSRpar, MSRvid, SMTeuroparl), LFT-scores + features always outperforms the scores obtained with a single logic form transformation in an unsupervised manner.

combining the scores obtained with the three logic form transformations and incorporating the additional features derived from the proofs improves performance.


The second is via supervised learning, where we explore different modeling techniques to capture the statistical regularities in creative compositional operations.

we will explore (1) compositional operations of vector space models, (2) kernels capturing the non-linear composition of different dimensions in the meaning space, (3) the use of neural networks as an alternative to incorporate nonlinearity in vector composition.


It contains a set of words that have been rated in terms of pleasure (affective valence), arousal, and dominance.

for our task we considered the valence dimension.


In Table 6 we report the results for the best performing formulae and learning algorithm on the GI PoS classes.

for ADJ there are 1,073 words, 922 for NOUN and 508 for VERB.


Each babble was turned into a set of nested queries by sequentially concatenating words.

the first query contained only the first word from the babble, the second query only the first two words, and so on.


If the first guess is bad, it would be reasonable to score a second guess, with some penalty.

there are several things that we would like our evaluation framework to describe.


In this paper, we propose a novel solution to tackle the challenging problem of storylines extraction and reconstruction.

we first investigate two requisite properties of an ideal storyline.


In fact, both of them involve a measurement of similarity between two story nodes, denoted by two word distributions (see Figure 2).

for story node I i , its distribution probability of word w is estimated as p( where the denominator is used for normalization.


In this way we verify the property of coverage.

shannon denotes the entropy H as follows of a discrete random variable X, which in fact is a word distribution.


Our works mainly tackle the problems of storylines extraction and reconstruction.

we investigate two requisite properties of an ideal storyline, Coherence and Diversity.


The current paper significantly elaborates on the latter, with a more detailed description of the algorithm and additional contrastive experiments including more data sets.

new experiments clearly demonstrate the benefit of the method in a realistic setting with statistically significant gains.


To do so in a graph based approach, one needs to keep separated the paths of different lengths ending in a given node.

only paths of the same length ending at a given point, with different predecessors, should be recombined so that disruption can be considered properly in subsequent steps of the algorithm.


We study the brand related language use through these reviews, with focuses on the brand satisfaction and gender factors.

we perform three tasks: automatic brand identification from raw text, joint brand-satisfaction prediction, and joint brandgender-satisfaction prediction.


5 We create two features to exploit narrative event chains for a candidate NP, c, if it serves as a subject or object.


Hence, given a data set D = {(m i , r i ), i = 1, ... , |D|} consisting of (mention, relationship) training pairs, one could learn the embeddings using constraints of the form: where 1 is the margin.

we want the relation that (weakly) labels a given mention to be scored higher than other relation by a margin of 1.


Training is carried out by Stochastic Gradient Descent (SGD), updating W and r at each step, following Bordes et al., 2013).

at the start of training the parameters to be learnt (the n v × k word/feature embeddings in W and the n r × k relation embeddings r ) are initialized to random weights.


Given a training set S = {(h i , r i , t i ), i = 1, ... , |S|} of relations extracted from the KB, this model learns vector embeddings of the entities and of the relationships using the idea that the functional relation induced by the r -labeled arcs of the KB should correspond to a translation of the embeddings.

given a k-dimensional embedding of the left-hand side (head) entity, adding the k-dimensional embedding of a given relation should yield the point (or close to the point) of the kdimensional embedding of the right-hand side (tail) entity.


The model thus gives the following score for the plausibility of a relation: A ranking loss is also used for training S kb .

the ranking objective is designed to assign higher scores to existing relations versus any other possibility: for each known triple (h, r , t), if we replaced the (i) head, (ii) relation or (iii) tail with some other possibility, the modified triple should have a lower score (i.e.


First, for each pair of entities (h, t) that appear in the test set, all the corresponding mentions M h,t in the test set are collected and a prediction is performed with: The predicted relationship can either be a valid relationship or NA -a marker that means that there is no relation between h and t (NA is added to R during training and is treated like other relationships).

ifr h,t is a relationship, a composite score is defined: only the top scoring non-NA predictions are re-scored.


In our experiments, we use MT word alignments extracted with the Berkeley Aligner (Liang et al., 2006) 1 .

we use the following equation to compute starting word embeddings: In this equation, S is the number of possible target language words that are aligned with the source word.


The vector representation allows us several choices for the node kernel function κ.

we consider: We note that the kernels above take strictly nonnegative values in [0, 1] (assuming word vector representations are normalized).


This corresponds to k+1 length walks on G starting from u and ending at any node.

w u,(v i ,v j ) corresponds to the sum of similarities of all common walks of length n in T and T that end in v i in T and v j in T .


(2011 Filtering Reviews: When computing above statistics over the set of reviews corresponding to each restaurant, we also consider removing a subset of reviews that might be dubious or just noise.

we remove reviews that are too far away (delta ≥ 2) from the average review rating.


Following a common approach to structured prediction problems, the score of a tree y is defined as a sum of local scores.

s , where E ranges over small connected subgraphs of y that can be scored individually.


In this paper, we propose a new cross-lingual adaptation approach for document classification based on learning cross-lingual discriminative distributed representations of words.

we propose to maximize the loglikelihood of the documents from both language domains under a cross-lingual logbilinear document model, while minimizing the prediction log-losses of labeled documents.


In this paper, we propose to tackle cross language text classification by inducing cross-lingual predictive data representations with both labeled and unlabeled documents from the two language domains.

we propose a cross-lingual log-bilinear document model to learn distributed representations of words, which can capture both the semantic sim-ilarities of words across languages and the predictive information with respect to the target classification task.


We suggest to use expert knowledge or crowdsourcing in the loop.

we present experiments where standard representations are augmented with features from a few hours of manual work, by machine learning experts or by turkers.


Seven researchers and graduate students spent five effective hours querying the datasets with regular expressions.

they spent three hours on the Stack Exchange dataset, and one hour on each of the two RateBeer datasets.


• We successfully apply the acquired data to train a novel supervised compression system which produces readable and informative compressions without employing a separate reranker.

we start with the unsupervised method of Filippova & Strube (2008) and replace the ad hoc edge weighting with a linear function over a rich feature representation.


First, we can eliminate the constraint z h ≥ z m by adding a penalty term to the objective function.

we can see that the penalty term −∞(1 − z h )z m excludes z h = 0, z m = 1 from the feasible set, and for z h ≥ z m , both problems have the same objective function value.


Due to the lack of training data for compressive summarization, we learn the subtree deletion model and the concept model separately.

the sentence compression dataset (Clarke and Lapata, 2008) (referred as CL08) is used for subtree deletion model training (θ arc ).


Given this input, we define a probabilistic context-free grammar (PCFG) that captures the structure of the database and how it can be verbalized.

we extend the model of Konstas and Lapata (2012) which also uses a PCFG to perform content selection and surface realization, but does not capture any aspect of document planning.


Konstas and Lapata (2012) use a modified version of the CYK parser (Kasami, 1965;Younger, 1967) to findĝ.

they intersect grammar G with a n-gram language model and calculate the most probable generationĝ as: where p(g, h | d) is the decoding likelihood for a sequence of words g = g 1 .


Data Since our aim was to evaluate the planning component of our model, we used datasets whose documents are at least a few sentences long.

we generated weather forecasts and troubleshooting guides for an operating system.


If a discourse tree explicitly represents parent-child relationships between textual units, we can apply the well-known tree-trimming approach to a discourse tree and reap the benefit of combinatorial optimization methods.

to apply the treetrimming approach, we need a tree whose all nodes represent textual units.


A nucleus is more important than a satellite in terms of the writer's purpose.

a satellite is a child of a nucleus in the RST-DT.


better results than MCP, KP and LEAD, although some of the comparisons are not significant.

tKP(G) achieved the highest ROUGE scores on all measures.


We instead define a few simple composition rules which over-generate and then use model features to simulate soft rules and categories.

we use POS tag features and features on the denotations of the predicted logical forms.


We now discuss the construction of a lexicon L, which is a mapping from natural language phrases to logical predicates accompanied by a set of features.

for a phrase w (e.g., "born in"), L(w) is a set of entries (z, s), where z is a predicate and s is the set of features.


Second, we introduce features on the POS tags involved in a composition, inspired by dependency parsing (Mc-Donald et al., 2005).

when we combine logical forms z 1 and z 2 via a join or bridging, we include a feature on the POS tag of (the first word spanned by) z 1 conjoined with the POS tag corresponding to z 2 .


Denotation features While it is clear that learning from denotations rather than logical forms is a drawback since it provides less information, it is less obvious that working with denotations actually gives us additional information.

we include four features indicating whether the denotation of the predicted logical form has size 0, 1, 2, or at least 3.


and performed a breadth-first search over questions (nodes), using the Google Suggest API supplying the edges of the graph.

we queried the question excluding the entity, the phrase before the entity, or the phrase after it; each query generates 5 candidate questions, which are added to the queue.


To create a blind test set for evaluation, our annotators labeled an additional 500 posts as patientspecific or general.

they labeled those 500 posts with PI sentences.


Given a source lexical chain LC s = {s j i } where the ith chain word s j i is from the jth sentence of the source document D s , we generate a target lexical chain LC t = {t j i } using maximum entropy (Max-Ent) classifiers.

we translate a word s j i in the source lexical chain into a target word t j i in the target lexical chain using a corresponding Max-Ent classifier as follows 1 .


We used a hierarchical phrased-based SMT system (Chiang, 2007) trained on large-scale data.

we aim at: • Measuring the impact of the threshold on the probability cohesion model and selecting the best threshold on a development test set.


Perhaps a more interesting question, and one that the current paper aims to address, is to what extent disambiguation can also provide benefits for tensor-based approaches, which in general constitute more powerful models for natural language (see discussion in Section 2).

this paper aims to: (a) propose disambiguation algorithms for a number of tensorbased distributional models; (b) examine the effect of disambiguation on tensors for relational words; and (c) meaningfully compare the effectiveness of tensor-based against vector mixture models in a number of tasks.


The disambiguation of a new word w under a context C can now be accomplished as follows: we create a context vector − → c for C as above, and we compare it with every sense vector of w; the word is assigned to the sense corresponding to the closest sense vector.

if S w is the set of sense vectors for w, − → c the context vector for C, and distance measure, the preferred senseŝ is given by: For the actual clustering step we follow the setting of Kartsaklis et al.


(2013), which worked well in tasks very similar to ours.

we perform hierarchical agglomerative clustering (HAC) using Ward's method as the inter-cluster distance, while the distance between vectors is measured with Pearson's correlation.


To make the analogy to SVD clear, we rewrite the results of Tucker decomposition by performing a nmode matrix product over the core tensor G with the matrix T. This produces a tensor S where each slice is a linear combination of the slices of G with coefficients given by T (see (Kolda and Bader, 2009) for detail).

we have An illustration is shown in Fig.


We therefore impose the constraint which requires translation pairs to be symmetric in order to qualify as potential new dimensions of the space.

given the current set of dimensions Z s , a translation pair (w S i , w T j ) has a possibility to be chosen as a new dimension from the pool of candidate dimensions if and only if it holds: T C(w S i ) = w T j and T C(w T j ) = w S i .


There has recently been a surge of interest for deep learning in natural language processing.

neural network language models (NNLMs) have been used to learn distributional word vectors (Bengio et al., 2003;Schwenk, 2007;Mikolov et al., 2010): the models jointly learn an embedding of words into an n-dimensional feature space.


We compare to the recent line of work on supervised compositional models.

we will describe and experimentally compare our new RNTN model to recursive neural networks (RNN) (Socher et al., 2011b) and matrix-vector RNNs  both of which have been applied to bag of words sentiment corpora.


Now, we can compute the error message for the two children of p 2 : where we define The children of p 2 , will then each take half of this vector and add their own softmax error message for the complete δ.

we have where δ p 2 ,down [d + 1 : 2d] indicates that p 1 is the right child of p 2 and hence takes the 2nd half of the error, for the final word vector derivative for a, it will be δ p 2 ,down The full derivative for slice V [k] for this trigram tree then is the sum at each node: and similarly for W .


DF-LDA cannot handle multiple senses because its definition of must-link is transitive.

if A and B form a must-link, and B and C form a must-link, it implies a must-link between A and C, indicating A, B, and C should be in the same topic.


The generalized Pólya urn (GPU) model differs from the SPU model in the replacement scheme during sampling.

when a ball is randomly drawn, certain numbers of additional balls of each color are returned to the urn, rather than just two balls of the same color as in SPU.


This problem can be dealt with the very sampling scheme of the GPU model (Chen et al., 2013a).

by adding additional , ′ , balls of color into while keeping the drawn ball, we increase the proportion (probability) of seeing the m-set under topic and thus promote m-set as a whole.


As the E-GPU model allows multi-urn interactions, when sampling a ball represents word from a word urn , , we want to transfer the balls representing cannotwords of (sharing a c-set with ) to other urns (see Step 3 a below).

decrease the proba-bilities of those cannot-words under this topic while increasing their corresponding probabilities under some other topics.


In order to correctly transfer a ball that represents word , it should be transferred to an urn which has a higher proportion of and its related words (i.e., words sharing msets with ).

we randomly sample an urn that has a higher proportion of any m-set of to transfer to (Step 3 b below).


• LDA-GPU: LDA with GPU (Mimno et al., 2011).

lDA-GPU applies GPU in lDA using co-document frequency.


Given a set of search queries q ∈ IR Q and docu-ments d ∈ IR D , where the j th dimension of a vector indicates the occurrence of the j th word for dictionaries of size Q and D, we want to learn a score f (q, d) between a query and a given document using the model 1 We take a pairwise ranking approach to optimization.

given labeled data in the form of a set R of tuples (q, d + , d − ), where d + is a relevant (or higher ranked) document and d − an irrelevant (or lower ranked) document for query q, the goal is to find a weight matrix W ∈ IR Q×D such that f (q, d + ) > f (q, d − ) for all data tuples from R. The scoring model learns weights for all possible correspondences of query terms and document terms by directly optimizing the ranking objective at hand.


For Japanese-to-English patent translation we used data provided by the organizers of the NTCIR 9 workshop for the JP-EN PatentMT subtask.

we used the data provided for NTCIR-7 (Fujii et al., 2008), consisting of 1.8 million parallel sentence pairs from the years 1993-2002 for training.


We see that using global constraints improves performance almost invariably on all measures in both full and temporal evaluations.

in the full evaluation Global improves recall by 12% and overall F 1 improves significantly by 3.7 points against Local (p < 0.01).


This indicates that our novel features (Section 3.2) are important for discriminating between process relations.

in the full evaluation Local improves precision more than in the temporal evaluation, suggesting that designing syntactic and semantic features for connectives is useful for distinguishing PREV, CAUSES, and ENABLES when the amount of training data is small.


The degree distribution of Global is more similar to the gold standard than Local.

the connectivity constraint ensures that there are no isolated nodes and shifts mass from nodes with degree 0 and 1 to nodes with degree 2.


By using triples rather than a pair-wise representation of subject-verb and verb-object, we achieve more coherent schemas than Chambers and Jurafsky (2009).

our schemas have higher topic coherence (92% compared to Chambers' 82%; make a higher percentage of valid assertions (94% compared with 61%); and have greater actor coherence (81% compared with 59%).


We dub our model ONETA (OrthoNormal Explicit Topic Analysis) and empirically show that on a cross-lingual retrieval task it outperforms ESA, LSI, and Latent Dirichlet Allocation (LDA) as well as a baseline consisting of translating each word into the target language, thus reducing the task to a standard monolingual matching task.

we quantify the effect of different approximation techniques for computing the orthonormal basis and investigate the effect of various methods for the normalization of frequency vectors.


Thus, if we take the first N 1 columns (documents) in X, it is possible to rearrange the rows of X with the result that there is some W 1 such that rows with index greater than W 1 have only zeroes in the columns up to N 1 .

we take a subset of N 1 documents and enumerate the words in such a way that the terms occurring in the first N 1 documents are enumerated 1, .


It is therefore necessary to re-define the λ.

we need to define the gradient of the evaluation criteria in AES, e.g.


As the model is trained by learning to rank algorithms, essays with higher model scores tend to get higher actual ratings.

essays with close model scores tend to get the same rating.


Despite its relative simplicity, we are not aware of previous work that looks at the distribution of constituents directly.

we are interested in examining the distribution of phrasal and/or clausal tags as follows: (i) Phrasal tag percent (PHR) -percentage distribution of phrasal tags.


As can be seen in  shown in Table 5 -Bottom, we find substantial differences in all genres.

discriminative unigrams that characterize less successful novels involve significantly more sentiment-laden words.


The relatively low F-scores for the metrics quantifying performance with respect to the cross of topic and speech act codes belie relatively good overall (marginal) predictive performance.

we achieve much better performance with respect to metrics that measure topic and speech act predictions independently of one another.


This is due to the very large output space under consideration (see Table 2).

averaged over ten runs, the MM model achieves a marginal mean topic F-score of .667 and marginal mean speech act F-score of .516.


Others have investigated visual structures of patient-provider interactions to qualitatively assess communication in care.

(Cretchley et al., 2010) leveraged concept maps to explore conversations between people with schizophrenia and their carers.


An easy way to use an EEC-set is to simply predict that all r i in the EEC-set are event-mentions, and hence are semantically equivalent.


The advantage of the joint model is analogous to that of cluster-based approaches for coreference resolution (CR).

a joint model can better capture constraints on multiple variables and can yield higher quality results than pairwise CR models (Rahman and Ng, 2009).


ResolverNYT: Since Resolver is supposed to perform better when given more accurate statistics from a larger corpus, we tried giving it more data.

we ran ReVerb on 1.8 million NY Times articles published between 1987 and 2007 obtain 60 million extractions (Sandhaus, 2008).


In this paper we focus on understanding those natural language constructs that will allow eliminating these "obvious" (to a human reader) mistakes from Wikification.

we focus on resolving coreference and a collection of local syntacticosemantic relations (Chan and Roth, 2011); better understanding the relational structure of the text allows us to generate title candidates more accurately given the text, rank these candidates better and determine when a mention in text has no corresponding title in Wikipedia and should be mapped to NIL, a key problem in Wikification.


To accomplish that we define our objective function as a Constrained Conditional Model (CCM) (Roth and Yih, 2004;Chang et al., 2012) that is used to reward or penalize a pair of candidates t k i , t l j by w (k,l) ij when they are chosen in the same document.

we choose the assignment Γ D that optimizes: Integral constraints Note that as in most NLP problems, the problem is very sparse, resulting in a tractable ILP that is solved quickly by off-the-shelf ILP packages (Gurobi Optimization, 2013).


These descriptions provide a clean "definition" of the entity, allowing us to abstract the inference to a limited "noun phrase entailment problem".

it allows us to determine whether the target mention corresponds to a candidate title.


We link certain mentions directly to their exact matching titles in Step 3 when there is very low ambiguity.

when no title is known for a mention that is relatively long and fuzzily matches the lexically retrieved title, we perform this aggressive linking.


We have demonstrated that, by incorporating textual relations and semantic knowledge as linguistic constraints in an inference framework, it is possible to significantly improve Wikification performance.

we have shown that our system is capable of making "intelligent" inferences that makes use of basic text understanding and has the ability to reason with it and verify it against relevant information sources.


Our goals of this work are to (1) explore the gender bias in the use of subjective language in social media, and (2) incorporate this bias into models to improve sentiment analysis for English, Spanish, and Russian.

in this paper we: • investigate multilingual lexical variations in the use of subjective language, and cross-cultural emoticon and hashtag usage on a large scale in Twitter data; 2 • show that gender bias in the use of subjective language can be used to improve sentiment analysis for multiple languages in Twitter.


are positive and negative polarity scores estimated using Eq.2 -3 such as: The gender-dependent rule-based classifiers are defined in a similar way.

⃗ f is replaced by ⃗ f M and ⃗ f F in Eq.5 and ⃗ .


Due to limited space, we only list the strict F-measure of opinion target extraction evaluated on opinioned sentences (same experimental setup as Table 3).

when w i is equal to 0, only explicit candidates are considered.


Empirical NLP, we propose, has a role to play in creating a more compact and easilyinterpretable way to understand the opinion space.

we envision applications to computational journalism, where there is high demand for transformation of and pattern discovery in unmanageable, unstructured, evolving data (including text) to inform the public (Cohen et al., 2011).


We view topics as lexicons, and propose that grounding a topic model with evidence beyond bags of words can lead to more lexicon-like representations.

our generative topic model grounds topics using the hierarchical organization of arguments within Debatepedia.


Motivated by past efforts to exploit prior knowledge (Zhao et al., 2010;Lin and He, 2009), we use the OpinionFinder sentiment lexicon 9 (Wilson et al., 2005) to construct η i and η o .

terms w in the lexicon were given parameters η i w = η o w = 0.01, and other terms were given η i w = η o w = 0.001, capturing our prior belief that opinion-expressing terms are likely to be used in expressing positions.


Although there has been a number of recent studies on event detection on Twitter, to the best of our knowledge, ours is the first that links the topic interests of users to their tweeting behaviors on events.

we propose a probabilistic latent variable model that identifies both topics and events on Twitter.


Generally speaking, RCRP assumes a Chinese Restaurant Process (CRP) (Blackwell and MacQueen, 1973) for items within an epoch and chains up the CRPs in adjacent epochs along the timeline.

in our case, the generative process can be described as follows.


We introduce a pseudo-observed binary variable r t,i for each tweet, where the value of r t,i is set to 1 for all tweets.

we assume that this variable is dependent on the hidden variables y and s. if y t,i is 0, i.e.


the latent topics, then basically we would like the topic distribution of this user to be close to that of the event.

we assume that each event k has associated with it an A-dimensional vector η k and a parameter η 0 k .


Intuitively, if a user's topic distribution is similar to the event-topic affinity vector of the event, then the user is likely to be interested in the event.

we use the first two months' data (April and May 2012) as training data to learn all the users' topic distributions.


We instead adopt a collaborative filtering strategy and rank the test users by their similarity with those training users who have tweeted about the event.

each of these methods produces a topic distribution θ u for each user.


In Table 5 we show a few highly related events for a few popular topics in our Twitter data set.

given a topic a we rank the meaningful events that contain at least 70 tweets based on η k,a .


Moreover, an authorship attribution system of tweets may have various applications.

a range of cybercrimes can be addressed using such a system, including identity fraud and phishing.


The two-dimensional work arrays must be stored in "symbol-major" order for this to work.

the parent VP for one work item is stored next to the parent VP for the next work item, while the VP symbol for the first work item is stored on the next "row" of the work array.


Unlike the workspace arrays, the arrays for the parse charts are stored in "span-major" order, transposed from how they are stored in the workspace arrays.

for a given span, the NP symbol is next to the same span's VP symbol (for example).


The goal is then i) to predict how well a particular candidate fits in the original context, and ii) given these predictions for each of the candidates, to correctly order the elements of the candidate set according to their contextual fitness.

a model is successful if it prioritizes plausible substitutions ahead of less likely synonyms (given the context).


The improvements are significant (using paired t-test, p < 0.01) for 3 out of 4 settings for EXPENS and in all settings for LAMBDAMART.

the results of LAMBDAMART are so far the best scores that have been reported for the best studied setting, i.e.


We propose a novel threelayer graph model, based on manifold ranking (Zhou et al, 2003a;2003b), to jointly model deceptive language at the offering-, review-and term-level.

rather than treating reviews within the same offering as independent units, there is a reinforcing relationship between offerings and reviews.


Accordingly, we propose a dimensionality-reduction approach, based on Latent Dirichlet Allocation (LDA) (Blei et al, 2003), to obtain a vector representation of reviews for the ranking algorithm that generalizes across reviews of different offerings.

we train an LDA-based topic model to view each review as a mixture of aspect-, city-, hotel-and review-specific topics (see Section 6).


We evaluate our approach on the task of identifying (ranking) manipulated hotels.

in the absence of gold standard offering-level labels, we introduce a novel evaluation procedure for this task, in which we rank numerous versions of each hotel, where each hotel version contains a different number of injected, known deceptive reviews.


We therefore investigate an LDA-based dimensionality-reduction approach (RLDA) to derive effective vector representations of reviews.

we model each document as a bag of words, generated from a mixture of: (a) "aspect" topics (that discuss various dimensions of the offering); (b) city-specific topics; (c) hotel-specific topics; (d) review-specific topics; 4 and (e) a background topic.


Our model has a similar general structure to standard LDA, but with additional machinery to handle different levels of information.

in order to model K aspects in a collection of R reviews, 4 These will be terms used in just a small number of reviews.


To qualitatively evaluate the ranking results, we use the Normalized Discounted Cumulative Gain (NDCG), which is commonly used to evaluate retrieval algorithms with respect to an ideal relevance-based ranking.

nDCG rewards rankings with the most relevant results at the top positions (Liu, 2009), which is also our objective, namely, to rank versions that have higher proportions of deceptive reviews nearer to the top.


We observe that our approach (using 2, 5 or 10 topics) generally outperforms the other approaches.

approaches that use our RLDA text representation (OUR APPROACH, TWO-LAYER MANIFOLD (R), and REVIEW-SVR+LDA (R)), which tries to remove city-and hotel-specific information, perform better than those that use the full vocabulary (REVIEW-SVR, TWO-LAYER MANIFOLD (S), and THREE-LAYER MANIFOLD (TF-IDF)).


The data we have does not contain the information on the reaction of the user to the recommendation.

we do not know if the textual argumentation would have been sufficient for convincing Theo6 to see the film.


These challenges have prompted some researchers to move away from MERT, in favor of linearly decomposable approximations of the evaluation metric (Chiang et al., 2009;Hopkins and May, 2011;Cherry and Foster, 2012), which correspond to easier optimization problems and which naturally incorporate regularization.

recent work (Chiang et al., 2009) has shown that adding thousands or tens of thousands of features can improve MT quality when weights are optimized using a margin-based approximation.


e s,M } corresponds to solving the following optimization problem: The function in this equation is piecewise linear (Papineni, 1999), which enables an efficient exhaustive computation.

this function is optimized by enumerating the up to M hypotheses that form the upper envelope of the model score function.


While some variants have been considered (Och, 2003;Flanigan et al., 2013), we use an expected BLEU approximation, assuming hypotheses are drawn from a log-linear distribution according to their parameter values (Smith and Eisner, 2006).

we assume the probability of a translation candidate e s,m is proportional to (exp (w h s,m )) µ , where w are the parameters being optimized, h s,m is the vector of the features for e s,m , and µ is a scaling parameter.


The purpose of adding noise is to make the optimization task more realistic.

after computing all pseudo-BLEU scores, we added noise to each feature vector h s,m by drawing from a zero-mean Gaussian with standard deviation 200.


The most stable voxels were those showing consistent variation across the different stimuli between scanning sessions.

and following a similar strategy to Mitchell et al.


As a second transformation, we apply dimensionality reduction to the two matrices.

we adopt the Singular Value Decomposition (SVD), one of the most effective methods to approximate the original data in lower dimensionality space (Schütze, 1997), and reduce the vectors to 50 dimensions.


Representational similarity analysis circumvents the previous problems by abstracting each fMRI/distributional data source to a common structure capturing the interrelationships between each pair of data items (e.g., words).

for each model/participant's fMRI data/anatomical region, the similarity structure was evaluated by taking the pairwise correlation (Pearson's correlation coefficient) between all unique category or word combinations.


It is however noteworthy that we can observe some interpretable selectivity in lobe*model combinations.

the Context model better matches the parietal lobe than the Object model, which in turn better captures the occipital and temporal lobes (Observations are qualitative).


per token (TpT) for our unlabeled corpora.

for each token, we counted the number of tags licensed by the dictionary, or all tags for word forms not in the dictionary.


Our notion of a document is meant in its most general form as a string of raw unstructured text.

the interestingness function should not rely on any document structure such as title tags, hyperlinks, etc., or Web interaction data.


Aggregate clicks can therefore serve as a proxy for interestingness.

for a given source document, target documents that attract the most clicks are more interesting than documents that attract fewer clicks 1 .


Thus, in our experiments, We collect a large dataset of user browsing events from a commercial Web browser.

we sample 18 million occurrences of a user click from one Wikipedia page to another during a one year period.


Intuitively, we want to learn to maximize ∆.

the DSSM is learned to represent documents as points in a hidden interestingness space, where the similarity between a document and its interesting documents is maximized.


To remedy this problem, in (Hochreiter and Schmidhuber, 1997) it was suggested to modify the architecture of a standard RNN in such a way that vanishing and exploding gradients are avoided during backpropagation.

no modification of the training algorithm is necessary.


In the last few years, neural networks have become increasingly popular in NLP applications.

neural language models (Bengio et al., 2003;Mnih and Hinton, 2007;Collobert and Weston, 2008) have demonstrated impressive performance at the task of language modeling.


First of all, we would like to investigate how our neural network approach might be improved by incorporating information from other sources.

we think of initializing our embedding matrices with distributed representations that come from a large-scale neural language model (Mikolov et al., 2013).


Finally, we want to investigate more advanced neural network architectures for the acquisition of selectional preferences.

neural tensor networks (Yu et al., 2013) have recently demonstrated impressive results in related fields like speech recognition, and might provide the necessary machinery to model multi-way selectional preferences in a more profound way.


Standard se-mantic space models extract meanings solely from linguistic data, even though we know that human semantic knowledge relies heavily on perceptual information (Louwerse, 2011).

there exists substantial evidence that many concepts are grounded in the perceptual system (Barsalou, 2008).


Since players are encouraged to produce as many terms per image, the dataset's increased coverage is at the expense of accuracy in the word-to-image mapping: a dog in a field with a house in the background might be a golden retriever in ImageNet and could have tags dog, golden retriever, grass, field, house, door in the ESP Dataset.

images in the ESP dataset do not make a distinction between objects in the foreground and in the background, or between the relative size of the objects (tags for images are provided in a random order, so the top tag is not necessarily the best one).


In the second step, we identify argumentative relations by classifying a pair of argument components as either support or non-support.

the contributions of this work are the following: First, we introduce a novel approach for identifying argumentative discourse structures.


This corpus contains annotations of argument components at the clause-level as well as argumentative relations.

it includes annotations of major claims, claims and premises, which are connected with argumentative support and attack relations.