Although it does not make sense to compute n r or n r+ on fractional counts, it does make sense to compute them on possible completions.

theoretically, this requires us to derive a new estimate for D. as this is not trivial, nearly all implementations simply use the original estimate (4).


Notice that the tokens w dk in document d are exchangeable: by collapsing out ψ d , we can regard them as having been generated from a CRP.

we use 20 iterations of EM with 100 samples per E-step for training, and use 1000 samples after training to estimate the posterior.


This model seems to be stupid, since we can easily notice that the bigram "is is" is quite frequent, and the two bigrams "is a" and "a is" have the same frequency.

we shall start with an analysis of training-set perplexity, since we can derive an exact formula for it, which will give us a sufficient idea for making an approximate analysis of testset perplexity.


This paper studies the idea of removing low-frequency words from a corpus, which is a common practice to reduce computational costs, from a theoretical standpoint.

comparing Zipf and Reuters, we see that C k is poorly estimated when the gram size is large, whereas π k is roughly correct.


Although these two cases might be unrealistic, the actual (theoretical) perplexity is expected to be between their values.

we conjecture that the word error rate also has a similar tendency as perplexity with respect to the reduced vocabulary size.


N-gram features are considered as more ambiguous compared to traditional lexical and syntactic features, and may introduce incorrect predictions, thus improving the recall at the cost of precision.

how we determine which relations expect certain types of subjects, which relations expect certain types of objects, etc.


On one hand, this result shows that using more unlabeled data can further improve the result.

we are not aware of other available medical relation extraction datasets that fit for our evaluations.


The matrix rank minimization problem is NPhard.

figure 5 demonstrates that the ranks of data matrices are approximately 2,000 for the initial optimization of DRMC-b and DRMC-1.


We first introduce features derived internal to the translation decoding process as well as externally from the source sentence analysis.

in this paper we propose an adaptive quality estimation that predicts sentence-level humantargeted translation error rate (HTER) (Snover et al., 2006) for a document-specific MT post-editing system.


Interactive scenarios have been used for evaluation purposes of translation systems for nearly 50 years, especially using human reading comprehension testing (Pfafflin, 1965;Fuji, 1999;Jones et al., 2005), and more recently, using face-toface conversation mediated via machine translation (Sakamoto et al., 2013).

interactive scenarios have been used for evaluation purposes of translation systems for nearly 50 years, especially using human reading comprehension testing (Pfafflin, 1965;Fuji, 1999;Jones et al., 2005), and more recently, using face-toface conversation mediated via machine translation (Sakamoto et al., 2013).


A pattern is generated from each complete match returned by LOOKUP after applying a simple heuristic: for each verb node v in the match, we enforce that negations and auxiliaries in T depending from x are also included in the pattern.

similarly, we include the parent of a noun in N if the dependency relation between the node and its parent is listed in N p .


However, the utility of such a representation for summarization depends on the quality of pattern clusters.

the first step consists in the linearization of the input tree t .


For each method, we have manually reviewed the 25 items where the difference between readability and informativeness was the largest, to understand in which cases grammatical, yet irrelevant compressions are produced.

the utility of such a representation for summarization depends on the quality of pattern clusters.


(Section 5.2) • How informative are the hierarchical summaries compared to the other methods?

each node in the ADG is a sentence from the dataset.


We cannot know a priori the number of clusters for a given topic.

sUMMA produces a hierarchy of relatively short summaries, in which the top level provides a general overview and users can navigate the hierarchy to drill down for more details on topics of interest.


After acquiring the timestamps, we must hierarchically cluster the sentences into sets that make sense to summarize together.

when there is little differentiation in news coverage, we prefer clusters evenly spaced across time.


The task is related to exploratory search (Marchionini, 2006).

we conclude with evaluation and discussion.


Evaluation indicates the algorithm improves on strong baselines.

to queryfocused summarization, the context of a sum-mary is not a single query, but the set of queries that led to the current step, their result sets and the corresponding summaries.


(R1) however talks about events which happened before 1 The produced summaries are truncated to fit within a 100-word limit imposed by the TAC-2011 guidelines.

we believe it is because the ROUGE measures that are used for evaluation are not suited for this purpose.


A distinguishing feature of the tectogrammatical analyses, vis a vis the other treebanks we are using, is that semantically empty words only take part in the analytical annotation layer and nodes are inserted at the tectogrammatical layer to represent covert elements of the sentence not present in the surface syntax of the analytical layer.

a pre-order traversal would result in tokens close to the root having few options, and in particular if the root has a single child, that node has no possible new heads unless one of its children has been assigned the root as its new head first.


The behaviour of α on the other hand is more complex, with structural noise being penalised harder than perturbations of the labels.

the third distance function, δ norm , takes into account a slightly different concern; namely that when comparing a long sentence and a short sentence, the distance has to be quite large simply to account for the difference in number of nodes, unlike comparing two short or two long sentences.


the Norwegian National Library for data from the NDT, and Emily Bender at the University of Washington for the SSD data.

5 the child nodes of a node are all processed before the node itself.


We calculated precision as the average ratio of correct hypernym lemmas (senses) to the total number of lemmas (senses) returned for all the pages in the dataset, recall as the number of correct lemmas (senses) over the total number of lemmas (senses) in the dataset, and coverage as the fraction of pages for which at least one lemma (sense) was returned, independently of its correctness.

wiBi provides over 3M hypernym pages chosen from a range of 94k distinct hypernyms, while others exhibit a considerably smaller range of distinct hypernyms (e.g., DBpedia by design, but also wikiNet, with around 11k distinct page hypernyms).


(2013) actually reported accuracy on this dataset.

by counting how many times each relation R was annotated, we can estimateP (R) and P (r).


The open-domain YA model learns to place more weight on LS features, which are unable to provide the same utility in the biology domain.

the combined dataset contains 7.7M words.


During composition we retain intermediate paths like M i 23 utilizing the ability to do lazy composition (Mohri and Pereira, 1998) in order to facilitate beam search through the multi-alignment.

we build our final machine by composing the three sub-machines as, where i = c or i = c + t. This gives us a combined weighted graph by mapping the output symbols of the first medical event sequence to the input symbols of the second medical event sequence.


Semantically similar phrases can be syntactically different.

for this task, annotators were asked to judge if a given article is objective or subjective.


Since all of the baselines we compared against assume alternatives are provided, we apply all methods to the same set of alternative fact candidates discovered by our method.

64% of trustworthy articles were independently labeled as objective.


Details of the resulting dataset are shown in the row "WKP Politicians" in Table 3. iii) General Knowledge Quiz: The third dataset consists of questions from a general knowledge quiz 4 .

additional aspects of a source influence its trustworthiness, besides external votes.


While prior methods assume the alternatives are known apriori, we developed a method for generating alternative fact candidates.

the number of places where a person can be born (one) is much fewer than the number of people that can die in a place (many).


We next describe our algorithm for detecting sense change from these sets of sense clusters.

similarly, for a candidate sense change from tv i to tv j , we say that the location of the sense change is tv j if and only if that sense change does not get detected by comparing tv i with any time interval tv k , intermediate between tv i and tv j .


We hypothesize that each different cluster corresponds to a particular sense of the target word.

the source time period here is 1909-1953.


Then, we remove the top 20% and the bottom 20% words from this list.

if a word undergoes sense change, this can be detected by comparing the sense clusters obtained from two different time periods by the algorithm outlined above.


Materna applied an LDA-based method to the BNC, which contains 1.4M verb uses, to induce seman-tic frames across verbs that can be considered to be verb classes (Materna, 2012;Materna, 2013).

we set the hyper-parameters α in (1) and β in (3) to 1.0.


The Web n-grams corpus has broad coverage but is limited to up to 5-grams, so it may not contain pattern-based evidence for various longer multiword terms and pairs.

this procedure only finds approximate marginal beliefs, and is not actually guaranteed to converge, but in practice can be quite effective for finding workable marginals in models for which exact inference is intractable, as is the case here.


For each edge variable y ij in the model, there is a corresponding factor E ij (small blue squares in Figure 2) that depends only on y ij .

currently available taxonomies such as WordNet are incomplete in coverage (Pennacchiotti and Hovy et al., 2009), unavailable in many domains and languages, and time-intensive to create or extend manually.


More specifically, we approach unsupervised constituent parsing from the perspective of structure learning as opposed to parameter learning.

we provide theoretical guarantees on the recovery of the correct underlying latent tree and characterize the associated sample complexity under our technique.


The Chow-Liu algorithm essentially computes the distances among all pairs of variables (the negative of the mutual information) and then finds the minimum cost tree.

we associate each sentence with an undirected latent tree graphical model, which is a tree consisting of both observed variables (corresponding to the words in the sentence) and an additional set of latent variables that are unobserved in the data.


For vowels, VM measures how well the inferred phonetic categorizations match the gold categories; for lexemes, it measures whether tokens have been assigned to the same lexemes both by the model and the gold standard.

the word-topic assignments are used to calculate unsmoothed situation-topic distributions θ used by the tLD model.


Under this view, contextual semantics is available to infants well before they have acquired large numbers of semantic minimal pairs.

although young children also have trouble with minimal pairs (Stager and Werker, 1997;Thiessen, 2007), the LD model may overestimate the degree of the problem.


A DP is parametrized as DP (α, H), where α is a real-valued hyperparameter and H is a base distribution.

if the learner hears kV 1 t and kV 2 t in different situational contexts, they are likely to be different lexical items (and V 1 and V 2 different phones), despite the lexical similarity between them.


Computational modeling provides a way to test the computational level of processing (Marr, 1982).

(2009), which shows that the current model provides comparable developmental modeling benefits to the BabySRL models.


Position relative to verb Probability Position relative to verb Figure 2: Visual representations of (Left) the initial model's expectations of where arguments will appear, given the initial parameters in Table 2 and (Right) the converged model's expectations of where arguments will appear.

3 Even though the infants had no extralinguistic knowledge about the verb, they consistently treated the verb as transitive if two nouns were present and intransitive if only one noun was present.


These positions may correspond to different semantic roles for different predicates (e.g.

the initial model conditions (see Figure 2) are most likely to realize an SVO ordering, although it is possible to obtain SOV (by sampling a negative number from the blue curve) or even OSV (by also sampling the red curve very close to 0).


The Wolof data has 36 input forms, each of the form V 1 tV 2 , where V 1 and V 2 are vowels that agree in height.

mEOT also is motivated by the general maxEnt framework, whereas most other OT formulations are ad hoc constructions specific to phonology.


Output EtE violates PARSE [atr], and has score 8.

6 Since data, matrix, and weight likelihoods all shape the learned constraints, there must be enough data for the model to avoid settling for a simple matrix that poorly explains the data.


While objectivity remains an important principle of journalistic professionalism, scholars and watchdog groups claim that the media are biased (Groseclose and Milyo, 2005;Gentzkow and Shapiro, 2010;Niven, 2003), backing up their assertions by publishing examples of obviously biased articles on their websites.

in this section, we describe a supervised RNN model for bias detection and highlight differences from previous work in training procedure and initialization.


To our knowledge, their semantic reordering models were PAS-specific.

for every two adjacent constituents XP i and XP i+1 with corresponding visit order v i and v i+1 , their reordering could be one of the following: Up to this point, we have generated a sequence of leftmost reordering types LRT = {lrt 1 , .


We learnt the generalized language models on the same split of the training corpus as the standard language model using modified Kneser-Ney smoothing and we also used the same set of test sequences for a direct comparison.

the authors compared Kneser-Ney smoothing, Katz backoff smoothing, caching, clustering, inclusion of higher order n-grams, sentence mixture and skip n-grams.


The main questions we ask are: how is the proposed model different from standard text regression/classification models?

depending on the amount of interactions in the question answering session, the complexities of the calls vary.


To evaluate the performance of our approach, we compare with a standard squared loss linear regression baseline, as well as strong baselines such as linear and non-linear support vector machines (SVMs) that are widely used in text regression tasks.

by considering the semiparametric case, we not only obtain some expressiveness from the nonparametric models, but also reduce the complexity of the task: we are only interested in the finite-dimensional components Σ in the Gaussian copula with O(n log n) complexity, which is not as computationally difficult as the completely nonparametric cases.


Domain Adaptation for SMT Training a SMT system using diverse data requires domain adaptation.

machine translation uses inherently multilingual data: an SMT system must translate a phrase or sentence from a source language to a different target language, so existing applications of topic models (Eidelman et al., 2012) are wilfully ignoring available information on the target side that could aid domain discovery.


The features are designed to share syntactic information about a word across its distinct semantic realizations in order to transfer syntactic information from CCGbank to semantic parsing.

our approach is most closely related to the distantly-supervised approach of Krishnamurthy and Mitchell (2012).


Both ASP and ASP-SYN perform reasonably well, within 2.5% of the performance of C&C at the same coverage level.

we conducted a second evaluation to measure these quantities.


This paper proposes a novel and effective method for the construction of semantic hierarchies based on word embeddings, which can be used to measure the semantic relationship between words.

many researchers * Email correspondence.


M Emb and M CilinE can also be combined.

their basic assumption that a hyponym can only be used in contexts where its hypernyms can be used and that a hypernym might be used in all of the contexts where its hyponyms are used may not always rational.


However, this produces an intractably large network.

1 Interestingly, this characterization echoes the principle of compositionality, which states that the meaning of a phrase is uniquely determined by the meaning of its parts and the rules that connect those parts.


For our example, the rule is: ∀x, y. guy(x) ∧ agent(y, x) ∧ walk(y) → result().

pSL employs a "lazy" approach to grounding that avoids the construction of irrelevant groundings.


The purpose of such a model is three-fold: i) to cover the content of query information optimally; ii) to generate a more readable and grammatical sentence; and iii) to favor strong connections between the concepts.

for example, "How were the bombings of the US embassies in Kenya and Tanzania conducted?


In order to generate such a sentence, we need to identify which sentences from the original document should be extracted and combined to generate abstract sentences.

the chat dataset results demonstrate the highest scores: 73% of the sentences generated by our phrasal query abstraction model are grammatically correct and 24% of the generated sentences are almost correct with only one grammatical error, while only 3% of the abstract sentences are grammatically incorrect.


Meeting: we use the AMI meeting corpus (Carletta et al., 2005) that consists of 140 multiparty meetings with a wide range of annotations, including generic abstractive summaries for each meeting.

in the stand alone version (extractive system) we limit the number of retrieved sentences to the desired length of the summary.


This means our systems can effectively aggregate the extracted sentences and generate abstract sentences based on the query content.

it is likely that most of the paths are not readable.


In fact, it would be of interest to investigate multi-objective optimisation techniques that can balance the needs of the lecturers to convey important content to the satisfaction of students.

for instance, the set of labels L = {temp 0 , temp 1 , ...temp 28 } could be transformed to {temp 0,1,2 , temp 28,3,17, ...}.


There is evidently a mismatch between the rules and the test-set; the content selection rules are based on heuristics provided by a L&T Expert rather than by the same pool of lecturers that created the test-set.

mL classification could be used when the goal of the generation is to replicate phenomena seen in the dataset, because it achieves high accuracy, precision and recall.


The structural configurations y and z are non-degenerate, i.e, the bigram configuration y represents an acyclic path while the dependency configuration z forms a tree.

in addition, the token configuration x can be rewritten in the form of a weighted combination of α(y) and β(z) to ensure its consistency with y and z.


Comparing the two approximation strategies shows a clear performance advantage for DP+LP over DP+LP→MST: the latter approach entails slower inference due to the overhead of running the Chu-Liu Edmonds algorithm at every dual update, and furthermore, the error introduced by approximating an integral solution re-sults in a significant decrease in dependency recall.

furthermore, much recent work has focused on the challenge of joint sentence extraction and compression, also known as compressive summarization Berg-Kirkpatrick et al., 2011;Almeida and Martins, 2013;Li et al., 2013;Qian and Liu, 2013), in which questions of efficiency are paramount due to the larger problems involved; however, these approaches largely restrict compression to pruning parse trees, thereby imposing a dependency on parser performance.


We focus in this work on a sentence-level compression system to approximate the ILP-based inference of Thadani and McKeown (2013) which does not restrict compressions to follow input parses but permits the generation of novel dependency relations in output compressions.

the effect of these constraints is diminished when solving an LP relaxation of the above problem.


Some issues remain problematic even for the structural model.

some comments do not contain any explicit positive or negative opinions, but provide detailed and well-argumented criticism, for example, this phone is heavy.


Intuitively, keyphrase extraction is about finding the important words and phrases from a document.

many existing approaches have a separate, heuristic module for extracting candidate keyphrases prior to keyphrase ranking/extraction.


TPR's superior performance strengthens the hypothesis of using topic clustering for keyphrase extraction.

the foreground corpus is composed of the set of documents from which keyphrases are to be extracted.


The features commonly used to represent an instance for supervised keyphrase extraction can be broadly divided into two categories.

we randomly selected 25 documents from each of these 3 A more detailed analysis of the results of the SemEval-2010 shared task and the approaches adopted by the participating systems can be found in Kim et al.


Intuitively, a phrase that has high scores for phraseness and informativeness is likely to be a keyphrase.

if a candidate phrase c 1 is more representative than another candidate phrase c 2 , c 1 should be preferred to c 2 .


1 This dictionary is intended to identify the prototypical syntagmatic patterns with which prepositions in use are associated, identifying linguistic units used sequentially to make well-formed structures and characterizing the relationship between these units.

the data in any of the fields may be altered (with the menu bar changing color to red) and then saved to the underlying databases.


Finally, we discuss works related to this study.

this suggests that both linear and polynomial regressions are suitable as a preprocessing step of the standard approach, while the logistic regression seems to be inappropriate according to the results shown in table 6.


Especially in the paid condition, a clear group appears in the top five positions, which were advertised as receiving prizes.

quality Enforcement Mechanisms Infection includes two game mechanics to limit adversarial players from creating many low quality annotations.


To compare with the video games, items were annotated using two additional methods: crowdsourcing and a non-video game with a purpose.

for both games, players were equally likely to select novel items, suggesting the games can serve a useful purpose of adding these missing relations in automatically constructed knowledge bases.


First, the annotation task should be a central and natural action with familiar video game mechanics.

based on agreement with the gold standard (Table 1, Col. 5), the estimated cost for crowdsourcing a correct true positive annotation increases to $0.014 for a concept-image and a $0.048 for concepts-concept annotation.


Design Infection is designed as a top-down shooter game in the style of Commando.

the adjudication process resolved these disputes, resulting in substantial agreement by all annotators on the final gold annotations.


Webgathered images were retrieved using Yahoo!

the annotation should be supplied by common actions such as collecting items, puzzles, or destroying objects, rather than through extrinsic tasks that players must complete in order to return to the game.


Instead of focusing on grammatical errors that are found to be highly representative of language proficiency, our interest is in capturing the range of forms that surface in language production and the degree of sophistication of such forms, collectively referred to as syntactic complexity in (Ortega, 2003).

the performance drop (around 0.05) resulting from a shallow analysis is relatively small compared to the drop observed while employing a deep syntactic analysis.


Another interesting finding is that, although the arceager algorithm achieves lower accuracies in the word-based pipeline models, it obtains comparative accuracies in the character-level models.

the definitions of AL c , AR c and SH c are the same as the word-based arc-standard model, while PW changes the top element on the stack into a full-word node, which can only take interword dependencies.


We can see that improved POS-tagging accuracies are achieved using the real inter-word dependencies when jointly performing inner-and inter-word dependencies.

we thank the anonymous reviewers for their constructive comments, and gratefully acknowledge the support of the National Basic Research Program (973 Program) of China via Grant 2014CB340503, the National Natural Science Foundation of China (NSFC) via Grant 61133012 and 61370164, the Singapore Ministry of Education (MOE) AcRF Tier 2 grant T2MOE201301 and SRG ISTD 2012 038 from Singapore University of Technology and Design.


Most verb errors are due to incorrect morpheme extraction by Morfessor.

we also compare our models with the case that there is much more training data and we do not do vocabulary expansion at all.


most noun errors result from affixes that are only compatible with a subset of all possible noun stems.


This representation contributes to reduce the negative impacts of erroneous "chars-to-word" alignments.

gP is integrated with estimation of parametric structural model.


They score a 1000-best list using only their model and are able to achieve the same BLEU as using all 12 standard MT features (21.8 vs 21.7).

8 By combining self-normalization and precomputation, we can achieve a speed of 1.4M lookups/second, which is on par with fast backoff LM implementations (Tanaka et al., 2013).


The updates also illustrate how γ balances the effect of the MST component of the score relative to the low-rank tensor score.

to add auxiliary word vector representations, we use the publicly available word vectors (Cirik   and Ş ensoy, 2013), learned from raw data (Globerson et al., 2007;Maron et al., 2010).


We can alternatively specify arc features in terms of rank-1 tensors by taking the Kronecker product of simpler feature vectors associated with the head (vector φ h ∈ R n ), and modifier (vector φ m ∈ R n ), as well as the arc itself (vector φ h,m ∈ R d ).

traditionally, parsing research has focused on modeling the direct connection between the features and the predicted syntactic relations such as head-modifier (arc) relations in dependency parsing.


Accurate scoring of syntactic structures such as head-modifier arcs in dependency parsing typically requires rich, highdimensional feature representations.

the resulting embedding is therefore tied to the syntactic roles of the words (and arcs), and learned in order to perform well in parsing.


As a result, the objective will be jointly convex with respect to θ and U and could be optimized using standard tools.

the model was evaluated on 14 languages, using dependency data from CoNLL 2008 and CoNLL 2006.


∆θ and ∆U ) during each update but does not include any overall regularization.

we then obtain parameter increments ∆θ and ∆U by solving min ∆θ, ∆U, ξ≥0 In this way, the optimization problem attempts to keep the parameter change as small as possible, while forcing it to achieve mostly zero loss on this single instance.


We present CoSimRank, a graph-theoretic similarity measure that is efficient because it can compute a single node similarity without having to compute the similarities of the entire graph.

we do not compare against this new measure as it uses the graph Laplacian and so cannot be computed for a single node pair.


(2006) introduce a similarity measure that is also based on the idea that nodes are similar when their neighbors are, but that is designed for bipartite graphs.

the counteracting effects of fewer iterations and more vector similarity computations can give either CoSimRank or PPR+cos an advantage, as is the case for synonym extraction and lexicon extraction, respectively.


The source code of our system PARASEMPRE is released at http://www-nlp.stanford.edu/ software/sempre/.

the basic template is a join of a binary and an entity, where a binary can either be one property p.e (#1 in the table) or two properties p 1 .p 2 .e (#2).


All tokens and concepts are lowercased before matches or fuzzy matches are done.

each stage is a discriminatively-trained linear structured predictor with rich features that make use of part-ofspeech tagging, named entity tagging, and dependency parsing.


In addition, akin to the first context function, we also added all dependency labels to the context set.

in order to speed up learning, we use an unnormalized output layer and a hinge-loss objective.


We noted that this renders every lexical unit as seen; in other words, at frame disambiguation time on our test set, for all instances, we only had to score the frames in F for a predicate with lexical unit (see §3 and §5.2).

for this context function, the block cardinality k was the sum of the number of scanned gold dependency path types and the number of dependency labels.


This is normally intractable in practice.

there are 67,723 and 4,348 unique word types in the training and test data after the preprocessing step.


According to them, there are 3 significant differences between topic-based WSI and generic topic modeling.

to the standard WSD formulation, Vickrey et al.


We assume that this property would fit with a word alignment task, and we propose an RNN-based word alignment model.

these models are roughly clustered into two groups: generative models, such as those proposed by Brown et al.


Every bidirectional alignment with finite objective score must align exactly one word in e to each word in f, encoding a permutation a.

first, a directional alignment is found from each word in e to a word f .


In their work, bilingual word embedding is trained to capture lexical translation information, and surrounding words are utilized to model context information.

the length of the phrase pair embedding is 20 × 4 = 80 .


We now focus on the structure independent features and experiment with their predictive value, according to types.

this validates our hypothesis that using the post structure results in better modeling of instructor's intervention.


Both models assume that posts of a thread structure it in form of a story or a "chain of events."

this representation is not natural for the MOOC domain where discussions are relatively more focused on the thread topic and are better organized using sections within the forums.


Each pinyin sylla-ble has a bunch of corresponding Chinese characters which share the same pronunciation represented by the syllable.

figure 8 shows the results on D with different N s, of which the N axis is drawn in logarithmic scale.


From our statistics on T, with 2% randomly generated typos, P r(L(S ′ , S) < 2) = 99.86%.

the vertex set V consists of the following parts: • Virtual start vertex S 0 and end vertex S E ; • Possible legal syllables fetched from dictionary D p according to the input pinyin sequence: • the letter itself as a fallback no matter if it is a legal pinyin syllable or not: the vertex weights w S are all set to 0.


In order to obtain an optimal result as measured by P C, we jointly tune the c i parameters to optimize the P C value achieved by our system on the same held-out validation data.

to do this, we train five maximum entropy classifiers for each prompt, one for each of the five thesis clarity errors, using MALLEt's (McCallum, 2002) implementation of maximum entropy classification.


We use these maximum likelihood probabilities for label assignment; for each node i, we assign the label To completely define our algorithm, we need to instantiate the potentials Ψ, in particular the priors and the compatibilities, which we discuss next.

what is more, it gives us a means to obtain connotation labels for the synsets themselves, which we use in the evaluations in the next sections.


For non-polysemous words, which constitute a significant portion of English vocabulary, learning the general connotation at the word-level (rather than at the sense-level) would be a natural operational choice.

in this classification task, we denote by Y the nodes the labels of which need to be assigned, and let y i refer to Y i 's label.


The key idea is that after enough iterations of message passes between the nodes, the "conversations" are likely to come to a consensus, which determines the marginal probabilities of all the unknown variables.

although one can use this sense in both positive and negative contexts, this sense of "abound" seems to collocate more often with items that are good to be abundant (e.g., "resources"), than unfortunate items being abundant (e.g., "complaints").


to this end, we extend the existing word embedding learning algorithm (Collobert et al., 2011) and develop three neural networks to effectively incorporate the supervision from sentiment polarity of text (e.g.


SSWE h is trained by predicting the positive ngram as [1,0] and the negative ngram as [0,1].

based on the above observation, the hard constraints in SSWE h should be relaxed.


Restaurant-Employee: We asked employees from selected restaurants (a waiter/waitress or cook) to each write positive-sentiment reviews of their restaurant.

one explanation for these observations is that informative (truthful) writing tends to be more introductory and descriptive, encoding more concrete details, when compared with imaginary writings.


Hotel-Customer: We borrowed from Ott et al.

recent studies show that deceptive opinion spam is not easily identified by human readers (Ott et al., 2011).


POS and LIWC features obtain around 0.5 precision and 1.0 recall, indicating that all doctor reviews are classified as deceptive by the classifier.

the two-class classifier is around 0.76 accurate in distinguishing between Turker and Employee reviews, despite both kinds of reviews being deceptive opinion spam.


In situated human-robot dialogue, humans and robots have mismatched capabilities of perceiving the shared environment.

its probabilistic labeling procedure can generate multiple grounding hypotheses to facilitate follow-up dialogue.


This fact suggests that a dialog system should be also capable of conducting multi-topic conversations with users to provide them a more natural interaction with the system.

the other direction of dialog topic tracking approaches made use of external knowledge sources including domain models (Roy and Subramaniam, 2006), heuristics (Young et al., 2007), and agendas (Bohus and Rudnicky, 2003;Lee et al., 2008).


This paper presented a composite kernel approach for dialog topic tracking.

the dialog topic at each turn can be determined not only by the user's intentions captured from the given utterances, but also by the system's decisions for dialog management purposes.


B-cubed and CEAF treat entities as sets of mentions and measure the agreement between key (or gold standard) entities and response (or system-generated) entities, while MUC and BLANC-gold are link-based.

rand Index is defined as the ratio between the number of correct within-cluster links plus the number of correct cross-cluster links, and the total number of links.


Once we have the partitions, then we compute the MUC score by: where K i is the i th key entity and p(K i ) is the set of partitions created by intersecting K i with response entities (cf.

we present our newly updated coreference scoring package in Section 3 together with the rescored CoNLL-2011/2012 outputs.


The effort required by a human annotator to detect sentiment is not uniform for all texts.

"simple time measurement" is not reliable because the annotator may spend time not doing any annotation due to fatigue or distraction.


According to this measure, citations are most similar to the books domain.

the dataset has 1768 citation sentences with polarity annotation: 190 are labeled as positive, 57 as negative, and the vast majority, 1521, are left neutral.


For the compreligion-sci dataset TS-LDA and ClassifyLDA have the same performance.

we extend Clas-sifyLDA algorithm by "sprinkling" topics to unlabeled documents.


Tree kernel is an effective technique for relation extraction.

for example, WordNet senses {New York#1, city#1, district#1, region#1, …} will be added as features to the [NN New York] node in figure 1.


We investigate word embeddings induced by two typical language models: Collobert and Weston (2008) embeddings (C&W) (Collobert and Weston, 2008;Turian et al., 2010) and Hierarchical log-bilinear embeddings (HLBL) (Mnih and Hinton, 2007;Mnih and Hinton, 2009;Turian et al., 2010).

the performance order across domains of the two baselines are the same.


SST and PTK would assign this pair a much lower score for having completely different ordering, but we suggest that cases such as these are very rare due to the relatively strict word ordering of English.

we tested one sub-task from TempEval-2013the extraction of temporal relations between an event and time expression within the same sentence.


This is estimated by the proportion of Wikipedia documents in that language which contain the word in question: where df (w i,l ) is the count of Wikipedia documents of language l which contain w i , and N l is the total number of Wikipedia documents in language l. In all experiments in this paper, we assume that we have access to at least one multilingual dictionary containing each of our target languages, but in absence of such a dictionary, sdict(w i,l ) could be set to 1 for all words w i,l in the language.

first, we present results over the synthetic dataset in Table 3.


Our goal is to use this feedback to improve the quality of the Wordnet synsets.

although our score is not in terms of percentage, we obtain the average score of 3.78/5.00 (or informally and possibly incorrectly, 75.60% precision) which we believe it is better than 55.30% obtained by  and 43.20% obtained by (Charoenporn et al., 2008).


For research purposes, we have obtained free access to the Microsoft Translator, which supports translations among 44 languages.

wordnets are intricate and substantive repositories of lexical knowledge and have become important resources for computational processing of natural languages and for information retrieval.


The rank value in the range 0.00 to 1.00.

we use the first approach to construct Wordnets.


This means the vertical offsets of neighboring glyphs are correlated, a relationship that is not captured by the original model.

we computed the precision and recall for the system's predictions about which words were italicized.


The robust model is fit using Orthant-Wise Limited-Memory Quasi Newton (OWL-QN), a technique for optimizing an L 1 -penalized objective (Andrew and Gao, 2007).

5 tumor and 4 normal samples should have their labels flipped.


The primary claim is that our algorithm performs better than cube pruning in terms of the trade-off between time and accuracy.

it is somewhat inefficient to separately consider the low-scoring child (countries, ).


In practice, neural network models for machine translation are usually trained by maximizing the likelihood of the training data, either via a crossentropy objective (Mikolov et al., 2010;Schwenk et al., 2012) or more recently, noise-contrastive estimation (Vaswani et al., 2013).

most features are local, meaning that all required information for them to assign a score is available within the state.


the second reason why alignment is important is that while one of the merits of t2S models is their ability to perform global re-ordering, it is difficult to learn good reorderings from bad alignments.


The most striking result is that T2S+all significantly exceeds all of the baselines, even including the pre/post-ordering baselines, which provide state-of-the-art results on this task.

there have been also been multiple reports noting the exact opposite result that sourceside syntax systems perform worse than Hiero, S2T, PBMT, or PBMT with pre-ordering (Ambati and Lavie, 2008;Xie et al., 2011;Kaljahi et al., 2012).


Word alignment is at the basis of most statistical machine translation.

this is not done in the current work however, so timing results will not be directly comparable to those found in (Dyer et al., 2013).


In Chinese, the dependent word of a plmod relation (e.g., "前" in Figure 2) occurs in the last position of the prepositional phrase.

in this case, the affect of the performance of the constituent parsers on pre-ordering is larger than that of the dependency ones so that the constituent parsers are likely to bring about more incorrect pre-orderings.


Language-Independent In this work, we integrate only language-independent features, and therefore do not consider morphological or linguistic features.

when considering words seen in the training data, CEC and GSEC have the same performance.


the recovery of errors in OOV words indicates how well the character-level model is doing independently of the MLE component.


Also, for practical applications, we believe that it is useful to distinguish sentences with minor errors from those with major errors that may disrupt communication.

we used the same learning algorithms as for our system (i.e., ridge regression for the ordinal task and logistic regression for the binary task).


The edit context includes the edited paragraph and one preceding and one following paragraph.

they find a correlation between the article edits and their discussions around the anniversaries of the events.


Experimental results on standard SIGHAN 2003 and 2005 bake-off evaluations show that our model outperforms the character and word baselines by a significant margin.

the dual form can then be decomposed into two sub-components (the two max problems in Eq.


Treating microblogs as standard texts and directly classifying them cannot achieve the goal of effective classification because of sparseness problem.

(c) Select relevant words from external knowledge to enrich the content of microblogs.


In our model, DWA, we optimize the likelihood using the EM as well.

another avenue for further study is to combine this method with monolingual language models, particularly in the context of semantic transfer into resource-poor languages.


and those with different search intents.

to test the model's performance with a larger, unseen dataset, we looked at how many retries it detected in the test set of potential retry pairs (n=7,254).


In this paper, we introduce a novel sliding window technique which avoids the errors produced by previous systems at the boundaries of the chunks used for alignment.

after the alignment, the captions are combined via majority voting at each position of the alignment matrix.


For example, DUC2005 consists of 50 tasks.

create term-based MAcD histogram where X-axis refers to T , and Y-axis denotes bursts of word w in A. Hereafter, referred to as bursts histogram.


This shows that integration of LDA and MACD is effective for topic detection.


For each prompt, we sample about 750 essays.

the quality of the information selected from the lecture is emphasized in excerpts from the scoring rubric for this test (below); essays are scored on a 1-5 scale: Score 5 successfully selects the important information from the lecture and coherently and accurately presents this information in relation to the relevant information presented in the reading.


1 Accordingly, large-scale assessments of writing incorporate tasks that test this skill.


The tagset is designed by treating each Chinese character as a singlecharacter word, and each (multi-character) word as a phrase of single-character words.


For example, LPath augments XPath with additional tree operators to give it further expressiveness (Lai and Bird, 2010).

clausal relations operate over longer distances in sentences, and so it is to be expected that showing longer stretches of context would perform bet- ter in these cases; that is indeed what the results showed.


(c) The same options in the phrases condition, shown as they appeared in an identification task for the relationship amod(life, ) (where different adjectives modify the noun 'life').

most potential users do not have programming expertise, and are not likely to be at ease composing rigidly-structured queries.


We also experiment with an additional wFST E that translates English words directly into Chinglish.

we then modify the initial, uniform English-to-Pinyin mapping probabilities by giving higher initial weight to mappings that respect observed start/end pairs.


It appears that the language pair in translation impacts the expected gains from discourse analysis on translation.

discourse devices are at play in the organization of information into complex sentences.


Table 2 shows very similar results using the Stanford parser, demonstrating the pattern is not limited to a single parsing model.

this yields the three similarity functions cited above: all three of these functions are special cases of symmetric ratio models.


Syntactic contexts capture different information than bag-of-word contexts, as we demonstrate using the sentence "Australian scientist discovers star with telescope".

other target words show clear differences between embeddings.


The pairwise distances between verbs were submitted to multidimensional scaling into two dimensions.

the effect is only significant for N ≥ 7; hence, the hypothesis that space density increases the odds of a coinage occurs in the construction is supported for measures of density based on these values of N. More generally, the p-value decreases as N in-creases, which means that the positive relation between DENSITY and OCCURRENCE is less systematic when DENSITY is measured with fewer neighbors.


We can only extract important content by trimming redundant parts from sentences.

we counted the number of sentences in the source document that each method used to generate a summary 5 .


We converted the rhetorical relations between EDUs to the relations between sentences to build the nested tree structure.

methods of EDU selection had an average of 5.77 and a median of five sentences.


We have demonstrated a semantic approach to automatic question generation that outperforms similar systems.

they generate questions of types: why, when, give an example, and yes/no.


Natural language generation faces many linguistic challenges.

most patterns use the lexical form of the main verb along with the appropriate form of the auxiliary do (do, does, did), for the subject-auxiliary inversion required in forming interrogatives.


When merging two spans, a new bigram is created, which connects the rightmost kept words in the left span and the leftmost kept word in the right span.

our approximate approach is much more efficient, about 10 times faster than TM13' system, and achieves competitive accuracy with the exact approach.


We report the ROUGE-1 and ROUGE-2 recall scores (average over the inputs) for each system.

there were 50 inputs, each contains around 10 documents on a common topic.


For example, a value of 10 corresponds to a significance level of 0.001 and is standardly used as the cutoff.

word surprise although computed for each word type separately, quantifies the surprise when incorporating the new counts of this word into the background multinomials.


The AA problem with limited training data was attempted in (Stamatatos, 2007;Luyckx and Daelemans, 2008).

our future work will extend the work by including more views such as the stylistic and vocabulary richness views.


Character ngrams are simple and easily available for any natural language.

three distinct views are used for building classifiers.


Sagarra and Alba (2006) compared the effectiveness of three learning methods including the semantic mapping, rote memorization (i.e., memorization by pure repetition, with no mnemonic aid) and keyword on beginner learners of a second language.

the preparation of the memorization tips for each new word is an activity that requires considerable time, linguistic competence and creativity.


Expanding the latter to your website sucks, brother certainly normalizes it to standard English, but one could argue that in so doing one is losing information that the writer is trying to convey using an informal style.

this system includes some manually built patterns and an address parser to find common abbreviations that occur in a recognizable context.


Ensuring high reliability is not always feasible, however; the advent of crowdsourcing brought about interest in algorithms for recovering from noisy annotations: Snow et al.

rehbein and ruppenhofer (2011) discuss the negative impact of systematic simulated annotation inconsistencies on active learning performance on a word-sense disambiguation task.


Such items might receive contradictory labels, without a clear way of settling the disagreement.

in light of , we want to check whether the presence of less reliable data (hard cases) in the training set adversely impacts performance on the highly reliable data (easy cases).


there are many sophisticated rubrics for calculating consistency.


Thus Sally rolled the ball entails that somebody applied force to the ball (namely: Sally), whereas The ball rolled does not.

the strongest evidence comes from experimental work on several so-called alternations (the passive, causative, locative, and dative alternations).


In the first frame, 8 verbs received the same annotation and 2 received others.

all verbs in the same class appear in the same set of syntactic frames.


Textbooks from Wikibooks were chosen to supply pedagogical text.

matter as a physical entity may seem out of place in  Table 4.


(2013) noted the pervasiveness of shell nouns in text and their tendency to "characterize and label" their antecedents.

such deixis can have a variety of referent types.


Saurí (2008) describes several classes of source-introducing predicates, which indicate how the source relates to the quoted claim.

for instance, consider the use of the word claims in figure 1, which conveys the author's doubt about the indirectly quoted content.


Detecting and reasoning about the certainty of propositional content has been identified as a key task for information extraction, and is now supported by the FactBank corpus of annotations for newstext (Saurí and Pustejovsky, 2009).

investigating the factuality judgments formed in response to such tweets is clearly an important problem for future research, but is outside the scope of this paper.


In recent years there has been an increasing focus on producing lists of words (lexica) with prior polarities, to be used in sentiment analysis.

the use of finer-grained models, accounting for the role of individual emotions, is still in its infancy.


An important finding is that LIWC can be effectively used as a bridge for crosscultural classification, with results that are comparable to the use of unigrams, which suggests that such specialized lexicons can be used for cross-cultural or cross-lingual classification.

these results are in line with previous research, which showed that LIWC word classes exhibit similar trends when distinguishing between deceptive and non-deceptive text (Newman et al., 2003).


We now investigate the significance of the initial model selection step used in the previous experiments.

rather, performance is dominated by the effects of random initialization (a problem for which we provide a correction while abiding by the same constraints as Canini et al.


The features below represent a first pass at candidate features, but many others could be explored.

the frequency is the number of annotators that suggested that simplification.


(2002) proposed a graph-based approach to dividing a longterm search session into search tasks.

a task name is composed of a task event and a task topic.


A complex task consists of several sub-tasks, and each sub-task goal may be composed of a sequence of search queries.

since some identified task names in Cluster_EQ_Rs may not semantically suitable, ECTM_Rs's approach can efficiently deal with this problem.


Furthermore, it requires us to define the number of clusters beforehand.

on average, we reduce the error rate by 56%.


We do not adopt a more fine-grained set because we annotate pairs that are far more ambiguous than those considered in previous efforts.

event tokens that were mentioned fewer than 20 times were excluded and only one TempEval task considered relations between events in different sentences.


Only obvious verbal ironies will be recognizable from intrinsic features alone.

annotators request context significantly more frequently for those comments that (are ultimately deemed to) contain an ironic sentence.


The clause John has kissed Mary introduces a state, but the fundamental aspectual class of the 'tenseless' clause John kiss Mary is dynamic.

to sum up, Inst features are essential for classifying multi-label verbs, and the LingInd features provide some useful prior.


expression in Equation 1: it is easier to estimate p(e i | f j ) than p( f j |e i ), since the vocabulary for the lexical knowledge features ( f j ) is usually more limited than that of the contexts (e i ) and hence estimation of the former suffices on a smaller amount of data than that of the latter.


Unsupervised domain adaptation is a fundamental problem for natural language processing, as we hope to apply our systems to datasets unlike those for which we have annotations.

we can use similar reasoning to compute the expectation of P. with probability (1 − p), the original features are preserved, and we add the outer-product x i x i ; with probability p, we add the outer-product x i q .


High-dimensional setting Structured prediction tasks often have much more features than simple bag-of-words representation, and performance relies on the rare features.

the corrupted input is divided to S subsets We obtain a projection matrix W s for each subset by reconstructing the pivot features from the features in this subset; we can then use the sum of all reconstructions as the new features, tanh( S s=1 W s X s ).


We call this the Relation Constrained Model (RCM).

clearly, our embeddings are much more effective at capturing semantic similarity.


To achieve this, we introduce a constraint that all positions that have similar characteristics must be selected at the same time.

algorithm 2 shows our Greedy+DP search algorithm.


For instance, Spanish also allows to omit pronouns.

there are three different ways to generate a subject.


While, the General-domain baseline trained on 16 million sentence pairs has a hierarchical phrase table containing 1.7 billion translation rules.

additionally, we adopt GIZa++ to get the word alignment of in-domain parallel data and form the word translation probability table.


Finding the last word anywhere boosts performance to 59.1%.

we observe the same computational problem as in the previous section (about 9.2% first pass failures, vs. 6.5%), reflected in a higher accuracy gap for 100ms and 1000ms time limits.


Creating cross-language article links among different online encyclopedias is now an important task in the unification of multilingual knowledge bases.

to train the LDA model, the pair English and Chinese articles are concatenated into a single document.


(B) whether one basic-edit addresses one single error.

the number of false positives (system proposes merges when it should not) is significantly reduced.


Here the hypothesis is that friends, who usually share common interests, tend to discuss similar topics and use similar words than non-friends.

table 3: Accuracy (%) of authorship attribution.


This task can be illustrated by considering the following topic derived from social media related to Education: school protest student fee choic motherlod tuition teacher anger polic where the top 10 words ranked by P (w i |t j ) for this topic are listed.

our research task of automatic labelling a topic consists on selecting a set of words that best describes the semantics of the terms involved in this topic.


WFSTs are likely to beat PFSTs as linguistic models, 4 just as CRFs beat MEMMs (Klein and Manning, 2002).

models of spelling errors are useful in a variety of settings including spelling correction itself and phylogenetic models of string variation  (mays et al., 1991;Church and Gale, 1991;Kukich, 1992;Andrews et al., 2014).


Each path from q I to a final state q F has • an input string x, given by the concatenation of its arcs' input labels; • an output string y, given similarly; • a probability, given by the product of its arcs' probabilities and the halt probability of q F .

log p θ (y k | x k ) depends only on the probabilities of edit arcs in T that appear in x k • T • y k , and the competing edit arcs from the same edit states q C .


This provides an indication of the overall quality of the label the system judges as the best one.

the best label proposed by our methods is judged to be better (as shown by the nDCG-1 and Top-1 Av.


These features increased greatly performance improvements (3rd column in Table 3).

we ran Kokoma Korean morpheme analyzer 4 on each sentence of the Donga corpus to divide words into morphemes to build latent morpheme representations.


Korean SRL research has been limited to domestically published Korean research on small corpora.

figure 1 displays a English/Korean sentence pair, highlighting the SOV word order of Korean as well as its rich morphological structure.


We model the conditional probability p(y|x) using a CRF model: where f m (y i−1 , y i , x, i) are the feature functions.

the Eomi and Josa categorization plays an important role in signaling semantic roles.


this research was supported by the the Basque Government (It344-10, S PE11UN114), the University of the Basque Country (GIU09/19) and the Spanish Ministry of Science and Innovation (MICINN, tIN2010-20218).


As discussed in Section 2.2.2, noun modifiers are sisters to the noun, instead of being adjoined, as in the PTB.

in future work, we will do a more proper cross-validation evaluation.


We address both tasks together with the regexes.

each structure within a circle is one etree, and the derivation as a whole indicates how these etrees are combined.


We call the resulting model Code-Switched LDA (csLDA).

to summarize, based on the model of  we will learn: • For each topic, a language specific word distribution.


In the case of code-switched social media data, we require a pertoken language variable.

training of polylingual topic models requires parallel or comparable corpora: document tuples from multiple languages that discuss the same topic.


We train a recurrent neural network language model (Mikolov et al., 2010;Mikolov, 2012b) on a large collection of tweets.

publicly available normalization datasets are annotated at word level.


(such as in document-level topic models).

the higher the β value, the more uniform the sampling.


We used KenLM for building models and evaluating them (Heafield, 2011).

language shows observable priming effects, sometimes called triggers, where the occurrence of a given term decreases the surprisal of some other term later in the same discourse (Lau et al., 1993;Church and Gale, 1995;Beeferman et al., 1997;Church, 2000).


This section describes our sentence-level sentiment tagger, from which we construct features for dispute detection (Section 4).

the accuracy is significantly higher than all the other systems (paired-t test, p < 0.05).


We have also used a named entity tagger trained specifically on the Twitter data 3 (Ritter et al., 2011) to directly extract named entities from tweets.

we have conducted experiments on a Twitter corpus and the results show that our proposed approach outperforms TwiCal, the state-ofthe-art open event extraction system, by 7.7% in F-measure.


We propose a simple Bayesian modelling approach which is able to directly extract event-related keywords from tweets without supervised learning.

the dataset was filtered by removing the events which are mentioned in less than 10 tweets.


Thus, we are motivated to develop a portable technique that generates representations of semantic content that are richer than keywords, and that can be applied to broad domains.

the information structures are to be used as representations of semantic content for text mining purposes.


This is a classical K-Nearest-Neighbor classifier baseline.

to select the classification threshold for ith relation type, we use the value which can achieve the best F-measure on training dataset (with an additional restriction that precision should > 10%).


Figure  2 shows that our model consistently outperforms all six algorithms at almost all recall levels and improves the maximum F -score by more than 13.5% relative to MIML (from 28.35% to 32.19%) as well as increases the area under precision-recall curve by more than 37% (from 11.74 to 16.1).

we define: • y r i ∈ {P, N } : r holds for the ith bag or not.


We also compare Guided DS with three state-of-the-art models: 1) MultiR and 2) MIML are two distant supervision models that support multi-instance learning and overlapping relations; 3) Mintz++ is a single-instance learning algorithm for distant supervision.

in some cases a small amount of human labeled data is available.


More concretely, in textual inference the candidate predicate and argument are typically identified, as they are aligned by the RTE system to a predicate and an argument of the Hypothesis.

one third of this potential improvement is achieved by our algorithm 1 .


It tends to be better than the previous measures on the upper bound, likely because of the contextual comparison it allows.

we have chosen to start with the domain interaction system (Dunn, 2013a(Dunn, , 2013b, which performed competitively in an evaluation with other systems (Dunn, 2013b).


This gives a total of 41 features for each sentence.

metaphorically used prepositions have been untagged as metaphoric.


This was verified by experiments on a corpus of 1-million sentence pairs on which traditional MCMC approaches would struggle (Xu et al., 2008).

uWS, spurred by the findings that infants are able to use statistical cues to determine word boundaries (Saffran et al., 1996), relies on statistical criteria instead of manually crafted standards.


Corlett and Penn (2010) solve the same problem using A * search.

for a cipher of length N we only explore N • B • (B LM + B lex ) many hypotheses.


The first natural approach is to extend MEANT's f-score based method of aggregating semantic parse accuracy, so as to also apply to aggregat-ing lexical translation probabilities within semantic role filler phrases.

(2012) showed empirically that cross-lingual semantic role reordering of the kind that MEANT is based upon is fully covered within ITG constraints.


The best single baseline MT system for DA is MSA-Pivot has a large room for improvement given the oracle upper bound (4.8% BLEU absolute).

we performed manual error analysis on a Dev set sample of 250 sen-tences distributed among the different dialects and genres.


The detiled training process is as follows: (1) Using the BTG system to perform force decoding on FBIS part of the bilingual training data 5 , and collect the sentences succeeded in force decoding (86,902 sentences in total) 6 .

it is too difficult to obtain all derivations.


We have presented a latent-variable violationfixing framework for general structured prediction problems with inexact search over hypergraphs.

a similar success on machine translation has been elusive, where the mainstream methods still tune on small datasets.


Punctuation parsing errors lead to low parsing accuracy on words.

the UAS on punctuations are generally below 85%.


Further, this sort of annotation seems well suited to incremental parsing with beam search, which has been shown to achieve high accuracies even for fully connected parsing (Zhang and Clark, 2011).

complexity of parsing with a hedge constrained CYK is reduced to O((nL 2 + n 2 )|G|).


The intermediate structures produced by the parser are highly informative, including predictions for properties and structural embeddings of upcoming words.

turboParser is trained on these extended PDAs and no adaptation of the training algorithm is needed.


The word five elements left of the newest word (time point 5) gets attached with an accuracy that is nearly as high as the accuracy for the whole sentence (final).

to previous approaches, we achieve state-of-the-art accuracy for whole sentences by abandoning strong monotonicity and aim at high stability instead, allowing the parser to improve intermediate results in light of new evidence.


We tried various sets of prefix lengths on the development set and found the best setting to use prefixes of length 4, 6, 8, and 12.

several methods have been proposed for intrinsic evaluation of word representa- tions; we discuss two here: Word similarity (sIM): One widely-used evaluation compares distances in the continuous space to human judgments of word similarity using the 353-pair dataset of Finkelstein et al.


Several researchers have made their trained representations publicly avail- able, which we use directly in our experiments.

we use ward's minimum variance algorithm (ward, 1963) for cluster distance and the Euclidean metric for vector distance (via MAT-LAB's linkage function with {method=ward, metric=euclidean}).


Short of running end-to-end parsing experiments, how can we choose which representations to use for parsing tasks?

we use the SENNA embeddings of Collobert et al.


This paper has demonstrated a simple and effective learning method for type-supervised, transductive part-of-speech tagging.

this paper describes observational initialization, a simple but effective learning technique for training type-supervised taggers.


Transductive learning arises in document clustering and corpus analysis applications.

the final SUPERVISED performance of 94.1% shows that there is substantial room for improvement over the UNIFORM initializer.


we introduced a model for leveraging situational information in learning vector-space representations of words that are sensitive to the speaker's social context.


We introduce a model for incorporating contextual information (such as geography) in learning vector-space representations of situated language.


We denote these sets C, for concrete, and A for abstract respectively.

according to the Dual Coding Theory, however, concrete concepts are precisely those with a salient perceptual representation.


For comparison, following approaches are implemented:  Monolingual: perform monolingual classification on the Chinese and English corpus individually, as shown in Figure 1.

this research work has been partially supported by two NSFC grants, No.61375073, and  No.61273320


E3: Saddam's clan is said to have left for a small village in the desert.

for a simplified implementation, we first get the  Where 0.9 is an empirical value which makes the translation probability become a dominative factor when the translation of the trigger is found in the translated sentence.


To address this, we check the common cooccurring entity pair of the relations.

each relation translation method translates an english relation r e into a list of Chinese relations, and we check whether a Chinese relation r C with the highest translation score is the correct translation.


• f F S1 and f F S2 : , With these features, we measure the similarity of a pair of couplings as follows.

unfortunately, this approach suffers from sparsity of the common entity pairs due to the incomparability of the corpora and those entities that cannot be translated by T N .


When its support set is {(Bush, China), (Mandelson, Moscow), (Rice, Israel)}, and that of visit is {(Bush, China), (Rice, Israel), (Medvedev, Cuba)}, we can infer their semantic equivalence based on the set intersection: In contrast, we propose to explore corpus latent features (LF), to complement the sparsity problem of EF: Out of 158 randomly chosen correct relation translation pairs we labeled, 64% has only one co-occurring entity pair, which makes EF not very effective to identify these relation translations.

note that the absolute numbers (due to the harsh evaluation criteria) may look low.


• [C3] f EX : The existence of a common entity pair using the seed entity translations (boolean).

to illustrate, two relations deploy and 在...部署 (deploy at) have similar temporal behaviors (Figure 2c).


This suggests an alternative explanation of the effectiveness of a bigram-based classifier in identifying the native language of a writer -that the character bigrams simply mirror differences in the word usage rather than the phonology of L1.

the fact that the two bigrams are also on the list for 2 It appears that only the relatively low frequency of most of the punctuation bigrams prevents them from dominating the sets of the indicative bigrams.


tention signal is from the state of a generative recurrent neural network (RNN) as decoder.

indeed, with the signal from the already learned word alignment, tagCNN can gain +0.25 BLEU over its generic counterpart, while for inCNN with the guiding signal from the proceeding words in target, the gain is more saliently +0.72 BLEU.


As shown, TCM yields a positive effect of around 0.5 BLEU and TER improvements in AR-EN and around 0.2 BLEU and TER improvements in ZH-EN.

the combination of the two sets produces a complementary gain in addition to the gains of the individual models as Row (S 10 ) shows.


For example, identifying the gender of a person is important for generating a good description.

table 3 shows the results of the model transfer experiment.


The objects are then parsed into a VDR structure using the VDR Parser trained on the automatically inferred training data.

transferring VDR from the VLT2K to the Pascal1K data set improves the generated descriptions from 7.4 → 8.2 Meteor points.


FA8750-13-2-0040, the National Science Foundation under grant no.

1 To create this training set, we used a simple online scene design interface that allows users to assemble scenes using available 3D models of common household objects (each model is annotated with a category label and has a unique ID).


Privacy restrictions are another reason why unsupervised/weakly-supervised methods can be helpful.

this example illustrates the strength of the joint reasoning approach in PSL-Joint.


In the first strategy, which we refer to as closest, we follow the conventional approach (Budanitsky and Hirst, 2006) and measure the similarity of the two words as the similarity of their closest senses, i.e.

in order to make use of the complementary information provided by our lexical knowledge base and to obtain more accurate similarity judgements, we introduce a graph vicinity factor, that combines the structural knowledge from BabelNet's semantic network and the distributional representation of sense embeddings.


We observed that the intended sense of the noun orthodontist occurs only 70 times in our annotated corpus.

the similarity of a pair of words w 1 and w 2 according to the weighted strategy is computed as: where the α parameter is a real-valued constant greater than one.


As our comparison baseline, we also report results for word embeddings, obtained using the Word2vec toolkit on the same corpus and with the same configuration (cf.

our approach lags behind on SCWS, highlighting the negative impact of taking the closest  senses as the intended meanings.


In this paper, when combining word-pair distances learned by different components of the contrasting inference layer, we use some top hidden layer(s) to provide a non-linear combination.

we also combined SCE/MCE with other components in the contrast inference layer, for which we used ten-fold cross validation to tune the weights of the top hidden layers on nine fold and test on the rest and repeated this for ten times to report the results.


To answer this, we downloaded the word pairs that Zhang et al.

as far as the embedding-based methods are concerned, PILSA (Yih et al., 2012) made a progress in achieving one of the best results, by priming LSA to encode contrasting meaning.


The statistics of the corpora used in baseline models are shown in Table 1.

a major problem with HAL is that the similarity measure will be dominated by the most frequent words due to its weight scheme.


In this paper, we use the same strategy as word2vec tools which uniformly samples from the set  The framework for HDC model.

existing models for learning word representations focus on either syntagmatic or paradigmatic relations alone.


As in part-of-speech tagging, we observe significant test-time speedups when applying our method of dynamic feature selection to dependency parsing.

they use a reinforcement learning approach whose computational tradeoffs are better suited to vision problems with expensive features.


RNN-random is the supervised RNN model described in section 3 with the relation vectors initialized randomly.

rNN + PrA Classifier combines the predictions of rNN and PrA Classifier.


Such KBs typically contain many millions of facts, most of them (en-tity1,relation,entity2) "triples" (also known as binary relations) such as (Barack Obama, presi-dentOf, USA) and (Brad Pitt, marriedTo, Angelina Jolie).

we also show that our zeroshot model is indeed capable of predicting new unseen relation types.


• Position feature (PF): It is necessary to spec-ify which words are the predicted trigger or candidate argument in the argument classification.

we propose a dynamic multi-pooling convolutional neural network (DMCNN), which uses a dynamic multi-pooling layer according to event triggers and arguments, to reserve more crucial information.


Although event extraction depends on name identification and entity mention co-reference, it is another difficult task in ACE evaluation and not the focus in the event extraction task.

table 3 illustrates that the methods based on convolutional neural networks (CNN and DM-CNN) outperform Embeddings+t.


Because of the ambiguity, a traditional approach may mislabel beats in S1 as a trigger of Attack.

we split each feature map into three parts according to the candidate argument and predicted trigger in the argument classification stage.


More recently, work after (Hasegawa et al., 2004) has developed weakly supervised forms of Information Extraction including schema induction in their objectives.

if this relevance score is higher than a threshold λ, then the document is considered as relevant.


based on this intuition, we propose a heuristic method that ignores segmentation boundaries that result in a translation of this type, and instead wait for the next translation unit, helping to avoid problems due to inappropriate segmentation boundaries.


Furthermore, T2S-Wait also maintains the same performance of T2S-Tag in BLEU and achieves much higher performance than any of the other methods in RIBES, particularly with regards to shorter translation units.

the second method is the state-of-the-art segmentation strategy proposed by Oda et al.


In this paper, we perform the first experiments applying syntax-based SMT to simultaneous translation, and propose two methods to prevent degradations in accuracy: a method to predict unseen syntactic constituents that help generate complete parse trees, and a method that waits for more input when the current utterance is not enough to generate a fluent translation.

the parser input may be an incomplete sequence of words (e.g.


From the figure we can see that lengths of up to 2 constituents cover the majority of cases for both L and R, but a significant number of cases require longer strings.

in the case of simultaneous translation, each translation unit is not necessarily segmented in a way that guarantees that the translation unit is a complete sentence, so each translation unit should be treated not as a whole, but as a part of a spoken sentence.


Another direction is to use more complex translation models such as hierarchical models (Chiang, 2007).

if there are two words x i and x j in x which satisfy neither x does not make a totally ordered set with the order relation), then x cannot be sorted, and the example is removed from the training data.


Along this direction our contributions are: • The first application of online multitask learning to QE, geared to the challenges posed by CAT technology.

indeed, despite the STL in downward error trend, it's worth remarking that an increased competitiveness would come at the cost of: i) collecting large amounts of annotated data and ii) integrating the model in a complex CAT architecture organized as a pool of independent QE components.


High C values lead to more aggressive weight updates.

on one side, processing such stream calls for supervised solutions that avoid the bot-tleneck of periodically retraining the QE models in a batch fashion.


Now let us define the latent answer-entailing structures.

the candidate answers may be pre-defined, as is the case in multiple-choice question answering, or may be undefined but restricted (e.g., to yes, no, or any noun phrase in the text).


4 (3) The third baseline (called RTE) uses textual entailment to answer MCTest questions.

rTE fares pretty poorly on the "multiple" questions indicating that of-the-shelf rTE systems cannot perform inference across large texts.


For these 11.5K elements, we associate their Italian (target space) label y i with a none chimera-5 P@1 38.4 31.1 P@5 54.2 46.1 P@10 60.4 51.3 Table 10: Cross-linguistic zero-shot experiment with data augmentation.

11 11 Pollution is of course an effect of overfitting, but we use this more specific term to refer to the tendency of training vectors to "pollute" nearest neighbour lists of mapped vectors.


5 Cross-modal experiments In the cross-modal experiments, we induce a mapping from visual to linguistic space.

still, more than 1/5 top predictions for cross-modal mapping with max-margin are hubs (vs. less than 1/10 for the original vectors).


More formally, at each step of stochastic gradient descent, given a source space vector x i , its target gold label/translation y i in Y T r and the mapped vectorŷ i , we compute s j = cos(ŷ i , y j ) − cos(y i , y j ), for all vectors y j in Y T r s.t.

given the vector representation of a word in language A, we apply the mapping to obtain an estimate of the vector representation of its meaning in language B, returning the nearest neighbour of the mapped vector in the B space as candidate translation.


Our starting point are the 5.1K labels in ImageNet (Deng et al., 2009) that occur at least 500 times in our English corpus and have concreteness score ≥5, according to Turney et al.

in practice, the accuracy in label retrieval with current mapping methods is still too low for practical uses.


We have constructed a dataset for French containing a large number of pairs of a compositional expression (adjective-noun) and a single noun that is semantically close or identical to the composed expression.

admittedly, choosing which noun is subordinate in noun-noun composition may be an interesting problem but it is out-side the scope of this paper.


In addition, LF's approach to adjectival meaning and composition has a theoretical drawback.

to be able to use the non-reduced space for the lexical function approach, one would have to learn matrices of size 10K ×10K for each adjective.


adjective-noun composition, or the composition of transitive verbs and direct objects (Mitchell and Lapata, 2008;Coecke et al., 2010; Baroni and Zamparelli, 2010)), the modeling of compositionality is still an important challenge for distributional semantics.

the negative values that result from svd are detrimental for the multiplicative approach.


In ablation experiments we tease apart our various contributions and modeling choices in order to shed some light on what matters in practice.


This allows us to capture a large variety of long range dependencies with a concise classifier.

for both of the other actions, it is often the case that we would like to capture a controller in the higher clause.


7 birthPlace, child, commander, deathPlace, keyPerson, knownFor, monarch, parent, partner, predecessor, relation, restingPlace, spouse, successor, territory 8 architect, associatedBand, associatedMusicalArtist, au-pages with 43,475 mentions, and we use 30,000 for training, and 13,375 for testing.

a key issue is that information extraction (IE) errors from text affect the quality of the KB, and propagate to the reasoning task.


For example, prepositional phrases (PPs) express crucial information that IE methods need to extract.

we could not evaluate the Stanford parser on the wSJ dataset.


Since prepositions specify details that transform binary relations to higher N-ary relations, our method can be used to read information that can augment binary relations already in KBs.


OntoNotes provides annotation of constituency trees only.

our analyses indicate that it may be possible to achieve small but significant increases in accuracy of dependency parsing through ensemble methods.


Translation Filter (TF) handles both Predicate Mismatch and Verb→Non-Verb translation shift errors.

table 6: Estimated precision and recall over seven languages.


A SUPER-VISED system was trained on the small Portuguese training set described in §6.3.

this work only addresses agent detection and requires translating the MPQA corpus.


The encoder and decoder structures allow us to learn a mapping within and across languages.

similarly, the same steps repeat for the latent representation y C in Chinese, which are shown in Figure 1(b).


In this paper, we only evaluate BSWE when dimension d varies from 50 to 500.

bilingual embedding learning algorithms focus on capturing syntactic and semantic similarities across languages, but ignore sentiment information.


Tang and Wan (2014) first proposed a bilingual sentiment embedding model using the original training data and the corresponding translations through a linear mapping rather than deep learning technique.

learning language-specific classifiers in each view fails to capture the common sentiment information of two languages during training process.


Scientific summarization: Early work in scientific summarization used abstracts of scientific articles to produce summaries of specific scientific papers (Kupiec et al., 1995).

mei and Zhai (2008) presented an impact-based summarization method for single-paper summarization that assigns relevance scores to sentences in a paper based on their similarity to the set of citing sentences that reference the paper.


(2010) convert input DAs into "semantic stacks", which correspond to natural language phrases and contain slots and their values on top of each other.

• Punctuation -nodes for commas, final punctuation, quotes, and brackets are introduced.


First a directed acyclic word graph is constructed based on the extracted events, and then a beam-search algorithm is used to find the best title based on path scoring.


In our experiments, we find that our pretraining method is almost always superior to strong baselines such as the popular domain adaptation method of Daumé III (2007).

the motivation for this definition is that similar label types often have similar or same word.


Table 2 shows the MAP performance of predicted slots for all experiments on ASR and manual transcripts.

the reason may be that inter-slot relations carry high-level semantics that align well with the structure of SDSs, but inter-word relations do not.


To further consider the global semantic structure, such as inter-word and inter-slot relations, we augment the latent MF-based model with a knowledge graph propagation model based on a slot-based semantic graph and a word-based lexical graph.

(b): Our matrix factorization method completes a partiallymissing matrix for implicit semantic parsing.


The method of auto-matic knowledge graph construction is introduced in Section 5, where we leverage distributed word embeddings associated with typed syntactic dependencies to model the relations (Mikolov et al., 2013b;Mikolov et al., 2013c;Levy and Goldberg, 2014;Chen et al., 2015).

our model learns the semantic slots for a domain-specific SDS in an unsupervised fashion, and carries out semantic parsing using latent MF techniques.


direct comparison with their work is difficult as they utilized the whole SWBD data while we only use the subcorpus with syntactic annotation which is only half the SWBD corpus and they also used extra corpus for language model training.


Similarly, Sil and Yates (2013) proposed to use a reranking approach to obtain overall better results on mention detection and entity disambiguation.

as shown in Table 3, the significant improvement in IR-driven evaluation indicates the importance of incorporating entity-entity information.


The Mixture of Language Model represents an entity as a document with multiple fields, where each field is given a different weight for generating the query terms.

one of the central components of our proposed Figure 1: An example of entity factoid hierarchy containing two factoids about Barack obama retrieval framework is a novel entity representation known as entity factoid hierarchy.


• Factoid Retrieval Model with Hierarchy (FRMwH).

it is not easy to generalize these models for open domain entity retrieval.


without background knowledge to evaluate the impact of introducing background knowledge.

our model first extracts these triples automatically from raw text and converts them into real-valued vectors based on the word semantics captured by Latent Dirichlet Allocation.


When evaluating the top 10 triples with the highest relevance weight, our framework outperforms the best baseline LDA by 4.4% in MAP and by 3.91% in P@N. When evaluating the top 5 triples, our framework performs even better and significantly outperforms the best baseline by 5.87% in MAP and by 17.21% in P@N. To analyze the results further, Ours-S, the simplified version of our model without iterative propagation, outperforms two strong baselines VSM and WE, which indicates the effectiveness of encoding distributional semantics.

the task of document enrichment is proposed to tackle this problem, and has been proved helpful in many NLP tasks such as web search (Pantel and Fuxman, 2011), coreference resolution (Bryl et al., 2010), document cluster (Hu et al., 2009) and entity disambiguation (Sen, 2012).


However, these methods primarily rely on special resources constructed with supervision or even manually, which are difficult to expand and in turn limit their applications in practice.

to evaluate our model, we conduct two series of experiments: (1) evaluation as a ranking problem, and (2) task-based evaluation.


The three error metric scores are given by: System S1 S2 S3 P C Baseline 1 .668 .428 .321 .000 Baseline 2 .652 .418 .267 .061 Our System .618 .392 .244 .212 Table 6: Five-fold cross-validation results for argument strength scoring.

we create features based on the 149 transitional phrases compiled by Study Guides and Strategies 4 .


In fact, they form a cyclic graph (Zesch and Gurevych, 2007).

a natural extension of this definition to a set of topics T is defined as Γ(T ) = ∪ t∈T Γ(t).


We operationalize the problem as a problem of ranking sentences from documents in a corpus that is related to the knowledge graph.

for ranking sentences we use a Random forest (Rf) classifier (Breiman, 2001).


A binary indicator x, which is generated by a Bernoulli distribution π x drawn from a Beta distribution with symmetric hyperparameter γ x , determines whether a word is generated by ψ z or ψ e .

moreover, there was work (Verhagen et al., 2005;Chambers and Jurafsky, 2008;Bethard, 2013;Chambers, 2013;Chambers et al., 2014) about temporal event extraction and tracking.


The patterns of reporting war events are similar to those of disasters, thus the quality of war chronicle is also good.

ψ e tends to describe the event-specific aspects.


If there exists one social relation between two mentions m p i and m q i of the morph m i , they are usually coreferential and assigned an association score 1.

to clean the name entity list, we adopt the temporal Distribution Assumption proposed in our recent work (Huang et al., 2013).


• Within-genre Unsupervised Learning: We directly train morph mention and entity embeddings from the large-scale tweets and web documents that we collect.

we adopt a dictionary of proper names (Li et al., 2012) and propose the following features: (i) Whether a term occurs in the dictionary.


For cases where adjectives should be disambiguated as nouns, e.g., "English" as a country to "England", we allow candidate mappings through the pertainment relation from WordNet.

table 6 shows results of employing individual configurations compared to the MFS baseline.


The integration of complementary objectives into our formalism demonstrates that robust disambiguation can be achieved by considering both the local and the global context of a mention.

babelfy (Moro et al., 2014) has shown the potential of combining these two tasks in a purely knowledge-driven approach that jointly finds connections between potential word senses on a global, document level.


Note that our formulation could easily be extended to using additional coefficients for each ob-jective.

babelfy (Moro et al., 2014) is a knowledgebased approach for joint WSD and EL that also uses a greedy densest subgraph algorithm for disambiguation.


They should not be applied during disambiguation because these priors can be very strong and are not domain independent.

the combination of the two most commonly employed reference knowledge bases for WSD and EL, WordNet (Fellbaum, 1998) and Wikipedia, in BabelNet (Navigli and Ponzetto, 2012), has enabled a new line of research towards the joint disambiguation of words and named entities.


Given a set of objectives O the overall objective function O is defined as the sum of all normalized objectives O ∈ O given a set of mentions M : .

existing methods or resources especially for NEs are either missing many important mappings 1 or contain many noisy mappings 2 .


Specifically, 69.3k (69.5%) of author-defined keyphrases (occurring in ACMCORPUS but not in AB-BREVCORPUS) are used as a keyword in only one paper.

for example, Color Perception and Blood Cell both appear in ACM articles but are not abbreviated.


In most situations the date is only 1-2 years apart.

our solution does not depend on a specific implementation, only that we are able to accurately identify abbreviation expansions.


Additionally, as a mixture model, LDA suffers from the drawbacks mentioned in Section 1 that are common to all mixture models.

this approach enables the model to learn multi-faceted information with a simple yet expressive structure.


In order to create the results presented in Table 6, we rank the trigrams which were selected as the most representative of any sentence in decreasing order of contribution value.

this effect of WPEs is reported by (Zeng et al., 2014).


Given a sentence x consisting of N words x = {w 1 , w 2 , ..., w N }, every word w n is converted into a real-valued vector r wn .

we use the backpropagation algorithm to compute gradients of the network.


The representation for the complete sentence is computed bottom-up by recursively combining the words according to the syntactic structure of the parse tree Their method is named the matrix-vector recursive neural network (MVRNN).

when using only the text span between the target nouns, the impact of WPE is much smaller.


In fact, the problem of incompatible comparison is unfortunately very common in the RE literature (Wang, 2008;Plank and Moschitti, 2013) and we believe there is a need to tackle this increasing confusion in this line of research.

we take half of bc as the only target development set, and use the remaining data and domains for testing.


This suggests little overlap in the contexts of query and counterpart terms.

it is difficult to find temporal counterpart terms by directly comparing context vectors across time.


it is difficult to collect multiple training pairs over various domains and for arbitrary time.


The figure displays the results of exploring paths up to a total length of 6 edges, performing backward search from the target nodes of up to j = 1, 2, 3 steps.

pRA encodes p (s → t; π j ), the probability of reaching target node t starting from source node s and following path π j , as a feature that describes the semantic relation between s and t. provided with a set of selected path types up to length , p = {π 1 , .


In a related study, and aware of the fact that there are often multiple ways to correct individual words in sentence, Rozovskaya and Roth (2010) instead chose to compute agreement at the sentence level.

the main reason for this is that these error types all typically have a scope much larger than most other categories in that they often involve changes at the structural or semantic level; e.g., changing an active to a passive or choosing a synonym.


While analysing the English and German part of the ICD ontology gold standard we noticed significant differences in the translations of the medical labels.

section 5 3 Translation performed on 25.02.2015 presents the automatic and manual evaluation of the translated labels.


The moped of the second example is a pun, one of whose meanings is the same inflected form of the verb mope ("to sulk") and the other of which is the noun moped ("motorized scooter").

these observations confirm the concerns we raised in §2.2 that pun disambiguators, unlike traditional WSD systems, cannot always rely on the output of a lemmatizer or part-of-speech tagger to narrow down the list of sense candidates.


For our second tie-breaking strategy, we posit that since humour derives from the resolution of semantic incongruity (Raskin, 1985;Attardo, 1994), puns are more likely to exploit coarse-grained homonymy than than fine-grained systematic polysemy.

there is some previous research on computational detection and comprehension of humour, though by and large it is not concerned specifically with puns; those studies which do analyze puns tend to have a phonological or syntactic rather than semantic bent.


The major contributions of this work are as follows: First, we have produced a new data set consisting of manually sense-annotated homographic puns.

1 there exists a class of language constructs known 1 Under this assumption, lexical ambiguity arises due to there being a plurality of words with the same surface form but different meanings, and the task of the interpreter is to select correctly among them.


the domain on which we train) vs. the target (i.e.

by applying algebraic operations on the word representations, we can perform numerous tasks in NLP, such as composing representations for larger textual units beyond individual words such as phrases (Mitchell and Lapata, 2008).


Following the bag-of-features model, we assume that a document D is represented by the set of pivots and non-pivots that occur in D (w ∈ d and c ∈ d).

dimensionality n of the representation is set to 300 for both GloVe and the proposed method.


For notational convenience we use w to denote non-pivots in both domains when the domain is clear from the context.

unlike those methods that use the softmax function to convert inner-products to probabilities, we directly use the inner-products without any further transformations, thereby avoiding computationally expensive distribution normalizations over the entire vocabulary.


Babelfy (Moro et al., 2014) is an approach with state-ofthe-art performance that relies on random walks on BabelNet multilingual semantic network (Navigli and Ponzetto, 2012a) and densest subgraph heuristics.

in our earlier work (Camacho-Collados et al., 2015), we conducted different experiments which demonstrated the improvement that lexical specificity can provide over the popular term frequency-inverse document frequency weighting scheme (Jones, 1972, tf-idf ).


The author was supported under ERC Starting Grant LOWLANDS No.

the difference in the two distributions illustrates why we need to control for topic frequency in our experiments.


As the semantic surprisal model represents a reweighted trigram model, there is a high correlation between the trigram model and the semantic surprisal model.


Z is drawn from a multinomial parameterized by a vector θ of length L. Definition 1 A Sparse Backoff Tree SBT (T , δ θ , Q(z)) for the discrete random variable Z consists of a rooted tree T containing L leaves, one for each value of Z; a coefficient δ a > 0 for each interior node a of T ; and a backoff distribution Q(z).

as argued above, existing hierarchical models require inference that becomes expensive as the topic space increases in size.


Latent variable topic models such as Latent Dirichlet Allocation (LDA) can discover topics from text in an unsupervised fashion.

lastly, SparselDA (Yao et al., 2009) is a method that improves the efficiency of inference in lDA by only generating portions of the sampling distribution when necessary.


Remarkably, an alternative approach is used in whole-sentence maximum entropy (WSME) language modeling (Rosenfeld et al., 2001).


Word embeddings play a crucial role in our model, since they allow us to model complex relations between the translations and the reference using syntactic and semantic vector representations.

we further experimented with a setup where we first used the logistic cost to pretrain the neural network, and then we switched to the Kendall cost in order to perform some finer tuning.


Existing MT evaluation measures typically output an absolute quality score by computing the similarity between the machine and the human translations.

we also need to make sure that we discourage ties.


Our MBOT is a synchronous grammar (Chiang, 2006) similar to a synchronous context-free grammar (SCFG), but instead of a single source and target fragment per rule, our rules are of the form s → (t 1 , .

we hope that our approach can also be applied successfully to other language pairs.