Of course, there will always be words that do not emerge, even in a large corpus; however, such rare words are too difficult for language learners to identify, and many are proper nouns, which are not helpful for measuring the vocabulary of second language learners.

our focus here is to measure how well the learners know a fixed set of words, that is, the given 8,000 words.


We tried C = 0.1, 0.5, 1.0, 2.0, 5.0, 10.0, 20.0, 50.0, and 100.0 for each of SVM and LR where the size of training data is 50 and chose the C value that performs best.

we set C = 5.0 for SVM and C = 50.0 for LR.


Since Dutch does not mark objects of the verb morphologically, NL-Turkish speakers have difficulty (e.g., unnecessary addition or omission of case markers) in determining the definiteness of the nominal complements in [N V] constructions (Dogruöz and Backus, 2009).

we expect this feature to differentiate well between NL-Turkish and TR-Turkish [N V] constructions and LVCs with yapmak/etmek as verbal complements.


To the best of our knowledge, finiteness has not been tested as a feature for identifying LVCs earlier.

we encoded the finiteness on yapmak/etmek as a binary (yes/no) feature in both data sets.


Although this requires further investigation, we can explain it by the thousands of features context generates: each contextual word is a feature.

it is very hard for our grammatical features to compete against contextual features but they do have an impact.


With the increasing use of handheld devices, reading device is regarded as an important factor for readability.

this paper investigates the relationship between readability and reading devices such as a smart phone, a tablet, and paper.


These factors reflect the easiness and understanding of the text and include text presentation format, font size, average ratio of annotated images, and sentence length (Hasegawa et al., 2008;Kitson, 1927;Ma et al., 2012;Oquist, 2006).

readability can be used to provide satisfiable services in text recommendation or text visualization.


Some factors are important to the readability of smart phone, but insignificant to that of paper.

we discover the importance of each readability factor for each device by analyzing the correlations.


Readability on mobile devices is not reflected only by the visualization factors, but also by textual factors.

this paper explores the readability factors that reflect the lexical and grammatical complexity of text and are affected by reading devices.


This is because some of their factors are computationally infeasible and language-dependent.


When a writer attempts to write many topics in a text, she tends to use many kinds of words simultaneously.

the text becomes longer and more complex.


The frequency of a word w is then reflected into P (w|B) computed from the independent background corpus B.

the factor LL is positively related with readability.


A reader regards a text as difficult when the sentences in the text have large parse tree depths or many subordinate clauses.

pTD and SBAR are related negatively with readability.


High values for these factors imply that the sentences in the text are related somehow.

these factors are believed to be related positively with readability.


We assume that w i|d , the weight of a readability factor f i , is dependent on the reading device d. Following the previous work of Pitler and Nenkova (2008), we also assume that each readability factor affects readability independently.

readability is calculated as a weighted linear sum of all readability factors.


One is an article, and the other is a device type.

the re-rank function is modeled as the readability-based re-ranking module suggests the news articles based on how easily the articles are read on a specific reading device.


The human annotators were asked to read and evaluate many articles within a relatively short period.

before the main experiments, we performed a pilot experiment on the memory effects of previously read articles and verified it empirically.


This is because the readability factors that affect the readers of news articles are different according to the reading device.

it is important to reflect the characteristics of a reading device when recommending news articles.


Let M SU Set = empty set For each word w in L: If Length(w) ≤ 2 Add w to M SU Set End if End for For each word w in L: If Length(w) > 2 and no word x in M SU Set is a substring of w Add w to M SU Set End if End for Return M SU Set Table 3: Algorithm for collecting MSUs from the PKU corpus The MSU-based method gives each MSU an extra indicative label.

any sequence labeling model is appropriate for the method.


On the other hand, we can tag the substrings of the full form to find the local representative characters in the substrings of the long full form.

we propose the sub-string based approach to given labeling results on sub-strings.


We should integrate their advantages while keeping the consistency between each candidate.

we further propose a global decoding strategy using Integer Linear Programming(ILP).


We can also figure out that the top-5 candidates include the reference abbreviation for most full forms.

reasonable decoding by considering all possible labeling of sequences may improve the performance.


We further select the top-ranked candidates for ILP decoding.

the MSU-based method can further improve the performance in the global decoding phase.


By decoding using ILP, both local and global information are incorporated.


Note that the event features used in our prediction model are generalized based on the algorithm introduced in Section 2.5.

though a specific event in the development test set might have never happened, its generalized form can be found in the training set.


In general, coreferencebased similarity can however be used as an additional feature for any task that distributional similarity is useful for.

our coreference resource can be applied to a variety of NLP tasks, e.g.


However, this approach cannot handle unknown words (Turian et al., 2010) and similarities among words cannot be represented (Mikolov et al., 2013).

distributed word representations (embeddings) become more and more popular.


Except for the publicly available coreference resolution system, our approach does not need external resources such as dictionaries or bilingual corpora and no human labor is required.

it can be easily applied to any corpus in any language as long as there exists a coreference resolution system in this language.


Loughran and McDonald (2011) states that a general purpose sentiment lexicon (e.g., the Harvard Psychosociological Dictionary) might misclassify common words in financial texts.

in this paper, we use a finance-specific lexicon that consists of the 6 word lists provided by (Loughran and McDonald, 2011) as seeds to expand keywords.


We choose to consider the frequency of a n-gram ng i as the number of its occurrence in d such that it is counted once per fragment.

the minimum value that could take a frequency is 1 if ng i appears only in one fragment, and its maximum value is |S| (the number of fragments in d) if ng i occurs in each fragment s i ∈ S. 3.


• Using non-standard spelling of words such as "SAbE" (finger) instead of "<sbE" in MSA.

broken plurals may also be non-standard.


Intuitively P alt (w i ) is proportional to the number of unique words that precede w i .

words that appear in many different contexts will be given higher weight than words that consistently appear after only a few contexts.


Then B (1) ∝ P (w i ) P (w i−1 ) and ∀w i−1 s.t.

c(w i−1 ) = 0: ..,i−n+2 be the best rank κ approximation of C n (: by selecting 1 < κ < V , we obtain count matrices and tensors between n and n − 1-grams.


Since KN smoothing alters the lower order distributions instead of simply using the MLE, varying the rank is not sufficient in order to generalize this suite of techniques.

pLRE computes low rank approximations of altered count matrices.


1 will generally not hold if this strategy is used.

we propose a generalized discounting scheme where each nonzero n-gram count is associated with a different discount ).


The Small-English dataset has a rather small vocabulary compared to the number of tokens, leading to lower data sparsity in the bigram.

the PLRE improvement is small for order = 2, but more substantial for order = 3.


PMI can be exploited as a semantic similarity measure (Han et al., 2013) and it is calculated as: To calculate the PMI value of a candidate word and a specific sense, we consider p(x) as the probability of the candidate word to occur in the corpus.

p(x) is calculated as p(x) = c(x)/N , where c(x) is the total count of the occurrences of the candidate word x in the corpus and N is the total co-occurrence count of all words in the corpus.


Manually producing or extending lexicons is much more labor intensive than generating VSM word vectors using a corpus.

lexicons are sparse with missing words and multiword terms as well as missing relationships between words.


Nevertheless, certain models remain somewhat opaque in this regard, particularly the recent window-based methods like skip-gram and ivLBL.

in this subsection we show how these models are related to our proposed model, as defined in Eqn.


noun phrase are noun phrases, and (subject-) verbobject phrases can be regarded as complete sentences.

different kinds of context information might be required for both groups.


Evaluating B k and C k requires O(nr 2 ) and the matrix inversion requires O(r 3 ).

the complexity of updating A is O(pr+nr 2 ) assuming n r. The updating rule of R k involves inverting an r 2 × r 2 matrix.


Therefore, the complexity of updating A is O(pr+nr 2 ) assuming n r. The updating rule of R k involves inverting an r 2 × r 2 matrix.

directly computing the inversion requires time complexity O(r 6 ) and space complexity O(r 4 ).


(2012) considered using QR decomposition to simplify the updates, it is still time consuming with the time complexity O(r 6 + pr 2 ).

the total time complexity is O(r 6 +pr 2 ) and the step of updating R k is the bottleneck in the optimization process.


Complexity Analysis For RESCAL, V and Σ can be computed by finding eigenvectors of A T A.

computing SVD of A costs O(nr 2 + r 3 ) = O(nr 2 ).


Using a similar derivation, the time complexity of updating R k in TRESCAL is O(nr 2 + pr).

the total complexity of each iteration is O(nr 2 + pr).


We suspect that the reason is that in our construction, each entity has its own vector representation, which is lacked in RI13.

the new triples that TRESCAL finds are very different from those found by RI13.


also contains mentions of Turing and King s College, but do not express the relation Attended.

weak supervision lead to noisy examples.


This means that the matrix Y contains exactly one 1 per line, which is equivalent to the constraint: Second, if the pair i of entities verifies the relation k we suppose that at least one relation mention candidate indeed express that relation.

we want to impose that for at least one relation mention candidate n such that E in = 1, we have Y nk = 1.


This is equivalent to the constraint: Third, if the pair i of entities does not verify the relation k, we suppose that no relation mention candidate express that relation.

we impose that for all mention candidate n such that E in = 1, we have Y nk = 0.


The componentwise projection operators associated to the constraints on Σ and Λ are defined by: The complexity of projecting Σ and Λ is O(N K).

the cost of those operations is ne gligible compared to the cost of computing the gradients of the dual cost function.


The embeddings are usually learnt by minimizing a global loss function of all the entities and relations in the knowledge graph.

the vector of an entity may encode global information from the entire graph, and hence scoring a candidate fact by designed vector operations plays a similar role to long range "reasoning" in the graph.


The capabilities and limitations of knowledge embedding and word embedding have inspired us to design a mechanism to mosaic the knowledge graph and the "word graph" together in a vector space so that we can score any candidate relational facts between entities and words 1 .

we propose a novel method to jointly embed entities and words into the same vector space.


it is almost impossible to estimate a vector for each r wv .


If we only have the knowledge model and text model, the entity embeddings and word embeddings will be in different spaces and any computing between them is meaningless.

we need mechanisms to align the two spaces into the same one.


As a result, for most of the anchors in Wikipedia, each of which refers to a Wikipedia page, we know that the surface phrase v of an anchor actually refers to the Freebase entity e v .

we define a likelihood for this part of anchors as Eq.


(6) in the same way by choosing corresponding negative distributions respectively.

the objectives of both the knowledge model L K (Eq.


However, when all triplets are ranked together, assigning scores in a more uniform scale is definitely an advantage.

the contradiction stems from the different training strategies of the two models and the consideration of relation-specific thresholds.


In addition, the number of entities mentioned by anchors is very limited com- pared with |E|.

interactions brought in by anchors are not as significant as that of the name graph.


On the other hand, neither Sm2r nor TransE is a probabilistic model.

we combine Sm2r with TransE or the "jointly" model according to the scheme proposed in  where for each candidate (h, r, t), if r =r δ(Score(h, r, t) < Score(h, r , t)) is less than τ , we increase Score Sm2r (h, r, t) by p. We search for the best β, τ , and p on another dataset-Wikipedia corpus distantly labeled by Freebase.


This method does not provide a well-formed grammatical abstract and the generated summary only contains words that occur in the original texts.

this approach is more extractive than abstractive.


We ran two different user studies because: i) for each product there might be many reviews to be included; ii) there is no guaranty that raters, in various evaluation settings, read   the reviews (partially or completely); and iii) there is no evidence regarding the depth that each rater would look into the reviews.

choosing between user study 1 and 2 is not a straightforward decision.


The main hypothesis is that the sentences in the gold related work sections should summarize the target paper and reference papers as well.

the sentences in the papers which are more similar to the sentences in the gold related work sections should be considered more important and suitable to be selected.


All the features are scaled into [0, 1].

we can learn SVR models based on the features and importance scores of the sentences, and then use the models to predict an importance score for each sentence in the test set.


Constraint (13): This constraint makes sure that at least one sentence in or is selected if bigram is selected.

we transform our optimization problem into a linear programing problem.


This information is used to find further coreferences that no single pair of documents would allow.

missing a coreference pair may result in losing the possibility of realizing further coreferences.


However, for two mentions matching a misspelling pattern which is highly accurate, the threshold for contextual evidence is lowered.

correcting a misspelling is not beneficial for a single mention only, but for the accuracy of the whole.


(b) shows, the source-target probability distributions are calculated in a complete probability space.

it will be more accurate than the traditional triangulation method.


The pivot-based translation is suitable for the scenario that there exists large amount of source-pivot and pivot-target bilingual corpora and only a little source-target bilingual data.

we randomly select 10K, 50K, 100K, 200K, 500K, 1M, 1.5M sentence pairs from the source-target bilingual corpora to simulate the lack of sourcetarget data.


Another issue is that, given a phrase segmentation, each source phrase is translated independently from the others, which can be problematic especially for short phrases.

the predictive translation of a source phrase does not access useful linguistic clues in the source sentence that are outside of the scope of the phrase.


As shown in Table 1, such a model could potentially increase the reference/MT-output recall by 12.3% with generation of new inflected forms, and by 11.2% without.

also when it comes to morphology, generation seems to be of secondary importance compared to better selection in our experimental setup.


A log-bilinear model instead could directly model word prediction through a factored representation of target words, similarly to Botha and Blunsom (2014).

no probability mass would be wasted over stem-suffix combinations that are not in the candidate generation function.


Each row of m i is convolved with the corresponding row in the previous layer resulting in a weight matrix whose number of columns decreases by one.

after 2k convolutional layers, the network transforms the source context matrix m cs i to a feature vectorq ∈ R d .


A general bilingual dictionary is then used to translate/project the target context vectors into the source language.

the source and target context vectors become directly comparable.


The hybrid model that combines the compositional and context clue, is based on a two-feature model.

the model converges using only a few hundred instances.


The high dimensionality of vectors, however, is a barrier to the performance of methods that employ VSMs.

a dimensionality reduction technique is employed to alleviate this problem.


the familiar Zipfian distribution of words).

extremely highdimensional vectors, which are also sparse-i.e.


Following the construction of the VSM, the dimension of the VSM is reduced in an independent process.

the VSM at reduced dimension is available for processing only after the whole sequence of these processes.


Hence, Li (2008) suggests using the geometric mean estimator instead of the median sample: 9 We suggest computing theL 1 ( u, v) in Equation 9 using arithmetic mean of logarithm-transformed values of |u i − v i |.

using the logarithmic identities, the multiplications and the power in Equation 9 are, respectively, transformed to a sum and a multiplication: Equation 10 for computingL 1 is more plausible for computational implementation than Equation 9 (e.g.


The procedure is repeated for all the 31 words in the list, one by one.

the procedure results in 31 sorted lists, each containing 30 words.


We expect the obtained relative 1 distances between each reference word and the 30 other words in an RMI-constructed VSM to be the same as the obtained relative distances in the original high-dimensional VSM.

for each VSM that is constructed by the RMI technique, the resulting sorted lists of words are compared by the sorted lists that are obtained from the original high-dimensional VSM.


the probability of preserving distances increases).


The proposed method for the generation of index vectors in RMI results in index vectors of non-zero elements that are real numbers.

index vectors and thus context vectors are arrays of floating point numbers.


3.1, we further expect the RNN Encoder-Decoder which was trained without any frequency information to score the phrase pairs based rather on the linguistic regularities than on the statistics of their occurrences in the corpus.

we focus on those pairs whose source phrase is long (more than 3 words per source phrase) and the probability of words not in the shortlist is always overestimated.


For example, the highlighted tree fragment of the forest in Figure 2 corresponds to the SCFG rule: Thus any valid SCFG rule can be formed by selecting a set of adjacent hyperedges from the forest and composing the minimal SCFG rules specified by each hyperedge.

the problem of SCFG rules extraction can be solved by sampling tree fragments from the phrase decomposition forest.


A tree can be specified by attaching a variable e n to each node n in the forest, indicating which hyperedge is turned on at the current node.

each assignment will specify a unique tree by tracing the edge variables from the root down to the leaves.


We can see from Equation 3 that the only history related to the current table assignment is the counts in z −i .

we define a table of counts N = {N C } C∈I which memorizes different categories of counts in z −i .


Assuming the j th document in the m th domain contains N j m words, each word is drawn according to H .

our four-level DP model can be summarized formally as follows: Here, we provide for our model the Chinese Restaurant Process (CRP) metaphor, which can create a partition of items into groups.


It is indeed difficult to find a standard evaluation corpus for our joint tasks, especially in different domains.

we spent a lot of time to collect and annotate a new corpus 1 composed of ten domains (including Physics, Computer, Agriculture, Sports, Disease, Environment, History, Art, Politics and Economy) and each domain is composed of 200 documents.


linear penalties for the GP and SVM respectively.

when training scores are more uniformly distributed (which is the case for fear), the GP model has to take the high scores into account, resulting in broader coverage of the full support.


Given the current fixation position s t , the text X, and the chosen saccadic type u t+1 , the amplitude is constrained to fall within a specific interval-for instance, within the characters of the currently fixated word for refixations.

we model the distribution over the saccade amplitude given the saccadic type (Equation 4) as truncated Gamma distributions, given by 1 is the Gamma distribution with shape parameter α and scale parameter β, and Γ is the Gamma function.


The natural choice of the code vector c c c is h h h. However, when testing an instance, the labeling is unknown (exactly what we try to predict), consequently we can not capture h h h with PCA to replace the code vector c c c in the prediction function Eq.(1).

we assume a linear transformation M M M from the training instances to their independent components, and use M M Mx x x as the approximate independent component.


In this example, the parser failed to determine the most salient EDU e 2 , that is the root EDU of the gold DEP-DT.

the summary extracted from this DEP-DT will have a low ROUGE score.


(2013) proved that TKP-HILDA outperformed other methods including Marcu's method (Marcu, 1998), a simple knapsack model, a maximum coverage model and LEAD method that simply takes the first L tokens (L = summary length).

we only employed TKP-HILDA as our baseline.


In addition, the theory of eventualities (Dölling, 2011) suggested that relations can be viewed as states that events start from and result in.

it is intuitive but challenging to extract all of them simultaneously in a single model.


The standard perceptron will cause invalid updates because of inexact search.

we apply earlyupdate (Collins and Roark, 2004), an instance of violation-fixing methods (Huang et al., 2012).


The co-occurrence of the full form and the abbreviation makes it easier for us to mine the abbreviation pairs from the large amount of news texts.

given a long full form, we can generate its abbreviation candidates from the given corpus, instead of doing the character tagging job.


We believe that a better way is to extract these abbreviation-full pairs from a natural text corpus where the full form and its abbreviation co-exist.


Suppose the length of a candidate is a, the length of the reference is b and the length of their LCS is c, then we can define precision P and recall R as: It is easy to see that F is a suitable s(x, y).

we can use the F-score as the value for s(x, y).


Intuitively, abbreviations tend to be short.

length can be an important feature for the reranking.


The reason for this phenomenon may lie in the fact that the verification data we use is news text, which tends to be formal.

when a reference is often used colloquially, our method may miss it.


Named entity linking (NEL) is the task of grounding textual mentions to entities in a KB, such as Wikipedia.

"named entity" here, has a somewhat broader definition than in NER -these are any entities in the KB, not just those expressed with proper names.


For NIL links, we use the NER type of PERSON, ORGANIZATION, or LOCATION, if available, mapping it to appropriate Freebase types.

of a larger training set, as well as model averaging, our baseline, which is otherwise equivalent to the methods of Hoffmann et al.


Experimental results also show significant differences in accuracy between newswire data and webblog data, and smaller but still recognizable difference between v events and all events.

we specifically look into distinctions between these data sets.


In particular, genres such as marketing flyers and info-graphics often augment textual information by its color, size, positioning, etc.

traditional text-based approaches to information extraction (IE) could underperform.


This means that facts are not always indefinitely true.

temporal scope has crucial implications for KB accuracy.


Our main insight is as follows: events that happen to an entity change the entity's state and therefore its facts.

by learning when a given entity undergoes a specific state change, we can directly infer the time scopes of its facts.


Definition 2 ( Aggregate State Vector for e) The aggregate state vector of a mean entity e for state change sc i , V s(e, sc i ), is made up of the contextual units from the CTPs of entities in the seed set S(sc i ) that undergo state change sc i .


Thus, we have: e,t:(e,t)∈S(sc i ) C e (t).

the state vector V s(e, sc i ) reflects events happening to e and the state of e at the time it undergoes the state change sc i .


Firstly, we have formulated these as ILP problems.

of this, the approximate inference therein is replaced by an exact inference procedure.


In practice, since future-cost precomputation already solves the best (monotonic) model-score for each span, is the only extra work for potential BLEU is to record (for each span) the subtranslation that achieves that best score.

the extra time for potential BLEU is negligible (the time complexity is O(n 2 ), but just as in future cost, the constant is much smaller than real decoding).


On the other hand, it is well-known that search errors can undermine the standard training for many beam search based NLP systems (Huang et al., 2012).

collins and Roark (2004) and Huang et al.


In this work, we refine the one-category grammar introduced by Chiang (2007) for HPBT in order to learn additional latent NT categories.

the following discussion is restricted to these kinds of grammars, although the method is equally applicable in other scenarios, e.g., the extended treeto-string transducer (xRs) formalism (Huang et al., 2006;Graehl et al., 2008) commonly used in syntax-directed translation, and phrase-based MT (Koehn et al., 2003).


In addition, when new data comes along, a new model must be generated again with all the data in Twitter-LDA because it does not assume online inference.

it cannot efficiently analyze the large number of tweets generated everyday.


In previous research, the simple method of counting first-person pronouns was used to measure the degree of self-disclosure (Joinson, 2001; Barak and Gluck-Ofri, 2007).

the absence of a first-person pronoun signals that the utterance belongs in the G level of self-disclosure.


Judges first read and discussed the definitions and examples of self-disclosure level shown in (Barak and Gluck-Ofri, 2007), then they worked separately on a Web-based platform.

of annotation, there are 122 G level converstaions, 147 M level and 32 H level con- versations, and inter-rater agreement using Fleiss kappa (Fleiss, 1971) is 0.68, which is substantial agreement result (Landis and Koch, 1977).


precision recall F1 0.82 0.86 0.84 Table 7: Performance for identifying properties.

far we have described how to automatically identify tweets referring to major life events.


We treat idioms as semantic outliers, and the identification of a semantic shift as outlier detection.

this topic representation allows us to differentiate idioms from literals using local semantic contexts.


We treat idioms as semantic outliers, and the identification of semantic shift as outlier detection.

this topic representation allows us to differentiate idioms from literals using the local semantics.


Some examples of such properties include 1) lexical fixedness: e.g., neither 'shoot the wind' nor 'hit the breeze' are valid variations of the idiom shoot the breeze and 2) syntactic fixedness: e.g., The guy kicked the bucket is potentially idiomatic whereas The bucket was kicked is not idiomatic anymore; and of course, 3) non-compositionality.

some approaches look at the tendency for words to occur in one particular order, or a fixed pattern.


we think that idioms have the tendency to appear in more affective contexts; and we think that incorporating more sophisticated sentiment analysis into our model will improve the results.


Knowledge of such common facts provides the context within which people communicate with language.

to create practical systems that can interact with the world and communicate with people, we need to leverage such knowledge to interpret language in context.


The authors demonstrated the promise of text to scene generation systems but also pointed out some fundamental issues which restrict the success of their system: much spatial knowledge is required which is hard to obtain.

users have to use unnatural language (e.g., "the stool is 1 feet to the south of the table") to express their intent.


For example, the oracle fscores for 5-and 20-best document-level parsing are only 56.91% and 57.65%, respectively.

the scope of improvement for the reranker is rather narrow at the document level.


In our reranking framework, we also include traditional feature vectors in addition to the trees.

each hypothesis h is represented as a tuple T, v composed of a tree T and a feature vector v. We then define a structural kernel (i.e., similarity) between two hypotheses h and h as follows: where DISCTK maps the DTs T and T to JRN or SRN and then applies STK, STK b or PTK defined in Sections 2.2 and 3, and F V is a standard kernel, e.g., linear, polynomial, gaussian, etc., over feature vectors (see next section).


In the last section we presented a framework for link-based error analysis, which extracts errors by comparing entity spanning trees to entity partitions.

we can accommodate different  Output: recall errors, precision errors notions of errors by varying the algorithm for computing spanning trees.


We first observe that for error extraction, the structure of the spanning trees of the subentities appearing in r S does not play a role.

edges present in r S are not candidates for errors, since they appear in both the reference entity r and the system output S. it does not matter which edges from the subentities are in the spanning tree.


The coreference resolution systems presented in the previous section are a representative sample of the state of the art.

by analyzing the errors they make, we can learn about remaining challenges in coreference resolution and analyze the qualitative differences between the systems.


We first observe a fundamental difference: evaluation metrics deal with scoring coreference chains, they provide no means of extracting recall or precision errors.

our analysis complements insights obtained via evaluation metrics.


The only frequent syntactic property shared is that bridging anaphors most often have a simple internal structure concerning modification.

we first create an initial list of potential bridging anaphora A which excludes NPs which have a complex syntactic structure.


An initial effort to extract partOf relations using WordNet yields low recall on the development set.

we use semantic connectivity expressed by prepositional patterns (e.g.


Essentially, the discourse hierarchy indicates the correlation degrees of semantic relations in the discourse, the deeper tree level of two discourse units, the higher correlation degree of their semantic relation.

a discourse relation is the ultimate factor for the choice of hierarchical discourse structure.


It is important to note that the distribution of states in the pruned space due to F prune on the testing data may be somewhat different from those on training data.

we train our scoring function via cross-validation by training the scoring function on heldout data that was not used to train the pruning function.


Recall that our features φ(s, a) for both pruning and scoring functions are defined over state-action pairs, where each state s consists of a set of clusters and an action a corresponds to merging an unprocessed mention m with a cluster C in state s or create one for itself.

φ(s, a) defines features over cluster-mention pairs (C, m).


However, in OntoNotes corpus, the training data does not include singleton clusters for the gold mentions.

only the large number (57%) of system mentions that are not part of gold mentions will constitute the set of singleton clusters.


This leads to a highly imbalanced learning problem because our model needs to learn (the weight of the single indicator feature) to predict NEW as the best action for a large set of mentions, which will bias our model to predict large number of NEW actions during testing.

we will generate many singleton clusters, which will hurt the recall of the mention detection after post-processing.


Our Prune-and-Score approach is a particular instantiation of the general idea of learning nearlysound constraints for pruning, and leveraging the learned constraints to learn improved heuristic functions for guiding the search (See (Chen et al., 2014) for another instantiation of this idea for multi-object tracking in videos).

other coreference resolution systems Björkelund and Kuhn, 2014) can also benefit from this idea.


The topic starter tries the suggested solution and reports his experience in the next post (sixth post).

we see that each individual post in a discussion thread serves a different purpose in the discussion and we posit that identifying the purpose of each such post is essential for creating effective summaries of the discussions.


Hence, selecting just a few sentences from a message may lead to loss of context and make the resulting summaries hard to comprehend.

in this work, we choose each individual message as a text unit and thus, the thread summaries are created by extracting most relevant posts from a discussion.


However, in our application, the state representation is not provided, but has to be inferred from a textual description.

it is not sufficient to extract features from text to supplement a simulation-based player.


Scores for all actions are predicted simultaneously, which is computationally more efficient than scoring each state-action pair separately.

by combining the representation generator and action scorer, we can obtain the approximation for the Q- An additional complexity in playing MUDgames is that the actions taken by the player are multi-word natural language commands such as eat apple or go east.


However, both the room descriptions (which are also varied and randomly provided to the agent) and the quest descriptions were adversarially created with negation and conjunction of facts to force an agent to actually understand the state in order to play well.

this domain provides an interesting challenge for language understanding.


However, this is a complex quest that requires the player to memorize game events and perform high-level planning which are beyond the scope of this current work.


However, since distributional models represent words as aggregated distributions of their contexts, and compute semantic similarity from these context distributions, the contexts that they use need to be generic enough to yield meaningful overlap between concepts (e.g., words).

distributional models cannot easily represent knowledge of the form "the value for property Y of word/concept X is Z".


Secondly, every permutation is factorizable into prime permutations only (Albert and Atkinson, 2005).

pETs expose maximal sharing between different permutations in terms of both phrases and their reordering.


A data-driven approach to learning these rewriting rules is hampered by the dearth of parallel data: we have few examples of text that have been both interpreted and translated.

we design syntactic transformation rules based on linguistic analysis of the source and the target languages.


Although our rules are designed to minimize long range reordering, there are exceptions.

6 applying a rule does not always reduce delay.


In this example, the delay of the original sentence is 2.5 word/seg; after rewriting, it is reduced to 1.7 word/seg.


However, combining RW and GD results in a better speed-accuracy tradeoff: the RW+GD curve completely dominates other curves in Figure 3a, 3c.

using more monotone translations improves simultaneous machine translation, and because RW-LM+GD is about  the same as GD, the major improvement likely comes from the translation model from rewritten sentences.


Moreover, all people have certain needs that they want to satisfy.

analyzing a comprehensive set of personal traits may help us understand the way we react to a particular brand.


Since different surveys used different scales, we normalized the trait scores by using their rank percentile (e.g., top 1%, top 5%).

all the normalized personal trait scores are between 0 and 1.


Since we need a large Twitter dataset to mine a list of general topics of interest, our current tweet collection is not sufficient.

we use a separate and much larger Twitter dataset from 10,000 randomly selected Twitter users.


For each user, we crawled his tweets and then aggregated them into a big document, one for each Twitter user.

we have 10,000 documents in our dataset.


Finally, for our models, since the overall correlation coefficients ρ are between 0.3 and 0.4, the strength of these correlations is moderate.

it may not be sufficient to build an accurate brand preference prediction system with only user features.


Processing a huge amount of text, especially during a hashtag burst time period, incurs expen-sive computational costs.

to guarantee a good recall in this step while still maintaining feasible computation, we apply entity linking only on a random sample of the complete tweet set.


Links come more from prominent entities to marginal ones to provide background, or more context for the topics.

starting from prominent entities, we can reach more entities in the graph of candidate entities effective metric that is agnostic to the scaling and time lag of time series (Yang and Leskovec, 2011).


Iterative Influence-Prominence Learning (IPL) IM itself is an NP-hard problem (Kempe et al., 2003).

we propose an approximation framework, which can jointly learn the influence scores of the entity and the entity prominence together.


Dataset There is no standard benchmark for our problem, since available datasets on microblog annotation (such as the Microposts challenge (Basave et al., 2014)) do not have global statistics, so we cannot identify the trending hashtags.


Searching for the best translation under this score definition is difficult in general.

approximate decoding algorithms such as beam search should be applied.


p.score = , when the current partial summary is S i−1 .

during each iteration of the greedy selection process, the compression procedure will also be affected by sentences that have already been included.


The lexicon consists of words and phrases and their degree of formality.

more formal words are marked on a positive scale and informal words like those occurring in colloquial language are marked on a negative scale.


Although parallel corpora have been used successfully for inducing bilingual lexicons for some languages (Och and Ney, 2003), these corpora are either too small or unavailable for many language pairs.

mono-lingual approaches that rely on comparable instead of parallel corpora have been developed (Fung and Yee, 1998;Koehn and Knight, 2002).


However, these models typically rely on the availability of bilingual seed lexicons to produce shared bilingual spaces, as well as large repositories of comparable data.

several approaches attempt to learn lexicons from large monolingual data sets in two languages (Koehn and Knight, 2002;Haghighi et al., 2008), but their performance again relies on language pair-dependent clues such as orthographic similarity.


In other words, what we are inter-ested in for this particular task is genuine similarity, rather than relatedness.

we can evaluate the quality of our similarity metrics by comparing their performance on similarity and relatedness tasks: if a metric performs well at measuring genuine similarity, this is indicative of its performance in the bilingual lexicon induction task.


The core idea behind BRAE is that a phrase and its correct translation should share the same semantic meaning.

they can supervise each other to learn their semantic phrase embeddings.


Obviously, we should have |S| ≤ |V | as it is a summary and should be small.

constraints on S can naturally be modeled as knapsack constraints: where c i is the non-negative cost of selecting unit i (e.g., the number of words in the sentence) and b is our budget.


Third, the countries are probably negative toward Imam, because the countries are negative toward fatwa and it is Imam who issued the fatwa.

the set P should also contain {(Imam, Prophet), (writer, fatwa)}, and the set N should also contain {(writer, Rushdie), (countries, Imam)}.


Joint prediction is critical for our task because it involves multiple, mutually constraining ambiguities (the source, polarity, and target).

this work aims at detecting both implicit and explicit sentiments expressed by an entity toward another entity/event (i.e., an eTarget) within the sentence.


SOURCE(y,s): S1 extracts the source of each opinion, S2 does not extract the source, and S3 assumes the source is always the writer.

for an opinion y, if the source s is assigned by S1, a ground atom SOURCE(y,s) is created with score 1.0.


Though the PSL inference does not need supervision and the SVM classifier for agents and themes in Section 4.4 is trained on a separate corpus, we still have to train the eTarget SVM classifier to assign local scores as described in Section 4.2.

the experiments are carried out using 5-fold cross validation.


But not all eTargets are equally important.

our first evaluation assesses the performance of extracting the most important eTarget.


As introduced in Section 3.1, a span-based target annotation of an opinion in MPQA 2.0 captures the most important target this opinion is expressed toward.

the head of the target span can be considered to be the most important eTarget of an opinion.


Twitter is a popular microblogging platform and is widely used for sentiment analysis.

it is an excellent source for collecting similes that people use in everyday conversation.


This can potentially bias the training data and limit the classifier's ability to learn to recognize affective similes that do not contain words with a positive or negative connotation.

we explore an additional approach where instead of judging the sentiment of the words in the simile, we analyze the words in the tweet surrounding the simile.


We did not create a classifier to solely identify neutral similes because neutral similes are much less common than positive/negative similes, making up only 8.7% of the extracted similes in our development set (Table 3).

obtaining a large set of neutral similes via manual annotation would have required substantially more manual annotation effort.


For Flickr30k, the descriptive phrases may cause data sparseness (the furry, black and white dog).

we extracted the lemmatised head word of each phrase, using a 'semantic head' variant of the head finding rules of Collins (2003) in Stanford CoreNLP.


The cosine distributions, Θ i , are based on the best matches for each category i, and normalized by the total mean and variance amongst all categoriesΘ i = Θ i −µ σ .

the figure illustrates how well each category is separated from each other, the larger separation the better.


Each sentence contains very little additional data, and even if the language model is adjusted after every individual word, it only obtains evidence of previous words in the sentence, whereas these words are relatively unlikely to occur again in the same sentence.

instead of adjusting individual word representations, our approach learns a distributed document vector to represent the specific unit of text that is currently being processed.


Since our implementation performs the un-rolling only at the end of each sentence, the updates would not have any effect.

we attach the document vector directly to the output layer, in parallel with the recurrent hidden component.


While the general RNNLM is a smooth static representation of the entire training data, the document vector is optimised to represent how each sentence differs from the main language model.

we performed a qualitative evaluation and found that the learned sentence vectors were also very good predictors of semantic similarity.


To train a large margin classifier, we train all the parameters such that the score of an utterance is bigger than a margin for its labels and less than the negative margin for all other labels.

the loss function is as follows: (2) where θ is all the parameters of the model, namely φ(w i ) (word embeddings), W word , W P arse , W previous , W next , W ih , and W ho .


In our setting, we observe posts over a target rumour i for one hour and over reference rumours (other than i) for two hours.

the training data set is represents number of posts observed for i th rumour).


Intuitively, documents in the same clique are likely to have similar topic distributions.

each of the L cliques l (the "birds of a feather" of our title) is assigned a separate Dirichlet prior π l over K topics.


As reported in the paper, using entity names severely pollutes the embeddings of words.


Our results for medium rare words -those that occur between 50 and 100 times -are less clear: either there are no improvements or improvements are small.


In such cases, distributional initialization makes the learning task easier since in addition to the contexts of the rare word, the learner now also has access to the global distribution of the rare word and can take advantage of weight sharing with other words that have similar distributional representations to smooth embeddings systematically.

distributional initialization is a form of smoothing: the embedding of a rare word is tied to the embeddings of other words via the links shown in Figure 1: the 1s in the lower "rare words" part of the illustrations for separate and mixed initialization.


Although GloVe and CBOW are the two best performing embeddings on the intrinsic tasks, they differ vastly in the amount of frequency information they encode.

we can conclude that most of the differences in frequency prediction are not due to intrinsic properties of natural language: it is not the case that frequent words naturally have only frequent neighbors.


This is a concern: the frequency of a word in the language plays a critical role in word processing of humans as well (Cattell, 1886).

we need to explicitly consider word frequency as a factor in the experiment design.


However, inference is cumbersome for LDA models with prior knowledge.

lDA models that use prior knowledge only work in small-scale scenarios.


Moreover, accurate training usually takes many sampling passes over the dataset.

for large datasets with millions or even billions of tokens, conventional Gibbs sampling takes too long to finish.


Moreover, scaling topic models to many topics can also reveal the hierarchical structure of topics (Downey et al., 2015).

there is a need for topic models that can both benefit from rich prior information and that can scale to large datasets.


A correlation between two words v and w indicates that they have a similar topic distribution, i.e., p(z|v) ≈ p(z|w).

1 the posterior topic assignments v and w will be correlated.


However, if the second term, exp f m (z, w, d), is dense, we still need to compute it explicitly T times (once for each topic) because we need the summation of P (z = t) for sampling.

the critical part of speeding up the sampler is finding a sparse representation of the second term.


It assumes that there is a one-to-one mapping between topics and labels, and it restricts each document's topics to be sampled only from those allowed by the documents label set.

labeled-lDA can be expressed in our model.


Moreover, the document-topic count is also sparse, which fits into the efficient learning framework.

for different types of prior knowledge, as long as we can define ψ(z, M ) appropriately so that f (z, w, d) are sparse, we are able to speed up learning.


Second, we find that somewhat surprisingly, compositional training also improves upon state-of-the-art performance for knowledge base completion, which is a special case of answering unit length path queries.

compositional training can also be seen as a new form of structural regularization for existing models.


It returns the top m highest scoring trees in D (in our experiments we used m = 200, 000), where the score for each tree is the perceptron-based score normalized by the sentence length.

we return the  Table 1: Statistics showing the accuracy for various definitions of projected trees: see §3.2 for definitions of P, P 100 etc.


When selecting mini-batches for online gradient updates, we select an equal number of classification instances from the source and target languages.

for each step |D s | = |D t |, effectively reweighting the cross-entropy components in (2) to ensure parity between the languages.


In a real world sce-nario, getting development data to tune this parameter is difficult.

we want a parameter that can work well cross-lingually.


Indeed, core problems in natural language processing such as translation (Cho et al., 2014;Sutskever et al., 2014;Luong et al., 2014), parsing (Vinyals et al., 2014), image captioning (Vinyals et al., 2015;Xu et al., 2015), or learning to execute small programs  employed virtually the same principles-the use of Recurrent Neural Networks (RNNs).

with regard to this line of research, our work comes closest to the recent machine translation work.


Furthermore, in his publication McDonald states clearly that different values of K did not actually have a major impact on the final performance of the model.

and for the sake of being able to successfully train the model with largescale data, the learning procedure is implemented as a distributed structured perceptron with iterative parameter mixing (McDonald et al., 2010), where each shard is processed with MIRA and K is set to 1.


Because manual syntactic annotations are relatively limited and highly expensive, it is necessary to use large-scale automatically-parsed sentences for training syntactic language models.

the syntactic structures that a word ordering system learns can be inaccurate.


On the other hand, the two models lead to different error distributions in word ordering.

we combined the advantages of both systems by integrating a syntactic model trained with relatively small data and an N-gram model trained with relatively large data.


The problem of inconsistency between the degree of certainty of expressing findings between the citing article and referenced article has been also reported (De Waard and Maat, 2012).

citations by themselves lack the related "context" from the original article.


While capturing domain concepts along with noun phrases helps improving the performance, adding related concepts to the citation vector causes drift from the original context as expressed in the reference article.

some decline in performance is incurred.


To create generalized models for incident type classification (and social media classification in general) the most important step is an appropriate feature generation.

in this paper we investigate the suitability of standard and novel features and different machine learning algorithms for the creation of generalized classification models for incident type classification.


To generate subsets we used the approach of (Schulz et al., 2013) of extracting microposts using incident-related keywords.

more than 200 keywords were identified for each class.


In conjunction with the named entities present in tweets, people tend to refer to street names (e.g.,I-95) or the number of injured people (e.g.,2-people).

we create a feature for the number of cardinal numbers present in a tweet.


Finally, we aim to pinpoint evidence in a clearly defined context, given by the pre-specified claim.

the developed system should not only find pieces of text that have general evidence characteristics but further identify which of these candidates can be used to support a specific claim.


Reaching a full coverage requires examining segments up to 25 sentences, which would vastly increase run time, for a relatively small gain.

for example, for a single paragraph with five sentences, our system will examine a total of 5 + 4 + 3 = 12 candidates.


The W2V baseline handles CDED as a purely semantic relatedness task using state of the art semantic relatedness measure of Word2Vec (Mikolov et al., 2013).

we use the average cosine similarity between the Word2Vec representations of all words in a given candidate to all words in the claim, to rank all relevant candidates with respect to each claim.


It is a central idea of the WMTs that automatic measures of machine translation quality are an imperfect substitute for human assessments.

manual evaluation of the system outputs are conducted and their results are reported as the final rankings of the workshops.


In our work, we wish to learn that they cause different state changes.

we can update the entity's fact and its temporal scope (Wijaya et al., 2014a).


For example, when a death happens, slots regarding birth e.g., birthdate, birthplace, may also be updated or added if they were missing before.

our method has to handle these sources of noise.


Additionally, most Wikipedia revisions only have text changes without the associated infobox change.

another line of future work is to also learn from these unlabeled documents.


The main goal of the simplified English resources is to make HeidelTime's original English resources amenable to automatic translation.

rules are addressed in a language-independent fashion (cf.


4 Obviously, it is crucial to use only the translations of the intended meaning.

in the case of multiple meanings of English patterns, we provide the names of Wiktionary's correct translation tables to our translation script, so that only translations of the correct meaning are generated.


This differs from a traditional CRF in that the feature function depends on the additional variables e w , which are the embeddings (as defined above).

the objective is no longer log-linear, but log-bilinear 8 .


6, especially in the case when none of predicate paths are similar enough to the relation words.

we trust the factor of P ( p | w) only when there exists a similar predicate path.


The latter one shows a more concrete representation of the relation, because deceased person covers small entities than person in the knowledge base.

the schemas cannot be ranked by using the support alone.


It is easy to derive that for different values of r, the only difference in P (r|e, d) is the sign within the logistic function.

we adopt the general representation of .


However, even if source-based feature vector holds a few dimensions (10 in our experiments), src LDT improves the precision on the basis of GM.

the precision can be enhanced further if we can develop more valuable features to represent the underlying document types.


But both types of systems frequently represent content that the user has already consumed, leading to delay and frustration.

identifying novel information has been an essential aspect of studies on news information retrieval.


Without the use of the context, the time complexity of our algorithm would depend on the number of sentences so far.

the features and the model for the context are important for efficiency.


This measure, however, does not necessarily reflect human perception of novelty given the prevalence of nonlinearity in human perception (Kingdom and Prins, 2009).

we explored the use of a simple nonlinear transformation of unseen word counts instead of the plain count ( Figure 2): where n is the number of new words and α, β and γ are parameters.


Unlike traditional question answering (QA), in CQA answering takes the form of commenting in a forum.

many comments are only loosely connected to the original question, and some are not answers at all, but are rather interactions between users.


Whereas CCA can handle only two modalities, we need to consider relations between three modalities because we use images in addition to the two languages.

we focus on an extension of CCA, generalized canonical correlation analysis (GCCA), to handle more than two inputs (Kettenring, 1971).


The use of words, however, varies strongly across domains, as illustrated in Figure 1 for a product, a hotel, and a movie review.

sentiment analysis suffers from domain dependence (Wu et al., 2010), i.e., high effectiveness is often achieved only in the domain an approach has been specifically modeled for.


Gleaning different sources of information, neural features and discrete linear features comple-ments each other.

a model that integrates both features can potentially achieve performance improvements.


One thing to note is that, our objective function is not differentiable because of the loss function l( x n , y n , Θ).

we use sub-gradients for l( x n , y n , Θ) instead, which can be computed by the formula: where ŷ is the predicted label sequence which corresponds to l( x n , y n , Θ).


A likely reason for this observation is that the neural model takes embedding inputs, which allow semantically similar words to be represented with similar vectors.

the neural model can better capture patterns that do not occur in the training data.


First, a passive solution is detecting whether an opinion includes a CFO and, if any, isolating that opinion from the target of opinion mining.

we can avoid potential errors as much as possible but the coverage is decreased.


For example, in "my daughter was pleased with toys in the room" the positive evaluation is restricted by the daughter's perspective.

the value of feature F6 takes 1 for the first phrase in a sentence excluding a conjunction, and 0 otherwise.


We combined the extracted words from the advertising slogans an annotated corpora, discarded redundancy, and standardized similar words, such as "kanko suru (do sightseeing) and "kanko (sightseeing)".


We removed sentences consisting only of opinion unit such as "The location is good" from the evaluation.

3 155 sentences remained, which comprise our corpus.


we conclude that each of our thirteen features was independently effective for extracting CFO and U-CFO in review sentences and that when used together the improvement was even greater.


The semantic concepts of entailment and contradiction are central to all aspects of natural language meaning (Katz, 1972;van Benthem, 2008), from the lexicon to the content of entire texts.

natural language inference (NLI) -characterizing and using these relations in computational systems (Fyodorov et al., 2000;Condoravdi et al., 2003;Bos and Markert, 2005;Dagan et al., 2006;MacCartney and Manning, 2009) -is essential in tasks ranging from information retrieval to semantic parsing to commonsense reasoning.


Table 8 shows the abstract questions we could use to query roles R0 and R1; and the generated questions, based on the set of predicted labels {R0:someone, R1:something}.

to generate a question to query a role r ∈ R, we simply return the most frequent abstract question that occurred in training data that matches the role being queried, and the set of other predicted labels.


In our experiments, a dataset that included 2,468 target entities and 350,498 microblogs generated a directed graph with billions of edges, which required more computer memory than was available even when using a sparse matrix.


When multiple target entities occur in one document, all of them are more likely to refer to the entities in the target domain.

the trust score for the document is higher, as indicated by d 4 (Figure 1).


One can normalize the final scores by dividing them by the largest score, but the relative ordering of the documents will not change.

one document that is more likely to contain any 1 We select the attenuation coefficient α = 0.85 and the number of iterations to be 20, which have been regarded as the standards in the PageRank literature (Page et al., 1999;Krishnan and Raj, 2006).


In this section, we construct a layered structure in the graph, where each layer denotes a document trust level that contains any true mention.

we define that the propagation direction from the high trust level to the low trust level.


Each stock has a unique ID, which has little ambiguity.

we used this regular expression pattern to extract seeds.


For example, the tweet "Cadillac Uber airport is classy" obtains a low score of 0.013 with 50% of the seeds, whereas it is identified successfully with the score of 0.588 when all of the seeds are used for propagation because more context features are introduced, such as "airport" and "Uber".

the proposed method achieves better performance as the seed set grows.


See examples of the pseudo mentions in Table 4.


A popular slice is defined as the slice of greater concern, which is normally frequently mentioned in the reference source, such as the slice of "the career of George W. Bush as the President" (high weight) versus "his childhood" (low weight).

the RM based biography-document relevance is only helpful to identify and recall the documents relevant to the popular slices but not to the unpopular ones.


Most are excluded since KBP annotators ignore relevant documents in which there isn't any filler for the assigned slots or, although exists, the fillers were inexactly identified by Slot Filling systems.

we manually went over the pool and extracted 4,405 relevant documents as our ground-truth.


Comprehensive and unbiased measurement of every known life slices is helpful in disambiguating entities that have similar backgrounds (definitely not the same in all).


Most target entities now in use for the evaluation are made to stand as "out of the ordinary", such as well-known enterprises, celebrities or domain experts.

a corpus contains abundant relevant documents of the entities but less about the little-known namesakes.


However, it is still rigid in two respects: (1) ER-MLN primarily relies on the predicates (also referred to as links or edges) for inference.

even if the words in the antecedent and setup have high entailment scores, the rule will still not "fire" if the edges do not match.


In other words, the more a G i c is structurally similar to its G m , the better the candidates in this G i c represent their mentions in G m .

we define the context similarity S m (m c , c) between a candidate c and its mention m c using Jaccard similarity in Equation 4: where Θ Gm (m c ) and Θ G i c (c) denote m c 's neighbor set in G m and c's neighbor set in G i c , respectively.


In other words, for each mention, the candidate entity types are not as diverse as those in the general domain.

the collective validation step in QCV does not take much effect since the weights of the involved relations are quite close to one another.


It is also true because most NLU applications combine a variety of information sources that include both text sources and knowledge bases; e.g., question answering systems need access to knowledge bases like gazetteers.

high-quality knowledge bases are critical for successful NLU.


Their method is based on KB information, more specifically entity descriptions in Wikipedia and Freebase.

in contrast to our approach, their system is not able to type entities that are not covered by existing KBs.


The main weakness of CM is that a large proportion of contexts does not contain sufficient information to infer all types of the entity; e.g., based on our distant supervised training data, we label every context of "Obama" with "author", "politician" and Obama's other types in the KB.

cM is trained on a noisy training set that contains only a relatively small number of informative contexts.


The top ranked type returned for 73.5% of entities was correct.

due to our ability to deal with NSE, we can type an additional 50,000 entities correctly.


On the other hand, the context words they cooccur with in a bag-of-words (BOW) sense are very similar.

this indicates that CM pays too much attention to BOW information and that its representation of contexts is limited in terms of generalization.


synsets, in S: this is the case, in the above example, for the ambiguous mention washington that has to be linked to either the president, the city or the state.

our approach comprises two successive stages: • A disambiguation stage (Section 5) where all KB i ∈ K are linked to S, either by inter-resource mapping (Figure 1(a)) or disambiguation ( Figure 1(b)), and all E i are merged into a unified set of entities E * .


Therefore, our approach comprises two successive stages: • A disambiguation stage (Section 5) where all KB i ∈ K are linked to S, either by inter-resource mapping (Figure 1(a)) or disambiguation ( Figure 1(b)), and all E i are merged into a unified set of entities E * .

of this process we obtain a set K S comprising all the KBs in K redefined using the common sense inventory S; • An alignment stage (Section 6, Figure 1(c)) where, for each pair of KBs KB S i , KB S j ∈ K S , we compare any relation pair r i , r j , r i ∈ R S i and r j ∈ R S j , in order to identify cross-resource alignments and merge relations sharing equivalent semantics into relation clusters (relation synsets).


Instead, in the unified KB * , relations with equivalent semantics should be considered as part of a single relation synset even when they come from different KBs.

at this stage, we apply an alignment algorithm to identify pairs of relations from different KBs having equivalent semantics.


Relations for which no alignment is found are turned into singleton relation synsets.

of this alignment procedure we obtain the unified set of relations R * .


Coverage displays the opposite behavior, decreasing exponentially with more confident outcomes, from 6 million triples to less than a thousand (for seeds with confidence δ dis > 0.95).

we chose δ dis = 0.25 as optimal threshold value throughout the rest of the evaluations.


Litanfeeth HYP: what can we get it leaks on feet While these errors are rare, they create major problems for language processing, since names tend to be important for many applications.

it is of interest to automatically detect such error regions for additional analysis or human correction.


As illustrated in the previous section, ASR hypotheses necessarily have errors in OOV regions, but because specific names are infrequent, even in-vocabulary names can have these types of error patterns.

developing a robust name error detector for ASR hypotheses is an important component of the system to resolve errors and ambiguity.


Although the Reddit discussions are different from BOLT data, they have a conversational nature and names are often observed within the discussions.

we hypothesize that they can help improve sentencelevel name prediction in general.


Stages 2-4 potentially introduce edges connecting names of different characters.

in the fifth stage, three heuristics are applied to add prohibitions on merging two names into the same character.


However, despite the lack of proper names, such characters behave and interact in similar ways as major characters, including having dialogue.

to discover such characters, we adopt a bootstrapping technique aimed at uncovering prototypical character behaviors from the novels themselves, inspired by the semantic predicate work of Flati and Navigli (2013).


These nodes are typically ambiguous first or last names.

the remaining set of nodes are merged to create sets of names, each associated with a different character.


The task of character recognition has largely been subsumed into the task of extracting the social network of novels.

three state-of-the-art systems for social network extraction were selected: the method described in Elson et al.


Finally, returning to the initially-posed question of how many characters are present, we find that despite the detection error in our method, the overall predicted number of characters is quite close to the actual: for Sherlock Holmes stories, the number of characters was estimated within 2.4 on average, for Pride and Prejudice our method predicted 72 compared with 73 actual characters, and for The Moonstone our method predicted 87 compared with 78.

we argue that our procedure can provide a reasonable estimate for the total number of characters.


In the SVM dual formulation that relies on kernels, i. e. similarity measures between documents, a linear kernel can be interpreted as the number of exact matching n-grams between two documents.

for short documents or when little training data is available, sparsity issues due to word synonymy arise, e. g., the sentences 'John likes hot beverages' and 'John loves warm drinks' have little overlap and therefore low linear kernel value (only 1) in the n-gram feature space, even with dependency tree representations and downward paths for n-grams as illustrated in Figure 1.


For example, instruction #8 mentions cooking spray, which is not on the ingredient list.

inst8 looks like it has no ingredients coming into it other than the output of inst7.


Importantly, our model is encouraged but not forced to find evidences of seed topics.

it has freedom to discover new topics beyond prior topics, as well as to detect which prior topics are not covered by current data.


As shown in the process, the prior probability of assigning a document to a topic is proportional to the number of documents already assigned to the topic.

the DPMM exhibits the "rich get richer" property.


As a result, GRNN is a more powerful model which outperforms the plain GRNN.

we mainly focus on the GRNN model in this paper.


Moreover, there is no clear guideline to distinguish strong and weak themes.

we here propose a method that extracts strong themes more effectively.


In the EM-based theme extraction scheme, the log odds score pz C d indicates the extent to which a document d is coupled with a specific theme C. If a cluster in-cludes a reasonable number of documents that have high pz C d s, it indicates that the cluster represents a strong theme.

we can obtain strong themes by collecting these clusters.


To simplify the notation, we define y ijk = (T j , π ijk , e ijk , s ijk ) the k-th derivation give x i and T j , and let Y i denote the set of all possible y ijk given x i and T .

to correctly solve x i is to find the correct y ijk ∈ Y i .


This can be simplified by computing arg max As we only consider assigning the number slots of the templates in T , generally, the size of the possible assignments per word problem is bearable, as shown in the Table 2.

we simply evaluate all the y ijk ∈ Y i .


Kushman's algorithm (2014) needs much more memory than our algorithm and can not run on a general laptop.

we tested their algorithm on a workstation with Intel E5-2620 CPU and 128G memory.


In this paper, we focus on Roman Urdu, an under-resourced language, that does not have any gold standard corpus with standard word forms.

we are restricted to the task of finding lexical variations in informal text.


Similarity of two nodes is measured using the cosine similarity of their corresponding sequences, a• b a b , which we compute using a function sim(•, •).

document groups are considered more similar when they have similar sets of topics in similar proportions.


Neither CR nor CCR links mention groups to corresponding KB entities.

they represent both in-KB entities and out-of-KB entities (e.g., long-tail or emerging entities that do not have a Wikipedia article) in the same way.


This is based on the observation that by increasing the value of the mention penalty, we are essentially forcing our model to predict more mentions.

the recall is a monotonic function with respect to the mention penalty.


For example, in the sentences "Klitschko is known for his powerful punches" and "Klitschko is the Mayor of Kiew," "Klitschko" will be associated with all its types, e.g., boxer, politician and mayor.

the labels in the training data can be misleading and negatively affect the extractors.


For example, "Messi" and "Cristiano Ronaldo" may both be mentioned in the context of soccer.

entity mentions similar to "Messi" in a sport context are likely to include soccer players.


We introduce a method for automatically pruning out hidden layer units, by adding a sparsity-inducing regularizer that encourages units to deactivate if not needed, so that they can be removed from the network.

after training with more units than necessary, a network is produced that has hidden layers correctly sized, saving both time and memory when actually putting the network to use.


So the unit can be removed without affecting the network at all.

we need a regularizer that pushes all the incoming connection weights to a unit together towards zero.


The first term becomes and the second term becomes The ∇L(u)(w − v) terms cancel out, and we can ignore terms not involving w, giving which is minimized by the update (4).

we have split the optimization step into two easier steps: first, do the update for L (3), then do the update for λR (4).


Since the 2,1 norm on matrices (1) is separable into the 2 norm of each row, we can treat each row separately.

for simplicity, assume that we have a single row and want to minimize The minimum is either at w = 0 (the tip of the cone) or where the partial derivatives are zero (Figure 3): Clearly, w and v must have the same direction and differ only in magnitude, that is, w = α v v .


Thus, for simplicity, assume that we have a single row and want to minimize The minimum is either at w = 0 (the tip of the cone) or where the partial derivatives are zero (Figure 3): Clearly, w and v must have the same direction and differ only in magnitude, that is, w = α v v .

substituting this into the above equation, we get the solution the update is As above, since the ∞,1 norm on matrices (2) is separable into the ∞ norm of each row, we can treat each row separately; thus, we want to minimize Figure 3: Examples of the two possible cases for the 2 gradient update.


However, at the end of training, all units are reactivated, as the goal of dropout is to reduce overfitting, not to reduce network size.

dropout and our method seem to be complementary.


The naive solution to this problem would be to find the highest-weighted path in the intersection Unfortunately, the intersection of WFSAs takes the Cartesian product of their state sets.

materializing this intersection would have taken time exponential in K. To put this another way, inference is NP-hard even on a "trivial" factor graph: a single variable X 1 attached to K factors.


In other words, a turn can't be anaphorically and rhetorically dependent on a turn that comes after it.

the nature of dialogue imposes an essential and important constraint on the attachment process that is not present for monologue or single-authored text, where an EDU may be dependent upon any EDU, later in the ordering or not: in dialogue there are no "backwards" rhetorical links such that an EDU in turn n by speaker a is rhetorically and anaphorically dependent upon an EDU in turn n + m of speaker b with a = b.


Since these relations can hold not only between adjacent but between arbitrary segments of the text, all possible combinations of segments are required to be tested.


In general, the classifier lacks precision compared to recall: It predicts too many edges.

the graph constructed from the predicted edges for one text very often does not form a tree.


Lexical and phrasal alignments can both be represented as pairs of words -in the form of manyto-many mappings among the two phrases' component words in the latter case.

without loss of generality, we formulate alignment as a binary classification task where given all word pairs across two sentences, the goal is to assign each a class label in {aligned, not aligned}.


Our observation of the aligner's behavior on the dev set of the MSR alignment corpus (Brockett, 2007) suggests that this happens primarily due to highly similar word pairs being left unaligned even in the absence of competing pairs because of relatively low contextual evidence.

aligner performance suffers in sentences with few common or similar words.


Specifically, we model the problem as that of assigning each span s ∈ S, from an over-generated set of candidate argument spans S, to a semantic role r ∈ R. The set of semantic roles R includes the special null role ∅, which is used to represent non-overt arguments.

our algorithm performs the SRL task in one step for a single predicate frame.


Local Estimation In local estimation, we treat each span-role assignment pair (s, r) ∈ S×R as an individual binary decision problem and maximize the corresponding log-likelihood of the training set.

5 Denote by z s,r ∈ {0, 1} the decision variable, such that z s,r = 1 iff span s is assigned role r. By passing the potential g NN (s, r; θ) through the logistic function, we obtain the log-likelihood the gold role for a given span according to the training data serves as the positive example, while all the other potential roles serve as negatives.


Scalability is often a concern when employing collective reasoning over large corpora, but our system can produce graphs with over 100K edges on conventional hardware.

we produce a large, complete, and high-precision hypernym graph that includes alignments and type information.


Rules 5 and 6 create a transitive closure of both WordNet and YAGO hierarchies.

of these rules, we can use indirect hyponyms (in rule 3) or indirect subtypes (in rule 4).


To keep this efficient, we use a closed set of possible verbs and assume a closed set of words (e.g., prepositions, adverbs) can only follow the start token in the argument bigram model.

annotating the verbs in a sentence determines a unique set of argument strings.


In addition, for the literal sense we also consider a special case, where the tweets are labeled with either positive or negative sentiment hashtags (e.g., #happy, #sad).

we consider two LSSD tasks: S vs. L and S vs. L sent , and aim to collect a balanced dataset for each target word.


As stated earlier, MVME algorithm aligns similar word pairs found in its inputs and this performs well for short texts (i.e., tweets).

the MVME algorithm combined with word embedding in kernel we results in very high F1.


Regarding the size of the training data, recall that the unsupervised alignment approach had extracted 70 target words (Section 2), although we have used 37 target words as we did not have enough training data for the remaining targets.

we plan to collect more training data for these targets as well as more target words (especially for the S vs. L sent task).


it is difficult to judge whether a .net site is more trustful than .org or not.

we let all open TLD domains have the same score.


For example, in the clue "The person who played at Eden Gardens played for India", the phrases "played at" and "played for" are used to express two different relations.

using a model that assigns meaning to each word may not be suitable here.


periments) for each source sentence in the training data.

the seg rev process is lightweight and shallow.


In contrast, fast align, although very fast, took approximately one hour for one round of training (using five iterations for its log-linear model) on the same corpus.

the additional time required in our approach is quite small and can be ignored compared with the training time of fast align.


The underlying set of events is thus much larger than for word-based models, while the training data (parallel corpora) are typically order of magnitude smaller than monolingual resources.

data sparsity issues for such models are particularly severe.


However, features that can be learned on a single document are limited in number and can be discarded after the document has been processed.

document-level sparse features are a powerful means to fit our model to local context with a comparatively small impact on efficiency.


However, QE is also concerned with how well the source segments can be translated.

we trained two models, one for each side of a given language pair.


However, the resulting triangulated phrase tablê T contains many spurious phrase pairs and noisy probability estimates.

early triangulation work (Wu and Wang, 2007) already realistically assumed access to a limited source-target parallel data from which a relatively high-quality source-target phrase table T can be directly estimated.


In this work, we use monolingual word embeddings for v s and v t , and set the vector f st to contain only the value of the triangulated score, such that f st :=ŵ st .

the matrix A is a linear transformation between the source and target embedding spaces, and h (now a scalar) quantifies how the triangulated scoresŵ are to be trusted.


For instance, when a rule is selected to translate sentence F given the partial translations in Figure 2, then the non-terminals in the target side of this rule must match the constituents selected so far.

rules r 1 and r 2 (Section 2.1) are not competing during rule selection.


Language use is known to be influenced by personality traits as well as by sociodemographic characteristics such as age or mother tongue.

it is possible to automatically identify these traits of the author from her texts.


MT models, even domain-specific, are user-generic.

the linguistic signals of user traits which are conveyed in the original language may not be preserved over translation.


Although personality projection in different languages is under-explored, it has been shown that the relationship between language use and personality traits varies between domains (Nowson and Gill, 2014).

while it would seem that there are cues which translate directly between languages, this may not always be the case.


However, state-of-the-art classification results are not our focus; rather, our intention is to understand the impact of translation on classification of socio-demographic and personality traits.

we fix our models with a set of parameters selected via cross-validation (CV) on the native language: the occurrence threshold is set to 5 and the SVD dimensionality to 500.


But as (Levy and Goldberg, 2014) showed that the Skip-gram loss function extracts interesting linguistic features, we wanted to use a loss function for the cross-lingual objective that will be closer to Skip-gram than Bil-BOWA.

we introduce a new task, Transgram, similar to Skip-gram.


Now the three polish words closest to v translate in English into "a train", "a train" and "railroad".

v is a better representation for the railroad sense of train fr .


The markedness measure can be analyzed in terms of point-wise mutual information (pmi), indicating the amount of information one relation has for the distribution of the words that follow it (Hume, 2011).

because pmi is proportionate to npmi, we can rewrite (2) as We also have the mutual information measure: For the mutual information of y i in Y :  the markedness measure can be understood as the Kullback-Leibler divergence of the univariate distribution of X from the conditional distribution of X given y i (a discourse relation in our case).


This method has similar performance with KAZB on their dataset, but does not have the two limitations mentioned above.


Table 5: Evaluation results The reason for such a high precision is that, by transforming NL text to DOL trees, the system "understands" the problem (or has structured and accurate information about quantity relations).

it is more likely to generate correct results than statistical methods who simply "guess" according to features.


Now our DOL language and CFG grammar already have a good coverage on common entity types, but the coverage on properties, relations, and actions is quite limited.

our parser fails to parse many sentences in general math word problems because they contain properties, relations or actions that are unknown to our system.


While we rely heavily on the relationships between words in-text and concept nodes expressed in parallel training data, we find this is not sufficient for complete coverage.

we also include a run-time module that generates AMRese base forms at the lexical level, expressing relationships such as those depicted in Table 4.


Unseen sentences are always parsed by an automatic parser, which is far from perfect and taskindependent.

a good solution is to give 3 An indirect interaction can be set up through pooling.


The fragment of semantics necessary to complete most instruction-following tasks is essentially predicate-argument structure, with limited influence from quantification and scoping.

the problem of sentence interpretation can reasonably be modeled as one of finding an alignment between language and the environment it describes.


As noted in the introduction, we approach instruction following as a sequence prediction problem.

we must place a distribution over sequences of actions conditioned on instructions.


If the first instruction go down the yellow hall were interpreted immediately, we would have a presupposition failure-the agent is facing a wall, and cannot move forward at all.

an implicit rotate action, unspecified by text, must be performed before any explicit instructions can be followed.


While in principle the normalizing constant can be computed using the forward algorithm, in practice the state spaces under consideration are so large that even this is intractable.

we make an additional approximation, constructing a setỸ of alternative actions and taking Ỹ is constructed by sampling alternative actions from the environment model.


To implement this, we view the two induced segmentations that constitute an alignment -e.g., (ph,oe,n,i,x) and (f,i,n,I,ks) -as strings including splitting signs.

we can compute the edit distance between the gold-standard segmented x string and the predicted segmentation, and analogously for the y sequence.


The training time of sequencebased methods tends to be long, especially when we use partially annotated data.

a simple method based on pointwise classification has been shown to be as accurate as sequence-based methods and fast enough to make active learning practically possible .


Their parameters are trained from manually segmented sentences whose words are annotated with phoneme sequences, and from sentences automatically annotated with NLP tools which are also based on machine learning models trained on the annotated sentences.

normal IM servers are not capable of presenting out-of-vocabulary (OOV) words (which provide large amounts of information on word boundaries) as conversion candidates.


For the LM unit, instead of words we propose to adopt word-pronunciation pairs u = y, w .

given a phoneme sequence y l 1 = y 1 y 2 • • • y l as the input, the goal of our IM engine is to output a word sequenceŵ m 1 that maximizes the probability P (w, y l 1 ) as follows: where the concatenation of y i in each u i is equal to the input: In addition u j (j ≤ 0) are special symbols introduced to simplify the notation and u m+1 is a special symbol indicating a sentence boundary.


The pSTC, however, contains two possible pronunciations for this word and allows our engine to present the OOV word " " for the input "yo-ko-a-ri."

when the user of our IM engine types "yoko-a-ri-ni-i-ku" and selects " (to) (go)," the engine can learn an OOV word " /yo-ko-a-ri" with context " /ni /i-ku".


IM users send the phoneme sequences for sentence fragments to the engine to avoid editing long conversion results that require many cursor movements.

the phoneme sequence and the final word sequence tend to be sentence fragments (as we noted above) and as a result they lose context information.


冬 天 (winter)， 能 (can) 穿 (wear) 多 少 (amount) 穿 (wear) 多 少 (amount)； 夏 天 (summer)，能 (can) 穿 (wear) 多 (more) 少 (little) 穿 (wear) 多 (more) 少 (little)。 Without the word "夏天 (summer)" or "冬天 (winter)", it is difficult to segment the phrase "能 穿多少穿多少".

we usually need utilize the non-local information for more accurate word segmentation.


The reason is that the number of its parameters is so large that the trained network has overfitted on training data.

it is necessary to capture the potential long-distance dependencies without increasing the size of the context window.


The LSTM provides a solution by incorporating memory units that allow the network to learn when to forget previous information and when to update the memory cells given new information.

it is a natural choice to apply LSTM neural network to word segmentation task since the LSTM neural network can learn from data with long range temporal dependencies (memory) due to the considerable time lag between the inputs and their corresponding outputs.


However, the primary shortcoming of these approaches is that they rely heavily on a large amount of labeled data, which is very timeconsuming and expensive to produce.

the scale of available manually labeled data has placed considerable limitations on the further enhancement of supervised CWS methods.


Similar to One-to-Many alignment, for any sequence of English words e i 2 i 1 and the Chinese word w n = c jn jm , the phrase tuple < e i 2 i 1 , c jn j 1 > is defined as an aligned Many-to-Many phrase tuple if it satisfies the following conditions: (1) j 1 ≤ j m , and j 1 is the beginning character of a word in w n Then, for any phrase tuple < e i 2 i 1 , c jn jm > that satisfies these conditions, the span < i 1 , i 2 , j 1 , j n > is defined as a Many-to-Many span and as a member of the set A M any .

for each span < i 1 , i 2 , j 1 , j n >, the Many-to-Many score can be calculated as follows: where t(c jn j 1 |e i 2 i 1 ) represents the translation probability of the phrase tuple < e i 2 i 1 , c jn j 1 >.


Given a Chinese word w n with an index i in the vocabulary, it is then straightforward to retrieve the word's vector representation via simple multiplication with a binary vector d that is equal to zero at all positions except that with index i: Because the word embeddings for the two languages (L ch and L en ) are learned separately and located in different vector spaces, we suppose that a transformation exists between these two semantic embedding spaces.

we collect all the One-to-Many phrase tuples < e 1 , c j 2 j 1 > that satisfy e 1 ∈ V en and c j 2 j 1 ∈ V ch from the entire corpus of bilingual data.


Using θ * , we can calculate the semantic gap for any possible span for w n , such as < i, j m , j n >, as follows: where p s and p t are the word vector representation of e i and c jn jm , respectively.

the semantic gap feature score of the word w n = c jn jm is derived as follows: Because the output scores of each sub-model described above are not probabilistic and they vary by orders of magnitude, we must first normalize the output scores of each sub-model.


We represent the normalization function by N orm(•).

for the substring c j 1 (j ∈ [1, J)) in c J 1 of the sentence tuple <e I 1 , c J 1 , a K 1 >, assuming that one of its candidate segmentations is , the feature score of the inner log-linear model is derived as follows: where f k (n) represents the phrase-level features.


Due to the O(|f | 3 |e| 3 ) time complexity for bi-parsing a bilingual sentence, previous studies relied on biparsing at the initialization step, and conducted Gibbs sampling by local operators (Blunsom et al., 2009;Levenberg et al., 2012) or sampling on fixed word alignments (Chung et al., 2014;Peng and Gildea, 2014).

the inference can easily result in local optimum, wherein induced derivation trees may strongly depend on the initial trees.


After learning a Bayesian model, it is not directly used in a decoder since it is composed of only minimum rules without considering phrases of various granularities.

it is a standard practice to obtain word alignment from derivation trees and to extract SCFG rules heuristically from the word-aligned data (Cohn and Haffari, 2013).


The major differences were the use of the minimal phrase pairs used in the previous work in which only minimal phrase pairs in the leaves of derivation trees were included in the model.

larger phrase pairs were forced to be constructed from those minimal rules.


Model 3 and our Back-off model dose not use any word class informations.

back+future infers better models by avoiding over pruning spans.


On one hand, aligners seek to find the alignment with the highest alignment model score, without regard to structural constraints.

sensible translation rules may not be extracted because they violate consistency constraints required by translation rule extraction (Och and Ney, 2004).


(2010) find that the standard alignment tools are not optimal for training syntax-based models.


Therefore, the objective of our search algorithm is defined as score(f , e, a, w, θ) = M (f , e, a, θ) + λC(f , e, a, w) where M (f , e, a, θ) is alignment model score, θ is a set of model parameters, C(f , e, a, w) is coverage (either hard or soft), and λ is a hyperparameter that controls the preference between alignment model score and coverage.

2 the decision rule is given bŷ a = argmax a∈A(f ,e) score(f , e, a, w, θ) where A(f , e) is a set of all possible alignments for the sentence pair.


We find w = 7 achieves the best result, which is consistent with the default setting in Moses.

we used C s+t and set w = 7 in the following experiments.


We find that our approach outperforms the baseline statistically significantly at p < 0.01 for four language pairs and p < 0.05 for one language pair.

using coverage to bridge word alignment and machine translation can hopefully benefit more languages.


The major difference is that our coverage regularizer is independent of model parameters.

alignment models can still be trained independently.


As stated previously, the translation of a sourceside content word s should be: 1) associated with s; 2) related to the translations of other source-side content words.

in the translation graph, the translation of s should be a target-side node which has: 1) an association edge with the node of s; 2) many relatedness edges with other target-side nodes that represent translations of other source words.


However, translation units in machine translation have long since shifted from words to phrases (sequence of words), of which syntactic and semantic information cannot be adequately captured and represented by word embeddings.

learning compact vector representations for phrases or even longer expressions is more crucial for successful "deep" SMT.


Similar to the conventional recursive antoencoder (RAE), BRAE neglects bilingual correspondences of sub-phrases.

it may combine "adopted" and "today" together to generate an undesirable target tree structure which violates word alignments.


Since phrase embeddings for the source and target language are learned separately in different vector spaces, a transformation matrix W (3) f ∈ R d×d is introduced to capture this semantic transformation in the source-to-target direction.


English-to-German: Concatenating all training data to train the MT pipeline has been shown to give the best results for English-to-German (Birch et al., 2014).

we did not do domainwise experiments, except for training a system on the in-domain IWSLT data for the sake of completeness.


A quick inspection of both datasets reveals that Chinese sentences without a comma were never translated into multiple sentences by more than one translator.

in our experiments we consider only sentences that contain at least one comma.


However, as Liu and Huang (2014) discussed, partial BLEU only evaluates the partial derivation itself without considering any of its context information, leading to a performance degradation.

in order to take the context information into account, we investigate a more reasonable metric, potential BLEU, in the next section.


We then sort the partial derivations in s [i,j] based on their potential Bleu +1 (line 12).

the sorting will guide us to first backtrace the partial derivations with better potential Bleu +1 scores.


Beam Size As we discussed before, searchaware tuning helps to accommodate search errors in decoding, and promotes good partial derivations.

we believe that even with a small beam, these good partial derivations can still survive with search-aware tuning, resulting in a good translation quality.


Similar to (Liu and Huang, 2014), there is also an monster phenomenon (Nakov et al., 2013) in our search-aware tuning setting.

here we perform search-aware tuning on only 109 short sentences (with less than 10 words) from nist02.


The key insight of this algorithm is the inner-to-outer dynamic-programming structure: inner modifiers (r) of a head (s) and their "best" labels (l * (s,r) ) are generated before outer ones (t).

using already computed "best" labels of inner dependency edges makes us get rid of maximizing over two labels, l and l. Moreover, we do not have to extend each incomplete span by the augmentation with a "label" index.


For the six u:big conditions, this is not surprising: the Gigaword corpus consists of news, so the large unlabeled data set is in reality the same domain as WSJ.

if large unlabeled data sets are available that are similar to the TD, then one might as well use STATIC tagging since the extra work required for ONLINE/BATCH is unlikely to pay off.


There is no large difference in variability ON-LINE vs. BATCH (see columns "std").

given that it has equal variability and higher performance, ONLINE is preferable to BATCH since it assumes no dataset available prior to the start of tagging.