Never-Ending Learning and Bootstrapping Our latent feature models are capable of never-ending learning (Carlson et al., 2010).

we can continue to train these models with incoming data, even if no structured annotation is available.


Given this ranking task, our evaluation is inspired by the TREC competitions and work in information retrieval (Manning et al., 2008).

we treat each relation as query and receive the top 1000 (run depth) entity pairs from each system.


This gives a set of relevant results that we can use to calculate recall and precision measures.

we can use these annotations to measure an average precision across the precision-recall curve, and an aggregate mean average precision (MAP) across all relations.


This is a simplification made for ease of computation, and subsequent work has found DLT to be more accurate cross-linguistically if the intervening elements are structurally defined rather than defined in terms of referents (Kwon et al., 2010).

simply having a particular referent intervene in a dependency projection may not have as great an effect on processing complexity as the syntactic construction the referent appears in.


However, it is certainly imaginable that f s could consist of specially tailored features that make generalization easier.

using a generalized f s would allow us to use this approach in more complex cases than considered here, for instance if the dependencies would be labeled with two different sets of grammatical function labels, or if one of the treebanks would use constituents rather than dependencies.


Here, we see that all three adaptation methods give statistically significant 4 improvements over the baseline when parsing the Tiger treebank.

the combined method gives a strong 0.7-point improvement, a 6% error reduction.


• The largest improvements (absolute and relative) all happen in treebanks that are harder to parse.

syntag and WsJ Part 2 are harder to parse due to their representation, and to some extent this may be true for Tiger as well -its learning curve rises more slowly than for TüBa-D/Z.


Again, we see that guided parsing is less effective if the guide uses an annotation style that is hard to parse.

for Swedish the Syntag parser never gives a very large improvement when guiding the Talbanken parser, and this is also true of both Italian parsers.


We evaluated the methods on eight treebanks in four languages, and had statistically significant improvements in all eight cases.

for English we saw a strong 1.2-point absolute improvement (an 11% relative error reduction) in the performance of a semantically oriented parser when trained on the full training set.


The combination of the two methods was also effective, in most cases outperforming both methods on their own.

when using the full training sets, this was the only method that had statistically significant improvements for all treebanks.


This paper describes an approach to learning a POS-tagger that can be applied in a truly lowresource scenario.

we discuss techniques that allow us to learn a tagger given only the amount of labeled data that a human annotator could provide in two hours.


Since we require a distribution for every token, we use a default distribution for such cases.

we use the unsupervised emission probability initialization of Garrette and Baldridge (2012), which captures both the estimated frequency of a tag and its openness using only a small tag dictionary and unlabeled text.


Formally, φ t (the word distribution for tuple t) has a Dirichlet(ω ( t) ) prior, where for each word w in the vector,ω w is a log-linear function: (1) where ω (B) is a corpus-wide precision scalar (the bias), ω w is a corpus-specific bias for word w, and ω (k) t k w is a bias parameter for word w for component t k of the kth factor.

each drug, route, and aspect has a weight vector over the vocabulary, and the prior for a particular triple is influenced by the weight vectors of each of the three factors.


We then use the learned parameters as priors over the corresponding f-LDA parameters.

we will place priors on the ω vectors, the Dirichlet hyperparameters which influence the word distributions.


However, such datasets cannot directly support our application.

our entailment graph is built over the extracted phrases (with max.


Parameter Settings: In order to make all the methods comparable, we chose for each method the parameter settings that give the gold-standard number of segments 5 .

we used a 11 × 11 rank mask for C99, as suggested by Choi (2000), the configurations included in the code (http://groups.csail.mit.edu/rbg/code) for Bayesseg and manually tuned parameters for MinCut.


These probabilities were then thresholded to give the gold-standard number of segments.

we apply a small amount of Gaussian smoothing to the marginal probabilities (except for Choi's dataset), like Puerver et al.


Although there are some exceptions, adding context information from the input features improves the performance of the models.

the MaxEnt models benefit from incorporating this extra information.


The SVM-based punctuation classifier relies on a five word and POS window in order to classify the target word.

token t 0 is classified given as input the window t −2 t −1 t o t 1 t 2 .


Recall that the parser we are using is based on a second order model.

the score of a dependency depends on some neighboring dependencies.


This might be due to the large intersection between our resources.

they do not have complementary information, and their combination will not  Table 10: LAS and UAS on TEST using DP with resource combination introduce much information.


Assume that we want to take the step ∆w i proportional to the average gradientĝ i calculated from n-best lists in which the feature is non-zero.


Therefore we should take the step ∆w i = αg i /n i .

we rescale each component g i of the gradient by 1/n i before taking the gradient step.


We leave to future work to experiment with feature sets designed for the large-scale discriminative setting.

we hope this framework will facilitate incorporation of richer linguistic knowledge into machine translation.


4 Applications We first investigate CTI in a well defined setting.

if we have a collection of terms such that its most important context is a "definition," e.g.


For evaluation purposes, each induced concept is mapped onto one human concept applying the following procedure: let HC i be the set of summary offsets where human concept i occurs, and let IC i be the set of summary offsets where automatic concept i occurs, then the induced concept j is mapped onto concept k such that: where |X| is the size of set X.

the induced concept is mapped onto the label it gives it a best match.


At each step, the algorithm finds the pair of labels that is currently the closest together according to the distance metrics of Equations (1) and (2), combines those two labels into a new one, and updates the set of P (s | t) and P (t | s) values appropriately.

the choice of label pair to collapse in each iteration can be expressed formally as arg min either a source label pair or a target label pair may be chosen by the algorithm in each iteration.


Our aim is to go beyond this limitation of the TSP model and use a richer set of features instead of using pairwise features only.

we are interested in features that allow us to examine triples of words/POS tags in the candidate reordering per-mutation (this is akin to going from bigram to trigram language models), and also structural features that allow us to examine the properties of the segmentation induced by the candidate permutation.


Although these efficiency tricks are important in the serial algorithms that serve as our baseline, not all of them are applicable to parallel architectures.

optimizations (1), (2), and (3) introduce order dependencies between queries; they are disregarded in our GPU implementation so that we can fully exploit parallelization opportunities.


Our experimental results demonstrate that at various pruning levels, our approach improves performance consistently.

at the level of 50% of rules being pruned, the discriminatively trained models performs better than the unpruned baseline grammar.


First, let the k-best hypotheses of the n-th sentence,Ê n = Ê 1 n , ...,Ê K n , approximate all its possible translation.

we assume that K k=1P (Ê k n |F n ) = 1, ∀n.


Our discriminative training procedure includes updating both λ and θ, and we follow (He and Deng, 2012) to optimize them in an alternate manner.

when we optimize θ via EBW, we keep λ fixed and when we optimize λ, we keep λ fixed.


We then explore the optimal setting for τ which controls the contribution of the regularization term.

we perform grid search, exploring values of τ from 0.1 to 0.75.


Table 2 compares the statistics of the pruned grammars and the unpruned one.

columns 4 and 5 show the total averaged probability mass of the remaining rules.


For all pruning levels, DT can compensate the loss associated with pruning.

at 50% level of pruning, there is a loss about 0.


(2010), who use it as a feature to predict how well a parser will perform when applied across domains.

2 the datasets A and B are compared by counting the number of bigram tokens in A that are unseen in B.


Thus each minibatch MIRA update involves an optimization over many more constraints than in pure online MIRA, which could potentially lead to a better margin.

we can view MIRA as an online version or stepwise approximation of SVM, and minibatch MIRA can be seen as a better approximation as well as a middleground between pure MIRA and SVM.


First, except for the largest minibatch size of 48, minibatch learning generally improves the accuracy  of the converged model, which is explained by our intuition that optimization with a larger constraint set could improve the margin.

m = 16 achieves the highest accuracy of 92.53, which is a 0.27 improvement over the baseline.


For example, minibatch of size 4 can reach 92.35 in 3.5 hours, and minibatch of size 24 in 3.7 hours, while the pure online baseline needs 6.9 hours.

just minibatch alone in serial mode can already speed up learning.


As in Figure 5, we observe similar convergence acceleration and higher accuracies with minibatch.

minibatch of size m = 16 provides the highest accuracy of 97.04, giving an improvement of 0.06.


However, besides these explicit social relations, the various kinds of interactions between online users often suggest other implicit relations.

in online discussion forums, users interact through textual posts and these exchanged texts often reveal whether two users are friends or foes, or whether two users share the same viewpoint towards a given issue.


PMF-UU: In this degenerate version of the model, we use only the user-user opinion matrix to learn the latent factor representation.

the objective function is modified such that we drop the sum of the square errors involving R and the regularizer on A. PMF-UA: In this degenerate version of the model, we use only the user-aspect opinion matrix to learn the latent factor representation.


Specifically, the objective function is modified such that we drop the sum of the square errors involving R and the regularizer on A. PMF-UA: In this degenerate version of the model, we use only the user-aspect opinion matrix to learn the latent factor representation.

the objective function is modified such that we drop the sum of the square errors involving S. In this section, we present our experiments that evaluate our model.


We observe that PMF-UOM achieves the best performance in terms of all the measures for almost all threads.

comparison with BL-1 and BL-2 shows that collaborative filtering can generalize the user preferences and help better group the users based on their viewpoints.


This leaves room for improvement in designing a system that can more easily adapt to previously unseen data.

the increasingly popular feature set produced by feature and context aggregation provides many opportunities for different types of optimization given the strong correlation between the training input and the feature values that are produced.


We use ROUGE evaluation metrics (Lin 2004) just like (Sharifi et al., 2010a;Sharifi et al., 2010b;Sharifi et al., 2010c), who evaluated summaries obtained with the PR Algorithm.

we use ROUGE-L, which uses the longest common subsequence (LCS) to compare summaries.


In this paper we use Markov random fields (MRFs) to model the joint distribution ܲ ܟ (, )܍ over a source-target translation phrase pair (, ,)܍ parameterized by .ܟ Different from the directional translation models, as in Equation 1, the MRF model is undirected, which we believe upholds the spirit of the use of bi-directional translation probabilities under the log-linear framework.

the agreement or the compatibility of a phrase pair is more effective to score translation quality than a directional translation probability which is modeled based on an imagined generative story does.


A number of the hypothesis producers detect and deal with informal words w present in a hypothesis by relying on bigram counts of w in a large corpus of formal texts.


In this work we distinguish between two kinds of deceptive opinion spam, depending on the sentiment expressed in the review.

reviews intended to promote or hype an offering, and which therefore express a positive sentiment towards the offering, are called positive deceptive opinion spam.


Nevertheless, it is important to subject our reviews to human judgments to validate their convincingness.

if human detection performance is found to be very high, then it would cast doubt on the usefulness of the Mechanical Turk approach for soliciting gold standard deceptive opinion spam.


(2011), we asked three volunteer undergraduate university students to read and make assessments on a subset of the negative review dataset described in Section 2.

we randomized all 40 deceptive and truthful reviews from each of four hotels (160 reviews total).


Second, the effect of deception on the pattern of pronoun frequency was not the same across positive and negative reviews.

while first person singular pronouns were produced more frequently in fake reviews than truthful, consistent with the case for positive reviews, the increase was diminished in the negative reviews examined here.


(2011)'s positive deceptive opinion spam dataset in conjunction with our own.

we have identified several features of language that seem to remain consistent across sentiment, such as decreased awareness of spatial details and exaggerated language.


In an attempt to attenuate these problems, we leveled the intensity of all recordings to a mean of 72dB using linear interpolation.

each sound was multiplied by a number such that its new average RMS intensity was 72dB; so that all sentences in the corpus ended up with the same average intensity.


(Shaw et al., 2011) propose a set of algorithms to create a reverse dictionary in the context of single language by using converse mapping.

given an English-English dictionary, they attempt to find the original words or terms given a synonymous word or phrase describing the meaning of a word.


appealable, accomodating, well-meaning, weakest, unsurpassed, unopposed, unobjectionable, unemployable, uncharacteristic or speechless).

these are less frequent words; many of them are actually subjective even though they are not listed in the sentiment lexicons.


Alternatively, we replace glue rules of HPB with reorderable ones: where the second (third) rule combines two translated phrases in a monotonic (inverted) way.

we set the translation probability of the first translation rule as 1 while estimating the probabilities of the other two rules from training data.


Similar to related work (Liu and Gildea, 2010;Xiong et al., 2012), we obtain the PAS of the source language (PAS-S) via a shallow semantic parser and project the PAS of the target language (PAS-T) using the word alignment derived from the translation process.

we use PropBank standard (Palmer et al., 2005;Xue, 2008) which defines a set of numbered core arguments (i.e., A0-A5) and adjunct-like arguments (e.g., AM-TMP for temporal, AM-MNR for manner).


The full derivation tree for a sentence is not used, but rather only that slice of it having e-trees with words that are in the nucleus being examined, which we call a derivation tree fragment.

for a given nucleus with a set of instances, we compare the derivation fragments for each instance.


The latter two also represent a measure for similarity of paired data.

all measures will be briefly described in the following: Unweighted average Recall The Unweighted average Recall (UaR) is defined as the sum of all class-wise recalls r c divided by the number of classes |C|: Recall r c for class c is defined as where δ is the Kronecker-delta, h i and r i represent the corresponding hypothesis-referencepair of rating i, and |R c | the total number of all ratings of class c. UaR for multi-class classification problems is the accuracy corrected by the effects of unbalanced data.


We note that CORPU SPART has a submodular problem structure, where greedy algorithms are often effective.

let |S| denote the vocabulary size of a set of documents S, and let S ⊆ S. Then for any document c the following inequality holds.


Specifically, let |S| denote the vocabulary size of a set of documents S, and let S ⊆ S. Then for any document c the following inequality holds.

adding a document c to the subset S increases vocabulary size at least as much as adding c to S; the vocabulary size function is submodular.


Given a selected document d, the MINIMUM method proposed in previous work selects node n t having the minimum number of word types after allocation of d to n t (Zhu and Ibarra, 1999).


However, our LVM training task differs significantly from those in which graph partitioning is typically employed.

graph partitioning tends to be used for scientific computing applications where communication is the bottleneck.


We use Conditional Random Fields (CRFs) for sequence labeling.

we use a first-order chain-structured CRF.


Those authors find that, in texts of the sort we are studying, attitude predicates are used predominantly to mark the source of information that is effectively asserted despite being embedded (Rooryck, 2001;Simons, 2007).

though X said p does not semantically entail p, it is often interpreted as a commitment to p, which correspondingly elevates mentions in p to main-clause status (Harris and Potts, 2009).


We discovered, however, that batch models were markedly inferior to more traditional models for our purposes because the influence of the initial model diminishes too quickly; here, we need particular topics in the model to correspond to particular styles, and we accomplish this by seeding the model with known instances of each style (see Section 3).

our initial β consists of distributions where the entire probability mass is divided amongst the seeds for each corresponding topic, and a full iteration over the corpus occurs before β is updated.


The model with independent training of each dimension (combo) did noticeably worse, supporting our conclusion that a multidimensional approach is warranted here.

striking is the much larger drop in overall accuracy as compared to pairwise accuracy, which suggests that the combo model is capturing the general trends but not distinguishing correlated styles as well.


Further discussion with the annotators found that some simply had a much lower threshold at which they would consider a sentence to contain an opinion.

this form of annotation is more affected by metasubjectivity than opinion annotation focused on opinion source perspective.


Then we may define the deletion rate at time T i as .

r T (t) is the fraction of tweets that are deleted during the one minute period (t, t + 1).


Compared to the FP detection component in the disfluency detection systems described in Section 2, this alternative approach explores different knowledge sources.

the AMs explore different front-end features compared to the lexical and prosodic features explored in those disfluency detection systems presented in Section 2.


Since WCPM depends on speaking rate as well as literacy, we are interested in identifying new measures that can be automatically computed for use in combination with WCPM to provide a better assessment of reading level.

we investigate finegrained measures that, if useful in identifying points of difficulty for readers, can lead to new approaches for assessing text difficulty.


Each participant read three word lists, three pseudo-word lists, one easy text passage, and one harder text passage, and the BRS is the average WCPM over the eight different readings.

the WCPM for each case is computed automatically using Ordinate's VersaReader system to transcribe the speech given the target text (Balogh et al., 2005).


A submodular function is defined as follows: Given a finite ground set of objects (samples) .

the incremental "value" of v decreases when the set in which v is considered grows from A to B.


We observe that the unigram morph distribution of the TOP-500K data set is the least uniform with KL distance of 6.6, whereas BOTTOM-500K has KL distance of 2.7 and the other two have KL distances of around 4.3.

this shows that TOP-500K has the lowest content variation, especially when compared to BOTTOM-500K.


In this work, we find that the learned word representations in fact capture meaningful syntactic and semantic regularities in a very simple way.

the regularities are observed as constant vector offsets between pairs of words sharing a particular relationship.


We represent all the tuple types 1 X and all the extraction pattern types Y contained in a given corpus through an undirected, weighted, bipartite graph G = (V, E) with vertices V = X ∪ Y and edges E ⊂ X × Y , where an edge (x, y) ∈ E indicates that tuple x occurrs with pattern y somewhere in the corpus.

edge weights are defined through a weight matrix W which holds the weight we use the count of how many times a tuple occurs with a pattern in the corpus and weights for unconnected vertices are zero.


Edge weights are constrained to be non-negative, which allows us to define the transition matrix P with The distance of two vertices is measured in terms of the average time of a random walk be-tween the two.

we adopt the notion of T-truncated hitting time (Sarkar and Moore, 2007) defined as the expected number of steps it takes until a random walk of at most T steps starting at v i reaches v j for the first time: The truncated hitting time h T (v j |v i ) can be approximately computed by sampling M independent random walks starting at v i of length T and computinĝ where {t 1 .


Gold standard seed and test tuples for a set of relations were obtained from YAGO (Suchanek et al., 2007).

we took all relations for which there are at least 300 tuples, each of which occurs at least once in the corpus.


The second one (IB2) corresponds to a standard bootstrapping algorithm which employs pruning after each step in order to reduce semantic drift.

scores are pruned after projecting from X onto Y and from Y onto X, retaining only the top N (t) = N 0 t scores at iteration t and setting all other scores to zero.


However, from a seed set size of 50 onwards, performance remains relatively constant.

iterative bootstrapping is not benefitting from the information provided by the additional labeled data, and thus has a poor learning performance.


Experiments show that terms with high RSI values are better candidates than those with high TF-IDF values when used by an HMMbased segmenter as state emissions.

terms with high RSI values are more likely to be structural cues in lectures independent of the lecture topic.


We believe this conclusion is applicable to numerous Twitter trend tasks, and we encourage others to investigate richer content analyses for these tasks.

the community interested in modeling author beliefs and influence (Diab et al., 2009;Prabhakaran et al., 2012b;Biran and Rambow, 2011) may find our task and data of interest.


DFA-EQUAL uses the process described in §3.1 to determine whether parse t evaluates to a regexp which is semantically equivalent to r, leading to the following objective: At testing time, for efficiency reasons, we calculate only the top parse.

if r = eval(t) is the regexp which results from evaluating parse t, then we generate t * = arg max t∈T ( w) p(t| w; θ, Λ), and return r * = eval(t * ).


We use the n-best parses to calculate an approximate version of the gradient.

t i is the set of n-best parses for training sample i, and C i includes all parses t in t i such that DFA-EQUAL(t, r i ).


We calculate the approximate gradient as: In contrast to our n-best technique, the past work has calculated equation (2) using a beam search approximation of the full inside-outside algorithm (Zettlemoyer and Collins, 2005;Kwiatkowski et al., 2010;Liang et al., 2011).

since the conditional probability of t given r does not factorize, a standard chart parser would need to maintain the full logical form (i.e.


It is also important to note that the runtime of the n-best algorithm scales much better.

as n increases, the n-best runtime increases as O(n| w| log(| w||P | + n), where P is the set of possible parse productions.


The improvement in performance of our model over the NBestParse-StringUnify, NBestParse-ExampleUnify and NBestParse-HeuristicUnify baselines highlights the importance of our DFA based semantic unification technique.

our model outperforms exact string based unification by over 30%, example based semantic unification by over 13% and our smart heuristic unification procedure by 9%.


We extend PROFINDER to leverage this intuition by incorporating a "stickiness" prior (Haghighi and Vanderwende, 2009) to encourage neighboring clauses to stay in the same frame.

along with introducing the background frame, the frame transition component now becomes where β is the stickiness parameter, and the event transition component correspondingly becomes Argument dependencies as caseframes As noticed in previous work such as Chambers and Jurafsky (2011), the combination of an event head and a dependency relation often gives a strong signal of the slot that is indicated.


Our generative model admits efficient inference by dynamic programming.

after collapsing the latent assignment of frame, event, and background into a single hidden variable for each clause, the expectation and most probable assignment can be computed using standard forward-backward and Viterbi algorithms on fixed tree structures.


Parameter learning can be done using EM by alternating the computation of expected counts and the maximization of multinomial parameters.

pROFINDER uses incremental EM, which has been shown to have better and faster convergence properties than standard EM (Liang and Klein, 2009).


We first evaluate our model on a standard entity extraction task, using the evaluation settings from Chambers and Jurafsky (2011) (henceforth, C&J) to enable a head-to-head comparison.

we use the MUC-4 data set (1992) , which contains 1300 training and development documents on terrorism in South America, with 200 additional documents for testing.


In PROFINDER, however, specificity can be easily estimated using the probability distributions learned during training.

we define the probability of an event head in a frame j as: and the probability of a frame given an event head as: We then follow the rest of C&J's procedure to score each learned frame with each MUC document.


In particular, we define the probability of an event head in a frame j as: and the probability of a frame given an event head as: We then follow the rest of C&J's procedure to score each learned frame with each MUC document.

a document is mapped to a frame if the average P F (w) in the document is above a threshold and the document contains at least one trigger word w with P (F |w ) > 0.2.


We converted the contributors into a form that is more similar to the previous MUC evaluation, so that we can fairly compare against previous work such as C&J that were designed to extract information into that form.

we extracted the head lemma from all the maximal noun phrases found in the contributor (Figure 3c) and treated them as gold-standard entity slots to extract.


In the following we use the rich mathematical framework of quantum theory to model semantic information.

we show how word meanings can be expressed as quantum states.


So we use it to show that dividing ρ by: normalizes its probability distribution over eigen kets: We represent the meaning of words by density matrices.

a lexical item w is modeled as an ensemble U w = {(p i , u i )} i of usages u i and the corresponding probabilities p i that w gets used "in the i-th manor".


These include a simple semantic space, where a word's meaning is a vector of co-occurrences with neighboring words (Mitchell and Lapata, 2010), a syntax-aware space based on weighted distributional triples that encode typed co-occurrence relations among words (Baroni and Lenci, 2010) and word embeddings computed with a neural language model (Bengio, 2001;Collobert and Weston, 2008) For all three models we used parameters that have been reported in the literature as optimal.

for the simple co-occurrence-based space we follow the settings of Mitchell and Lapata (2010): a context window of five words on either side of the target word and 2,000 vector dimensions (i.e., the 2000 most common context words in the BNC).


We would also like to model semantic composition as our approach can do this easily by taking advantage of the notion of quantum measurement.

we  can work out the meaning of a dependency tree by measuring the meaning of its heads in the context of their dependents.


Chunking We use the POS/NER/DEP tags directly just as one would in a chunking task.

suppose t represents the current token position and pos [t]  Our intuition is these chunking features should allow for learning which types of words tend to be answers.


Thus we also include a version of the POS/NER/DEP based feature for each token: We make an assumption that each sentence produces a candidate answer and then vote among all answer candidates to select the most-voted as the answer to the original question.

this overproduceand-vote strategy applies voting in two places: 1.


The recall from which the official Fmeasure is computed is weighted by snippet citations (a citation is a reference to the original document that supports the correct facet).

a snippet with more citations leads to a higher recall than one with less citations.


First, current utterance is affected by temporal history of preceding utterances.

when a new utterance is made it is likely to be related more closely with latest utterances than old ones.


This is because graphs are effective to express the relationship between words, and the graph operations that are required for keyword extraction are also efficiently performed.

whenever an utterance is spoken, it is represented as a graph (G 1 ) of which nodes are the potential keywords in the utterance.


Therefore, the history graph G 2 = (V 2 , E 2 ) is an undirected graph of keywords in the preceding utterances.

all vertices in V 2 are keywords extracted from one or more previous utterances, and the edge between two keywords implies that they co-occurred at least once.


Given the current utterance graph G 1 = (V 1 , E 1 ) and the history graph is a set of vertices from G 2 which are directly connected to the words in V 1 .

v 3 contains the words of G 1 and their direct neighbor words in G 2 .


When it appears in only one of the graphs, w 4 ij is set to be the weight of its corresponding graph.

ij ∈ E 1 and e 4 ij / ∈ E 3 , w 3 ij otherwise.


Only nouns are considered as potential keywords.

only the words whose POS tag is NNG (common noun) or NNP (proper noun) can be a keyword.


Thus, the proposed model does not reflect the temporal importance of preceding utterances if w 2 ij = 1 always.

under w 2 ij = 1, old utterances are regarded as important as recent utterances.


Note that the aforementioned six relation types are chosen by (1) discarding During, During Inv, and Identity, and (2) combining the two relation types in each of the five pairs, namely (Before, After), (IBefore, IAfter), (Includes, Is Included), (Begins, Begun By), and (Ends, Ended By), into a single type because they are inverses of each other.

if a relation instance (e 1 , e 2 ) is anno-  Table 1: The 14 temporal relations and their frequency of occurrences in TimeBank (v1.2).


Since our task is to predict the relation between them, we hypothesize that pairwise features, which are computed based on both elements, could better capture the relationship between them.

we introduce pairwise versions of the head word feature and the two prepositional lexemebased features in the baseline.


Given the potential usefulness of lexical relations for temporal relation classification, we create features based on four types of lexical relations present in Webster's online thesaurus 5 , namely synonyms, related-words, near-antonyms, and antonyms.

for each event e appearing in TimeBank, we first use the head word of e to retrieve four lists, which are the lists corresponding to the synonyms, related words, near-antonyms, and antonyms of e. Then, given a training/test instance involving e 1 and e 2 , we create eight binary features: whether e 1 appears in e 2 's list of synonyms/related words/near-antonyms/antonyms, and whether e 2 appears in e 1 's list of synonyms/related words/nearantonyms/antonyms.


We hypothesize that other WordNet lexical relations could also be useful for the task.

we employ four types of WordNet relations, namely hypernyms, hyponyms, troponyms, and similar, to create eight binary features for temporal relation classification.


Consequently, we create features for temporal relation classification based on four types of PropBank-style predicate-argument relations, namely directional, manner, temporal, and cause.

using SENNA's output, we create four binary features that encode whether argument e 2 is related to predicate e 1 through the four types of relations, and we create another four binary features that encode whether argument e 1 is related to predicate e 2 through the four types of relations.


It may not be immediately clear how to interpret the results under, for instance, the "All Rules" column.

it may not be clear what it means to add the six types of features incrementally to a rule-based system.


The highest accuracy and macro F-score are achieved when all types of features are used in combination with the "Rules + Features + Rules as Features" architecture.

this system achieves an accuracy of 53.4% and a macro F-score of 36.6% on the 2000-instance test set.


A closer examination of these results reveals that the hand-crafted rules used by the system correctly classify 239 of the 305 test instances to which they are applicable.

the rules achieve a precision of 78.3% and a recall of 15.3% on the test data.


In comparison to the baseline, both accuracy and macro F-score increase significantly when our system is used in combination with all feature types.

our system yields a relative error reduction of 16-25% for event-event classification and 6-9% for event-time classification over the baseline.


We defined the auxiliary feature functions -the expert knowledge on unlabeled data as 1 : where is an indicator function, and is a conditional probability specified in the form of by experts.

we took as the reference point when calculating g * (•).


We believe there is more to RM adaptation than that.

even if one The American list of goods that would incur tariffs in retaliation would certainly not be accepted by the Chinese government.


The state tree is explored in a best-first manner.

when the algorithm visits a node, it considers that node's best child.


Loading time was measured by running the decoder with empty input.


Statistical methods to modeling language semantics have proven to deliver good results in many natural language processing applications.

probabilistic topic models have been successfully used for tasks such as summarization (Gong and Liu, 2001;Hennig, 2009), text segmentation (Misra et al., 2009), lexical substitution (Dinu and Lapata, 2010) or word sense disambiguation (Cai et al., 2007;Boyd-Graber et al., 2007).


(2003)) topic model for estimating the semantic closeness of candidate terms, and explore different ways of utilizing LDA's topic information in automatic lexical chainers.

we use the GibbsLDA++ 5 framework for topic model estimation and inference, and examine the following LDA parameters: number of topics T , Dirichlet hyperparameters for documenttopic distribution α and topic-term distribution β.


More formally, let samples (d,w) be the vector of assignments that have been collected for a certain word w in a certain document d with each samples referring to the i-th sampled topic ID for (d, w).

samples (d,w) can be seen as the Markov chain for a particular word in a particular document.


Further let z (d,w) be the topic ID that was most assigned to the word w with respect to the samples in samples (d,w) .

z (d,w) is defined to be the sampled mode in samples (d,w) in case of multiple modes a random mode is chosen, which never happened in our experiments.


In our task, we only consider a prediction as a success if the system can determine which person in Wikipedia "Robert" is referring to.

our definition of entities depends on the given knowledge base, rather than human judgment.


Threaded discussion forums provide an important social media platform that allows netizens to express their opinions, to ask for advice, and to form online communities.

responses to major sociopolitical events and issues can often be found in discussion forums, which serve as an important source of public feedback.


In the initialization stage of Gibbs sampling, for a reply post to a recipient, we initialize its corresponding reply polarity s according to all the labeled polarity of interaction words.

if the majority of labeled interaction words are positive, we set s = 1, otherwise we set s = 0.


For our task, we take a simpler approach and use a sentiment lexicon together with some heuristics to predict the polarity of interaction expressions.

we first identify interaction sentences following the strategies from Hassan et al.


We made sure that the annotators understand the issue and the two major viewpoints before they annotated the posts.

as the Chinese data sets are about some controversial issues around the entities involved, we then defined two major viewpoints as support and not support the entity who initiated the event.


Our proposed model captures these observations in a principled way.

to incorporate the user interaction information, we proposed a novel generative process.


There is an important feature about this problem which makes it amenable to transfer learning so that we do not need to label data in every domain.

for a particular kind of intention such as buying, the ways to express the intention in different domains are often very similar.


Note that for this set of experiments, the source data all contain labeled posts from three domains and the target data contain unlabeled posts in one domain.

for each target domain, we merge three other domains for training and the target domain for testing.


Building on this observation, our technique for estimating the user's explicit meaning uses a probability threshold to select those individual frame elements which are most likely to be in the frame for a complete utterance, according to the predictive model.

at each partial result r j , we estimate the user's explicit meaning using a constructed frame: This approximation could work well if, in practice, the most probable frame elements prove to match fairly closely the user's non-incremental utterance meaning at the point this frame is constructed.


An analysis was then performed to compare the disambiguation performance of a supervised WSD system on a given ambiguous word against its corresponding pseudoword.

for our manually sense-tagged corpus we used the Senseval-3 English lexical sample dataset (Mihalcea et al., 2004), which contains 3593 and 1807 sense-tagged sentences for 20 ambiguous nouns (with an average polysemy degree of 5.8) in its training and test sets, respectively.


In addition to assessing the representativeness of pseudosenses, their degree of distinguishability has to be determined.

we have to determine how easily each pseudosense can be distinguished from the others in a pseudoword.


The overall score of 0.79 shows that similarity-based pseudowords effectively preserve the distinguishability of senses of their real counterparts.

they do not tend to have over-generalized pseudosenses which cover more than one sense.


We hope that the availability of this resource will enable large-scale experiments in tasks such as semantic role labeling, semantic parsing, and Word Sense Disambiguation.

as future work, we plan to utilize the generated pseudosense-tagged dataset to perform an in-depth study of different WSD paradigms.


To simulate an all-word scenario, we perform a 10-fold cross validation in our experiments, splitting the dataset into equal-sized folds randomly on the target word level.

all sentences for a particular target word fall into the same fold and thus either the training or the test set (but never both).


a number that denotes 'how good the candidate is, given the context'.

the best substitution candidate s (characterized by a set of features F) from a set of candidates S is obtained as argmax s∈S [P (good|F)], the next best as the argmax of the remaining elements, and so on.


This is especially important for the LexSub dataset, which contains examples from all major parts of speech (the TWSI dataset contains only noun targets).

we use: • 1-3-grams of main POS categories in a window around t, e.g.


intelligent, glimmery, polished, happy, ...) as potential paraphrases.

for the sentence "He was bright and independent and proud.


In order to assess the contribution of each group to the overall system performance, we performed a feature ablation experiment.

we trained the MaxEnt model with using all feature groups (this equals the model in Table 2) and then with leaving each of the feature groups out once.


These two drawbacks led us to consider an alternative method for the construction of the Tucker model.

we consider the factor matrices as given (as the output from our first step), and proceed to compute the core tensor G. Additionally, we do not use a latent representation for the first mode, which means that the first mode is represented by its original instances.


In our opinion, research on the modeling of compositional phenomena within a distributional framework would substantially benefit from new evaluation frameworks.

we think of a lexical substitution or paraphrasing task along the lines of McCarthy and Navigli (2009), but specifically aimed at the assessment of compositional phenomena.


Event summarization in this paper aims to generate a representative and concise textual description of the scheduled events that are being lively reported on Twitter, providing people with an alternative means of observing the world beyond the traditional journalism.

we investigate scheduled events of different types, including six of the NBA (National Basketball Association) sports games and a representative conference event, namely the Apple CEO's keynote speech in the Apple Worldwide Developers Conference (WWDC 2012) 1 .


In this graph G, each vertex is a sentence and an edge from s i to s j indicates that s j can be placed right after s i in a coherent summary.


This suggests a paradigm shift in the generation task when referring to visible objects, if the goal is to produce human-like reference.

this suggests moving from selecting properties that rule out other scene objects to selecting properties that are salient for the speaker (visually, conversationally, based on previous experiences, etc.).


To generate reference in this model, nodes correspond to general visual attributes and may generate forms for visual properties (attribute:value pairs).

a property such as color:red is generated from the attribute node color and a property such as size:tall is generated from the attribute node size.


We can penalize longer descriptions with a penalty proportional to the length of the property set under construction.

given a prior likelihood estimate for including an attribute att, α att , and the property set constructed so far A, we compute whether to add a prop-a.


Given a matrix A, the goal of CSSP is to select a subset of k columns whose span most closely captures the range of the full matrix.

the matrixÃ formed by orthogonally projecting A onto the k-dimensional space spanned by the selected columns should be a good approximation to A.


To evaluate the quality of this approximation, we can measure the Frobenius norm of the residual matrix ||A − A k || F .

4 The Eckart-Young theorem (Eckart and Young, 1936) states that A k is optimal in the following sense: truncated SVD gives the best rank k approximation to A in terms of minimizing the Frobenius norm of the residual matrix.


In CSSP, the goal is similar, with the added constraint that the approximation to A must be obtained by projecting onto the subspace spanned by a k-subset of the original rows of A.

5 Formally, the goal is to produce a (k × m) matrix S formed from rows of A, such that is minimized over all n k possible choices for S. Here S + is the (m × k) Moore-Penrose pseudoinverse of S, and S + S gives the orthogonal projector onto the rowspace of S. our goal is to select k data points which serve as a good approximate basis for all the data points.


While each of the joint factors all improve over the baselines on RF, the full model with all the joint factors does not perform as well as with some factors excluded.


For a thematic fit task, the correlation between calculated estimates and human judgements can be expected to improve.

good role fillers that are very different from one another and belong to different senses of a verb can all be assigned thematic fit scores as high as those of good role fillers of monosemous verbs.


This task has also been useful for comparing among DM models and the different thematic fit datasets.

we can qualitatively evaluate how reliable syntax can be for determining the semantic notion of thematic fit, and the relative strength of human intuitions on verb-imposed restrictions on the various roles (agent, patient, instrument, and location).


Note that rank accuracy and reciprocal rank produce identical orderings of methods.

whatever method performs best in terms of rank accuracy will also perform best in terms of reciprocal rank.


In contrast, we propose an approach that can handle arbitrarily complex scoring functions.

we employ a randomized greedy algorithm that jointly predicts segmentations, POS tags and dependency trees.


We also assume access to a morphological analyzer and a POS tagger that provide candidate analyses.

for each token x i , the algorithm is provided with candidate segmentations S i , and candidate POS tags T i and T i,j .


We improve the dependency tree y via a similar hill-climbing process.

we greedily update the head y i,j of each morpheme in a bottom-up order as follows where Y i,j is the set of candidate heads such that changing y i,j to any candidate does not violate the tree constraint.


We observe that most sentences converge quickly.

13 more than 97% of the sentences converge within first 300 restarts.


On both Arabic and Chinese, our model achieves improvement on the three tasks over state-of-the-art systems and pipeline variants of our system.

we demonstrate that OOV words benefits more from the power of joint prediction.


For this, after every action, in addition to updating the stack we also keep track of the dependencies resolved and update the dependency graph accordingly 1 .

we build the dependency graph for the sentence in parallel to the CCG derivation.


For NonInc, this dependency is resolved only after all the four remaining words in the sentence are shifted.

it has to wait for four more words before this dependency is resolved and hence the waiting time is four.


Furthermore, semantically empty elements are widespread in the DeepBank (around 21.5%), compared to a low rate of 4.3% in PAS.

the latter is somewhat more dense and consequently more syntactic.


Given the plethora of documents, it can be highly useful to have a system which can automatically predict relevant citations, for this could (1) aid authors in citing rele-vant, useful sources which they may otherwise not know about; and (2) aid readers in finding useful documents which otherwise might not have been discovered, due to the documents' being unpopular or poorly cited by many authors.

we are interested in citation prediction -that is, we aim to predict which sources each report document cites.


Hofmann and Cohn's (2001) PHITS seminal work on citation prediction included a system that was based on probabilistic latent semantic analysis (PLSA) (Hofmann, 1999).

they extended PLSA by representing each distinct link to a document as a separate word token -as shown in Equation 1 and represented by s l .


Very recently, (Wilson et al., 2014) used topic modelling to help predict movie recommendations.

one feature into their system was the KL-divergence between candidate items' topic distributions, but applying this towards citation prediction has yet to be done.


Of the past research which uses generative models for citation prediction, we believe LinkLDA is the only other system in which a source's prior citation probability plays any role in training the model.

in LinkLDA, the prediction metric is identical to ours in that the topics are marginalized over topics (Equation 3).


In attempt to combine the effectiveness of LDA in generating useful topics with the ability of dis- criminative classifiers to learn important features for classification, we use logistic regression with a linear kernel.

we train using L2regularization, which during test time allows us to get a probability estimate for each queried vector (i.e., a report-source pair).


Instead of using only the text from the abstracts, like in their research, to make the comparison more fair we used text from the entire documents -just like we did for the rest of our systems.

adhering to their naming convention, the features from their system that we used are: citation-count, venue-citation-count, authorcitation-count, author-h-index, age (# years between report and source), terms-citing, topics, authors, authors-cited-article, and authors-cited-author.


Consequently, we downsampled the negative examples.

for each report, we included all positive examples (the cited sources), and for each positive example, we included 5 randomly selected negative examples (sources).


Character n-grams are generally too short to represent any deep syntax, but some of them can reflect morphology to some degree.

we consider the following affix-like features by looking at n-grams that begin or end a word: prefix A character n-gram that covers the first n characters of a word that is at least n + 1 characters long.


3 riejohnson.com/cnn_download.html   fication, vice versa on topic classification, and the winner generally outperforms the conventional bagof-n-gram vector-based methods, as well as previous CNN models for text which are more complex.

to our knowledge, this is the first work that has successfully used word order to improve topic classification performance.


We regard the output of a convolution layer as an 'image' so that the output of each computation unit is considered to be a 'pixel' of m channels where m is the number of weight vectors (i.e., the number of rows of W) or the number of neurons.

a convolution layer converts image regions to m-dim vectors, and the locations of the regions are inherited through this conversion.


Note that the embedded vectors for the text regions listed in the same row are close to each other as they have a large value in the same component.

table 5 also shows that the proximity of the embedded vectors tends to reflect the proximity in terms of the relations to the target classes (positive/negative sentiment).


These patterns require tri-grams for disambiguation, and seq-CNN successfully makes use of them even though the exact tri-grams were not seen during training, as a result of learning, e.g., "am X satisfied" with non-negative X (e.g., "am very satisfied", "am so satisfied") to be predictive of the positive class through training.

cNN can effectively use word order when bag-of-n-gram-based approaches fail.


Mei and Zhai (2008) presented a sophisticated generative approach that frames summarisation under an Information Retrieval (IR) context.

an impact language model for a paper is first built as a mixture of a language model estimated from the paper's own text, and a weighted citation language model based on its collective citation contexts, using a compound coefficient reflecting both a sentence's proximity to the citation label (anchor) in the citing paper and the citing paper's authority calculated from the citation network using PageRank (Brin and Page, 1998).


We also plan to apply our method to the problem of multi-document summarisation.

we are very interested to test our system's performance on automatically generating a technical survey of a scientific paradigm, which thanks to the authors of (Mohammad et al., 2009;Qazvinian et al., 2013), has been established as a well-defined task with high-quality open data.


We present a novel method for aligning a sequence of instructions to a video of someone carrying out a task.

we focus on the cooking domain, where the instructions correspond to the recipe.


In contrast to declarative information, procedural knowledge tends to be inherently multimodal.

both language and perceptual information are typically used to parsimoniously describe procedures, as evidenced by the large number of "howto" videos and illustrated guides on the open web.


For each recipe, we apply a suite of in-house NLP tools, similar to the Stanford Core NLP pipeline.

we perform POS tagging, entity chunking, and constituency parsing (based on a reimplementation of (Petrov et al., 2006)).


1 Following (Druck and Pang, 2012), we use the parse tree structure to partition each sentence into "micro steps".

we split at any token categorized by the parser as a conjunction only if that token's parent in the sentence's constituency parse is a verb phrase.


To evaluate the quality of these methods, we created a random subset of 900 clips from each corpus using stratified sampling.

we picked an action uniformly at random, and then picked a corresponding object for that action from its support set uniformly at random, and finally picked a clip with that (action, object) label uniformly at random from the clip corpuses produced in Section 3; this ensures the test set is not dominated by frequent actions or objects.


We put together various benchmarks covering diverse aspects of meaning, to gain insights on the effect of perceptual information on different similarity facets.

we test on general relatedness (MEN, Bruni et al.


The owl example shows how multimodal   models pick taxonomically closer neighbours of concrete objects, since often closely related things also look similar .

both multimodal models get rid of squirrels and offer other birds of prey as nearest neighbours.


The multimodal representations induced by our models should be better suited than purely textbased vectors to label or retrieve images.

given that the quantitative and qualitative results collected so far suggest that the models propagate visual information across words, we apply them to image labeling and retrieval in the challenging zeroshot setup (see Section 2 above).


We conjecture that the representations naturally induced by our models display a similar property.

the entropy of our multimodal vectors, being an expression of how varied the information they encode is, should correlate with the degree of abstractness of the corresponding words.


We reuse the exemplar accounts used in the follower constraint in Section 3.3.

rather than using the fact that a user follows "dennis kucinich" as a soft constraint, we classify such a user as a Democrat.


We can see that for three of the four tasks (race, pol, pol-f), label regularization accuracy is either the same as logistic or within 2%.

using no user-annotated data, we can obtain accuracy competitive with logistic regression.


These differences suggest that the constraint selection algorithms allow label regularization to be more robust to noisy and conflicting constraints.

using constraint selection, we can view constraint engineering akin to feature engineering in discriminative, supervised learning methods -developers can add many types of constraints to the model without (much) fear of reducing accuracy.


This value is used to generate a probability distribution for the next word in the sequence.

the forward pass proceeds with the following recurrence, for t = 1, .


In contrast to these approaches, this paper presents a data-driven model that accomplishes (2) by blending two given words into a portmanteau.

with an input of "friend" and "enemy," we want to generate "frenemy."


using dynamic programming (Coster and Kauchak, 2011;Barzilay and Elhadad, 2003).

the alignment is computed by a recursive function that optimizes alignment of one or two consecutive sentences in one article to sentences in the other article.


The fragments are generated based on the second level of the syntactic parse tree.

each fragment is a S, SBAR, or SINV node at this level.


We develop a parallel corpus of aligned sentence pairs between standard and simple Wikipedia, together with their similarity scores.

we use our best case method to align sentences from 22k standard and simple articles, which were download in April 2014.


Lexical features have proven indispensable for the good performance of most applications dealing with language.

more generalized characterizations of the lexicon (Brown et al., 1992;Wilson et al., 2005;Feng et al., 2013;Ji and Lin, 2009;Resnik, 1995) have become key in overcoming issues with lexical sparseness and in providing practical semantic information for natural language processing systems (Miller et al., 2004;Rutherford and Xue, 2014;Velikovich et al., 2010;Dodge et al., 2012).


For complexity, the reference corpus is normal Wikipedia and the combined data is normal and simplified Wikipedia together.

we map a phrase w onto a style dimension via: We assign formality scores to phrases up to three words in length that occur at least three times total in ALL, regardless of whether they occur in both corpora.


Empirical process theory provides standard bounds of the form of Equation 4 (Kakade, 2011) in terms of the size of a cover of the function class under consideration (here Z(•; θ)).

given some α, we must construct a finite set of Z(•; θ) such that someẐ is everywhere a distance of at most α from every Z.


As hoped, sample complexity grows as the number of features, and not the number of contexts.

skip-gram models that treat context words independently will have sample efficiency multiplicative, rather than exponential, in the size of the conditioning context.


To train our model, we maximize conditional log likelihood of the training data augmented with a loss function via softmax-margin (Gimpel and Smith, 2010).

during training, we maximize L to be token-level asymmetric Hamming distance (where the output is viewed as binary edited/nonedited).


In addition, we fire features on repeated words and part-of-speech tags in order to capture the fact that the repair is typically a partial copy of the reparandum, with possibly a word or two switched out.

we fire features on the distance to any duplicate words or parts-of-speech in a window around the current token, conjoined with the  word identity itself or its POS tag (see the Duplicate box in Figure 2).


In Table 2 we see that neither source of predicted ToBI breaks does much to improve performance.

the gains from using raw acoustic features are substantially greater despite the fact that the pre-  Table 3: Disfluency prediction results on the test set; our base system outperforms that of Honnibal and Johnson (2014), a state-of-the-art system on this dataset, and incorporating prosody further improves performance.


This recovery proceeds incrementally (word by word), for RE of arbitrary length.

if U denotes the current word, we are interested in P (I|U ), the current hypothesis about the intended referent, given the observed word.


Further challenge is that available offerings constantly change and manually entered data often results in out-dated inventories.

the commercial real estate industry in the US (unlike residential real estate 1 ) does not have a centralized database or an established source of information.


Performance gains are more significant for named entities that are typically visually salient and are otherwise difficult (or impossible) to identify in a textonly version of the flyers.

improvements were most significant for named entities referring to space information.


In addition, we have shown that ubiquitous online formats such as PDF and HTML often exploit the interaction of textual and visual elements.

in the marketing domain, information is often augmented or conveyed by non-textual features such as positioning, font size, color, and images.


In this section, we present the basic formalism that will drive our attempts at multi-target translation.

we propose a generalization of SCFGs, which we will call multi-synchronous context free grammars (MSCFGs).


We define the set of multilingual phrases associated with f j i as the cross product of these sets.

if f j i is associated with 2 phrases in T1, and 3 phrases in T2, then there will be a total of 2 * 3 = 6 phrase triples extracted as associated with f j i .


In our MSCFG formalism, we also add new features regarding the additional targets.

in the case where we have one additional target α 2 , we add the log translation probabilities P (γ|α 2 ) and P (α 2 |γ), log lexical probabilities P lex (γ|α 2 ) and P lex (α 2 |γ), and word penalty for α 2 .


BLEU is used for both tuning and evaluating all models.

we tune and evaluate all models based on T1 BLEU, simulating a situation similar to that in the introduction, where we want to use a large LM in T2 to help translation in T1.


The result of experiments using all five languages as T1, and the remaining four languages as T2 for all of these methods is shown in First, looking at the overall results, we can see that MSCFGs with one of the choices of T2 tends to outperform SCFG for all instances of T1.

the gain for the full MSCFG model is large for the cases where the two target languages are French and Spanish, with en-fr/es achieving a gain of 1.46 BLEU points, and en-es/fr achieving a gain of 0.76 BLEU points over the baseline SCFG.


In addition, if u ∈ S and some phonological alternation p ∈ P maps u to a surface form s ∈ p(u) ∈ S, then (s, u) ∈ X .

we require that potential underlying forms appear as surface substrings somewhere in the data D (which means this model cannot handle e.g., absolute neutralisation).


Note that the set X is the same for all contexts c; we show below that restricting attention to just those surface/underlying pairs appearing in the context c degrades the model's performance.

the model benefits from the implicit negative evidence provided by underlying/surface pairs that do not occur in a given context.


To incorporate these hypotheses, we propose continuous space representations of linguistic typology.

we use an autoencoder (see (Bengio, 2009) for a review) to map each language into the latent space.


It handled missing values using multiple correspondence analysis (MCA).

we used the imputeMCA function to predict missing feature values.


ALC-CONT performed clustering in the continuous space, using cosine distance.

we can examine the effects of the typology evaluator and precision parameters.


Separately from the frequency score, a penalty score is computed that penalizes interpretations containing extraneous tokens.

the penalty counts the number of nouns or adjectives located outside the modifier and head.


The scores provided for each variant type correspond to either non-exclusive (upper portion of the table) or exclusive (lower portion) contribution of that variant type towards some extracted interpretations.

in the lower portion, the scores capture the fraction of the top 10 interpretations that are produced only by that particular variant type.


In this paper, we study the language of memes by jointly learning the image, the description, and the popular votes.

we propose a robust nonparanormal approach to transform all vision and text features into the cumulative density function space.


Our parser achieves an F-score of 63% (Row 3) and the result is 5% better than the first published result reported in (Flanigan et al., 2014) with the same training and test set (Row 2).

we also conducted experiments on the test set by replacing the parsed graph with gold 1 A script to create the train/dev/test partitions is available at the following URL: http://goo.gl/vA32iI 2 we used CoreNLP toolkit v3.3.1 and parser model wsjPCFG.ser.gz trained on the wSJ treebank sections 02-21. relation labels or/and gold concept labels.


We use convex geometry to show that the behaviour of training methods such as MERT (Och, 2003;Macherey et al., 2008), MIRA (Crammer et al., 2006), PRO (Hopkins and May, 2011), and others converge with a high feature dimension.

we analyse how robustness decreases in linear models as feature dimension increases.


We also note that non-linear models methods, such as neural networks (Schwenk et al., 2006;Kalchbrenner and Blunsom, 2013;Devlin et al., 2014;Cho et al., 2014) and decision forests (Criminisi et al., 2011) are not bound by these analyses.

neural networks are non-linear functions of the features, and decision forests actively reduce the number of features for individual trees in the forrest.


and lexico-structural, or deepsyntactic, (ATTR(ibutive), APPEND(itive) and CO-ORD(inative)) relations.

they do not contain any punctuation and functional nodes, i.e., governed elements, auxiliaries and determiners.


The evaluation shows that all straightforward mappings are performed correctly; English auxiliaries, that complementizers, infinitive markers and determiners are introduced, and so are Spanish auxiliaries, reflexive pronouns, and determiners.

the rules produce well-formed SSyntSs of all possible combinations of auxiliaries, conjunctions and/or prepositions for verbs, determiners and/or prepositions for nouns, adjectives and adverbs.


BLEU is the average of n-gram precisions and includes a brevity penalty, which reduces the score if the length of the output sentence is shorter than the gold.


3 and 4, we could simplify training by breaking up the estimation process into two steps.

we train alignment parameters, P (•| •) or domain parameters, P (D| •, •) first, hold them fixed before training the other kind of the parameters.


Nevertheless, a more effective way is to make use of the domain-specific seed samples and the pool of the rest sentence pairs in the heterogeneous data.

6 we train the model on each of the samples, assign-ing the derived probabilities as the initialization for their corresponding domain-conditioned alignment parameters.


Other studies have established consistent links between temporal orientation and demographic characteristics.

as one ages they think less about the immediate present and more about the future (Friedman, 2000;Nurmi, 2005;Steinberg et al., 2009), and females tend to think a bit more about the future than males (Keough et al., 1999).


Still, we validate that our approach is in line with psychological theory (discussed in Section 2) by correlating user-level temporal orientations with outcomes whose associations have been previously established: conscientiousness, age, and gender number of users; results controlled for age and gender.

future orientation should be positively correlated with conscientiousness, age, and being female, while present orientation should be negatively correlated.


The primary goal of the level one segmentation is to separate token replacements which are most centrally thought of as part of the normalization task from other instances that may require additional pragmatic inference.

we separate edits coarsely into three categories: • Token Replacements.


In all cases, to examine the effects of each nor-malization edit we model our analyses as ablation studies.

for every category in the taxonomy, we examine the effect of performing all normalization edits except the relevant case.


Another commonality among the analyses is that performance is measured relative to the top performance of the tool, not the task.


The states should then go back to doing something that actually makes sense and doesn't reward people like Britney Spears for being white trash.

s2:1 all well and good, but it is not the religious ceremony and sanction that gays are looking for.


This paper presents a framework to infer temporally-anchored spatial knowledge from verbal semantic roles.

our goal is to infer whether something is located somewhere or not located somewhere, and temporally anchor this spatial information.


Fifth, they considered that the security guard was probably located at the intersec-    tion before and after.

annotators understood that (1) the car was moving down a road and arrived at the intersection; (2) then, it was pulled over by a security guard who is probably stationed at the intersection; and (3) after the stop, the car probably continued with its route but the guard probably stayed at the intersection.


Finally, annotators answered around 50% of questions with probYES or probNO.

they found it likely that spatial information can be inferred, but were not completely certain.


Their formulation is similar to the previous sentence extraction method except that it allows the root node of a subtree to be other than the root node of the input tree.

their formulation allows multiple candidate root nodes for a subtree.


Our algorithm also fills a table for problem solving, but the table's size is |Z| × (L + 1).

the size of the table equals the number of nodes of the ZDD Algorithm 1 Dynamic Programming Algorithm Input: ZDD Z that represent T , length limit L, and wi, li for else 16: i ← lo(i) 17: return r that represents a set of subtrees T .


Pseudo-word methods consider a set of real words as pseudo-senses of an artificial pseudo-word (Pilehvar and Navigli, 2014).

we adopted a simple approach following Otrusina and Smrz (2010) to generate our pseduo-words.


Therefore, we annotated our test set (1152 utterances) by hand.

for the SMS/chat test set, we recruited three linguistically-informed native Mandarin speakers to annotate Chinese clauses containing empty categories.


This is a rulebased approach that mimics the intuitions described in Section 2.

for a Chinese utterance, we check whether the current utterance has an overt pronominal subject.


while we focus on resolving deictic non-anaphoric zero pronouns.

for an end-to-end system that resolves both deictic and anaphoric zero pronouns, one could first take our approach and whenever our model predicts "third person," which is often an anaphoric reference, one could apply one of these prior approaches for further reference resolution.


Despite this, their unsupervised model only agrees with their supervised model on 55% of zero pronoun antecedents, suggesting that this hypothesis is weak.

the complementarity of zero versus overt pronoun usage has been studied within various domains of linguistics.


Without human curation, however, the web data introduces significant noise.

everyday captions often contain extraneous information that is not directly relevant to what the image shows (Kuznetsova et al., 2013b;.


Goldberg and Elhadad (2013) and (Szántó and Farkas, 2014) show that using such information in the wordpreterminal emission probabilities of PCFGs can improve parsing accuracy.

szántó and Farkas (2014) show that this approach performs as well as an MA in some cases.


PDT and MTE have been annotated using two different guidelines that without further annotation effort could only be merged by reducing them to a common subset.

we removed features such as sub POS tags as well as markers for (in)animacy.


Further study (Kennedy and Inkpen, 2006;Andreevskaia and Bergler, 2008;Qiu et al., 2009) shows that corpus-based and lexicon-based approaches have complementary performances.

the corpus-based approach has high precision but low recall on positive instances, while the lexicon-based approach has high recall but low precision on positive instances.


The goal is to use these values to construct a sentiment lexicon, which assigns sentiment scores to each word.

we need the probability that each word w appears in a certain sentiment class, i.e.


The values of σ r p 0 are shown in Table 2.

for the accuracy based test-sets we found MRDS= σ p 0 that satisfied the following: we calculated the posterior probability p(θ B ≤ θ A ) with a flat prior of β(1, 1) to solve the following: 4 σ p0 = min{σ|∀ 0<θ< min(1−σ, 0.9) p(θ B ≤θ A |θ A =θ,θ B =θ + σ, n) < p 0 } Here θ A and θ B are probability of correctness of algorithms A, B andθ A ,θ B are observed empirical accuracies.


We trained our best MVLSA system with data from all views and by using the individual best settings of the hyper-parameters.

the configuration we used was as follows: n j = Count 1 4 , t = 12.5K, m = 500, k = 300, v = 16.


Our approach utilizes translations of words in other languages as bridges between synonymous words in English.

for each sense s of word w in Wiktionary, we first get all the available translations.


Unlike in other annotation schemes, each error is defined by its locus (regardless of the position of the incorrect tokens in the sentence) and all its alternative corrections must be mutually exclusive.

corrections are grouped whenever they refer to the same underlying error, even if the tokens involved are not contiguous.


It is clear that for error correction these counts should be weighted differently.

we would like to: • Reward correction more than preservation (i.e.


Here we begin to move beyond the ideas inspired by cross-language IR.

we posit that an optimal subset contains the beginning and ending terms of the region, along with a series of intra-terms that connect the two.


The key idea facilitating the easy extension to complex models and to non-parallel data settings is in our regularizer, which operates on the model parameters as opposed to their inferences.

mIR was designed to reward model pairs whose translation tables respect the invertibility intuition.


In the non-parallel data setting, only the target sequence f is observed and the source sequence e is hidden.

the model assigns the following probability to the observed data: the sequence f can arise from any sequence e by first selecting e ∼ p(e) and then proceeding according to the parallel-data generative story (Eq.


Each entry T i j represents the total probability mass mapped from source word e i to source word e j by first applying the source-to-target mapping T 1 and then the targetto-source mapping T 2 .

each diagonal entry T ii holds the probability of mapping a source symbol back onto itself, a quantity we intuitively believe should be high.


Since T is stochastic, setting it to the identity matrix I maximizes its trace.

the more T 1 and T 2 behave as (pseudo-)inverses of each other, the higher the trace is.


The probability of the nth sample under the kth model Θ k (for k ∈ {1, 2}) is denoted p k (x n k ; Θ k ).

in the parallel data setting, the probability of x n k under its model is: 1 whereas in the non-parallel data setting, the probability is defined as: Θ 2 ) = p(e n ; Θ 2 ).


The only extra work is in the Mstep, which optimizes a single (concave) objective function.

let z n denote the missing data, where, in the parallel data setting, only the alignment is missing (z n k = a n k ) and in the non-parallel data setting, both alignment and source symbol are missing (z n 1 = (a n 1 , e n ), z n 2 = (a n 2 , f n )).


Now, let C e, f 1 and C e, f 2 denote the expected counts for the t-table parameters.

c e, f k denotes the expected number of times a source-symbol type e is seen aligned to a target-symbol type f according to the posterior q k .


(2008) propose PostCAT which uses Posterior Regularization (Ganchev et al., 2010) to enforce posterior agreement between the two models.

they add a KL-projection step after the E-step of the EM algorithm which returns the posterior q(z | x) closest in KL-Divergence to an E-step posterior, but which also upholds certain constraints.


(2007) cast the two directional models as a single joint model by reparameterization and normalization.

both directional models, consisting of a t-table only, are reparameterized as:  They then maximize the likelihood of observed monolingual sequences from both languages: where, for example: Here, p(e) denotes the probability of e according to a fixed source language model.


Alignment F-scores are reported in Table 1.

the best results were obtained by MIR, when applied to the fertility based IBM4 modelwe obtained gains of +2.1% (Chinese-English) and +0.3% (Czech-English) compared to the best competitor.


An English-to-Japanese phoneme mapping wFST that encodes a phoneme t-table t 1 which was designed according to the best setting reported by Ravi and Knight (2009).

t 1 is restricted to either 1-to-1 or 1-to-2 phoneme mappings and maintains consonant parity.


9 The remaining required wFSTs were obtained by inverting the forward model wFSTs (that is, wFSTs 2,3,4 above), and the cascade was composed in the reverse direction.

by inverting t 1 , we obtained the Japanese-to-English t-table t 2 that allows only 2-to-1 or 1-to-1 phoneme mappings.


Training was done using the Carmel finite-state toolkit.

10 baseline and oracle rely on Carmel exclusively, while for MIR and bi-EM, we manipulated Carmel to output the E-step posteriors, which we then used to construct and solve the Mstep objective using our own implementation.


To do this, we incorporate the typed dependency grammar theory (De Marneffe and Manning, 2008) in a stateof-the-art frame-semantic driven unsupervised slot induction framework .

we build two knowledge graphs: a slotbased semantic knowledge graph, and a word-based lexical knowledge graph.


Then with each induced slot s i and its corresponding trained semantic decoder M i , an SLU model can be built to predict whether the semantic slot occurs in the given utterance in a fully unsupervised way.

the SLU model is able to transform the testing utterance into semantic representations without human involvement.


For the edges in E ws , we estimate the edge weights based on the frequency that the slot candidates and the words are parsed as slot-filler pairs.

the edge weight between the slot-filler w i and the slot candidate s j ,r(w i , s j ), is equal to how many times the filler w i corresponds to the slot candidate s j in the parsing results.


The other (k) is more linguistically motivated: a genuine pattern is more likely to be used for long stems, as affixation is a general operation for producing lexical derivations in many languages.

we set k = 5 and n = 2, as proposed in (Gaussier, 1999).


Our system uses Le and Zuidema (2014)'s reranker (Section 4.3).

it is worth noting that, in this case, each phase with iterated reranking could be seen as an approximation of hard-EM (see Equation 2) where the first step is replaced by instead of searching over the treebank space, the search is limited in a neighbour set N(D i ) generated by k-best parser P i .


Preconditions related to rootedness, singleheadedness, and acyclicity follow the arc-eager system straightforwardly: each transition that adds an arc (h, m) checks that m is not the root, m does not already have a head, and that h is not a descendant of m. Preconditions used to guarantee that trees output by the system are within the desired class are listed in Table 4.

they ensure that all crossed arcs are incident to registers, and that each pair of registers entails an interval corresponding to a selfcontained set of crossed edges.


The features for the two other transition systems are based on the same set, but with slight modifications to account for the different relevant domains of locality.

for the swap transition system, we updated the features to account for the fact that this transition system is based on the arc-standard model and so the most relevant positions are the top two tokens on the stack.


Feature embeddings can easily be used in feature-rich sequence labeling algorithms such as conditional random fields or structured perceptron, but our pilot experiments suggest that with sufficiently rich features, classification-based methods can be extremely competitive on these datasets, at a fraction of the computational cost.

we apply a support vector machine (SVM) classifier, Component Feature template Orthography (3) w i contains number, uppercase character, or hyphen adding dense features from FEMA (and the alternative representation learning techniques) to a set of basic features.


Our current approach assumes a fixed ontology, but we hope to explore a more bi-directional relationship between ontology and VSM in future work.

ly we envisage simultaneously incrementing ontologies with structure learning in addition to improving VSMs.


The problem of dealing with different distributions between training and test data is broadly investigated by the machine learning community.

approaches for dealing with domain drift are proposed within the scope of transfer learning, whose aim is to explore knowledge from one or more source tasks (henceforth, we use the terms domain and task interchangeably) and apply it to a target task (Pan and Yang, 2010).


The λ ∈ [0, 1] parameter controls the level of regularization applied to the model.

the sparsity of the predicted model is controlled via λ which weights the 1 -norm across all tasks.


We assign a weight w to each of the training instances, computed as the inverse of its class frequency in the training set.

w is obtained by dividing the total number of training samples by the number of training samples belonging to the class of the given utterance.


We thus consider matching minimum sub-events with two protein arguments.

we preprocessed complex events in K to identify minimum logical forms containing two protein arguments from each complex event, where arguments not directly leading to either protein are skipped.


To render learning tractable, the key idea is to augment the local event and argument states so that they contain sufficient information for evaluating φ K (z).

the semantic state z(n i ) needs to represent not only the semantic assignment to n i (e.g., a NEG-REG event trigger), but also the set of (possibly incomplete) sub-events in the subtree under n i .


We thus adopted a novel approach to evaluation by simulating a grounded learning scenario using the GENIA event extraction dataset (Kim et al., 2009).

we generated a set of complex events from the annotations of training sentences as the database.


This motivates us to explore prototype-driven learning (Haghighi and Klein, 2006) in combination with grounded learning.

we simulated expert selection by picking the top five most frequent trigger words for each event type from training data.


The GENIA dataset offers an excellent opportunity to study the robustness of grounded learning in light of such mismatch.

we simu- lated a grounded learning scenario with an incomplete database by populating events from the annotations of a random fraction of training text, and then learning GUSPEE with this database and all training text.


However, we can convert the extraction task into classifying minimum sub-events between proteins, for which existing methods can be applied.

we used binary sub-events in S(K) (Section 3.3) for distant supervision, and evaluated on classifying test sentences.


Since the language concepts (or bigrams) can be considered as key phrases of the documents, the other line related to our work is how to extract and measure the importance of key phrases from documents.

our work is related to key phrase extraction by using external resources.


Second, when we use the features from only one external resource, the results from some resources are competitive compared to that from the system using internal features.

when using the LM scores, Wiki or Word Embedding features, the results are slightly better than the internal features.


WSJ0109One can infer from the discourse that the problems require international experience, but Mr. Baum and Mr. Harper don't have that experience even without the discourse connective "however".

the truth value of this proposition is not affected by the presence or absence of this discourse connective.


Addressing these hard coreference problems requires significant amounts of background knowledge, along with an inference paradigm that can make use of it in supporting the coreference decision.

in Ex.1 one needs to know that "a limb bends" is more likely than "a bird bends".


In this paper, we develop the notion of Predicate Schemas, instantiate them with automatically acquired knowledge, and show how to compile it into constraints that are used to resolve coreference within a general Integer Linear Programming (ILP) driven approach to coreference resolution.

we study two types of Predicate Schemas that, as we show, cover a large fraction of the challenging cases.


Let M be the set of all mentions in a given text snippet, and P the set of all pronouns, such that P ⊂ M. We train a coreference model by learning a pairwise mention scoring function.

given a mention-pair (u, v) ∈ M (u is the antecedent of v), we learn a left-linking scoring function ) is a pairwise feature vector and w is the weight vector.


(2011) proposed to assemble a set of sentences which comply with Winograd's schema.

there are pairs of sentences which are identical except for minor differences which lead to different references of the same pronoun in both sentences.


Given a noun phrase, the segmentation problem is thus choosing a segmentation that maximizes the following conditional probability: Complementary to the segmentation problem is the semantic linking problem, which is to link s i to an attribute a i , which is the label of the i th attribute in the entity description vector.

we wish to maximize the probability of the attribute label sequence a given the segments of the noun phrase: Taking consecutive words with the same attribute label as the same semantic segment, the noun phrase segmentation and semantic linking problem is then: argmax a p (< a 1 , a 2 ,..., a n > | < w 1 , w 2 ,..., w n >) In the tag sequence <a 1 , a 2 , …, a n >, if a i and a i+1 are the same, then w i and w i+1 are assigned to the same semantic segment with tag a i .


Given the linear nature of our data, we employ a linear chain CRF.

given a sequence of words w, the probability of a label sequence a is defined as w,a i ,a i-1 ) is a feature function.


PPA is a form of frontotemporal dementia which is characterized by progressive language impairment without other notable cognitive impairment.

we consider two subtypes of PPA: semantic dementia (SD) and progressive nonfluent aphasia (PNFA).


While the performance of ASR and parsing components are often optimized independently of each other, in particular in case of the ASR to minimize recognitions errors, research has shown that ASR transcriptions with a lower error rate can in fact yield worse understanding performance Bayer and Riccardi, 2012) and that joint approaches to recognition and understanding can yield improved performance (Wang and Acero, 2006b;Deoras et al., 2013).

wang and Acero (2006b) have shown that applying the same grammar for speech recognition and understanding can yield improved understanding performance compared to applying a standard n-gram model with the ASR, since dependencies between acoustics and semantics can be captured.


Several approaches have addressed unsupervised (Solan et al., 2005;van Zaanen and Adriaans, 2001) and semi-supervised (Wong and Meng, 2001;Siu and Meng, 1999;Meng and Siu, 2002) induction of grammars, where the latter may comprise manual post-processing of automatically induced rules.

in order to be applicable as an SLU model, semantic information must be added manually, since only syntactic structures can be induced automatically in this case.


While in data-driven SLU research typically pipeline-based systems are applied, a few joint approaches have been proposed (Deoras et al., 2013;Wang and Acero, 2006b;Bayer and Riccardi, 2012).

the work presented here is most similar to the approach presented by Wang and Acero (2006b).


Specifically, the work presented here is most similar to the approach presented by Wang and Acero (2006b).

we also attempt to learn grammars applicable for both speech recognition and understanding.


In this paper, we explore the induction of semantic grammars under weak supervision provided in the form of ambiguous representations of the semantic context as explored in the NLP field of Semantic Parsing .

the training data comprises of a set of textual utterances coupled with symbolic context information from which we induce semantic parsers and derive different LMs for application with an ASR.


Recently, this task has been extended to consider speech data, both in learning and applying a parser.

the approach of Gaspers and Cimiano (2014) relied on transcriptions made by a task-independent phoneme recognizer as input.


lexical knowledge is used to segment NL such that all semantic referents observed in an mr are expressed by individual sequences, and hypotheses concerning a mapping between NL and semantics are created.

for instance, given an input example (1) NL: purple eight kicks to purple seven mr1: playmode(play on) mr2: pass(purple8, purple7) mr3: pass(purple2, purple5) the following alignment might be created: (2) assuming that the algorithm has learned in step 1 that "purple eight" and "purple seven" refer to purpl8 and purple7, respectively, it may use this knowledge to hypothesize that NL is an instantiation of a pattern "X 1 kicks to X 2 ", which in turn refers to the predicate pass(ARG 1 , ARG 2 ) with a mapping X 1 → ARG 1 , X 2 → ARG 2 .


Given the alignments induced in step 3, a parser is estimated by computing co-occurrence frequencies at different levels.

association scores are computed at three different levels, i.e.


Based on the written training data we created different LMs.

we created rule-based recognition grammars using the algorithm and further LMs, such as trigram models, for comparison.


In case of the inventory of syntactic constructions, alternative expansions of learned syntactic patterns were defined, and in case of the lexicon, alternative expansions of learned lexical units were defined.

with respect to the lexicon we defined a rule <ref> which comprises the learned lexical units.


In grammar creation, we also investigated the influence of occurrence frequencies of syntactic patterns and lexical units to enhance grammatical rules with weights.

we created both weighted and unweighted grammars.


We computed different language models for comparison; these were mainly stochastic LMs.

we created standard trigram language models from the written training data without making use of concurrent perceptual context information using SRILM (Stolcke, 2002).


Notice, however, that it is not common to apply grammars learned in an unsupervised fashion directly for SLU.

with respect to semantic parsing, automatically induced grammars are typically post-processed manually, which we refrained from doing, since the focus of this paper is on the automatic creation of speech recognition and understanding components.


We evaluated the word error rate (WER) as well as parsing accuracy for different language models and combinations thereof.

in case of applying recognition grammars we applied these also in combination with an n-gram back off LM.


In particular, in case of applying recognition grammars we applied these also in combination with an n-gram back off LM.

the n-gram model was applied in case of utterances which were rejected by the recognizer as out of grammar (OOG), as these might still be parsed subsequently by applying approximate matching.


In the following, we will discuss results for applying our induced grammars as an LM compared to using standard trigrams models (solely trained on the indomain data) as a baseline, since these yielded the best results.

we do not discuss the results achieved by the grammars induced in a purely syntactic manner as they performed worse than semantic grammars in all experiments, and we do not discuss the experiments for the interpolated/adapted Figure 2: A subset of weighted speech recognition grammar rules public <utterance> = /6/ <ref> again passes to <ref> | /199/ <ref> kicks to <ref> | ... <ref> = /15/ pink goalie | /132/ pink nine | /10/ pink one | ... trigram models as they performed worse than the in-domain trigram models with the exception of a very slight improvement when applied as a back off model for SLU.


Applying a back off trigram model yields only little improvement in parsing performance, although this may to some extent be due to not tuning recognition parameters.

if the ASR would be tuned to reject more OOG utterances correctly, these utterances might instead be recognized by a trigram model and probably parsed correctly by applying approximate matching.


in our case parsing performance is not directly dependent on the WER but rather on the type of errors made.

with respect to semantic parsing it is important that words carrying important meaning are recognized correctly.


Applying the semantically motivated grammars may have been beneficial in recognizing the semantic referents correctly because the system can explicitly learn them and their appearances in certain patterns in contrast to the trigram model.

if an utterance "pink nine passes the ball to pink seven" appears during recognition and "pink seven" has not been observed in the context of the preceding words during training, then the n-gram model would assign a low probability, likely leading to a recognition error such as "pink eleven".


This paper demonstrates that a data-aware crowdsourcing model incorporating a generative multinomial data model enjoys a strong competitive advantage over its discriminative log-linear counterpart in the typical crowdsourcing setting.

the generative approach is better except when the annotators are highly accurate in which case simple majority vote is often sufficient.


Broadly speaking, variational approaches to posterior inference transform inference into an optimization problem by searching within some family of tractable approximate distributions Q for the distribution q ∈ Q that minimizes distributional divergence from an intractable target posterior p * .

under the mean-field assumption we confine our search to distributions Q that are fully factorized.


One key innovation of our approach is multigranularity: we compute similarity between the two paraphrase candidates on multiple levels.

we compute similarity at four levels in this paper: unigram, short ngram, long ngram and sentence.


Different feature maps are expected to extract different kinds of sentence features, and can be implemented in the same convolution layer in parallel.

we use f sn = 6 feature maps on this level following Kalchbrenner et al.


Hall and Klein (2010) avoid this: they fit a full bigram model by inclusive KL divergence, which refuses to prune any values (see section 3).

they minimized D(µ F →V τ || q θ τ ), where τ was a simple fixed function (a 0-gram model) included so that they were working with distributions (see section 4.3).


The former leads to a smaller WFSA encoding.

it is cheaper to add abd once abc is already present, as a state already exists that represents the context ab.


Bhargava and Kondrak (2012) generalize this idea to improve transliteration accuracy by utilizing either transliterations from other languages, or phonetic transcriptions in the source language.

they apply an SVM reranker to the topn outputs of a base spelling-based model.


Figure 2 shows an example of three pairwise alignments that are mutually inconsistent: the English letter e is aligned to the phoneme [i] and to the grapheme ベ(BE), which are not aligned to each other Our solution is to select one of the input strings as the pivot for aligning the remaining two strings.

we align the pivot string to each of the other two strings through one-to-many alignments, where the maximum length of aligned substrings in the pivot string is set to one.


In our implementation, we merge different phonetic transcriptions of a given word into a single abstract vector representation.

we replace each phoneme with a phonetic feature vector according to a phonological feature chart, which includes features such as labial, voiced, and tense.


In order to circumvent these limitations, we propose to directly incorporate supplemental transliterations into the generation process.

we train our generalized joint model on the graphemes of the source word, as well as on the graphemes of supplemental transliterations.


It is well known that prosodic information is used by infants in early language acquisition.

prosodic boundaries have been shown to help infants with sentence and wordlevel segmentation.


Prosodic information is thought to play a fundamental role in early language acquisition, and provide infants with rich structural information about their language (Christophe et al., 1997).

prosody has been claimed to help infants find word boundaries (Christophe and Dupoux, 1996).


In order to be able to phrase (in)compatibility detection not only in continuous terms, but also as dichotomous tasks, we further produced a list of unambiguously (in)compatible pairs from the ends of the rating scale.

we manually inspected a subset of the list (before any computational simulation was run), and picked a mean 3.7 rating (exclusive) as minimum value for compatible pairs, and 1.6 (inclusive) as maximum score for incompatible ones.