The antonym pairs in column 2 were derived from the corresponding paraphrase pairs in column 1. such as hypernyms, holonyms, meronyms, and near-synonyms also tend to occur together more often than chance.

separating antonyms from pairs linked by other relationships has proven to be difficult.


The evaluation was focused solely on English, and it rarely concerned other languages (Adi et al., 2017;Conneau et al., 2017).

many 'core' methods to learn distributed sentence representations are largely under-explored in a variety of typologically diverse languages, and still lack a demonstration of their usefulness in actual downstream tasks.


This demonstrate that sentiment is encoded not only within a sentence, but also in its textual context.

a relatively small and accessible dataset (Wikipedia) is sufficient to provide a reliable model in most languages.


• selecting at most 1 tweet for every tweeterquery term combination.

the master set of tweets is not heavily skewed towards some tweeters or query terms.


The correlations obtained when repeating the experiment with three annotations for each 4-tuple is expected to be even higher.

the numbers shown in Table 2 are a lower bound on the quality of annotations obtained with three annotations per 4-tuple.


Specifically, for each split we randomly select 5% of training pairs for validation and take the remainder of the training set that does not share a lemma with the validation set as that fold's training instances.

the validation set size remains constant for each fold, but the training set sizes may vary between folds.


1 Entities are instances of concepts, i.e., they refer to specific individual objects in the real world, for example, Donald Trump is an instance of the concept politician.

entities are generally associated with a rich set of numeric and relational attributes (for politician instances: size, office, etc.).


In the BAKING A CAKE scenario, for example, there is little to no difference between mentions of making the dough and preparing ingredients.

these two events are often confused: Approximately 50% of the instances labeled as PREPARE INGREDIENTS are actually instances of MAKE DOUGH.


Our BATS experiment confirms the overall superiority of 3CosMul to 3CosAdd, although the difference is less dramatic.

lRCos considerably outdoes its competitors, although it does not manage to avoid the similarity problem.


We develop a simple log-linear model, in the spirit of traditional web-based QA systems (Kwok et al., 2001;Brill et al., 2002), that answers questions by querying the web and extracting the answer from returned web snippets.

our evaluation scheme is suitable for semantic parsing benchmarks in which the knowledge required for answering questions is covered by the web (in contrast with virtual assitants for which the knowledge is specific to an application).


For example, the word "wife" is an atomic binary relation in natural language, but expressed with a complex binary λx.λy.Spouse(x, y) ∧ Gender(x, Female) in knowledge-bases.

some of the complexity of understanding natural language is removed when working with a natural language representation.


While semantic parsers generally return a set, in COMPLEXQUESTIONS 87% of the answers are a singleton set.

features A candidate span a often has multiple mentions in the result set R. our feature function φ(•) computes the average of the features extracted from each mention.


Our candidate extraction step finds the correct answer in the top-K candidates in 65.9% of development examples and 62.7% of test examples.

our test F 1 on examples for which candidate extraction succeeded (WEBQA-SUBSET) is 51.9 (53.4 p@1, 67.5 MRR).


According to Frisson and McElree (2008), event recovery is triggered by the type clash, and the extra processing load is due to "the deployment of operations to construct a semantic representation of the event".

logical metonymy raises two major questions: i.)


8 On the other hand, the σ e k score weights the salience of the unified event e k by combining the weights of e i and e j into a new weight assigned to e k .

in this work, we compute activation of an event e simply by summing the activation scores of the single lexical items cuing it (i.e., the conditional probabilities of the event given each lexical item in the input sentence): the score σ e k measures the degree to which the unified event is activated by the linguistic expressions composing it.


In this work, we compute activation of an event e simply by summing the activation scores of the single lexical items cuing it (i.e., the conditional probabilities of the event given each lexical item in the input sentence): Thus, the score σ e k measures the degree to which the unified event is activated by the linguistic expressions composing it.

events that are cued by many constructions in the sentence should incrementally increase their salience.


With "cared about the strategy," it is TOPIC-ness that the preposition brings to the table; with "anger of Bipasha," it is the conceptualization of anger as an attribute that somebody possesses.

we turn to theories in Cognitive Semantics to define the phenomenon of construal as a means of understanding the contributions that are emerging from the adpositions with respect to the expressed event or situation.


Which rolefunction pairs are attested in particular languages, and are any universal?

far we have seen that certain scene roles, such as EXPERIENCER, STIM-ULUS, and PROFESSIONALASPECT, invite many different adposition construals-is this universally true?


Apply SCAN to the single terminal edge that makes up G to obtain [b(G), q : X → G • , φ q ] and finally apply COMPLETE to this and the axiom reach the goal Only if: Assume the goal can be reached from the axiom and G = e. Then the item [b(e), q : X → e, φ q ] must have been reached at some point for some q ∈ P G .

q : X → e is a production and so e = G ∈ L X (G).


However, building these frame lexicons is very expensive and time consuming.

it remains difficult to port applications from resource-rich languages or domains to data impoverished ones.


While this relation indicator can be used to capture arbitrary relational information, in this paper we set it to a combination of the directionality of the argument with respect to the predicate (L or R), and the preposition immediately preceding the argument phrase (or None if there isn't one).

for example, we have relational indicators such as "L-on", "R-before", "L-because", "R-None", etc.


For example, the predicate isPerpendicular(AB, CD) is mapped to the constraint  straints can be non-trivial to write and often require manual engineering.

gEOS's constraint set is incomplete and it cannot solve a number of SAT style geometry problems.


In our case, the execution model is a program sequence constructed from a set of theorem applications.

our work provides a way to integrate semantic parsing with probabilistic programming.


Experiments have shown that event knowledge, in terms of the prototypical participants of events and their structured compositions, plays a crucial role in human sentence processing, especially from the perspective of thematic fit: the extent to which humans perceive given event participants as "fitting" given predicate-role combinations (Ferretti et al., 2001;McRae et al., 2005;Bicknell et al., 2010).

computational models of language processing should also consist of event representations that reflect thematic fit.


While the collection focuses on German PVs, the representation of the meaning components (source and target domains, as well as directions) is language-independent.

the setup for collecting the meaning components that we present below should also be applicable to other languages, regarding complex verbs as well as regarding other language-specific targets that undergo meaning shifts.


We started out by assuming that "meaning shifts for German PVs frequently take place when combining a BV from a concrete source domain with a particle, resulting in a PV meaning (possibly among other meanings) related to an abstract target domain".

the generated PV sentences are expected to (i) represent shifted, non-literal language meanings and to (ii) exhibit abstract meanings, both considerably more often than the generated BV sentences.


Nevertheless, for the main theme of this sentence, Japanese Prime minister met US President, the two person names are equally important as the location they met.

we must look into the semantics of the individual words when computing their saliences.


Moreover, no such dataset exists where human annotators have manually rated words for their salience.

we resort to extrinsic evaluation, where, we first use (1) to create the sentence embedding for a given sentence using pretrained word embeddings and the NWS scores computed using the proposed method.


• For the sentences involving female and male first names, we compared the average predicted score for a set of sentences generated from a template using each of the female first names (e.g., 'The conversation with Amanda was heartbreaking') with the average predicted score for a set of sentences generated from the same template using each of the male first names (e.g., 'The conversation with Alonzo was heartbreaking').

eleven pairs of scores (ten pairs of scores from ten noun phrase pairs and one pair of scores from the averages on name subsets) were examined for each template-emotion word instantiation.


Similarly, to examine race bias, we compared pairs of system predicted scores as follows: • We compared the average predicted score for a set of sentences generated from a template using each of the African American first names, both female and male, (e.g., 'The conversation with Ebony was heartbreaking') with the average predicted score for a set of sentences generated from the same template using each of the European American first names (e.g., 'The conversation with Amanda was heartbreaking').

one pair of scores was examined for each template-emotion word instantiation.


We then turn to the consensus-dev subset, which consists of 100 sentences taken from the Wall Street Journal corpus.

gold standard syntax parses for these sentences can be obtained from CCGBank (Hockenmaier and Steedman, 2007).


These types are commonly recognized by named entity recognition systems, such as Stanford NER Tool .

in this scenario, an EL system can solely focus on entity disambiguation.


This means that a successful system for EL on question data needs to be able to recognize and to link all categories of entities.

we aim to show that comprehensive modeling of different context levels will result in a better generalization and performance across various entity categories.


These systems usually rely on an off-theshelf named entity recognizer to extract entity mentions in the input.

such approaches can not handle entities of types other than those that are supplied by the named entity recognizer.


Unfortunately, the described EL systems for short texts are not available as stand-alone tools.

the modern QA approaches mostly rely on off-the-shelf entity linkers that were designed for other domains.


QA systems need at least one entity per question to attempt to find the correct answer.

evaluating using the main entity shows how the entity linking system fulfills this minimum requirement.


According to the Distributional Hypothesis (Harris, 1954;Firth, 1968), similar linguistic contexts tend to imply similar meanings of words.

we suggest to perform a distributional semantic analysis in order to quantitatively investigate the contexts that concrete and abstract words frequently co-occur within.


On the contrary, it is rejected by the contextual ratio of abstract nouns as they primarily co-occur with other abstract nouns.

as we based our hypothesis on the theory of embodied cognition, the observed contextual pattern of abstract nouns challenges this theory.


While synsets "05665769; 13781820" have only the term "bonding", "00148653" includes as well the lemma "soldering" which is not in DepecheMood.

from step 2, "soldering" will have the same scores as "bonding".


not the ability of the model to identify POS tags, or the ability to cluster words in groups, or something else) in terms of properness of distances in words pairs with certain types of semantic relations (particularly, semantic similarity).

we could not say that a strong difference in performances of word embeddings on these two tasks could be highly expected.


In the scenario we are focusing on, sentence vectors are computed from a Siamese network, and thus it can be said that the vectors are in the same semantic space.

the effect of considering interdimensional interaction is less significant than that of multimodal pooling (e.g.


The inference from Fred believes John likes Mary to John likes Mary is clearly not semantically valid.

the AMR triples cannot be interpreted as stating semantic-web style facts; rather they state sub-formulas of a logical form.


All of these remarks also apply to the applications of (I), (J), and (K); these do not occur in the input tree, and the algorithm must make a choice somehow.

we do not know the worst-case complexity of our algorithm.


For example, it will assign the supertag N/N to most, but NP/N to other quantifiers.

in order to handle most, one has to manually change the parse trees.


their sibling nodes that define the frame structure.

to split frame x to two frames y and z, we replace the entire set x → v} with two similar sets where x gets replaced with y and z, respectively.


P Empty is another variant of P we get by removing all the annotated edges (that is, only nodes are kept).

they did not get any information from the annotated edges in P and any improvement came from bootstrapping alone.


We experimented with 300-dimension Fasttext embeddings and 100dimension Glove embeddings for CoNLL 2003 English data and saw no appreciable differences (± 0.2%).

we report scores with 100-dimension Glove embeddings due to the reduced training time and fewer parameters.


This is likely because the RETRIEVAL outputs with high interestingness scores frequently contained more surprising content such as new character names or new actions/events that were not found in the context.

a high interestingness score was not as strongly correlated with overall quality as with HUMAN outputs, for which interesting continuations were less likely to contain erroneous new material.


For SEQ2SEQ, the continuations can only be interesting if they are first somewhat relevant to the context; nonsensical output was rarely annotated as interesting.

the SEQ2SEQ relevance and interestingness scores have a higher correlation than for HUMAN or RETRIEVAL.


This has two main benefits: (i) Definitional sentences express (sense-specific) corpus-independent meanings of words, hence definition-driven approaches enable strong generalization-once trained, the model is expected to work well in opendomain testbeds; (ii) Global context from a large corpus and definitions provide complementary information for words.

our model, HYPERDEF, once trained on taskagnostic data, gets state-of-the-art results in multiple benchmarks 1 .


Thus, we formulate two prediction tasks: In the first task, we want to predict judgments of individuals on assertions based on other judgments by the same person.

the first task is formulated as follows: given a set of assertions a 1 , ..., a n relevant to an issue and the judgments of a person p i on a 1 , ..., a n−1 an automatic system has to predict p i 's judgment on the assertion a n .


The mass of the distribution is concentrated in the positive range of possible values, which indicates that the participants more often agree with the assertions than they disagree.

baselines accounting for this imbalance perform strongly in predicting judgments on assertions.


We suspect that this effect results from the distribution of similarity scores that is skewed to the positive range of possible scores.

the SVM probably assigns too much weight to ngrams that signal a negative score.


Note that collaborative filtering requires knowledge of how others judged the assertion for which the system tries to make a prediction.

these strategies are not applicable if we want to predict judgments on a previously unseen assertion.


As a simple collaborative filtering strategy, we predict how the majority of other persons judged an assertion.

we average the judgments of all other users and predict agree if this value is positive and disagree if the value is negative.


Our model performs best and achieves a significant boost in MR and Hits@10 compared to the baselines, while IKRL slightly outperforms TransE in terms of MR only.

hits@10 (%)  the results confirm the robustness of our method for large-scale datasets.


(LIFT(e),time(e)=t 1 , AGENT(e)=jack 1 @t 1 , AFFECTED(e)=ball 1 @t 1 ) Key to this framework is a theory of scales.

for example, height(o@t) maps a temporally situated object to the set of values on the height scale that this object takes over period t. Note that the object o may take different values on a scale sc over a given time interval t. sc(o@t) is a set of values.


The roles for the predicate Towards are FIGURE and GROUND, and an IMPRO fills the GROUND role of the predicate.

we have a role chain, from the head verb through RESULT and GROUND.


Then, they regard vertices that do not belong to any community as peripheral vertices.

the detected peripheries might not be strongly related to the associated cores because they are not densely interconnected with the cores in general.


Adding the vertex j to P i increases Q, if W ij − p is positive.

p i associated with core i must be the neighbours of vertex i with an edge weight of W ij > p. we have whereW (4) indicates that the maximisation of Q is equivalent to partitioning of the set of vertices V into C and V \ C such that the sum of edge weights given by (5) between C and V \ C is maximised.


However, solving the max-cut problem is NP-hard (Karp, 1972).

we use the Kernighan-Lin's algorithm (Kernighan and Lin, 1970) to find a good (but generally a suboptimal) solution.


Note how this expansion is quite different from the expansion of {Ford, Chevy}, and the difference is one of Similar How, since whether a word (say, BMW or FDR) belongs in the expansion depends not just on how much commonality it shares with Ford but on what commonality it shares.

this task allows the same surface form to belong to multiple sets, by virtue of being similar to items in distinct sets for different reasons.


Experiments on the development set showed that target word BERT representations and USE sentence embeddings are the best-suited for WiC.

'embedding-based features' here refers to these two representations.


Framing word prediction as an XML problem allows us to define a unifying framework for word embeddings.

we can systematically analyze the problem of training word embeddings using lessons from the XML literature.


In most extrinsic tasks, embeddings are usually used as a representation of examples which are the inputs of a classifier.

the performance on these extrinsic tasks is determined by two factors: the quality of representation and the quality of that classifier.


3) provide a systematic way to combine feature engineering and embedding learning.

the rich history of feature engineering in NLP becomes applicable for constructing word embeddings.


The problems outlined above are well known in SRL.

existing compositions (except f ⊗ ) can only model relations superficially for tasks currently used to train state of the art sentence encoders, like NLI or discourse connectives prediction.


3 Additionally, using C β (Table 6) instead of (Tables 3 and 4) for transfer learning in relational transfer tasks (PDTB, MRPC, SICK) yields a significant improvement on average, even when m = was used for training (p < 0.001).

we believe f C β ,− is an interesting composition for inference or evaluation of models regardless of how they were trained.


However, we hypothesize that each relation may contain useful information about the others, and training on only one relation inevitably neglects some relevant information.

training jointly on multiple relations may improve performance on one or more relations.


The analyses have focused on wordlevel models, yet character-level models have been shown to outperform word-level models in some NLP tasks, such as text classification (Zhang et al., 2015), named entity recognition (Kuru et al., 2016), and time normalization (Laparra et al., 2018a).

there is a need to study pre-trained contextualized character embeddings, to see if they also yield improvements, and if so, to analyze where those benefits are coming from.


Here, the internal classifier determines that the utterance indeed expresses that the user encountered a technical problem.

the control moves along the positive edge to the Headset problem node.


This node is mapped to the first bin in section 1 of the bin vector.

the regular counter is set to 1 for this bin.


This is made difficult by the fact that bot testing often involves somewhat realistic simulations of conversations.

in this binary classification task, a bot should be classified as either a production bot or not, given all of its conversations.


The intuition behind the clustering step is that it splits a corpus into qualitatively similar groups of documents.

we expect it to be possible to summarize the subject of each cluster by a small collection of topic words.


Let V 2 be a set of vertices, where each vertex corresponds to a document d in the corpus D. We add an edge between a word w and a document d if w (k, 1 − δ)covers d in the sense of cosine similarity between words and documents, s(w, d), in the embedded space.

the graph G We also have for each document, a cluster assignment from the spectral-clustering step, i.e.


also harder to answer, even for a model that makes use of a commonsense database.

we argue that MCScript2.0 is the first resource which makes it possible to evaluate how far models are able to exploit script knowledge for automated text understanding.


During the collection process, workers were not shown a text, but only a very short description of the text scenario.

many questions ask about general aspects of the scenario, without referring to actual details.


If connection holds then projected vectors of head and tail are connected by a translation vector with low error.

entities have different representations for hyperplanes of different relations where they are involved.


As expected due to the small-world nature of the Word-Net graph (Steyvers and Tenenbaum, 2005), most synsets are not similar at all: with JCN-S, the overwhelming majority of pairs feature similarity very close to zero; with LCH, most pairs have similarity below 1.0.

we filtered low-similarity pairs out, using similarity threshold of 0.1 for the JCN-S and ShP datasets, 0.3 for the WuP dataset and 1.5 for the LCH dataset (due to substantial differences in similarities distributions, as shown in Figure 1).


Particularly, in the case of WordNet, each node (synset) has 36 synsets in its V 2 on average, and half of the nodes do not have any neighbors at all.

only 2,935,829 pairwise similarity calculations are needed, 1,000 times less than when calculating similarities between all synset pairs.


In applications one could replace path-based measures with dot product between path2vec embeddings, gaining significant speedup in distance computation between nodes.

our model could be used to speed up various other graph-based algorithms that make use of node distance computations, such as Floyd (1962) algorithm, Dijkstra (1959) algorithm, or algorithms for computing node betweenness centrality (Brandes, 2001).


(2017), who inferred the attributes from the text of the users, which could arguably confound the interpretation of the results, as domains are defined using the same information available to the classifier.

we used only information from user profiles to obtain their demographic attributes.


While our data collection includes only public data, due to the potential sensitivity of user profile information, we stored only data necessary for this study.

we anonymized the personal information and deleted user images after retrieving the demographic attributes from the Microsoft API.


The goal is to perform well at predicting document classes, while the demographic attribute tasks are modeled primarily for the purpose of learning characteristics of the demographic groups.

the model aims to learn discriminative features for text classification while learning to be invariant to the linguistic characteristics of the demographic groups.


Because language styles vary across groups, as shown in Section 2.2, information from each task could be useful to the other.

our intuition is that while we model the document and demographic predictions as independent tasks, the shared feature space allows the model to transfer knowledge from the demographic tasks to the text classification task and vice versa.


Second, current benchmarks for natural language inference (Bowman et al., 2015), do not include challenging examples with negation.

state-of-the-art approaches (Chen et al., 2017) trained on these benchmarks are unable to solve text-hypothesis pairs that contain negation.


A subsequent proof by Lewis (1976) found that there is no single proposition based on the antecedent and consequent such that its probability will consistently match the conditional probability.

one could have a propositional theory of conditionals, or validate the Equation -but not both.


We found that in the CoNLL dataset, roughly 34% of the PER and GPE named entities that are the head of a mention of some gold cluster in the test set are also the head of a mention of a gold cluster in the train set.

there is considerable overlap, or leakage, between the names in the train and test sets.


For each dataset, highest score for each dataset is bolded and is underlined if the difference between it and the other model's score is statistically significant (p < 0.20 per a stratified approximate randomization test similar to that of (Noreen, 1989)  the system must determine whether a given pronoun refers to one, both, or neither of two given names.

the task can be viewed a binary classification task in which the input is a (pronoun, name) pair and the output is True if the pair is coreferent and False otherwise.


In addition, we create 10 binary features (one per semantic class) indicating whether any of the head words belongs to each class, based on the classifier's predicted labels.

for each event expression the semantic classifier generates 50 semantic features.


This inflates the number of parameters in the LSTM's weight matrices.

the likelihood of overfitting is increased -an issue which is further aggravated by the fact that SPR2 data are significantly fewer than SPR1 data.


However, we still observed small skews towards CONTRADICTION.

we investigate whether our methods reduce the probability of predicting CONTRADICTION when a hypothesis contains an indicator word.


The threshold that is being used can only commutatively compound the thresholds of both quantifiers that are used.

the model would not distinguish between the sentences "Every bass player is taller than most guitarists" and "Most bass players are taller than every guitarist", although their semantics are distinct.


It would not evaluate the hypothesis in that model, yielding no result at all.

the consequence relation is not monotone, as the addition of an extra inconsistent premise makes the computation diverge.


GF only provides binary clausal polarity, whereas our implementation of predicates requires a manyvalued logic.

vP negation in examples such as "John isn't a guitarist" rules out both that John is a guitarist, and that John is in the undecided area between θ − guitarist and θ + guitarist .


To address this problem, for each hypothesis s i ∈ S a , we take the corresponding reference r t i ∈ R t a as the one with the least Levenshtein Distance (Navarro, 2001)  As per this alignment, in the above example, we will have correct alignments (s 1 , r 3 2 ) and (s 2 , r 3 1 ).

for each simplified sentence s i we have 3 corresponding references r 1 i , r 2 i and r 3 i .


Each side of the bipartite graph can be assigned to a page, such that no two edges on the same page cross.


We built our work on an existing corpus of highquality video material, which is restricted to the cooking domain.

the corpus covers only a limited inventory of activity types and action verbs.


Eq(6) is tight if c appears in the parse tree: z c = 1, otherwise Eq 7is tight.

minimize ψ with respect to p, we have The second equation holds since, for any two factors, c and c ′ , g c (or h c ) and g c ′ (or h c ′ ) are separable.


For any p, p c ∈ ∆, and z ∈ Z, ψ(p, z) ≥ ϕ(z).

ψ(p, z) is an upper bound function of ϕ(z).


The minimal gap between different scores is 1 N ×T after averaging, where N is the number of training samples, and T is the iteration number for perceptron training.

the upper bound can be tightened as U B = ⌊N T ψ⌋ N T .


Since the size of the set {z j |j > k} is |S| − k, hence there are at least |S| − k parse trees whose scores ϕ(z j ) are less than ψ(p, z k ).

because ϕ(z ′ ) ≥ ψ(p, z k ), hence z ′ is at least the k th best parse tree in subset S. we can search the k best parse trees in this way: for each sub-problem, we use DP to derive the k best parse trees.


Let γ w be the sentiment profile of the word w. where Σ i∈{pos,neg,neu} p i w = 1.

a sentiment profile of a word is essentially a 3-sided coin, corresponding to its probability of coming out positive, negative, and neutral, respectively.


Instead, we propose a simpler method to produce small forests which have more accurate oracles by forcing the beam search shift-reduce parser to keep the correct state in the beam buffer.

the correct tree will always be encoded in a packed forest.


(2011) were unavailable for us as a baseline.

we utilize the alignment approach by Meyer and Gurevych (2011) to create a similarity-based baseline, with minor modifications.


However, intuitively not all links in an article are equally meaningful.

for the SR configuration, we decided to retain only the category links and the links within the first paragraph of the article.


Using only monosemous links (LM) improves the recall considerably, but unlike the WN-OW alignment, it stays fairly low.

even when using semantic relations and monosemous links in conjunction (SR+LM), the recall can only be increased slightly, leading to an overall F-Measure of 0.39.


Here, lcd is uninformative, since screen already denotes a unique object in the environment.

it is not important to learn an accurate classifier for lcd.


Unlike algorithms such as Perceptron and stochastic gradient descent, our method keeps track of dual variables and updates the weight vector more aggressively.

this training process is as efficient as existing online learning methods, and yet derives consistently better models, as evaluated on four benchmark NLP datasets for part-ofspeech tagging, named-entity recognition and dependency parsing.


While syntactic functions of arguments change with the form of the event (e.g., active vs. passive forms), the semantic roles of arguments remain invariant to their syntactic realization.

since the first works, SRL systems have assumed access to the syntactic structure of the sentence (Gildea and Jurafsky, 2002;Carreras and Màrquez, 2005).


4, as well as the corresponding constraints cTree and cSubtree.

we note that the syntactic paths π are not tied to any consistency constraint other than each of the paths being a well-formed sequence of dependencies linking the predicate to the argument.


Based on the above analysis, we can conclude that the tree structure that is independent from Tree-bank resources and simultaneously considers the bilingual mapping inside the bilingual sentence pairs would be a good choice for building treebased translation models.

complying with the above conditions, we propose an unsupervised tree structure for treebased translation models in this study.


Under this initial STSG, the sampler modifies the initial U-trees (initial sample) to create a series of new ones (new samples) by the Gibbs operators.

new STSGs are created based on the new U-trees simultaneously and used for the next sampling operation.


For a U-tree of a given sentence, we define snode as the non-root node covering at least two words.

the set of s-node contains all the tree nodes except the root node, the pre-terminal nodes and leaf nodes, which we call non-s-node.


For a specific U-tree, the states of s-nodes are fixed.

by changing an s-node's state, we can easily transform this U-tree to another one, i.e., from the current sample to a new one.


To avoid confusion, we split the Two-level-Rotate operator into two operators: Two-level-left-Rotate operator, which works with the parent node and its left child, and Two-level-right-Rotate operator, which only considers the parent node and its right child 10 .

the operator used in Figure 4 is a Twolevel-right-Rotate operator.


This indicates that the U-trees are more consistent with the word alignment and are good at capturing bilingual mapping information.

because parse trees are always constrained by cross-lingual structure divergence, we believe that the proposed U-trees would result in a better translation grammar.


The best value for this parameter depends on the size of the training set, characteristics of the language being learned, and also the evaluation metric being used, as in some cases the best SBF1 and EMMA scores are obtained with completely different values.

we tuned the value of the perplexity threshold on the labelled set for each language and evaluation metric for different unlabelled training set sizes.


Such an order is chosen on the basis of where the head a h and the dependent a d inheriting the gap are placed within the involved trees.

in our algorithm we need to represent several intermediate parsing states.


It means that these two words are attested together on the Web with respect to the intensity patterns more than with other candidate words.

we try to respect the order of such word pairs more in the final ranking when we are breaking constraint-violating cycles.


Although intuitive in theory, the extracted pairwise scores are frequently too sparse for this to work.

many adjectives have no score with a particular headword.


We assume the distortion procedure is influenced by three factors: words distance, head-modifier cohesion and modifier-modifier cohesion.

we further decompose the distortion model into three terms as follows: where is the words distance term, ℎ is the head-modifier cohesion term and is the modifier-modifier cohesion term.


The head-modifier cohesion term ℎ is used to penalize the distortion probability according to relationships between the head node and its children (modifiers).

we define ℎ as the product of probabilities for all head-modifier pairs taking as head node: , ∈ where } is the headmodifier cohesion relationship between and one of its child , ℎ is the corresponding probability, and are the aligned words for and .


Obtaining the Viterbi alignment itself is not so easy for our model.

we take the Viterbi alignment of the sequence-based model (HMM model) as the starting point, and iterate the hillclimbing algorithm (Brown et al., 1993) many times to get the best alignment greedily.


Concretely, when sampling the aligned on iteration .

we sample the aligned position ( +1) as follows:  ) calculated with Eq.


However, the training procedure is very timeconsuming, and they trained the model with only 100 hand-annotated sentence pairs.

their method is not suitable for large-scale tasks.


In other words, they increase the diversity of model candidates for parser ensemble.

pseudo-grammar-based models lead to further improvements for parser combination.


During the PSMT decoding process, the output sentence is built from left to right, while the input sentence positions can be covered in different orders.

reordering in PSMT can be viewed as the problem of choosing the input permutation that leads to the highest-scoring output sentence.


Relaxing this kind of constraints means dramatically increasing the size of the search space and making the reordering model's task extremely complex.

even in language pairs where long reordering is regularly observed, PSMT quality degrades when long word movements are allowed to the decoder.


Assuming a one-to-one correspondence between source and target phrases, reordering in PSMT can be viewed as the problem of searching through a set of permutations of the input sentence.

two sub-problems arise: defining the set of allowed permutations (reordering constraints) and scoring the allowed permutations according to some likelihood criterion (reordering model).


1 illustrates the scoring process: when a partial translation hypothesis H is expanded by covering a new source phrasef , the model returns the log-probability of translating the words off in that particular order, just after the last translated word of H. In details, this is done by converting the phraseinternal word alignment 4 to a source permutation, in just the same way it was done to produce the model's training examples.

the global score is independent from phrase segmentation, and normalized across outputs of different lengths: that is, the probability of any complete hypothesis decomposes into J factors, where J is the length of the input sentence.


The accuracies for the folds from 10 fold cross validation in the feature vector combination method were also found to be significantly higher than those from word features only (using a paired Wilcoxon signedrank test).

both topic and writing features are clearly useful for identifying great articles.


We then generate two features for each pair, PPMI(x i , x j , left) and PPMI(x i , x j , right).

there are 2n(n − 1) PPMI values in the second set of features.


In theory, the set of all IsA relations, I, and the set of all metaphor relations, M , do not overlap, because by definition, the source concept and the target concept in a metaphor are not the same thing.


(6) to the following: Here, P (y|x, h x ) means when x is interpreted as an h x , the probability of y as a target metaphorical concept for h x .

given h x , y is independent with x, so P (y|x, h x ) can be simply replaced by P (y|h x ).


5 Furthermore, C is independent of x in a type 2 or 3 metaphor, since a metaphor is an unusual use of x (the target) within a given context.

p (C|x, y) = p (C|y), where p (C|y) is available from Eq.


Given a treebank oracle, a gold sequence of transitions is derived, and a predictor is trained to predict transitions along this gold sequence, without considering any parser state outside this sequence.

once the parser strays from the golden path at test time, it ventures into unknown territory and is forced to react to situations it has never been trained for.


Thus, in both cases, we need to make sure that a transition is applied during training only if it is considered correct by the oracle; else we cannot guarantee that later oracle predictions will be correct.

on line 9, we either remain in the same configuration (easyfirst) or follow the oracle prediction and go to t o (c) (left-to-right systems); on line 11, we in fact also go to t o (c), because in this case we have t p (c) = t o (c).


However, they are still only guaranteed to be correct on a subset of the possible configurations.

when using a non-deterministic oracle for training in Algorithm 2, the function o(t; c, T ) returns true if T is reachable from c and t ∈ o n (c, T ).


But for configurations from which T cannot be derived, the dynamic oracle returns transitions leading to the best parse A (in terms of Hamming distance from T ) which is reachable from c. This is the behavior expected from a dynamic oracle, as defined in Section 4.1.

in order to derive a dynamic oracle for an arc-decomposable transition system, it is sufficient to show that the transition cost function C(t; c, T ) can be computed efficiently for that system.


We estimate alignments jointly on all data sets.

tEtRA may have artificially good phrase tables.


We train k binary one-vs-all classifiers that use local context only and no sequence information (where k is the number of tags).


ALL accuracies suffer little when leaving out suffixes, but the feature space is much smaller: about 3000 dimensions.

for domains where we expect few OOVs, omitting suffix features could be considered.


Latent sequence states are a form of word representation.

it would be interesting to compare them to the non-sequence-based distributional representation that FLORS uses.


From this initial document set (hereafter referred to as WI-KICONTENT), we only retained languages that had more than 1000 documents (44 languages), and generated documents for WIKIPEDIAMULTI as follows: 1. randomly select the number of languages K (1≤K≤5) 2. randomly select a set of K languages S = {L i ∈L for i = 1• • •K} without replacement 3. randomly select a document for each L i ∈S from WIKICONTENT without replacement 4. take the top 1 K lines of the document 5. join the K sections into a single document.

of the procedure, the relative proportion of each language in a multilingual document tends not to be uniform, as it is conditioned on the length of the original document from which it was sourced, independent of the other K −1 for the other languages that it was combined with.


Specifically, given a collection of documents retrieved for a target language, the task is to identify the documents that contain text in English in addition to the target language.

we re-train each system for each target language, using only training data for English and the target language.


Since MTurk does not provide any information about workers demographics, including their language competencies, it can be hard to exclude such workers.

naive data collection on MTurk may result in noisy data.


This allows AGs to learn reusable sub-trees such as words, sequences of words, or smaller units such as Onsets and Codas.

while ordinary PCFGs have a finite number of parameters (one probability for each rule), Adaptor Grammars in addition have a parameter for every possible complete tree rooted in any of its adapted non-terminals, leading to a potentially infinite number of such parameters.


By the c-feasibility assumption, there exists some solution having ck labels.

some label from that solution covers at least a 1 /ck fraction of the remaining words.


Temporal representation and reasoning in the medical record are difficult due to: (1) the diversity of time expressions; (2) the complexity of determining temporal relations among events (which are often left to inference); (3) the difficulty of handling the temporal granularity of an event; and (4) general issues in natural language processing (e.g., ambiguity, anaphora, ellipsis, conjunction).

the signals used for reconstructing a timeline can be both domain-specific and complex, and are often left implicit, requiring significant domain knowledge to accurately detect and interpret.


The clinical domain includes no shortage of inferences, shorthands, and unusual use of language, but the structure of the underlying timeline is not unique.

of this, we have been able to adopt most of the framework from ISO-TimeML, adapting the guidelines where needed, as well as reframing the focus of what gets annotated.


The isolated analysis approach is inappropriate because i) it sacrifices acuracy by ignoring the intricate interplay among different types of relations; ii) it could lead to conflicting predictions such as estimating a relation candidate as both negative and comparative.

in this paper, we identify instances of both sentiment polarities and comparative relations for entities of interest simultaneously.


For example, we can encode constraints to discourage an attribute to participate in a polarity relation and a comparative relation at the same time.

the system extracts a set of correlated instances of sentiment-oriented relations from a given sentence.


Approaches rely on linguistically motivated rules and assume the existence of independent keywords in sentences which indicate comparative relations.

these methods fall short of extracting comparative relations based on domain dependent information.


How-ever, if we represent a candidate by all these words, it is very likely that the instances of different relation types share overly similar features, because a mention is often involved in more than one relation candidate, as shown in Figure 2.

the instances of different relations represented by overly similar features can easily confuse the learning algorithm.


Thus, it is critical to select proper constituents or sentences as textual evidences for each relation candidate in both training and testing.

we divide the task of sentimentoriented relation extraction into two subtasks : i) identifying the most likely MRGs; ii) assigning proper textual evidences to each edge of MRGs to support their relation assertions.


The selection between two alternative structures is formulated as τ b m i + τ t m i = 1.

as this influences only the edges incident to an attribute mention, we keep all the constraints introduced in the previous section unchanged except for constraint (3), which is modified as we can have either binary edges or ternary edges for an attribute mention.


The textual features of a relation candidate are all lexical and sentiment predictor features within a surface distance of four words from the mentions of the candidate.

this baseline does not need the inference constraints of SENTI-LSSVM for the selection of textual evidences.


We denote a segmentation of the N tokens of corpus w N 1 into M N segments by specifying segment boundary markers Setting a boundary marker s i =a means that we put a segment boundary before the a-th word token (or the end-of-corpus marker for a=N +1).

our corpus is segmented into token sequences j=1 .


On the other hand, cautiously using only the most recent data risks overfitting to short-term trends and missing important timeinsensitive effects (Blei and Lafferty, 2006;Wang et al., 2008).

in this paper, we take steps toward methods for capturing long-range temporal dynamics in language use.


Nonetheless, we want to update our model so that it can make a better prediction at t + 1.

we can only minimize the bound until timestep t. .


As described in §2.2, our model is evaluated by the loss suffered at every timestep, where the loss is defined as the negative log likelihood of the model on text at timestep w t .

at each timestep t, we need to predict (the distribution of) w t .


It is difficult to establish any comprehensive taxonomy of multiword idioms, let alone develop linguistic criteria and corpus resources that cut across these types.

the voluminous literature on MWEs in computational linguistics-see §7, Baldwin and Kim (2010), and Ramisch (2012) for surveys-has been fragmented, looking (for example) at subclasses of phrasal verbs or nominal compounds in isolation.


The problem of identifying the densest subgraph of size at least k is NP-hard (Feige et al., 1999).

we define a heuristic for k-partite graphs inspired by a 2-approximation greedy algorithm for arbitrary graphs (Charikar, 2000;Khuller and Saha, 2009).


The naive way to translate nodes using Tr is to use all translations for a given word.

for each edge in the source DM between lemmas s 1 and s 2 , we ob- The score σ T of a target edge is defined as the mean of the scores of all source edges that map to it.


It follows a different strategy, however: it collects cooccurrence counts from a German corpus and translates the context dimensions into the English Roget categories.

it crucially requires a large target language corpus, which our crosslingual methods (Section 3) avoid.


For instance, "spurs" can refer to two distinct sports teams (San Antonio Spurs, which is a basketball team in the US, and Tottenham Hotspur F.C., which is a soccer team in the UK), besides being a non-entity verb or noun.

retrieving tweets via keyword queries inevitably mixes different entities.


For efficiency, we only adopt the first order model.

the prediction function can be decomposed for each anchor a independently, e * = arg max e∈E(a) w T Φ(m, a, e).


In fact, six of the entities are not dominant, marked by in Table 1.

merely linking to the most popular one (i.e., the base system LP) is not a good strategy.


However, if an entity e is "trending" at some time t or location l, that is, e is mentioned by more tweets than usual at t or l, we shall expect P (e|t, l) > P (e).

the system is more likely to link to e instead of ∅ based on our spatiotemporal model (3), resulting in higher recall.


We achieve a relaxation by using sums of hingeloss terms to interpolate c over the continuous domain [0, 1] and d, over [0, 1] 3 (even though they are defined only for binary domains).

the HL-MRF formulation is equivalent to Eq.


(2006) distribute the text-level scores from their SVM classifier with the corpus, so we simply work with those, after transforming them into probabilities via logistic regression (a standard technique called Platt scaling (Platt, 1999)).

let q u and q v be the probabilistic sentiment predictions for u and v on a given bill.


810,' receives a probability of 98% of expressing a positive opinion on H.R.

(i.e., House of Representatives) bill 810, whereas the prediction for the speech containing the words, 'I urge my colleagues to vote against both H.R.


Chanceadjusted agreement levels ranged from very high to chance levels, with similar variation for pairwise agreement (Passonneau et al., 2012a).

the annotations for certain words appear to be low quality.


For example, in a binary task with 95% prevalence of one category, two 90% accurate annotators would have negative chanceadjusted agreements of 0.9−(.95 2 +.05 2 ) 1−(.95 2 +.05 2 ) = −.053.

high chance-adjusted interannotator agreement is not a necessary condition for a high-quality corpus.


Training these models took far longer than our neural-network model, due to the need to use the forward-backward algorithm with a 425×425-dimensional transition matrix during training (rather than considering each word's category independently).

we only experimented with the Turian-50 embeddings with a 7word context window, which attained the best performance using the neural network.


However, our approach requires discriminative training, which makes our method not comparable with their results.

we choose the MSM2013 dataset in our experiment and compare our system with the MSM2013 official runs.


However, for 3-wise CRFs, the naive dual decomposition requires many small slaves to cover the order-3 factors.


We are: (1) selecting a subset of candidate phrases, and (2) re-ordering the selected phrases (hence making the problem NP-hard).

we encode our problem using Integer Linear Programming (ILP) (Roth and tau Yih, 2004;Clarke and Lapata, 2008) and use the CPLEX (ILOG, Inc, 2006) solver.


For example, N T i = {NN,NP, ...} if p i corresponds to an "object" (noun-phrase).

equation 8 enforces the correspondence between phrase types and nonterminal symbols at the tree leafs.


As noted by recent studies (Mason and Charniak, 2013;Kuznetsova et al., 2013;Jamieson et al., 2010), naturally existing image captions often include contextual information that does not directly describe visual content, which ultimately hinders their usefulness for describing other images.

to improve the fidelity of the generated descriptions, we explore image caption generalization as an  optional pre-processing step.


We want to keep words referring to actual objects in the image.

we use V (x j ), a visual similarity score, as our confidence of an object corresponding to word x j .


Neither BLEU nor METEOR directly measure grammatical correctness over long distances and may not correspond perfectly to human judgments.

we supplement automatic evaluation with human evaluation.


The math function UNIQUE on Austin in Figure 2b indi-cates Austin is the only value of x which can satisfy the query graph in Figure 2c.

the denotation 1 of the NL query graph is {AUSTIN}.


While MIRA (Chiang et al., 2008), Rampion (Gimpel and Smith, 2012), and HOLS (Flanigan et al., 2013) have been shown to be effective over larger feature sets, they are difficult to explicitly regularize -this will become important in Section 4.2.

we use the PRO optimizer (Hopkins and May, 2011) as our baseline learner since it has been shown to perform comparably to MERT for a small number of features, and to significantly outperform MERT for a large number of features (Hopkins and May, 2011;Ganitkevitch et al., 2012).


We view these arguments as being arranged in a stack with j 1 X 1 at the top and j m X m at the bottom.

another way of writing the category X above is as X D A˛, where˛is a (possibly empty) stack of m arguments.


Every rule is obtained by choosing a specific degree d 0 and specific directions (forward or backward) for each of the slashes j i , while X , Y and the Y i are variables ranging over categories.

for every degree d 0 there are 2 d forward rules and 2 d backward rules.


The arity restriction implies that the category of every node u on the spine properly between f and r takes the form Xˇu, jˇuj > 0, and that the category at the root takes the form Xˇ, jˇj 0.

the category X is never exposed in c, except perhaps at r. As we will see in Section 5.4, this property, together with a careful selection of "split nodes", will allow us to decompose derivations into smaller, shareable parts.


Here the length ofˇis bounded by the maximum degree d of a composition rule in the grammar, and ar.Y / is bounded by the maximum arity a of an argument from the (finite) set L of arguments in the lexicon (recall Section 4.4).


This requires the distinction of several possible cases, resulting in four different realizations of the application rule (Vijay-Shanker and Weir, 1993, p. 616).

the V&W algorithm uses nine (forward) inference rules, against our algorithm in Figure 5 which uses only three.


However, because the alternatives are based on the error analysis of the participating systems, the revised set may be biased .


In the case of the ∅ candidate, the word "camera" acts as the target.

the confusion set for the article classifier is modified as follows: instead of the three articles (as shown in Sec.


When successful, crowdsourcing enables gathering annotations at scale; however, its performance is still limited by (1) the difficulty of expressing the annotation task as a simply-understood task suitable for the layman, (2) the cost of collecting many annotations, and (3) the tediousness of the task, which can fail to attract workers.

several groups have proposed an alternate annotation method using games: an annotation task is converted into a game which, as a result of game play, produces annotations (Pe-Than et al., 2012;.


When a player chooses an image in the mystery gate, that image receives n-1 positive votes for it being a good depiction of the sense; the remaining unselected images receive one negative vote.

an image's rating is the cumulative sum of the positive and negative votes it receives.


Second, many games rely on a sense of novelty, i.e., not seeing the same pictures repeatedly; however, limiting the images to those in G can create issues where too few   Puzzle Racer is intended to produce a sense-image mapping comparable to what would be produced by crowdsourcing.

we performed a largescale study involving over 100 players and 16,000 images.


Because the game's costs are fixed, the cost per annotation is driven down as players compete.

puzzle Racer reduces the annotation cost to ≤27% of that required by crowdsourcing.


(2014)) and more apt video games could be developed for text-based annotations such as PP-attachment or pronoun resolution.

in our first future work, we plan to develop new types of video games for textual items as well as extend the current games for new semantic tasks such as selectional preferences and frame annotation.


Past variational inference techniques for adaptor grammars assume a preprocessing step that looks at all available data to establish the support of these nonparametric distributions (Cohen et al., 2010).

these past approaches are not directly amenable to online inference.


We first establish by induction on δ * the following ( ): c and λ, λ, λ, λ ∈ δ * by definition of the extended transition function.


As before, letting q = Suff k−1 (w), it follows that Suff k−1 (w) = Suff k−1 (q) so tails f (w) = tails f (q), and Suff k−1 (wa) = Suff k−1 (qa) so tails f (wa) = tails f (qa).


By extension there will be |q 2 | − k states between q 1 and q 2 , each of which will be merged, triggering pushback of u, so that by the time q 1 and q 2 are merged, δ 1 (q 2 , #) = ux = z = δ 1 (q 2 , #).

all non-determinism can be removed, and so T is subsequential.


The first item of the seed, S , covers all and only the states of the target: the left projection of these pairs is thus linear in n and every right projection is at most n • m + p. Thus the size of S is at most n Concerning the second part, S , its cardinality is at most n • |Σ| (in the rare case where Q = Q).

each element of the left projection of S is of length (k + 1) and each element of its right projection is at most of length (k + 1) the size of the characteristic sample is in , which is clearly polynomial in the size of the target transducer.


This confirms the effectiveness of loss-augmented training of our sequence models for opinion extraction.

all following results are based on the loss-augmented version of our models.


However, in our joint inference approaches, the attribute classification models are trained independently from the segmentation model, and the training examples for the classifiers only consist of correctly labeled expressions ("unjust" as a nested opinion expression in this example would not be considered in the training data for the attribute classifier).

the joint inference approaches do not suffer from this issue.


According to Vygotsky's zone of proximal development (Vygotsky, 1978), the range of suitable material is very small.

creating a test that fits this narrow target zone is a tedious and timeconsuming task.


Psycholinguistic analyses (Sigott, 2006;Grotjahn and Stemmer, 2002) discuss in detail that both strategies are required for successful C-test solving.

we developed a model for C-test difficulty that incorporates features from both processing levels (see Figure 4).


Note that the automatic prediction is compared with the actual error rates, not the human predicated ones.

it is possible to outperform human performance with automatic methods and provide a very helpful tool.


However, as the actual error rates are continuously distributed, gaps that are close to the class boundaries are more likely to be mislabeled.

we also test regression prediction using the actual error rates instead of the artificially determined classes.


Therefore, when generating a CSC training example in DAGGER (lines 7-12), we do not need to know whether an action y j t is correct or not, we only evaluate what the effect of y j t is on the loss incurred by the complete action sequence.

it does not need to decompose over the actions taken to evaluate them.


This configuration parallels the information used in our model but fails to exploit raw data.


This translates to low edge weights in the underlying similarity graph structure (between mentions), thus forming favorable candidates in the initial phases of cluster splitting using minimum edgecut based graph partitioning.

cROcS inherently incorporates early partition (during the clustering phase) of such possibly singleton mention clusters from the "main data", thereby helping in de-noising and efficiency.


The gain observed is not as high as that for the John Smith dataset, as in the WePS-2 corpus documents are longer, giving richer context with fewer ambiguous entity mentions.

simpler methods also perform fairly well.


We observe that the trained value of θ (obtained previously) for both the WePS-2 and the NYT datasets are close to the optimal setting as seen from Table 4 and provide nearly similar F1 score performance.

we set θ = 0.65 and consider the entire input corpora as test set for the remainder of our experiments.


Note that while the special edge e * ,∅ is needed for the direct correspondence with the ILP score, its weight is constant across variable assignments.

this edge only adds a constant offset of c ∅ * to the ILP solution.


Similarly, our goal in this study was not to provide state of the art results for a particular task, but to demonstrate a framework for learning structures that are richer than previous structured models.

our experiments focused on understanding how SPRITE compares to commonly used models with similar structures, and how the different variants compare under different metrics.


However, if we know that the topic of the document concerns the effects of low temperatures on physical health, then the negative emotional reaction sense should become less likely.

in this case, knowing the topic helps narrow down the set of plausible senses.


As in §6.1, for each instance in the original dataset, we find the most similar sentence in ukWac for each instance using word embeddings and add it into the dataset.

the number of instances is doubled, and we use the enriched dataset for our sense-topic model.


Our data comes from the FAQs produced by computer and operating system manufacturers and other well-managed websites.

the text in the FAQ solutions is of high quality.


In serial implementations of on-demand extraction, very frequent patterns are a major computational bottleneck (Callison-Burch et al., 2005;Lopez, 2008b).

for patterns occurring more than n times, for some fixed n (typically between 100 and 1000), these implementations deterministically sample n matches, and only extract translations of these matches.


To measure end-to-end translation speed using pycdec, per-sentence grammars are first extracted for a batch of sentences and written to disk, then read from disk during decoding.

we report times separately for grammar extraction, disk I/O, and decoding (which includes time for reading the grammar files from disk back into memory).


Since the extractor should generalize over sentences with dissimilar expressions, it is crucial to identify sentences with  diverse event phrases.

we also measured the accuracy and the count of a "diverse" condition: only consider the subset of sentences with distinct event phrases.


In both of these algorithms, orthographic relatedness is based on simple deterministic rules.

semantic relatedness plays an essential role in the success of these methods.


While language models that operate on words linked through a dependency chain -called syntactic n-grams (Sidorov et al., 2013) -can improve translation, some of the improvement is invisible to an n-gram metric such as BLEU.

tuning to BLEU does not show the full value of a syntactic language model.


This model predicts the context of a word given the word itself, and through this process embeds words into a latent conceptual space with a fixed number of dimensions.

related words tend to have vector representations that are close to each other in this space.


For example, by chaining even small numbers of direct associations, such as breakfast -pancakes, pancakes -hashbrowns, hashbrowns -potato, and potato -field, a model that does not control for semantic drift may be tempted to answer questions about breakfast venues with answers that discuss wheat fields or soccer fields.

keeping these indirect associations short and on-context is critical for QA.


In particular, the factorization's loss function is not based on L 2 , and is much less sensitive to extreme and infinite values due to a sigmoid function surrounding w • c. Furthermore, the loss is weighted, causing rare (w, c) pairs to affect the objective much less than frequent ones.

while many cells in M PMI equal log 0 = −∞, the cost incurred for reconstructing these cells as a small negative value, such as −5 instead of as −∞, is negligible.


However, our experiments show that those word embeddings are often not suited for FCT.

we propose to Table 1: Feature templates for word w i in phrase p. t(w): POS tag; c(w): word cluster (when w is a function word, i.e.


a preposition word or conjunction word, there is no need to have smoothed version of the word features based on clusters.

we directly use the word forms as features as shown in line 3 of the table); h: position of head word of the phrase p; Dis(i − j): distance between w i and w j (distance in tokens).


In contrast to other applications, like vision, where a single input can cover most or all of the model parameters, word embeddings are unique to each word, so a word will have its embedding updated only when the word appears in a training instance.

only words that appear in the labeled data will benefit from finetuning and, by changing only part of the embedding space, the performance may be worse overall.


Language modeling provides a method to update all embeddings based on a large unlabeled corpus.

we combine the language modeling object (Eq.


When a word's embedding is changed in a task-specific way, it will impact the rest of the embedding space through the LM objective.

all words can benefit from the task-specific training.


A common approach to composition, a weighted sum of the embeddings (which we include in our experiments as Weighted SUM), is a special case of FCT with no non-lexical features, and a special case of RNN if we restrict the W matrix of RNN to be diagonal.

rNN and FCT can be viewed as two different ways of improving the expressive strength of Weighted SUM.


Since each thread takes a similar amount of time, the actual weights will be roughly c 1 = c 1 * n 1 and c 2 = c 2 * n 2 .

we first fix the numbers of threads and then tune c 1 and c 2 .


Training the FCT models with pre-trained word embeddings requires running the skip-gram model on NYT data for 2 iterations: one for word2vec training and one for learning FCT.

we also run the word2vec model for two epochs to provide embeddings for the baselines.


While the model trained by HS is efficient to evaluate perplexities, NCE training requires summation over all words in the vocabulary in the denominator of the softmax to compute perplexity, an impracticality for large vocabulary.

we report NCE loss with a fixed set of samples for NCE trained models.


From this point of view, our work is closely related to the discriminative training methods for low-rank tensors in NLP (Cao and Khudanpur, 2014;Lei et al., 2014), while it can handle more complex ngram-to-ngram tasks, where the label y also has its embedding composed from basic word embeddings.

our model can capture the above work as special cases.


(2013), which suggests inferring additional links using the aliases for each article.

we compile a list of alternative titles, including anchor texts and NE redirects (i.e., the linked phrases and redirected pages that refer to NE articles).


The recall, however, of the DL classifier is high, covering as many NEs as possible in all possible domains.

the DL classifier is better than the SSL classifier in detecting NEs that appear in ambiguous contexts (they can be used for different NE types) and with no obvious clues (NE indicators).


The values of hyperparameters A 0 offered a natural method to include any prior knowledge.

they can be regarded as pseudo-counts of prior observations and they can be chosen to represent any prior level of uncertainty in the confusion matrices, Π.


The accuracy and diversity of the single classifiers are the main conditions for a combined classifier to have better accuracy than any of its components (Dietterich, 2000a).

in the next section we take into consideration various classifier combination methods in order to aggregate the best decisions of SSL and DL classifiers, and to improve overall performance.


These low false positive rates and the high percentage of the NEs that are detected and missed by the two classifiers in a mutually exclusive way can be exploited to obtain better results, more specifically, to increase recall without negatively affecting precision.

we restricted the combi-nation process to only include situations where the base classifiers agree or disagree on the NE type of a certain word.


This way of using the IBCC model results in a performance level that is superior to all of the individual classifiers and other voting-based combined classifiers.

the IBCC model leads to a 12% increase in the performance of the best base classifier, while voting methods increase the performance by around 7% -10%.


Existing data sets, such as Free917 (Cai and Yates, 2013) and We-bQuestions (Berant et al., 2013), would not allow us to evaluate performance on this subset of language.

we evaluate our system on a new data set with unconstrained language.


In contrast, our approach uses an open predicate vocabulary and can learn denotations for words whose semantics cannot be represented using Freebase predicates.

our approach can answer many questions that these Freebase semantic parsers cannot (see Section 7.5).


The input data for training our DBN model are the original feature vectors on training and unlabeled data.

to train our DBN model, we first need to extract the original features for syntactic parsing on these data.


This is because our syntactic parser uses history-based features which depend on previous actions taken when parsing a sentence.

features on unlabeled data can only be extracted after the data are parsed.


Using the LFRs, we test the our adapted systems on both in-domain and out-of-domain data.

we get many test results, each corresponding to a splitting configuration.


They took each feature in the original problem and made three versions of it: a general version, a source-specific version and a target-specific version.

the augmented source data contains only general and source-specific versions; the augmented target data contains general and target-specific versions.


But we note that All performs better than both OnlySrc and OnlyTgt on the target domain test, although its training data contains some auto parsed data.

the data of the target domain, labeled or unlabeled, are potential in alleviating the adaptation problem of different domains.


The method measures each token by the z-score of its log-odds-ratio as: It uses a background corpus when calculating the log-odds-ratio δ t for token t, and controls for its variance σ 2 .

it is capable of detecting differences even in very frequent tokens.


Our LF-DMM model uses the DMM model assumption that all the words in a document share the same topic.

the process of generating a document in a document collection with our LF-DMM is as follows: a distribution over topics θ is drawn for the document collection; then the model draws a topic indicator z d for the entire document d; for every i th word w d i in the document d, a binary indicator variable s d i is sampled from a Bernoulli distribution to determine whether the Dirichlet multinomial or latent feature component will be used to generate the word w d i , and finally the word is generated from the same topic z d by the determined component.


Evaluating on just this intersection, we find that the inclusion of entity semantics yields an improvement in accuracy from 37.5% to 39.1%.

while the overall improvements offered by entity mention semantics are relatively small, this is due in part to the poor recall of the state-of-the-art coreference resolution system; if coreference improved, the impact of the entity mention semantics would increase correspondingly.


This indicates that the annotations reflect topical similarity in addition to capturing functional or definitional similarity.

we had the data re-annotated by two authors of this paper who are native English speakers.


For JN and NN, we extracted pairs containing aligned, adjacent tokens in the two phrases with the appropriate part-of-speech tag.

we extracted pairs like easy job, simple task for the JN section and town meeting, town council for the NN section.


It is also common to express syntactic information through derivation or composition.

these words, orthographically written together, actually have word-internal syntactic struc-tures.


9 We expect this bias to help learning, by limiting the search space and allowing the algorithm to harness training data in a more efficient way.

bDSM should be better at learning with less data, where SVM will be prone to overfitting.


Such resources are unavailable for many languages, and expensive to create.

a model that can jointly learn the sound patterns and the lexicon of a language would open up the possibility of automatically developing SLP capabilities for any language.


Generates a parse tree , which deterministically sets the value of z i,t .

the latent variables our model defines for each utterance are: In the next section, we derive inference methods for all the latent variables except for { θ q } q∈Nag∪N noisy-channel , which we integrate out during the inference process.


The new PCFG G is thus a grammar that can be used to parse the terminals v i and generate derivations that are rooted at the start symbol of the AG.

we transform the task of sampling d i and o i to the task of generating a parse for v i using G , which can be efficiently solved by using an variant of the Inside algorithm for PCFGs (Lari and Young, 1990;Johnson et al., 2007;Goodman, 1998;Finkel et al., 2006).


The parameter vectors φ and θ are specific to the language being generated.

under our generative story, a language is created as follows: 1.


However, a contributing factor may be the increased error rate of the phonological learner, visible even with oracle data.

we suspect that a S θ model with better generalization would improve our results at all training sizes.


GPs have been shown to achieve state of the art performance in various regression tasks (Hensman et al., 2013;.


the hyperparameters: The gradients of G depend on the underlying kernel.

we can employ any kind of valid kernel in this procedure as long as its gradients can be computed.


From Equation 2 we know that the kernel is a double summation over the ∆ function.

all gradients are also double summations, but over the gradients of ∆.


Abstract entities such as companies were consistently annotated as having the potential to be aware.

in B for example, Ford is annotated as being aware that Mazda makes the Tracer for the company.


Yet in practice, we will not be able to find the true p θ (model misspecification) nor exactly compute the marginals of p θ (computational intractability).

this paper proposes a training procedure that compensates for the system's approximations, adjusting θ to reduce the actual loss of h θ (x) as measured at training time.


ERMA can still be used with such a badly misspecified model-one of its advantages-but no amount of training can raise CLL's objective above −∞, since any non-projective gold tree will always have probability 0.

for CLL only, we replace each gold tree in training data with a minimum-loss projective tree (Carreras, 2007).


The only context feature that helps us discriminate between the different possible disambiguations is Cleveland; the remaining features (such as Money magazine, United States) could easily appear in the context of the other candidates.

combining the evidence from all features may blur the distinction between these entities.


We treat mentions (or mention clusters) as independent and identically distributed; the complete model is shown in Figure  1.

given a test mention (w , b ), we can compute the entity posterior by marginalizing out the latent variable k: the entity posterior is a product of the name score (p(e|w )) and context score ( k b k p(k|e)/β k ).


All mentions in a document (or a suitable batch of documents) can be sent in parallel to the selected servers.

the time to resolve all mentions in a document is proportional to the maximum response time of an entity server, rather than to the sum of per-mention response times.


Note that clusters will often include proper names mentions, referential common noun phrases, and referential pronouns.

plato detects and links both nominal and named mentions.


Note that in text, not all event arguments are always present for an event mention; they may even be distributed over different sentences.

whether two event mentions are coreferential should be determined based on the context.


However, the coreference relations may also exist across documents -the same event may be described in multiple documents.

it is ideal to have a twolevel clustering model that can group event mentions within a document and further group them across documents.


The posterior is intractable due to a combinatorial number of possible link configurations.

we approximate the posterior using Markov Chain Monte Carlo (MCMC) sampling, and specifically using a Gibbs sampler.


For example, we managed to harvest over 5 million relation instances more than PATTY, using a much smaller corpus (sin-gle sentences as opposed to full Wikipedia articles) and generating a number of distinct relations that was six times less than PATTY's.

we obtained an average number of extractions that was substantially higher than those of our OIE competitors.


The challenge in this setting is that the vocabulary of the target logical language often contains thousands of logical predicates, and there is a mismatch between the structure of the natural language and the logical language.

the space of possible semantic parses for even a short utterance grows quickly.


Second, semantic parsers explore a much larger search space than syntactic parsers, due to the high level of uncertainty when translating to logical form.

we hold a beam of parses for each chart cell, and modify learning for this setup.


All derivations in a stream g have the same child derivations, as they were constructed by one application of a semantic function f .

the difference in their scores is only due to new features created when applying f .


This gives rise to a score s B (b) = φ B (b) θ that depends on the semantic function only.

we can sort B before parsing, so that when the function f is called, we do not need to instantiate the derivations.


Our objective is to construct a maximum noncrossing dependency graph on the full vertex set.

the goal items of the deduction system are all items over the full interval OE1; n. The inference rules of the deduction system are shown in Figure 2.


In an n-SCFG, the elementary structures are rewrite rules of n − 1 target sides: where X is a non-terminal symbol, γ 1 is the source side string of terminal and non-terminal symbols, and γ 2 , ...γ n are the target side strings.

at each derivation step, one non-terminal in γ 1 is chosen and all the corresponding non-terminals with the same index in {γ 2 , ..., γ n } are rewritten using a single rule.


While all evaluation up to this point has used language models to disambiguate paraphrases, we can assume that human users will be even better at judging whether or not a paraphrase makes sense.

we perform an additional evaluation in which human annotators evaluate the paraphrases generated from the systems.


We use the Geoquery n-gram model for the language model used during decoding and NNLM language model to finally rerank the nbest list.

of experiments, even though this system obtained a respectable BLEU score of 57.5, the parsing accuracies were much lower than the direct keyword-MR system at 64.8 F-measure.


An edge was introduced for every word pair sharing the same word cluster and a feature for the cluster is fired.

there are 256 possible cluster features on an edge, though in our case only a single one can fire.


However, the seed lexicons used in §4.1 have been artifically constructed from aggregating attributes of word types over the treebank.

it word exchange cluster * lowercase(word) capitalization {1,2,3}-g suffix * digit {1,2,3}-g prefix * punctuation Table 6: Features used to train the morphological tagger on the universal dependency treebank.


More interestingly, it can be seen that for predicting VFORM:GER i.e, continuous verb forms, the lowest weighted features are those morphological transformations that substitute "ing" with something else.

if there exists an edge between the words studying and study, containing the feature: suffix:ing:{null}, the model would correctly predict that studying is VFORM:GER as study is not so and the negative feature weight can flip the label values.


We are currently using the binary relevance method which trains a binary classifier for every attribute independently (Godbole and Sarawagi, 2004;Zhang and Zhou, 2005) with paradigm projection as a post-processing step ( §3.3).

we are accounting for attribute correlations only at the end.


Finally, the internal memory state, m t and new output state h t , of the LSTM at t are computed as where indicates elementwise vector multiplication and φ is, as before, some non-linear function (we use tanh).

g i determines to what extent the new input word is considered at each time step, g f determines to what extent the existing state of the internal memory is retained or forgotten in computing the new internal memory, and g o determines how much this memory is considered when computing the output state at t. The sentence-final memory state of the LSTM, m N , a 'summary' of all the information in the sentence, is then projected via an extra non-linear projection (parameterised by a further weight matrix) to a target embedding space.


While the median rank for OneLook on this evaluation is lower, the NLMs retrieve the correct answer in the top ten candidates approximately as frequently, within the top 100 candidates more frequently and with lower variance in ranking over the test set.

nLMs seem to generalise more 'consistenly' than OneLook on this dataset, in that they generally assign a reasonably high ranking to the correct word.


However, inspecting the WordNik training data, there is no mention of cold or anything to do with climate in the definitions of Eskimo, Scandinavian, Scandinavia etc.

the embedding models must have learned that coldness is a characteristic of Scandinavia, Siberia, Russia, relates to Eskimos etc.


With the exception of AMBRA, all other participating systems used external resources (such as Wikipedia and Google n-grams); it is thus fair to assume they had access to at least as much training data as our SCAN model.

the gap in performance can not solely be attributed to a difference in the size of the training data.


The two groups were then asked a set of questions on how to tackle crime in the city.

while the first group tended to opt for preventive measures (e.g.


LDA also assumes that the distribution of each topic is fixed across all documents in a corpus.

a topic associated with, e.g., war will have the same distribution over the lexicon regardless of whether the document was taken from a pro-war editorial or an anti-war speech.


Similarly, it can be shown that the marginal distribution of i ) for ∈ {a, b}.

the integrand above is proportional to the beta-binomial distribution with number of trials N a k + N b k , successes x + y, and parameters β o and ( Then expression (1) above becomes: We cannot observe N a kb , N b kn , N a k and N b k explicitly, but we can estimate them by obtaining posterior samples from our Gibbs sampler.


A speaker's level of formality can reveal information about their familiarity with a person, opinions of a topic, and goals for an interaction (Hovy, 1987;Endrass et al., 2011).

the ability to recognize formality is an integral part of dialogue systems (Mairesse, 2008;Mairesse and Walker, 2011;Battaglino and Bickmore, 2015), sociolinguistic analyses (Danescu-Niculescu-Mizil et al., 2012;Justo et al., 2014;Krishnan and Eisenstein, 2015), human-computer interaction (Johnson et al., 2005;Khosmood and Walker, 2010), summarization (Sidhaye and Cheung, 2015), and automatic writing assessment (Felice and Deane, 2012).


In Samala consonant harmony, the tier is sibilants (Rose and Walker, 2004), whereas in Koorete, the tier is sibilants and {b,r,g,d} (McMullin and Hansson, forthcoming).

it is of interest to understand how both the tier and permissible tier 2-factors for TSL 2 grammars might be learned efficiently.


Continuing with Example 3, a would not satisfy condition (b) given I.

given that i = 0, in order for a to satisfy condition (b), for all get tier would infer that a is an exclusive blocker.


On other hand, condition (b) requires one search through P for every path element p ∈ P in the worst case, which is O(n 4 ).

the time complexity for get tier is O(|Σ|(n 2 + n 4 )) = O(n 4 ) since |Σ| is a constant.


||D|| is bounded by O(|Σ| 3 ) which, since |Σ| is a constant, is effectively O(1).


By hypothesis H i = H i , and so there is always guaranteed to be some τ 1 , X , τ 2 ∈ P T i where X ⊆ H i .

condition (b) will always be satisfied for σ i .


We now know that both Cases (1) and (2) are true; thus, assuming the RH, H i+1 = H i+1 .


In the setting addressed here, the subdomains are not known in advance and we have to consider them latent in the training data.

we introduce a random variable z ∈ {1, .


(2007) dataset has been the benchmark for most recent work on the two tasks as well as our own.

we situate our description in the specific context of this dataset.


Since in our case (in difference from the standard Wikification) a mention may refer to multiple concepts, using the current top ranked concept candidate from other mentions (Ratinov et al., 2011) may lose some useful information.

we develop an approach that is similar to Cucerzan (2007).


However, unlike the biomedical ontologies which have many common attributes and relatively uniform structure, different verb sense inventories vary in format and content: some resources have descriptions or example sentences of the senses, but others only have the names of semantic roles; some have relations between senses but some do not.

the question of what features would be useful in this case could be very different from those proposed in this paper and would require additional research.


We compare the performance of existing metrics over different gold-standard annotations, and show that automatic evaluation with our new annotation scheme has very strong correlation with expert rankings (ρ = 0.82).

we advocate for a fundamental and necessary shift in the goal of GEC, from correcting small, labeled error types, to producing text that has native fluency.


Additionally, it would be difficult, if possible, to reliably calculate IAA without coded alignments between the new and original sentences.

we look at two alternate measures: the percent of sentences to which different annotators made the same correction(s) and the sTER between two annotators' corrections, reported in Table 4.


The threshold t=0.07 together with a discriminating filter of f =0.07 result in the best precision, and t=0.07 in the best F1 score.

we perform extrinsic evaluation in a VSD task on S-test with WaS-de (t=0.07) and on the combinations U-WaS-de (union with S-train) and B-WaS-de (backoff-variant).


Methods for Named Entity Recognition and Disambiguation (NERD) perform NER and NED in two separate stages.

nED may be penalized with respect to precision by nER false positives, and suffers in recall from nER false negatives.


Figure 2 shows an example of such a tree model.

while the linear-chain model adds factors between labels of adjacent tokens only based on their positions in the sentence, the tree model adds factors based on the dependency parse tree to enhance the coherence of labels across tokens.


In addition, our graphical models often consider only a specific subset of candidate labels as assignments to the output variables y i ∈ Y.

we formulate the feature-generation process as a set of feature functions that depend on both (per-factor subsets of) X and Y.


For example, the two entities David Beckham and Manchester United are highly coherent as they share many tokens in their contexts, such as "champions", "league", "premier", "cup", etc.


All CRFs have the same features, but differ in their factors.

some features are not effective for the linear model and the tree model.


In practice, we are not interested in predicting x, as x is observable, but rather in inducing an appropriate latent representation (i.e., r).

it is crucial to design the model in such a way that a good r (the one predictive of x) indeed encodes relations rather than some other form of abstraction.


In contrast to the previous model, it does not model the interaction between arguments: it is easy to see that p(e 1 |e 2 , r, θ) for this model (expression (2)) does not depend on e 2 (i.e., on u e 2 and c 2r ).

such a decoder would be more similar to generative models of relations which typically assume that arguments are conditionally independent (Yao et al., 2011).


Ω am corresponding to anchors a h ∈ A(h) which can be convexly combined to realize all Given O, we can recover the A-HMM parameters as follows.

first, we recover the stationary state distributionπ as: The emission parameters O are given by Bayes' theorem: Using the fact that the emission probabilities are position-independent, we see that the initial state distribution π satisfies u 1 = Oπ: • Recover the parameters: Output: A-HMM parameters (π, T, O) π can be recovered as π = O + u 1 where O + is the Moore-Penrose pseudoinverse of O.


(2013)'s framework to learn various models including HMMs, but they address a more general problem.

their algorithm draws from Anandkumar et al.


Due to the variety of word choices and inherent ambiguities in natural language, bag-of-word approaches with simple surface-form word matching tend to produce brittle results with poor prediction accuracy (Bilotti et al., 2007).

researchers put more emphasis on exploiting syntactic and semantic structure.


As we explained in the final paragraph of Section 3, we perform all-ap pooling for all blocks, not just for b k .

we get one sentence representation each for s 0 and s 1 for each block b 1 , .


They use the intersection of languages covered in the Google dependency treebanks project and those contained in the Europarl corpus.

they only consider closely related Indo-European languages for which high-quality tokenization can be obtained with simple heuristics.


We only want to assume what we can in fact assume for truly low-resource languages.

for the target languages, we do not assume the avail-ability of any labeled data, tag dictionaries, typological information, etc.


The intuitive explanation often given -that smaller models generalize better-turns out to be fallacious, since the training method for creating embeddings makes no reference to analogy solving.

there is no a priori reason why lowdimensional model parameters (i.e., lower model capacity) should lead to better performance in analogy solving, just as there is no reason they are better at some other unrelated task like predicting the weather.


Assume a simplified version of our model, where a small window of k words is generated as follows: sample c ∼ C, where C is a uniformly random unit vector, then sample Furthermore, assume Z c = Z for any c. Lemma 3.1.

in the simplified version of our model, the Maximum-a-Posteriori (MAP) estimate of c using the MAP estimate of c t gives essentially the same expression as CBOW apart from the rescaling, which is often omitted due to computational efficiency in empirical works.


For instance, if one uses our PMI objective, then the weighted average of the termwise error in (2.5) is 17%, and the expression in (4.1) above contains six inner products.

in principle the approximation error could lead to a failure of the method and the emergence of linear relationship, but it does not.


Similar results are also obtained for word embedings by GloVe and word2vec.

the first singular vector can be taken as the direction associated with this relation, while the other components are like random noise, in line with our model.


Briefly, the reason this holds is as follows: in the regime when d is small ( √ d = o(log 2 n)), any word vector v w and discourse c satisfies that , and since E [1F ] = exp(−Ω(log 2 n)), the claim follows directly; In the regime when d is large ( √ d = Ω(log 2 n)), we can use concentration inequalities to show that except with a small probability exp(−Ω(d)) = exp(−Ω(log 2 n)), a uniform sample from the sphere behaves equivalently to sampling all of the coordinates from a standard Gaussian distribution with mean 0 and variance 1 d , in which case the claim is not too difficult to show using Gaussian tail bounds.


We use the pairwise ranking optimization (PRO) algorithm (Hopkins and May, 2011) implemented in the open-source Joshua toolkit (Ganitkevitch et al., 2012a;Post et al., 2013) for tuning.

specifically, we train the system to distinguish a good candidate output j from a bad candidate j , measured by an objective function o (section 3.1), for an input sentence i: the optimization reduces to a binary classification problem.


Unlike LSA based methods, CCA based methods are scale invariant and can capture multiview information such as the left and right contexts of the words.

the eigenword embeddings of Dhillon et al.


Previous work has shown that sacrificing lexical features amounts to a substantial decrease in the performance of a dependency parser (Cohen et al., 2011;Täckström et al., 2012;Tiedemann, 2015;Guo et al., 2015).

we extend the token representation in MALOPA by concatenating learned embeddings of multilingual word clusters, and pretrained multilingual embeddings of word types.


16 Some WALS features are not annotated for all languages.

we use the average value of all languages in the same genus.


Our main goal in this work is not to provide top parsing accuracies, but rather to evaluate the ability of the proposed compositional architecture to learn and capture the structural cues that are needed for accurate parsing.

we are most interested in the random initialization setup: what can the network learn from the training corpus alone, without relying on external resources.


could make a difference in someone's life.

far, quantitative evidence for effective conversation strategies has been scarce, since most studies on counseling have been limited to very small sample sizes and qualitative observations (e.g., Labov and Fanshel, (1977); Haberstroh et al., (2007)).


The primary goal of this paper is to study strategies that lead to conversations with positive outcomes.

we require a ground-truth notion of conversation quality.


While the counselor-level analysis is of primary concern, we will also differentiate between counselor behavior in "positive" versus "negative" conversations (i.e., those that will eventually make the texter feel better vs. not).

in the remainder of the paper we differentiate between more vs. less successful counselors and positive vs. negative conver-  sations.


We compare success rates of counselors across different issues and find that successful counselors have a higher fraction of positive conversations across all issues and that less successful counselors typically do not excel at a particular issue.

we conclude that counseling quality is a general trait or skill and supporting that the split into more and less successful counselors is meaningful.


This suggests that more successful counselors act with more control over the conversa-tion, which could explain why they are quicker to make it through the initial conversation stages.

far, we have studied conversation dynamics and their relation to conversation success from the counselor perspective.


Using compressed bitvector representations and Huffman codes to define the alphabet partitioning, the space usage of the wavelet tree and associated RANK structures of the BWT is bound by H k (T)n + o(n log σ) bits (Grossi et al., 2003).

the space usage is proportional to the order k entropy (H k (T)) of the text.


Even 7,000 training examplesone for each language or dialect on Earth-would be a small dataset by contemporary standards.

it is challenging to develop systems that will discover structure in new languages in the same way that an image segmentation method, for example, will discover structure in new images.


While datasets have grown in recent years, so have models: e.g., neural networks have many parameters to train.

it is still common to create synthetic training examples-often by adding noise to real inputs or otherwise transforming them in ways that are expected to preserve their labels.


First, we retain the immediate dominance structure and lexical items of the substrate trees, altering only their linear precedence relations.

each sentence remains topically coherent; nouns continue to be distinguished by case according to their role in the clause structure; wh-words continue to ccommand gaps; different verbs (e.g., transitive vs. intransitive) continue to be associated with different subcategorization frames; and so on.


For this feature and the next one, we extend the summation in (1) to allow 0 ≤ i < j ≤ n + 1, taking t 0 = r 0 = BOS ("beginning of sequence") and t n+1 = r n+1 = EOS ("end of sequence").

a bigram feature such as A.DET.EOS would fire on DET when it falls at the end of the sequence.


Each word in a synthetic sentence is annotated with its original position in the substrate sentence.

all synthetic treebanks derived from the same substrate treebank are node-to-node aligned to the substrate treebank and hence to one another.


Some may be unnatural, but others are similar to other real languages, including held-out dev languages.

dutch (nl) and Estonian (et) have close synthetic neighbors within this small sample, although they have no close real neighbors.


We also add "cheating results" where English itself is used as the substrate (left column) and/or the superstrate (solid black line at top).

the large black dot at the upper left is a supervised English parser.


This does not increase the diversity of the pool as much as when we add new real languages.

we are particularly interested in generating a wider range of synthetic languages.