Since some words occur in the vocabulary and also the input article, they will have non-zero probabilities of being newly generated as well as being copied.

the total probability of w being the next word generated in the summary, denoted by p is given by, The second term allows the framework to choose a word to copy from the input text using the attention distribution.


Language-model (LM) Bridge The LM bridge utilizes a pretrained neural language model p LM (Y ) as constraint, which motivates to propose target examples conforming to language fluency.

similar to the uniform bridge case, we can re-write the loss function to a KL-divergence: the LM bridge is also static and can be seen as an extension of the uniform bridge, where the exponentiated similarity score is re-weighted by a pretrained LM score, and renormalized: where Z is the partition function.


Since the KL constraint is a moving target when the generator is updated, the coaching bridge should not remain static.

we perform iterative optimization to train the bridge and the generator jointly.


In coaching GBN, due to the fact that the input of abstractive summarization X contains more information than the summary target Y ⇤ , directly training the bridge p ⌘ (Y |Y ⇤ ) to understand the generator p ✓ (Y |X) is infeasible.

we re-design the coaching bridge to receive both source and target input X, Y and we enlarge its vocabulary size to 88,883 to encompass more information about the source side.


As shown in Figure 2, long sentences have deep dependency trees, which have long distances from root node to words at leaf nodes.

improving compression performance for sentences with such deep dependency trees can help to compress longer sentences.


In this example, the compressed sentence should keep the phrase "from Kyrgyz Republic and Tajikistan".

the compressed sentence must also keep the dependency chain "import", "resolution" and "signed" because the phrase is a child of this chain.


We explain the structure and function of each layer of NSEM in more details below: Input Layer (d, h): the input layer aims to represent the main body d and title h with distributed embedding ⃗ d and ⃗ h. Let the subscript pos denotes the relevant title h pos (positive example) and subscript neg denotes an irrelevant title h neg (negative example).

for news articles, we pay more attention to the key elements of events such as location l, person p , organization o and keywords w. an event is described by a quadruple ⟨l, p, o, w⟩.


Hence, when constructing storyline for the main body d in current time period t, we use the model in previous time period t − 1 and predict the storyline distribution p t−1 (s d ).

then we measure current storyline distribution p t (s d ) and predicted distribution p t−1 (s d ) by KL divergence which can be defined as below: the final objective function is to minimize: where α and β are the weights controlling the contributions of the two loss terms.


Algorithm 1 Training procedure for NSEM at the time period t Require: main bodies d; titles h; model parameter Φ t−1 at the time period t − 1 1: Initialize Φ t 2: Calculate its storyline distribution based on Φ t−1 4: end for 5: repeat 6: Calculate the storyline distribution Calculate the storyline distribution p(s h i,pos ) Sample an irrelevant title h i,neg where h i,neg ∩ h i,pos = ∅ Calculate the storyline distribution p(s h i,neg ) 12: Update model parameter Φ t end for 18: until Convergence As the number of storylines at each time period is assumed to be the same, some newly emerging storylines might be incorrectly linked with previous storylines.

post-processing is needed to filter out such erroneous linkings.


Of particular interest, the method based on submodular maximization has three advantages: (1) Many objective functions used for document summarization are known to be monotone and submodular (Lin and Bilmes, 2011;J Kurisinkel et al., 2016); examples of such functions include the coverage function, diversity reward function, and ROUGE.

the method can deliver high performance by using monotone submodular objective functions that are suitable for the given tasks.


Figure 1 (a) illustrates an example of a document tree.

given document tree T, a summary preserves the original dependency relations if it forms a subtree rooted at r in T. our aim is to find a rooted subtree of T that includes informative textual units.


Given document tree T, a summary preserves the original dependency relations if it forms a subtree rooted at r in T. Therefore, our aim is to find a rooted subtree of T that includes informative textual units.

for each v ∈ V , the length of v is denoted by v ≥ 0; for example, v is the number of words or characters in chunk v. If S ⊆ V is a subset of the textual units included in an obtained summary, its total length must be less than or equal to the given length limitation value L ≥ 0; namely, the following knapsack constraint must be satisfied: v∈S v ≤ L. The quality of summary S is evaluated by a monotone submodular function g. compressive summarization is formulated as STKP: At first glance, it may seem that the following naive greedy approach works well for this problem: Starting from root r, we sequentially add the most beneficial child to the current solution until the knapsack constraint is violated.


We observed that the maximum number, λ, of leaves in a sentence tree was about 22 on average, which leads to a 2.2%-approximation guarantee of our algorithm.

our method empirically performs much better than the theoretical guarantee; this is often the case with the greedy algorithm for submodular maximization problems, in particular when the problems have complex constraints.


Our training algorithm allows to explore the space of possible summaries, making our model more robust to unseen data.

reinforcement learning helps extractive summarization in two ways: (a) it directly optimizes the evaluation metric instead of maximizing the likelihood of the ground-truth labels and (b) it makes our model better at discriminating among sentences; a sentence is ranked high for selection if it often occurs in high scoring summaries.


The relational summarization problem is somewhat different: we begin with a pair of query terms, (t 1 ) and (t 2 ), and we wish to learn the nature of their relationship.

any statement which coherently describes any relationship between the two query terms is potentially of interest, even if it does not match prior expectations of what constitutes a relation.


In sentence compression corpora, gold standard compressions must be acceptable sentences.

compressions from the dataset which happen to begin and end with a named entity, 7 once extracted from source sentences, can serve as positive examples of acceptable relation statements.


This indicates that using ROUGE recall as regressand in a sentences regression framework is not very promising.

the results are a first confirmation of the previously described intuition that predicting precision scores can be better than predicting recall scores.


Yet this loss works in an opposite di-rection to the original objective of VAEs that minimizes the minimum description length.

it may be in danger of forcibly moving the information that is better modeled in the decoder to the latent variable.


However, conditioning on the context makes the range of training target x t very sparse; even in a large-scale conversation corpus such as Ubuntu Dialog (Lowe et al., 2015), there exist one or very few target utterances per context.

hierarchical RNNs, given their autoregressive power, can easily overfit to training data without using the latent variable.


However, the expressive power of hierarchical RNN decoders makes the model still prone to ignore latent variables z conv and z utt t .

our second idea is to apply an utterance drop regularization to effectively regularize the hierarchical RNNs, in order to facilitate the use of latent variables.


The resulting customer frustration may not surface in easily detectable ways such as the appearance of all caps, shouting to a speech recognizer, or the use of profanity or extreme punctuation.

the chatbot will continue as if the conversation is proceeding well, usually leading to conversational breakdown.


We followed the methodology presented in (Jovita et al., 2015) claiming that the best answer given by the system has the highest similarity value between the customer turn and the agent answer.

if the similarity was < 0.8 we considered this as an erroneous detection.


These studies employed bag-of-words representations which do not capture term similarity and cannot distinguish word importance and relationships between words in a message.

better representations of messages and their relationships are needed.


the collection of linked paths, these hidden variables can reveal the underlying semantics between these two entities.

the link classification problem can be decomposed into two modules -acquire underlying paths (Path Finder) and infer relation from la-tent representation (Path Reasoner).


17: Sort S i and find positive rank ra + M AP ← 1 1+ra + 20: end procedure which indicates "no-relation" between two entities.

we manage to decompose the data ) into a series of tuples (e q , r q , e i ), where r q = r q for positive samples and r q = n/a for negative samples.


pairs, meanwhile eliminating many noisy links.

we first mainly summarize three different types and investigate their changing curve under different beam size conditions: 1.


However, dense annotation is time-consuming and unstable human judgments on "salient" pairs are not improved at all.

a high proportion of "vague" or "nolink" pairs appears in these dense corpora such as TimeBank-Dense .


The reason is that most events reported in news articles usually occur in precisely the same day as DCT.

the TORDER Event-DCT is benefited from the low proportion of vague relations, which sharply outperforms the TLINK Event-DCT by 16.3% overall exact match.


Pseudo Entities are words and phrases that frequently occur in Wikipedia, and co-occur with mentions of KG entities in the web corpus.

eLDeN uses a web corpus to find pseudo entities and refines the cooccurrences with Pointwise Mutual Information (PMI) (Church and Hanks, 1989) measure.


However, our systems produce, for example, negative reviews about products that are mostly discussed positively in the training set.

the classifier often gives unreliable predictions on system outputs.


However, we find that empirically it is often easy to fool the discriminator without actually removing the attribute information.

we explicitly separate attribute and content by taking advantage of the prior knowledge that the attribute is localized to parts of the sentence.


The interpretation of expressed sentiment is an inherently subjective exercise, the gold-standard of which is the sentiment perceived by other humans.

it is crucial to better understand sentiment annotation itself to inform future classifier design.


However, for word embeddings (the default input for word emotion induction) there does not seem to be any meaningful order of their components.

these more sophisticated DL methods are, for the time being, not applicable for the study at hand.


Since EN is a subset of EN+, we can compare these performance figures against our own results on the EN data set where we achieved r = .918, .730 and .825, respectively.

our proposed method did actually outperform human reliability for Dominance and is competitive for Valence and Arousal, as well.


However, the AffectEvent dataset contains a very large set of events that were extracted from the same blog corpus, but not manually labeled with affective polarity.

we explored two weakly supervised learning methods to exploit this large set of unlabeled events.


For each human need category, we select the unlabeled event that has been assigned to that category with the highest confidence.

each category will have one additional labeled event at each iteration.


For each hu-man need category, we then select the event that has been assigned to the category with the highest confidence value as a new instance to label.

each category will receive two additional labeled events at each iteration, one from the event expression classifier and another one from the event context classifier.


As arguments are highly contextualized, warrants are usually presupposed and left implicit.

the comprehension does not only require language understanding and logic skills, but also depends on common sense.


Paradoxically, this effect works in the models' favour in the antonym swapping case (I TA1 ) because all the gold-labels remain as contradiction.

ignoring the transformation will avoid any loss in performance.


Notably, all perform poorly on I TH instances in which the gold label is changed, with ESIM and CE performing below the level of chance.

on these instances, the models tend to predict the label of the original unaltered training instance and inference in this case is similar to nearest-neighbour prediction.


This could help eliminate any sharp peak in information density, as illustrated in Figure 1 (bottom left).

if the speaker's goal is to transfer information as smoothly as possible, the less predictable the upcoming clause, the more inclined the speaker would be to produce the function word that.


This framework was selected rather than newer or more task-specific frameworks as it is the same underlying memory model of ACT-R, which has been used to explain a wide variety of language phenomena (e.g., Vasishth and Lewis, 2004;Reitter et al., 2011), but also has been used to explain everything from decision-making (e.g., Marewski and Mehlhorn, 2011) to visual attention in graphical user interfaces (e.g., Byrne et al., 1999).

by using this model, our work naturally builds upon a large body of work, using the same mechanisms to explain a variety of tasks.


(2006) showed that for large sample sizes of language, Pointwise-Mutual Information is nearly identical.

we use Semilar's PMI database computed on the Wikipedia corpus (Rus et al., 2013;Church and Hanks, 1990).


As words themselves naturally can require different amounts of time to speak, we instead use the adjusted delay which is computed by taking the average of all of the durations of that word (as found in Switchboard) and subtracting it from the given duration.

the adjusted delay could be a positive or a negative number, representing slowdowns and speedups, respectively.


By this logic, if grammatical encoding proceeds in a purely incremental fashion, then lexical retrieval does not, and vice versa.

it is reasonable to believe that the grading of incrementality found in natural human discourse is not only variable from situation to situation, but it may be variable amongst competing processes for any given situation.


What is in question is whether we really process local syntax using structured, memoryhungry models (i.e., with syntax trees); we note that in natural language processing, skip-grams can capture local, non-incremental relationships among words.

the relationship between working memory, syntax trees, and skip-grams appears to be of continued interest.


Furthermore, the GECO corpus has a slightly different eye movement feature set than what we have for the Dundee corpus.

we treat the two eye movement corpora as two individual modalities in order to assess their individual contributions.


One issue still remains that characters in this dataset are often mentioned by several aliases (e.g., nicknames, honorifics) such that it is not trivial to cluster mentions implying the same character using simple string matching.

an entity dictionary is created for each character whose key is the name of the character and the value is a list of aliases for the character, manually inspected throughout the entire show.


The attention score a q j,i on memory m q i ∈ M q for a RNN hidden state t j in the decoder part is computed as follows: where W q 1 and W q 2 are weight vectors associated with m q i and t j , respectively.

the attention vector on the query sequences is given by: Similarly, the attention scores and attention vectors on the other three memories can be derived by replacing q with r, c, and h in Equations 14, 15.


For instance, there are many file directories with computer names in Ubuntu dialog corpus.

qualified domain-specific datasets are desperately required to evaluate NRG models with different architectures reasonably.


The quality of the data in Reddit movie category has been discussed by Stoddard (2015) and Jamnik and Lane (2017), who point out that the popularity is a good indication of relative quality and the movie category is one of the most popular boards in Reddit.

the data in Reddit movie category is originally high-quality.


Currently, few studies focus on the evaluation based on the group of references, which is more meaningful and reasonable for NRG models.

we proposed three metrics: MaxBLEU, Mean Diversity Score, and Probabilistic Diversity Score, to quantify both the relevance and diversity of the generated responses.


Yet, these candidates only capture n-gram level similarity with the probably diverged semantics.

in the second stage, we utilize paragraph vector algorithm (a.k.a Doc2vec 6 ) (Le and Mikolov, 2014) to resort the selected similar queries in the semantic space and only queries of similarity score higher than a certain threshold (i.e., 0.9) are reserved.


9: end for 10: return k MDS(k), k PDS(k) we propose two evaluative metrics based on the MaxBLEU metric for diversity measurement: a) Mean Diversity Score (MDS) and b) Probabilistic Diversity Score (PDS).


Proprietary commercial software tools often termed as computerassisted coding (CAC) systems are already in use in many healthcare facilities and were found to be helpful in increasing medical coder productivity (Dougherty et al., 2013).

progress in automated EMR coding methods is expected to directly impact real world operations.


Additionally, if coders become more efficient, hospitals may hire fewer coders to reduce their operating costs.

automated coding methods are expected to help with expedited coding, cost savings, and error control.


Next, for every step of the training process (for every mini-batch M ), we search T \ M to find the e nearest neighbors (using Euclidean distance) per instance to form our support set S. Likewise, we add e random examples from T \ M to the support set.

our support set S contains up to |M |e + e instances.


We would expect medical coders to use our system to become more efficient in coding EMRs.

we would rank the labels based on model confidence and medical coders would choose the correct labels from the top few.


Embeddings represent similarities between concepts and stocks if they co-exist literally in a context window during embedding training.

irrelevant (relevant) stocks are less (more) similar to the concept c, since they infrequently (frequently) co-occur, which alleviates the problem brought by imbalanced datasets in that irrelevant stocks can be spotted at ease.


Formally, a MDP is a list [Z, A, T, R], where Z = {z} is a set of states, A = {a} is a set of actions, T (z, a) is a transition function, which determines the next state z = T (z, a) after performing action a on z, and R is a reward function.

we describe each in detail below: States: Each state z is a list of lists:  at each state, we have n candidate concepts v 1 ...v n .


Words are represented in the form of binarized vectors.

the consumption of memory space is significantly reduced.


In addition, for binarization, we do not need to determine the position of each unique values in advance.

binarization is more flexible than quantization.


To find the relevance of each memory m t λ 's context with q i , a match between both is computed.

we do this by taking an inner product as follows: Here, sof tmax(x i ) = e x i ∑ j e x j and attention vector p λ = {p t λ } is a probability distribution over the input memories  the output representation o λ contains weighted contextual summary accumulated from the memory.


Contrasting to CMN, the model generates the memory representations for each historical utterance using an embedding matrix B as used in equation 7, without sequential modeling.

for utterance u i , both memories are created as M λ using {m t λ = B.u t u t ∈ hist λ and t ∈ [1, K]} for λ ∈ {a, b}.


CMN Self : In this baseline, we use only self history for classifying emotion of utterance u i .

if u i is spoken by person P a , then only hist a is considered.


CMN N A : Single layer variant of the CMN with no attention module.

its output o λ (equation 8) is generated using a uniform probability distribution p λ , i.e., {p t λ = 1 K } K t=1 .


In other words, our model automatically learns that the convex timedecay attention is more suitable for modeling contexts from the dialogue data than the other two types.

we can conclude that in complex dialogues, the recent utterances contain majority of salient information for spoken language understanding, where the attention decay trend follows a convex curve.


It can be found that our proposed time-decay attention can effectively turn the attention to the correct contexts in order to correctly predict the dialogue act and attribute.

the proposed attention mechanisms are demonstrated to be effective for improving understanding performance in such complex humanhuman conversations.


2.2 Factors that affect duration of segments and pauses Since oral reading involves saying the text aloud, the durations of individual segments, words and phrases as well as location and duration of silent pauses are subject to constraints that have been extensively studied in literature on phonetic timing; see White (2014); Hirschberg (2002) for a review.

it has been long known that different segments have different intrinsic durations which account for a lot of variation in segmental durations (Peterson and Lehiste, 1960;Klatt, 1976;van Santen, 1992): for example, high vowels tend to be shorter than low vowels.


9 In other cases the interpretation of the feature was less straightforward.

the feature with the second highest coefficient after timing for both narrators was that which counted occurrences of members of the Weasley family.


This opens up a possibility for a sophisticated assessment of oral reading using both TTS and human benchmark to separate reading that adheres to basic timing constraints of English speech (which constitutes a demonstrably big part of fluent reading) from a more nuanced expressive reading that TTS is not currently doing, but good human readers are.

beyond assessment context, our findings can also inform work on text-to-speech synthesis for book-length texts.


The speaker may communicate with the listener in free-form naturallanguage dialogue to achieve this goal.

a model of the speaker must process representations of the colors in the context and produce an utterance to distinguish the target color from the others.


However, this is an unfair comparison: the monolingual model's high perplexity is dominated by low probabilities assigned to rare tokens in the opposite-language data that it did not see.

perplexity ceases to be a measure of language modeling ability and assumes the role of a proxy for the out-of-vocabulary rate.


"hot dog" is matched to the dog category).


For BL, neither frequency nor our concreteness scores are significantly correlated with USF.

in three of our four datasets, our measure tends to predict human concreteness judgments better than frequency.


Then they assigned the face based on candidate names from the time-stamped script.

these studies used speaker recognition as an essential step to construct cast-specific face classifiers.


These mentions allow us to obtain cues about the speaker name and the absence or the presence of the mentioned character in the surrounding subtitles.

they affect the probability that the mentioned character is the speaker or not.


goal is to assign names to speakers, and not necessarily solve this coreference problem, we consider the assignment of the subtitle segments to any of the speaker's aliases to be correct.

during the evaluation, we map all the characters' aliases from our model's output to the names in the ground truth annotations.


This can be seen as a type of domain transfer for the biLM.

in most cases we used a fine-tuned biLM in the downstream task.


We define • the hidden state vector h s,t ∈ Q N given by where h s,−1 = h −1 and we use standard matrix product and point-wise vector addition, Finally, the RNN R computes the weighted language R : Σ * → R, which is given for every in- In other words, each component h s,t (n) of the hidden state vector is the ReLU activation applied to a linear combination of all the components of the previous hidden state vector h s,t−1 together with a summand W st that depends on the t-th input letter s t .

we often specify h s,t (n) as linear combination instead of specifying the matrix W and the vectors W a .


In addition, we have h s,t (2) = t + 1 and h s,t (3) = σ 3(t − T − 1) .

hence we have which shows that the probability otherwise of predicting $ increases over time and eventually (for t 3T ) far outweighs the probability of predicting a. in this case the RNN is consistent (see Lemma 16 in the appendix).


If M does not halt, then R(a n ) ≤ 1 1+e 2(n+1) ≤ 1 1+e 2 < 0.12 for all n ∈ N. On the other hand, if M halts after T steps, then using Lemma 14 in the appendix.

a string with weight above 0.12 exists if and only if M halts, so the best string problem is also undecidable.


Furthermore, an argument can be defined in a variety of ways depending on the problem being solved.

we focus on the specific domain of opinionated texts such as those found in online reviews.


Pointer mechanism: Due to the limitation of the vocabulary size, some keywords may not be in the target vocabulary, which will certainly lead to a lack of them in the final summary.

we take the key information representation k, the context vector c t and the decoder hidden state s t as inputs to calculate a soft switch p sw , which is used to choose between generating a word from the target vocabulary or copying a word from the input text: where w T k , w T c , w T s and b sw are parameters, σ is the sigmoid function.


We hope the predicted value of v(x, y p1 ) can be larger than v(x, y p2 ) if AvgCos(x, y p1 ) > AvgCos(x, y p2 ).

the loss function of the prediction-guide network is as follows: where AvgCos(x, y p1 ) > AvgCos(x, y p2 ).


In our proposed framework, an input of a decoder contains not only the output from the last step but one from the last decoding layer.

we design two types of teacher forcing techniques -inner-layer and interlayer.


Since SARI depends on the differences between a system's output and the input sentence, large differences may yield very good SARI even though the output is ungrammatical.

when tuning with SARI, we ignored epochs in which the BLEU score of the output is too low, using a threshold ς.


Additionally, there is a board called joke that we could acquire large amount of Chinese humor samples.

we use some political-related words to extract political jokes from PTT and treat them as the positive samples.


And typically we make the window width of the filters the same as the embedding dimension.

the filter size varies according to a sliding window size we decide.


Current state-of-the-art systems rarely alter the source sentences enough to yield semantically unfaithful outputs (Choshen and Abend, 2018b).

their human rankings are not determined by their semantic faithfulness, rendering them unuseful for validating USIM.


In general, the human written domain knowledge functions are not expected to be perfectmost likely only examining a subset of features and not covering all cases.

we are now faced with two challenges: (1) the domain knowledge functions have limited generalization; (2) the domain knowledge functions provide a ranking, but do not provide an inference (search) procedure.


For particle verbs not covered by GermaNet, we used the existing verbs as a seed set and applied a nearest-prototype (centroid) classifier to all other BVs and PVs, with a centroid for each of the 15 classes.

we were able to assign class labels to all verbs in our dataset.


Instead, w t+i is predicted by 2c predictors according to where it appears in w t 's context.

every word in SSG should have 2c output embeddings for the 2c relative positions.


On the output side, SSSG has two "word" vectors respectively for left and right context, while DSG has one "word" vector and one "direction" vector.

the direction vector of DSG can be used to explicitly predict whether the context is on the left or right in word prediction, while SSSG doesn't.


For many applications, bilingual lexicon induction of rare and domainspecific words is of critical importance.

we design a new task to evaluate bilingual word embeddings on rare words in different domains.


Annotating all sentences would dilute the impact of the positive examples, further complicate the annotation scheme, and increase the cost of the corpus creation.

we opt to construct a balanced corpus of positive and negative examples for a specific kind of redundancy in a specific configuration.


We then create an emotion vector for each sentence by counting the presence of words of the sentence in each of the ten different types of emotions of the NRC Emotion Lexicons.

the sentence emotion vector has a dimension of 10.


Moreover, the standard deviations for all emotions are significantly different across the two categories (p < 0.001).

emotion concentration (µ) and variation (σ) for Successful books are higher than for Unsuccessful books for all emotions in the NRC lexicon.


W ad ∈ R C×d ad is a parameter and sof tmax( Loss Function: We use categorical crossentropy as the loss function which is averaged over all aspects for a sentence.

stochastic loss for sentence S is calculated as: Here, m is the number of aspects for a sentence and C is the number of sentiment categories.


These reductions can mostly be attributed to the correlations between local similarity and synchrony per feature and between the lexical measures.

the analysis again confirms a lack of correlation across features since more significant dimensionality reduction would otherwise be possible.


Entrainment is measured in various ways, even with regard to the same features.

it would be possible to continue our search using different entrainment measures on our features.


However, all our measures meaningfully and diversely capture entrainment.

it seems unlikely that alternative measures would yield fundamentally different outcomes, such as strong correlations across features.


Compounds, again, behave in the opposite way.

the less confidence there is for an MWT, the more confidence we find in its subterms.


Language is inherently sequential and a writer repeats the same event to remind the readers about the main event.

the frequent and extended repetitions of the central event facilitate to minimize the cognitive effort needed by the reader for understanding a text.


We extend the BiLSTM-CRF model by adding entity embedding channel to the embedding layer.

x i is the concatenation of word embedding, character embedding and its entity embedding, Entity embeddings are derived from a noisy gazetteer created using Wikipedia articles.


(2017b) also show that complex structure models are prone to the structure-based overfitting.

we propose the structure regularized BRCNN.


On the other hand, while encoding definitional information using deep learning techniques has been addressed in the past (Hill et al., 2015;Noraset et al., 2016), to the best of our knowledge no previous work has tackled the problem of DE by reconciling both the linguistic lessons learned in the past decades (e.g., the importance of lexico syntactic patterns or long-distance relations between definiendum and definiens) 1 and the processing potential of neural networks.

we propose to bridge this gap by learning high level features over candidate definitions via convolutional filters, and then apply recurrent neural networks to learn long term dependencies over these feature maps.


Note, however, that non-arc-decomposability in the 2-Planar parser not only comes from cycles (as in Covington) but also from situations where, due to a poor assignment of planes to already-built arcs, no possible plane assignment allows building a set of pending gold arcs.

the loss calculation technique of the Covington dynamic oracle is not directly applicable to the 2-Planar parser.


This is because the standard training procedure assumes that the reference tree is within reach of the parser, which is not the case for non-projective examples.

projective parsers cannot make any use of such samples and common practice is to filter them out, thereby wasting potentially valuable training material.


Arc-decomposability does not always hold, however, in which case there are extra costs to take into account: by definition of non-arcdecomposable systems, some arcs are incompatible (they are not unreachable, they can simply not be reached together).

at some point, adding a gold arc will imply renouncing to another gold arc, thereby inserting an error.


(2009), English MWEs exhibit lower variability than non-MWEs.

variability measures can help identify MWEs in running text.


In our incremental RSA, speaker models take both a target image and a partial caption pc.

s 0 is a neurally trained conditional distribution s t 0 (u|w, pc t ), where t is the current timestep of the unrolling and u is a character.


The max-pooling layer will choose the highest score from the sub-images as the final score.

the layer could easily handle statements about single-subimage-existence based reasoning (e.g., the 4 positively-classified examples in Fig.


Clipping the maximum distance also enables the model to generalize to sequence lengths not seen during training.


Additionally, relative position representations can be shared across sequences.

the overall self-attention space complexity increases from O(bhnd z ) to O(bhnd z + n 2 d a ).


Because our test set is large, it is difficult to evaluate some of the existing methods, which are computationally demanding.

we introduce WIKI-50, a set of 50 randomly sampled test documents from WIKI-727K.


The COND model also computes a shared representation, h s , directly from x, using a shared CNN s , and the two representations are concatenated together to form input to linear softmax classification function c for predicting class label y.

far, the approach resembles Daumé (c) Baseline CNN model with adversarial loss.


Furthermore, this performance difference essentially vanishes when we remove the inflectional markings.

1 in highly inflected languages, either the utterances have more content or the models are worse.


To the best of our knowledge, this disambiguation task is novel.

we resort to comparing three variants of our model in lieu of a previously published baseline.


We used perplexity on development data to jointly choose 2 Our vocabulary and parameter set are determined from the lexicon.

we create a regularized parameter ω , yielding a smoothed estimate p( ), even if the training count c( ) = 0. the learning rate, the initial random θ (from among several random restarts), the regularization coefficient λ ∈ {10 −1 , 10 −2 , 10 −3 , 10 −4 } and the neural network architecture.


We hypothesize that the main determining factor in this behavior is the relatively small size of the data and its distribution, which does not require using long-range context in most examples.

we leverage a strong language model that was pre-trained on large corpora as a fixed encoder which supplies additional contextualized token representations.


Most questions in the dataset have answer spans embedded in sentences that are syntactically similar to the question.

during training, the model is rarely punished for answering questions based on syntactic similarity, and learns it as a reliable approach to Q&A.


The five categories cover a majority of entities in the dataset.

we do not use other entity types provided by Freebase.


To define a "community," volunteers manually selected a set of users that fit with a theme that they had familiarity with.

the specific 16 communities were determined based on themes of interest to the authors and their friends and colleagues, where we could be reasonably confident about membership decisions.


However, such an approach based on close-world assumption is not always practical, especially for domains with limited reviews or the "out-of-vocabulary" problem.

we propose a convolutional neural network (CNN) based model which leverages both word-level and character-based representations.


On the other hand, while generating a word by the decoder, there can be a preference of focusing on a record type say, temperature, over some other type say, windSpeed.

dynamic attention is used across records.


One possibility is that simply encouraging workers to spend more time writing their summaries improved performance, but fitting a linear model we find the correlation coefficient between time and F 1 is −0.06, indicating no linear correlation between time and accuracy across conditions.

we believe that the significant accuracy improvement observed in the Pin-Refine condition is the result of active priming with aggregated TIUs.


But "Trump won" can only occur in the specific situation where Trump won something, which is unlikely to be described in a summary more than once.

we simply get rid of the frequency count.


In this case, the NLP pipeline does not identify Barney Cousins as an animate character, possibly due to his unusual name.

the method received as input a summary in which only three characters were identified for the story on the right.


Given that emojis may also mislead humans (Miller et al., 2017), the automated prediction of emojis may help to achieve better language understanding.

by modeling the semantics of emojis, we can improve highly-subjective tasks like sentiment analysis, emotion recognition and irony detection (Felbo et al., 2017).


Computing s c (i, j) only requires manipulating matrices of size M × |g| and M × M .

we instead propose to use s c (i, j) to compute a rough sketch of likely antecedents.


As shown in Figure 2, the coarse-to-fine model is quite insensitive to more aggressive pruning.

our final model considers only 50 antecedents per span.


Despite using far less computation, it outperforms the baseline because the coarse scores s c (i, j) can be computed for all antecedents, enabling the model to potentially predict a coreference link between any two spans in the document.

we observe a much higher recall when adopting the coarse-to-fine approach.


Each post describes how the character that is assumed by that specific player responds to the game situation.


The objective of a task oriented SDS is to fulfill the users goal, but as the goal is not observable for the SDS, the SDS needs to gather enough information to correctly fulfill it.

in each turn, the DM can decompose its decision in two steps: first, decide between taking an action in order to gather information about the user goal (information gathering actions) or taking an action to fulfill the user goal or a part of it (information providing actions) and second, select a (primitive) action to execute from the previously selected subset.


The differences between the slots (size, value distribution...) are accounted by the feature function φ s (b).

π m (b m ) is defined as: If π m (b m ) = a m i , the sub-policy run is π i : Else, if π m (b m ) = a m d , π d is selected.


2 Note that at test time, most of the h tokens will have been seen before in the training data (due to Zipf's law), and for these tokens it is very difficult to beat a baseline that normalizes each h to the most common m seen for it in training.

3 in practice, normalization systems should typically only be applied to unseen tokens.


CE forgets word identities and uses common parameters for all words in a cluster.

it is expected that it should perform best only when all models are restricted to be extremely small.


Moreover, audio recordings are often sub-optimal, using digital recorders on a desk, which is the case of the data used in this study.

the ability to detect cognitive conditions must accommodate the presence of noisy data, of which we sought to evaluate.


subjects consistently spoke like other subjects, and testers consistently spoke like other testers).

with the availability of highly accurate transcriptions of the same structure (neuropsychological exams), a highly accurate text-based speaker diarization can be conducted.


They use an unexpected pun in an image description during inference/retrieval.

they do not require to be trained with witty captions.


In following time-steps, the decoder generates new words conditioned on all previously decoded words.

the decoder attempts to generate sentences that flow well based on previously uttered words.


Similarly, for a given layer of a CNN with dimension q, we can create a matrix of embeddings for the same k concepts (R k×q ), and compute a correlation matrix in image space: I ∈ R k×k .

we have taken embedding spaces for the same set of k concepts and created new spaces of dimension k based on the correlation between concepts.


For example, man made objects tend to have more straight lines and natural objects are more curved.

it is logical that the early layers of a CNN could distinguish between some pairs of objects using only the most basic of visual features.


Meanwhile, to understand each of them requires understanding many action components (e.g., pouring a glass of milk is a complicated action sequence).

we hypothesize that it is beneficial to learn and align both the high-level (global) and low-level (local) temporal transitions of multiple modalities.


The following factors are expected to influence reading: experiment procedures; text presentation; recording hardware, software and quality; sampling rates; initial calibration and filtering, as well as human factors such as head movements and lack of attention.

separate normalization for each dataset should better preserve the signal within each corpus and for the same reason the type-aggregation was computed on the normalized feature values.


This implies that the eye-tracking feature values were averaged for each word type over all occurrences in the training data.

for instance, the sum of the features of all n occurrences of the token "island" are averaged over the number of occurrences n. for each corpus as well as for the aggregated corpora, a lexicon of lower-cased word types with their averaged eye-tracking feature values was compiled.


While the high number of unknown values is certainly a limitation of our approach, it shows at once the possibility of not requiring original gaze features at prediction time.

the trained NER models can be applied robustly on unseen data.


A possible explanation for this finding is that in English the plural form is almost always more frequent than the singular one, as the latter only marks third person singular, whereas the former is identical to the infinitive and other forms.

if the network reverts to unigram probabilities, it will tend to prefer the plural.


During training, if an instance from q i is correctly classified by the learner, it will be "promoted" to q i+1 (solid/green arrows in Figure 1), otherwise it will be "demoted" to the previous queue, q i−1 (dashed/red arrows in Figure 1).

as the learner trains through time, higher queues will accumulate instances that are easier for the learner, while lower queues will accumulate harder instances.


ner queue of size n = 3 and the following movement patterns for four individual instances that ultimately home in on q 0 (recall that correct prediction promotes an instance to a higher queue, while wrong prediction demotes it to a lower queue): Although all these instances ultimately home in on the same queue, they may have different contributions to the training of a model because there is a considerable difference in the ability of the downstream network in learning their labels.

if there is a large deviation among movement patterns of instances of the same queue, better data sampling policies could be developed, perhaps through finer-grained queue-level sampling.


The garden path should be eliminated in sentences such as (5-b), the UNREDUCED condition, where the words "who was" clarify that the verb "brought" is part of an RC, rather than the main verb of the sentence.

we quantify the garden path effect as the surprisal at the disambiguator for the REDUCED minus UNREDUCED conditions.


This is because BLEU requires exact four-gram matches only, but low four-gram accuracy may obscure a high unigram accuracy and inexact translations that partially capture the semantics of an utterance, and these can still be very useful in situations like language documentation and crisis response.

we also report word-level unigram precision and recall, taking into account stem, synonym, and paraphrase matches.


Our initial analysis of a textonly disfluency detection system shows that despite high performance of such models, there exists a big gap in the performance of text-based approaches for some types of disfluencies, such as restarts and non-trivial or long rephrases.

prosody cues, which can be indicative of interruption points, have a potential to contribute towards detection of more difficult types of disfluencies.


Despite psychological findings that rare words have long access times (Balota and Chumbley, 1985;Jescheniak and Levelt, 1994;Griffin and Bock, 1998), listeners expect interpreters to quickly understand the source words and generate accurate translations.

professional simultaneous interpreters often work in pairs (Millán and Bartrina, 2012); while one interpreter performs, the other notes certain challenging items, such as dates, lists, names, or numbers (Jones, 2002).


Computers are ideally suited to the task of recalling items given their ability to store large amounts of information, which can be accessed almost instantaneously.

there has been recent interest in developing computer-assisted interpretation (CAI; Plancqueel and Werner;Fantinuoli (2016Fantinuoli ( , 2017b) tools that have the ability to display glossary terms mentioned by a speaker, such as names, numbers, and entities, to an interpreter in a real-time setting.


For example, a human with no expertise can hardly discriminate the sound of Guitar from Banjo.

we set aside the musical territory for future exploration.


Hence, the lower-level features can be useful to capture such characteristics of natural sound, although they may lack the semantics of the higher-level features.

the joint use of these two levels of features can be mutually beneficial.


Allowing only short edits means we can apply focused analysis on the tipping point between regular and funny.

our task is to edit a headline to make it funny, where an edit is defined as the insertion of a single-word noun or verb to replace an existing entity or single-word noun or verb.


Defined in Equation 5, the final probability distribution P t is a mix of the generation distribution P gen t and the copy distribution P copy t .

the fixed vocabulary is extended by all the words appearing in the source sentence.


Without the encoderdecoder attention, the generating work gets hard.

the copying part of the model will be boosted for the correct sentences.


Previous experiments with an implementation of the vanilla parser already struggled with small subsets (we used grammars extracted from 250-1500 parse trees) of the NeGra corpus.

we omit evaluation of the vanilla parser.


In principle, one could give more weight to older languages or to languages that are known to be more conservative, but this knowledge is not available for all languages families.

we resort to using all the available data from descendent languages without distinction.


Furthermore, languages do not evolve completely randomly, but follow some linguistic universals and have to keep a balance between speakability, learnability and understandability.

languages can share grammatical features without necessarily being genetically related, either by contact or by mere chance.


Faroese (fo) is morphologically rich and that should help, however its North-Germanic relatives are morphologically much simpler.

the model does not learn to rely on morphological attributes nor on word endings for the character model as much.


the tree fragments being constructed) with data structures with linear-time sequential access (either a stack, or a stack coupled with a double-ended queue).

these systems need to perform at least n actions to construct a new constituent from two subtrees separated by n intervening subtrees.


The following auto-regressive decoding process is usually used: Parameters ✓ are typically learned by maximizing the marginal log likelihood log p However, this marginal term is intractable to compute for many decoder choices.

variational inference is considered, and the true posterior p ✓ (z|x) / p ✓ (x|z)p(z) is approximated via the variational distribution q (z|x) is (often known as the inference model or encoder), implemented via a -parameterized neural network.


A lower quality z introduces more difficulties in reconstructing x via Path A.

the model is forced to learn an easier solution to decoding: generating x via Path B only.


Also, by projectivity, a single B-gadget can have at most 1 weight 2 region connectivity arc.

the number of weight 2 arcs would be maximised by ensuring arcs exiting the B-region originate from the same B-gadget, so only one B-gadget does not have weight 2 entering arcs.


s a t encodes all past marks and depends on the full hidden state so far: However, we make s x t encode only the sequence of past input marks, ignoring all others.

t encodes only the sequence of past output marks.


By using a large ensemble, we reduce both the bias (from normalized importance sampling) and the variance of the gradient estimate; we found that smaller ensembles did not work as well.


But they produce lots of false-positives including for correct English sentences (e.g., "play next song").


Based on manual exploration, we also found that the classifiers fail to recognize complex cheating behaviours such as Example 13 in Table 1 as discussed in Section 3.

new approaches are required to accurately detect paraphrasing issues.


Compared with other QA data collection tasks, human involvement in FreebaseQA is relatively light since each person only needs to make a one-out-of-three choice instead of composing a question or sentence from scratch.

using this method, we may significantly reduce the cost of QA data collection.


Our baseline QA results have also shown that FreebaseQA is a much more difficult KBQA task than either WebQSP or Sim-pleQuestions.

freebaseQA may be an invaluable asset to the investigation of more advanced machine learning methods for factoid KBQA problems.


One problem with this approach is that the retrieved subgraph can be fairly large.

it is desirable to rank the subgraph by relevance to the mention and only preserve the most relevant facts.


Then, the score for ranking the subgraph is the weighted sum of the literal score and the semantic score where τ is a hyper-parameter whose value need to be tuned on the validation set.

for each question q, we can get the top-n ranked subgraph G n q↓ as well as the corresponding top-n ranked candidate subjects S n q↓ and relations R n q↓ .


Let I = I + I − be the index set of S, |I ⇔ n2 where from (10) to (11) we expand the sums and reorder the terms.

we obtain the well-order loss Lwell-order(Sms, Spr) = where S ms , S pr are the scores for (mention, subject), (pattern, relation) pairs for a question, I, J are the index sets for candidate subjects, relations in the ranked subgraph, +, − indicate the correct candidate and incorrect candidate, [•] + = max(•, 0), and λ > 0 is a margin term.


To see this, for a question and a fixed number of candidate facts in subgraph, |I| = |J|, the incorrect number of subjects |I − | is usually larger than the incorrect number of relations |J − |, which causes larger loss for mismatching subjects.

the model is forced to match subjects more correctly, and in turn, prune the relations with incorrect subjects and reduce the size of J − , leading to smaller loss.


multiple-choice) rather than a span among existing passages (i.e., 'answer span'); or when (3) the need arises to understand long and complex questions and reason over multiple passages, rather than simple text matching.

it is essential to incorporate commonsense knowledge or to improve retrieval capability to better capture partially related evidence (Chen et al., 2017).


Two main difficulties for these methods are that processing through all relations in a KG is not practical as the combination of these relations is nearly infinite, and that the number of candidate paths grows exponentially with the path length and quickly becomes intractable for large knowledge graphs.

current relation-chain-based methods set the maximum length of candidate paths to 1, 2 or 3.


However, they are synthetic, the relations are simply direction tokens, and the questions are just sequences of direction instructions.

in this paper, it is only used to test the model's ability of making long sequential decisions instead of understanding questions.


In many scenarios, we need to comprehend the relationships of entities across documents before answering questions.

reading comprehension tasks with multiple hops were proposed to make it available for machine to tackle such problems, e.g.


Most queries cannot be answered depending on a single document, and multi-step reasoning chains across documents are needed.

it is possible that understanding a part of paragraphs loses effectiveness for multi-hop inference, which posts a huge challenge for previous models.


However, the lack of attention and complementary features limits their performance.

our BAG model achieves the best performance under all data configurations.


For each document D and each codeword µ i , the differences x t − µ i of the vectors x t ∈ C i ∩ D and the codeword µ i are accumulated into column vectors: where D ⊂ X is the set of word embeddings in a given text document.

the final VLAWE embedding for a given document D is obtained by stacking together the d-dimensional residual vectors v i,D , where d is equal to the dimension of the word embeddings: the VLAWE document embedding is has k • d components.


Our proposed model is novel in the sense that our approach applies attention over multi-modal information of the contextual utterances in a single step.

it ensures to reveal the contributing features across multiple modalities and contextual utterances simultaneously for sentiment and emotion analysis.


For example, people are usually calm and present a neutral emotion while only in some particular situations, they express strong emotions, like anger or fear.

we need to be sensitive to the minority emotions while relieving the effect of the majority emotions.


Generally, recognizing strong emotions may provide more value than detecting the neutral emotion (Hsu and Ku, 2018).

in Friends and EmotionPush, UWA is a more favorite evaluation metric because WA is heavily compromised with the large proportion of the neutral emotion.


In our method, there are three objects W s , W t and D bi to update through the self-learning procedure.

we iterates over the following three steps: 1.


Re-normalization is applied as a final step after we have obtained W s and W t .

the normalized embeddings S and t are not aligned along the first axis, i.e., the i-th row of S does not correspond to the i-th row of t. an initial bilingual dictionary is required in order to access the correspondence between the two languages.


2 This formulation shows that both terms of this objective can be seen as a maximum of linear functions of either W s or a p .

our objective is convex with respect to either W s or a p , thus can be efficiently minimized by using the projected gradient descent algorithm.


Despite having nearly 90,000 parallel sentences, the eu-en dataset only contains 2,000 human-translated sentences; the others are translations of Wikipedia page titles and localization files.

we regard the eu-en dataset as very low-resource.


However, it is impossible to evaluate the hidden representation on all those tasks; moreover, due to relationship between tokens (Hu et al., 2016) in Y, not all partitions are reasonable.

motivated by the analysis of VC Dimension (Vapnik, 1995), we construct a sequence of nested partitions with an entailment structure: The benefit is that a spectrum of task hardness can be constructed due to the increased partition or task cardinalities.


Then we balance task diversity through a 5 times cardinality difference between tasks from the previous chosen task.

we can obtain 4 tasks with s(l) = 5, 8, 11, 20 for the Zh⇒En task and s(l) = 5, 7, 10, 21 for the En⇒De task, where l = 2, 3, 4, 5 of the 6-layer decoder.


In an evaluation on four tasks, we show that attentive mimicking outperforms previous work for both rare and medium-frequency words.

compared to previous work, attentive mimicking improves embeddings for a much larger part of the vocabulary, including the mediumfrequency range.


Given existing NLP techniques, it may not be possible to correctly identify all style-related words in a text.

there is a tradeoff between identifying more style-related words and incorrectly marking some other (content) words as style-related.


Bigrams (two-word sequences) are especially important there since they are the smallest unit formed by composing words.

it would be useful to have large semantic relatedness datasets involving bigrams.


Further, since our goal is to create a gold standard relatedness dataset, we wanted it to include term pairs across the whole range of semantic relatedness: from maximally unrelated to maximally related.

a key challenge in term-pair selection is obtaining pairs with a wide range of semantic relatedness scores, without knowing their true semantic relatedness in advance.


The output of the decoder is sent to a dense layer with sigmoid units.

at each word position, we predict multiple intents.


At the same time, characterizing the communicative role of imagery, par-ticularly photographic imagery, involves a special problem: distinguishing the content that the author specifically aimed to depict from merely incidental details that happen to appear in the scene (Stone and Stojnic, 2015).

rather than start from a taxonomy of discourse relations like that used in PDTB, we characterize the different kinds of inferential relationships involved in interpreting imagery separately.


However, utilizing the previous system utterance together with the current user utterance always at each turn may add noise.

we use a gating mechanism based on both utterances to determine the relevance of the previous system utterance in the current turn.


Such systems are prone to error accumulation from a separate SLU module, in absence of necessary dialog context required to interpret the user utterance.

later work on DST moved away from separate SLU modules and inferred the dialog state directly from user utterance and dialog history (Henderson et al., 2014b,c;Zilka and Jurcicek, 2015).


For the creation of manually annotated datasets, randomly sampling microposts from large social media platforms typically results in a too small proportion of abusive comments (Wulczyn et al., 2017;Founta et al., 2018).

more focused sampling strategies have to be applied which 0 Present affiliation: Leibniz ScienceCampus, Heidelberg/Mannheim, Germany 1 http://thelawdictionary.org/ cause biases in the resulting datasets.


Following from this, the computation of a multiclass bias subspace does not have any linear constraints, though it does come with a loss of resolution.

we can compute the principal components required to compute the "bias subspace" by simply adding an additional term for each additional bias class to each defining set.


engineers are stereotypically men, and nurses are stereotypically women.

downstream applications that use these pretrained word embeddings also reflect this bias.


Each lexicon contributes exactly one pseudocount, divided among positive, negative, and neutral, to what would otherwise be a symmetric, uniform Dirichlet distribution.

of this construction, words that appear in more lexica will have more concentrated Dirichlets.


For fair comparisons with FS-MTL, here we use the same deep Bi-LSTM CRF model for SRL as well.

the encoder outputs are the hidden vectors from deep Bi-LSTMs.


As shown, the results are consistent with our intuition.


For each set of classes, we consider a setting where directed relations are predicted with one where the direction is ignored.

in the directed prediction scenario, each emotion constitutes two classes to be predicted for both possible directions (therefore, 16, 10, and 4 labels exist).


Even human readers are sometimes not able to precisely recognize satire (Allcott and Gentzkow, 2017).

an automatic system for satire detection is both relevant for downstream applications and could help humans to better understand the characteristics of satire.


Unfortunately, we cannot analyze this potential limitation since their data set does not contain any information on the publication source 3 .

we create a new corpus in German (see Section 4.1) including this information and investigate our hypothesis on it.


However, they require massive parallel training datasets (Koehn and Knowles, 2017).

there has been extensive work on utilizing non-parallel corpora to boost the performance of SEQ2SEQ models (Sennrich et al., 2016;Gülçehre et al., 2015), mostly in neural machine translation where models that require absolutely no parallel corpora have also been pro- posed (Artetxe et al., 2018;Lample et al., 2018b).


Sampling a specific word y t from the softmax (Eq.

3) is equivalent to adding (element-wise) to the logits an independent noise sample ξ from the Gumbel distribution 1 and taking the arg max: using the GS trick, Eq.


It is also reported in (Bell et al., 2009) that content words are shorter when more frequent while function words are not so.

word length effect is strongly exists in content words but is weak in function words.


For the naive LSH-based partitioning, we can only set d, which does not directly correspond to a similarity.

we propose a second, more transparent grouping technique with this property.


This can be easily represented as a loop that traverses the input sentence from left to right, linking each word to another from the same sentence or to the dummy root.

we just need n steps to process the n words of a given sentence and build a dependency tree.


(2018) since, as in other transition-based parsers such as (Covington, 2001;Gómez-Rodríguez and Nivre, 2010), cycles can be incrementally identified in amortized constant time by keeping track of connected components using path compression and union by rank.

the left-to-right algorithm requires n steps to produce a parse.


This is necessary because two different words might be attached to the same head node and the latter can be located in the sentence either before or after w i .

the sequence s i , required by the attention score function (Eq.


where the presence of character < means that w i−1 has an incoming arc from the right, k copies of character \ mean that w i has k outgoing arcs towards the left, k copies of / mean that w i−1 has k outgoing arcs towards the right, and the presence of > means that w i has an incoming arc from the left.

each right dependency from a word i to j is encoded by a (/, >) pair in the label components x i+1 and x j , and each left dependency from j to i by a (<, \) pair in the label components x i+1 and x j .


In practice, annotators are typically instructed to annotate named entities for complete word spans only (Settles et al., 2008;Surdeanu et al., 2010).

sub-entity level annotations or O labels 2 should not be assumed to be avail-2 Why should the O labels be assumed unavailable?


We CoNLL  notice that incomplete annotation issue is very common in the industry setup.

we also consider two new datasets from industry -Taobao and Youku datasets 8 consisting of product and video titles in Chinese.


Previous work solved this task by recognizing and classifying event triggers, which is defined as the word or phrase that most clearly expresses an event occurrence.

existing approaches required both annotated triggers and event types in training data.


This work focuses on detecting events without triggers.

we remove trigger annotations from the corpus.


Actually, "died" and "fired" are the trigger words of Death and Attack events, respectively.

we argue that, although annotated triggers are unavailable, our model still can exploit trigger information for this task.


For tracking each of the sub-events, the timing aspect is an important concept (i.e., consecutive tweets in time).

a sequential model could successfully exploit chronological relations between the tweets in a Twitter stream as an informative feature for sub-event detection.


in this work, we propose to use the bin-level evaluation, since it is a more natural way to measure the duration of a sub-event in a supervised sequence labeling setting.


In a more general case, where multiple edge types exist, we expect them to have different impacts on the aggregation.

we model these edge types with different weights in Eq.


Given an entity pair, we want the observed KB relations between them to have higher scores than the unobserved ones.

a pair-wise ranking based loss function is used to train our model: where p pos refers to a positive relation, p neg refers to a uniformly sampled negative relation, and γ is the margin hyper-parameter.


In this case, directly appending representation of pronouns does not provide extra useful information for making fine-grained predictions.

instead of using the concatenation operator, we propose to construct a stronger interaction between the mention and context with an attentionbased matching module, which has shown its effectiveness in recent natural language inference models (Mou et al., 2016;Chen et al., 2017).


A qualitative analysis of an interactive inference network shows that it is good at capturing word overlaps, antonyms and paraphrases between sentence pairs (Gong et al., 2018).

we believe this network is suitable for capturing the argument compatibility between two event mentions.


Therefore, in lifelong learning research, the learner is usually constrained on the memory size, denoted as a constant B.

at each step k, the learner is allowed to keep training samples from {T (j) |j = 1, .


(2) for each task, we want to select the most informative samples to store in the memory, instead of random sampling like in (Lopez-Paz and Ranzato, 2017).

the budget of memory can be better utilized.


However, distribution approximation itself is a hard problem and will be inefficient due to its combinatorial optimization nature.

many recent works such as GEM ignore this step and randomly select samples from each task to store in the memory.


This simplest distribution approximation does not give an improvement in our experiments because of the huge information loss.

we propose a better approach of sample selection by clustering over the embedding space from the model, and choose one representative from each cluster to store in the memory.


Moreover, the training data is noisy since it is automatically annotated by distant supervision.

we cannot just use part of it for testing.


The interaction between a customer and the customer care representative contributes substantially to the overall customer experience.

it is imperative for customer care agents and chatbots engaging with humans to be personal, cordial and emphatic to ensure customer satisfaction and retention.


For the growth of any company or application it is necessary for the customer care agents to be cordial and amicable to the customer.

along with handling queries, it is important for agents to provide customer satisfaction by greeting, empathizing, appreciating feedback, apologizing at the right time, and thus build a strong relation with the customer.


Moreover, such language conditioning shall ensure that responses are more human-like.

the major motivation behind this task is to create systems that are able to converse with humans efficiently and generate replies in accordance with the emotions of the customer.


The Pointer generator model (Model 2) is aided by the copying mechanism.

it is better modeled to include portions of the content from the generic response into the courteous response.


Given that the initial word representations in a span n i are x 1 , ..., x T , we denote their representations x * 1 , ..., x * T after encoded by the biLSTM.

then we obtain the inner-span attention by where α t is computed via a standard feed-forward neural network we have the weighted embedding of each spanx i througĥ Afterwards, we concatenate the starting (x * start ) and ending (x * end ) embedding of each span, as well as its weighted embedding (x i ) and the length feature (φ(i)) to form its final representation e: Once the span representation of n ∈ N and p are obtained, we compute F c for each n with a standard feed-forward neural network: where is the element-wise multiplication.


The weighting component receives all features k for n and n , and the span representations e n and e n as input, where e n and e n help selecting appropriate knowledge based on the context.

for a candidate pair (n, n ) and a knowledge source i, its knowledge attention score is computed via where o i n,p = [e n , k i n,p ] and o i n ,p = [e n , k i n ,p ] are the concatenation of span representation and external knowledge embedding for candidate n and n respectively.


As a result, for a candidate pair (n, n ) and a knowledge source i, its knowledge attention score is computed via where o i n,p = [e n , k i n,p ] and o i n ,p = [e n , k i n ,p ] are the concatenation of span representation and external knowledge embedding for candidate n and n respectively.

the weight for features from different knowledge sources is thus computed via Similar to the weighting component, for each feature i, we compute its score f i k (n, n , p) for n against n in the scoring component through where it is worth noting that we exclude e in this component for the reason that, in practice, the dimension of e is normally much higher than k. it could dominate the computation if e and k is concatenated.


In Example B, the distance between 'an accident' and the pronoun 'it' is too far.

the 'None' result from the End2end model indicates that the contextual information is not enough to make the decision.


While in Example B, the target pronoun 'it' cannot describe human, thus 'father' and 'friend' are 0 on the AG feature while 'hospital' and 'accident' are 1.

the attention module emphasizes AG more than other knowledge types.


We split the sentence with pronouns and nouns into two sub-sequences separated by the previous word of the pronoun.

the example sentence in the Figure 2 can be split into two subsentences as shown below: • S x : " Two Northwest Airlines pilots failed to make radio contact with ground controllers for more than an hour and" • S y : "overflew their Minneapolis destination by 150 miles before discovering the mistake and turning around".


If we let G n be the DAG that uses t 2 exactly n times, then the language L defined by this automaton is n∈N G n .

since w(G n ) = w(t 1 )w(t 2 ) n w(t 3 ) and w(t 1 ), w(t 2 ) and w(t 3 ) are positive, w satisfies R1 and: w also satisfies R2 and the weighted automaton in Example 3 is probabilistic.


Although potentially useful information on word relationships is hidden in their internal subwordlevel structure, 1 subword-agnostic word representation models do not take these structure features into account and are effectively unable to represent rare words accurately, or unseen words at all.

there has been a surge of interest in subword-informed word representation architectures aiming to address these gaps.


The atoms of corpus-level DP form the base measure for document levels DPs of words and entities.

the atoms of corpus-level DP can represent word topics, entity mixture components, or both of them.


We use a method based on Alternating Direction Method of Multipliers (ADMM) (Boyd, 2011) optimization to learn a linear mapping back to original label space.

during training, we can select and learn the most informative label subset using a submodular maximization framework of linear cost.


We choose a model parameter γ > 0 giving us our final submodular function: The main step of our proposed framework is to propagate the predicted value for the selected label subset to the full set of labels in order to recover the original space.

we aim to find a linear relation including the dependency of the selected labels and all the other labels.


FastXML finds a partitioning tree by using nonlinear binary classifiers to partition the samples at each node, which is a very complicated and unstable problem for highdimensional spaces.

for both SLEEC and FastXML methods, they need an ensemble of the learners in order to overcome this instability and achieve a good prediction accuracy.


In DS, we align the entity pair (head, tail) from a triple head, rel, tail extracted from a huge knowledge base (e.g., Freebase, Wikidata) with sentences from free texts (e.g., Wikipedia, New York Times) to obtain the training examples, and the label of such an example is the corresponding relation rel.

dS can automatically create a set of training data for each entity pair.


Formally, the relation r t at t ∈ (t 1 , t 2 ] can be infered by r t 1 .

we can model the problem as a sequence labeling problem with noisy inputs.


Although it is challenging for a machine to achieve, humans are able to learn new concepts by transferring knowledge from known to unknown domains based on high-level descriptions and semantic representations (Thrun and Pratt, 1998).

without labelled data of unseen classes, a zero-shot learning framework is expected to exploit supportive semantic knowledge (e.g., class descriptions, relations among classes, and external domain knowledge) to generally infer the features of unseen classes using patterns learned from seen classes.


divided by the total number of nodes in the set.

the vector associated to each set shows how w j is semantically close to that set.


Based on the construction method, v w,c quantified the relatedness between words and the class but, unlike v w and v c , it did not include detailed semantic meaning.

the classifier using v w,c only could not find out the correct unseen class and neither   vw; vc; vw,c] 0.418 0.754 0.302 0.500 Table 6: The accuracy of the zero-shot classifier in Phase 2 given documents from unseen classes only.


As a point of note, for our experiments, we obtained optimal results by using κ = 2.

to simplify the description of our experiments, we name the parameter β 1 as β (the parameter β 2 is then identical to 1 − β).


PLSA (Hofmann, 1999) or LSA, (Deerwester et al., 1990), it was shown in (Blei et al., 2003) that LDA outperforms each of these.

we use the stronger baseline of LDA in our experiments.


w rand is a random token in the sentence that is not the head of w mod .

the datasets used in these tasks are balanced.


However, we can introduce a variational distribution p(z|x) to model π (z|x).

plugging this into (6) we observe that Moreover, H(q, p) is an upper bound on H(Z|X) for any p with equality iff p matches the true posterior distribution π.


Once again, it cannot be written as a sum of local objectives because the entropy term involves log ofq(z) computed from all samples.

it is not clear if stochastic optimization will be effective.


We train the model end-to-end with only the traditional predictive language modeling perplexity objective.

our system can model named entities in text more accurately as demonstrated by reduced perplexities compared to traditional LM baselines.


The decision of whether the word is a general term or a named entity from a given type is controlled by a gating mechanism conditioned on the context observed so far.

kALM learns to predict whether the context observed is indicative of a named entity of a given type and what tokens are likely to be entities of a given type.


Then we concatenate the resulting vectors with the source embeddings as inputs for the baseline Bi-RNN Encoder.

the encoder process can be formalized as follows: xn ⊕ s n .


Actually, this part can be further fine tuned along with the NMT learning, by treating them as one kind model parameters.

there arises a question that whether fine-tuning the parser model parameters can bring better performance.


The major drawback is that its bottomup composing strategy is insufficient for bottom nodes.

bi-directional extensions have been suggested Yang et al., 2017).


Furthermore, longer sentences are intuitively harder to translate due to the propagation of errors made early on when generating the target language sentence.

a simple way to define the difficulty of a sentence Note that we can compute this difficulty metric on either the source language sentence or the target language sentence.


Those extracted-and-edited sentences serve as pivotal points in the target language space to guide the unsupervised learning.

the learned target language distribution could be closer to the real one.


A proper initialization can also help align the semantic spaces and extract potential parallel pairs within them.

following the previous methods, we use the inferred bilingual dictionary as described in Conneau et al.


Despite the popularity of back-translation in the previous methods (Lample et al., 2018a;Artetxe et al., 2018;Lample et al., 2018b), we argue that it suffers from the low-quality pseudo language pairs.

in this work, we propose a new paradigm, extract-edit, to address this issue by extracting and editing potential real parallel sentences.


The rough operation like this will result in sub-optimal translation performance.

in order to assess the quality of the translated sentence t * and train the translation model V s→t , we introduce an evaluation network R for evaluating the relative similarities between the source and target sentences among all sentence pairs.


Although we can infer that the models would perform even better with k > 10 from Figure 3, more computational resources will be required for that and we already observe a decelerated growth of BLEU scores from k = 8 to k = 10.

in this paper, we set k = 10 for the full models.


The Effect of Comparative Translation Finally, we aim to roughly evaluate the effect of the proposed comparable translation loss in our model.

we compare our model with a twostaged NMT system, where we extract and edit the parallel pairs and retrain the NMT system with the standard maximum likelihood estimation (MLE) loss in a supervised way (by taking the extractedand-edited sentences as the ground-truth targets).


A limitation of this method is that it could lose fine-grained order information in each latent dimension and have issues with long responses that involve multiple dialog acts.

we propose a novel method, Attention Fusion, to combine categorical latent actions with the decoder.


Then we have: The decoder's next state is updated by h i+1 = RNN(h i , w i+1 ), h i ) and h 0 is computed via summation-fusion.

attention fusion lets the decoder focus on different latent dimensions at each generation step.


the decoder only sees z sampled from q(z|x, c) and never experiences z sampled from p θ (z|c), which is always used at testing time.

in this paper, we propose a simplified ELBO for encoder-decoder models with stochastic latent variables: Essentially this simplified objective sets the posterior network the same as our encoder, i.e.


It is challenging to quantify the performance of RL-based neural generation systems because it is possible for a model to achieve high task reward and yet not generate human language (Das et al., 2017).

we propose a novel measure, the Language Constrained Reward (LCR) curve as an additional robust measure.


In this paper, we propose a SPACEFUSION model to jointly optimize diversity and relevance that essentially fuses the latent space of a sequenceto-sequence model and that of an autoencoder model by leveraging novel regularization terms.

our approach induces a latent space in which the distance and direction from the predicted response vector roughly match the relevance and diversity, respectively.


At any decode time step t, the decoder can either generate a word from the decode vocabulary or copy a word from the memory.

the decoder computes: (1) generate distribution P g (y t ) over the decode vocabulary, and (2) copy distribution P c (y t ) over words in the memory.


Attention over tokens on the other hand can push the attention weights towards either the subject or object in the KB tuple, based on the query's request.

using both in conjunction helps BOSSNET perform significantly better than the baselines and illustrates the importance of the BOSS memory in comparison to a flat memory layout.


One of the main limitations of mainstream word embeddings lies in their static nature, i.e., a word is associated with the same embedding, independently from the context in which it appears.

these embeddings are unable to reflect the dynamic nature of ambiguous words 1 , in that they can correspond to different (potentially unrelated) meanings depending on their usage in context (Camacho-Collados and Pilehvar, 2018).


Gains are much larger on state-of-the-art models like BERT, while the BOW model learns almost nothing from new examples.

performance changes are more drastic on PAWS QQP than on QQP.


The original ICNALE is not error annotated.

we sampled a total number of 1,736 sentences, which are manually annotated with grammatical errors based on KJ 's annotation scheme.


However, experimental results have shown that the model rankings on both JF-LEG and FCE are different (Figure 1).

single-corpus evaluation is deemed weak regardless of its diversity.


Empirical evaluation results revealed that the model performance and rankings considerably vary depending on the corpus, suggesting that a single-corpus evaluation can be unreliable.

cross-corpora evaluation should be applied to GEC models.


To reduce model complexity, we replace the fully-connected structure with a star-shaped topology, in which every two non-adjacent nodes are connected through a shared relay node.

complexity is reduced from quadratic to linear, while preserving the capacity to capture both local composition and long-range dependency.


The radical connections focus on the non-local compositions and the ring connections focus on the local composition.

star-Transformer works for modestly sized datasets and does not rely on heavy pre-training.


Ring Connections Since text input is a sequence, we bake such prior as an inductive bias.

we connect the adjacent satellite nodes to capture the relationship of local compositions.


Compare with the standard Transformer, all positions are processed in parallel, pair-wise connec-  tions are replaced with a "gather and dispatch" mechanism.

we accelerate the Transformer 10 times on the simulation task and 4.5 times on real tasks.


And the symbol "Ours + Char" means an additional character-level pre-trained embedding JMT (Hashimoto et al., 2017) is used.

the total size of embedding should be 400 which as a result of the concatenation of GloVe and JMT.


Without the ring connections, it loses its performance heavily on real tasks.

both the radical and ring connections are necessary to our model.


However, asynchronous domains (forum, email) differ from synchronous (MRDA) in their underlying conversational structure (Louis and Cohen, 2015), in style (spoken vs. written), and in vocabulary usage (meetings on some focused agenda vs. conversations on any topic of interests in a public forum).

we hypothesize that to make the best use of labeled data from synchronous domains, one needs to model the shift in domains.


Despite this, its modeling strength is limited compared to structured models that use global inference to model consistency in the output, especially when there are strong dependencies between output labels (Collobert et al., 2011).

instead of classifying sentences independently with a Softmax layer, our second method is to model them jointly with a CRF layer (Lafferty et al., 2001).


11 For this paper, we exclude § 130 StGB 12 , that covers incitement to hatred against a national, racial, religious group or a group defined by their ethnic origins, due to an additional complexity of the assessment.

to understand their elements in detail, it is crucial to refer to the more general legal concept behind these criminal offenses: as illustrated in Fig  the scope of protection of § 185 to § 187 StGB follows the respective interpretation of the right to honor.


24 In the German Criminal Code, this is reflected by § 193 StGB: even if a statement falls under the scope of said criminal offenses, it might still be 21 BVerfGE 99,185,197;E 97,381,403.

22 we do not implement subsequent conditions of § 186, § 187 StGB, as shown in Figure 1, respectively whether facts cannot be proven to be true ( § 186 StGB) or whether the untruth was intended and known ( § 187 StGB).


On the constitutional level, then, the decision of whether a social media posting constitutes a punishable criminal offense and leads to the platform provider's deletion obligation can thus ultimately be perceived as a balancing between freedom of expression and the right to honor.

the court would have to balance these concurrent rights depending on the case at hand.


Such a case will result in a positive penalty, while the penalty will be 0 in all other cases.

l sim constrains similar labels to have similar outputs, while l sub reinforces each cooccurring subsumption pair to satisfy the dependency of the parent label on the child label.


Previous research (Collobert et al., 2011;Socher et al., 2011) has shown the usefulness of using pretrained word vectors to improve the performance of various models.

in our experiments, we have used the publicly available pre-trained GloVe word vectors for Twitter by (Pennington et al., 2014a).


Finally, we validated annotations by only accepting annotations if at least 3 among 4 annotators agreed on.

we presume the quality of attained ground truth labels is dependable.


For embedding models, we determined the minimum count parameter to 5, culminating in the elimination of infrequent words.

we have 3, 100-dimensional word embedding tensor in which first, second and third channels consist of word2vec, fastText and GloVe embeddings respectively.


We are interested in whether (a) the knowledge in the optimization process of HOTFLIP can be distilled into a neural model, and (b) whether this model can generalize to a black-box attack.

the outline of our approach is as follows: 1.


NVD only officially publishes a threat after all evaluations are completed.

there is a time delay between CVE entry established date and the official publication date.


Even though we still observe improvements in the performance with more than 5 MF layers, the difference is small.

we select T = 5 as it produces the best trade-off between accuracy and computational complexity.


In this second step, the gradient that is backpropagated is flipped by a Gradient-Reversal layer.

5 the model parameters are updated such that the classifier becomes worse at solving the task.


These results show significant density of embeddings in semantic space.

we optimize the number of clusters for creating VLAEs by resorting to validation data and measuring taskspecific performance.


During the full annotation task, we decided to search for more political keywords as they tend to be richer in offensive content.

we sampled our full dataset, so that 50% of the tweets come from political keywords, and the other 50% come from non-political keywords.


This function description provides crucial clues to determine the type of transduced as positive regulation.

to capture such knowledge from external KBs, for each entity, we first learn a KB concept embedding from its properties, and then automatically incorporate the KB representation into its Tree-LSTM hidden state with a gate function.


The Genia event extraction shared task provides the annotations of all entity mentions.

for each trigger, we use all the entity mentions that occur in the same sentence as its candidate arguments, and then assign an argument role or None.


tially effective solution for a small labeled dataset by using pre-trained language models for various NLP tasks (Ramachandran et al., 2017;Howard and Ruder, 2018).

we focus on transferring knowledge from unlabeled in-domain dialogues as well as a general out-of-domain corpus.


We assume that counselors and clients have different language models because they have distinctive roles in the dialogue.

we train a counselor language model and a client language model separately.


The size of attention layer is set to 500.

the model is trained by three steps: 1) training word vectors and two language models, 2) training seq2seq conversation model with pretrained LMs, and 3) fine-tuning task-specific classification layers, after removing softmax of the conversation model.


Note that these pretrained LMs are character-based.

words in the target data set are first converted into a sequence of characters, and then fed into the LMs.


Yet, only 1,863 annotated sentences (12,783 tokens) were available for German.

in the experiments, we performed 5-fold cross-validation.


We found that BPE can segment a noisy word into two or more known subword units that have no real relationship to the actual word.

using representations of wrong subword units could hurt the performance.


We regard the lemma as a lexicographic convention, often used to better organize dictionaries.

the choice of which inflected form is the lemma is motivated by tradition and convenience, e.g., the lemma is the infinitive for verbs in some Indo-European languages, rather than by linguistic or cognitive concerns.


If k > |L|, purity can take a value of 1 if each cluster is a subset of a ground partition.

purity is expected to increase as k grows higher than |L|.


In other words, purity does not depend on word frequency.

it is not an approximation of PoS tagging accuracy, like the M-1 measure used by Bansal et al.


Such few observations likely do not provide enough information for Brown or Exchange to properly place those words.

from the already computed clusterings, we remove words with a frequency less than 5 and recalculate the two quality measures.


2 However, the Exchange algorithm is an iterative clustering algorithm that has a complete and valid cluster partitioning at the end of each iteration.

we can also measure morphosyntactic content in each of these clusterings.


The conditional distribution for x T is where the quantity n k|d S is added and thus transferred from the source document.

the calculation of P x incorporates the knowledge transferred from the other language.


The training of topic models is unsupervised, i.e., there is no ground truth for labeling a topic, which makes it difficult to analyze the effect of transfer learning.

after calculating P x T , we take an additional step called reverse valida-tion, where we design and calculate a measurecircular validation loss-to quantify the transfer.


The conditional distribution for sampling a topic z x for a token x during sampling can be factorized into document-topic and topic-word levels: where ⊗ is element-wise multiplication.

we have the following inequality: and the KL-divergence term in Theorem 3 is simply the sum of the KL-divergences between the conditional and prior distributions on all levels.


In this case, Q φ,x , the prior for P φ,x , is simply a Kdimensional uniform distribution U.

then: at levels where transfer happens (documentor word-level), a low crosslingual entropy is preferred, to offset the impact of monolingual entropy where no transfer happens.


That is, z (t) , and only sample the target language.

taking the difference between the expectation at iterations t and t − 1, we have we have theorem 3.


From Theorem 2, for target language, with probability at least 1 − δ, For the source language, similarly, with probability at least 1 − δ, Given a word type w T , we notice that only the KL-divergence term in ϵ(x T , w S ) varies among different tokens x T .

we use KL w S and KL w T to denote the maximal values of KLdivergence over all the tokens, Let n = min {n w T , n w S }, and n ⋆ = max {n w T , n w S }.


We then focus on the topic-dependent part of the last inequality.

we have the following inequality: Plug in Lemma 1, we see that Appendix C Dataset Details For all the languages, we use existing stemmers to stem words in the corpora and the entries in Wiktionary.


Note that although all the language pairs share the same set of English test documents, the document representations are inferred from different topic models trained specifically for that language pair.

the document representations for the same English document are different across different language pairs.


To learn in this paradigm, instead of using the pre-trained word embeddings as fixed training samples, at every training step we obtain samples from the Gaussian mixture space.

our method is exploring the entire embedding space instead of only the specific points assigned for observed words.


The density can be computed using the idea of normalizing flow described above.

for the the monolingual embedding spaces, we need to define tractable density functions p(x) and p(y).


However previous work (Zhang et al., 2017) has noted that word frequency information is a strong signal -words with similar frequency are likely to be matched -and thus we use π(y j |x i ) to incorporate this prior knowledge.

in this work, we assume that the frequencies of aligned bilingual words in the individual monolingual corpus should be correlated, and to match words that are ranked similarly, we model the Gaussian mixture weights as the negative absolute difference between log-scale word ranks and normalize over all the target Gaussian component words by a softmax function with temperature τ : if a word x i has similarly frequency rank as word y j , the sample x from the Gaussian distribution centered at x i will be assigned higher weight for the componentp(y|y j ) =p(W xy x|y j ).


However, they still remain relatively close to their per-token anchor.

these anchors can still serve as a good approximation for learning alignments.


This is motivated by our observations in Section 3.1 that context-dependent embeddings are well clustered around their centers.

in the case where a dictionary is available, we solve Eq.


In order to synthesize a dictionary, we use distance in the anchor space.

far we assumed that embeddings for both source and target languages are pretrained separately.


Millions of user-generated messages are produced every day, far outpacing the human being's reading and understanding capacity.

the current decade has witnessed the increasing demand for effectively discovering gist information from large microblog texts.


For example, less than 15% tweets contain at least one hashtag (Wang et al., 2011;Khabiri et al., 2012).

for a multitude of posts without human-annotated hashtags, there exists a pressing need for automating the hashtag annotation process for them.


The other two are discrete and merely used as arguments to VIPER.

they are only required to specify nearest neighbors for standard input characters.


We invert the scores g of G2P by 1/g since lower scores are better for edit distance.

s * (0) is always 1 and s * (p) is the relative performance compared to the clean case of no perturbations.


For instance, at p = 0.5, POS improves by about 20pp, while AT alone had an effect of only 12pp and the effect of CE was even negative.

it appears that AT is able to kick-start the benefits of CE, especially in the case when they alone are not effective.


This would result in a 'perfect' cosine similarity, which would be advantageous in the previous experiment.

we perform an additional experiment where we examine if  Table 6: Results of the intrinsic evaluation where we compare clean sentences to their perturbed counterparts as well as randomly chosen sentences.


E.g., if we perturb all characters in a sentence (p = 1.0), the SELMo embeddings of the perturbed sentence are in 1% of the cases more similar to the original sentence whereas this is the case for more than 34-39% for VELMo.

vELMo embeddings better capture similarity between visually similar words.


For 3 of 6 communities, even the popularity oracle does not beat post time baseline, and in all cases, the mean performance of the controversy predictor exceeds the oracle by t = 180.

in our setting, controversy predictors and popularity predictors learn disjoint patterns.


Conditional Random Field Compressed sentences have to remain grammatical, which implies a dependence between values in y.

without any restrictions on the dependence between choices, it is intractable to marginalize over all possible y with a scoring function for a given (x, y) pair, we assume that only neighboring values in y are dependent, and apply a linear-chain CRF (Lafferty et al., 2001) that uses h as its features to this problem.


Semi-Markov Conditional Random Field Although compressed sentences should often include entire phrases, the CRF does not take into account local dependencies beyond neighboring words.

we relax the Markov assumption and score longer spans of words.


Previous findings indicate that titles help with retention only when presented towards the beginning of a text (Dooling and Mullet, 1973).

we place texts in the left margin at the top of a paragraph as shown in the example in Figure 3.


models with low HUSE are guaranteed to be distinguishable from humans).

the models which we identify as having low-HUSE are reliably distinguishable regardless of the crowdworker quality.