As in other works, we use newstest-2014 as test set.

in order to follow the standard WMT evaluation setting, we use mteval-v14.pl on untokenized hypothesis to calculate case-sensitive BLEU scores.


In recent years, neural networks with distributed word representations (i.e., word embeddings) (Mikolov et al., 2013;Pennington et al., 2014) have been introduced to calculate word scores automatically for CRFs (Chiu and Nichols, 2016;.

semi-Markov conditional random fields (SCRFs) (Sarawagi and Cohen, 2005) have been proposed for the tasks of assigning labels to the segments of input sequences, e.g., NER.


Traditionally, the input of LDA is a documentterm matrix of term frequencies (TF), according to the bag-of-words model (BoW).

wilson and Chew (2010) showed that point-wise mutual information (PMI) term weighting model can be successfully applied to eliminate stop words from topic descriptors.


In general, NE Independent model showed improvement in coherence up to a certain value of α (different in each case), followed by a decline, reaching very low values for NE Independent (x10).

nE Document Dependent model does not introduce new parameters into LDA and manages to achieve best performance in the majority of settings, thus being more stable and easy to use.


AS Reader (Kadlec et al., 2016) which models the question and the paragraph using GRU followed by attention mechanism got 74.1% accuracy on the SciQ test set.

for a question, they used a different corpus to extract the text passage.


Using questionoption tuple as input gave significant advantage to our model.

there is a lot of scope for future work.


Indeed, the top-systems were built using Support Vector Machines (SVMs) trained with Tree Kernels (TKs), which were applied to a syntactic representation of question text (Filice et al., 2016Barrón-Cedeño et al., 2016).

nns-based models struggled to get good accuracy as (i) large training sets are typically not available 2 , and (ii) effectively exploiting full-syntactic parse information in nns is still an open issue.


This suggests that NNs require more data to learn complex relational syntactic patterns expressed by TKs.

the plot in Figure  1 shows that the improvement reaches a plateau around 100k examples.


set and almost 3% on the test set over TK models.

using too much automatically labeled data may hurt the performance on the test set.


(2017), who use Nystrom methods to compact the TK representation in embedding vectors and use the latter to train a feed forward NNs.

we present a simpler approach, where NNs learn syntactic properties directly from data.


Indeed, the latter only exploit lexical similarity measures, which are typically also generated by NNs.

even if our conjecture were wrong, the bottom line would be that, thanks to our approach, we can have NN models comparable to TK-based approaches, by also avoiding to use syntactic parsing and expensive TK processing at deployment time.


This is trivial for statistical machine translation (Koehn, 2009) because there is no overlap between the translation units of a hypothesis, i.e., we have a 0-1 coverage vector.

it is not the case for NMT where the coverage is modeled in a soft way.


This approach was developed based on the idea that the training cost is a useful measure to determine the translation quality of a sentence.

some of the sentences that can be potentially improved by training may be deleted using this method.


As a solution, various studies proposed segmenting words into sub-word units and performing translation at the sub-lexical level.

statistical word segmentation methods have recently shown to be prone to morphological errors, which can lead to inaccurate translations.


In fact, the second approach is now prominent and has established a pre-processing step for constructing a vocabulary of sub-word units before training the NMT model.

several studies have shown that segmenting words into sub-word units without preserving morpheme boundaries can lead to loss of semantic and syntactic information and, thus, inaccurate translations (Niehues et al., 2016;Ataman et al., 2017;Pinnis et al., 2017;Huck et al., 2017;Tamchyna et al., 2017).


Note that in this case there is now only an embedding vector (of dimension 512 in our experiments) for each speaker.

the resulting domain embedding are non-trivial to interpret (i.e.


We hypothesize that an NMT ensemble would be strengthened if its component models were complementary in this way.

ensembling often requires component models to make predictions relating to the same output sequence position at each time step.


Previous research has used very large batches to improve training convergence while requiring fewer model updates (Smith et al., 2017;Neishi et al., 2017).

with such large batches the model size may exceed available GPU memory.


It is also possible to constrain decoding of linearized trees and derivations to wellformed outputs.

we found that this gives little improvement in BLEU over unconstrained decoding although it remains an interesting line of research.


Furthermore, artificially generated partial feedback does not contain noise, given that the reference translation is adequate.


The previous work uses the bag-of-word to constraint the latent variable, and the latent variable is the output of the encoder.

we use the bag-of-word to supervise the distribution of the generated words, which is the output of the decoder.


Compared with the previous work, our method directly supervises the predicted distribution to improve the whole model, including the encoder, the decoder and the output layer.

the previous work only supervises the output of the encoder, and only the encoder is trained.


The beam search finds good candidate translations by considering multiple hypotheses of translations simultaneously.

as the algorithm searches in a monotonic left-to-right order, a hypothesis can not be revisited once it is discarded.


In this section, we introduce an extended decoding algorithm of the beam search, which maintains a single priority queue that contains all visited hypotheses.

to the standard beam search, which only considers hypotheses with the same length in each step, the proposed algorithm selected arbitrary hypotheses from the queue that may differ in length.


Finally, similarly to the second variant, the last variant, Pseudofit fus-*, adds a supplementary representation of the target word.

this representation is not an additional pseudosense but an aggregation of its already existing pseudo-senses, which can be viewed as another global representation of the target word.


On EVAL, Hearst-pattern based methods get penalized by OOV words, due to the large number of verbs and adjectives in the dataset, which are not captured by our patterns.

in 7 of the 9 datasets, at least one of the sparse models outperforms all distributional measures, showing that Hearst patterns can provide strong performance on large corpora.


Thus the alignment model can be considered as an implicit part of the translation model.

separating the alignment model from the lexicon model has its own advantages: First of all, this leads to more flexibility in modeling and training: The models can not only be trained separately, but they can also have different model types, such as neural models, count-based models, etc.


Neural coreference is promising since it allows cross-lingual transfer using multilingual embedding.

most of the recent neural coreference models (Wiseman et al., 2015(Wiseman et al., , 2016Manning, 2015, 2016;Lee et al., 2017) have focused on training and testing on the same language.


The tasks of EL and coreference are intrinsically related, prompting joint models (Durrett and Klein, 2014;Hajishirzi et al., 2013).

the recent SOTA was obtained using pipeline models of coreference and EL (Sil et al., 2018).


For instance, according to the ACE-2005 annotation guideline, in the sentence "Jane and John are married", an ED system should be able to identify the word "married" as a trigger of the event "Marry".

it may be difficult to identify events from isolated sentences, because the same event trigger might represent different event types in different contexts.


If we only examine the first sentence, it is hard to determine whether the trigger "leave" indicates a "Transport" event meaning that he wants to leave the current place, or an "End-Position" event indicating that he will stop working for his current organization.

if we can capture the contextual information of this sentence, it is more confident for us to label "leave" as the trigger of an "End-Position" event.


The tasks that dialogue systems are trying to solve are becoming increasingly complex, requiring scalability to multi-domain, semantically rich dialogues.

most current approaches have difficulty scaling up with domains because of the dependency of the model parameters on the dialogue ontology.


The information about semantic similarity is held by d usr and d sys , which are fed to a non-linear layer to output a binary decision: where w d ∈ R 2L and b d are learnable parameters that map the semantic similarity to a belief state probability P t (d) of a domain d at a turn t. Slots and values are tracked using a similar architecture as for domain tracking (Figure 1).

to correctly model the context of the systemuser dialogue at each turn, three different cases are considered when computing the similarity vectors: 1.


The reason may be twofold: Firstly, as discussed in previous works (Joty et al., 2013), it is important to address discourse structure characteristics, e.g., through modeling lexical chains in a discourse, for discourse parsing, especially in dealing with long span scenarios.

most existing approaches mainly focus on studying the semantic and syntactic aspects of EDU pairs, in a more local view.


Here, lexical cohesion reflects the semantic relationship of words, and can be modeled as the recurrence of words, synonym and contextual words.

previous works do not well model the discourse cohesion within the discourse parsing task, or do not even take this issue into account.


The main reason might be that MST-full follows a global graph-based dependency parsing framework, where their high order methods (in cubic time complexity) can directly analyze the relationship between any EDUs pairs in the discourse, while, we choose the transition-based local method with linear time complexity, which can only investigate the top EDUs in S and B according to the selected actions, thus usually has a lower performance than the global graph-based methods, but with a lower (linear) time complexity.

the neural network components help us maintain much fewer features than MST-full, which carefully selects 6 different sets of features that are usually obtained using extra tools and resources.


We follow the criterion of Polanyi (1988) and Irmer (2011) which treats clauses as EDUs.

since a discourse unit is a semantic concept but a clause is defined syntactically, in some cases segmentation by clauses is still not the most proper strategy.


Thus, the success of such systems relies entirely on the ability of the map to make the predicted vectors similar to the target vectors in terms of semantic or neighborhood structure.

1 whether neural nets achieve this goal in general has not been investigated yet.


Well-known theoretical work shows that networks with as few as one hidden layer are able to approximate any function (Hornik et al., 1989).

this result does not reveal much neither about test performance nor about the semantic structure of the mapped vectors.


Many recent state-of-the-art models for constituency parsing are transition based, decomposing production of each parse tree into a sequence of action decisions Cross and Huang, 2016;Liu and Zhang, 2017;, building on a long line of work in transition-based parsing (Nivre, 2003;Yamada and Matsumoto, 2003;Henderson, 2004;Zhang and Clark, 2011;Chen and Manning, 2014;Andor et al., 2016;Kiperwasser and Goldberg, 2016).

models of this type, which decompose structure prediction into sequential decisions, can be prone to two issues (Ranzato et al., 2016;Wiseman and Rush, 2016).


negative labeled F1, for trees), we use the risk objective: which measures the model's expected cost over possible outputs y for each of the training exam- Minimizing a risk objective has a long history in structured prediction (Povey and Woodland, 2002;Smith and Eisner, 2006;Li and Eisner, 2009;Gimpel and Smith, 2010) but often relies on the cost function decomposing according to the output structure.

we can avoid any restrictions on the cost using reinforcement learning-style approaches (Xu et al., 2016;Shen et al., 2016;Edunov et al., 2017) where cost is ascribed to the entire output structure -albeit at the expense of introducing a potentially difficult credit assignment problem.


For example, the oracle for RNNG lacks F1 optimality guarantees, and softmax margin without exploration often underperforms likelihood for this parser.

exploration improves softmax margin training across all parsers and conditions.


(2017a) counts the incorrect labels (i, j, X) in the predicted tree: Note that X can be null ∅, and t * (i,j) denotes the gold label for span (i, j), which could also be ∅.

6 there are two cases where t * (i,j) = ∅: a subspan (i, j) due to binarization (e.g., a span combining the first two subtrees in a ternary branching node), or an invalid span in t that crosses a gold span in t * .


Alphabets (e.g., the Latin, Cyrillic, and Greek scripts) are the most common and treat vowel and consonant let- ters equally.

abjads (e.g., the Arabic and Hebrew scripts) do not write most vowels explicitly.


Traditional input methods for abugidas are similar to those for alphabets, mapping two or three different symbols onto each key and requiring users to type each character and diacritic exactly.

we are able to substantially simplify inputting abugidas by encoding them in a lossy (or "fuzzy") way.


Because they are the essence of the entire source paper, which can directly reflect the quality of the source paper.

the methods module of the source paper has little effect on the probability of being accepted according to Table 4.


Therefore, we combine HISK and BOSWE in the dual (kernel) form, by simply summing up the two corresponding kernel matrices.

summing up kernel matrices is equivalent to feature vector concatenation in the primal Hilbert space.


Predicting how Congressional legislators will vote is important for understanding their past and future behavior.

previous work on roll-call prediction has been limited to single session settings, thus did not consider generalization across sessions.


In addition to enabling prediction, associating text with ideology allows for a further degree of interpretability.

all previous work incorporating text into roll call prediction have limited their evaluation to in-session training and testing.


1 As legislators typically serve for multiple sessions, and similar bills are proposed across sessions, we want to be able to leverage this data across sessions to inform our model.

the generalizability of previous methods to a crosssession setting is unknown.


Contrary to our hypothesis, MWE achieves higher accuracy than Meta-Only.

it remains unclear whether this signal is related to ideology or other contextual information.


these benchmarks were mostly derived independently of any NLP problems.


With the unigram distribution, for any training word, all the other vocabulary words can be selected as noise words because of their non-zero frequency.

with the bigram distribution, some vocabulary words may never co-occur with a given training word, which makes them impossible to be selected for this training word.


In the literature, much more attention has been paid to studies on what is said.

recently, capturing how it is said, such as stylistic variations, has also proven to be useful for natural language processing tasks such as classification, analysis, and generation (Pavlick and Tetreault, 2016;Wang et al., 2017).


These results indicate that the proposed CBOW-SEP-CTX model jointly learns two different types of lexical similar-ities, i.e., the stylistic and syntactic/semantic similarities in the different parts of the vectors.

our stylistic vector also captured the topic similarity, such as "サンタ (Santa Clause)" and "トナカイ (reindeer)" (the fourth row of Table 2).


The main idea behind these works is to develop neural architectures that are able to learn continuous features and capture the intricate relation between a target and context words.

to sufficiently train these models, substantial aspect-level annotated data is required, which is expensive to obtain in practice.


To some extent, humor reflects a kind of intelligence.

from both theoretical and computational perspectives, it is hard for computers to build a mechanism for understanding humor like human beings.


The most frequent discourse relations in humorous data include condition, background and Contrast.

non-humorous texts contain same-unit and attribution more.


The proposed approach is close to (Liu et al., 2015), where only the annotated data for aspect extraction is used.

we will show that our approach is more effective even compared with baselines using additional supervisions and/or resources.


The proposed embedding mechanism is related to cross domain embeddings (Bollegala et al., 2015(Bollegala et al., , 2017 and domain-specific embeddings (Xu et al., 2018a,b).

we require the domain of the domain embeddings must exactly match the domain of the aspect extraction task.


In Section 5 we refer to some alternatives and show that they do not achieve better results than the one presented above.

we do not claim that our blending method is the only option or even the best one.


The main target in this work is to investigate the effect of audio, visual, and text modalities, and different fusion methods in personality recognition, rather than proposing the method with the best accuracy.

we still repeat the accuracy of the reported methods in Table 2 and two winners of the ChaLearn 2016 competition DCC (Güçlütürk et al., 2016) and evolgen (Subramaniam et al., 2016) in Table 3.


It is likely that the limited backpropagation method learns something similar to a linear combination of channels, just like the decision-level method.

the full backpropagation method yields significantly higher results for all traits except Agreeableness.


As an example of incorrect output, the model fails to assign a high score to (prince, royalty), possibly due to the usage patterns of these words being different in context.

it assigns an unexpectedly high score to (kid, parent), likely due to the high distributional similarity of these words.


As the vocabulary is finite, it is possible to evaluate the uncertainty measures for all possible inputs to synthesize the most uncertain query.

such a greedy policy is expensive and prone to selecting outliers.


Neural Machine Translation (NMT) has shown remarkable progress in recent years.

it requires large amounts of bilingual data to learn a translation model with reasonable quality (Koehn and Knowles, 2017).


encoder, decoder or the attention mechanism.

this approach has two limitations: (i) it fully shares the components, and (ii) the shared component(s) are shared among all of the tasks.


With the development of several multilingual datasets used for semantic parsing, recent research efforts have looked into the problem of learning semantic parsers in a multilingual setup (Duong et al., 2017; Susanto and Lu, 2017a).

how to improve the performance of a monolingual semantic parser for a specific language by leveraging data annotated in different languages remains a research question that is under-explored.


Such a model allows two types of input signals: single source SL-SINGLE and multi-source SL-MULTI.

semantic parsing with cross-lingual features has not been explored, while many recent works in various NLP tasks show the effectiveness of shared information cross different languages.


For example, the two semantic units STATE : smallest one ( density (STATE)) and STATE : smallest one ( population (STATE)) share similar representations.

we also found that occasionally semantic units conveying opposite meanings are also grouped together.


When this news headline is fed into modern tools for Named Entity Disambiguation (NED), virtually all of them would map the mention Schumacher onto the former Formula One champion Michael Schumacher, as the best-fitting entity from a Wikipedia-centric knowledge base (KB).

knowing that Sunday refers to August 14, 1949, i.e., ignoring the surface form but exploiting normalized information, it becomes clear that the text actually refers to the German politician Kurt Schumacher.


As shown in Table 4, Attr2Seq tends to cover more aspects in generation, many of which are not discussed in real reviews.

expansionNet better captures the distribution of aspects that are discussed in real reviews.


(2017) and approaching their best RDF-aware method.

manual inspection reveal many cases of unwanted behaviors in the resulting outputs: (1) many resulting sentences are unsupported by the input: they contain correct facts about relevant entities, but these facts were not mentioned in the input sentence; (2) some facts are repeated-the same fact is mentioned in multiple output sentences; and (3) some facts are missingmentioned in the input but omitted in the output.


While the set of complex sentences is still divided roughly to 80%/10%/10% as in the original split, now there are nearly no simple sentences in  We believe this split strikes a good balance between challenge and feasibility: to succeed, a model needs to learn to identify relations in the complex sentence, link them to their arguments, and produce a rephrasing of them.

it is not required to generalize to unseen relations.


In fact, we can also leverage this abstraction to visualize the simplified LSTM's weights as is commonly done with attention (see Appendix A for visualization).

there are three major differences in how the weights w t j are computed.


In particular, real-time counting languages cut across the traditional Chomsky hierarchy: real-time k-counter machines can recognize at least one context-free language (a n b n ), and at least one context-sensitive one (a n b n c n ).

they cannot recognize the context free language given by the grammar S → x|aSa|bSb (palindromes).


SRNN The finite-precision SRNN cannot designate unbounded counting dimensions.

the SRNN update equation is: By properly setting U and W, one can get certain dimensions of h to update according to the value of x, by h this counting behavior is within a tanh activation.


Here, again, the two counting dimensions are clearly identified-indicating the LSTM learned the canonical 2-counter solutionalthough the slightly-imprecise counting also starts to show.

figures 1c and 1d show the state values of the GRU-networks.


Previous approaches to machine comprehension are usually based on pairwise sequence matching, where either the passage is matched against the sequence that concatenates both the question and a candidate answer (Yin et al., 2016), or the passage is matched against the question alone followed by a second step of selecting an answer using the matching result of the first step (Lai et al., 2017;Zhou et al., 2018).

these approaches may not be suitable for multi-choice reading comprehension since questions and answers are often equally important.


Matching the passage only against the question may not be meaningful and may lead to loss of information from the original passage, as we can see from the first example question in Figure 1.

concatenating the question and the answer into a single sequence for matching may not work, either, due to the loss of interaction information between a question and an answer.


We can see that the performance of our model on different types of questions in the RACE dataset is quite similar.

our model is only based on wordlevel matching and may not have the ability of reasoning.


Congratulations (bow-theknee emoticon)" Comparing Default with LogReg and LinSVM, we can see that the linear models performed better than the default RNN model without pretraining, when the labeled data size is less than or equal to 20K.

looking at the results of Dial, our method improved Default even for these cases (5K to 20K), and Dial clearly outperformed the linear models.


(S1) @Anonymous doing a great job... #not What do I pay my extortionate council taxes for?

#Disgrace #Ongo-ingProblem http://t.co/FQZUUwKSoN the reliability of the self-labeled data is an important issue.


d (k+1) is the representation of document s (k+1) using Self-Attentive Encoder,û is the output words after the second-pass decoder.

to the original Deliberation Network (Xia et al., 2017), where they propose a complex joint learning framework using Monte Carlo Method, we minimize the following loss as Xiong et al.


BLEU measures n-gram overlap between a generated response and a gold response.

since there is only one reference for each response and there may exist multiple feasible responses, BLEU scores are extremely low.


The original system wrongly classified the user intention as shopping since this is a common conversational pattern in shopping.

our utterance rewriter is able to recover the omitted information "under the weather in Beijing".


This has been often attributed to the manner and extent to which these models use the dialog history when generating responses.

there has been little empirical investigation to validate these speculations.


Sequence-to-sequence models (Sutskever et al., 2014) has become one of the most popular approaches to dialog systems, for it provides a high degree of automation and flexibility.

they are known to suffer from the "dullresponse" problem (Li et al., 2015).


On one hand, the highest frequency of source-side copy helps address sparsity and results in the highest precision and recall.

we see space for improvement, especially on the relatively low recall of target-side copy, which is probably due to its low frequency.


It is demonstrated that the annotation of only preterminal categories is sufficient to adapt a CCG parser to new domains.

the solution is limited to a specific parser's architecture, making non-trivial the application of the method to the current state-of-the-art parsers Yoshikawa et al., 2017;Stanojević and Steedman, 2019), which require full parse annotation.


We relax these assumptions by using dependency tree, which is a simpler representation of the syntactic structure, i.e., it lacks information of longrange dependencies and conjunct spans of a coordination structure.

due to its simplicity and flexibility, it is easier to train an annotator, and there exist plenty of accessible dependency-based resources, which we exploit in this work.


14 We report labeled bracket F1 scores between the resulting trees and the gold trees in the true Switchboard corpus, using the EVALB script.

15 the reported scores suffer from the compound effect of failures in CCG parsing as well as ones occurred in the conversion to the constituency trees.


Figure 3 is one of such cases, which the plain depccg falsely analyzes as one huge NP phrase.

after fine-tuning, it successfully produces the correct "If S 1 and S 2 , S 3 " structure, recognizing that the equal sign is a predicate.


In the previous experiments, we rely purely on unsupervised NMT for pivot translation, assuming that the translation on each hop cannot leverage any bilingual sentence pairs.

there in-   deed exist plenty of bilingual sentence pairs between some languages, especially among the popular languages of the world, such as the official languages of the United Nations and the European Union.


The final loss function for the mapping matrix is: L W |D enables the model to leverage the distributional information available from the two embedding spaces, thereby using all available monolingual data.

l W |S allows for the correct alignment of labeled pairs when available in the form of a small seed dictionary.


This baseline does not make use of various advances in NMT architectures and training tricks.

to the baseline, we use a BiDeep RNN architecture (Miceli Barone et al., 2017), label smoothing (Szegedy et al., 2016), dropout (Srivastava et al., 2014), word dropout (Sennrich et al., 2016a), layer normalization (Ba et al., 2016) and tied embeddings (Press and Wolf, 2017).


The IT task is very small: training on IT data alone results in over-fitting, with a 17 BLEU improvement under fine-tuning.

no-reg fine-tuning rapidly forgets previous tasks.


Large KBs such as DBpedia (Auer et al., 2007), Wikidata (Vrandecic and Krötzsch, 2014) and Yago (Suchanek et al., 2007) contain millions of facts about entities, which are represented in the form of subject-predicate-object triples.

these KBs are far from complete and mandate continuous enrichment and curation.


The encoder-decoder with attention model (Bahdanau et al., 2015) has been used in machine translation.

in the relation extraction task, the attention model cannot capture the multiword entity names.


For example, the first bin in the left subplot shows that discovered topic 1 has 91 unique words, all belonging to common topic C1.

the first bin in the right subplot shows that discovered topic 1 has 100 unique words, 38 belonging to common topic C1 and 58 to common topic C2.


All five identified topics contain words from the 2 common topics.

in the aggregated dataset, the first identified topic contains a mixture of words from the 2 common topics, while the remaining 4 are almost entirely comprised of words from the 4 spatially distinct topics.


The combination of multilingual BERT, monolingual BPEmb, and character embeddings is best overall (92.0) among models trained only on monolingual NER data.

this ensemble of contextual and non-contextual subword embeddings is inferior to MultiBPEmb (93.2), which was first trained on multilingual data from all languages collectively, and then separately finetuned to each language.


On one hand, we wish to guide training and prediction with neural networks using logic, which is non-differentiable.

we seek to retain the advantages of gradient-based learning without having to redesign the training scheme.


The statement A 1 ∧ B 1 → A 2 ∧ B 2 is cyclic with respect to the graph.

the statement for compiling conditional statements into differentiable statements that augment a given network.


Designing the distance function The key consideration in the compilation step is the choice of an appropriate distance function for logical statements.

the ideal distance function we seek is the indicator for the statement Z: since the function d ideal is not differentiable, we need smooth surrogates.


Because the implication is bidirectional in biconditional statement, it violates our acyclicity requirement in §3.1.

since the auxiliary neuron state does not depend on any other nodes, we can still create an acyclic sub-graph by defining the new node to be the distance function itself.


Our work is connected to active learning, for example, to approaches that use reinforcement learning to learn a policy for a dynamic active learning strategy (Fang et al., 2017), or to learn a curriculum to order noisy examples (Kumar et al., 2019), or to the approach of  who use imitation learning to select batches of data to be labeled.

the action space these approaches consider is restricted to the decision whether or not to select particular data and is designed for a fixed budget, neither do they incorporate feedback cost in their frameworks.


Using only full feedback (blue) as in standard supervised learning or learning from post-edits, the overall highest improvement can be reached (visible only after the cutoff of 80k edits; see Appendix A.2 for the comparison over a wider window of time).

it comes at a very high cost (417k characters in total to reach +0.6 BLEU).


All of these works follow a common paradigm: use an LSTM/GRU over the word sequence, extract contextual features at each time step, and apply some kind of pooling on top of that.

a few works adopt some different methods.


BERT is effective across domains, providing between 25% and 55% error reduction over the base neural parsers.

as for word embeddings, the pre-trained BERT representations do not generally provide a larger error reduction in out-of-domain settings than in in-domain (although a possible confound is that the BERT model is fine-tuned on the relatively small amount of in-domain treebank data, along with the other parser parameters).


When using BERT encoder representations, the Chart parser (with its unstructured decoder) and In-Order parser (with its conditioning on a representation of previously-constructed structure) obtain roughly comparable F1 (shown in the first two columns of Table 5), with In-Order better on seven out of nine corpora but often by slight margins.

these aggregate F1 scores decompose along the structure of the tree, and are dominated by the short spans which make up the bulk of any treebank.


Repeating this mechanism in consecutive layers allows for information to flow over long distances.

for each input token, each attention head scales linearly in memory and time in the context size, or attention span.


This indicates that lower layers in a Transformer model do not really require a long attention span in this particular task.

few attention heads in the higher layers have very long spans, exceeding several thousand.


Users usually have both long-term preferences and short-term interests.

existing news recommendation methods usually learn single representations of users, which may be insufficient.


They represented a session by using the topic distribution of browsed news in this session, and the representations of users were built from their session representations weighted by the time.

these methods heavily rely on manual feature engineering, which needs massive domain knowledge to craft.


We see that the explanatory power of L&M variables is lost after we additionally include H4N RE in a regression: all three L&M variables are not significant.

h4N RE continues to be significant in all experiments, with large standard coefficients.


The best L&M dictionary is again neg lm with standard coefficient 0.0472 and t = 3.30.

h4N RE has the highest explanatory value for volatility.


For example, Taylor (1953) studies the relation of cloze tests and readability.

to C-tests (Klein-Braley and Raatz, 1982), cloze tests remove whole words to produce a gap leading to more ambiguous solutions.


Another drawback of the naïve strategy is that it is difficult to control for the topic of the underlying text and in the worst case, the necessity to search through a whole corpus for selecting a fitting C-test.

to the naïve strategy, our proposed manipulation strategies are designed to be used in real time and manipulate any given C-test within 15 seconds at an acceptable quality.


In a more detailed analysis, we find two main sources of problems demanding further investigation: First, the difficulty prediction quality when deviating from DEF and second, the increasing ambiguity in harder C-tests.

it underestimates the d(T ) = 0.11 for T SEL,dec 4 (the same text used in figure 1), for which we found an actual error rate of 0.28.


In addition to the NTS system adapted to use our phrase table, we also tested a baseline which greedily applied the phrase table at all possible points in a sentence.

this system was ranked as least understandable more often than any other system.


Therefore, the result that audio-only MDRM performs better than textonly MDRM (1.412 vs. 1.431) may need careful interpretation as we have to stop audio-only model training early to prevent overfitting.

using both audio features and text features, the model usually converges in 20 epochs without over-fitting.


Our experiment results also consistently show that complex deep models such as bc-LSTM (Poria et al., 2017) or our proposed deep regression model outperform shallow models (such as SVR) by large margin in short-term prediction (τ =3 or 7).

the margin becomes smaller as we predict a relative long-term stock volatility (τ =15 or 30).


For example, comparing with tf-idf bag-of-words model at τ = 3, our MDRM reduces prediction error by 19.1% (1.371 vs. 1.695).

at τ = 30, the prediction error reduction is 12.8% (0.217 vs. 0.249).


At the time of pouring, there was no secret mechanism in place to inform the particular participant who brought the wine being poured.

given the fact that (1) it was common knowledge (each one knows it, if each one knows that the others know it, if each one knows that each one knows that the others know it, and so on) that each grape-region combo was assigned to no one or one individual for each session, and (2) all participants were required to bring a wine of the assigned grape-region that they know very well and ensure it of a classical style most representative of the grape and region, the task of detecting self-brought wines becomes trivial to our participants, and therefore the informing mechanism stands.


In practice, given a claim of interest, people may search for related articles from multiple sources and collect evidence for the claim; they can then determine the veracity of the claim by deciding whether the evidence found supports or refutes the claim.

most existing work that attempted to study trustworthiness of sources assumed that sources make assertions directly.


Here we allow P s can be different with H s , since providing true evidence for a true claim is more difficult than just providing a true claim.

considering that those all reflect the trustworthiness of s, we assume they share a similar distribution over sources in our problem.


It makes sense that evidence-based method (leveraging indirect assertions) can beat the claim-based method (leveraging direct assertions only) by using more information to reduce potential noise.

using sources and claims only is more noisy, especially with many bad information sources.


Based on the results of MJ-EVI, we can observe that simply calculating accuracy by estimated "correct" evidence cannot achieve a highly correlated estimation of sources: the entailment tool provides noisy evidence.

sim-Com, which directly counts estimated "correct" claims by MJ-EVI, can improve the estimation.


Among different factors, evidence contributes the most when estimating the veracity of claims, which can also help the estimation of the trustworthiness.

the usefulness of evidence highly depends on the quality of the NLP tool.


As noise increases, the accuracy, Pearson and Spearman score drop lower.

the JELTA method is consistently better than the alternatives.


(2016) use adversarial and information maximization objectives to produce interpretable latent representations that can be tweaked to adjust writing style for handwritten digits, as well as lighting and orientation for face models.

this problem is less explored in natural language processing.


(2017) propose to control the sentiment by using discriminators to reconstruct sentiment and content from generated sentences.

there is no evidence that the latent space would be disentangled by simply reconstructing a sentence.


The multi-task loss only ensures that the style space contains style information.

the content space might also contain style information, which is undesirable for disentanglement.


Our insight of combining the two auxiliary losses is a simple yet effective way of disentangling latent space.

j mul(s) and j adv(s) only regularize the style information, leading to gradual drop of content preserving scores.


Sophisticated sequence-to-sequence architectures (Gehring et al., 2017;Vaswani et al., 2017)   2018b), rescoring (Chollampatt and Ng, 2018a), iterative decoding strategies (Ge et al., 2018;Lichtarge et al., 2018), synthetic (Xie et al., 2018) and semi-supervised corpora (Lichtarge et al., 2018), and other task-specific techniques  has achieved impressive results on this task.

all prior work ignores document-wide context for GEC, and uses sentence-level models.


They also use ROUGE to make extraction labels and to provide rewards in their RL training phase.

our model extracts multiple sentences and rewrites them together into a single subject line.


Therefore, an ecologically valid model of semantic change should show that the words change more as the time interval for comparison increases, for the vocabulary as a whole.

if a model captures stochastic fluctuations in the words' vectors instead of true semantic change, then such a shift in the distribution will be less prominent.


For the semantic change words (upper plot), all four models show a noticeable peak when the new sense was first injected (step 2), followed by a steady decrease in acd until step 6.

the stable words only show the steady decrease starting from step 1, without any noticeable peaks.


Compared with the texts in the Essays domain, some texts in the Travel Guides domain state much about the histories and anecdota of the tourist attractions.

besides the historical stories, for instance, the text "Good for the health is just one of the many magical qualities that are attributed to these beautiful emerald-green or turquoise stones."


Previous research on affective analysis focuses on text modality (Liu and Zhang, 2012;Cambria and Hussain, 2015), which is a hot research topic in the NLP community.

recent research suggests that information from text is not sufficient for mining opinion of humans (Poria et al., 2017a;D'Mello and Kory, 2015;Cambria, 2016), espe-cially under the situation where sarcasm or ambiguity occurs.


Previous work has shown that earnings calls disclose more information than company filings alone (Frankel et al., 1999) and influence investor sentiment in the short term (Bowen et al., 2002).

recently company executives and investors have questioned their value (Koller and Darr, 2017;Melloy, 2018).


This operation is performed iteratively, allowing the information to be modified and propagated across multiple links as the number of iterations increases.

to most multi-task learning schemes which share information through learning a common feature representation, IMN not only allows shared features, but also explicitly models the interactions between tasks through the message passing mechanism, allowing different tasks to better influence each other.


proposed to model the problem as a sequence labeling task with a unified tagging scheme.


This again indicates that domain-specific knowledge has already been captured by domain embeddings, while knowledge obtained from DD and DS via parameter sharing could be redundant in this case.


As observed in example 2, both PIPELINE and INABSA extract "Pizza".

since no opinion is expressed in the given sentence, "Pizza" should not be considered as an aspect term.


There are also other multimodal emotion and sentiment analysis datasets, such as MOSEI , MOSI (Zadeh et al., 2016b), and MOUD (Pérez-Rosas et al., 2013), but they contain individual narratives instead of dialogues.

emotionLines (Chen et al., 2018) is a dataset that contains dialogues from the popular TV-series Friends with more than two speakers.


MOSI (Zadeh et al., 2016b), MO-SEI , and MOUD (Pérez-Rosas et al., 2013) are such examples that have drawn significant interest from the research community.

iEMOCAP and SEMAiNE are two popular dyadic conversational datasets where each utterance in a dialogue is labeled by emotion.


that are aligned to the components of MELD.

mELD is different in terms of both complexity and quantity.


Multimodal fusion helps in improving the emotion recognition performance by 3%.

multimodal classifier performs worse than the textual classifier in classifying sadness.


The PRET+MULT framework proposed in (He et al., 2018) is a successful attempt by adopting pre-training and multi-task learning approaches.

their model only shares shallow embedding and LSTM layers between ASC and DSC (document-level sentiment classification) tasks.


the method in (He et al., 2018) is based on an existing AT-LSTM model (Wang et al., 2016b), whereas our framework is a totally new one which employs capsule network with carefully designed strategies for ASC tasks.


Humans can easily identify the positive polarity towards aspect [screen].

the single-task variant TransCap{S} and most baselines give a false negative prediction.


In aspect-level sentiment classification (ASC), it is prevalent to equip dominant neural models with attention mechanisms, for the sake of acquiring the importance of each context word on the given aspect.

such a mechanism tends to excessively focus on a few frequent words with sentiment polarities, while ignoring infrequent ones.


By doing so, the weights of words extracted first will be reduced, and those of words extracted later will be increased, avoiding the over-fitting of high-frequency context words with sentiment polarities and the under-fitting of lowfrequency ones.

for the words in s m (x) with misleading effects on the sentiment prediction of x, we want to reduce their effects and thus directly set their expected weights as 0.


Trabelsi and Zaïane (2015) used an augmented LDA to automatically extract coherent words and phrases describing arguing expressions and apply constrained cluster-ing to group similar viewpoints of topics.

to previous work, we apply argument clustering on a dataset containing both relevant and non-relevant arguments for a large number of different topics which is closer to a more realistic setup.


Agglomerative clustering is a strict partitioning algorithm, i.e., each object belongs to exactly one cluster.

an argument can address more than one aspect of a topic, therefore, arguments could belong to more than one cluster.


Humans are better than the proposed BERT-model at estimating the pairwise similarity of arguments.

when combined with a clustering method, the performances are on-par.


Our contributions are four-fold: (1) We propose a MTL approach to coherence assessment and compare it against a number of baselines.

we experimentally demonstrate that such a framework allows us to exploit more effectively the inter-dependencies between the two prediction tasks and achieve state-of-the-art results in predicting document-level coherence; (2) we assess the extent to which the information encoded in the network generalizes to different domains and prediction tasks, and demonstrate the effectiveness of our approach not only on standard binary evaluation tasks on the wall Street Journal (wSJ), but also on more realistic tasks involving the prediction of varying degrees of coherence in people's everyday writing; (3) to existing work that has only investigated the impact of a specific set of grammatical roles (i.e., subject and object) on coherence, we instead investigate a large set of GR types, and train the model to predict the type of role dependents participate in.


For dialogues with multiple interlocutors, extraction of their discourse structures could provide useful semantic information to the "downstream" models used, for example, in the production of intelligent meeting man-agers or the analysis of user interactions in online fora.

despite considerable efforts on computational discourse-analysis (Duverle and Prendinger, 2009;Joty et al., 2013;Ji and Eisenstein, 2014;Surdeanu et al., 2015;Yoshida et al., 2014;Li et al., 2016), we are still a long way from usable discourse models, especially for dialogue.


Taking into account this structure has shown to help many NLP end tasks, including summarization (Hirao et al., 2013;Durrett et al., 2016), machine translation (Joty et al., 2017), and sentiment analysis (Ji and Smith, 2017).

annotating discourse requires considerable effort by trained experts and may not always yield a structure appropriate for the end task.


Structured attention at the sentence level helps performance for all except WQ, where no form of attention helps.

structured attention at the document level yields mostly negative results, in contrast to the improvements reported in L&L.


In this paper, we frame zero-shot learning as a challenge for pragmatic modeling and explore zero-shot reference games, where a speaker needs to describe a novel-category object in an image to an addressee who may or may not know the category.

to standard reference games, this game explicitly targets a situation where relatively common words like object names are likely to be more inaccurate than other words like e.g.


it is not aware that it encounters a novel category and frequently generates names of known categories encountered during training.

even in this simple model, we find a certain portion of output expressions that do not contain any name (e.g.


Although recent end-to-end neural coreference models have advanced the state-of-the-art performance for coreference resolution, they are still trained with heuristic loss functions and make a sequence of local decisions for each pair of mentions.

as studied in Clark and Manning (2016a); Yin et al.


While it is true that coherence is a property of a passage as a whole, capturing long-term dependencies in sequences remains a fundamental challenge when training neural networks in practice (Trinh et al., 2018).

it is plausible that much of global coherence can be decomposed into a series of local decisions, as demonstrated by foundational theories such as Centering Theory.


Although Romanian and Moldavian are supposed to be hard to discriminate, since Romania and the Republic of Moldova share the same literary standard (Minahan, 2013), the empirical results seem to point in the other direction, to our surprise.

we should note that the high accuracy rates attained by the proposed classifiers can be explained through a combination of two factors.


We note that the goal of this work was not to achieve state-of-the-art results on English WSD compared to manually-annotated corpora.

performing competitively on standard benchmarks represents one step further towards getting rid of the limitation imposed by resources like SemCor.


Parallel corpora were exploited also in the more recent work of Taghipour and Ng (2015, OMSTI) 11 , who presented a semi-automatic approach that creates a novel semantically-annotated dataset by leveraging the manual effort made to align senses across different languages.

recent methods have been able to fully automatise the whole process while simulta-neously producing high-quality resources.


We find that overfitting to BLI may severely hurt downstream performance, warranting the coupling of BLI experiments with downstream evaluations in order to paint a more informative picture of CLE models' properties.

2 Projection-Based CLEs: Methodology to more recent unsupervised models, CLE models typically require bilingual signal: aligned words, sentences, or documents.


The results highlight VECMAP (Artetxe et al., 2018b) as the most robust choice among unsupervised models: besides being the only model to produce successful runs for all language pairs, it also significantly outperforms other unsupervised models-both when considering all language pairs and only the subset where other models produce successful runs.

vECMAP still performs worse (p ≤ 0.0002) than PROC-B (trained on only 1K pairs) and all supervised models trained on 3K or 5K word pairs.


Given the importance of SP, the automatic acquisition of SP has become a well-known research subject in the SP Evaluation Set #R #W #P (McRae et al., 1998) 2 641 821 (Keller and Lapata, 2003) 3 571 540 (Padó et al., 2006) 3 180 207 SP-10K 5 2.5K 10K NLP community.

current SP acquisition models are limited based on existing evaluation methods.


As shown in Table 7, almost 50% of SP pairs in the perfect group are covered by OMCS.

only about 6% of SP pairs from the impossible group are covered.


For the perfect group, we find that human-defined commonsense triplets often have neatly corresponding SP pairs.

for the impossible group, SP pairs are covered by OMCS either because of incidental overlap with a non-keyword, e.g., 'child' in 'child wagon', or because of the low quality of some OMCS triplets.


But as we only label 4,000 multihop pairs, the overall coverage is limited.

automatic SP acquisition method PP can cover more questions, but the precision also drops due to the noise of the collected SP knowledge.


Currently, the most popular evaluation method for SP acquisition is the pseudodisambiguation (Ritter et al., 2010;de Cruys, 2014).

pseudo-disambiguation can be easily influenced by the aforementioned noisiness of evaluation corpora and cannot represent ground truth SP.


If we only consider is distracted, without also considering correct predictions, we might conclude that the distractor hypothesis is correct: the 192 instances in the group are all cases where BiDAF predicts a wrong span that has the same entity type as the ground truth, and the group accounts for 5.7% of all BiDAF errors.

looking at the groups in succession reveals a different, and more complete story: BiDAF predicts the exact correct span (exact match) 68% of the time overall, which rises to 80% when the ground truth is an entity.


The author of D2 observed examples like Figure 8(c) in his samples, but decided ultimately that what mattered was just the returned short text, not the span index.

d1's author carefully refined his initial query precisely to rule out cases like Figure 8(c).


(2018) made a similar attempt to balance the trade-off in Slice Finder, a framework that uses statistical techniques to identify large and interpretable slices that models perform poorly on.

their purely automated data slicing does not allow users to customize groups based on their own hypotheses.


Two widely used metrics F1 and AUC are used in our experiments.

some relational facts present in both the training and dev/test sets, thus a model may memorize their relations during training and achieve a better performance on the dev/test set in an undesirable way, introducing evaluation bias.


On the one hand, jointly predicting the evidence provides better explainability.

identifying supporting evidence and reasoning relational facts from text are nat-  urally dual tasks with potential mutual enhancement.


(2018b) further combine external recommendations with human annotation to build large-scale high-quality datasets.

these RE datasets limit relations to single sentences.


Recently, some document-level RE datasets have also been constructed.

these datasets are either constructed via distant supervision Peng et al., 2017) with inevitable wrong labeling problem, or limited in specific domain Peng et al., 2017).


However, these datasets are either constructed via distant supervision Peng et al., 2017) with inevitable wrong labeling problem, or limited in specific domain Peng et al., 2017).

docREd is constructed by crowd-workers with rich information, and is not limited in any specific domain, which makes it suitable to train and evaluate general-purpose document-level RE systems.


For each idiom, we picked up the 20 most similar idioms whose embedding similarity score to the input idiom is less than some threshold.

according 可是有一个时期大家#idiom-0#，不大敢露面， 只有她一个人倚在阳台上看排队的兵走过。 there was a period when everyone #idiom-0# and was scared to show up.


Indeed, perplexity is still frequently used to evaluate models, and each of the models mentioned in the previous section, including CopulaLDA-designed to improve local topic quality-uses perplexity to evaluate the model.

while held-out perplexity can test the generalization of predictive models, it is negatively correlated with human evaluations of global topic quality (Chang et al., 2009).


Leveraging this intuition, where rank(w i , z i ) is the rank of i th word w i in its assigned topic z i when sorted by probability, we define AVGRANK as With this evaluation the lower bound is 1, although this would require that every token be assigned to a topic for which its word is the mode.

this is only possible if the number of topics is equal to the vocabulary size.


Our players obtain an aggregated F of 90.5, which is very high.

collecting judgements from real players tends to be slower than using a crowdsourcing service.


We can see that operating out of their original domains, the automated pipelines, while still outperforming the Stanford pipeline by around 4 percentage points, do not outperform aggregated users.

they do appear to serve well as agents to train participants to perform annotations, as participants annotate to a high level of accuracy.


Confidence in annotators is not modelled.

it has been extended to incorporate the reliability of the annotator with a similar method that also combines Expectation Maximization with CRF in an NER and NP chunking task (Rodrigues et al., 2014  In this paper, we presented a hybrid mention detection method combining state-of-the-art automatic mention detectors with a gamified, twoplayer interface to collect markable judgements.


It is because this slot usually has a large number of possible values that is hard to recognize.

number-related slots such as arrive by, people, and stay usually have the lowest error rates.


use distributional representation learning to leverage semantic information from word embeddings to and resolve lexical/morphological ambiguity.


Xu and Hu (2018) use the index-based pointer network for different slots, and show the ability to point to unknown values.

many of them require a predefined domain ontology, and the models were only evaluated on single-domain setting (DSTC2).


Learning features of the Smart Home category helps overcome such conflicts.

a word in different tasks in the same domain can still have different slot types.


(2012) developed a nonparametric Bayesian model to learn task subspaces and features jointly.

with the advent of deep learning, MTL with deep neural networks has been successfully applied to different applications (Zhang et al., 2018;Masumura et al., 2018;Fares et al., 2018;Guo et al., 2018).


A flat MR will struggle to represent 1) the correspondence of arguments to dialog acts; 2) what attributes to group and contrast and 3) semantic equivalence of arguments like date time1 and date time2.

our MRs ease discourse-level learning and encourage reuse of arguments across multiple dialog acts.


Recently, a joint model (Zhang et al., 2019b) was proposed to connect the contextual information and human-designed features together for pronoun coreference resolution task (with gold mention support) and achieved the state-of-the-art performance.

their model still requires the complex features designed by experts, which is expensive and difficult to acquire, and requires the support of the gold mentions.


Considering that neural models are intensive datadriven and normally restricted by data nature, they are not easily applied in a cross-domain setting.

if a model is required to perform in real applications, it has to show promising performance on those cases out of the training data.


Their results proved that such knowledge is helpful when appropriately used for coreference resolution.

external knowledge is often omitted in their models.


2 The development of new large-scale datasets has led to a flurry of various neural network architectures for solving NLI.

recent work has found that many NLI datasets contain biases, or annotation artifacts, i.e., features present in hypotheses that enable models to perform surprisingly well using only the hypothesis, without learning the relationship between two texts (Gururangan et al., 2018;Poliak et al., 2018b;Tsuchiya, 2018).


In a sense this is counter-intuitive, since p θ is being trained to unlearn bias, while p φ,θ is being trained to learn it.

if the models are trained separately, they may learn to co-adapt with each other (Elazar & Goldberg, 2018).


The only exception seems to be SCITAIL, where we do not improve despite it having different biases than SNLI.

when we strengthen α and β (below), Method 1 outperforms the baseline.


However, removing such attributes from text representations may be difficult (Elazar & Goldberg, 2018).

to this line of work, our final goal is not the removal of such attributes per se; instead, we strive for more robust representations that better transfer to other datasets, similar to Li et al.


(2018) incorporate external linguistic resources and use a GAN-style framework to adversarially train robust NLI models.

we do not use external resources and we are interested in mitigating hypothesisonly biases.


ADD-ONE-RTE and the recast datasets also model NLI as a binary prediction task.

their label sets are ENTAILED and NOT-ENTAILED.


Existing FV methods formulate FV as a natural language inference (NLI) (Angeli and Manning, 2014) task.

they utilize simple evidence combination methods such as concatenating the evidence or just dealing with each evidence-claim pair.


Some workers still had frequent low agreement with the majority.

in most cases we obtained a clear majority annotation.


The "voted" version corrects the reference, when one of the three scribes misses the annotation.

when two scribes pick different valid labels and the third misses them, the "voted" reference is not better than the single reference.


A counselor might therefore seek to adapt training principles to their own personality, finding a voice that distinguishes them from other counselors (betweencounselor diversification).

change is far from guaranteed, and several forces potentially counteract the two vectors of change outlined above.


As in the mental health setting, these studies have largely considered the behavior of interlocutors in a single conversation, tying it to outcomes such as persuasion, problemsolving and incivility (Curhan and Pentland, 2007;Rosenthal and McKeown, 2015;Wang et al., 2017;Zhang et al., 2018), or to social correlates such as power, gender and age (Gonzales et al., 2010;Ireland et al., 2011;Danescu-Niculescu-Mizil et al., 2012;Nguyen et al., 2013;Prabhakaran et al., 2014).

our work seeks to chart the development of linguistic tendencies of individuals as they engage in many conversations.


This could suggest that as counselors encounter more diverse situations, they become linguistically better equipped to approach them in a specialized fashion.

in conversational components which have functions that generalize across texter problems there is no systematic increase in diversity (hello), and there is even a slight decrease (goodbye)-perhaps indicating that counselors develop routines to close the conversation.


With shared contextual encoder representation consisting of character and word embedding based models, the proposed architecture provides an effective solution for knowledge transfer across tasks, thus consolidating the ability to perform well on unseen text.

this proposed ar- chitecture is not scalable, the number of decoders scales linearly with the number of attributes.


Indeed, we show a very high performance of the baseline by length over UPKCon-vArg in §5.1.

a data set that includes only evidence poses a more challenging task.


In addition, the pre-processing step which generates the huge set of linguistic features, used by Habernal and Gurevych (2016a) and Simpson and Gurevych (2018), takes a lot of time and is not suitable for many languages.

our network does not depend on task specific features, and still achieves state of the art results on the task of argument convincingness classification and ranking.


We find there words related to argumentation (argue, claim), or studies and polls (found,  ures (DR., Clinton, W. Bush) or court orders (supreme, v. 8 ).

when subtracting the true convincing distribution from the true nonconvincing distribution (Figure 2b) one gets opinion words (support, opposes, vote), partial change (reduce, amend, part), non-emphasized actions (said, proposed, concern).


If we argue against the DC (Con stance), we may choose any plausible alternative as our EC, following a line of argument such as "EC is a better alternative to DC".

when we argue in favor of DC (Pro stance), the typical argument changes to "if we don't choose DC then we are left with EC", which requires EC to be the "default" alternative to DC.


Among others, emotions are encoded in both lexical and acoustic information where each modality contributes to the overall emotional state of a given speaker.

in some situations, one modality can be more insightful to derive emotions than the other.


Previous studies usually extract features from the modalities in independent modules, and then they concatenate the corresponding utterance representations from the acoustics and lexical features to feed into the next layers of their models.

we argue that a more natural way to understand emotions is to align lexical and acoustic information, which simulates the way humans process both modalities simultaneously.


For negative sampling in the contrastive loss function, we empirically found that using anger with happiness and neutral with sadness generally worked well since the acoustic patterns are similar.

we saw some informative pairs when happiness and anger were coupled with neutral.


This framework was followed by most of the recent studies in this field Gui et al., 2016a;Li et al., 2018;Xu et al., 2019;Yu et al., 2019).

there are two shortcomings in the current ECE task.


In addition to a document as the input, ECE needs to provide annotated emotion at first before cause extraction.

the output of our ECPE task is a pair of emotion-cause, without the need of providing emotion annotation in advance.


Till now, two component Bi-LSTM at the upper layer are independent to each other.

as we have mentioned, the two sub-tasks (emotion extraction and cause extraction) are not mutually independent.


It is clear that by removing the emotion annotations, the F1 score of CANN drops dramatically (about 34.69%).

our method does not need the emotion annotations and achieve 65.07% in F1 measure, which significantly outperforms the CANN-E model by 27.1%.


On the one hand, its goal is not direct.

the mistakes made in the first step will affect the results of the second step.


This kind of argument can be made, mutatis mutandis, when debating quite different topics, such as legalizing organ trade, or banning pornography.

it is not always relevant when debating whether to legalize or ban something; For example, when debating whether to legalize polygamy or ban breastfeeding in public, the black market argument seems less appropriate.


A somewhat similar approach is suggested in work of Walton andGordon (2017, 2018), where arguments are constructed from a database (Reed and Rowe, 2004) of smaller argumentative building blocks.

these building blocks are topic-specific and can not readily provide arguments for topics not in the database.


We observe that LexRank tends to extract long and comprehensive sentences that yield high graph centrality; the abstractive pointer-generator networks, despite the promising results, can sometimes fail to generate meaningful summaries (e.g., "a third of all 3-year-olds • • • have been given to a child").

our DPP method is able to select a balanced set of representative and diverse summary sentences.


It was also applied for optimizing the neural summarization model for headline generation with respect to ROUGE (Ayana et al., 2017), which is based on an overlap of words with reference summaries (Lin, 2004).

how to use MRT under a length constraint was an open problem; thus we propose a global optimization under length constraint (GOLC) for neural summarization models.


Models trained with GOLC showed better ROUGE scores than those of maximum loglikelihood based methods while generating summaries satisfying the length constraint.

to the approximately 20% and 50% of overlength summaries generated by the other state-ofthe-art models, our method only generated 6.70% of overlength summaries on CNN/Daily and 7.8% on long summary of Mainichi while improving ROUGE scores.


Each Q θ (y − |x) for summaries shorter than the length constraint increases because∆(y, y − ) < 0.

each Q θ (y + |x) for overlength summaries decreases be-cause∆(y, y + ) > 0.


Since short summary lengths distribute approximately 10 to 17, lengths of summaries generated by PG (MLE), which does not has capability of controlling summary length, are less than the length constraint 17.

pG w/ LE (MLE) and pG w/ LE (GOLC) tend to generate as the same length of summary as the length constraint.


Specifically, these work instantiate their encoder-decoder framework by choosing recurrent neural networks (Cheng and Lapata, 2016;Nallapati et al., 2017;Zhou et al., 2018) as encoder, auto-regressive decoder (Chen and Bansal, 2018;Jadhav and Rajan, 2018;Zhou et al., 2018) or non auto-regressive decoder (Isonuma et al., 2017;Narayan et al., 2018;Arumae and Liu, 2018) as decoder, based on pre-trained word representations (Mikolov et al., 2013;Pennington et al., 2014).

how to use Transformer in extractive summarization is still a missing issue.


5, we can find that contextindependent word representations can not contribute much to current models.

when the models are equipped with BERT, we are excited to observe that the performances of all types of architectures are improved by a large margin.


For instance, structural features like centrality and repetitions are still among the most used proxies for Importance (Yao et al., 2017;Kedzie et al., 2018).

such features just correlate with Importance in standard datasets.


Information theory provides the means to rigorously discuss the abstract concept of information, which seems particularly well suited as an entry point for a theory of summarization.

information theory concentrates on uncertainty (entropy) about which message was chosen from a set of possible messages, ignoring the semantics of messages (Shannon, 1948).


A simple model could be the uniform distribution over known information: n if the user knows ω i , and 0 otherwise.

k can control other variants of the summarization task: A personalized k p models the preferences of a user by setting low probabilities to the semantic units of interest.


Single document summarization (SDS) systems have benefited from advances in neural encoder-decoder model thanks to the availability of large datasets.

multidocument summarization (MDS) of news articles has been limited to datasets of a couple of hundred examples.


The three source documents discuss the same event and contain overlaps in content: the fact that Meng Wanzhou was arrested is stated explicitly in Source 1 and 3 and indirectly in Source 2.

some sources contain information not mentioned in the others which should be included in the summary: Source 3 states that (Wanzhou) is being sought for extradition by the US while only Source 2 mentioned the attitude of the Chinese side.


(2018) trains abstractive sequence-to-sequence models on a large corpus of Wikipedia text with citations and search engine results as input documents.

no analogous dataset exists in the news domain.


A potential problem is that the applicability of a high-quality operator is often restricted.

one can increase the application probability by selecting not only the top-ranked n-gram but a small number of different near-top n-grams.


This indicates that the meaning becomes a strong clue to assign a domain to the document.

in the implicit semantic space created by using the neural language model such as the Word2Vec, a word is represented as one vector even if it has several senses.


A restriction of the subject domain makes the problem of polysemy less problematic.

even in texts from a restricted subject domain such as Wall Street Journal corpus (Douglas and Janet, 1992), one encounters quite a large number of polysemous words.


Most of these approaches demonstrated that neural network models are powerful for learning effective features from textual input.

most of them for learning word vectors only allow a single context-independent representation for each word even if it has several senses.


Among the load of works like assigning reviewers, ensuring timely receipt of reviews, slot-filling against the non-responding reviewer, taking informed decisions, communicating to the authors, etc., editors/program chairs are usually overwhelmed with many such demanding yet crucial tasks.

the major hurdle lies in to decide the acceptance and rejection of the manuscripts based on the reviews received from the reviewers.


Moreover, even a preliminary study into the inners of the peer review system is itself very difficult because of data confidentiality and copyright issues of the publishers.

the silver lining is that the peer review system is evolving with the likes of OpenReviews 2 , author response periods/rebuttals, increased effective communications between authors and reviewers, open access initiatives, peer review workshops, review forms with objective questionnaires, etc.


(2018) relies on elementary handcrafted features extracted only from the paper; does not consider the review features whereas we include the review features along with the sentiment information in our deep neural architecture.

we also find that our approach with only Review+Sentiment performs inferior to the Paper+Review method in Kang et al.


Furthermore, the corruption mechanism is nothing other than traditional dropout on the input layer.

coupled with the word2vec-style loss and training methods, this paper offers little on the novelty front.it is very efficient at generation time, requiring only an average of the word embeddings rather than a complicated inference step as in Doc2Vec.


With further exploration, we aim to mould the ongoing research to an efficient AI-enabled system that would assist the journal editors or conference chairs in making informed decisions.

considering the sensitivity of the topic, we would like to further dive deep into exploring the subtle nuances that leads into the grading of peer review aspects.


The new end-to-end speech recognition approach (Graves et al., 2006;Graves and Jaitly, 2014;Hannun et al., 2014;Miao et al., 2015;Chorowski et al., 2015;Chan et al., 2016; integrates all available information within a single neural network model, allows to make fusing conversational-context information possible.

these are limited to encode only one preceding utterance and learn from a few hundred hours of annotated speech corpus, leading to minimal improvements.


Several recent studies have considered to embed a longer context information within a end-toend framework Kim and Metze, 2019).

with our method which can learn a better conversationalcontext representation with a gated network that incorporate external word/sentence embeddings from multiple preceding sentence history, their methods are limited to learn conversationalcontext representation from one preceding sentence in annotated speech training set.


Table 1: SEQ vs. CAMB system results on words only and on words and phrases systems achieve similar results: the SEQ model beats the CAMB model only marginally, with the largest difference of +1.05% on the WIKINEWS data.

it is worth highlighting that the CAMB system does not perform any phrase classification per se and simply marks all phrases as complex.


The topic information of news is critical for learning accurate news and user representations for news recommendation.

it is not considered in many existing news recommendation methods.


Thus, the performance improves when λ increases from 0.

when λ becomes too large, the performance of our approach declines.


Some graphic design applications such as Adobe Spark 1 perform automatic text layout using templates that include images and text with different fonts and colors.

their text layout algorithms are mainly driven by visual attributes like word length, rather than the semantics of the text or the user's intent, which can lead to unintended emphasis and the wrong message.


Keyword detection mainly focuses on finding important nouns or noun phrases.

social media text is much shorter, and users tend to emphasize a subset of words with different roles to convey specific intent.


From Figure 2, it can be seen that, despite small improvements, different featurizations provide similar APS values.

focusing more on the early parts of the PRC, i.e., high precision, we can see that there is a significant improvement of recall.


Up to now, the model is completely agnostic of sequence positions.

position information is crucial in natural language, so a mechanism to represent such information in the model is needed.


A simple way to realize self-attentional modeling for lattice inputs would be to linearize the lattice in topological order and then apply the above model.

such a strategy would ignore the lattice structure and relate queries to keys that cannot possibly appear together according to the lattice.


Below, we demonstrate that the binary masking approach ( § 4.1.1) is biased such that computed representations are impacted by path duplication.

the probabilistic approach ( § 4.1.2) is invariant to path duplication.


For the sequential lattice with binary mask, it is: Here, C is the softmax normalization term that ensures that exponentiated similarities sum up to 1.

the lattice with duplication results in a doubled influence of v a : The probabilistic approach yields the same result as the binary approach for the sequential lattice (Equation 11).


For lexical cohesion, Wong and Kit (2012) count the ratio between the number of repeated and lexically similar content words over the total number of content words in a target document.

guillou (2013); Carpuat and Simard (2012) find that translations generated by a machine translation system tend to be similarly or more lexically consistent, as measured by a similar metric, than human ones.


This paradigm is widely used in Multi-NMT systems due to simple implementation and convenient deployment.


As shown in part II, Multi-NMT Baselines underperform the NMT Baselines on all cases.

our method performs better than NMT Baselines, and it achieves the improvement up to 1.76 BLEU points on Nl→It translation task.


Both statistical (Lample et al., 2018b;Artetxe et al., 2018b) and neural (Artetxe et al., 2018c;Lample et al., 2018a) MT approaches were proposed which are promising directions to overcome the data sparsity problem.

various issues of the approaches still have to be solved, e.g., better word reordering during translation or tuning system parame-ters.


our conservative approach also misses true parallel pairs resulting in a significant drop in recall.

we argue that precision is more important for downstream tasks, since noise in the data often hurts performance.


In addition, it mentions US presidency related entities twice, once as Clinton and once confusing it with Obama.

by using parallel sentences the results are more fluent and accurate.


The empirical results in this section show that the quality of pre-trained UBWE is important to UNMT.

the quality of UBWE decreases significantly during UNMT training.


Because there are many shared subwords in these similar language pairs, this method achieves better performance than other UBWE methods.

this initialization does not work for distant language pairs such as English-Japanese and English-Chinese, where there are few shared subwords.


This indicates that a larger dictionary size helps estimate a better UBWE agreement.

larger dictionary size did not always obtain a higher BLEU as shown in Table 3.


A simple but commonly used solution is pivoting: we first translate from Spanish to English, and then from English to French with two separately trained single-pair models or a single multilingual model.

it comes with two drawbacks: (1) at least 2× higher latency than that of a comparable direct translation model; (2) the models used in pivot-based translation are not trained taking into account the new language pair, making it difficult, especially for the second model, to cope with errors created by the first model.


After training, one obtains the optimized W and then easily infers word alignment for a test sentence pair x, y via the MAP strategy as defined in equation 3 by setting Note that if word embeddings and hidden units learned by NMT models capture enough information for word alignment, the above method can obtain excellent word alignment.

because the dataset for supervision in training definitely include some data intrinsic word alignment information, it is unclear how much word alignment is only from NMT models.


Previous experiments mainly consider the alignment for the reference, and show that EAM is better at aligning a reference word to source words than PD.

in order to better understand the translation process of a NMT model, it is helpful to analyze the alignment of real translations derived from the NMT model itself.


The main reason is that EAM relies on an external aligned dataset with supervision from statistical word aligner FAST ALIGN, and thus the characteristic of its alignment result are similar to that of FAST ALIGN, leading to the understanding biased to FAST ALIGN.

pD only relies on prediction from a neural model to define the relevance, it has been successfully used to understand and interpret a neural model (Zintgraf et al., 2017).


As shown in Figure 4(a), as a functional word, 'a' should not be aligned to any source word.

in Figure 4(b) PD incorrectly aligned 'a' to 'háng hǎi jīa' by only considering the contributions from the source side, and this leads to a misunderstanding for why 'a' is translated.


(2017) first developed a nonautoregressive NMT system which produces the outputs in parallel and the inference speed is thus significantly boosted.

it comes at the cost that the translation quality is largely sacrificed since the intrinsic dependency within the natural language sentence is abandoned.


Several others use reinforcement learning (RL) to develop an agent to predict read and write decisions (Satija and Pineau, 2016;Gu et al., 2017;Alinejad et al., 2018).

due to computational challenges, they pre-train an NMT model on full sentences and then train an agent that sees the fixed NMT model as part of its environment.


At the lowest comparable latency (4 tokens), MILk is 1.5 BLEU points ahead of wait-k. EnFr is a much easier language pair: both MILk and wait-k maintain the BLEU of full attention at lags of 10 tokens.

we were surprised to see that this does not mean we can safely deploy very low ks for wait-k; its quality drops off surprisingly quickly at k = 8 (DAL=8.4, BLEU=39.8).


The omitted BLEU/DAL pairs are 19.5/2.5 for DeEn and 28.9/2.9 for EnFr, both of which trade very large losses in BLEU for small gains in lag.

wait-k's ability to function at all at such low latencies is notable.


We observe that, in the early stage of training, the validation loss of RNN decreases faster than Transformer.


By considering the reasoning patterns, one can discover that Luc Besson could speak English following a reasoning logic that Luc Besson directed Léon: The Professional and this film is in English indicates Luc Besson could speak English.

most existing GNNs can only process multi-hop relational reasoning on pre-defined graphs and cannot be directly applied in natural language relational reasoning.


3 that as the number of layers grows, the curves get higher and higher precision, indicating considering more hops in reasoning leads to better performance.

the improvement of the third layer is much smaller on the overall distantly supervised test set than the one on the dense subset.


In practical scenario, relation extraction needs to first identify entity pairs that have relation and then assign a correct relation class.

the number of non-relation entity pairs in context (negative instances) usually far exceeds the others (positive instances), which negatively affects a model's performance.


Strictly speaking, most existing studies of relation extraction treat the task as relation classification.

relation extraction often comes with an extremely imbalanced dataset where the number of non-relation entity pairs far exceeds the others, making it a more challenging yet more practical task than relation classification.


One possible reason is that the all models are closely related to each other.

how they   affect each other in this joint settings is still an open question.


For S3, our "GCN" identifies a PHYS relation between "[units] PER " and "[captial] GPE ",  while the "NN" does not find this relation even the entities are correct.

both models do not identify the relation ART between "[units] PER " and "[weapons] WEA ".


The BERT model has been successfully applied to various NLP tasks.

the final prediction layers used in the original model is not applicable to MRE tasks.


Note that the system (Rotsztejn et al., 2018) integrates many techniques like feature-engineering, model combination, pretraining embeddings on in-domain data, and artificial data generation, while our model is almost a direct adaption from the ACE architecture.

compared to the top singlemodel result (Luan et al., 2018), which makes use of additional word and entity embeddings pretrained on in-domain data, our methods demonstrate clear advantage as a single model.


Adding structured prediction layer to BERT (i.e., BERT SP ) also leads to a similar amount of improvement.

the gap between BERT SP method with and without entity-aware attention is small.


In a supervised setting, discriminative approaches, such as deep neural network classifiers, have demonstrated substantial improvement.

these models are hard to train without supervision, and the currently proposed solutions are unstable.


Most relation extraction systems need to be trained on a labeled dataset.

human annotation is expensive, and virtually impractical when a large number of relations is involved.


Later, the OpenIE approaches (Banko et al., 2007;Angeli et al., 2015) relied upon the hypothesis that the surface form of the relation conveyed by a sentence appears in the path between the two entities in its dependency tree.

these latter works are too dependent on the raw surface form and suffer from bad generalization.


paraphrases, relation aliases, and entity types (Vashishth et al., 2018).

we observe that these models are often biased towards recognizing a limited set of relations with high precision, while ignoring those in the long tail (see Section 5.2).


PCNN+ATT has the highest average precision at 94.3%, 3% higher than the 91.2% of RE-SIDE and 5% higher than our model.

we see that this is mainly due to PCNN+ATT's very high P@100 and P@200 scores.


Similarly, our approach predicts a larger set of distinct relation types with high confidence among the top predictions.

to RESIDE, which uses explicitly provided side information and linguistic features, our approach only utilizes features implicitly captured in pre-trained language representations.


Instead, we use a recently released sentence-level test set (Ren et al., 2017) for evaluation.

there also exist several problems in this test set (see Sec.


To automatically obtain a large training dataset, DS has been proposed (Mintz et al., 2009).

dS also introduces noisy data, making dS-based RC more challenging.


They use Bi-LSTM on each sentence for automatic feature learning, and the extracted hidden features are shared by a sequential entity tagger and a shortest dependency path relation classifier.

while introducing shared parameters for joint entity recognition and relation extraction, they must still pipeline the entity mentions predicted by the tagger to form mention pairs for the relation classifier.


(2018) improved the performance of the vanilla LSTM (Hochreiter and Schmidhuber, 1997) by utilizing RL to discover structured representations and  interpreted the sentiment prediction of neural models by employing RL to find the decision-changing phrases.

nRE models are unique because we only care about the semantic inter-entity relation mentioned in the sentence.


We conjecture that this behavior is due to the gap between maximizing the likelihood of the NRE model and the ground-truth instance selection.

dIAG-NRE can contribute both stable and interpretable improvements with the help of human-readable patterns.


The extracted named entities can benefit various subsequent NLP tasks, including syntactic parsing (Koo and Collins, 2010), question answering (Krishnamurthy and Mitchell, 2015) and relation extraction (Lao and Cohen, 2010).

accurately recognizing representative entities in natural language remains challenging.


Neural language representation models such as BERT pre-trained on large-scale corpora can well capture rich semantic patterns from plain text, and be fine-tuned to consistently improve the performance of various NLP tasks.

the existing pre-trained language models rarely consider incorporating knowledge graphs (KGs), which can provide rich structured knowledge facts for better language understanding.


From the results, we observe that: (1) BERT achieves comparable results with NFGEC on the macro and micro metrics.

bERT has lower accuracy than the best NFGEC model.


Following the success of KG representation learning, recent work embeds entities in different KGs into a low-dimensional vector space with the help of seed alignments (Chen et al., 2017).

the limited seeds and structural differences take great negative impacts on the quality of KG embeddings, which performs alignment poorly.


While other neural architectures for graphs exist, we believe that GGNN is more suitable for the Chinese NER task for its better capability of capturing the local textual information compared to other GNNs such as GCN (Kipf and Welling, 2017).

the traditional GGNN (Li et al., 2016) is unable to distinguish edges with different labels.


These models can potentially generate any sequence, and are thus sometimes referred to as open vocabulary.

they tend to underperform compared to word level models when trained on the same data.


For this section, we restrict ourselves to predicting characters and bigrams for simplicity.

our approach is straightforward to apply to n-grams or words.


Neural language models are typically trained by predicting the next word given a past context (Bengio et al., 2003).

natural sentences are not constructed as simple linear word sequences, as they usually contain complex syntactic information.


For example, in Figure 4b, the TCN model assigned the highest syntactic height to the word "market" and induced the phrase "expect a rough market" for the context "the fund managers".

in a ground-truth dependency tree, the verb "expect" is the word directly connected to the root node and therefore has the highest syntactic height.


θ is randomly and uniformly sampled from [−π, π].

our early experiments show that, at least within the context of NLP applications, this initialization performed comparable or worse than the standard Glorot initialization.


We observe a −0.1% degrade in performance on the partial adaptation and −4.3% degrade on the full adaptation of the Quaternion Transformer.

we note that the drop in performance with respect to parameter savings is still quite decent, e.g., saving 32% parameters for a drop of only 0.1 BLEU points.


Appendix A.2): For using the sparsemax mapping in an attention mechanism, Martins and Astudillo 2016show that it is differentiable almost everywhere, with where At first glance, sparsemax appears very different from softmax, and a strategy for producing other sparse probability mappings is not obvious.

the connection becomes clear when considering the variational form of softmax (Wainwright and Jordan, 2008): where H S (p) := − j p j log p j is the well-known Gibbs-Boltzmann-Shannon entropy with base e. Likewise, letting H G (p) := 1 2 j p j (1 − p j ) be the Gini entropy, we can rearrange Eq.


(2018) found non-monotonic models (with either soft attention or hard alignment) outperform popular monotonic models (Aharoni and Goldberg, 2017) in the three above mentioned tasks.

the inductive bias of monotonicity, if correct, should help learn a better model or, at least, learn the same model.


This query chain can be decomposed into two parts: a key represented by i k and a query represented by The former is modulated through the weight matrix W k , and tightly associated with the corresponding input token.

information carried by the key remains intact during the evolution of time step t. the latter, induced by the weight matrix W q , highly depends on the position and length of this chain, which dynamically changes between different token pairs.


However, for LSTM, GRU and ATR, LN results in significant computational overhead (about 27%∼71%).

quasi recurrent models like SRU and LRN only suffer a marginal speed decrease.


In recent years, deep neural networks have achieved outstanding success in natural language processing (NLP), computer vision and speech recognition.

these deep models are datahungry and generalize poorly from small datasets, very much unlike humans (Lake et al., 2015).


This finding also agrees with the observation  we found in multi-label classification: our approach has superior generalization capability in low-resource setting with few training examples.

to the strongest baseline HD-LSTM with 34.51M and 0.03 seconds for one batch, our approach has 17.84M parameters and takes 0.06 seconds in an acceleration setting, and 0.12 seconds without acceleration.


For example, ASP with three-source tasks achieves 82.23% and 66.92% accuracy, respectively, in SNLI and MNLI, which are lower than 82.28% and 67.39% accuracy with its best performance with two-source tasks.

tARS overcomes such challenges, for example, 83.12% > 82.67% and 68.24% > 67.79% in SNLI and MNLI, except for QQP, which can be further improved by asymmetric MtL techniques (Lee et al., 2016).


Early works mainly focused on the shared representation methods (Liu et al., 2017;Tong et al., 2018;Lin et al., 2018), using a single shared encoder between all tasks while keeping several task-dependent output layers.

the sparseness of the shared space, when shared by K tasks, was observed (Sachan and Neubig, 2018).


Our method is based on the observation that high-dimensional multimodal time series data often exhibit correlations across time and modalities which leads to low-rank tensor representations.

the presence of noise or incomplete values breaks these correlations and results in tensor representations of higher rank.


facial behaviors, and body postures (Mihalcea, 2012;Rossiter, 2011).

as much as more modalities are required for improved performance, we now face a challenge of imperfect data where data might be 1) incomplete due to mismatched modalities or sensor failure, or 2) corrupted with random or structured noise.


This leads to redundancy in these overparametrized tensors which explains their low rank (Figure 1).

the presence of noise or incomplete values breaks these natural correlations and leads to higher rank tensor representations.


Recently, neural models such as cascaded residual autoencoders (Tran et al., 2017), deep adversarial learning (Cai et al., 2018), or translation-based learning (Pham et al., 2019) have also been proposed.

these methods often require knowing which of the entries or modalities are imperfect beforehand.


Given our intuitions above, it would then seem natural to augment the discriminative objective function with a term to minimize the rank of M. In practice, the rank of an order-M tensor is computed using the nuclear norm X * which is defined as (Friedland and Lim, 2014), When M = 2, this reduces to the matrix nuclear norm (sum of singular values).

computing the rank of a tensor or its nuclear norm is NP-hard for tensors of order ≥ 3 (Friedland and Lim, 2014).


Thus one pinyin may correspond to ten or more Chinese characters on the average.

pinyin IME may benefit from decoding longer pinyin sequence for more efficient inputting.


This sensitivity helps language users segment a continuous stream of speech (Vitevitch et al., 1997), incorporate new words into the lexicon (Storkel et al., 2006), and reconstruct parts of an utterance that may have been obscured by noise.

the details of how language learners infer these phonotactic generalizations from incoming acoustic data are still unclear.


(2011) that were trained on a comparable, but smaller, amount of unsyllabified data (20,000 vs 30,000 words).

a Wilcoxon rank sum test on ρ yielded no significant difference between feature-naive and feature-aware models in this regard (W = 282; p = 0.56).


These attributes suggest that CLMs can model regularities that exist within words, such as morphological inflection.

even large language modeling (LM) corpora have sparse coverage of inflected forms for morphologically-rich languages, which has been shown to make word and character language modeling more difficult (Gerz et al., 2018b;Cotterell et al., 2018).


In machine translation, subword tokenization with byte pair encoding (BPE) addresses the problem of unknown words and improves performance (Sennrich et al., 2016).

segmentation is potentially ambiguous, and it is unclear whether preset tokenization offers the best performance for target tasks.


As the word embedding model is a fundamental component in many NLP systems, mitigating bias in embeddings plays a key role in the reduction of bias that is propagated to downstream tasks (e.g., (Zhao et al., 2018a)).

it is debatable if debiasing word embeddings is a philosophically right step towards mitigating bias in NLP.


Future work can look to apply existing methods or devise new techniques towards mitigating gender bias in other languages as well.


For example, profession-related nouns such as professor, doctor, programmer have shown to be stereotypically male-biased, whereas nurse, homemaker to be stereotypically female-biased, and a debiasing method must remove such biases.

one would expect 1 , beard to be associated with male nouns and bikini to be associ-ated with female nouns, and preserving such gender biases would be useful, for example, for a recommendation system (Garimella et al., 2017).


For both datasets, we uncover strong associations between 3 Our findings also hold for the widely used data from Waseem and Hovy (2016).

because of severe limitations of that dataset (see Schmidt  inferred AAE dialect and various hate speech categories, specifically the "offensive" label from DWMW17 (r = 0.42) and the "abusive" label from FDCL18 (r = 0.35), providing evidence that dialect-based bias is present in these corpora.