Finally, our analysis of case category prediction on nouns shows that more common cases such as the nominative, accusative, and genitive are predicted better, especially in languages with fixed word order.

cases that appear less frequently and on shifting positions (such as the instrumental), as well as those not associated with specific prepositions, are less well predicted.


They argue that summarize-then-translate is preferable to avoid both the computational expense of translating more sentences and sentence extraction errors caused by incorrect translations.

summarize-then-translate can only be used when the source language is high-resource (Wan et al.


The explicit use of syntactic information has been proved useful for neural machine translation (NMT).

previous methods resort to either tree-structured neural networks or long linearized sequences, both of which are inefficient.


Automating this process with meta-learning is thus an attractive proposition.

it comes with many potential pitfalls such as failing to match a human-designed curriculum, or significantly increasing training time.


The RL reward that directly corresponds to this goal would be the highest likelihood value reached during an NMT training run.

as we use only one NMT training run, having a single reward per run is infeasible.


We also tried computing the diagonal ofF on held-out data, as 5 github.com/thompsonb/sockeye_ewc there is some evidence that estimating Fisher on held out data reduces overfitting in natural gradient descent (Pascanu and Bengio, 2013).

we again found no meaningful differences.


We also experimented with personal pronouns as a proxy for persons.

we found them unsuitable since they are also often used as referring expressions to other entities, such as animals.


The pretrained model is then fine-tuned to the target task.

the fine-tuning procedure of the language model to the target task does not include an auxiliary objective.


Our proposed model is closely related to ULMFiT.

uLMFiT trains a LM and fine-tunes it to the target dataset, before transferring it to a classification model.


The resulting MAE was 0.450 for factuality and 1.184 for bias prediction, which is slightly better then our results (yet, very comparable for factuality).

our goal here is to emphasize the advantages of modeling the two tasks jointly.


To maximize their benefit to the scientific community, it is crucial to understand and evaluate the construction and limitation of reviews themselves.

minimal work has been done to analyze reviews' content and structure, let alone to evaluate their qualities.


This alone-that cross-domain performance can be so strikingly worse-is a significant result, providing the first estimate of how performance degrades across these domains for this task.

when we train an identically parameterized model on the training partition of the literary data and evaluate it on the literary test partition, performance naturally improves substantially to an F-score of 68.3.


Previous research on automated abusive language detection in Twitter has shown that communitybased profiling of users is a promising technique for this task.

existing approaches only capture shallow properties of online communities by modeling followerfollowing relationships.


The embeddings (called author profiles) are generated by applying a node embedding framework to an undirected unlabeled community graph where nodes denote the authors and edges the follower-following relationships amongst them on Twitter.

these profiles do not capture the linguistic behavior of the authors and their communities and do not convey whether their tweets tend to be abusive or not.


We propose an approach for learning author profiles using GCNs applied to the extended graph.

to node2vec, our method allows us to additionally propagate information with respect to whether tweets composed by authors and their communities are abusive or not.


GCN on its own achieves a high performance, particularly on the sexism class where its performance is typical of a community-based profiling approach, i.e., high recall at the expense of precision.

on the racism class, its recall is hindered by the same factor that Mishra et al.


Additionally, word embeddings are insensitive to fine-grained semantic distinctions, such as antonymy, due to their construction nature.

the sense representations used in our experiments (DeConf) were constructed by exploiting the knowledge encoded in WordNet.


By fine-tuning on, in some cases, as few as 100 examples, both NLI models are able to recover almost the entire performance gap on both the word overlap and negation challenge datasets (Outcome 1).

both models struggle to adapt to the spelling error and length mismatch challenge datasets (Outcome 2).


Indeed, it seems unlikely that a model will learn to perform algebraic numerical reasoning based on as few as 50 NLI examples.

a closer look at this dataset provides a potential explanation for this finding.


To capture fine-grained features, on the one hand, some works are concerned with matching QA pairs relationship in a more complex and diverse way, e.g., CNTN (Qiu and Huang, 2015) and MV-LSTM(Wan et al., 2016).

latent representation models aim to jointly learn lexical and semantic information from QA sentences and influence the vector generation directly, e.g., attention mechanism (Bahdanau et al., 2015).


Relevant relationship in answer selection datasets is binary, only including relevance and irrelevance.

in the real CQA applications, it is difficult to verify whether the answers are completely correct or not.


Specifically, they do: where the ST estimator lets the gradients of the stochastic layer dz dg φ (x) ≈ 1.

the ST estimator is backpropagating with respect to the sample-independent.


This has led to several attempts to use GANs for text generation, with a generator using either a recurrent neural network (RNN) Guo et al., 2017;Press et al., 2017;Rajeswar et al., 2017), or a Convolutional Neural Network (CNN) (Gulrajani et al., 2017;Rajeswar et al., 2017).

evaluating GANs is more difficult than evaluating LMs.


In RNNbased GANs, the previous output token is used at inference time as the input x t Guo et al., 2017;Press et al., 2017;Rajeswar et al., 2017).

when evaluating with BPC or perplexity, the gold token x t is given as input.


The output distribution of the generator is expected to converge to the real data distribution during the training.

the discriminator f w is expected to discern real samples from generated ones by outputting zeros and ones, respectively.


We hypothesize that MRS is particularly well-suited for text generation, as it is explicitly compositional, capturing the contribution to sentence meaning of all parts of the surface form (Bender et al., 2015).

semantic representations such as Abstract Meaning Representation (AMR; Banarescu et al., 2013) seek to abstract away from the syntax of a sentence as much as possible.


The method of plan construction will likely not generalize "as is" to other datasets, and the plan structure itself may also be found to be lacking for more demanding generation tasks.

on a higher level, our proposal is very general: intermediary plan structures can be helpful, and one should consider ways of obtaining them, and of using them.


4 Based on this, we could represent each sentence and each plan as a sequence of entities, and verify the sequences match.

using this criterion is complicated by the fact that it is not trivial to map between the entities in the plan (that originate from the RDF triplets) and the entities in the text.


possible plans, making this method prohibitive for even moderately sized input graphs.

it is sufficient for the WebNLG dataset in which n ≤ 7.


Table 4 indicates that for decreasingly probable plans our realizer does worse in the first criterion.

for both parts of the test set, if the realizer managed to express all of the entities, it expressed them in the requested order, meaning the outputs are consistent with plans.


1 Increases in computing power and model capacity have made it possible to generate mostlygrammatical sentence-length strings of natural language text.

generating several sentences related to a topic and which display overall coherence and discourse-relatedness is an open challenge.


The KB provides information about the hotel's services: complimentary breakfast, free wifi, spa.

it may not include information about the menu/times for the breakfast, credentials for the wifi, or the cancellation policy for a spa appointment at the hotel.


Given the wide range of information that may be of interest to guests, it is not clear how to extend the KB in the most effective way.

the conversational logs, which many hotels keep, contain the actual questions from guests, and can therefore be used as a resource for extending the KB.


3 shows that subjective words that represents speakers attitude (e.g., "ready", "guests", "time") had significantly different embeddings in the question and answer encoders.

objective words such as menu, or activity (e.g., "bacon", "cruise", "weekday") had similar embeddings although the two encoders do not directly share the embedding parameters.


The extracted tuples from NEURON can be used to extend a KB for a specific domain.

automatically fusing the tuples with existing facts in   Figure 4: Human-in-the-loop system for extending a domain-specific KB.


Recently, OPENIE systems based on end-to-end frameworks, such as sequence tagging (Stanovsky et al., 2018) or sequence-tosequence generation (Cui et al., 2018), have been shown to alleviate such engineering efforts.

all these systems focus on sentence-level extraction.


Alternatively, a CQA dataset could be transformed into declarative sentences (Demszky et al., 2018) for a conventional OPENIE system.

such a two-stage approach is susceptible to error propagation.


Even if (similarly to WIKIHOP creators) one considers a coarse-to-fine approach, where a set of potentially relevant documents is provided, re-encoding them in a query-specific way remains the bottleneck.

to other proposed methods (e.g., (Dhingra et al., 2018;Raison et al., 2018;Seo et al., 2016)), we avoid training expensive document encoders.


In particular, we observe a negative Pearson's correlation (-0.687) between accuracy and the number of candidate answers.

the performance does not decrease steeply.


Knowledge bases (KBs) (such as Freebase (Dong et al., 2015;Xu et al., 2016;Yao and Van Durme, 2014) or DBpedia (Lopez et al., 2010;Unger et al., 2012)) have been used for question answering (Yu and Lam, 2018).

the ever-changing nature of online businesses, where new products and services appear constantly, makes it prohibitive to build a highquality KB to cover all new products and services.


This amount is large enough to ameliorate the flaws of BERT that has almost no questions on the left side and no textual span predictions based on both the question and the document on the right side.

a small amount of finetuning examples is not sufficient to turn BERT to be more task-aware, as shown in Sec.


In Figure 1, for the question 'When did princess Diana die', Rematch correctly recognizes the relation die and links it to dbo:deathYear.

when the question slightly changed to "Where did princess Diana die?"


It is important to note that most of these approaches use state of the art machine learning techniques and require a large amount of training data.

when these tools applied to short text in a new domain such as question answering (QA) or key word based search, the performance is limited.


The scale and diversity of this dataset makes it particularly suited for use in training deeplearning models to solve word problems.

there is a significant amount of unwanted noise in the dataset, including problems with incorrect solutions, problems that are unsolvable without brute-force enumeration of solutions, and rationales that contain few or none of the steps required to solve the corresponding problem.


By augmenting our dataset with these formalisms, we are able to cover most types of math word problems 3 .

to other representations like simultaneous equations, our formalisms ensure that every problem-solving step is aligned to a previous one.


Discussions As we mentioned in section 3, the continuous nature of our formalism allows us to solve problems requiring systems of equations.

there are other types of word prob-Error type Problem Hard problems (45%) Jane and Ashley take 8 days and 40 days respectively to complete a project when they work on it alone.


And the gated recurrent unit is used to control the contribution of shortcut paths and path between adjacent characters.

as the study of  shown, the gate mechanism fails to choose the right path sometimes.


Arabic text is typically written without short vowels (or diacritics).

their presence is required for properly verbalizing Arabic and is hence essential for applications such as text to speech.


They are a natural choice to handle unknown words.

bPE does not fit in our scenario as it may create source and target segments of different lengths.


Using a Transformer model led to nearly identical WER to using our NMT model with attention.

their results are somewhat complimentary.


Heuristics can improve incorporation of this data: a relabeling heuristic (Pair) helps on HEAD and a filtering heuristic (Overlap) is helpful in both settings.

our trainable filtering and relabeling models outperform both of these techniques.


The rule-based approach has the advantage of not requiring manual annotation, while also allowing easy access to adding and removing individual rules.

language is continuously evolving, and there are exceptions to most grammar rules we know.


Difficult cases (such as long distance subject-verb relations) are often ignored in order to ensure high precision, at the expense of the recall of the system.

our rulebased system is not limited to the detection of simple cases of SVA errors.


In this regard, as experimented in (Rei and Yannako-udakis, 2016), training on more data in the same domain is a valid solution for improving the performance of LSTM models.

when also adding artificially generated data to the training set, we reach higher scores only on 2 out of the 4 benchmarks.


We demonstrate that error generation is much less sensitive to parsing errors and irregularities than rule-based systems for detecting subject-verb agreement.

artificial error generation enables us to utilise much more training data, and therefore can develop more robust neural models for SVA error detection that do not overfit the available, manually annotated training data.


We employ 102 features obtained from WALS 9 related to word order and morphosyntactic alignment, further reduced to 50 dimensions using PCA.

none these criteria correlates significantly to tagging accuracy, as we elaborate in Section 5.1.


We explore model confidence, as measured by perplexity and typological similarities, as intuitive criteria for PL choice.

both criteria prove to be not correlated with tagging accuracy scores.


RNN is a prevalent family of models for many NLP tasks and has demonstrated compelling performances (Mikolov et al., 2010;Sutskever et al., 2014;Peters et al., 2018).

its sequential nature makes it heavily reliant on word order information, which exposes to the risk of encoding language-specific order information that cannot generalize across languages.


Traditional document representation models such as Bag-of-words (BoW) and TF-IDF show competitive performance in some tasks (Wang and Manning, 2012).

these models treat words as flat tokens which may neglect other useful information such as word order and semantic distance.


For example, if all documents in the corpus are about machine learning, then the con-cept of machine learning becomes less critical in the embedding.

if the corpus contains documents from different areas of computer science, then the concept of machine learning should be encoded in any document relevant to it.


Obviously, good can be replaced with synonyms like nice without significantly altering the meaning of the sentence.

since the synonyms are treated as independent tokens in PV and Doc2VecC, the lexicon good must be predicted exactly.


To address the above concerns we propose a novel model for learning document embedding unsupervisedly.

with previous work (PV and Doc2Vec), we model documents according to two aspects.


7) for the baseline models are small, which support our assumption that these methods are not able to model interdocument features properly.

distances for SDDE are significantly larger.


On this point, CNNs can be likened to players of the twenty questions who guess the answers by iteratively asking questions and receiving information.

differences exist between humans and traditional CNNs in the manner in which they play this game.


Rather, they would prefer questions related to the target's profession or origin that are practical for inferring the answer.

typical CNNs use the same set of filters in any circumstances (Kim, 2014;Johnson and Zhang, 2017;.


Without adaptive convolutions, RNNs show better performance than CNNs on most datasets.

when adaptive convolutions are applied, our baselines perform better than RNN variants on every dataset.


In the case of Turkish, its morphological and syntactic features seem to explain some of the relatively low results.

with the small sample of languages and the many potential influencing factors at play, we are aware that it is not possible to draw any strong conclusions.


This co-extraction strategy can also be adopted in neural networks with multi-task learning (Wang et al., 2016(Wang et al., , 2017Li and Lam, 2017).

in all these works, the extracted targets and opinion words are separated.


The Pipeline model performs much better than rule-based methods, obtaining an especially high precision, showing that machine-learning methods can obtain better opinion words extraction.

pipeline model is still not ideal, and the F1score is approximately 10% lower than our proposed model in several datasets.


TC-BiLSTM includes the target information by concatenation and obtains better general performance than LSTM and BiLSTM.

tC-BiLStM is still over 10% lower than IOG and is slightly inferior to Pipeline, showing that the concatenation is not a good way to incorporate the target information for tOWE.


Admittedly, by task definition, abstractive summarization is more challenging than extractive summarization.

we argue that such inferior performance is partly due to some biases of existing summarization datasets.


The Idebate dataset is collected from short arguments of debates on controversial topics, and thus the text is rather formal.

our dataset contains the posts of interesting stories happened in daily life, and thus the text is more unstructured and informal.


(2017), which use a memory network for text summarization.

they only deal with extractive summarization by storing embeddings of individual sentences into memory.


Based on their observations, we hypothesize that sentences in a good summary should have a well-distributed connection with as many sentences as possible in the original text, because a good summary is supposed to cover all the important information in the text.

sentences in a bad summary may fail to form a welldistributed connection with sentences in the original text.


To learn these patterns automatically, we apply a CNN model on the similarity matrix to predict the quality of the summary.

it should be noted that CNNs usually work best when a large amount of training data is available, whereas the summary data we have collected represents a relatively small dataset.


That is why existing research employs large-scale automatically extracted compression pairs, such as the first sentence and the presumably shorter headline of a news article.

such easy-to-extract source data is only available for a few tasks, domains, and genres and the corresponding models do not generalize well from the task of headline generation to other text compression tasks.


Excluding the use of human abstracts, all systems resulted in similar performance times.

we observe a large margin in QA accuracy in our full system compared to the abstractive and our supervised approach.


We find that the two approaches (CL and RL) achieve overall relatively similar task success rates.

evaluating on task success is only one side of the coin.


(Lu et al., 2017) uses a knowledge-transferring method between generative and discriminative task settings.

this requires the models on both settings to be pretrained well enough.


(2018), which proposed to use a speaker to synthesize new instructions and implement pragmatic reasoning.

we observe there is some performance gap between seen and unseen environments.


Given a reference document and a question, the expected answer is a short span from the document.

answers in datasets such as SearchQA (Dunn et al., 2017) and NarrativeQA (Kočiskỳ et al., 2018) are free-form human generated texts based on given documents (Nguyen et al., 2016;Reddy et al., 2018;.


Such sensitivity suggests a preference for using our sample reweighting technique.

the ratio Samples/Groups CED H Q H A Examples (NewsQA) Q: Where is the drought hitting?


From overall statistics, we also find samples in NewsQA have a higher score than those in MARCO.

if we look at MARCO questions that start with "when" or "who" (i.e., probability natural questions with short answers), the scores go up dramatically.


Tymoshenko et al., 2017;Xiong et al., 2016a;Radford et al., 2018;Li et al., 2018, inter alia).

an undesired effect of this focus on neural approaches was that other methods have fallen out of focus, including strong unsupervised benchmarks that are necessary to highlight the true gains of supervised approaches.


(2018) used a sequence to sequence model (Sutskever et al., 2014) to generate an enhanced query for ARC which retrieves better supporting passages.

in general, all these approaches rely on annotated training data, and, some, on structured KBs, which are expensive to create (Jauhar et al., 2016).


(2018) in its complementary view of GCN and BiLSTM.


Wordbased models (Zhang et al., 2016;Cai and Zhao, 2016;Cai et al., 2017;Yang et al., 2017) directly segment a character sequence into words and can easily achieve the benefits of word-level information.

these models cannot usually conduct exact inference because of strategies, such as beam-search decoding and constraints of maximum word length, which are necessary as the number of candidate segmentations increases exponentially with the sentence length.


We fine-tuned all word embeddings during training of the proposed segmenter.

we randomly initialized all character embeddings, since pre-trained character embeddings did not improve performance in our preliminary experiments.


Their model also considers the importance of multiple words by integrating character and word information into an LSTM cell vector using a gatemechanism.

their model might not fully exploit word information, since word information is given to only the first and last characters of the word.


Su and Lee (2017) also introduce a pixel-based model that learns character features from font images.

their model is not shown to be better than word2vec model because it has little flexibility and fixed character features.


(2017) provide a new way to automatically extract character-level features, creating an image for the character and running it through a convolutional neural network to produce a visual character embedding.

this method does not utilize the rich semantic information of contextual words.


Similarly, our model does not require any annotated target data.

to their work, our model is efficient and feature-free.


In some studies, pre-processing is applied in order to improve the performance of CWS models, including substituting consecutive digits and English letters, Chinese idioms and long words with unique symbols (Cai and Zhao, 2016;Chen et al., 2015).

we do not deploy such techniques for fair comparison, focusing only on the possible improvements brought by word embeddings.


Especially for Chinese, there is a lot of work on joint word segmentation and POS tagging, e.g., (Zhang and Clark, 2008;Sun, 2011;Hatori et al., 2012;Zheng et al., 2013;Kong et al., 2015;Cai and Zhao, 2016;Chen et al., 2017;Shao et al., 2017), of which some use CRFs to predict one POS tag per character.

this is hard to transfer to languages like English and Vietnamese where single characters are less informative and tokens are much longer, resulting in a larger combinatory label space.


(2016), which uniformly distributes some small factor like 0.1 to incorrect labels.


For NN-based analyzers, we count a dictionary as 1 MB because they need a character-to-id mapping to work.

the list of characters contains non-frequently used characters, some of which could be treated as UNKs without any accuracy loss.


This motivated the development of special disfluency detection systems, and special mechanisms added to parsers specifically to handle disfluencies.

we show here that neural parsers can find EDITED disfluency nodes, and the best neural parsers find them with an accuracy surpassing that of specialized disfluency detection systems, thus making these specialized mechanisms unnecessary.


Thus, such conversational context information, higher-level knowledge that spans across sentences, provides important information that can improve speech recognition.

the long conversations typically split into short sentencelevel audios to make building speech recognition models computationally feasible in current stateof-the-art recognition systems (Xiong et al., 2017;Saon et al., 2017).


We then freeze the model when training the tagging LSTM.

in the pretrain + fine-tune setting, the dynamic speaker model is fine-tuned together with the tagging LSTM.


Previous work on disfluency removal has treated it as a sequence labeling task using word or spanlevel labels.

in some cases, simply removing disfluencies from an utterance can create ill-formed output.


Document-level event factuality identification is fundamental for document-level NLP applications, such as machine reading comprehension, which aims to have machines read a text passage According to Politico.com, it is said the United States will reach(CT+) an agreement with Mexico on the new trade deal that will replace North American Free Trade Agreement (NAFTA) before December, 2017.

mexican Economy minister Ildefonso Guajardo denied that they plan to reach(CT-) any agreement with the U.S. on the trade deal talks.


According to Figure 1, document-level event factuality can not be deduced from each sentence-level factuality separately, but depends on the comprehensive semantic information of sentences.

no suitable model for document-level task has been proposed yet.


Ignoring the inter-label dependency also enables efficient optimization algorithm, which further reduces computational cost.

ignoring label dependency inherently limits prediction accuracy.


It is a simple but very effective and accurate multi-class text classifier, as demonstrated in both sentiment analysis and multi-class classification (Grave et al., 2017).

fastText may not be directly applicable for more complicated problems, like XML.


The effectiveness and robustness of Rank-AE have been demonstrated in the previous section.

it is not clear to us  yet that if the effectiveness benefits from the proposed components, such as attention mechanism and marginal ranking loss.


Modeling the instance selection in one bag as a sequential decision process, a reinforcement learning agent is trained to determine whether an instance is valuable or not and construct a new bag with less noisy instances.

unbiased methods, such as REINFORCE, could usually take much time to train.


The SCVB0 algorithm outperforms stochastic VB (Hoffman et al., 2010) on large corpora by converging faster and often to a better solution (Foulds et al., 2013).

the SCVB0 algorithm does not leverage sparsity, and hence requires O(K) operations per word token.


Sequence labeling models are built on a recurrent neural network to sequentially go through a source text and learn the likelihood of each word in the source text to be a keyphrase word (Zhang et al., 2016;Luan et al., 2017;Gollapalli et al., 2017).

to these extractive methods, our approach can generate both absent and present keyphrases.


One possible reason is that scoring absent keyphrases requires a stronger text understanding ability than scoring present keyphrases.

as shown in the row of "gk, rk" on Table3, this problem can be solved by incorporating the retrieved keyphrases which provide external information to this module.


However, such heuristic pruning significantly reduces the amount of training data, and thus cannot make full use of distantly annotated data.

the other thread of works try to incorporate such imperfect annotation by partiallabel loss (PLL).


Recently, (Xin et al., 2018b) proposed utilizing a pretrained language model measures the compatibility between context and type names, and use it to repel the interference of noisy labels.

the compatibility got by language model may not be right and type information is defined by corpus and annotation guidelines rather than type names as is mentioned in (Azad et al., 2018).


Traditionally, labels are collected from multiple annotators/experts who are assumed to provide reliable labels.

in reality, these experts may have varying levels of expertise depending on the domains, and thus may disagree on labelling in certain cases (Aroyo and Welty, 2013).


In crowdsourcing, each instance is presented to multiple expert or non-expert annotators for labelling.

labels collected in this manner could be noisy, since some annotators could produce a significant number of incorrect labels.


By 2016, this approach became dominant with many studies demonstrating that using SDP brings better experimental results than previous approaches that used the whole sentence (Xu et al., 2015a,b;Mehryary et al., 2016;Cai et al., 2016;Le et al., 2018).

using the SDP may lead to the omission of useful information (i.e., negation, adverbs, prepositions, etc.).


Our RbSP model yields an F1-score of 86.3%, outperforms other comparative models, except Multi-Att-CNN model of  with multi-level attention CNN.

we have tried to re-implement the Multi-Att-CNN, but we failed to reproduce the positive result in the original paper.


The F1 reductions illustrate the contributions of all proposals to the final result.

the impact levels vary with different components.


Notably, another thread of SP-based approaches try to exploit IR-based techniques (Yao and Van Durme, 2014;Bast and Haussmann, 2015;Yang et al., 2014;Yih et al., 2015;Bao et al., 2016;Yavuz et al., 2016;Liang et al., 2016) by computing the similarity of two sequences as features, leveraging a neural network-based answer type prediction model, or training end-to-end neural symbolic machine via REINFORCE (Williams, 1992).

most SP-based approaches more or less rely on handcrafted rules or features, which limits their scalability and transferability.


The idea of bidirectional attention proposed in this work is similar to those applied in machine reading comprehension (Wang and Jiang, 2016;Seo et al., 2016;Xiong et al., 2016).

these previous works focus on capturing the interactions between two bodies of text, in this work, we focus on modeling the interactions between one body of text and a KB.


2, the answer context of Jon A. Husted includes the government position title secretary of state and starting date 2011-01-09.

for simple questions without constraints, the answer context is unnec-essary and can potentially incorporate noise.


Question answering for reading comprehension in general has seen a great deal of recent work (Rajpurkar et al., 2016;Joshi et al., 2017), and there have been many recent attempts to construct QA datasets that require advanced reasoning abilities Welbl et al., 2018;Mihaylov et al., 2018;Zellers et al., 2018;.

these attempts typically involve engineering data to be more difficult by, for example, explicitly prompting users to write multi-step questions Mihaylov et al., 2018), or filtering out easy questions (Zellers et al., 2018).


This work builds upon the Natural Questions (NQ) (Kwiatkowski et al., 2019), which contains some natural yes/no questions.

there are too few (about 1% of the corpus) to make yes/no QA a very important aspect of that task.


Questions were often about entertainment media (including T.V., movies, and music), along with other popular topics like sports.

there are still a good portion of questions asking for more general factual knowledge, including ones about historical events or the natural world.


We do find a correlation between the nature of the question and the likelihood of a "yes" answer.

this correlation is too weak to help outperform the majority baseline because, even if the topic or type is known, it is never best to guess the minority class.


While possible, it would require a great deal of thought and creativity.

detecting when a yes/no question can be answered using these strategies seems much easier and more intuitive.


In those countries, escapees who do not break any other laws are not charged for anything and no extra time is added to their sentence.

in Mexico, officers are allowed to shoot prisoners attempting to escape, and an escape is illegal if violence is used against prison personnel or property, or if prison inmates or officials aid the escape.


Riding on the GM Theta platform, the unibody is mechanically similar to the Saturn Vue and the Suzuki XL7.

the Equinox and the Torrent are larger than the Vue, riding on a 112.5 in (2,858mm) wheelbase, 5.9 in (150mm) longer than the Vue.


Traditional Key-value Memory Neural Networks (KV-MemNNs) are proved to be effective to support shallow reasoning over a collection of documents in domain specific Question Answering or Reading Comprehension tasks.

extending KV-MemNNs to Knowledge Based Question Answering (KB-QA) is not trivia, which should properly decompose a complex question into a sequence of queries against the memory, and update the query representations to support multi-hop reasoning over the memory.


The KV-MemNNs have been shown to support shallow reasoning in domain-specific knowledge based question answering (KB-QA) tasks such as MovieQA [Tapaswi et al., 2016].

when applied to a more challenging scenario, e.g., open domain KB-QA, the KV-MemNNs models do not perform as well as expected, possibly due to two reasons.


Intuitively, this loss function makes our model generate shorter paths to reach answers from the question.

it encourages the query updating method to mask the information already addressed in previous hops for the next query representation.


We find that for 48 questions, the model with CQU (model (f)) repeatedly selects the same keys that have the highest relevance probabilities at the first two hops.

when replacing CQU with KVQU, the model addresses different keys at the first two hops (examples shown in Table 2).


Then with the hop size increasing, the performance drops.

the model with the SQ method also achieves the best performance when the hop size is 3.


So we haveX This is similar to a standard attention mechanism, where attended representation is computed by summing the scaled representations.

such scaled addition is not possible when the outputs from lower layers are not of the same shapes, as in the following case.


if the relevant sentences are far apart, the model finds it difficult to handle such long-range cross sentence dependencies in its contextualized representations.


Models such as Decomposable Attention (Parikh et al., 2016) and ESIM (Chen et al., 2017), on the other hand, find alignments between the hypothesis and premise words through crossattention.

these improvements in entailment models have not yet translated to improvements in end tasks such as question answering.


Entailment models trained on this dataset show minor improvements on the Aristo Reasoning Challenge Musa et al., 2018).

these QA systems make independent predictions and can not combine in-formation from multiple supporting sentences.


Recently, machine translation techniques have been also applied to explain non-standard English expressions (Ni and Wang, 2017).

our deciphering task is not the same as machine translation in that hate symbols are short and cannot be modeled as language.


However, in their task, for each word to be defined, its pre-trained word embedding is required as an input, which is actually the prior knowledge of the words.

such kind of prior knowledge is not available in our decipherment task.


We use GRU (Cho et al., 2014) for decoding.

instead of also using GRU for encoding, we found that LSTM (Hochreiter and Schmidhuber, 1997) performs better on our task.


This means the Seq2Seq model can better explain the hate symbols when Twitter users intentionally misspell or abbreviate common slur terms.

the Variational Decipher tends to be better at deciphering hate symbols with unseen definitions, so it can be applied to explain newly created hate symbols on Twitter.


Traditionally, the output layer of PCNN/CNN would learn label specific parameters optimized by a cross-entropy loss.

the label specific parameters spaces are unique to each relation, matrices associated with the long-tails can only be exposed to very few facts during training, resulting in poor generalization.


To alleviate the wrong labeling problem, distant supervised methods have been implemented with multi-instance learning and neural networks (Riedel et al., 2010;Hoffmann et al., 2011;Zeng et al., 2015;Lin et al., 2016Lin et al., , 2017.

previous works focus on positive instances and few methods have addressed the issue of false negative instances.


In this model, the different tasks share span representations that only incorporate broader context indirectly via the gradients passed back to the LSTM layer.

dYGIE uses dynamic graph propagation to explicitly incorporate rich contextual information into the span representations.


For example, in the sentence "[They] ORG PER might have been using Northshore...", DYGIE predicted They to be of ORG type because the most confident antecedent is those companies in the previous sentence: "The money was invested in those companies."

they is actually referring to these fund managers earlier in the document, which belongs to PER category.


To remove this bottleneck, researchers have explored alternative ways to automatically create labeled data and learn models from such potentially noisy labels (Dalvi et al., 2011;Gentile et al., 2015;Furche et al., 2014;Lockard et al., 2018).

these approaches cannot find triples for predicates that are not in the seed ontology.


This is the first work that could truly be considered an OpenIE approach to semi-structured extraction.

as we show in our experiments, the constraints of their heuristic limit the recall of this approach.


All the domain shared features and domain specific features are just mixed together.

the output of the shared encoder of our full model is well distributed.


Only one existing work has endeavoured to consider the full document context , thus proposing a more generalised approach to document-level NMT.

the model is restrictive as the document-level attention computed is sentence-based and static (computed only once for the sentence being translated).


The former module is similar to the Memory Networks architecture of  in that it uses sentencelevel information.

there are two key differences: (i) we use MultiHead attention as in the Transformer architecture, and (ii) our context attention is dynamic such that we have a separate attention for each query word.


We again add the Document-level Context Layer alongside the decoder stack as in Figure 3.

instead of choosing the keys and values to be monolingual as in the encoder, we follow Tu et al.


, y t−1 up to step t − 1 during decoding, the probability that the model makes an error at the next step t is maximized.

in the log-semiring, this translates into the following loss function: to previous methods, which don't consider meaning preservation, we propose simple modifications of the approach presented in §3.1 to create adversarial perturbations at the word level that are more likely to preserve meaning.


Note that for a word-based model, every OOV will look the same (a special <unk> token), however the choice of OOV will still have an influence on the output of the model because we use unk-replacement.

we refer the base attack without constraints as Unconstrained hereforth.


In MT, methods have been proposed to attack word-based (Zhao et al., 2018;Cheng et al., 2018) and character-based (Belinkov and Bisk, 2018;Ebrahimi et al., 2018a) models.

these works side-step the question of meaning preservation in the source: they mostly focus on target side evaluation.


The vocabulary size |V | is usually more than ten thousands for datasets covering many sentences with a variety of topics.

for example, at most 100 unique words are selected when generating a sentence of length 100.


Effectiveness with fewer pre-training steps We followed the standard practice that the models are pre-trained by maximum likelihood before starting reinforcement learning.

such pre-training may have a negative effect in reinforcement learning.


Machine learning algorithms are gradually taking over from the human operators in tasks such as machine translation (Bahdanau et al., 2014), optical character recognition (Mithe et al., 2013), and face recognition (Parkhi et al., 2015).

some real-world applications require higher accuracy than the results achieved by state-of-the-art algorithms, which makes it difficult to directly apply these algorithms in certain scenarios.


Existing uncertainty models are usually based on Bayesian models, which is Traditional Bayesian models such as Gaussian Process (GP), can measure uncertainty of model.

as a nonparametric model, the time complexity of GP is increased by the size of data, which makes it intractable in many real world applications.


Obviously, the feature distance of the intra-class is large, sometimes even exceeding those of the inter-class distance near the decision boundary.

the features trained by metric learning, shown in the right-hand figure, exhibit clear gaps between the inter-class predictions.


To the best of our knowledge, predicting helpful posts in generic open-ended discussion forums has not been studied before.

there is significant amounts of related work on similar directions; where researchers evaluate the quality (which may not correlate with perceived helpfulness by the community users) of posts in specific domains such as health (Oh et al., 2012;Oh and Worrall, 2013;Beloborodov et al., 2014) and online education (Chandrasekaran et al., 2015;Chandrasekaran and Kan, 2019;Jenders et al., 2016).


As a preliminary experiment, we tried with a traditional bagof-words based model.

we do not include it in the baseline portfolio given its poor performance on our datasets.


In addition, an appropriate training loss is chosen to change the style while preserving the content.

these approaches are limited by their use of loss functions that must be differentiable with respect to the model parameters, since they rely on gradient descent to update the parameters.


The simplest means of G2P conversion is to build up a lexicon or dictionary containing the mapping from words to their corresponding pronunciations.

it fails to provide pronunciations for unknown words and inclusion of newer words increases memory requirement.


For most of the words (around 70K words), we observe that the phonetic transcriptions are exactly same in each of the 4 set.

for the remaining 30K words (29105 words to be exact), we observe that at least one set provides different transcription.


It is a case of an incorrectly labeled instance, a typical occurrence in distant supervision.


In sisg, the embeddings of OOV (or unseen) words are computed from the embedings of their subwords.

sisg requires word segmentation as a prepossessing step, and the way of collecting co-occurrence information is dependent on the results of explicit word segmentation.


The proposed scne is much simpler, but due to its simpleness, the embedding target of scne should contains many non-words, which seems to be a problem (see Figure 1).

our experimental results show that scne successfully captures the semantics of words and even sentences for unsegmented languages without using any knowledge of word boundaries (see Section 3).


Recent studies (Feng et al., 2018;Qin et al., 2018b,a) introduce reinforcement learning (RL) and adversarial learning to filter out incorrectly labeled sentences and achieve significant improvements.

there are two remaining challenges of noisy labeling problem.


(2015)  learning and adversarial learning to explicitly remove incorrectly labeled sentences.

they neglect the useful inherent information of those sentences which should be replaced correctly.


The experiments show promising results of our approach on this task, compared with the recent deep models commonly used for encoding textual representations (Conneau et al., 2017).

when the number of the unseen relation types increases, the performance of our model become far from the results obtained in the one-shot image classification (Koch et al., 2015), opening an interesting challenge for future work.


Based on our assumption, the aim is to learn how to encode the relational representations through the different paraphrases of the same relation.

the model also needs negative examples during the training phase.


Formally, given is a new dense representation of w ij which encodes also the information of the whole sentence.

word Attention with Context Vector only certain words in a sentence express the semantics of a relation, therefore we need a strategy to automatically identify them during the training.


To be effective, supervised DNNs rely on large amounts of carefully labeled training data.

it is not always realistic to assume that example labels are clean.


This temporal representation of a sample X is fed as input to a classifier on the training set D with sample categories y.

as mentioned in Section 2, we cannot access the true noise-free samples labels and instead, observe noisy labels corrupted by an unknown noise distribution.


In order to avoid this situation, and to prevent overfitting, we apply l 2 regularization to the noise model.

we want the noise model to overfit the label noise.


This suggests that the base model is just fitting the noisy labels.

when we train an SVM on the TRPr representations with true labels as targets, the SVM achieves the proposed model performance.


With each iteration, we can see the formation of clusters in Row 1.

in Row 2, when the noisy labels are superimposed, the clusters are not well separated.


The results of our analysis imply that early in training, representing part of speech is the natural way to get initial high performance.

as training progresses, it increasingly benefits the model to represent categories with longer-range dependencies, such as topic.


This approach can utilize larger corpora since training a Word2vec model is much faster than training a RNNLM.

in our preliminary experiments with this approach, we did not observe any improvement when using word embeddings trained on a large corpus.


1 With the development of large-scale knowledge bases (KBs) such as Freebase (Bollacker et al., 2008), relation extraction has been extended to larger scales comparable to KBs using the distant supervision (Mintz et al., 2009).

when the training corpus does not support the annotated relations showing in the KB, such approach could fail to find sufficient training examples.


As shown in Figure 1, in the KB, we have entities Pink Floyd, Animals, etc., with some existing relations notable work and has member in the KB.

when doing relation discovery, we can only get supporting sentences that suggest new relations based on and influenced by.


This has enabled Neural Machine Translation (NMT) sequence-tosequence (S2S) models and techniques to be applied to the GEC task (Tao et al., 2018b;Chollampatt and Ng, 2018;.

the efficacy of NMT techniques is degraded for low-resource tasks (Koehn and Knowles, 2017).


This roundtrip translation introduces relatively clean errors, so the generated corpus is much less noisy than the human-derived Wikipedia corpus.

these synthetic corruptions, unlike human errors, are limited to the domain of errors that the translation models are prone to making.


In our experiments with round-trip translation, we used target sentences drawn from Wikipedia to maintain a reasonable comparability between the two techniques.

there is no constraint preventing the application of roundtrip translation to diverse data sources; any source of clean text can be turned into a parallel GEC corpus.


(2016) (hereafter called 'Large Data LSTMs') learned a contingency between licensors and NPIs: the NPIs in examples like (3) were lower-surprisal when linearly preceded by negative licensors.

both papers reported that these models failed to constrain the contingency along the correct structural lines: negative NPI surprisal was decreased at least as much by a preceding negative distractor as by a negative licensor.


Standard autodifferentiation toolkits can be used to compute gradients of the energy with respect to the output once the output space has been relaxed.

one challenge is maintaining constraints on the variables being optimized.


the inference network output is in Y R (x) while gradient descent works with the more relaxed space Y R ′ (x).


The concept of adversarial agent is related to recent studies on adversarial examples for image classifiers-it has been shown that a carefully designed small perturbation can easily make neural networks mis-classify (Goodfellow et al., 2014;Szegedy et al., 2013;Moosavi Dezfooli et al., 2016;Carlini and Wagner, 2017;Cheng et al., 2019), and several recent works has extended these attacks to natural language processing models such as sentiment analysis (Gao et al., 2018;Yang et al., 2018) and machine translation (Ebrahimi et al., 2018;.

all of the previous work consider attacking a static model, where except input im-age/sentence there is no interaction between the attacker and the target model.


The use of state annotations allows a cleaner separation of the reasoning and natural language aspects of dialogues.

it is very expensive to annotate every state in a large amount of training data.


All the above-mentioned work focus on the static setting, i.e., the input does not depend on the model's output.

in our work, one agent's input depends on the other agent's output, which makes the input undecidable in the beginning.


We observe that the reactive attack could achieve better results than black-box method with 5.40 score advantage against SV agent and 4.98 score advantage against RL agent.

preemptive attack is not that successful-it gets 2.81 advantage against SV agent and 0.77 score advantage against RL agent.


We found that in the original model, it is easy for an attacker to find a sentence to quickly end the dialogue.

after adversarial training, it becomes much harder to find such sentences.


These dense representation models for link prediction include tensor factorization [Nickel et al., 2011, Socher et al., 2013, algebraic operations [Bordes et al., 2011, 2013b, Dasgupta et al., 2018, multiple embeddings [Wang et al., 2014, Lin et al., 2015, Ji et al., 2015, Zhang et al., 2018, and complex neural models [Dettmers et al., 2018, Nguyen et al., 2018.

there are only a few studies [Kadlec et al., 2017, Sharma et al., 2018 that investigate the quality of the different KG models.


Then, we compute the score change as: A.2 Modifications of the Form s, r , o In this section we approximate the effect of attack in the form of s, r , o .

to s , r , o attacks, for this scenario we need to consider the change in the e s , upon applying the attack, in approximation of the change in the score as well.


Gómez-Rodríguez and Vilares (2018) reduced constituent parsing to sequence tagging, where only n tagging actions need to be made, and obtained one of the fastest parsers to date.

the performance is well below the state of the art (Dyer et al., 2016;Stern et al., 2017;Kitaev and Klein, 2018a).


when a short constituent must be predicted (|n t | ≤ 2).

with a relative encoding scheme, the F-score was low for words where the corresponding n t has a large negative value (as showed in Figure 3).


Similarly, we tested whether it was beneficial to feed the output of task N into task C , and viceversa.

all these results did not improve those of the hard-sharing model.


Our model builds a coordinate tree in a bottom-up manner and predicts inner conjuncts accurately.

the baseline model predicts the entire span of a coordinate structure and splits them into conjuncts in a top-down fashion.


It is worthy of note that the pretrained word embedding is beneficial information for the task.

the use of contextual embedding, ELMo, does not improve performance.


Recent works, including those considered stateof-the-art, ignore event-nouns or build a single model for solving both Japanese predicate argument structure analysis (PASA) and eventnoun argument structure analysis (ENASA).

because there are interactions between predicates and event-nouns, it is not sufficient to target only predicates.


For example, predicates seldom have arguments in the same bunsetsu.

event-nouns often have arguments in the same bunsetsu, compound nouns, for example.


Therefore, the single model wrongly predicted that the ACC argument does not exist, but multi-RNN was able to predict the answer using the specific knowledge of event-nouns.

in Example (e), the single model correctly predicted the answer, but the multi-RNN model failed to predict the correct arguments.


Compared to only including the features, the model can benefit from adding both, the character level layer and the pretrained word vectors, while the results for tagging proper nouns and articles are still slightly worse than the baseline.

the final experimental setup can optimize the results for every POS tag compared to the baseline, see figure 3.


Parsing a Chinese address into semantically meaningful structures can be regarded as a special type of chunking task (Abney, 1991), where we need to perform address-specific Chinese word segmentation (Xue, 2003;Peng et al., 2004;Zhao et al., 2006) while assigning a semantic label to each chunk.

existing models designed for chunking may not be readily applicable in this task.


All these baseline models are capable of encoding transition patterns between neighboring chunks, which can partially capture certain structural information.

certain Chinese address-specific structural information is not explicitly captured in such models.


On the one hand, nowadays, fake news articles are easily propagated through various online media platforms and have become a grand threat to the trustworthiness of information.

our understanding of the language of fake news is still minimal.


Sentence fusion is challenging because it requires understanding the discourse semantics between the input sentences.

consider the example in Figure 1: a coherent fusion of the sentences requires understanding that the second sentence contrasts the first one, in order to insert the discourse connective "".


The rules specify how to modify the sentences such that they are "unfused" into two independent sentences.

e.g., in Figure 1 one rule will delete the discourse connective "", and another will replace the pronoun "he" with the named entity "Zeitler".


Importantly, sentence fusion models trained on our data cannot simply learn to invert rule application, because information is lost and can be recovered only by understanding the text semantics .

as mentioned, learning to insert "" in Figure 1 requires inferring that the sentences contrast.


Since the pioneering work in machine translation (Sutskever et al., 2014), sequence-tosequence (SEQ2SEQ) models have led much recent progress in open-domain dialogue generation, especially single-turn generation where the input is a prompt and the output is a response.

sEQ2sEQ methods are known to favor universal responses, e.g., "I don't know what you are talking about" (sordoni et al., 2015;Li et al., 2016a).


One promising line of research tackling this issue is to improve the specificity of responses, building on the intuition that generic responses frequently appear in the training data or consist of frequent words (Yao et al., 2016;Zhang et al., 2018b;.

past work in sentence specificity-the "quality of belonging or relating uniquely to a particular subject" 1has shown that word frequency is only one aspect of specificity, and that specificity involves a wide range of phenomena including word usage, sentence structure (Louis and Nenkova, 2011;Li and Nenkova, 2015;Lugini and Litman, 2017) and discourse context (Dixon, 1987;Lassonde and O'Brien, 2009).


Compared with diverse decoding (Li et al., 2016c), we observed during development that sentences generated by different dropouts tend to have diverse semantics (hence more likely to have different plausibility levels).

sentences from diversity decoding often have similar structure and phrases across candidates.


Similar but insignificant improvements are observed on Open-Subtitles.

using the maximum levels led to significant improvements over the baseline or the median level on all metrics.


To solve this task, we propose a neural description model that consists of two context encoders and a description decoder.

to the existing methods for non-standard English explanation (Ni and Wang, 2017) and definition generation (Noraset et al., 2017; Gadetsky et al., 2018), our model appropriately takes important clues from both local and global contexts.


Paraphrasing (Androutsopoulos and Malakasiotis, 2010; Madnani and Dorr, 2010) (or text simplification (Siddharthan, 2014)) can be used to rephrase words with unknown senses.

the target of paraphrase acquisition are words/phrases with no specified context.


R D , and then propose to add random dropout noise into the embedding input to simulate the dropout process as follows: where OneHot is a function to transform a word x into its one-hot form OneHot(x) ∈ R V , and  are given as the evidence, we first define an objective function as L(f θ (x), y) and then derive its lower bound as follows (withp = 1 − p): where P(b) is the prior distribution, and Bern(p) denotes the Bernouli approximate posterior with parameter p. Here we use E(x) as the simplied we separate the text classification model's parameters θ with the embedding parameters W and assume the classification model f θ directly takes embedding E as input.

the Bernouli distribution is hard to reparameterize, where we need to enumerate 2 V different values to compute the expectation over the stochastic dropout vector b.


Through our experiments, we have empirically demonstrated that the commonly adopted frequency-based vocabulary selection is already a very strong mechanism, further applying our proposed VVD can further improve the compression ratio.

due to the time and memory complexity issues, our algorithm and evaluation are more suitable for classificationbased application.


One possible solution is to merely discard (less important) words from the vocabulary, which can straightforwardly reduce the memory requirement.

such a naive method can cause another well-known drawback regarding the inapplicability of OOV words.


A practical approach is to partially take advantage of a certain range of smaller N -grams, such as N = 1 to 3 (row (b)) or N = 3 to 6 (row (c)).

smaller subword settings, such as row (b), might markedly degrade the performance from the original word embeddings.


This result indicates that KVQ-FH, SUM-FH and SUM-F successfully predicted the OOV word embeddings.

we also observed no significant difference between KVQ-FH, SUM-FH and SUM-F for both the H = 0.5M and H = 0.2M settings.


One might be surprised the improved results by KVQ-FH since the model sizes of KVQ-FH were relatively very small comparing with the original embeddings.

this may be a reasonable observation since our method additionally offered the embeddings of OOV words that cannot be handled by the original embeddings.


Indeed, for dependency parsing ensemble learning has shown to improve accuracy (Surdeanu and Manning, 2010;Kuncoro et al., 2016;.

ensembles are computationally demanding due to the large number of participating models.


For all languages in the multi-task setting the gain from including +SGLD in +ADV is statistically significant.

the gain decays as the amount of training data increases: from 3.8% on Irish (ga) to 1.1% on Persian (fa).


In other words, for this task, if one perturbs the attention weights when it is inferred that the patient is diabetic, this does change the output, which is intuitively agreeable.

this behavior again is an exception to the rule.


Treating the problem as question-answering will not replace the need for exploration in text-adventure games.

we hypothesize that it will cut down on the amount of exploration needed during testing time, theoretically allowing it to complete quests faster; one of the challenges of text adventure games is that the quests are puzzles and even after training, execution of the policy requires a significant amount of exploration.


The pre-training acts as a form of transfer learning from different, but re-lated games.

question-answering alone cannot solve the text-adventure playing problem because there will always be some trial and error required.


Some of these errors are fixed by the current RL model.

there are instances where even the RL-trained model outputs the wrong event type.


Several other works (Cao et al., 2017;Iyyer et al., 2018) exist for paraphrasing, though they have either been superseded by newer models or are not-directly applicable to our settings.

most of these methods focus on the issue of generating semantically similar paraphrases, while paying little attention towards diversity.


Submodular functions have been applied to a wide variety of problems in machine learning (Iyer and Bilmes, 2013; Jegelka and Bilmes, 2011; Krause and Guestrin, 2011;Kolmogorov and Zabih, 2002) and have recently attracted much attention in several NLP tasks including document summarization (Lin and Bilmes, 2011), data selection in machine translation (Kirchhoff and Bilmes, 2014) and goal-oriented chatbot training (Dimovski et al., 2018).

their application to sequence generation is largely unexplored.


Both of these Algorithm 1: Greedy selection for submodular optimization (Cardinality constraint) Input: approaches try to boost the generalization abilities of downstream classification models through word-level substitutions.

they are inherently restrictive in terms of the diversity they can offer.


We propose to identify the persuasive strategy employed in each sentence in each request.

constructing a large dataset with persuasion strategies labeled at the sentence level is timeconsuming and expensive.


Note that other persuasion tactics such as Reciprocity -"feel obligated to return something after receiving something of value from another" -and Authority -"comply with the requests of authority in an unthinking way to guide their decisions"-are also widely used in persuasive communication.

in this context, we did not observe enough instances of them.


(2017) also used this approach to adapt a dataset of AD narratives to their MCI classification task.

to the previous work on AD classification, we measure not only which information units are mentioned, but also the order in which they are mentioned.


The WEIGHT adaptation technique is not feasible when the source data is smaller than the target data, and the LININT technique performs poorly, as it assigns too much importance the smaller and out-of-domain dataset.

we do see marginal improvements using ALL and AUGMENT, reflecting the value of increasing the training set size by roughly 10%.


Only 4 of the "ratio" features and none of the density or efficiency features have a median value greater than zero.

these features are relevant to the task, and potentially more generalisable (e.g., total concept efficiency differs between the French AD and HC groups with p < 0.001 on a t-test, and represents an aggregate score rather than depending on the presence or absence of a single information unit).


When using ELMo with BM we see an improvement in recall.

adding knowledge on top improves the precision by 2.24 and 4.04 pp.


We see an improvement over random paths and no selection, but not important enough.

using both single and multi-hop paths in conjunction with relevance ranking improves the performance considerably (rows 4-8).


Temporal resolution is an NLP task that, in general, is domain-agnostic because temporal information is represented using a limited lexicon.

domain-specific aspects of temporal resolution are present in clinical texts.


Following is the identification of the "PM" AMPM entity; however, if this module finds an AMPM entity it then proceeds to look for an HourOfDay entity preceeding the AMPM substring.

an HourOfDay had already been identified, and the AMPM module neglected checking this.


Prior work in machine reading has made great strides in sentence-level binary relation extraction.

generalizing extraction to n-ary relations poses new challenges.


Each mutation in the global mutation-gene map is mapped to more than 20 genes on average.

within the context of a document, each mutation is (usually) associated with just a single gene.


Concretely, where w α ∈ R 1×d and H a ∈ R d×|a| , denoting hidden size by d and article length by |a|.

5 We recruited … … PPV and SVV were higher in PROP with respect to DEX (p < 0.01) …  the text span relevant to a classification will depend on the prompt under consideration.


To summarize this history, content-aware models (Chen et al., 2016a;Kim et al., 2017) similar to attention models in machine translation (Bahdanau et al., 2014) have been proposed.

content-aware models are likely to select the wrong history when the histories are similar in content.


Further, placing the KB results and the past utterances in same memory forces the architecture to encode them using a single strategy.

our work uses two different memories for past utterances and KB results.


A detailed study reveals that the use of triples cannot handle cases when a user queries with non-subject entries or in cases when the response requires inferencing over multiple entries.

our model is able to handle such cases since we use a compound representation of entire result (bag of words over values) while attending on that result.


Success of deep learning techniques have renewed the interest in development of dialogue systems.

current systems struggle to have consistent long term conversations with the users and fail to build rapport.


It is natural to consider applying text classification techniques for topic spotting.

as we empirically show in this paper, text classification techniques do not perform well in this setting.


For most of the classes the classifier is able to predict accurately.

the model gets confused between the classes which are semantically close (w.r.t.


This may be partly due to the fact that in a discussion, the first few utterances explicitly talk about the topic.

as we have seen, since SWBD covers many different topics which are semantically close to each other but are assigned distinct classes, it is equally challenging to predict the topic with the same model.


As can be seen from Table 3, alternative word generation orders all make it harder for standard leftto-right sequence decoder to learn to predict the next word.

using a doublyrecurrent architecture, specifically the surface decoder and the ancestor decoder, can improve perplexity scores for top-down word generation over the left-to-right decoder.


Furthermore, incorporating syntax facilitates response generation with richer vocabularies and more complex structures.

as shown in our error analysis, there is still room for improvement on discourse and pragmatics level.


A possibility is that models do encode information in the entity representations, but it doesn't get used in this task because of how the utterance is encoded in the hidden layer, or that results are due to some quirk in the specific setup of the task.

we replicated the results in a setup that does not encode whole utterances but works with single attributes and relations.


We still think that this hypothesis is plausible.

the architectures tested do not yet provide convincing support for it, at least for the data analyzed in this paper.


(Kim et al., 2018a) extended SHORTLISTER by using additional contextual information to rerank the predictions of SHORTLISTER.

none of them can continuously accommodate new domains without full model retrains.


Since data collection for machine learning models for this task is time-consuming, it is desirable to make use of existing data in a high-resource language to train models in low-resource languages.

development of such models has largely been hindered by the lack of multilingual training data.


For this reason, one would like to make use of methods that enable transfer learning from a highresource language to a low-resource language.

the development of sophisticated crosslingual transfer methods for intent detection and slot filling has so far been hindered by the lack of multilingual data sets that have been annotated according to the same guidelines.


We further find that the optimal choice of transfer method depends on the size of the available training data in the target language: In the zero-shot case where no target language data is available, translating the training data gives the best results.

if a small amount of training data is available, we find that jointly training on the high-resource and lowresource data works better than training on translated data.


Not surprisingly, since we have an order of magnitude more data for English, the model trained and tested on English data still performs better than any of the evaluated methods for the other two languages.

the gap between the numbers for English and the numbers for the other two languages does get considerably smaller for the cross-lingual models.


The Spanish results also suggest that in the zero-shot case, the multilingual encoder embeddings lead to better results than the XLU embeddings.

also the models that use cross-lingual embeddings perform very poorly and contrary to the case where we have some training data in the target language, in the zero-shot scenario, the translation method works considerably better than any other of the transfer methods.


Typically, evaluating goal-oriented dialogue systems is done via human-generated judgment like a task completion test or user satisfaction score (Walker et al., 1997;Möller et al., 2006).

the task of evaluating open-ended dialogue systems is not well defined as there is no * Equal Contribution clear explicit goal for conversations.


introduced an automatic evaluation system called ADEM which learns to score responses from an annotated dataset of human responses scores.

such system is heavily biased towards the training data and struggles with generalization capabilities on unseen datasets.


(Venkatesh et al., 2018) introduced a framework for evaluating the quality of the conversations based on topical diversity, coherence, engagement and conversational depth and showed that these metrics conform with human evaluation.

a big part of their metrics relies on human labels, which makes the evaluation system not scalable.


Given this similarity, it is natural to adopt existing QA models (Seo et al., 2016;Yu et al., 2018) for this task.

the documents in Holl-E dataset are longer, and the authors specifically report that they were unable to run these models when the entire document was given as input.


2017, after word-level adversarial training, we also adopt a refinement step to construct a bilingual dictionary for the top-k most frequent words in the source language 3 based on V s and V t , and further optimize U with Equation 2 in a supervised way.

once s is projected into the same semantic space as t, we can regard both sentences as coming from one unified language and directly project annotations from s to t. name tagging not only relies on word level features, but also on sequential contextual features for entity type classification.


The produced new synthetic parallel corpus can be used together with the original parallel data to increase the size of the training data, and eventually to improve NMT systems significantly and consistently.

on the source side, the synthetic data only contain data that can be generated by the back-translation system trained on some existing parallel data.


Even without an accurate translation model, we still have the possibility of extracting sentence pairs from unrelated source and target monolingual data.

this is very challenging since we have no guarantee that there are sentence pairs actually retrievable from a given Source sentence der Mann wurde festgenommen .


Recent methods addressed the task of finding word translations from monolingual data without any supervision (Lample et al., 2018a;Artetxe et al., 2018a).

marie and Fujita (2018) presented a method for inducing phrase tables from monolingual data using a weaklysupervised framework.


For en→tr, partial also significantly outperformed the baseline system, by 1.1 BLEU points.

backtr and copy brought larger improvements.


This new knowledge is unbiased by the existing parallel data since we induce the phrase table without using any given parallel data.

we can train a phrase table on the parallel data used  to train the NMT system and use it to extract partial translations.


(2017) proposed a method to train an RNN-based NMT system on partially aligned translations only.

this method cannot straightforwardly be applied to the state-of-theart Transformer architecture.


We also incorporate cycle consistency along with the adversarial loss.

while all these methods learn the mapping in the original embedding space, our approach learns it in the latent code space considering both the mapper and the target encoder as adversary.


The weight initialization reduces the requirement on the training data for the source-target language pair by transferring knowledge from the parent task, thereby improving the performance on the child task.

the divergence between the source and the assisting language can adversely impact the benefits obtained from transfer learning.


In an end-to-end solution, it would be ideal to use bilingual embeddings or obtain word-by-word translations via bilingual embeddings (Xie et al., 2018).

publicly available bilingual embeddings for English-Indian languages are not good enough for obtaining good-quality, bilingual representations (Smith et al., 2017;Jawanpuria et al., 2019) and publicly available bilingual dictionaries have limited coverage.


One may expect that the gaps between the different models should become smaller and even close with more updates, as the models with more languages see less examples per language in each batch, thus requiring more updates to improve in terms of BLEU.

in our setting these gaps did not close even after the models converged, leaving 2.73 average BLEU difference be-tween the 5-to-5 and the 103-to-103 model.


Extrapolating this trend, it is conceivable that the NMT models would simply need more training data than our current datasets provide in order to consistently outperform the SMT approach.

there appears to be no correlation between the size of the training set (cf.


Recent work has shown that supervision available in English for this task (e.g., lexical resources) can be transferred to other languages via crosslingual word embeddings.

this kind of transfer misses monolingual distributional information available in a target language, such as contrast relations that are indicative of antonymy (e.g., hot…while…cold).


These mixed results suggest that while crosslingual transfer in neural network models is a promising direction, the best blend of polyglot and language-specific elements may depend on the task and architecture.

we find overall contextual representations from polyglot language models succeed in a range of settings, even where multilingual word type embeddings do not, and are a useful technique for crosslingual transfer.


For example, dividing all Java methods in the dataset into 80% in training, 10% in validation, and 10% in testing.

this results in a situation where it is possible for code from one project to be in both the testing set and the training set.


However, this possibility is theoretical and a negative effect has never been shown.

we split by project: randomly divide the Java projects into training/validation/test groups, then place all methods from e.g.


(LeClair and McMillan, 2019) report that this search turns out to be quite aggressive, catching nearly all auto-generated code in the repository.

as with RQ 1 , the effect of this filter is theoretical and has not been measured in practice.


Recent work on neural summarization has mainly focused on sequence-to-sequence (seq2seq) architectures (Sutskever et al., 2014), a formulation particularly suited and initially employed for abstractive summarization (Rush et al., 2015).

state-of-the-art results have been achieved by RNN-based methods which are extractive.


(2018) trained an abstractive sentence compression model using attention-based sequence-to-sequence architecture (Rush et al., 2015) to map a sentence in the document selected by the extractive model to a sentence in the summary.

as the sentences in the document and in the summary are not aligned for compression, their compression model is significantly inferior to the extractive model.


A full-overlap, i.e., perc match of 1 is likely to be a case where the ShortcutText disseminates information of its prominent section.

a non-overlap is very likely to be click-bait or noise.


At first glance, filtering those abstractive Shortcut-Texts that constitute only frequent words should intuitively prune uninteresting ones and save ones that are similar to the example in Table 3.

a closer look at the pruned ShortcutTexts shows several interesting teasers with substrings comprised of out-of-place frequent-words, e.g., Las Vegas gunman Stephen bought nearly %% guns legally.


As a very low dr score corresponds to unusualness, a presence of very low dr values among the nonoverlapping words of the ShortcutText suggest a high likelihood of it being a teaser, and therefore, we compile them as teasers.

the filtering requires a threshold dr value that defines anything lower than it as a very low dr. Also, computing dr requires domain information of the text.


Traditional probabilistic topic models such as Latent Dirichlet Allocation are not able to utilize large numbers of topics (Wallach et al., 2009).

we successfully train anchor-based topic models with thousands of topics.


Ordinarily, this factorization is NP-Hard (Arora et al., 2012).

given a set of anchor words, or words which uniquely identify a topic, the computation requires only O(KV 2 + K 2 V I) where K is the number of topics, V is the size of the vocabulary, and I (typically around 100) is the average number of iterations (Arora et al., 2013).


Our model using tandem anchors built from randomly selected verses performs the best for nearly all levels of false positive rate.

the Gram-Schmidt based anchors produce better true positive rates for very low false positive rates.


With OpenBible+5, which is considerably more selective, this cost rises to approximately $1,000,000 USD.

with traditional coarse-grained topic modeling, this cost is anywhere from $40,000 USD using the TSKE as the ground truth, to $17,500,000 USD using Open-Bible+5 as the ground truth.


For example, in the case of the LDS edition, it took a committee of experts seven years of work to produce their crossreference resource, even with the aid of hundreds of volunteers.

without those experts and volunteers, the cost would have been even greater.


In our everyday chitchat , there is a conversation initiator, who proactively casts an initial utterance to start chatting.

most existing conversation systems cannot play this role.


There have also been efforts to develop systems that can chat with users about specific documents such as Wikipedia articles (Zhou et al., 2018) or reviews (Moghe et al., 2018).

these studies did not investigate how to initiate such conversations, and as a result, their models assume that the initial utterance is always given by users.


In particular, P E was very effective in the setting of very short headlines (len = 30).

the proposed method slightly lowered ROUGE-2 scores from the bare Transformer on len = 30, 50.


We infer that the bare Transformer can generate headlines whose lengths are close to 30 and 50 because the majority of the training set consists of headlines whose lengths are less than or equal to 50.

most of the generated headlines breached the length constraints, as explained in Section 3.4.


LRP E also generated headlines with a precise length but its variance is larger than those of previous studies in very short lengths, i.e., len = 10 and 13 in Japanese.

we consider LRP E is enough for real applications because the averaged difference between its output and the desired length is small, e.g., 0.1 for len = 10.


This strategy is guaranteed to work, since it can accelerate the computation of gradients, which is the substance of the backward propagation.

there is still a big concern: once the baseline models are simplified, will their performance be compromised?


Compared with GRU, MGU contains less trainable parameters, consumes less training time, and achieves similar performance.

in this paper, to better carry out our proposed "the lower the simpler" strategy, we propose Scalar Gated Unit (SGU), which is an even more simplified variant of GRU: Hidden State: By comparing the formulation of SGU with that of GRU, it is easy to see that both the update gate z t and the reset gate r t change from the vectors in GRU to the scalars in SGU.


Deep learning has emerged as a compelling solution to many NLP tasks with remarkable performances.

due to their opacity, such models are hard to interpret and trust.


(2017) is limited to regularizing model with gradient of the model's input.

we extend this concept to the intermediate layers of deep models, which is demonstrated to be beneficial based on the experimental results; 2) Ross et al.


In textual data, it is often unrealistic to annotate a word (even a stop word) to be completely irrelevant.

it can be reasonably easy to identify group of words that are positively linked to a class.


Question answering from text (Richardson et al., 2013;Hill et al., 2015;Hermann et al., 2015;Rajpurkar et al., 2016) is a key challenge problem for NLP that tests whether models can extract information based on a query.

even sophisticated models that perform well on QA benchmarks (Seo et al., 2017;Shen et al., 2017; may only be doing shallow pattern matching of the question against the supporting passage (Weissenborn et al., 2017).


Comparing MC-BiDAF++ and MC-MemNet on the multiple-choice setting of both datasets as shown in Table 3, the models appear to have similar capabilities to learn multi-hop reasoning.

looking at the no-context baseline for comparison, we find that it is only around 2% lower than the two relatively more complex models.


> 1), 2D-CSANs further improve the translation quality.

when the number of heads in attention goes up, the translation quality inversely drops.


A few of our LSTM reg runs attain state-of-theart test F 1 scores on AAPD.

in the interest of robustness, we report the mean value, as mentioned in Section 4.2.


Pre-trained representations mostly impacts the training time of the sequence to sequence model (see Appendix B): SRC-ELMO slows throughput during training by about 5.3x and SRC-FT is even slower because of the need to backpropagate through the LM for fine-tuning (9.2x).

inference is only 12-14% slower than the baseline when adding pre-trained embeddings to the encoder (SRC-ELMO, SRC-FT).


Comparing the deterministic and stochastic encoders in WAE, we observe the same trade-off between reconstruction and sampling.

our proposed stochastic encoder, with λ KL = 0.1 for WAE, consistently outperforms VAE in the contradictory metrics BLEU and PPL.


We expect all the models to implicitly capture aspects of world knowledge.

the discrete latent variable models provide Viterbi alignments over the training data, from which we can compile a look-up table with the extracted knowledge.


Other systems induce situation-specific frames from text (Cheung et al., 2013;Balasubramanian et al., 2013).

these methods do not explicitly target the commonsense correspondence between differing levels of detail of complex events.


fixation duration) (Demberg and Keller, 2008;Frank and Bod, 2011;van Schijndel and Schuler, 2015;Shain et al., 2016).

this approach has important disadvantages.


There is evidence that both frequency (unigram log probability) and predictability (5-gram surprisal) in isolation reliably index processing difficulty, as shown by the significance of both effects over the baseline.

when the effects are compared to each other, predictability explains significantly more variance than frequency but not vice versa.


To overcome this drawback, one can propose to take benefit from randomly initialised units, that are by design nonbiased.

it is common to face small target-datasets that contain too few data to learn such neurons from scratch.


Back to the extra predictor (FC layer of random branch), it is important to note that both branches are equally important for making a decision for every class, i.e., no weight is applied on the dimensions ofŷ p w i andŷ r w i .

this latter is sub-optimal since we, a priori, do not know which kind of units (random or pre-trained) is better for making a decision.


As expected, experiments on 10× larger Europarl datasets for 3 languages show that neural models outperform -gram models in less challenging data-intensive scenarios.

training on large datasets comes at the expense of training efficiency for neural models: e.g., according to Figure 1 training non-Bayesian -grams is around 15,000× quicker than training neural models.


As with the UDT experiments, there is little difference between Lematus 20-ch and Lematus 0ch in the smallest data setting.

when the number of training types increases to 5k, the benefits of context begin to show, with Lematus 20-ch yielding a 1.6% statistically significant (p < 0.001) improvement over Lematus 0-ch.


Recent work has improved our ability to detect linguistic knowledge in word representations.

current methods for detecting syntactic knowledge do not test whether syntax trees are represented in their entirety.


Most of State-of-the-art QA models are mainly based on Convolution Neural Network (CNN), Recurrent Neural Network (RNN) and many variants thereof (Wang and Nyberg, 2015;Yang et al., 2016;Hu et al., 2014;Tan et al., 2015).

with opaque structures of convolutional kernels and recurrent cells, these models are hard to understand for humans.


Meanwhile, adding convolutional layers on top of a density matrix is more of an empirical workaround than an implementation of a theoretical framework.

a complex-valued matching network is built on top of a quantum-theoretical framework for natural language.


In all of our experiments, we mask 15% of all WordPiece tokens in each sequence at random.

to denoising auto-encoders (Vincent et al., 2008), we only predict the masked words rather than reconstructing the entire input.


We recognize that it would also be possible to train separate LTR and RTL models and represent each token as the concatenation of the two models, as ELMo does.

: (a) this is twice as expensive as a single bidirectional model; (b) this is non-intuitive for tasks like QA, since the RTL model would not be able to condition the answer on the question; (c) this it is strictly less powerful than a deep bidirectional model, since it can use both left and right context at every layer.


Answer: The MLM model does converge slightly slower than the LTR model.

in terms of absolute accuracy the MLM model begins to outperform the LTR model almost immediately.


In this paper, we propose a method for reducing bias in machine learning classifiers without relying on protected attributes.

to previous work, our method eliminates the need to specify which biases are to be mitigated, and allows simultaneous mitigation of multiple biases, including those that relate to group intersections.


The box came hurtling down the hill and rolled up the slope Despite the high acceptability of the transitive alternate of roll in (5), infelicity is produced for the unaccusative alternate in (6).

closer observation of these sentences reveals that the issue at stake here is not whether the subject of the unaccusative roll can be instantiated with box, but that the act of rolling must be inherently facilitated.


All the NP arguments are shared in parallel between ba and its subordinate verb; the subject of ba is identical to the subject of the subordinate verb, and the object of ba is also the object of the subordinate verb.

(7)b shows a structure-sharing between the subjects of ba and its subordinate verb only; the objects are not shared.


It is left open in this paper how the intentions of a speaker relate to the meanings of his words.

sometimes the intent of the speaker is strongly linked with the meaning.


The first formalises to n -0 -U -EikokujinH wakatta ( u )], this is then operates as per normal.

the negation of the second sentence would be 31tO wakattaH(U Eikokujin)].


An embedded question is not a query directed toward the addressee.

in the case of questions, as shown in (8), a sense of query is directed to the addressee: In (8a) and (8b), we clearly see partitioning of relevant situations.


(We will discuss both examples further in Section 5).

even in the cases where there is a difference in meaning, what we are looking for is the relevant difference in both syntax and semantics --that is, along what semantic lines do these two words differ, and how is this difference related to their synactic behavior (and vice versa)?


Since they do have totally different (logical) event structures, previous theories may have to treat them as homophones.

the conceptual tie is so salient that we feel it is counterintuitive to assign them to two different senses.


The two types of modules also represent the two basic atomic terms in formal semantics: event and individuals.

individuals are understood in the context (i.e.


First, only boundaries (including stand-alone complete events) can be identified with a temporal point, such as in (6) Since complete and boundary events both have a delimiting temporal point (but contain no time course), the durational phrase can only be interpreted as the distance between reference point in time and that delimiting temporal time (the death time in 8a).

the durational phrase will be interpreted as a time course of an process (8b).


In example (14a), the focus is on incremental theme and therefore the measure phrase describes the resulting number of wounds.

in (14b) there is no such locus and therefore the measure attached to the cognate object describes the frequency of the activity.


We propose new method for the alignment betwee the languages that do not belong to the same language family.

to most of the previously proposed methods that rely heavily on statistics, our method attempts to use linguistic knowledge to overcome the problems of statistical model.


In the alignment of some language pairs that do not belong to the same language family, word list (Wu, 1994) or functional word information (Shin and Choi, 1996) was employed additionally.

to the systems that mainly rely on statistical approach, the English-Chinese alignment system developed by Ker (1997) used a class-based algorithm in its word alignment, and the syste did not use any statistical technique.


Syntactic approaches take this form as made up of two independent words o-V-ni and nar without further analysing how the two words contribute their respective information to the construction.

lexicalist approaches can take the whole form as one word as required from its syntactic behavior, and analyse the lexical compostion of the two component forms.


For we have a corresponding object-exaltation form o-V sur.

another variant of SHF is possible using the form o-V da, whose second component is a copulative verb.


It has been suggested that (1) the implicit target must be an integral part of the event denoted by the predicate and (2) its denotation is linguistically pre-specified enough.

these two properties still do not constitute a sufficient condition of the implicit target.


This kind of semantic decomposition would accommodate the data in (10) as well, if to practice calligraphy is reanalyzed as for Chinese ink to be applied on the surface of paper: In (15), the formal role of the embedded predicate is: burnt(sardine).

the implicit target, what is scraped off, is not the sardine itself but the burnt substance attached on its surface.


What is necessary is that the qualia structure encode this kind of semantic extension as well as the decomposition: ink pot bottle containing ink & turned-over(ink pot) ---> exist-out-of(ink, ink pot).

this is not entailment; the ink may well not come out if the ink pot is turned over.


These three patterns seem to be identical types.

a careful examination of these apparently similar constructions reveals that each of these patterns has its own syntactic, semantic, and even pragmatic properties.


In SLK43, i) the axiom of Gentzens LK in the form A-4 3 A is not always assumed.


Sets are incorporated into the rules and categories of CCG in order to allow flexibility for handling scrambled word orders directly with single lexical categories.

multiset-CCG achieves higher power than CCG and other mildly context-sensitive formalisms.


Set-CCG also accepts scrambled orders with single lexical categories by incorporating sets into the categories and rules of CCG.

the directional specifications of lexical categories and rules are retained.


It is true that sentences in which complements or adverbials are placed after the predicates of the sentences exist in Korean.

it is very rare phenomenon, and we will focus on predicate-final sentences.


This new constraint LOC is set up for languages like English which show a strong locality tendency in anaphoric interpretation.

to LPC, LOC finds its antecedent in local domain as we see in the following example: (19) Johni thinks that Tomj criticizes himselPi/j.


, then A is the source category and B is the target category, where f is a standard interpretation function.

not every metaphor has the form "A is B" with common nouns "A" and "B".


According to Fauconnier, these examples demonstrate that it has no sense to ask after the absolute truth of several counterfactual sentences.


Collocation is the habitual co-occurrence of two or more words in a text, and is an important feature for vocabulary learning.

due to the traditional grammar-based EFL pedagogy, the collocational property in relation to each item of vocabulary has been neglected in EFL class [6] [7].


Corpus analysis tools such as WordSmith [12] and Qwick [13] are very popular and useful software for concordance and simple collocation search.

they only take raw text as input, and thus fail to perform more sophisticated functions, such as the search of collocations in terms of their lemmas and parts-of-speech.


An expression such as tori-hada-ga tatu (bird-skin-NOM stand: skin turning into goose-flesh) would be an example.

there are resemblance metaphors that are not based on literal similarity.


If we generalize the above expressions into formulae in accordance with Lakoff and Johnson's convention, we may get something like WOMEN ARE ANIMALS or WOMEN ARE PLANTS.

these names are insufficient; they do not grasp the details of mapping precisely enough.


The shorthand codes recorded by the machine were transcribed into English words via a Computer Aided Transcription (CAT) system.

it is incapable of processing Cantonese, the dialect spoken by the predominant majority of Chinese litigants in Hong Kong.


This saves the stenographer from the tedious conversion of Arabic numerals into phonetically-based shorthand code during online input.

it prevents the CAT system from capturing some potential regularities of numerals in the corpus.


Indeed, object topicalization appears occasionally in place of a passive in English-to-Japanese translation.

an examination of the properties of object topicalization reveals that the resemblance is only partial.


letter TOP hide-PST 'The letter, (I) hid (it).'

while the agent in (2a) can be anyone, the one in (2b) is limited to the writer or the protagonist in this particular context.


Indeed, Lambrecht [8, p.123] takes this structure to be synonymous with the "subjectaccented" sentence (of which object topicalization is an example).

(6) and (lb) differ with regard to the information status of the logical object.


Temporal clauses deal with time, and conditional clauses deal with hypotheticality.

the postposed adverbial clauses function very differently from preposed ones.


In English, it can have two representations: "A before B" or "after A, B" to fit this time sequence.

there's only one device in Chinese "A zhihou/yihou, B" and that's why we have much more after-clauses in discourse superficially.


Some previous semantic analyses assume that the interpretation of (1) involves a semantic version of conjunction reduction, with its denotation corresponding to a set of propositions (Karttunen [5], Groenendijk & Stokhof [4], Roberts [10]).

there can be other semantic mechanisms that yield wide scope of disjunction in ( I ).


Larson's theory predicts that the sentences in (12) are derived from their sources (13) without any problem, since there is no island or finite clause that blocks either-movement.

(12a-c) are degraded contrary to Larson's prediction.