Question-answering (QA) data often encodes essential information in many facets. This paper studies a natural question: Can we get supervision from QA data for other tasks (typ-ically, non-QA ones)? For example, can we use QAMR (Michael et al., 2017) to improve named entity recognition? We suggest that simply further pre-training BERT is often not the best option, and propose the question-answer driven sentence encoding ( QUASE ) framework. QUASE learns representations from QA data, using BERT or other state-of-the-art contextual language models. In particular, we observe the need to distinguish between two types of sentence encodings, depending on whether the target task is a singleor multi-sentence input; in both cases, the resulting encoding is shown to be an easy-to-use plugin for many downstream tasks. This work may point out an alternative way to supervise NLP tasks. 1 1 Introduction It is labor-intensive to acquire human annotations for NLP tasks which require research expertise. For instance, one needs to know thousands of semantic frames in order to provide semantic role labelings (SRL) (Palmer et al., 2010). It is thus an important research direction to investigate how to get supervision signals from indirect data and improve one's target task. This paper studies the case of learning from question-answering (QA) data for other tasks (typically not QA). We choose QA because (1) a growing interest of QA has led to many large-scale QA datasets available to the community; (2) a QA task often requires comprehensive understanding of language and may encode rich information that  Part of this work was done while the author was at the University of Illinois at Urbana-Champaign. is useful for other tasks; (3) it is much easier to answer questions relative to a sentence than to annotate linguistics phenomena in it, making this a plausible supervision signal (Roth, 2017). There has been work showing that QA data for task A can help another QA task T , conceptually by further pre-training the same model on A (an often larger) before training on T (a smaller) (Talmor and Berant, 2019; Sun et al., 2019). However, it remains unclear how to use these QA data when the target task does not share the same model as the QA task, which is often the case when the target task is not QA. For instance, QA-SRL (He et al., 2015), which uses QA pairs to represent those predicate-argument structures in SRL, should be intuitively helpful for SRL parsing, but the significant difference in their surface forms prevents us from using the same model in both tasks. The success of modern language modeling techniques, e.g., ELMo (Peters et al., 2018), BERT (Devlin et al., 2019), and many others, has pointed out an alternative solution to this problem. That is, to further pre-train 2 a neural language model (LM) on these QA data in certain ways , obtain a sentence encoder, and use the sentence encoder for the target task, either by fine-tuning or as additional feature vectors. We call this general framework question-answer driven sentence encoding ( QUASE ) . A straightforward implementation of QUASE is to first further pre-train BERT (or other LMs) on the QA data in the standard way, as if this QA task is the target, and then fine-tune it on the real target task. This implementation is technically similar to STILTS (Phang et al., 2018), except that 2 We clarify three types of training: pre-training, further pre-training, and fine-tuning. Pre-training refers to the training of sentence encoders on unlabeled text; further pre-training refers to continuing training the sentence encoders on an intermediate, non target-task-specific labeled data (e.g. QA data); fine-tuning refers to training on the target task in the fine-tuning approach. STILTS is mainly further pre-trained on textual entailment (TE) data. However, similar to the observations made in STILTS and their follow-up works (Wang et al., 2019), we find that additional QA data does not necessarily help the target task using the implementation above. While it is unclear how to predict this behaviour , we do find that this happens a lot for tasks whose input is a single sentence, e.g., SRL and named entity recognition (NER), instead of a sentence pair, e.g., TE. This might be because QA is itself a paired-sentence task, and the implementation above (i.e., to further pre-train BERT on QA data) may learn certain attention patterns that can transfer to another paired-sentence task more easily than to a single-sentence task. Therefore, we argue that, for single-sentence target tasks, QUASE should restrict the interaction between the two sentence inputs when it further pre-trains on QA data. We propose a new neural structure for this and name the resulting implementation s-Q UASE, where s stands for single; in contrast, we name the straightforward implementation mentioned above p-Q UASE for paired. Results show that s-Q UASE outperforms p-Q UASE significantly on 3 single-sentence tasksSRL, NER, and semantic dependency parsing (SDP)indicating the importance of this distinction. Let QUASEA be the QUASE further pre-trained on QA data A . We extensively compare 6 different choices of A : TriviaQA (Joshi et al., 2017), NewsQA (Trischler et al., 2017), SQuAD (Ra-jpurkar et al., 2016), relation extraction (RE) dataset in QA format (QA-RE for short) (Levy et al., 2017), Large QA-SRL (FitzGerald et al., 2018), and QAMR (Michael et al., 2017). Interestingly, we find that if we use s-Q UASE for single-sentence tasks and p-Q UASE for paired-sentence tasks, then QUASEQAMR improves all 7 tasks 3 in low resource settings, with an average error reduction rate of 7.1% compared to BERT. 4 While the set of tasks we experimented with here is nonexhaustive, we think that QUASEQAMR has the potential of improving on a wide range of tasks. This work has three important implications. First, it provides supporting evidence to an important alternative to supervising NLP tasks: using QA to annotate language, which has been discussed in works such as QA-SRL, QAMR, and 3 SRL, SDP, NER, RE, co-reference resolution (Coref), TE and machine reading comprehension (MRC). QA-RE. If it is difficult to teach annotators the formalism of a certain task, perhaps we can instead collect QA data that query the target phenomena and thus get supervision from QA for the original task (and possibly more). Second, the distinction between s-Q UASE and p-Q UASE suggests that sentence encoders should consider some properties of the target task (e.g., this work distinguishes between singleand multi-sentence tasks). Third, the good performance of QUASEQAMR suggests that predicate-argument identification is an important capability that many tasks rely on; in contrast, many prior works observed that only language modeling would improve target tasks generally. This work aims to find an effective way to use readily available QA data to improve a target task that is typically not QA. A natural choice nowadays given the success of language modelsis to further pre-train sentence encoders, e.g. BERT, on QA data in certain ways , and then use the new encoder in a target task. This general framework is called QUASE in this work, and the assumption is that the sentence encoders learned from QA data have useful information for the target task. A straightforward implementation of QUASE is to further pre-train BERT on QA data in the standard way , i.e., fine-tune BERT as if this QA dataset is the target task, and then fine-tune BERT on the real target task. However, we find that this straightforward implementation is less effective or even negatively impacts target tasks with single-sentence input; similar observations were also made in STILTS (Phang et al., 2018) and its follow-ups (Wang et al., 2019): They further pretrain sentence encoders, e.g., ELMo, BERT, and GPT (Radford et al., 2018), on TE data and find that it is not effective for the syntax-oriented CoLA task and the SST sentiment task in GLUE, which are both single-sentence tasks (Wang et al., 2018). One plausible reason is that the step of further pre-training on QA data does not take into account some properties of the target task, for instance, the number of input sentences. QA is inherently a paired-sentence task; a typical setup is, given a context sentence and a question sentence, predict the answer span. Further pre-training BERT on QA data will inevitably learn how to attend to the context given the question. This is preferable when the target task is also taking a pair of sentences [CLS] Q 1 Q 2  QM [SEP] Question [CLS] S 1 S 2  SN [SEP] Sentence BERT BERT T[CLS] T 1 T 2  TM T[SEP] T'[CLS] T' 1 T' 2  T' N T'[SEP] Sentence2Question and Question2Sentence Attention G[CLS] G 1 G 2  GM G[SEP] Question Modeling Sentence Modeling U[CLS] U 1 U 2  UM U[SEP] H[CLS] H 1 H 2  HN H[SEP] Interaction Layer C[CLS] C 1 C 2  CN C[SEP] Classification Layer START/END s-QuASE as input, while it may be irrelevant or harmful for single-sentence tasks. It points out that we may need two types of sentence encodings when further pre-training BERT on QA data, depending on the type of the target task. The following subsection discusses this issue in detail. Standard sentence encoding is the problem of converting a sentence S =[ w 1 , w 2 ,    , w n ] to a sequence of vectors h ( S ) =[ h 1 , h 2 ,    , h n ] (e.g., skip-thoughts (Kiros et al., 2015)). Ideally, h ( S ) should encode all the information in S , so that it is task-agnostic: given a target task, one can simply probe h ( S ) and retrieve relevant information. In practice, however, only the information relevant to the training task of h ( S ) is kept. For instance, when we have a task with multi-sentence input (e.g., QA and TE), the attention pattern A among these sentences will affect the final sentence encoding, which we call h A ( S ) ; in comparison, we denote the sentence encoding learned from single-sentence tasks by h ( S ) , since there is no cross-sentence attention A . In a perfect world, the standard sentence encoding h ( S ) expresses also the conditional sentence encoding h A ( S ) . However, we believe that there is a trade-off between the quality and the quantity of semantic information a model can encode. Our empirical results corroborate this conclusion and more details can be found in Appendix A.2. The distinction between the sentence encodings types may explain the negative impact of using QA data for some single-sentence tasks: Further pre-training BERT on QA data essentially produces a sentence encoding with cross-sentence attentions h A ( S ) , while the single-sentence tasks expect h ( S ) . These two sentence encodings may be very different: One view is from the theory of information bottleneck (Tishby et al., 1999; Tishby and Zaslavsky, 2015), which argues that training a neural network on a certain task is extracting an approximate minimal sufficient statistic of the input sentences with regard to the target task; information irrelevant to the target task is maximally compressed. In our case, this corresponds to the process where the conditional sentence encoding compresses the information irrelevant to the relation, which will enhance the quality but reduce the quantity of the sentence information. In order to fix this issue, we need to know how to learn h ( S ) from QA data. However, since QA is a paired-sentence task, the attention pattern between the context sentence and the question sentence is important for successful further pre-training on QA. Therefore, we propose that if the target task is single-sentence input, then further pre-training on QA data should also focus on single-sentence encodings in the initial layers; the context sentence should not interact with the question sentence until the very last few layers. This change is expected to hurt the capability to solve the auxiliary QA task, but it is later proved to transfer better to the target task. This new treatment is called s-Q UASE with s representing single-sentence, while the straightforward implementation mentioned above is called p-Q UASE where p means paired-sentence. The specific structures are shown in Fig. 1. 2.2.1 s-Q UASE The architecture of s-Q UASE is shown in Fig. 1(a). When further pre-training it on QA data, the context sentence and the question sentence are fed into two pipelines. We use the same Sentence2Question and Question2Sentence attention as used in BiDAF (Seo et al., 2017). Above that, Sentence Model-ing, Question Modeling, and Interaction Layer are all bidirectional transformers (Vaswani et al., 2017) with 2 layers, 2 layers, and 1 layer, respectively. Finally, we use the same classification layer as BERT, which is needed for training on QA data. Overall, this implementation restricts interactions between the paired-sentence input, especially from the question to the context, because when serving the target task, this attention will not be available. Using s-Q UASE in target tasks. Given a sentence S , s-Q UASE can provide a sequence of hidden vectors h ( S ) , i.e., the output of the Sentence Modeling layer in Fig. 1(a). Although h ( S ) does not rely on the question sentence, h ( S ) is optimized so that upper layers can use it to handle those questions in the QA training data, so h ( S ) indeed captures information related to the phenomena queried by those QA pairs. For single-sentence tasks, we use h ( S ) from s-Q UASE as additional features, and concatenate it to the word embeddings in the input layer of any specific neural model. 5 2.2.2 p-Q UASE The architecture of p-Q UASE is shown in Fig. 1(b), which is the standard way of pre-training BERT. That is, when further pre-training it on QA data, the context sentence and the question sentence form a single sequence (separated by special tokens) and are fed into BERT. 5 We mainly use concatenation in both types of QUASE. However, we also use replacement in some experiments and we will note these cases later in this paper. Using p-Q UASE in target tasks. Given a sentence pair S (concatenated), p-Q UASE produces h A ( S ) , i.e., the output of the BERT module in Fig. 1(b). One can of course continue fine-tuning p-Q UASE on the target task, but we find that adding p-Q UASE to an existing model for the target task is empirically better (although not very significant); specifically, we try to add h A ( S ) to the final layer before the classification layer, and we also allow p-Q UASE to be updated when training on the target task, although it is conceivable that other usages may lead to even stronger results. For instance, when the target task is token classification, e.g., MRC, we can simply concatenate the vectors of h A ( S ) at each timestamp to any existing model; when the target task is sentence classification, e.g., TE, we apply max-pooling and average-pooling on h A ( S ) , respectively, and concatenate the two resulting vectors to any existing model before the final classification layer. Modern LMs are essentially sentence encoders pre-trained on unlabeled data and they outperform early sentence encoders such as skip-thoughts (Kiros et al., 2015). While an LM like BERT can handle lexical and syntactic variations quite well, it still needs to learn from some annotations to acquire the definition of many tasks, especially those requiring complex semantics (Tenney et al., 2019). Although we extensively use BERT here, we think that the specific choice of LM is orthogonal to our proposal of learning from QA data. Stronger LMs, e.g., RoBERTa (Liu et al., 2019) or XLNet (Yang et al., 2019), may only strengthen the proposal here. This is because a stronger LM represents unlabeled data better, while the proposed work is about how to represent labeled data better. CoVe (McCann et al., 2017) is another attempt to learn from indirect data, translation data specifically. However, it does not outperform ELMo or BERT in many NLP tasks (Peters et al., 2018) and probing analysis (Tenney et al., 2019). In contrast, our QUASE will show stronger experimental results than BERT on multiple tasks. In addition, we think QA data is generally cheaper to collect than translation data. The proposed work is highly relevant to Phang et al. (2018) and their follow-up works (Wang et al., 2019), which use further pre-training on data-rich intermediate supervised tasks and aim Single-sentence Paired-sentence System SRL RE TE MRC BERT 34.17 62.99 78.29 79.90 BERTQAMR 32.92 50.16 78.73 82.96 Table 1: The naive way of training BERT on QAMR (BERTQAMR ) negatively impacts single-sentence tasks. to improve another target task. The key differences are as follows: First, we distinguish two types of sentence encodings, which provide explanation to their puzzle that sentence-pair tasks seem to benefit more from further pre-training than single-sentence tasks do. Second, they only focus on fine-tuning based methods which cannot be easily plugged in many single-sentence tasks such as SRL and Coref, while we analyze both fine-tuning based and feature-based approaches. Third, they mainly use TE signals for further pre-training, and evaluate their models on GLUE (Wang et al., 2018) which is a suite of tasks very similar to TE. Our work instead makes use of QA data to help tasks that are typically not QA. Fourth, from their suite of further pre-training tasks, they observe that only further pre-training on language modeling tasks has the power to improve a target task in general, while we find that QAMR may also have this potential, indicating the universality of predicate-argument structures in NLP tasks. Our work is also related to Sentence-BERT (Reimers and Gurevych, 2019) in terms of providing a better sentence representation. However, their focus was deriving semantically meaningful sentence embeddings that can be compared using cosine-similarity, which reduces the computational cost of finding the most similar pairs. In contrast, QUASE provides a better sentence encoder in the same format as BERT (a sequence of word embeddings) to better support tasks that require complex semantics. In this section, we conduct thorough experiments to show that QUASE is a good framework to get supervision from QA data for other tasks. We first give an overview of the datasets and models used in these experiments before diving into the details of each experiment. Specifically, we use PropBank (Kingsbury and Palmer, 2002) (SRL), the dataset from the SemEval'15 shared task (Oepen et al., 2015) with DELPH-IN MRS-Derived Semantic Dependencies target representation (SDP), CoNLL'03 (Tjong Kim Sang and De Meulder, 2003) (NER), the dataset in SemEval'10 Task 8 (Hendrickx et al., 2009) (RE), the dataset in the CoNLL'12 shared task (Pradhan et al., 2012) (Coref), MNLI (Williams et al., 2018) (TE), and SQuAD 1.0 (Ra-jpurkar et al., 2016) (MRC). In Table 4, we use CoNLL'12 English subset of OntoNotes 5.0 (Prad-han et al., 2013), which is larger than PropBank. The performance of TE and MRC is evaluated on the development set. 6 For single-sentence tasks, we use both simple baselines (e.g., BiLSTM and CNN; see Appendix B.1) and near-state-of-the-art models published in recent years. As in ELMo, we use the deep neural model in He et al. (2017) for SRL, the model in Peters et al. (2018) for NER, and the end-to-end neural model in Lee et al. (2017) for Coref. We also use the biaffine network in Dozat and Manning (2018) for SDP but we removed part-of-speech tags from its input, and the attention-based BiLSTM in Zhou et al. (2016) is the strong baseline for RE. In addition, we replace the original word embeddings in these models (e.g., GloVe (Pennington et al., 2014)) by BERT. Throughout this paper, we use the pre-trained case-insensitive BERT-base implementation. More details on our experimental setting can be found in Appendix B, including the details of simple models in B.1, some common experimental settings of QUASE in B.2, and s-Q UASE combined with other SOTA embeddings (ELMo and Flair (Akbik et al., 2018)) in B.3. We first consider a straightforward method to use QA data for other tasksto further pre-train BERT on these QA data. We compare BERT further pre-trained on QAMR (denoted by BERTQAMR ) with BERT on two single-sentence tasks (SRL and RE) and two paired-sentence tasks (TE and MRC). We use a feature-based approach for single-sentence tasks and a fine-tuning approach for paired-sentence tasks. The reason is two-fold. On the one hand, current SOTAs of all single-sentence tasks considered in this paper are still 6 For TE, we mean matched examples in MNLI. feature-based. How to efficiently use sentence encoders (e.g. BERT) in a fine-tuning approach for some complicated tasks (e.g. SRL and SDP) is unclear. On the other hand, the fine-tuning approach shows great advantage over feature-based on many paired-sentence tasks (e.g. TE and MRC). Similar to Phang et al. (2018), we find in Table 1 that the two single-sentence tasks benefit less than the two paired-sentence tasks from BERTQAMR , which indicates that simply further pre-training BERT is not enough. We then compare s-Q UASEQAMR and p-Q UASEQAMR on three single-sentence tasks (SRL, SDP and NER) and two paired-sentence tasks (TE and MRC) to show that it is important to distinguish two types of sentence representations. Rather than concatenating two embeddings as proposed in Sec. 2.2, here we replace BERT embeddings with QUASE embeddings for convenience. The results are shown in Table 2. We find that s-Q UASE has a great advantage over p-Q UASE on single-sentence tasks and p-Q UASE is better than s-Q UASE on paired-sentence tasks. The proposal of two types of sentence encoders tackles the problem one may encounter when there is only further pre-training BERT on QAMR for single-sentence tasks. In summary, it is necessary to distinguish two types of sentence representations for single-sentence tasks and paired-sentence tasks. To see whether adding QUASE to BERT reduces the sample complexity, we compare QUASEQAMR with BERT on one single-sentence task (SRL) and one paired-sentence task (MRC) with different percentages of training examples. For convenience, we replace BERT embeddings with QUASE embeddings for SRL. As shown in Figure 2, we find that s-Q UASEQAMR outperforms BERT on SRL with small training data, and p-Q UASEQAMR outperforms BERT on MRC with small training data. The results support that (1) adding QUASE to BERT reduces the sample complexity, (2) QUASE is very important in the low-resource setting. For instance, s-Q UASEQAMR achieves an F1 score of 61 in SRL with 30% ( 27 K ) training examples (compared to 50 . 92 F1 by BERT). And p-Q UASEQAMR achieves 69 . 81 average F1 on MRC with 0 . 1% (about 100 ) training examples (com-pared to 13 . 29 F1 by BERT). We compare BERT with QUASE further pre-trained with the same numbre of QA pairs on 6 different QA datasets (TriviaQA (Joshi et al., 2017), NewsQA (Trischler et al., 2017), SQuAD, QA-RE (Levy et al., 2017), Large QA-SRL (FitzGerald et al., 2018), and QAMR). s-Q UASE further pre-trained on different QA datasets are evaluated on four single-sentence tasks in a feature-based approach: SRL, SDP, NER and RE. p-Q UASE further pre-trained on different QA datasets is evaluated on one task (TE) in a fine-tuning approach. In Table 3, we find that the best options are quite different across different target tasks, which is expected because a task usually benefits more from a more similar QA dataset. However, we also find that QAMR is generally a good further-pre-training choice for QUASE. This is consistent with our intuition: First, QAMR has a simpler concept class than other paragraph-level QA datasets, such as TriviaQA, NewsQA and SQuAD. It is easier for QUASE to learn a good representation with QAMR to help sentence-level tasks. Second, QAMR is more general than other sentence-level QA datasets, such as QA-RE and Large QA-SRL. 7 Therefore, we think that the capability to identify predicate-argument structures can generally help many sentence-level tasks, as we discuss next. Here we compare QUASEQAMR with BERT on 5 single-sentence tasks and 2 paired-sentence tasks, where QUASEQAMR is further pre-trained on the training set ( 51 K QA pairs) of the QAMR dataset. As shown in Table 4, we find that QUASEQAMR 7 Although the average performance of QUASEQAMR on five tasks is slightly below QUASE Large QA  SRL , for which the benefit mostly comes from SRL. QUASE is mainly designed to improve a lot of tasks, so QAMR is a better choice in our setup, but in practice, we do not limit QUASE to any specific QA dataset and one can use the best one for corresponding target tasks. has a better performance than BERT on both single-sentence tasks and paired-sentence tasks, especially in the low-resource setting 8 , indicating that QUASEQAMR can provide extra features compared to BERT. Admittedly, the improvement in the Full setting is not significantly large, but we think that this is expected because large direct training data are available (such as SRL with 278 K training examples in OntoNotes). However, it is still promising that 51 K indirect QA pairs can improve downstream tasks in the low-resource setting (i.e. several thousands direct training examples). That is because they help the scalability of machine learning methods, especially for some specific domains or some low-resource languages where direct training data do not exist in large scale. In this section we discuss a few issues pertaining to improving QUASE by using additional QA datasets and the comparison of QUASE with related symbolic representations. We investigate whether adding the Large QA-SRL dataset (FitzGerald et al., 2018) or the QA-RE 9 dataset into QAMR in the further pre-training stage can help SRL and RE. We use s-Q UASE embeddings to replace BERT embeddings instead of concatenating the two embeddings. The effectiveness of adding existing resources (Large QA-SRL or QA-RE) into QAMR in the further pre-training stage of s-Q UASE on SRL and RE are shown in Table 5. We find that adding related QA signals (Large QA-SRL for SRL and QA-RE for RE) into QAMR can help improve specific tasks. Noteworthy is the fact that QA-RE can help SRL (Large QA-SRL can also help RE), though the improvement is minor compared to Large QA-SRL (QA-RE). These results indicate that adding more QA signals related to the sentence can help get a better sentence representation in general. 8 Another interesting finding is that simple models usually benefit more from QUASE embeddings than SOTA models. 9 Because the training set of QA-RE is too large, we randomly choose 100 , 000 training examples. Moreover, inducing these representations requires costly annotation by experts. Proposals such as QA-SRL, QAMR, semantic proto-roles (Reisinger et al., 2015), and universal dependencies (White et al., 2016) avoid some of these issues by using natural language annotations, but it is unclear how other tasks can take advantage of them. QUASE is proposed to facilitate inducing distributed representations instead of symbolic representations from QA signals; it benefits from cheaper annotation and flexibility, and can also be easily used in downstream tasks. The following probing analysis, based on the Xinhua subset in the AMR dataset, shows that s-Q UASEQAMR embeddings encode more semantics related to AMR than BERT embeddings. Specifically, we use the same edge probing model as Tenney et al. (2019), and find that the probing accuracy ( 73 . 59 ) of s-Q UASEQAMR embeddings is higher than that ( 71 . 58 ) of BERT. At the same time, we find that p-Q UASEQAMR can achieve 76 . 91 F1 on the PTB set of QA-SRL, indicating that p-Q UASEQAMR can capture enough information related to SRL to have a good zero-shot SRL performance. More details can be found in Appendix C.1. Another fact worth noting is that AMR can be used to improve downstream tasks, such as MRC (Sachan and Xing, 2016), TE (Lien and Kouylekov, 2015), RE (Garg et al., 2019) and SRL (Song et al., 2018). The benefits of QUASEQAMR on downstream tasks show that we can take advantage of AMR by learning from much cheaper QA signals dedicated to it. QUASE is designed to learn distributed representations from QA signals to help down-stream tasks. We further show the difficulties of learning two types of corresponding symbolic representations from QA signals, which indicates that the two other possible methods are not as tractable as ours. One option of symbolic representation is the QAMR graph. Michael et al. (2017) show that question generation for QAMR representations can only achieve a precision of 28% , and a recall of 24% , even with fuzzy matching (multi-BLEU 10 > 0 . 8 ). Furthermore, it is still unclear how to use the complex QAMR graph in downstream tasks. These results indicate that learning a QAMR parser for down-stream tasks is mainly hindered by question generation, and how to use the full information of QAMR for downstream tasks is still unclear. Another choice of symbolic representation is AMR, since QAMR is proposed to replace AMR. We consider a simpler setting, learning an SRL parser from Large QA-SRL. We propose three models in different perspectives, but the best performance of them is only 54 . 10 F1, even with fuzzy matching (Intersection/Union  0 . 5 ). More details can be found in Appendix C.2. Although a lot of methods (Khashabi et al., 2018; Marcheggiani and Titov, 2017; Strubell et al., 2018) can be adopted to use SRL/AMR in downstream tasks, the difficulty of learning a good SRL/AMR parser from QA signals hinders this direction. The difficulties of learning the two types of symbolic representations from QA signals indicate that our proposal of learning distributed representations from QA signals is a better way of making use of the latent semantic information in QA pairs for down-stream tasks. In this paper, we investigate an important problem in NLP: Can we make use of low-cost signals, such as QA data, to help related tasks? We retrieve signals from sentence-level QA pairs to help NLP tasks via two types of sentence encoding approaches. For tasks with a single-sentence input, such as SRL and NER, we propose s-Q UASE that provides latent sentence-level representations; for tasks with a sentence pair input, such as TE and MRC we propose p-Q UASE, that generates latent 10 An average of BLEU1BLEU4 scores. representations related to attentions. Experiments on a wide range of tasks show that the distinction of s-Q UASE and p-Q UASE is highly effective, and QUASEQAMR has the potential to improve on many tasks, especially in the low-resource setting. This material is based upon work supported by the US Defense Advanced Research Projects Agency (DARPA) under contracts FA8750-19-2-0201, W911NF-15-1-0461, and FA8750-19-2-1004, a grant from the Army Research Office (ARO), and Google Cloud. The views expressed are those of the authors and do not reflect the of-ficial policy or position of the Department of Defense or the U.S. Government.
abstain method result objective abstain result abstain abstain abstain abstain method method abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain result result abstain abstain objective abstain abstain method result result abstain abstain abstain method method abstain abstain abstain abstain abstain other abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain method abstain method abstain abstain abstain abstain abstain method abstain abstain abstain method method abstain abstain abstain other other other other method other other other other objective method other other objective method other abstain abstain method other other abstain abstain method method abstain abstain method method method method abstain abstain method method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain method method abstain abstain abstain abstain abstain abstain result abstain method abstain result abstain abstain abstain method abstain abstain abstain abstain result result abstain abstain abstain result abstain result abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain objective method objective abstain abstain other other