ERASER : A Benchmark to Evaluate Rationalized NLP Models Jay DeYoung   , Sarthak Jain   , Nazneen Fatema Rajani   , Eric Lehman  , Caiming Xiong  , Richard Socher  , and Byron C. Wallace   Equal contribution. Abstract State-of-the-art models in NLP are now predominantly based on deep neural networks that are opaque in terms of how they come to make predictions. This limitation has increased interest in designing more interpretable deep models for NLP that reveal the reasoning' behind model outputs. But work in this direction has been conducted on different datasets and tasks with correspondingly unique aims and metrics; this makes it difficult to track progress. We propose the E valuating R ationales A nd S imple E nglish R easoning ( ERASER ) benchmark to advance research on interpretable models in NLP. This benchmark comprises multiple datasets and tasks for which human annotations of rationales (sup-porting evidence) have been collected. We propose several metrics that aim to capture how well the rationales provided by models align with human rationales, and also how faithful these rationales are (i.e., the degree to which provided rationales influenced the corresponding predictions). Our hope is that releasing this benchmark facilitates progress on designing more interpretable NLP systems. The benchmark, code, and documentation are available at https://www.eraserbenchmark.com/ 1 Introduction Interest has recently grown in designing NLP systems that can reveal why models make specific predictions. But work in this direction has been conducted on different datasets and using different metrics to quantify performance; this has made it difficult to compare methods and track progress. We aim to address this issue by releasing a standardized benchmark of datasets  repurposed and augmented from pre-existing corpora, spanning a range of NLP tasks  and associated metrics for measuring different properties of rationales. We refer to this as the E valuating R ationales A nd S imple E nglish R easoning ( ERASER ) benchmark. In curating and releasing ERASER we take inspiration from the stickiness of the GLUE (Wang et al., 2019b) and SuperGLUE (Wang et al., 2019a) benchmarks for evaluating progress in natural language understanding tasks, which have driven rapid progress on models for general language representation learning. We believe the still somewhat nascent subfield of interpretable NLP stands to ben-efit similarly from an analogous collection of standardized datasets and tasks; we hope these will aid the design of standardized metrics to measure different properties of interpretability', and we propose a set of such metrics as a starting point. Interpretability is a broad topic with many possible realizations (Doshi-Velez and Kim, 2017; Lipton, 2016). In ERASER we focus specifically on rationales , i.e., snippets that support outputs. All datasets in ERASER include such rationales, explicitly marked by human annotators. By definition, rationales should be sufficient to make predictions, but they may not be comprehensive . Therefore, for some datasets, we have also collected comprehensive rationales (in which all evidence supporting an output has been marked) on test instances. The quality' of extracted rationales will depend on their intended use. Therefore, we propose an initial set of metrics to evaluate rationales that are meant to measure different varieties of inter-pretability'. Broadly, this includes measures of agreement with human-provided rationales, and assessments of faithfulness . The latter aim to capture the extent to which rationales provided by a model in fact informed its predictions. We believe these provide a reasonable start, but view the problem of designing metrics for evaluating rationales  especially for measuring faithfulness  as a topic for further research that ERASER can facilitate. And while we will provide a leaderboard', this is better viewed as a results board'; we do not privilege any one metric. Instead, ERASER permits comparison between models that provide rationales with respect to different criteria of interest. We implement baseline models and report their performance across the corpora in ERASER. We find that no single off-the-shelf' architecture is readily adaptable to datasets with very different instance lengths and associated rationale snippets (Section 3). This highlights a need for new models that can consume potentially lengthy inputs and adaptively provide rationales at a task-appropriate level of granularity. ERASER provides a resource to develop such models. In sum, we introduce the ERASER benchmark ( www.eraserbenchmark.com ), a unified set of diverse NLP datasets (these are repurposed and augmented from existing corpora, 1 including sentiment analysis, Natural Language Inference, and QA tasks, among others) in a standardized format featuring human rationales for decisions, along with starter code and tools, baseline models, and standardized (initial) metrics for rationales. Interpretability in NLP is a large, fast-growing area; we do not attempt to provide a comprehensive overview here. Instead we focus on directions particularly relevant to ERASER, i.e., prior work on models that provide rationales for their predictions. 1 We ask users of the benchmark to cite all original papers, and provide a BibTeX entry for doing so on the website. rationales (marked by humans) are provided during training. However, such direct supervision will not always be available, motivating work on methods that can explain (or rationalize) model predictions using only instance-level supervision. In the context of modern neural models for text classification, one might use variants of attention (Bahdanau et al., 2015) to extract rationales. Attention mechanisms learn to assign soft weights to (usually contextualized) token representations, and so one can extract highly weighted tokens as rationales. However, attention weights do not in general provide faithful explanations for predictions (Jain and Wallace, 2019; Serrano and Smith, 2019; Wiegreffe and Pinter, 2019; Zhong et al., 2019; Pruthi et al., 2020; Brunner et al., 2020; Moradi et al., 2019; Vashishth et al., 2019). This likely owes to encoders entangling inputs, complicating the interpretation of attention weights on inputs over contextualized representations of the same. 2 By contrast, hard attention mechanisms discretely extract snippets from the input to pass to the classifier, by construction providing faithful explanations. Recent work has proposed hard attention mechanisms as a means of providing explanations. Lei et al. (2016) proposed instantiating two models with their own parameters; one to extract rationales, and one that consumes these to make a prediction. They trained these models jointly via REINFORCE (Williams, 1992) style optimization. Recently, Jain et al. (2020) proposed a variant of this two-model setup that uses heuristic feature scores to derive pseudo-labels on tokens comprising rationales; one model can then be used to perform hard extraction in this way, while a second (independent) model can make predictions on the basis of these. Elsewhere, Chang et al. (2019) introduced the notion of classwise rationales that explains support for different output classes using a game theoretic framework. Finally, other recent work has proposed using a differentiable binary mask over inputs, which also avoids recourse to REINFORCE (Bastings et al., 2019). Post-hoc explanation . Another strand of interpretability work considers post-hoc explanation methods, which seek to explain why a model made a specific prediction for a given input. Commonly 2 Interestingly, Zhong et al. (2019) find that attention sometimes provides plausible but not faithful rationales. Elsewhere, Pruthi et al. (2020) show that one can easily learn to deceive via attention weights. These findings highlight that one should be mindful of the criteria one wants rationales to fulfill. these take the form of token-level importance scores. Gradient-based explanations are a standard example (Sundararajan et al., 2017; Smilkov et al., 2017). These enjoy a clear semantics (describing how perturbing inputs locally affects outputs), but may nonetheless exhibit counterintuitive behaviors (Feng et al., 2018). Gradients of course assume model differentiability. Other methods do not require any model properties. Examples include LIME (Ribeiro et al., 2016) and Alvarez-Melis and Jaakkola (2017); these methods approximate model behavior locally by having it repeatedly make predictions over perturbed inputs and fitting a simple, explainable model over the outputs. Acquiring rationales . Aside from interpretability considerations, collecting rationales from annotators may afford greater efficiency in terms of model performance realized given a fixed amount of annotator effort (Zaidan and Eisner, 2008). In particular, recent work by McDonnell et al. (2017, 2016) has observed that at least for some tasks, asking annotators to provide rationales justifying their categorizations does not impose much additional effort. Combining rationale annotation with active learning (Settles, 2012) is another promising direction (Wallace et al., 2010; Sharma et al., 2015). Learning from rationales . Work on learning from rationales marked by annotators for text classification dates back over a decade (Zaidan et al., 2007). Earlier efforts proposed extending standard discriminative models like Support Vector Machines (SVMs) with regularization terms that penalized parameter estimates which disagreed with provided rationales (Zaidan et al., 2007; Small et al., 2011). Other efforts have attempted to specify generative models of rationales (Zaidan and Eisner, 2008). More recent work has aimed to exploit rationales in training neural text classifiers. Zhang et al. (2016) proposed a rationale-augmented Convolutional Neural Network (CNN) for text classification, explicitly trained to identify sentences supporting categorizations. Strout et al. (2019) showed that providing this model with rationales during training yields predicted rationales that are preferred by humans (compared to rationales produced without explicit supervision). Other work has proposed pipeline' approaches in which independent models are trained to perform rationale extraction and classification on the basis of these, respectively (Lehman et al., 2019; Chen et al., 2019), assuming Name Size (train/dev/test) Tokens Comp? Rajani et al. (2019) fine-tuned a Transformer-based language model (Radford et al., 2018) on free-text rationales provided by humans, with an objective of generating open-ended explanations to improve performance on downstream tasks. Evaluating rationales . Work on evaluating rationales has often compared these to human judgments (Strout et al., 2019; Doshi-Velez and Kim, 2017), or elicited other human evaluations of explanations (Ribeiro et al., 2016; Lundberg and Lee, 2017; Nguyen, 2018). There has also been work on visual evaluations of saliency maps (Li et al., 2016; Ding et al., 2017; Sundararajan et al., 2017). Measuring agreement between extracted and human rationales (or collecting subjective assessments of them) assesses the plausibility of rationales, but such approaches do not establish whether the model actually relied on these particular rationales to make a prediction. We refer to rationales that correspond to the inputs most relied upon to come to a disposition as faithful . Most automatic evaluations of faithfulness measure the impact of perturbing or erasing words or tokens identified as important on model output (Ar-ras et al., 2017; Montavon et al., 2017; Serrano and Smith, 2019; Samek et al., 2016; Jain and Wallace, 2019). We build upon these methods in Section 4. Finally, we note that a recent article urges the community to evaluate faithfulness on a continuous scale of acceptability, rather than viewing this as a binary proposition (Jacovi and Goldberg, 2020). For all datasets in ERASER we distribute both reference labels and rationales marked by humans as supporting these in a standardized format. We delineate train, validation, and test splits for all corpora (see Appendix A for processing details). We ensure that these splits comprise disjoint sets of source documents to avoid contamination. 3 We have made the decision to distribute the test sets publicly, 4 in part because we do not view the cor-rect' metrics to use as settled. We plan to acquire additional human annotations on held-out portions of some of the included corpora so as to offer hidden test set evaluation opportunities in the future. Evidence inference (Lehman et al., 2019). A dataset of full-text articles describing randomized controlled trials (RCTs). The task is to infer whether a given intervention is reported to either significantly increase , significantly decrease , or have no significant effect on a specified outcome , as compared to a comparator of interest. Rationales have been marked as supporting these inferences. As the original annotations are not necessarily exhaustive, we collected exhaustive rationale annotations on a subset of the validation and test data. 5 BoolQ (Clark et al., 2019). This corpus consists of passages selected from Wikipedia, and yes/no questions generated from these passages. As the original Wikipedia article versions used were not maintained, we have made a best-effort attempt to recover these, and then find within them the passages answering the corresponding questions. For public release, we acquired comprehensive annotations on a subset of documents in our test set. 5 Movie Reviews (Zaidan and Eisner, 2008). Includes positive/negative sentiment labels on movie reviews. Original rationale annotations were not necessarily comprehensive; we thus collected comprehensive rationales on the final two folds of the original dataset (Pang and Lee, 2004). 5 In contrast to most other datasets, the rationale annotations here are span level as opposed to sentence level. FEVER (Thorne et al., 2018). Short for Fact Extraction and VERification; entails verifying claims from textual sources. Specifically, each claim is to be classified as supported , refuted or not enough information with reference to a collection of source 3 Except for BoolQ, wherein source documents in the original train and validation set were not disjoint and we preserve this structure in our dataset. Questions , of course, are disjoint. 4 Consequently, for datasets that have been part of previous benchmarks with other aims (namely, GLUE/superGLUE) but which we have re-purposed for work on rationales in ERASER, e.g., BoolQ (Clark et al., 2019), we have carved out for release test sets from the original validation sets. 5 Annotation details are in Appendix B. texts. We take a subset of this dataset, including only supported and refuted claims. MultiRC (Khashabi et al., 2018). A reading comprehension dataset composed of questions with multiple correct answers that by construction depend on information from multiple sentences. Here each rationale is associated with a question, while answers are independent of one another. We convert each rationale/question/answer triplet into an instance within our dataset. Each answer candidate then has a label of True or False . Commonsense Explanations (CoS-E) (Rajani et al., 2019). This corpus comprises multiple-choice questions and answers from (Talmor et al., 2019) along with supporting rationales. The rationales in this case come in the form both of highlighted (extracted) supporting snippets and free-text, open-ended descriptions of reasoning. Given our focus on extractive rationales, ERASER includes only the former for now. Following Talmor et al. (2019), we repartition the training and validation sets to provide a canonical test split. e-SNLI (Camburu et al., 2018). This dataset augments the SNLI corpus (Bowman et al., 2015) with rationales marked in the premise and/or hypothesis (and natural language explanations, which we do not use). For entailment pairs, annotators were required to highlight at least one word in the premise. For contradiction pairs, annotators had to highlight at least one word in both the premise and the hypothesis; for neutral pairs, they were only allowed to highlight words in the hypothesis. Human Agreement We report human agreement over extracted rationales for multiple annotators and documents in Table 2. All datasets have a high Cohen  (Cohen, 1960); with substantial or better agreement. In ERASER models are evaluated both for their predictive performance and with respect to the rationales that they extract. For the former, we rely on the established metrics for the respective tasks. Here we describe the metrics we propose to evaluate the quality of extracted rationales. We do not claim that these are necessarily the best metrics for evaluating rationales, however. Indeed, we hope the release of ERASER will spur additional research into how best to measure the quality of model explanations in the context of NLP. The simplest means of evaluating extracted rationales is to measure how well they agree with those marked by humans. We consider two classes of metrics, appropriate for models that perform discrete and soft' selection, respectively. For the discrete case, measuring exact matches between predicted and reference rationales is likely too harsh. 6 We thus consider more relaxed measures. These include Intersection-Over-Union (IOU), borrowed from computer vision (Evering-ham et al., 2010), which permits credit assignment for partial matches. We define IOU on a token level: for two spans, it is the size of the overlap of the tokens they cover divided by the size of their union. We count a prediction as a match if it overlaps with any of the ground truth rationales by more than some threshold (here, 0.5). We use these partial matches to calculate an F1 score. We also measure token -level precision and recall, and use these to derive token-level F1 scores. Metrics for continuous or soft token scoring models consider token rankings, rewarding models for assigning higher scores to marked tokens. In particular, we take the Area Under the Precision-Recall curve (AUPRC) constructed by sweeping a threshold over token scores. We define additional metrics for soft scoring models below. In general, the rationales we have for tasks are sufficient to make judgments, but not necessarily comprehensive . However, for some datasets we have explicitly collected comprehensive rationales for at least a subset of the test set. Therefore, on these datasets recall evaluates comprehensiveness directly (it does so only noisily on other datasets). As discussed above, a model may provide rationales that are plausible (agreeable to humans) but that it did not rely on for its output. In many settings one may want rationales that actually explain model predictions, i.e., rationales extracted for an instance in this case ought to have meaningfully influenced its prediction for the same. We call these faithful rationales. How best to measure rationale faithfulness is an open question. In this first version of ERASER we propose simple metrics motivated by prior work (Zaidan et al., 2007; Yu et al., 2019). In particular, following Yu et al. (2019) we define metrics intended to measure the comprehensiveness (were all features needed to make a prediction se-lected?) and sufficiency (do the extracted rationales contain enough signal to come to a disposition?) of rationales, respectively. Comprehensiveness . To calculate rationale comprehensiveness we create contrast examples (Zaidan et al., 2007): We construct a contrast example for x i ,  x i , which is x i with the predicted rationales r i removed. Assuming a classification setting, let m ( x i ) j be the original prediction provided by a model m for the predicted class j . Then we consider the predicted probability from the model for the same class once the supporting rationales are stripped. Intuitively, the model ought to be less confident in its prediction once rationales are removed from x i . We can measure this as: comprehensiveness = m ( x i ) j  m ( x i / r i ) j (1) A high score here implies that the rationales were indeed influential in the prediction, while a low score suggests that they were not. A negative value Where do you find the most amount of leafs? here means that the model became more confident in its prediction after the rationales were removed; this would seem counter-intuitive if the rationales were indeed the reason for its prediction. Sufficiency . This captures the degree to which the snippets within the extracted rationales are adequate for a model to make a prediction. These metrics are illustrated in Figure 2. As defined, the above measures have assumed discrete rationales r i . We would also like to evaluate the faithfulness of continuous importance scores assigned to tokens by models. Here we adopt a simple approach for this. We convert soft scores over features s i provided by a model into discrete rationales r i by taking the top  k d values, where k d is a threshold for dataset d . We set k d to the average rationale length provided by humans for dataset d (see Table 4). Intuitively, this says: How much does the model prediction change if we remove a number of tokens equal to what humans use (on average for this dataset) in order of the importance scores assigned to these by the model. Once we have discretized the soft scores into rationales in this way, we compute the faithfulness scores as per Equations 1 and 2. This approach is conceptually simple. It is also computationally cheap to evaluate, in contrast to measures that require per-token measurements, e.g., importance score correlations with leave-one-out' scores (Jain and Wallace, 2019), or counting how many important' tokens need to be erased before a prediction flips (Serrano and Smith, 2019). However, the necessity of discretizing continuous scores forces us to pick a particular threshold k . We can also consider the behavior of these measures as a function of k , inspired by the measurements proposed in Samek et al. (2016) in the context of evaluating saliency maps for image classification. They suggested ranking pixel regions by importance and then measuring the change in output as they are removed in rank order. Our datasets comprise documents and rationales with quite different lengths; to make this measure comparable across datasets, we construct bins designating the number of tokens to be deleted. Denoting the tokens up to and including bin k for instance i by r ik , we define an aggregate comprehensiveness measure: 1 B + 1 ( B  k = 0 m ( x i ) j  m ( x i / r ik ) j ) (3) This is defined for sufficiency analogously. Here we group tokens into k = 5 bins by grouping them into the top 1%, 5%, 10%, 20% and 50% of tokens, with respect to the corresponding importance score. We refer to these metrics as Area Over the Perturbation Curve (AOPC). 7 These AOPC sufficiency and comprehensiveness measures score a particular token ordering under a model. As a point of reference, we also report these when random scores are assigned to tokens. 7 Our AOPC metrics are similar in concept to ROAR (Hooker et al., 2019) except that we re-use an existing model as opposed to retraining for each fraction. Our focus in this work is primarily on the ERASER benchmark itself, rather than on any particular model(s). But to establish a starting point for future work, we evaluate several baseline models across the corpora in ERASER. 8 We broadly classify these into models that assign soft' (continuous) scores to tokens, and those that perform a hard' (discrete) selection over inputs. We additionally consider models specifically designed to select individual tokens (and very short sequences) as rationales, as compared to longer snippets. All of our implementations are in PyTorch (Paszke et al., 2019) and are available in the ERASER repository. 9 All datasets in ERASER comprise inputs, rationales, and labels. But they differ considerably in document and rationale lengths (Table A). This motivated use of different models for datasets, appropriate to their sizes and rationale granularities. We hope that this benchmark motivates design of models that provide rationales that can flexibly adapt to varying input lengths and expected rationale granularities. Indeed, only with such models can we perform comparisons across all datasets. Models that perform hard selection may be viewed as comprising two independent modules: an encoder which is responsible for extracting snippets of inputs, and a decoder that makes a prediction based only on the text provided by the encoder. We consider two variants of such models. Lei et al. (2016) . In this model, an encoder induces a binary mask over inputs x , z . The decoder accepts the tokens in x unmasked by z to make a prediction  y . These modules are trained jointly via REINFORCE (Williams, 1992) style estimation, minimizing the loss over expected binary vectors z yielded from the encoder. One of the advantages of this approach is that it need not have access to marked rationales; it can learn to rationalize on the basis of instance labels alone. However, given that we do have rationales in the training data, we experiment with a variant in which we train the encoder explicitly using rationale-level annotations. In our implementation of Lei et al. (2016), we drop in two independent BERT (Devlin et al., 2019) or GloVe (Pennington et al., 2014) base modules 8 This is not intended to be comprehensive. with bidirectional LSTMs (Hochreiter and Schmid-huber, 1997) on top to induce contextualized representations of tokens for the encoder and decoder, respectively. The encoder generates a scalar (de-noting the probability of selecting that token) for each LSTM hidden state using a feedfoward layer and sigmoid. In the variant using human rationales during training, we minimize cross entropy loss over rationale predictions. The final loss is then a composite of classification loss, regularizers on rationales (Lei et al., 2016), and loss over rationale predictions, when available. Pipeline models . These are simple models in which we first train the encoder to extract rationales, and then train the decoder to perform prediction using only rationales. No parameters are shared between the two models. Here we first consider a simple pipeline that first segments inputs into sentences. It passes these, one at a time, through a Gated Recurrent Unit (GRU) (Cho et al., 2014), to yield hidden representations that we compose via an attentive decoding layer (Bahdanau et al., 2015). This aggregate representation is then passed to a classification module which predicts whether the corresponding sentence is a rationale (or not). A second model, using effectively the same architecture but parameterized independently, consumes the outputs (rationales) from the first to make predictions. This simple model is described at length in prior work (Lehman et al., 2019). We further consider a BERT-to-BERT' pipeline, where we replace each stage with a BERT module for prediction (Devlin et al., 2019). In pipeline models, we train each stage independently. The rationale identification stage is trained using approximate sentence boundaries from our source annotations, with randomly sampled negative examples at each epoch. The classification stage uses the same positive rationales as the identification stage, a type of teacher forcing (Williams and Zipser, 1989) (details in Appendix C). We consider a model that passes tokens through BERT (Devlin et al., 2019) to induce contextualized representations that are then passed to a bidirectional LSTM (Hochreiter and Schmidhuber, 1997). The hidden representations from the LSTM are collapsed into a single vector using additive attention (Bahdanau et al., 2015). The LSTM layer allows us to bypass the 512 word limit imposed by Perf. BERT; when we exceed this, we effectively start encoding a new' sequence (setting the positional index to 0) via BERT. The hope is that the LSTM learns to compensate for this. Evidence Inference and BoolQ comprise very long ( > 1000 token) inputs; we were unable to run BERT over these. We instead resorted to swapping GloVe 300d embeddings (Pennington et al., 2014) in place of BERT representations for tokens. spans. To soft score features we consider: Simple gradients, attention induced over contextualized representations, and LIME (Ribeiro et al., 2016). Here we present initial results for the baseline models discussed in Section 5, with respect to the metrics proposed in Section 4. We present results in two parts, reflecting the two classes of rationales discussed above: Hard' approaches that perform discrete selection of snippets, and soft' methods that assign continuous importance scores to tokens. In Table 3 we evaluate models that perform discrete selection of rationales. We view these as inherently faithful, because by construction we know which snippets the decoder used to make a prediction. 10 Therefore, for these methods we report only metrics that measure agreement with human annotations. 10 This assumes independent encoders and decoders. Due to computational constraints, we were unable to run our BERT-based implementation of Lei et al. (2016) over larger corpora. Conversely, the simple pipeline of Lehman et al. (2019) assumes a setting in which rationale are sentences, and so is not appropriate for datasets in which rationales tend to comprise only very short spans. Again, in our view this highlights the need for models that can rationalize at varying levels of granularity, depending on what is appropriate. We observe that for the rationalizing model of Lei et al. (2016), exploiting rationale-level supervision often (though not always) improves agreement with human-provided rationales, as in prior work (Zhang et al., 2016; Strout et al., 2019). Interestingly, this does not seem strongly correlated with predictive performance. Lei et al. (2016) outperforms the simple pipeline model when using a BERT encoder. Further, Lei et al. (2016) outperforms the BERT-to-BERT' pipeline on the comparable datasets for the final prediction tasks. This may be an artifact of the amount of text each model can select: BERT-to-BERT' is limited to sentences, while Lei et al. (2016) can select any subset of the text. Designing extraction models that learn to adaptively select contiguous rationales of appropriate length for a given task seems a potentially promising direction. In Table 4 we report metrics for models that assign continuous importance scores to individual tokens. For these models we again measure downstream (task) performance (macro F1 or ac-curacy). Here the models are actually the same, and so downstream performance is equivalent. To assess the quality of token scores with respect to human annotations, we report the Area Under the Precision Recall Curve (AUPRC). These scoring functions assign only soft scores to inputs (and may still use all inputs to come to a particular prediction), so we report the metrics intended to measure faithfulness defined above: comprehensiveness and sufficiency, averaged over bins' of tokens ordered by importance scores. To provide a point of reference for these metrics  which depend on the underlying model  we report results when rationales are randomly selected (averaged over 10 runs). Both simple gradient and LIME-based scoring yield more comprehensive rationales than attention weights, consistent with prior work (Jain and Wallace, 2019; Serrano and Smith, 2019). Attention fares better in terms of AUPRC  suggesting better agreement with human rationales  which is also in line with prior findings that it may provide plausible, but not faithful, explanation (Zhong et al., 2019). Interestingly, LIME does particularly well across these tasks in terms of faithfulness. From the Random' results that we conclude models with overall poor performance on their final tasks tend to have an overall poor ordering, with marginal differences in comprehensiveness and sufficiency between them. For models that with high sufficiency scores: Movies, FEVER, CoS-E, and e-SNLI, we find that random removal is particularly damaging to performance, indicating poor absolute ranking; whereas those with high comprehensiveness are sensitive to rationale length. We have introduced a new publicly available resource: the Evaluating Rationales And Simple English Reasoning (ERASER) benchmark. This comprises seven datasets, all of which include both instance level labels and corresponding supporting snippets (rationales') marked by human annotators. We have augmented many of these datasets with additional annotations, and converted them into a standard format comprising inputs, rationales, and outputs. ERASER is intended to facilitate progress on explainable models for NLP. We proposed several metrics intended to measure the quality of rationales extracted by models, both in terms of agreement with human annotations, and in terms of faithfulness'. We believe these metrics provide reasonable means of comparison of specific aspects of interpretability, but we view the problem of measuring faithfulness, in particular, a topic ripe for additional research (which ERASER can facilitate). Our hope is that ERASER enables future work on designing more interpretable NLP models, and comparing their relative strengths across a variety of tasks, datasets, and desired criteria. It also serves as an ideal starting point for several future directions such as better evaluation metrics for interpretability, causal analysis of NLP models and datasets of rationales in other languages. This work was supported in part by the NSF (CA-REER award 1750978), and by the Army Research Office (W911NF1810328).
abstain abstain abstain abstain objective abstain objective method other abstain objective abstain method objective abstain method abstain abstain method abstain objective abstain abstain method result abstain method result abstain abstain objective method method method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method method abstain abstain abstain method abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain method abstain abstain abstain abstain abstain method abstain abstain abstain abstain method abstain abstain method abstain method method abstain method abstain method abstain method abstain method method abstain method method abstain method abstain abstain abstain abstain abstain abstain method abstain abstain abstain method abstain method abstain abstain abstain abstain abstain abstain method method method abstain method method abstain abstain method abstain abstain method method abstain abstain abstain abstain method abstain method method method abstain abstain abstain abstain abstain method abstain method abstain abstain abstain abstain abstain method abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain method method method abstain method abstain abstain abstain abstain abstain abstain abstain method abstain abstain method method method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain method abstain abstain abstain abstain abstain abstain objective abstain objective abstain objective method method abstain other