Recent years have witnessed the emergence of a variety of post-hoc interpretations that aim to uncover how natural language processing (NLP) models make predictions. Despite the surge of new interpretation methods, it remains an open problem how to define and quantitatively measure the faithfulness of interpretations, i.e., to what extent interpretations reflect the reasoning process by a model. We propose two new criteria, sensitivity and stability, that provide complementary notions of faithfulness to the existed removal-based criteria. Our results show that the conclusion for how faithful interpretations are could vary substantially based on different notions. Motivated by the desiderata of sensitivity and stability, we introduce a new class of interpretation methods that adopt techniques from adversarial robustness. Empirical results show that our proposed methods are effective under the new criteria and overcome limitations of gradient-based methods on removal-based criteria. Besides text classification, we also apply interpretation methods and metrics to dependency parsing. Our results shed light on understanding the diverse set of interpretations. As complex NLP models are widely deployed in real-world applications, there is an increasing interest in understanding how these models come to certain decisions. As a result, the line of research on interpretation techniques grows rapidly, facilitating a broad range of model analysis, from building user trust on models (Ribeiro et al., 2016; Hase and Bansal, 2020) to exposing subtle biases (Zhao et al., 2017; Doshi-Velez and Kim, 2017). In this paper, we focus on post-hoc interpretations in NLP. Given a trained model and a specific input text, post-hoc interpretations assign an importance score to each token in the input which indicates its contribution to the model output. Current methods in this direction can be roughly divided into three categories: gradient-based methods (Simonyan et al., 2014; Li et al., 2016); reference-based methods (Sundararajan et al., 2017; Shrikumar et al., 2017); and perturbation-based methods (Zeiler and Fergus, 2014; Ribeiro et al., 2016). Despite the emergence of new techniques, one critical issue is that there is little consensus on how to define and evaluate the faithfulness of these techniques, i.e., whether they reflect the true reasoning process by a model. A widely employed criterion, especially in NLP, is the removal-based criterion (DeYoung et al., 2020), which removes or only preserves a set of tokens given by interpretations and measures how much the model prediction would change. However, as pointed out in prior work (Bastings and Filippova, 2020; Ancona et al., 2018), the corrupted version of an input produced during evaluations falls out of the distribution that models are trained on, and thus results in an inaccurate measurement of faithfulness. This limitation prevents removal-based metrics from being used as the golden standard for evaluating interpretations. To remedy this, we complement the removal-based criterion with two other criteria, sensitivity and stability , which are overlooked in prior works. Sensitivity is based on the notion that models should be more sensitive to perturbations on tokens identified by a faithful explanation. In contrast to the removal-based criterion, which completely removes important tokens, the sensitivity criterion adds small but adversarial perturbations in a local region of the token embedding, and thus preserves the structure of input sentences as well as interactions between context words. This criterion is recently discussed in Hsieh et al. (2020) in computer vision, while we provide comprehensive analyses on various NLP models and tasks. Note that while the removal-based criterion asks the question: if some important tokens did not exist', what would happen , the sensitivity criterion asks: if some im-2631 portant tokens were changed' adversarially, what would happen . Stability assumes that a faithful interpretation should not produce substantially different explanations for two inputs that the model finds similar. There are several attempts to generate such a pair of inputs. The most relevant one is Ghorbani et al. (2019). However, their method is only applicable to differentiable interpretations. Our work proposes a new paradigm based on adversarial word substitution that employs a black-box algorithm to generate a semantically related neighbor of the original input, which is specially designed for NLP and applicable to all interpretations techniques. The above two metrics highlight the connection between interpretability and robustness. Experiments show that interpretations which perform well on the removal-based criterion might not do well on the new criteria. Motivated by the limitations of existing interpretations and the desiderata of sensitivity, we propose robustness-based methods , based on projected gradient descent (PGD) attacks (Madry et al., 2018) and certifying robustness (Jia et al., 2019; Huang et al., 2019; Shi et al., 2020; Xu et al., 2020). We demonstrate that the new methods achieve top performance under sensitivity and stability. Moreover, as a simple improvement to gradient-based methods, our methods avoid the gradient saturation issues of gradient-based methods under the removal-based criterion. Another limitation of removal-based metrics emerges when interpreting dependency parsing  when input tokens are removed, the tree structure is drastically changed and a model might not be able to produce a meaningful parse tree. Thus, there are little discussion for dependency parsing interpretations. In this paper, we propose a new paradigm to interpret dependency parsers leveraging prepositional phrase (PP) attachment ambiguity examples. To our best knowledge, this is the first work to study interpretations on dependency parsing. We demonstrate that sensitivity does not change the output tree structure as much as removal-based ones do, and provide analyses for interpretation methods with our paradigm and metrics. Our contributions can be summarized as follows. 1. We discuss two overlooked notions of faithfulness in NLP interpretations. Our notions emphasize the connection between interpretability and robustness. We systematically evaluate interpretations under these notions, including existed removal-based ones. The code for this paper could be found at https://github.com/uclanlp/ NLP-Interpretation-Faithfulness . 2. We propose new robustness-based interpretations inspired by the sensitivity metric and demonstrate their effectiveness under both sensitivity and stability. 3. We propose a novel paradigm to evaluate interpretations on the dependency parsing task. A faithful post-hoc interpretation identifies the important parts of the input a model prediction relies on. Let x = [ x 1 ; x 2 ; . . . ; x n ] be a sequence of tokens. e (  ) denotes the token embedding function. An NLP model f takes the embedding matrix e ( x )  R n  d as input and provides its prediction f ( e ( x )) = y . Let s y ( e ( x )) denote the output score of f ( e ( x )) on y . The exact form of s y ( e ( x )) is defined in Appendix D. An interpretation assigns an importance score to each token which indicates its contribution to the model decision. We first review the well-established removal-based criterion and emphasize its relation to the two criteria defined in this paper 1) sensitivity , and 2) stability , for which we propose novel paradigms to adapt them to various NLP tasks. Removal-based Criterion A well-established notion of interpretation faithfulness is that the presence of important tokens should have more meaningful influence on the model's decision than random tokens, quantified by the removal-based criterion. We adopt the comprehensiveness and the sufficiency score in DeYoung et al. (2020). The comprehensiveness score measures how much the model performance would drop after the set of relevant" tokens identified by an interpretation is removed. A higher comprehensiveness score suggests the tokens are more influential to the model output, and thus a more faithful explanation. The sufficiency score measures to what extent the original model performance is maintained when we solely preserve relevant tokens. A lower sufficiency score means less change in the model prediction, and thus a more faithful explanation. See DeYoung et al. (2020) for detailed definitions. Note that completely removing input tokens produces incomplete texts. Large perturbation of this kind lead to several issues as pointed out by prior studies (Feng et al., 2018; Bastings and Filippova, 2020). 2632 Ours: Sensitivity Instead of removing important tokens, the sensitivity criterion adds local but adversarial noise to embedding vectors of the important tokens and measures the magnitude of the noise needed to change the model prediction. This is inspired by the notion that models should be more sensitive to perturbations being added to relevant tokens compared to random or irrelevant tokens. From the adversarial robustness perspective (Hsieh et al., 2020), this notion implies that by perturbing the most relevant tokens, we can reach the local decision boundary of a model with the minimum perturbation magnitude. Given the sequence of relevant tokens r k , sensitivity adds perturbation to its embedding e ( r k ) but keeps the remaining token embeddings unchanged. Then, it measures the minimal perturbation norm, denoted as  r k , that changes the model prediction for this instance:  r k = min   r k  F s.t. f ( e ( x ) +  r k )  = y, where  F is the Frobenius norm of a matrix, and  r k  R n  d denotes the perturbation matrix where only the columns for tokens in r k have non-zero elements. Since the exact computation of  r k is intractable, we use the PGD attack (Madry et al., 2018) with a binary search to approximate  r k . A lower  r k suggests a more faithful interpretation. In practice, we vary the size of r k , compute multiple  r k , and summarize them with the area under the curve (AUC) score. Ours: Stability Another desired property of faithfulness is that a faithful interpretation should not give substantially different importance orders for two input points that the model finds similar. To construct a pair of similar inputs, we propose to generate contrast examples to the original one by synonym substitutions. A contrast example of x ,  x , satisfies (1) has at most k different but synonymous tokens with x ; (2) the prediction score at  x changes less than  compared to the score at x . The goal of these two conditions is to generate (almost) natural examples where the changes of model outputs are smaller than a threshold  . Given all contrast examples, we search for the one that leads to the largest rank difference D between the importance order for x , m ( x ) and the alternated order m ( x ) : arg max  x D ( m ( x ) , m ( x )) , s.t. | s y ( e ( x ))  s y ( e ( x )) |  ,  x   x  0  k. Specifically, we first extract synonyms for each token x i following Alzantot et al. (2018). Then, in the decreasing order of m ( x ) , we greedily search for a substitution of each token that induces the largest change in m ( x ) and repeat this process until the model output score changes by more than  or the pre-defined constraint k is reached. Finally, we measure the difference D between two importance ranks using Spearman's rank order correlation (Spearman, 1961). We call this criterion stability . A higher score indicates that the ranks between this input pair are more similar, and thus a more faithful interpretation. Note that instead of using the gradient information of interpretation methods to perturb importance ranks like Ghorbani et al. (2019), our algorithm treats interpretations as black-boxes, which makes it applicable to non-differentiable ones. Also, compared to Ding and Koehn (2021), who manually construct similar input pairs, our method is a fully automatic one as suggested by their paper. 3 Interpretations via Adversarial Robustness Techniques Experiments indicate that existing methods do not work well with the sensitivity and stability metrics (Sec. 4.2). In this section, we define a new class of interpretation methods by adopting techniques in adversarial robustness to remedy this. We first give a brief review of existing interpretation approaches and then introduce our new methods. 3.1 Existing Interpretation Methods We roughly divide the existing methods into three categories: gradient-based methods , reference-based methods , and perturbation-based methods , and discuss the representatives of them. Gradient-based methods The first class of methods leverage the gradient at each input token. To aggregate the gradient vector at each token into a single importance score, we consider two methods: 1) using the L 2 norm, (cid:13)(cid:13)(cid:13) s y ( e ( x )) e ( x i ) (cid:13)(cid:13)(cid:13) 2 , referred to as Vanilla Gradient (VaGrad) (Simonyan et al., 2014), and 2) using the dot product of gradient and input, (cid:16) s y ( e ( x )) e ( x i ) (cid:17)   e ( x i ) , referred to as Gradient  Input (GradInp) (Li et al., 2016). Reference-based methods These methods distribute the difference between model outputs on a reference point and on the input as the importance score for each token. We consider Inte-2633 grated Gradient (IngGrad) (Sundararajan et al., 2017) and DeepLIFT (Shrikumar et al., 2017). IngGrad computes the linear intergral of the gradients from the reference point to the input. DeepLIFT decomposes the difference between each neuron activation and its reference activation' and back-propagates it to each input token. We use DeepLIFT with the Rescale rule. Note DeepLIFT diverges from IngGrad when multiplicative interactions among tokens exist (Ancona et al., 2018). Perturbation-based methods Methods in this class query model outputs on perturbed inputs. We choose Occlusion (Zeiler and Fergus, 2014) and LIME (Ribeiro et al., 2016). Occlusion replaces one token at a time by a reference value and uses the corresponding drop on model performance to represent the importance of each token. LIME uses a linear model to fit model outputs on the neighborhood of input x and represents token importance by the weights in the trained linear model. 3.2 Proposed Robustness-based Methods We propose two methods inspired from the PGD attack (Madry et al., 2018) and the certifying robustness algorithms (Xu et al., 2020) in adversarial robustness. VaPGD and PGDInp The PGD attack in adversarial robustness considers a small vicinity of the input and takes several mini-steps" within the vicinity to search for an adversarial example. Consider the token embeddings for the input x , we perform t iterations of the standard PGD procedure starting from e (0) = e ( x ) : e ( j ) = P (cid:16) e ( j  1)    s y (cid:16) e ( j  1) (cid:17)(cid:17) , j =1 , 2 , . . . , t. P represents the operation that projects the new instance at each step back to the vicinity of e ( x ) , and  is the step size. Intuitively, e ( t )  e ( x ) tells us the descent direction of model confidence. Similar to the gradient-based methods, the importance of each token x i can be either represented by (cid:13)(cid:13)(cid:13) e ( t ) i  e ( x i ) (cid:13)(cid:13)(cid:13) 2 , where e ( t ) i is the i-th column in e ( t ) , referred to as Vanilla PGD (VaPGD), or by (cid:16) e ( x i )  e ( t ) i (cid:17)   e ( x i ) , referred to as PGD  Input (PGDInp) Note that different from the PGD attack we use for approximating the sensitivity criterion, we manually decide the magnitude of the vicinity of e ( x ) instead of using a binary search. We add perturbations to the whole sentence at the same time. Also, the final e ( t ) does not necessarily change the model prediction. Certify Certifying robustness algorithms also consider a vicinity of the original input and aim to provide guaranteed lower and upper bounds of a model output within that region. We use the linear relaxation based perturbation analysis (LiRPA) discussed in (Shi et al., 2020; Xu et al., 2020). LiRPA looks for two linear functions that bound the model. Specifically, LiRPA computes W , W , b , and b that satisfy (cid:80) i W i e ( x  i )+ b  s y ( e ( x  ))  (cid:80) i W i e ( x  i ) + b for any point e ( x  ) that lies within the L 2 ball of e ( x ) with size  . We use the IBP+backward method in Xu et al. (2020). It uses Interval Bound Propagation (Gowal et al., 2018; Mirman et al., 2018) to compute bounds of internal neurons of the model and then constructs the two linear functions with a bound back-propagation process (Zhang et al., 2018; Singh et al., 2019). Finally, the importance score of the i -th token in the input is represented by W i  e ( x i ) , where W i is the i -th row of W . We call this method Certify . Robustness-based vs. Gradient-based Gradient-based methods provide a linear approximation of the model decision boundary at the single input, which is not accurate for non-linear models. Robustness-based methods instead search multiple steps in neighbors and approximate the steepest descent direction better. We also empirically show that robustness-based methods avoid the saturation issue of gradient-based methods, i.e, gradient becomes zero at some inputs. See Appendix H. Note that VaPGD (PGDInp) degrades to VaGrad (Grad-Inp) when the number of iterations is 1. Robustness-based vs. IngGrad IngGrad leverages the average gradient in a segment between the input and a reference. It is likely to neglect local properties desired by the sensitivity criterion. Robustness-based methods instead search in the vicinity of the input, and thus local properties are better preserved. See results in Sec. 4.2. In this section, we present the results on text classification tasks under the three criteria. We find that the correlation between interpretation faithfulness based on different criteria are relatively low in some cases. Results verify the effectiveness of our new methods. Datasets We conduct experiments on three text classification datasets: SST-2 (Socher et al., 2013), Yelp (Zhang et al., 2015), and AGNews (Zhang et al., 2015) following Jain and Wallace (2019)'s preprocessing approach. All of them are converted to binary classification tasks. SST-2 and Yelp are sentiment classification tasks where models predict whether a review is negative (0) or positive (1). AGNews is to discriminate between world (0) and business (1) articles. See Appendix A for statistics of the three datasets. When evaluating interpretation methods, for each dataset, we select 200 random samples (100 samples from class 0 and 100 samples from class 1) from the test set. Models For text classification, we consider two model architectures: BERT (Devlin et al., 2019) and BiLSTM (Hochreiter and Schmidhuber, 1997). Interpretation Methods Besides our robustness-based interpretations PGDInp , VaPGD , and Certify , we experiment with six others from three existing categories: VaGrad , GradInp (gradient-based); IngGrad , DeepLIFT (reference-based); and Occlusion , LIME (perturbation-based). We also include a random baseline Random that randomly assigns importance scores. We use comprehensiveness ( Comp. ), sufficiency ( Suff. ), sensitivity ( Sens. ), and stability ( Stab. ) as metrics. Overall Results Results of interpretations for BERT and BiLSTM are presented in Table 1 and 2. The interpretations' performance are averaged over three runs on models trained from different random seeds. Results verify the effectiveness of our proposed robustness-based methods. Specifically, VaPGD achieves the best performance under the sensitivity and the stability criteria for both BERT and BiLSTM. Our methods also outperform their gradient-based counterparts under removal-based criteria. Especially, when interpreting BERT on SST-2 and AGNews, GradInp has near random performance. PGDInp can avoid these unreasonable behaviors. See Appendix H for a qualitative study on this, where we find PGDInp does not suffer from the saturation issue as GradInp. Also notice that the superior performance of robsutness-based methods are consistent on BERT and BiLSTM+GloVe, which demonstrate that it is not influenced by the embeddings being used. However, the performance of other methods tend to be inconsistent under different measurements. For example, under the removal-based criterion, IngGrad performs well for BiLSTM, which gives four out of six best numbers. But, IngGrad has very limited performance under the sensitivity metric, especially for BiLSTM on SST-2 and Yelp. Similar issues exist for LIME and Occlusion. Also, one might fail to recognize the faithfulness of VaPGD by solely looking at the removal-based criterion. Thus, when deploying interpretation methods on real tasks, we advocate for a careful selection of the method you use based on the underlying faithfulness notion that aligned with your goal. Performance Curves To show how the size of the relevant set affects interpretation performance, we plot the comprehensiveness and the sensitivity curves when increasing the number of tokens being removed (perturbed). Consider interpreting BERT on Yelp as an example, we collect two groups of examples from the test set of Yelp based on input lengths, where examples in each group are of 30  5 and 120  5 tokens long, and remove (perturb) the topk most important tokens given by interpretations. Results are shown in Figure 2. As shown in the figure, Occlusion is able to discover a smaller set of impactful tokens, under both metrics. However, when the size of the relevant 2635 SST-2 Yelp AGNews Methods Comp. set is increased, the performance of IngGrad under the comprehensiveness metric and the performance of VaPGD under the sensitivity metric gradually surpass Occlusion and other methods. This implies that the two methods are better at identifying a relevant set with more tokens. Interpolation Analysis To check whether the comprehensiveness and sensitivity scores can reflect the relative importance of each token in the relevant set, we conduct an interpolation analysis that gradually replaces each token in the relevant set with a random token outside of the set. Specifically, we select 50 examples from SST-2 and test on BERT with relevant sets given by LIME and VaPGD. For each example, we extract a relevant set consists of the top four most important tokens and gradually replace each token, from the least to the most important one, with a random token. We denote the relevant set at each step as S 0 , S 1 , ..., S 4 , where S 0 is the original relevant set containing the top four tokens and S 4 is the set of four random tokens. The performance change at step i is represented by f ( i ) = | M ( S 0 )  M ( S i ) | | M ( S 0 )  M ( S 4 ) | , where M is the comprehensiveness or sensitivity score. We expect that a good metric should induce a monotonically increasing function f . Further, f should be strictly convex as that indicates the importance of each token is different. We plot the curve in Figure 3. Results show that both the comprehensiveness and sensitivity metrics generate a monotonically increasing function, which indicates that they fully consider each token in the relevant set. Also, notice that based on the comprehensiveness metric, the contribution of each token tends to distribute evenly within the relevant set, which contradicts the fact that tokens in the set have different contribution to the prediction, while the importance rank is better preserved based on the sensitivity metric,. we qualitatively study the notions of faithfulness defined by comprehensiveness ( comp. ) and sensitivity ( sens. ), and discuss two main differences. evaluations, which could possibly break the interaction between removed tokens and context tokens, and underestimate the importance of context tokens. In Figure 1(a), the tokens not' and hold' together determine the negative sentiment of the sentence. Sens. considers both not' and hold' as important tokens as one expects. However, comp. regards hold' less important than will'. Second, sens. measures token importance by how much model performance would change after adversarially perturbing' that token. In this sense, both positive and negative pertinent tokens will be deemed important. In contrast, comp. only considers positive pertinent ones. In Figure 1(b), which is predicted as positive, removing the negative verb hate' would not influence model performance much. However, adversarially perturbing hate' (e.g. change hate' to a more negative verb) might change the model prediction from positive to negative. Thus, sens. prefers interpretations that identify hate' as an important token like VaPGD. The full version of Figure 1(b) is in Appendix E. Some contrast examples generated for the stability criterion are presented in Appendix F. 5 Experiments on Structured Prediction Structured prediction tasks are in the center of NLP applications. However, applying interpretation methods and criteria to these tasks are difficult because 1) the required output is a structure instead of a single score. It is hard to define the contribution of each token to a structured output, and 2) compared to text classification tasks, removing parts of the input like what removal-based criteria do, would cause more drastic changes to model predictions as well as the groundtruth. Therefore, existing works often conduct experiments only on binary or multi-class text classification tasks. To remedy these issues, we investigate interpretations for dependency parsing, with an special focus on analyzing how models resolve the PP attachment ambiguity, which avoids interpreting the structured output as a whole. We show that our sensitivity metric is a better metric for dependency parsing as it causes negligible changes to model outputs compared to removal-based metrics. Our paradigm focuses on the PP attachment ambiguity, which involves both syntactic and semantics considerations. A dependency parser needs to determine either the preposition in PP attaches to the preceding noun phrase NP (NP-attachment) or the verb phrase VP (VP-attachment) (Hindle and Rooth, 1993). The basic structure of ambiguity is VP  NP  PP. For example, in the sentence I saw a cat with a telescope , a parser uses the semantics of the noun phrase a telescope to predict the head of with , which is saw . If we change a telescope to 2637 PTB-SD Method Comp. a tail , the head of with would become the preceding noun cat . We will later call nouns in PPs like telescope disambiguating nouns", as they provide semantic information for a parser to disambiguate PP attachment ambiguity. The main advantage of this paradigm is that disambiguating nouns can be viewed as proxy groundtruths for faithfulness as parsers must rely on them to make decisions. Experimental Setup We use DeepBiaffine, a graph-based dependency parser as the target model (Dozat and Manning, 2017). We extract 100 examples that contain the PP attachment ambiguity from the English Penn Treebank converted to Stanford Dependencies 3.5.0 (PTB-SD). We consider the same interpretation methods as before, and they assign an importance score to each token in the sentence to indicate how much it impacts the model prediction on PP attachment arcs. We test the faithfulness of the attributions using comprehensiveness and sensitivity. See Appendix A  C for details. Results are shown in Table 3. Similar to the results on text classification tasks, we find that perturbation-based methods like LIME and Occlusion perform well under the comprehensiveness score, while VaPGD performs the best under sensitivity. PGDInp and Certify are slightly better than GradInp under both the two metrics. Qualitatively, we find that according to interpretation methods, important tokens for a PP-attachment decision converge tothe preposition itself, the preceding noun or verb, and the disambiguating noun. This is close to human expectations. An example is shown in Appendix E. PGD Occlusion IngGrad GradInp Comp. Metric Check Removing even a small piece of inputs breaks the dependency tree. It will be hard to distinguish either the decision process behind the model has changed or the removal of important tokens actually causes the performance drop. Thus, we expect a better metric to have less influence on the tree structure of a sentence. In Table 4, we show that evaluating interpretations with sensitivity leads to smaller changes in the output dependency tree compared to comprehensiveness, suggesting sensitivity a more compatible metric for the dependency parsing task interpretations. Disambiguating Noun Analysis Disambiguating nouns are expected to be identified as important signals by faithful interpretations. We summarize how many times they are actually recognized as the top-k most important words by interpretation methods, where k is the interval varies in 10-20%, . . . , 90-100% of total tokens in an example. Results in Figure 4 demonstrate that interpretation methods from the same category have high correlations when extracting disambiguating nouns. For example, VaGrad and VaPGD leveraging gradients only, tend to position disambiguating nouns on the top of their importance lists, which is consistent with human judgments. Likewise, the perturbation-based methods, Occlusion and LIME, also put the disambiguation words to very similar positions. Interpretation methods Various post-hoc interpretation methods are proposed to explain the behaviors of black-box models. These methods can be roughly categorized into three classes: gradient-based methods (Simonyan et al., 2014; Li et al., 2016), which leverage local gradient information; reference-based methods (Shrikumar et al., 2017; Sundararajan et al., 2017), which consider the model output difference between the original point and a reference point; and perturbation-based methods (Ribeiro et al., 2016; Zeiler and Fergus, 2014; Lundberg and Lee, 2017), which query model outputs on perturbed data. In our work, we propose 2638 0 20 Gradient-based Robustness-basedw/ Input GradInp Gradient-based Robustness-basedw/o Input VaGrad Perturbation-based Occlusion 0.5 1.0 0 20 PGDInp 0.5 1.0 VaPGD 0.5 1.0 LIME Figure 4: Where do interpretations place the disambiguating nouns. new interpretation methods called robustness-based methods, which adopt techniques in the adversarial robustness domain and bridge the gap between the gradient-based and the reference-based methods. Evaluating interpretation methods One line of studies explores approaches to evaluate interpretations. Several studies propose measurements for faithfulness. A large proportion of them occlude tokens identified as important by interpretations and measure the confidence change of models (DeY-oung et al., 2020; Jain and Wallace, 2019; Zaidan and Eisner, 2008; Serrano and Smith, 2019). Some other works propose to evaluate the faithfulness by checking to what extent they satisfy some desired axioms (Ancona et al., 2018; Sundararajan et al., 2017; Shrikumar et al., 2017). Besides, Alvarez-Melis and Jaakkola (2018); Ghorbani et al. (2019); Kindermans et al. (2019); Yeh et al. (2019) reveal limitations in interpretation faithfulness through testing the robustness of interpretations. Another group of studies measure the plausibility of interpretations, i.e., whether the explanations conform with human judgments (Doshi-Velez and Kim, 2017; Ribeiro et al., 2016), or assist humans or student models to predict model behaviors on new data (Hase and Bansal, 2020; Pruthi et al., 2020). Note that although there exist many hybrid works that evaluate both the faithfulness and the plausibility of interpretations by combining a suite of diagnostic tests (DeYoung et al., 2020; Atanasova et al., 2020; Liu et al., 2020), Jacovi and Goldberg (2020) advocate to explicitly distinguish between the two measurements. In our work, we focus on interpretation faithfulness but consider two new metrics. We apply them to the dependency parsing task. Also, notice that the stability could be regarded as an automatic input consistency tests suggested by Ding and Koehn (2021). In our work, we enhanced the existed definition of interpretation faithfulness by two other notions and proposed corresponding quantitative metrics: sensitivity and stability, for each of them. We studied interpretations under the two notions along with the existed one. We found that interpretations have inconsistent performance regarding different criteria. We proposed a new class of interpretations, motivated by the adversarial robustness techniques, which achieves the best performance under the sensitivity and the stability criteria. We further proposed a novel paradigm to evaluate interpretations on the dependency parsing task, which moves beyond text classification in the literature. Our study shed light on understanding the behavior of model interpretations and suggested the community to put more efforts on defining an appropriate evaluation pipeline for interpretation faithfulness. We thank anonymous reviewers, UCLA PLUS-Lab and UCLA-NLP for their helpful feedback. This work is supported in part by NSF 1927554, 2008173, 2048280, CISCO, and a Sloan fellowship. This paper does not contain direct social influences. However, we believe the model analysis and interpretation techniques discussed in this paper are critical for deploying deep learning based models to real-world applications. Following previous work in this direction such as Jacovi and Goldberg (2020), we advocate to carefully consider the explanations obtained from interpretation methods as they may not always reflect the true reasoning process behind model predictions. Besides the three notions of faithfulness discussed in this paper, there are other important aspects for measuring interpretations that could be applied to evaluate interpretations. Also, We are not claiming that the proposed paradigm are per-fect as faithfulness measurements. For example, we recognize that it requires further and detailed 2639 analysis on either the model itself or the interpretation methods lead to a low performance on the stability metric, although we do try to make sure models behaviors do not change substantially between an input pair. Moreover, experiments in this paper are all based on mainstream English corpora. Although our techniques are not language specific, there could be different conclusions given the varying properties of languages. For example, the discussion for dependency parsing could be easily affected by the language one considers.
abstain abstain objective result objective objective method result abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain objective abstain abstain abstain objective abstain abstain abstain objective objective objective objective abstain objective method other objective objective abstain abstain abstain abstain abstain abstain abstain abstain method other method abstain abstain method method abstain abstain method abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result result abstain method abstain abstain abstain abstain method method method method method abstain abstain abstain abstain result abstain abstain result abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain method method method method method abstain result abstain result abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain other other objective other other other other other other other other objective method other objective method result objective objective method other other abstain method abstain method abstain abstain abstain abstain abstain