Regularizing Black-box Models for Improved Interpretability
Most of the work on interpretable machine learning has focused on designingeither inherently interpretable models, which typically trade-off accuracyfor interpretability, or post-hoc explanation systems, which lack guarantees about their explanation quality. We explore an alternative to theseapproaches by directly regularizing a black-box model for interpretabilityat training time. Our approach explicitly connects three key aspects ofinterpretable machine learning: (i) the model’s internal interpretability, (ii)the explanation system used at test time, and (iii) the metrics that measureexplanation quality. Our regularization results in substantial improvementin terms of the explanation fidelity and stability metrics across a range ofdatasets and black-box explanation systems while slightly improving accuracy. Finally, we justify theoretically that the benefits of our regularizationgeneralize to unseen points.
This paper proposes an approach for local post-hoc explanation with introduction of a new regularization that helps regulate the *fidelity* and *stability* of the output explanation (in the style of LIME). The fidelity regularizer is essentially the squared error of the explainer as compared to the given model in the local neighborhood, whereas the stability regularizer measures the total squared differences between the explanation outcomes between the sample in question and other samples in the local neighborhood. In the experimental evaluation section, the authors evaluates the performance fo the proposed regularizers, used as part of both LIME and MAPLE, against three interpretability metrics: point fidelity, neighborhood fidelity and stability. The results verify that indeed the use of the regularizers improve the performance of both LIME and MAPLE over the unregularized versions, with respect to the corresponding metrics. This is in a way *expected*, since the regularizer used in the method and that in the metric are closely related, and is an unsatisfying aspect of the work. Using image data, they also demonstrate that qualitatively the use of stability regularizer seems to significantly improve the saliency of the output explanation. The paper is well written, the proposal is reasonable, but the contribution is modest and experimental evaluation is not entirely convincing.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Regularizing Black-box Models for Improved Interpretability
Most of the work on interpretable machine learning has focused on designingeither inherently interpretable models, which typically trade-off accuracyfor interpretability, or post-hoc explanation systems, which lack guarantees about their explanation quality. We explore an alternative to theseapproaches by directly regularizing a black-box model for interpretabilityat training time. Our approach explicitly connects three key aspects ofinterpretable machine learning: (i) the model’s internal interpretability, (ii)the explanation system used at test time, and (iii) the metrics that measureexplanation quality. Our regularization results in substantial improvementin terms of the explanation fidelity and stability metrics across a range ofdatasets and black-box explanation systems while slightly improving accuracy. Finally, we justify theoretically that the benefits of our regularizationgeneralize to unseen points.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



This paper proposes an approach for local post-hoc explanation with introduction of a new regularization that helps regulate the *fidelity* and *stability* of the output explanation (in the style of LIME). The fidelity regularizer is essentially the squared error of the explainer as compared to the given model in the local neighborhood, whereas the stability regularizer measures the total squared differences between the explanation outcomes between the sample in question and other samples in the local neighborhood. In the experimental evaluation section, the authors evaluates the performance fo the proposed regularizers, used as part of both LIME and MAPLE, against three interpretability metrics: point fidelity, neighborhood fidelity and stability. The results verify that indeed the use of the regularizers improve the performance of both LIME and MAPLE over the unregularized versions, with respect to the corresponding metrics. This is in a way *expected*, since the regularizer used in the method and that in the metric are closely related, and is an unsatisfying aspect of the work. Using image data, they also demonstrate that qualitatively the use of stability regularizer seems to significantly improve the saliency of the output explanation. The paper is well written, the proposal is reasonable, but the contribution is modest and experimental evaluation is not entirely convincing.
assistant
This paper proposes an approach for local post-hoc explanation with introduction of a new regularization that helps regulate the *fidelity* and *stability* of the output explanation (in the style of LIME). The fidelity regularizer is essentially the squared error of the explainer as compared to the given model in the local neighborhood, whereas the stability regularizer measures the total squared differences between the explanation outcomes between the sample in question and other samples in the local neighborhood. In the experimental evaluation section, the authors evaluates the performance fo the proposed regularizers, used as part of both LIME and MAPLE, against three interpretability metrics: point fidelity, neighborhood fidelity and stability. The results verify that indeed the use of the regularizers improve the performance of both LIME and MAPLE over the unregularized versions, with respect to the corresponding metrics. This is in a way *expected*, since the regularizer used in the method and that in the metric are closely related, and is an unsatisfying aspect of the work. Using image data, they also demonstrate that qualitatively the use of stability regularizer seems to significantly improve the saliency of the output explanation. The paper is well written, the proposal is reasonable, but the contribution is modest and experimental evaluation is not entirely convincing.