Towards a Unified Evaluation of Explanation Methods without Ground Truth
This paper proposes a set of criteria to evaluate the objectiveness of explanation methods of neural networks, which is crucial for the development of explainable AI, but it also presents significant challenges. The core challenge is that people usually cannot obtain ground-truth explanations of the neural network. To this end, we design four metrics to evaluate the explanation result without ground-truth explanations. Our metrics can be broadly applied to nine benchmark methods of interpreting neural networks, which provides new insights of explanation methods.
This paper considers the important problem of evaluating explanation methods for neural models without recourse to `ground-truth* explanations. The authors rightly highlight that plausibility (to humans) does not imply faithfulness to the underlying models. They then define a suite of metrics, largely drawing upon known techniques, to evaluate explanations. For instance, the authors propose evaluating the agreement between model attributions and Shapley values. The latter are approximated in two ways: Only pixels with high (or low) attribution values are considered; and sampling is used to make estimation tractable. Overall, this is a nice paper that investigates an important problem --- I particularly appreciated the comparison of multiple attribution methods across these suite of metrics. However, the results are something of a mixed bag, and I was left without a clear takeaway. I am not sure the ICLR audience will get much out of this submission. It would be nice if the authors could provide additional guidance concerning the interpretation of the various metrics they propose, and for which scenarios they view these as appropriate. Strengths --- + The topic of explainability is of increasing importance, and the authors focus on an important question regarding this: How can we evaluate attribution methods? + The approach of defining a suite of metrics, rather than only one, is welcome given that different applications will call for different types of attribution. + The comparison of existing methods for attribution across a range of metrics is a nice contribution. That said, the results are sort of a mixed bag and it is not obvious what one should take away from this. Weaknesses --- - Much of this work (and the proposed metrics) seems to derive from prior efforts. I*m not sure how much novelty there is here, although there is a value in bringing these metrics together. - To the latter point, however, the authors do not really speak to the pros, cons, and applicability or relevance of the different metrics considered. A user of this suite of metrics would therefore be left with four separate measures, and it is not immediately clear how to interpret these, or what the relative performance of explainability methods with respect to these metrics tells us. - I am not sure I followed the unexplainable feature component metric (3.3). I found the notation and terminology here difficult to parse; can the authors please clarify what they mean by *unexplainable feature components*, and also clarify what represent (I think induced feature representations of the image I but again am not certain). From what I gather this is somehow measuring the peakiness (or lack thereof) of attribution values, but it is not obvious to me that this is an inherently desirable property. - The robustness metric seems to make the assumption that the model does not capture interactions, i.e., a pixel j may be important only when seen together with some other pixel i. I*m not sure this assumption is warranted; perhaps the authors can elaborate to provide further intuition.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Towards a Unified Evaluation of Explanation Methods without Ground Truth
This paper proposes a set of criteria to evaluate the objectiveness of explanation methods of neural networks, which is crucial for the development of explainable AI, but it also presents significant challenges. The core challenge is that people usually cannot obtain ground-truth explanations of the neural network. To this end, we design four metrics to evaluate the explanation result without ground-truth explanations. Our metrics can be broadly applied to nine benchmark methods of interpreting neural networks, which provides new insights of explanation methods.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



This paper considers the important problem of evaluating explanation methods for neural models without recourse to `ground-truth* explanations. The authors rightly highlight that plausibility (to humans) does not imply faithfulness to the underlying models. They then define a suite of metrics, largely drawing upon known techniques, to evaluate explanations. For instance, the authors propose evaluating the agreement between model attributions and Shapley values. The latter are approximated in two ways: Only pixels with high (or low) attribution values are considered; and sampling is used to make estimation tractable. Overall, this is a nice paper that investigates an important problem --- I particularly appreciated the comparison of multiple attribution methods across these suite of metrics. However, the results are something of a mixed bag, and I was left without a clear takeaway. I am not sure the ICLR audience will get much out of this submission. It would be nice if the authors could provide additional guidance concerning the interpretation of the various metrics they propose, and for which scenarios they view these as appropriate. Strengths --- + The topic of explainability is of increasing importance, and the authors focus on an important question regarding this: How can we evaluate attribution methods? + The approach of defining a suite of metrics, rather than only one, is welcome given that different applications will call for different types of attribution. + The comparison of existing methods for attribution across a range of metrics is a nice contribution. That said, the results are sort of a mixed bag and it is not obvious what one should take away from this. Weaknesses --- - Much of this work (and the proposed metrics) seems to derive from prior efforts. I*m not sure how much novelty there is here, although there is a value in bringing these metrics together. - To the latter point, however, the authors do not really speak to the pros, cons, and applicability or relevance of the different metrics considered. A user of this suite of metrics would therefore be left with four separate measures, and it is not immediately clear how to interpret these, or what the relative performance of explainability methods with respect to these metrics tells us. - I am not sure I followed the unexplainable feature component metric (3.3). I found the notation and terminology here difficult to parse; can the authors please clarify what they mean by *unexplainable feature components*, and also clarify what represent (I think induced feature representations of the image I but again am not certain). From what I gather this is somehow measuring the peakiness (or lack thereof) of attribution values, but it is not obvious to me that this is an inherently desirable property. - The robustness metric seems to make the assumption that the model does not capture interactions, i.e., a pixel j may be important only when seen together with some other pixel i. I*m not sure this assumption is warranted; perhaps the authors can elaborate to provide further intuition.
assistant
This paper considers the important problem of evaluating explanation methods for neural models without recourse to `ground-truth* explanations. The authors rightly highlight that plausibility (to humans) does not imply faithfulness to the underlying models. They then define a suite of metrics, largely drawing upon known techniques, to evaluate explanations. For instance, the authors propose evaluating the agreement between model attributions and Shapley values. The latter are approximated in two ways: Only pixels with high (or low) attribution values are considered; and sampling is used to make estimation tractable. Overall, this is a nice paper that investigates an important problem --- I particularly appreciated the comparison of multiple attribution methods across these suite of metrics. However, the results are something of a mixed bag, and I was left without a clear takeaway. I am not sure the ICLR audience will get much out of this submission. It would be nice if the authors could provide additional guidance concerning the interpretation of the various metrics they propose, and for which scenarios they view these as appropriate. Strengths --- + The topic of explainability is of increasing importance, and the authors focus on an important question regarding this: How can we evaluate attribution methods? + The approach of defining a suite of metrics, rather than only one, is welcome given that different applications will call for different types of attribution. + The comparison of existing methods for attribution across a range of metrics is a nice contribution. That said, the results are sort of a mixed bag and it is not obvious what one should take away from this. Weaknesses --- - Much of this work (and the proposed metrics) seems to derive from prior efforts. I*m not sure how much novelty there is here, although there is a value in bringing these metrics together. - To the latter point, however, the authors do not really speak to the pros, cons, and applicability or relevance of the different metrics considered. A user of this suite of metrics would therefore be left with four separate measures, and it is not immediately clear how to interpret these, or what the relative performance of explainability methods with respect to these metrics tells us. - I am not sure I followed the unexplainable feature component metric (3.3). I found the notation and terminology here difficult to parse; can the authors please clarify what they mean by *unexplainable feature components*, and also clarify what represent (I think induced feature representations of the image I but again am not certain). From what I gather this is somehow measuring the peakiness (or lack thereof) of attribution values, but it is not obvious to me that this is an inherently desirable property. - The robustness metric seems to make the assumption that the model does not capture interactions, i.e., a pixel j may be important only when seen together with some other pixel i. I*m not sure this assumption is warranted; perhaps the authors can elaborate to provide further intuition.