Aggregating explanation methods for neural networks stabilizes explanations
Despite a growing literature on explaining neural networks, no consensus has been reached on how to explain a neural network decision or how to evaluate an explanation. Our contributions in this paper are twofold. First, we investigate schemes to combine explanation methods and reduce model uncertainty to obtain a single aggregated explanation. The aggregation is more robust and aligns better with the neural network than any single explanation method.. Second, we propose a new approach to evaluating explanation methods that circumvents the need for manual evaluation and is not reliant on the alignment of neural networks and humans decision processes.
The paper presents a study on explanation methods, proposing an interesting way to aggregate their results and providing empirical evidence that aggregation can improve the quality of the explanations. The paper considered only methods using CNN for classifying images, leaving other applications for future investigation. The results show aggregation of different explanation methods leads to better explanations, and is therefore exploitable, for high dimensional images while degrades with low-dimensional ones. In the latter case, it happens that the aggregated explanation explains a bit worse than the best non-aggregated explanation (from the graph it seems a very small difference though). This is odd because I would have assumed to see an improvement (at least a small one) using information from more than one system. The paper also presents a score for evaluating explanation methods, which shows good results. The paper is interesting and well written, the experimental campaign extensive enough and the methods are presented clearly. There are some minor problems to be solved: - In the abstract, there are two periods before the last sentence. - When defining the size of matrices I suggest using 	imes instead of x to improve readability - The caption in Figure 1 says *The decrease of the class score over the number of removed segments is reported.*, but I cannot find where this decrease is reported. - On page 5, in the parentheses it is reported that a given explanation method is no better than random choice. I would say that it is better (or not worse) than random choice, otherwise the methods would not provide any useful information. - On page 6, forty images are considered in the text while fifty images are considered in the caption of Table 1. Please align the number with the correct one.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Aggregating explanation methods for neural networks stabilizes explanations
Despite a growing literature on explaining neural networks, no consensus has been reached on how to explain a neural network decision or how to evaluate an explanation. Our contributions in this paper are twofold. First, we investigate schemes to combine explanation methods and reduce model uncertainty to obtain a single aggregated explanation. The aggregation is more robust and aligns better with the neural network than any single explanation method.. Second, we propose a new approach to evaluating explanation methods that circumvents the need for manual evaluation and is not reliant on the alignment of neural networks and humans decision processes.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



The paper presents a study on explanation methods, proposing an interesting way to aggregate their results and providing empirical evidence that aggregation can improve the quality of the explanations. The paper considered only methods using CNN for classifying images, leaving other applications for future investigation. The results show aggregation of different explanation methods leads to better explanations, and is therefore exploitable, for high dimensional images while degrades with low-dimensional ones. In the latter case, it happens that the aggregated explanation explains a bit worse than the best non-aggregated explanation (from the graph it seems a very small difference though). This is odd because I would have assumed to see an improvement (at least a small one) using information from more than one system. The paper also presents a score for evaluating explanation methods, which shows good results. The paper is interesting and well written, the experimental campaign extensive enough and the methods are presented clearly. There are some minor problems to be solved: - In the abstract, there are two periods before the last sentence. - When defining the size of matrices I suggest using 	imes instead of x to improve readability - The caption in Figure 1 says *The decrease of the class score over the number of removed segments is reported.*, but I cannot find where this decrease is reported. - On page 5, in the parentheses it is reported that a given explanation method is no better than random choice. I would say that it is better (or not worse) than random choice, otherwise the methods would not provide any useful information. - On page 6, forty images are considered in the text while fifty images are considered in the caption of Table 1. Please align the number with the correct one.
assistant
The paper presents a study on explanation methods, proposing an interesting way to aggregate their results and providing empirical evidence that aggregation can improve the quality of the explanations. The paper considered only methods using CNN for classifying images, leaving other applications for future investigation. The results show aggregation of different explanation methods leads to better explanations, and is therefore exploitable, for high dimensional images while degrades with low-dimensional ones. In the latter case, it happens that the aggregated explanation explains a bit worse than the best non-aggregated explanation (from the graph it seems a very small difference though). This is odd because I would have assumed to see an improvement (at least a small one) using information from more than one system. The paper also presents a score for evaluating explanation methods, which shows good results. The paper is interesting and well written, the experimental campaign extensive enough and the methods are presented clearly. There are some minor problems to be solved: - In the abstract, there are two periods before the last sentence. - When defining the size of matrices I suggest using 	imes instead of x to improve readability - The caption in Figure 1 says *The decrease of the class score over the number of removed segments is reported. *, but I cannot find where this decrease is reported. - On page 5, in the parentheses it is reported that a given explanation method is no better than random choice. I would say that it is better (or not worse) than random choice, otherwise the methods would not provide any useful information. - On page 6, forty images are considered in the text while fifty images are considered in the caption of Table 1. Please align the number with the correct one.