Explaining Neural Networks Semantically and Quantitatively
This paper presents a method to explain the knowledge encoded in a convolutional neural network (CNN) quantitatively and semantically. How to analyze the specific rationale of each prediction made by the CNN presents one of key issues of understanding neural networks, but it is also of significant practical values in certain applications. In this study, we propose to distill knowledge from the CNN into an explainable additive model, so that we can use the explainable model to provide a quantitative explanation for the CNN prediction. We analyze the typical bias-interpreting problem of the explainable model and develop prior losses to guide the learning of the explainable additive model. Experimental results have demonstrated the effectiveness of our method.
 This work proposes to explain classification decisions via quantitative estimates of the contributions of individual visual concepts to the classification score. It learns an explainer model on top of a performer model (as distillation); the explainer returns the contributions of different concepts. To prevent the model from only selecting a few concepts for the explanation, the authors use a prior-enforcing loss. The approach is clear but evidence that the proposed work will have an impact in practice is not convincing. The quantitative results don*t show the proposed method is useful. Table 1 shows entropy over concept contributions, but the authors don*t convince the reader that this is the right metric to use. They argue using too few concepts (which of course leads to low entropy) is undesirable, but there is no evidence/proof why this is undesirable. Table 2 uses prediction accuracy, which does not seem to be in line with explainability. The qualitative figures are somewhat interesting, but they are not sufficient. It is easy to look at a qualitative figure and say "sure, this makes sense" but this does not mean the proposed method is useful, especially when there is no result from a baseline, whose output might also make perfect sense. Finally, the authors claim pixel-level visualizations aren*t useful, but there*s no proof of why that is. A user study of what*s useful, or a quantitative evaluation of the usefulness of different visualizations for downstream tasks, would be good. In terms of writing, the first few pages tend to be repetitive yet vague about what exactly will be done (generally ok for introduction, but a bit too vague). It is not clear what "subjectively visualizing" means. Finally, I found the end of second para in Sec. 3 unclear.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Explaining Neural Networks Semantically and Quantitatively
This paper presents a method to explain the knowledge encoded in a convolutional neural network (CNN) quantitatively and semantically. How to analyze the specific rationale of each prediction made by the CNN presents one of key issues of understanding neural networks, but it is also of significant practical values in certain applications. In this study, we propose to distill knowledge from the CNN into an explainable additive model, so that we can use the explainable model to provide a quantitative explanation for the CNN prediction. We analyze the typical bias-interpreting problem of the explainable model and develop prior losses to guide the learning of the explainable additive model. Experimental results have demonstrated the effectiveness of our method.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



 This work proposes to explain classification decisions via quantitative estimates of the contributions of individual visual concepts to the classification score. It learns an explainer model on top of a performer model (as distillation); the explainer returns the contributions of different concepts. To prevent the model from only selecting a few concepts for the explanation, the authors use a prior-enforcing loss. The approach is clear but evidence that the proposed work will have an impact in practice is not convincing. The quantitative results don*t show the proposed method is useful. Table 1 shows entropy over concept contributions, but the authors don*t convince the reader that this is the right metric to use. They argue using too few concepts (which of course leads to low entropy) is undesirable, but there is no evidence/proof why this is undesirable. Table 2 uses prediction accuracy, which does not seem to be in line with explainability. The qualitative figures are somewhat interesting, but they are not sufficient. It is easy to look at a qualitative figure and say "sure, this makes sense" but this does not mean the proposed method is useful, especially when there is no result from a baseline, whose output might also make perfect sense. Finally, the authors claim pixel-level visualizations aren*t useful, but there*s no proof of why that is. A user study of what*s useful, or a quantitative evaluation of the usefulness of different visualizations for downstream tasks, would be good. In terms of writing, the first few pages tend to be repetitive yet vague about what exactly will be done (generally ok for introduction, but a bit too vague). It is not clear what "subjectively visualizing" means. Finally, I found the end of second para in Sec. 3 unclear.
assistant
 This work proposes to explain classification decisions via quantitative estimates of the contributions of individual visual concepts to the classification score. It learns an explainer model on top of a performer model (as distillation); the explainer returns the contributions of different concepts. To prevent the model from only selecting a few concepts for the explanation, the authors use a prior-enforcing loss. The approach is clear but evidence that the proposed work will have an impact in practice is not convincing. The quantitative results don*t show the proposed method is useful. Table 1 shows entropy over concept contributions, but the authors don*t convince the reader that this is the right metric to use. They argue using too few concepts (which of course leads to low entropy) is undesirable, but there is no evidence/proof why this is undesirable. Table 2 uses prediction accuracy, which does not seem to be in line with explainability. The qualitative figures are somewhat interesting, but they are not sufficient. It is easy to look at a qualitative figure and say "sure, this makes sense" but this does not mean the proposed method is useful, especially when there is no result from a baseline, whose output might also make perfect sense. Finally, the authors claim pixel-level visualizations aren*t useful, but there*s no proof of why that is. A user study of what*s useful, or a quantitative evaluation of the usefulness of different visualizations for downstream tasks, would be good. In terms of writing, the first few pages tend to be repetitive yet vague about what exactly will be done (generally ok for introduction, but a bit too vague). It is not clear what "subjectively visualizing" means. Finally, I found the end of second para in Sec. 3 unclear.