How Important is a Neuron
The problem of attributing a deep network’s prediction to its input/base features is well-studied (cf. Simonyan et al. (2013)). We introduce the notion of conductance to extend the notion of attribution to understanding the importance of hidden units. Informally, the conductance of a hidden unit of a deep network is the flow of attribution via this hidden unit. We can use conductance to understand the importance of a hidden unit to the prediction for a specific input, or over a set of inputs. We justify conductance in multiple ways via a qualitative comparison with other methods, via some axiomatic results, and via an empirical evaluation based on a feature selection task. The empirical evaluations are done using the Inception network over ImageNet data, and a convolutinal network over text data. In both cases, we demonstrate the effectiveness of conductance in identifying interesting insights about the internal workings of these networks.
 The idea is nice. It is well aligned with tools that are needed to understand neural networks. However, the experiments feel like they are missing motivation as to why this method is being used. The paper does not provide very significant evidence that this method is useful. The negation example is nice but this doesn*t seem to display the potential power of the method to understand a neural network. More motivation for experimental section is needed. If the authors don*t discuss a motivation then how will a reader know how to apply the tool? It seems there is no conclusion to take away from the experiments in section 5 (convolutions). The authors should rethink the structure of the experimental section from the standpoint of convincing someone to use this method. In section 4.1 the authors have a good discussion on what is wrong with other methods in order to motivate their approach but then they don*t deliver significant evidence in the later part of the section. The paper needs more discussion and experiments to explain how and why to use this approach. While the authors say "attributing a deep network’s prediction to its input is well-studied" they don*t compare directly against these methods. There are many typos and grammar errors While I think the paper could be much more impactful if the experimental section was greatly reworked; I believe the first 5 pages of the paper are a very good contribution and it should be accepted.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
How Important is a Neuron
The problem of attributing a deep network’s prediction to its input/base features is well-studied (cf. Simonyan et al. (2013)). We introduce the notion of conductance to extend the notion of attribution to understanding the importance of hidden units. Informally, the conductance of a hidden unit of a deep network is the flow of attribution via this hidden unit. We can use conductance to understand the importance of a hidden unit to the prediction for a specific input, or over a set of inputs. We justify conductance in multiple ways via a qualitative comparison with other methods, via some axiomatic results, and via an empirical evaluation based on a feature selection task. The empirical evaluations are done using the Inception network over ImageNet data, and a convolutinal network over text data. In both cases, we demonstrate the effectiveness of conductance in identifying interesting insights about the internal workings of these networks.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



 The idea is nice. It is well aligned with tools that are needed to understand neural networks. However, the experiments feel like they are missing motivation as to why this method is being used. The paper does not provide very significant evidence that this method is useful. The negation example is nice but this doesn*t seem to display the potential power of the method to understand a neural network. More motivation for experimental section is needed. If the authors don*t discuss a motivation then how will a reader know how to apply the tool? It seems there is no conclusion to take away from the experiments in section 5 (convolutions). The authors should rethink the structure of the experimental section from the standpoint of convincing someone to use this method. In section 4.1 the authors have a good discussion on what is wrong with other methods in order to motivate their approach but then they don*t deliver significant evidence in the later part of the section. The paper needs more discussion and experiments to explain how and why to use this approach. While the authors say "attributing a deep network’s prediction to its input is well-studied" they don*t compare directly against these methods. There are many typos and grammar errors While I think the paper could be much more impactful if the experimental section was greatly reworked; I believe the first 5 pages of the paper are a very good contribution and it should be accepted.
assistant
 The idea is nice. It is well aligned with tools that are needed to understand neural networks. However, the experiments feel like they are missing motivation as to why this method is being used. The paper does not provide very significant evidence that this method is useful. The negation example is nice but this doesn*t seem to display the potential power of the method to understand a neural network. More motivation for experimental section is needed. If the authors don*t discuss a motivation then how will a reader know how to apply the tool? It seems there is no conclusion to take away from the experiments in section 5 (convolutions). The authors should rethink the structure of the experimental section from the standpoint of convincing someone to use this method. In section 4.1 the authors have a good discussion on what is wrong with other methods in order to motivate their approach but then they don*t deliver significant evidence in the later part of the section. The paper needs more discussion and experiments to explain how and why to use this approach. While the authors say "attributing a deep network’s prediction to its input is well-studied" they don*t compare directly against these methods. There are many typos and grammar errors While I think the paper could be much more impactful if the experimental section was greatly reworked; I believe the first 5 pages of the paper are a very good contribution and it should be accepted.