WHITE NOISE ANALYSIS OF NEURAL NETWORKS
A white noise analysis of modern deep neural networks is presented to characterize their biases at the whole network level or the single neuron level. Our analysis is based on two popular and related methods in psychophysics and neurophysiology namely classification images and spike triggered analysis. These methods have been widely used to understand the underlying mechanisms of sensory systems in humans and monkeys. We leverage them to investigate the inherent biases of deep neural networks and to obtain a first-order approximation of their functionality. We emphasize on convolutional neural networks (CNNs) since they are currently the state of the art methods in computer vision and are a decent model of human visual processing. In addition, we study multi-layer perceptrons, logistic regression, and recurrent neural networks. Experiments over four classic datasets, MNIST, Fashion-MNIST, CIFAR-10, and ImageNet, show that the computed bias maps resemble the target classes and when used for classification lead to an over two-fold performance than the chance level. Further, we show that classification images can be used to attack a black-box classifier and to detect adversarial patch attacks. Finally, we utilize spike triggered averaging to derive the filters of CNNs and explore how the behavior of network changes when neurons in different layers are modulated. Our effort illustrates a successful example of borrowing from neuroscience to study ANNs and highlights the importance of cross-fertilization and synergy across machine learning, deep learning, and computational neuroscience.
This is an interesting paper that uses two techniques from neuroscience (eletrophysiology) to interpret CNNs. The *classification images* technique allows the authors to build an input specific bias map for CNNs, from different images that are randomly generated using different methods. STA is also used to visualize filters. The authors find interesting similarities between CNNs and animal/human brain such as the shift of the psychometric curve. The paper is well written and the experiments are thorough. This paper has comprehensive use of neuroscience method, nicely bringing the two fields together. Even though the authors mention other approaches, perhaps one weakness is the limited discussion of other visualization tools: what is the difference between other more frequent approaches such as activation maximization and the approach the authors present. What could we understand with these tools that cannot be understood through activation maximization (i.e. using backprop to find optimal stimuli)? One problem with classification images is that the pattern is limited to low level patterns unless the random images have structure in them. The authors solve this problem using more complex random sample (e.g. with Gabor features). But a more in depth discussion is necessarily about the additional understanding that is procured from such an approach in comparison to activation maximization. It*s an important discussion because even neuroscientists are now interested (and finally able) to properly perform activation maximization to test their hypotheses more strongly (see Bashivan et al 2019 science for a great example). Even though I think the lack of this discussion is currently a limitation, I think this paper is a good opportunity to have this discussion and bring together the specifics of these two different approaches.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
WHITE NOISE ANALYSIS OF NEURAL NETWORKS
A white noise analysis of modern deep neural networks is presented to characterize their biases at the whole network level or the single neuron level. Our analysis is based on two popular and related methods in psychophysics and neurophysiology namely classification images and spike triggered analysis. These methods have been widely used to understand the underlying mechanisms of sensory systems in humans and monkeys. We leverage them to investigate the inherent biases of deep neural networks and to obtain a first-order approximation of their functionality. We emphasize on convolutional neural networks (CNNs) since they are currently the state of the art methods in computer vision and are a decent model of human visual processing. In addition, we study multi-layer perceptrons, logistic regression, and recurrent neural networks. Experiments over four classic datasets, MNIST, Fashion-MNIST, CIFAR-10, and ImageNet, show that the computed bias maps resemble the target classes and when used for classification lead to an over two-fold performance than the chance level. Further, we show that classification images can be used to attack a black-box classifier and to detect adversarial patch attacks. Finally, we utilize spike triggered averaging to derive the filters of CNNs and explore how the behavior of network changes when neurons in different layers are modulated. Our effort illustrates a successful example of borrowing from neuroscience to study ANNs and highlights the importance of cross-fertilization and synergy across machine learning, deep learning, and computational neuroscience.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



This is an interesting paper that uses two techniques from neuroscience (eletrophysiology) to interpret CNNs. The *classification images* technique allows the authors to build an input specific bias map for CNNs, from different images that are randomly generated using different methods. STA is also used to visualize filters. The authors find interesting similarities between CNNs and animal/human brain such as the shift of the psychometric curve. The paper is well written and the experiments are thorough. This paper has comprehensive use of neuroscience method, nicely bringing the two fields together. Even though the authors mention other approaches, perhaps one weakness is the limited discussion of other visualization tools: what is the difference between other more frequent approaches such as activation maximization and the approach the authors present. What could we understand with these tools that cannot be understood through activation maximization (i.e. using backprop to find optimal stimuli)? One problem with classification images is that the pattern is limited to low level patterns unless the random images have structure in them. The authors solve this problem using more complex random sample (e.g. with Gabor features). But a more in depth discussion is necessarily about the additional understanding that is procured from such an approach in comparison to activation maximization. It*s an important discussion because even neuroscientists are now interested (and finally able) to properly perform activation maximization to test their hypotheses more strongly (see Bashivan et al 2019 science for a great example). Even though I think the lack of this discussion is currently a limitation, I think this paper is a good opportunity to have this discussion and bring together the specifics of these two different approaches.
assistant
This is an interesting paper that uses two techniques from neuroscience (eletrophysiology) to interpret CNNs. The *classification images* technique allows the authors to build an input specific bias map for CNNs, from different images that are randomly generated using different methods. STA is also used to visualize filters. The authors find interesting similarities between CNNs and animal/human brain such as the shift of the psychometric curve. The paper is well written and the experiments are thorough. This paper has comprehensive use of neuroscience method, nicely bringing the two fields together. Even though the authors mention other approaches, perhaps one weakness is the limited discussion of other visualization tools: what is the difference between other more frequent approaches such as activation maximization and the approach the authors present. What could we understand with these tools that cannot be understood through activation maximization (i.e. using backprop to find optimal stimuli)? One problem with classification images is that the pattern is limited to low level patterns unless the random images have structure in them. The authors solve this problem using more complex random sample (e.g. with Gabor features). But a more in depth discussion is necessarily about the additional understanding that is procured from such an approach in comparison to activation maximization. It*s an important discussion because even neuroscientists are now interested (and finally able) to properly perform activation maximization to test their hypotheses more strongly (see Bashivan et al 2019 science for a great example). Even though I think the lack of this discussion is currently a limitation, I think this paper is a good opportunity to have this discussion and bring together the specifics of these two different approaches.