Are there any *object detectors* in the hidden layers of CNNs trained to identify objects or scenes?
Various methods of measuring unit selectivity have been developed with the aim of better understanding how neural networks work. But the different measures provide divergent estimates of selectivity, and this has led to different conclusions regarding the conditions in which selective object representations are learned and the functional relevance of these representations. In an attempt to better characterize object selectivity, we undertake a comparison of various selectivity measures on a large set of units in AlexNet, including localist selectivity, precision, class-conditional mean activity selectivity (CCMAS), network dissection, the human interpretation of activation maximization (AM) images, and standard signal-detection measures. We find that the different measures provide different estimates of object selectivity, with precision and CCMAS measures providing misleadingly high estimates. Indeed, the most selective units had a poor hit-rate or a high false-alarm rate (or both) in object classification, making them poor object detectors. We fail to find any units that are even remotely as selective as the *grandmother cell* units reported in recurrent neural networks. In order to generalize these results, we compared selectivity measures on a few units in VGG-16 and GoogLeNet trained on the ImageNet or Places-365 datasets that have been described as *object detectors*. Again, we find poor hit-rates and high false-alarm rates for object classification.
This work investigates the collection of methods that have been proposed to find units in neural networks that are selective for certain object classes. Previous works have used different measures of selectivity (with sometimes contradictory results), and the authors investigate the degree to which these units qualify as “object detectors”. This research area is important for understanding deep networks because claims have been made as to the relative importance (or lack thereof) of these individual units as identified by different measures vis-a-vis distributed representations -- the identification of such units would be interesting for understanding the predictions of classification networks. The authors find that (1) different proposed measures of selectivity are not consistent and (2) units identified as selective cannot be considered object detectors due to the high false alarm / low hit rates, analyzing a large number of selectivity measures. I would have liked to see experiments on more recent architectures (the focus of the paper is on a dated architecture (AlexNet)); there is analysis on units in GoogLeNet and VGG-16 but it would also be interesting to see results for more modern architectures (e.g. DenseNet and ResNet). Overall, I think that the authors have presented a strong meta-analysis and compelling argument for further study in rigorously identifying the presence (or lack thereof) of selective units in neural networks and the degree to which they may be considered *object detectors.*
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Are there any *object detectors* in the hidden layers of CNNs trained to identify objects or scenes?
Various methods of measuring unit selectivity have been developed with the aim of better understanding how neural networks work. But the different measures provide divergent estimates of selectivity, and this has led to different conclusions regarding the conditions in which selective object representations are learned and the functional relevance of these representations. In an attempt to better characterize object selectivity, we undertake a comparison of various selectivity measures on a large set of units in AlexNet, including localist selectivity, precision, class-conditional mean activity selectivity (CCMAS), network dissection, the human interpretation of activation maximization (AM) images, and standard signal-detection measures. We find that the different measures provide different estimates of object selectivity, with precision and CCMAS measures providing misleadingly high estimates. Indeed, the most selective units had a poor hit-rate or a high false-alarm rate (or both) in object classification, making them poor object detectors. We fail to find any units that are even remotely as selective as the *grandmother cell* units reported in recurrent neural networks. In order to generalize these results, we compared selectivity measures on a few units in VGG-16 and GoogLeNet trained on the ImageNet or Places-365 datasets that have been described as *object detectors*. Again, we find poor hit-rates and high false-alarm rates for object classification.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



This work investigates the collection of methods that have been proposed to find units in neural networks that are selective for certain object classes. Previous works have used different measures of selectivity (with sometimes contradictory results), and the authors investigate the degree to which these units qualify as “object detectors”. This research area is important for understanding deep networks because claims have been made as to the relative importance (or lack thereof) of these individual units as identified by different measures vis-a-vis distributed representations -- the identification of such units would be interesting for understanding the predictions of classification networks. The authors find that (1) different proposed measures of selectivity are not consistent and (2) units identified as selective cannot be considered object detectors due to the high false alarm / low hit rates, analyzing a large number of selectivity measures. I would have liked to see experiments on more recent architectures (the focus of the paper is on a dated architecture (AlexNet)); there is analysis on units in GoogLeNet and VGG-16 but it would also be interesting to see results for more modern architectures (e.g. DenseNet and ResNet). Overall, I think that the authors have presented a strong meta-analysis and compelling argument for further study in rigorously identifying the presence (or lack thereof) of selective units in neural networks and the degree to which they may be considered *object detectors.*
assistant
This work investigates the collection of methods that have been proposed to find units in neural networks that are selective for certain object classes. Previous works have used different measures of selectivity (with sometimes contradictory results), and the authors investigate the degree to which these units qualify as “object detectors”. This research area is important for understanding deep networks because claims have been made as to the relative importance (or lack thereof) of these individual units as identified by different measures vis-a-vis distributed representations -- the identification of such units would be interesting for understanding the predictions of classification networks. The authors find that (1) different proposed measures of selectivity are not consistent and (2) units identified as selective cannot be considered object detectors due to the high false alarm / low hit rates, analyzing a large number of selectivity measures. I would have liked to see experiments on more recent architectures (the focus of the paper is on a dated architecture (AlexNet)); there is analysis on units in GoogLeNet and VGG-16 but it would also be interesting to see results for more modern architectures (e.g. DenseNet and ResNet). Overall, I think that the authors have presented a strong meta-analysis and compelling argument for further study in rigorously identifying the presence (or lack thereof) of selective units in neural networks and the degree to which they may be considered *object detectors. *