On the importance of single directions for generalization
Despite their ability to memorize large datasets, deep neural networks often achieve good generalization performance. However, the differences between the learned solutions of networks which generalize and those which do not remain unclear. Additionally, the tuning properties of single directions (defined as the activation of a single unit or some linear combination of units in response to some input) have been highlighted, but their importance has not been evaluated. Here, we connect these lines of inquiry to demonstrate that a network’s reliance on single directions is a good predictor of its generalization performance, across networks trained on datasets with different fractions of corrupted labels, across ensembles of networks trained on datasets with unmodified labels, across different hyper- parameters, and over the course of training. While dropout only regularizes this quantity up to a point, batch normalization implicitly discourages single direction reliance, in part by decreasing the class selectivity of individual units. Finally, we find that class selectivity is a poor predictor of task importance, suggesting not only that networks which generalize well minimize their dependence on individual units by reducing their selectivity, but also that individually selective units may not be necessary for strong network performance.
 Summary: - nets that rely on single directions are probably overfitting - batch norm helps not having large single directions - high class selectivity of single units is a bad measure to find "important" neurons that help a NN generalize. The experiments that this paper does are quite interesting, somewhat confirming intuitions that the community had, and bringing new insights into generalization. The presentation is good overall, but many minor improvements could help with readability. Remarks: - The first thing you should say in this paper is what you mean by "single direction", at least an intuition, to be refined later. The second sentence of section 2 could easily be plugged in your abstract. - You should already mention in section 2.1 that you are using ReLUs, otherwise clamping to 0 might take a different sense. - considering the lack of page limit at ICLR, making *all* your figures bigger would be beneficial to readability. - Figure 2*s y values drop rapidly as a function of x, maybe make x have a log scale or something that zooms in near 0 would help readability. - Figure 3b*s discrete regimes is very weird, did you actually look at how much these clusters converged to the same solution in parameter space? - Figure 4a is nice, but an additional figure zooming in on the first 2 epochs would be really great, because that AUC curve goes up really fast in the beginning. - Arpit et al. find that there is more cross-class information being shared for true labels than random labels. Considering you find that low class selectivity is an indicator of good generalization, would it make sense to look at "cross-class selectivity"? If a neuron learns a feature shared by 2 or more classes, then it has this interesting property of offering a discrimination potential for multiple classes at the same time, rather than just 1, making it more "useful" potentially, maybe less adversary prone? - You say in the figure captions that you use random orderings of the features to perform ablation, but nowhere in the main text (which would be nice).
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
On the importance of single directions for generalization
Despite their ability to memorize large datasets, deep neural networks often achieve good generalization performance. However, the differences between the learned solutions of networks which generalize and those which do not remain unclear. Additionally, the tuning properties of single directions (defined as the activation of a single unit or some linear combination of units in response to some input) have been highlighted, but their importance has not been evaluated. Here, we connect these lines of inquiry to demonstrate that a network’s reliance on single directions is a good predictor of its generalization performance, across networks trained on datasets with different fractions of corrupted labels, across ensembles of networks trained on datasets with unmodified labels, across different hyper- parameters, and over the course of training. While dropout only regularizes this quantity up to a point, batch normalization implicitly discourages single direction reliance, in part by decreasing the class selectivity of individual units. Finally, we find that class selectivity is a poor predictor of task importance, suggesting not only that networks which generalize well minimize their dependence on individual units by reducing their selectivity, but also that individually selective units may not be necessary for strong network performance.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



 Summary: - nets that rely on single directions are probably overfitting - batch norm helps not having large single directions - high class selectivity of single units is a bad measure to find "important" neurons that help a NN generalize. The experiments that this paper does are quite interesting, somewhat confirming intuitions that the community had, and bringing new insights into generalization. The presentation is good overall, but many minor improvements could help with readability. Remarks: - The first thing you should say in this paper is what you mean by "single direction", at least an intuition, to be refined later. The second sentence of section 2 could easily be plugged in your abstract. - You should already mention in section 2.1 that you are using ReLUs, otherwise clamping to 0 might take a different sense. - considering the lack of page limit at ICLR, making *all* your figures bigger would be beneficial to readability. - Figure 2*s y values drop rapidly as a function of x, maybe make x have a log scale or something that zooms in near 0 would help readability. - Figure 3b*s discrete regimes is very weird, did you actually look at how much these clusters converged to the same solution in parameter space? - Figure 4a is nice, but an additional figure zooming in on the first 2 epochs would be really great, because that AUC curve goes up really fast in the beginning. - Arpit et al. find that there is more cross-class information being shared for true labels than random labels. Considering you find that low class selectivity is an indicator of good generalization, would it make sense to look at "cross-class selectivity"? If a neuron learns a feature shared by 2 or more classes, then it has this interesting property of offering a discrimination potential for multiple classes at the same time, rather than just 1, making it more "useful" potentially, maybe less adversary prone? - You say in the figure captions that you use random orderings of the features to perform ablation, but nowhere in the main text (which would be nice).
assistant
 Summary: - nets that rely on single directions are probably overfitting - batch norm helps not having large single directions - high class selectivity of single units is a bad measure to find "important" neurons that help a NN generalize. The experiments that this paper does are quite interesting, somewhat confirming intuitions that the community had, and bringing new insights into generalization. The presentation is good overall, but many minor improvements could help with readability. Remarks: - The first thing you should say in this paper is what you mean by "single direction", at least an intuition, to be refined later. The second sentence of section 2 could easily be plugged in your abstract. - You should already mention in section 2.1 that you are using ReLUs, otherwise clamping to 0 might take a different sense. - considering the lack of page limit at ICLR, making *all* your figures bigger would be beneficial to readability. - Figure 2*s y values drop rapidly as a function of x, maybe make x have a log scale or something that zooms in near 0 would help readability. - Figure 3b*s discrete regimes is very weird, did you actually look at how much these clusters converged to the same solution in parameter space? - Figure 4a is nice, but an additional figure zooming in on the first 2 epochs would be really great, because that AUC curve goes up really fast in the beginning. - Arpit et al. find that there is more cross-class information being shared for true labels than random labels. Considering you find that low class selectivity is an indicator of good generalization, would it make sense to look at "cross-class selectivity"? If a neuron learns a feature shared by 2 or more classes, then it has this interesting property of offering a discrimination potential for multiple classes at the same time, rather than just 1, making it more "useful" potentially, maybe less adversary prone? - You say in the figure captions that you use random orderings of the features to perform ablation, but nowhere in the main text (which would be nice).