Robustness and/or Redundancy Emerge in Overparametrized Deep Neural Networks
Deep neural networks (DNNs) perform well on a variety of tasks despite the fact that most used in practice are vastly overparametrized and even capable of perfectly fitting randomly labeled data. Recent evidence suggests that developing *compressible* representations is key for adjusting the complexity of overparametrized networks to the task at hand and avoiding overfitting (Arora et al., 2018; Zhou et al., 2018). In this paper, we provide new empirical evidence that supports this hypothesis, identifying two independent mechanisms that emerge when the network’s width is increased: robustness (having units that can be removed without affecting accuracy) and redundancy (having units with similar activity). In a series of experiments with AlexNet, ResNet and Inception networks in the CIFAR-10 and ImageNet datasets, and also using shallow networks with synthetic data, we show that DNNs consistently increase either their robustness, their redundancy, or both at greater widths for a comprehensive set of hyperparameters. These results suggest that networks in the deep learning regime adjust their effective capacity by developing either robustness or redundancy.
The authors conduct an empirical study of the representations that deep neural networks learn as a function of model size, as measured by two different metrics: how quickly performance degrades when random neurons are removed (*robustness*), or how similar neurons are (*redundancy*). They show that deep neural networks tend to become more robust and more redundant as model size increases, though this is not necessarily true. In general, I believe that this sort of empirical approach is useful, and I appreciate the authors* efforts to study the phenomena of robustness and redundancy empirically. However, in my opinion, I do not believe that this paper is ready for publication in its current state. My central concern is the current imprecision of the paper, which undermines its scientific contribution. Many terms are ill-defined or fuzzy. For example, in section 2, 6 different capacity-constrained features are enumerated but not well-defined, including *semantically redundant units* that are *units which are not linearly correlated to others and which provide a different representation from the previous one, similarly related to the learning task.* What does it mean to *provide a different representation from the previous one* (and what previous one?), or to be *similarly related to the learning task*? Critically, there is a significant amount of speculation throughout the paper that is not clearly delineated from experimental observation. This speculation involves the above ill-defined terms and/or are not backed up. As non-exhaustive examples: 1) In page 3, robustness (as measured by neuron dropouts) is asserted to aid in distinguishing *redundant functional units* and *semantically redundant units* from other units because of differences in how robustness of those units change with model size; but this is not backed up and as far as I can tell, not experimentally investigated. Similar statements about redundancy and similarity are made in page 3. 2) In page 3, *L is more robust and equally or less redundant than S*: what is a holistic class representation? Why is that less robust than a *bag-like representation*? 3) Section 4 (results) has a lot of speculation as well. For example, *we take this discrepancy between robustness and redundancy trends as strong evidence that these models... are autoregularizing largely by forming qualitatively different representations... with units that are merely semantically redundant.* These terms are not clearly defined, and there is scant evidence to back up these conjectures. As it stands, it is difficult to accept the paper because of the many speculative claims it contains. The experimental findings are interesting and should be reported; and implications and conjectures should be discussed; but they should be carefully separated. I encourage the authors to critically examine the paper and remove or reword claims that are not directly supported by the results. On the experiments: a) For Figure 3, shouldn*t the inceptionv3 and inceptionv3-layer graphs be identical when the model size factor is 1x, since the models are identical? What explains the large discrepancies? b) The number of similar neurons per neuron (e.g., Figs 3c and 4c) should increase with model size just because there are more neurons, right? (Mathematically, if we consider a set of random vectors in , then as increases the number of vectors that correlate with each other should increase.) Is the increase that is experimentally observed any different from this? == Another comment (no need to respond): Experimental procedures should also be well-defined. For example, since neuron ablations are central to the experiments, the procedure should be explained in the main text instead of being buried in a citation.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Robustness and/or Redundancy Emerge in Overparametrized Deep Neural Networks
Deep neural networks (DNNs) perform well on a variety of tasks despite the fact that most used in practice are vastly overparametrized and even capable of perfectly fitting randomly labeled data. Recent evidence suggests that developing *compressible* representations is key for adjusting the complexity of overparametrized networks to the task at hand and avoiding overfitting (Arora et al., 2018; Zhou et al., 2018). In this paper, we provide new empirical evidence that supports this hypothesis, identifying two independent mechanisms that emerge when the network’s width is increased: robustness (having units that can be removed without affecting accuracy) and redundancy (having units with similar activity). In a series of experiments with AlexNet, ResNet and Inception networks in the CIFAR-10 and ImageNet datasets, and also using shallow networks with synthetic data, we show that DNNs consistently increase either their robustness, their redundancy, or both at greater widths for a comprehensive set of hyperparameters. These results suggest that networks in the deep learning regime adjust their effective capacity by developing either robustness or redundancy.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



The authors conduct an empirical study of the representations that deep neural networks learn as a function of model size, as measured by two different metrics: how quickly performance degrades when random neurons are removed (*robustness*), or how similar neurons are (*redundancy*). They show that deep neural networks tend to become more robust and more redundant as model size increases, though this is not necessarily true. In general, I believe that this sort of empirical approach is useful, and I appreciate the authors* efforts to study the phenomena of robustness and redundancy empirically. However, in my opinion, I do not believe that this paper is ready for publication in its current state. My central concern is the current imprecision of the paper, which undermines its scientific contribution. Many terms are ill-defined or fuzzy. For example, in section 2, 6 different capacity-constrained features are enumerated but not well-defined, including *semantically redundant units* that are *units which are not linearly correlated to others and which provide a different representation from the previous one, similarly related to the learning task.* What does it mean to *provide a different representation from the previous one* (and what previous one?), or to be *similarly related to the learning task*? Critically, there is a significant amount of speculation throughout the paper that is not clearly delineated from experimental observation. This speculation involves the above ill-defined terms and/or are not backed up. As non-exhaustive examples: 1) In page 3, robustness (as measured by neuron dropouts) is asserted to aid in distinguishing *redundant functional units* and *semantically redundant units* from other units because of differences in how robustness of those units change with model size; but this is not backed up and as far as I can tell, not experimentally investigated. Similar statements about redundancy and similarity are made in page 3. 2) In page 3, *L is more robust and equally or less redundant than S*: what is a holistic class representation? Why is that less robust than a *bag-like representation*? 3) Section 4 (results) has a lot of speculation as well. For example, *we take this discrepancy between robustness and redundancy trends as strong evidence that these models... are autoregularizing largely by forming qualitatively different representations... with units that are merely semantically redundant.* These terms are not clearly defined, and there is scant evidence to back up these conjectures. As it stands, it is difficult to accept the paper because of the many speculative claims it contains. The experimental findings are interesting and should be reported; and implications and conjectures should be discussed; but they should be carefully separated. I encourage the authors to critically examine the paper and remove or reword claims that are not directly supported by the results. On the experiments: a) For Figure 3, shouldn*t the inceptionv3 and inceptionv3-layer graphs be identical when the model size factor is 1x, since the models are identical? What explains the large discrepancies? b) The number of similar neurons per neuron (e.g., Figs 3c and 4c) should increase with model size just because there are more neurons, right? (Mathematically, if we consider a set of random vectors in , then as increases the number of vectors that correlate with each other should increase.) Is the increase that is experimentally observed any different from this? == Another comment (no need to respond): Experimental procedures should also be well-defined. For example, since neuron ablations are central to the experiments, the procedure should be explained in the main text instead of being buried in a citation.
assistant
The authors conduct an empirical study of the representations that deep neural networks learn as a function of model size, as measured by two different metrics: how quickly performance degrades when random neurons are removed (*robustness*), or how similar neurons are (*redundancy*). They show that deep neural networks tend to become more robust and more redundant as model size increases, though this is not necessarily true. In general, I believe that this sort of empirical approach is useful, and I appreciate the authors* efforts to study the phenomena of robustness and redundancy empirically. However, in my opinion, I do not believe that this paper is ready for publication in its current state. My central concern is the current imprecision of the paper, which undermines its scientific contribution. Many terms are ill-defined or fuzzy. For example, in section 2, 6 different capacity-constrained features are enumerated but not well-defined, including *semantically redundant units* that are *units which are not linearly correlated to others and which provide a different representation from the previous one, similarly related to the learning task. * What does it mean to *provide a different representation from the previous one* (and what previous one? ), or to be *similarly related to the learning task*? Critically, there is a significant amount of speculation throughout the paper that is not clearly delineated from experimental observation. This speculation involves the above ill-defined terms and/or are not backed up. As non-exhaustive examples: 1) In page 3, robustness (as measured by neuron dropouts) is asserted to aid in distinguishing *redundant functional units* and *semantically redundant units* from other units because of differences in how robustness of those units change with model size; but this is not backed up and as far as I can tell, not experimentally investigated. Similar statements about redundancy and similarity are made in page 3. 2) In page 3, *L is more robust and equally or less redundant than S*: what is a holistic class representation? Why is that less robust than a *bag-like representation*? 3) Section 4 (results) has a lot of speculation as well. For example, *we take this discrepancy between robustness and redundancy trends as strong evidence that these models... are autoregularizing largely by forming qualitatively different representations... with units that are merely semantically redundant. * These terms are not clearly defined, and there is scant evidence to back up these conjectures. As it stands, it is difficult to accept the paper because of the many speculative claims it contains. The experimental findings are interesting and should be reported; and implications and conjectures should be discussed; but they should be carefully separated. I encourage the authors to critically examine the paper and remove or reword claims that are not directly supported by the results. On the experiments: a) For Figure 3, shouldn*t the inceptionv3 and inceptionv3-layer graphs be identical when the model size factor is 1x, since the models are identical? What explains the large discrepancies? b) The number of similar neurons per neuron (e.g., Figs 3c and 4c) should increase with model size just because there are more neurons, right? (Mathematically, if we consider a set of random vectors in , then as increases the number of vectors that correlate with each other should increase.) Is the increase that is experimentally observed any different from this? == Another comment (no need to respond): Experimental procedures should also be well-defined. For example, since neuron ablations are central to the experiments, the procedure should be explained in the main text instead of being buried in a citation.