Slimmable Neural Networks
We present a simple and general method to train a single neural network executable at different widths (number of channels in a layer), permitting instant and adaptive accuracy-efficiency trade-offs at runtime. Instead of training individual networks with different width configurations, we train a shared network with switchable batch normalization. At runtime, the network can adjust its width on the fly according to on-device benchmarks and resource constraints, rather than downloading and offloading different models. Our trained networks, named slimmable neural networks, achieve similar (and in many cases better) ImageNet classification accuracy than individually trained models of MobileNet v1, MobileNet v2, ShuffleNet and ResNet-50 at different widths respectively. We also demonstrate better performance of slimmable models compared with individual ones across a wide range of applications including COCO bounding-box object detection, instance segmentation and person keypoint detection without tuning hyper-parameters. Lastly we visualize and discuss the learned features of slimmable networks. Code and models are available at: https://github.com/JiahuiYu/slimmable_networks
 This paper presents a straightforward looking approach for creating a neural networks that can run under different resource constraints, e.g. less computation but lower quality solution and expensive high quality solution, while all the networks are having the same filters. The idea is to share the filters of the cheapest network with those of the larger more expensive networksa and train all those networks jointly with weight sharing. One important practical observation is that the batch-normalization parameters should not be shared between those filters in order to get good results. However, the most interesting surprising observation, that is the main novelty of the work that even the highest quality vision network get substantially better by this training methodology as compared to be training alone without any weight sharing with the smaller networks, when trained for object detection and segmentation purposes (but not for recognition). This is a highly unexpected result and provides a new unanticipated way of training better segmentation models. It is especially nice that the paper does not pretend that this phenomenon is well understood but leaves its proper explanation for future work. I think a lot of interesting work is to be expected along these lines.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Slimmable Neural Networks
We present a simple and general method to train a single neural network executable at different widths (number of channels in a layer), permitting instant and adaptive accuracy-efficiency trade-offs at runtime. Instead of training individual networks with different width configurations, we train a shared network with switchable batch normalization. At runtime, the network can adjust its width on the fly according to on-device benchmarks and resource constraints, rather than downloading and offloading different models. Our trained networks, named slimmable neural networks, achieve similar (and in many cases better) ImageNet classification accuracy than individually trained models of MobileNet v1, MobileNet v2, ShuffleNet and ResNet-50 at different widths respectively. We also demonstrate better performance of slimmable models compared with individual ones across a wide range of applications including COCO bounding-box object detection, instance segmentation and person keypoint detection without tuning hyper-parameters. Lastly we visualize and discuss the learned features of slimmable networks. Code and models are available at: https://github.com/JiahuiYu/slimmable_networks
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



 This paper presents a straightforward looking approach for creating a neural networks that can run under different resource constraints, e.g. less computation but lower quality solution and expensive high quality solution, while all the networks are having the same filters. The idea is to share the filters of the cheapest network with those of the larger more expensive networksa and train all those networks jointly with weight sharing. One important practical observation is that the batch-normalization parameters should not be shared between those filters in order to get good results. However, the most interesting surprising observation, that is the main novelty of the work that even the highest quality vision network get substantially better by this training methodology as compared to be training alone without any weight sharing with the smaller networks, when trained for object detection and segmentation purposes (but not for recognition). This is a highly unexpected result and provides a new unanticipated way of training better segmentation models. It is especially nice that the paper does not pretend that this phenomenon is well understood but leaves its proper explanation for future work. I think a lot of interesting work is to be expected along these lines.
assistant
 This paper presents a straightforward looking approach for creating a neural networks that can run under different resource constraints, e.g. less computation but lower quality solution and expensive high quality solution, while all the networks are having the same filters. The idea is to share the filters of the cheapest network with those of the larger more expensive networksa and train all those networks jointly with weight sharing. One important practical observation is that the batch-normalization parameters should not be shared between those filters in order to get good results. However, the most interesting surprising observation, that is the main novelty of the work that even the highest quality vision network get substantially better by this training methodology as compared to be training alone without any weight sharing with the smaller networks, when trained for object detection and segmentation purposes (but not for recognition). This is a highly unexpected result and provides a new unanticipated way of training better segmentation models. It is especially nice that the paper does not pretend that this phenomenon is well understood but leaves its proper explanation for future work. I think a lot of interesting work is to be expected along these lines.