AutoGrow: Automatic Layer Growing in Deep Convolutional Networks
Depth is a key component of Deep Neural Networks (DNNs), however, designing depth is heuristic and requires many human efforts. We propose AutoGrow to automate depth discovery in DNNs: starting from a shallow seed architecture, AutoGrow grows new layers if the growth improves the accuracy; otherwise, stops growing and thus discovers the depth. We propose robust growing and stopping policies to generalize to different network architectures and datasets. Our experiments show that by applying the same policy to different network architectures, AutoGrow can always discover near-optimal depth on various datasets of MNIST, FashionMNIST, SVHN, CIFAR10, CIFAR100 and ImageNet. For example, in terms of accuracy-computation trade-off, AutoGrow discovers a better depth combination in ResNets than human experts. Our AutoGrow is efficient. It discovers depth within similar time of training a single DNN.
The paper presents a meta-learning algorithm to automatically detemine the depth of neural network through a policy to add depth if this bring improvement on accuracy. I have conserved opinion based on the technique being used here is extremely simple, basically is an implementation of naive greedy algorithm in such a scenario, which implies the problem may not be intrinsically hard, or even useful. The paper consists of detailed narrative about how these procedure are conducted, but still, it is really hard for me to find the true merit to appreciate, and why this brings a nontrivial and usefull contribution. The tables, visualization figures also didnot imply too much about whether this is more than overfitting on previous works with hand-chosen depth.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
AutoGrow: Automatic Layer Growing in Deep Convolutional Networks
Depth is a key component of Deep Neural Networks (DNNs), however, designing depth is heuristic and requires many human efforts. We propose AutoGrow to automate depth discovery in DNNs: starting from a shallow seed architecture, AutoGrow grows new layers if the growth improves the accuracy; otherwise, stops growing and thus discovers the depth. We propose robust growing and stopping policies to generalize to different network architectures and datasets. Our experiments show that by applying the same policy to different network architectures, AutoGrow can always discover near-optimal depth on various datasets of MNIST, FashionMNIST, SVHN, CIFAR10, CIFAR100 and ImageNet. For example, in terms of accuracy-computation trade-off, AutoGrow discovers a better depth combination in ResNets than human experts. Our AutoGrow is efficient. It discovers depth within similar time of training a single DNN.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



The paper presents a meta-learning algorithm to automatically detemine the depth of neural network through a policy to add depth if this bring improvement on accuracy. I have conserved opinion based on the technique being used here is extremely simple, basically is an implementation of naive greedy algorithm in such a scenario, which implies the problem may not be intrinsically hard, or even useful. The paper consists of detailed narrative about how these procedure are conducted, but still, it is really hard for me to find the true merit to appreciate, and why this brings a nontrivial and usefull contribution. The tables, visualization figures also didnot imply too much about whether this is more than overfitting on previous works with hand-chosen depth.
assistant
The paper presents a meta-learning algorithm to automatically detemine the depth of neural network through a policy to add depth if this bring improvement on accuracy. I have conserved opinion based on the technique being used here is extremely simple, basically is an implementation of naive greedy algorithm in such a scenario, which implies the problem may not be intrinsically hard, or even useful. The paper consists of detailed narrative about how these procedure are conducted, but still, it is really hard for me to find the true merit to appreciate, and why this brings a nontrivial and usefull contribution. The tables, visualization figures also didnot imply too much about whether this is more than overfitting on previous works with hand-chosen depth.