Penetrating the Fog: the Path to Efficient CNN Models
With the increasing demand to deploy convolutional neural networks (CNNs) on mobile platforms, the sparse kernel approach was proposed, which could save more parameters than the standard convolution while maintaining accuracy. However, despite the great potential, no prior research has pointed out how to craft an sparse kernel design with such potential (i.e., effective design), and all prior works just adopt simple combinations of existing sparse kernels such as group convolution. Meanwhile due to the large design space it is also impossible to try all combinations of existing sparse kernels. In this paper, we are the first in the field to consider how to craft an effective sparse kernel design by eliminating the large design space. Specifically, we present a sparse kernel scheme to illustrate how to reduce the space from three aspects. First, in terms of composition we remove designs composed of repeated layers. Second, to remove designs with large accuracy degradation, we find an unified property named~emph{information field} behind various sparse kernel designs, which could directly indicate the final accuracy. Last, we remove designs in two cases where a better parameter efficiency could be achieved. Additionally, we provide detailed efficiency analysis on the final 4 designs in our scheme. Experimental results validate the idea of our scheme by showing that our scheme is able to find designs which are more efficient in using parameters and computation with similar or higher accuracy.
 This paper addressed an interesting problem of reducing the kernel to achieve CNN models, which is important and attracts lots of research work. However, the methods don*t have very good justifications. For example, in Section 3.1, the authors mentioned that "Specifically, in normal CNNs it is quite common to have multiple stages/blocks which contain repeated patterns such as layers or structures." It is still unclear why it is better to replace these so-called repeated patterns. The defined "information field" is not clearly explained and the benefits are also not demonstrated.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Penetrating the Fog: the Path to Efficient CNN Models
With the increasing demand to deploy convolutional neural networks (CNNs) on mobile platforms, the sparse kernel approach was proposed, which could save more parameters than the standard convolution while maintaining accuracy. However, despite the great potential, no prior research has pointed out how to craft an sparse kernel design with such potential (i.e., effective design), and all prior works just adopt simple combinations of existing sparse kernels such as group convolution. Meanwhile due to the large design space it is also impossible to try all combinations of existing sparse kernels. In this paper, we are the first in the field to consider how to craft an effective sparse kernel design by eliminating the large design space. Specifically, we present a sparse kernel scheme to illustrate how to reduce the space from three aspects. First, in terms of composition we remove designs composed of repeated layers. Second, to remove designs with large accuracy degradation, we find an unified property named~emph{information field} behind various sparse kernel designs, which could directly indicate the final accuracy. Last, we remove designs in two cases where a better parameter efficiency could be achieved. Additionally, we provide detailed efficiency analysis on the final 4 designs in our scheme. Experimental results validate the idea of our scheme by showing that our scheme is able to find designs which are more efficient in using parameters and computation with similar or higher accuracy.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



 This paper addressed an interesting problem of reducing the kernel to achieve CNN models, which is important and attracts lots of research work. However, the methods don*t have very good justifications. For example, in Section 3.1, the authors mentioned that "Specifically, in normal CNNs it is quite common to have multiple stages/blocks which contain repeated patterns such as layers or structures." It is still unclear why it is better to replace these so-called repeated patterns. The defined "information field" is not clearly explained and the benefits are also not demonstrated.
assistant
 This paper addressed an interesting problem of reducing the kernel to achieve CNN models, which is important and attracts lots of research work. However, the methods don*t have very good justifications. For example, in Section 3.1, the authors mentioned that "Specifically, in normal CNNs it is quite common to have multiple stages/blocks which contain repeated patterns such as layers or structures." It is still unclear why it is better to replace these so-called repeated patterns. The defined "information field" is not clearly explained and the benefits are also not demonstrated.