Convolution operator is the core of convolutional neural networks (CNNs) and occupies the most computation cost. To make CNNs more efficient, many methods have been proposed to either design lightweight networks or compress models. Although some efficient network structures have been proposed, such as MobileNet or ShuffleNet, we find that there still exists redundant information between convolution kernels. To address this issue, we propose a novel dynamic convolution method named DyNet in this paper, which can adaptively generate convolution kernels based on image contents. To demonstrate the effectiveness, we apply DyNet on multiple state-of-the-art CNNs. The experiment results show that DyNet can reduce the computation cost remarkably, while maintaining the performance nearly unchanged. Specifically, for ShuffleNetV2 (1.0), MobileNetV2 (1.0), ResNet18 and ResNet50, DyNet reduces 37.0%, 54.7%, 67.2% and 71.3% FLOPs respectively while the Top-1 accuracy on ImageNet only changes by +1.0%, −0.27%, −0.6% and −0.08%. Meanwhile, DyNet further accelerates the inference speed of MobileNetV2 (1.0), ResNet18 and ResNet50 by 1.87×, 1.32× and 1.48× on CPU platform respectively. To verify the scalability, we also apply DyNet on segmentation task, the results show that DyNet can reduces 69.3% FLOPs while maintaining the Mean IoU on segmentation task.
Main contribution of the paper
- The paper proposes a dynamic convolution selection method can be applied to arbitrary classification networks based on the global average pooled (GAP) feature map info.
- The method obtained improvements over various networks (SuffleNet v2, MobileNet v2, ResNet 18) on ImageNet.
Methods
- Given the set of fixed convolutional filters, the method dynamically selects the (weighted sum) kernels by given a kind of channel attention.
- The GAP of the features gives the channel attention on each stage, and the method applies the dynamic selection of the kernels.
- The number of channels in skip-connection shluld be the same because it should be elementwise multiplied with the channel attention acquired from GAP.
- The author slightly revises the baseline networks to set the networks integrated with the proposed method to have smaller Flops.
Questions
- According to Figure 4, it seems that the proposed add-on requires many parameters because it would include a FC layer for each block. But we cannot find the number of parameters in this paper.
- The parameter $g_t$ is defined as 6. The experiment shows the ablation to the case $g_t$ =1, but what if we set the parameters to other numbers?
Strong points
- The proposed model achieved improvement with fewer Flops on large scale image classification dataset.
- The method shows effectiveness when it is attached to various classification networks.
Concerns
- The main concern of the reviewer is that the model shares the core contribution to the existing method; squeeze-and-excitation network (SEnet, Hu et.al.). The method also proposes the attention-based scaling of channels, where the attention comes from GAP, so the reviewer thinks that it is possible to explain this work as some variation of SEnet.
The author should clarify the difference and the strong points of the proposed block compared to SEnet.
- Also, the reviewer cannot guarantee that the networks trained by the proposed method can transfer the knowledge to other tasks such as detection. 
The reviewer thinks that it is a critical part because one of the primal reasons for training the network is to use them as the pre-trained backbone for the other tasks.
Regarding this, the baseline methods (MobileNet V2, Shufflenet v2, ResNet)  are widely used as a pre-trained backbone for object detection, and the papers mention the CoCo object detection results using the pre-trained backbones from their method. The reviewer thinks that the experiment regarding this should be included.
- The other thing is that the parameter increases. As in the question, the reviewer thinks that the number of parameters would be increased. The reviewer agrees that some recent works focus more on Flops, but the number of parameters is also discussed in general, when telling about the 'model size'.
Conclusion
- The author proposed a dynamic kernel selection method (add-on), which can enhance the classification accuracy of the baseline network. 
- However, the reviewer cannot convince the novelty of the proposed approach and usefulness of the pre-trained backbone network from the proposed method when applying it to the other tasks (Object detection).
Inquiries
- Clarifying the difference between SEnet.
- Testing the ImageNet trained network of the proposed method into an object detection task (as the pre-trained backbone).
- Discussing the number of the parameter as well.