Scaling Up Your Kernels: Large Kernel Design in ConvNets towards Universal Representations
<|reference_start|>Scaling Up Your Kernels: Large Kernel Design in ConvNets towards Universal Representations: This paper proposes the paradigm of large convolutional kernels in designing modern Convolutional Neural Networks (ConvNets). We establish that employing a few large kernels, instead of stacking multiple smaller ones, can be a superior design strategy. Our work introduces a set of architecture design guidelines for large-kernel ConvNets that optimize their efficiency and performance. We propose the UniRepLKNet architecture, which offers systematical architecture design principles specifically crafted for large-kernel ConvNets, emphasizing their unique ability to capture extensive spatial information without deep layer stacking. This results in a model that not only surpasses its predecessors with an ImageNet accuracy of 88.0%, an ADE20K mIoU of 55.6%, and a COCO box AP of 56.4% but also demonstrates impressive scalability and performance on various modalities such as time-series forecasting, audio, point cloud, and video recognition. These results indicate the universal modeling abilities of large-kernel ConvNets with faster inference speed compared with vision transformers. Our findings reveal that large-kernel ConvNets possess larger effective receptive fields and a higher shape bias, moving away from the texture bias typical of smaller-kernel CNNs. All codes and models are publicly available at https://github.com/AILab-CVC/UniRepLKNet promoting further research and development in the community.<|reference_end|>
@article{zhang2024scaling,
    title={Scaling Up Your Kernels: Large Kernel Design in ConvNets towards
  Universal Representations},
    author={Yiyuan Zhang, Xiaohan Ding, Xiangyu Yue},
    journal={arXiv preprint arXiv:2410.08049},
    year={2024},
    archivePrefix={arXiv},
    eprint={2410.08049},
    primaryClass={cs.CV cs.AI cs.LG}
}