AudioRepInceptionNeXt: A lightweight single-stream architecture for efficient audio recognition
<|reference_start|>AudioRepInceptionNeXt: A lightweight single-stream architecture for efficient audio recognition: Recent research has successfully adapted vision-based convolutional neural network (CNN) architectures for audio recognition tasks using Mel-Spectrograms. However, these CNNs have high computational costs and memory requirements, limiting their deployment on low-end edge devices. Motivated by the success of efficient vision models like InceptionNeXt and ConvNeXt, we propose AudioRepInceptionNeXt, a single-stream architecture. Its basic building block breaks down the parallel multi-branch depth-wise convolutions with descending scales of k x k kernels into a cascade of two multi-branch depth-wise convolutions. The first multi-branch consists of parallel multi-scale 1 x k depth-wise convolutional layers followed by a similar multi-branch employing parallel multi-scale k x 1 depth-wise convolutional layers. This reduces computational and memory footprint while separating time and frequency processing of Mel-Spectrograms. The large kernels capture global frequencies and long activities, while small kernels get local frequencies and short activities. We also reparameterize the multi-branch design during inference to further boost speed without losing accuracy. Experiments show that AudioRepInceptionNeXt reduces parameters and computations by 50%+ and improves inference speed 1.28x over state-of-the-art CNNs like the Slow-Fast while maintaining comparable accuracy. It also learns robustly across a variety of audio recognition tasks. Codes are available at https://github.com/StevenLauHKHK/AudioRepInceptionNeXt.<|reference_end|>
@article{lau2024audiorepinceptionnext:,
    title={AudioRepInceptionNeXt: A lightweight single-stream architecture for
  efficient audio recognition},
    author={Kin Wai Lau, Yasar Abbas Ur Rehman, Lai-Man Po},
    journal={arXiv preprint arXiv:2404.13551},
    year={2024},
    archivePrefix={arXiv},
    eprint={2404.13551},
    primaryClass={cs.SD eess.AS}
}