  We present SplitMixer, a simple and lightweight isotropic MLP-like
architecture, for visual recognition. It contains two types of interleaving
convolutional operations to mix information across spatial locations (spatial
mixing) and channels (channel mixing). The first one includes sequentially
applying two depthwise 1D kernels, instead of a 2D kernel, to mix spatial
information. The second one is splitting the channels into overlapping or
non-overlapping segments, with or without shared parameters, and applying our
proposed channel mixing approaches or 3D convolution to mix channel
information. Depending on design choices, a number of SplitMixer variants can
be constructed to balance accuracy, the number of parameters, and speed. We
show, both theoretically and experimentally, that SplitMixer performs on par
with the state-of-the-art MLP-like models while having a significantly lower
number of parameters and FLOPS. For example, without strong data augmentation
and optimization, SplitMixer achieves around 94% accuracy on CIFAR-10 with only
0.28M parameters, while ConvMixer achieves the same accuracy with about 0.6M
parameters. The well-known MLP-Mixer achieves 85.45% with 17.1M parameters. On
CIFAR-100 dataset, SplitMixer achieves around 73% accuracy, on par with
ConvMixer, but with about 52% fewer parameters and FLOPS. We hope that our
results spark further research towards finding more efficient vision
architectures and facilitate the development of MLP-like models. Code is
available at https://github.com/aliborji/splitmixer.

Computer Vision and Pattern Recognition;Artificial Intelligence;Machine Learning