What happens when we apply similar pure convolution blocks on patches of images? We can train a network with 0.8 million parameters for 10 epochs on CIFAR-10 and get ~83% top-1 test accuracy without having to use any fancy regularization. ConvMixer (the recently talked about architecture on Twitter):

  
      

      keras.io
  

  
    

Keras documentation: Image classification with ConvMixer 8



  

  
    
    
  

  


There are a few visualizations of the internals of ConvMixer that might be useful for the community.
Learned patch embeddings:

Convolution kernel from the middle of the network showing varying locality spans: