A new interesting Convmixer approach is avaialable (ICLR 2022 submission) :

  
      

      OpenReview
  

  
    

Patches Are All You Need?

  Although convolutional networks have been the dominant architecture for vision tasks for many years, recent experiments have shown that Transformer-based models, most notably the Vision Transformer...


  

  
    
    
  

  



  

      Medium – 2 Nov 21
  

  
    

ConvMixer: Patches Are All You Need? Overview and thoughts 🤷

  CNNs don’t always have to progressively decrease resolution. A revolutionary idea that might shape the next-gen architectures for Computer…

  
    Reading time: 8 min read