Hey all !
This is Nikolas from AI Summer.
I am pretty interested in self-attention and transformers in computer vision. I have started an open source project to collect my process of re-implementing different modules in self attention and transformers architectures in computer vision.
If there is anybody that is interested in the same stuff please do let me know.
Here is a list of articles that I wrote:

How Attention works in Deep Learning: understanding the attention mechanism in sequence models 6
How Transformers work in deep learning and NLP: an intuitive introduction
How the Vision Transformer (ViT) works 3
 Understanding einsum for Deep learning: implement a transformer with multi-head self-attention from scratch 2

Based on all my research from zero to hero for self attention I also created this repo:
  
      
      github.com
  
  
    

The-AI-Summer/self-attention-cv 15


  Implementation of self-attention mechanisms for general purpose. Focused on computer vision modules. Ongoing repository. pip package coming soon...

  
  
    
    
  
  


Would love to hear that more people are interested to re-implement papers in this direction.
Have a fab. day!