@ptrblck I think i want to do something like this except I am unsure how to add self-attention to intermediate resnet blocks
  
    
    
    Attention in image classification vision
  
  
    Hi, 
I’m using a resnet to extract features and the last layer gives me an output of (batchsize x 2048 x 1 x 1). 
When i passed this onto this code SelfAttention i get a the same dimensions out. Is this the correct behaviour? 
Can i use this vector to flatten and pass it to fully connected layer in order to classify my own classes? 
Thank you.
  


also, for self-attention, there are so many options. Is there any self-built module within pytorch that you would suggest?
I found these alternatives, what is your take?

Attention in image classification - #3 by AdilZouitine 4
GitHub - Chenglin-Yang/LESA: Locally Enhanced Self-Attention: Rethinking Self-Attention as Local and C 1
GitHub - leaderj1001/Stand-Alone-Self-Attention: Implementing Stand-Alone Self-Attention in Vision Mod 2