If you haven’t seen them already, you might find these articles useful.

  
      
      arXiv.org
  
  
    

Attention Is All You Need 1

The dominant sequence transduction models are based on complex recurrent or
convolutional neural networks in an encoder-decoder configuration. The best
performing models also connect the encoder and decoder through an attention
mechanism. We propose...


  
  
    
    
  
  



  
      jalammar.github.io
  
  
    

The Illustrated Transformer 3

Discussions:
Hacker News (65 points, 4 comments), Reddit r/MachineLearning (29 points, 3 comments)


Translations: Chinese (Simplified), French, Japanese, Korean, Russian, Spanish

Watch: MIT’s Deep Learning State of the Art lecture referencing this...