I am glad to present my implementation of the “Fastformer: Additive Attention Can Be All You Need” paper.
This is a Transformer variant based on additive attention that can handle long sequences efficiently with linear complexity. Fastformer is much more efficient than many existing Transformer models and can meanwhile achieve comparable or even better long text modeling performance.

  
      

      github.com
  

  
    

GitHub - Rishit-dagli/Fast-Transformer: An implementation of Fastformer:... 26


  An implementation of Fastformer: Additive Attention Can Be All You Need in TensorFlow - GitHub - Rishit-dagli/Fast-Transformer: An implementation of Fastformer: Additive Attention Can Be All You Ne...