Transformers from Scratch (2021)
An early explainer of transformers, which is a quicker read, that I found very useful when they were still new to me, is The Illustrated Transformer[1], by Jay Alammar.<p>A more recent academic but high-level explanation of transformers, very good for detail on the different flow flavors (e.g. encoder-decoder vs decoder only), is Formal Algorithms for Transformers[2], from DeepMind.<p>[1] <a href="https://jalammar.github.io/illustrated-transformer/" rel="nofollow">https://jalammar.github.io/illustrated-transformer/</a>
[2] <a href="https://arxiv.org/abs/2207.09238" rel="nofollow">https://arxiv.org/abs/2207.09238</a>