AI, Large Language Models, Transformers, Machine Learning, Getting Started.
attention layer combines across positions (horizontally). The feed-forward layer combines across dimensions (vertically). And this is the secret sauce behind the power of the transformers — the ability to analyze data from different directions. To summarize the ideas above, here are the key points: