AI, Large Language Models, Transformers, Machine Learning, Getting Started.
transformers, FFN is essentially a regular multilayer perceptron acting on a batch of independent data vectors. Combined with attention, it produces the correct ‘position-dimension’ combination. How do Attention and FFN work? So, without further ado, let’s dive into how attention-weighting and FFN