Transformers, Deep Learning, Machine Learning, Editors Pick, Data Science.
learn, let’s introduce some more learnable parameters! Instead of using our embedding matrix directly, lets pass this through three, independent linear layers (matrix multiplications); this should enable the model to ‘focus’ on different parts of the embeddings. This is depicted in the image below: