Transformers, Deep Learning, Machine Learning, Editors Pick, Data Science.
we need to establish a notion of similarity between two embeddings. One way to represent this similarity is by using the dot product, where we would like to learn embeddings such that higher scores indicate that two words are more similar. As, for each token, we need to calculate its relevance with