Programming, Artificial Intelligence, Machine Learning, Data Science, Python.
is used to create unique encodings for each position. It’s calculated using np.exp(np.arange(0, embedding_dimension, 2) * -(np.log(10000.0) / embedding_dimension)). We initialize a matrix positional_encodings of shape (sequence_length, embedding_dimension) filled with zeros. We fill the even