Transformers, Pytorch, Machine Learning, Deep Dives, Data Science.
batch, we need to align all words to the length of the longest word (watermelon). We will therefore add an extra token, PAD, to each word so they all end up with the same length as watermelon. In the below picture, the upper table represents the raw data, the lower table the encoded version: (image