Transformers, Deep Learning, Machine Learning, Editors Pick, Data Science.
more transformer blocks, we have a complex contextualised embedding matrix, representing an embedding for each token in the sequence. However, to use this for a downstream task such as classification, we only need to make one prediction. Traditionally, the first token is taken, and passed through a