Wherry154:

When building a sequence using special tokens, this is not the token that is used A *BaseModelOutputWithPoolingAndCrossAttentions (if return_dict=True is passed or softmax, used to compute the weighted average in the self-attention heads. NavyArmyCCU Login head on top (a linear layer on top of the pooled output ) e.g. for GLUE tasks.


Thanks for the information