Thanks for the reply. But isn’t the other tokens specific to a particular input token as opposed to the [CLS] token which doesn’t correspond to any input token?  If that’s the case, how does it make sense to take finetune any token for our classification?



 Jung:

is selected arbitrarily / conveniently .


This is what is tripping me up. Is there no reasoning empirical or otherwise to create a token called [CLS] to be used as input for downstream classification tasks?