[CLS] token /  is used as the aggregate sequence representation for classification tasks
[CLS]  is  fed into an output layer for classification. How will be built this token. Is there something special done for it during training? in which way does happen this aggregations of sequences?

Thanks for response
[CLS], [SEP] are "special token" of bert. Its included in the vocabulary of size 30522. Its starts with like 101, 103 something.<|||||>of course these two tokens are special tokens. 

 **is  used  as  the  aggregate sequence representation for classification tasks.**

in which way does happen this aggregations?

>>The first token of every sequence is always a special classification  token  ([CLS]).  The  final  hidden  state corresponding  to  this  token  is  used  as  the  aggregate sequence representation for classification tasks.

from papert https://arxiv.org/pdf/1810.04805.pdf<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
