I may be wrong when I said any tokens would do . If you have time, maybe you can just make an experiment about that.
My intuition is that at first each other token may indeed represent each original input token. But if you finetune any of them (backpropagation), it can also perform as good as [CLS]. (Never actually tried it).
One thing from my experience on Kaggle NLP competitions, however, is that the use of GlobalPooling1D is not inferior to [CLS] .