@BramVanroy @sgugger,
I studied this a while ago and can’t remember all the details. I think it has something to do with Bert being trained at the token-level and it’s loss objective. Here is a paper about Sentence-Bert 15 that is a good reference.  Also, here is the Sentence Transformers github 6 by the author. Just reading through the issues on there was a wealth of information on the topic.