try Sentence transformer which is much better for clustering from feature extraction than vanilla BERT or RoBERTa.

Hello, but why would sentence-transformer perform better than vanilla pre-trained BERT/ any-other transoformer ?