Solved by nielsr in post #4 
              
              
                Actually, that’s not possible, unless you compute cosine similarity between the mean of the last hidden state and the embedding vectors of each token in BERT’s vocabulary. You can do that easily using sklearn. 
The embedding matrix of BERT can be obtained as follows: 
from transformers import BertM…