Hi
I would like to plot semantic space for specific words. Usually, we use word embeddings for this. But model I use (xlm-roberta) deala with language on the level of part of words (BPE tokens). It means that if I give to the model a word ‘hello’ I will get 2-3 vectors for each part of this word, right?
model(**tokenizer('hello', return_tensors="tf"),output_hidden_states=True)
# hidden states are embedding I suppose

So, what is the best approach to get one single vector to plot?
To be clear what do I mean by semantic space:

850×719 102 KB