I was wondering what kind of embedding is used in the embedding function provided by pytorch. It’s not clear what is actually happening. For example is a pre-trained embedding being used to project the word tokens to its hypothetical space? Is there a distance measure being used? Or is it embedding models like word2vec?