As far as I am aware, the Embedding class “forward-pass” converts each word-index in a LongTensor into its corresponding word embedding. However, I am wondering if there is a method to adapt Embedding to accept a categorical vector instead of word-index? Specifically, each vector represents a distribution over the entire vocabulary, and the Embedding would return a linear combination of the word embedding based on the vector. So for example:
If the vector is [0.7, 0.2, 0.0, 0.1], the embedding would be 0.7emb0+0.2emb1+0.0emb2+0.1emb3.
While I suppose one way to do this is to create a matrix of dimension [vocab_size, embedding_size] to substitute the Embedding, I have some pretrained Embeddings that I would prefer to (continue) using.
Any help is appreciated.