Thanks for the explanation.
If I understand the use case correctly, you would like to use the pretrained embeddings to create the continuous output features for categorical inputs and train a new classifier with it?
If so, then this would be supported and you could either store the output activations for all inputs directly using forward hooks or just use the nn.Embedding layer in isolation, feed the input to it, transform the output to numpy arrays, and train your new classifier with it.