If you don’t want to train the embedding, then you can detach the output.
To use a pretrained embedding you would have to make sure you are loading the right weights.
Besides that it will just work as a lookup table.
The RuntimeError regarding the second backward pass seems to point to another issue and it’s a bit weird that the detach() solves it.
Could you post a code snippet to reprodice this issue using a random embedding?