John_Watkins:

While I suppose one way to do this is to create a matrix of dimension [vocab_size, embedding_size] to substitute the Embedding, I have some pretrained Embeddings that I would prefer to (continue) using.


I haven’t tried it, but this should work just fine since both the embedding layer or your “manual” linear layer have the same shape (vocab_size, embed_dim). And it should be possible to set the weights of the linear layer to the pretrained word vectors.