nn.EmbeddingBag != nn.Embedding
The first will give you the average of all the embeddings in the sequence while the second will give you the embeddings in a sequence.  I think you want to use the latter rather than the former.