Generally speaking, you’d represent the words either as word indexes (like eg ‘the’ is 1, ‘to’ is 2, etc… ), or as embeddings.
You can convert word indexes to embeddings by passing a LongTensor containing the indexes (not one-hot, just like eg [5,3,10,17,12], one integer per word), into an nn.Embedding.
You should never need to fluff the word indices up into actual physical one-hot. Nor do you need to use sparse tensors: nn.Embedding handles this all for you, internally.