Another quick question.
If I want to achieve the function of nn.Embeddingbag with nn.Embedding, is padding the only way I can do?
Assuming the followings are inputs to nn.Embeddingbag:
idx = [0,3,2,4,5,9,23]
offset = [0,2,4]
In this case, the actual input to nn.Embedding is [[0,3],[2,4],[5,9,23]].
So to make a batch from this input, do I need to pad?
I think i can increase the indices by 1 and pad 0 up to the max length of inputs, which is 3 in this case. And I can use padding_idx option in nn.Embedding.
However, my concern is that if the max length is large, then I would have to store a lot of 0s, which will be very memory inefficient.
Is there any way that I can achieve this efficiently?