EmbeddingBag are filled with Nan in GPU for empty bags, but zeros in CPU.
It should be zeros according to the pytorch document.

image.png1332×300 42.2 KB

here is the test code:
u_embedding = nn.EmbeddingBag(180,100)
offset = torch.LongTensor([0,2,2])
word_in = torch.LongTensor([234,234,23,234,53])
out = u_embedding(word_in,offset)
#out[1] is zeros

u_embedding = nn.EmbeddingBag(180,100).cuda()
offset = torch.cuda.LongTensor([0,2,2])
word_in = torch.cuda.LongTensor([234,234,23,234,53])
out = u_embedding(word_in,offset)
#out[1] is Nan

Is this a bug? what should i do if want to using GPU and set zeros by default for empty bags?
Is there a setup like ''padding_idx" in nn.EmbeddingBag layers?