When you say sparse embedding, you mean the nn.Embedding with sparse = True?
If so, that means that it generates a sparse gradient. So the op just before needs to be able to handle such gradient properly.