padding_idx is ignored in computing backward gradients.
Here’s the code location reflecting that: https://github.com/pytorch/pytorch/blob/1848cad10802db9fa0aa066d9de195958120d863/aten/src/ATen/native/Embedding.cpp#L117 100