I am using many zero-paddings in my batch data. The padding index is 0. So I want my model to ignore the padding effect of my trainings. So could someone check my code is correct or not?
 optimizer.zero_grad()
 loss.backward()

 for name, param in model.named_parameters():
       if param.grad is not None:
           if 'A.' in name or 'W.weight' in name:
               param.grad.data[0] = 0

 optimizer.step()


Reference
After changing weight of the nn.Embedding, the values at the padding_index also changed