If you don’t have a very large number of embedding layers, you could address the parameters directly, e.g. “model.emb.weight” if the embedding is named emb.
Also, I would probably follow Soumith’s suggestion to reset the padding embedding vector after the step instead of changing the grad before. Very likely both work, but if you are set to keep them fixed, resetting feels slightly more straightforward than setting the grad to zero in order to change the update to zero.
Best regards
Thomas