Solved by ptrblck in post #4 
              
              
                Are you sure the gradients are zeros everywhere in the embedding weight parameter? 
Note that only the selected weight “rows” will get a valid gradient, so depending on the way you’ve check it (e.g. by printing the .grad attribute only) it could look as if it’s all zeros. 
Here is a small example to…