Solved by albanD in post #2 
              
              
                Hi, 
You won’t be able to get gradients wrt to the input of the embedding layer I’m afraid. Since, as you pointed out, they are not of contiguous dtype. 
You might want to do use that technique on the output of the embedding layer instead?