I’m not sure that’s possible with an Embedding layer, since, as you’ve already explained, you should pass an index tensor containing long values.
If there would be a method to compute gradients of type long (which I’m unaware of), this would mean your indices would change due to the gradients.
Is this assumption correct?
For an NLP use case, the gradient would change the “words” which were encoded using the index. Is there any underlying relation between your indices, i.e. are the indices 2 and 3 nearer to each other than 2 and 100?