Well, the thing is that Embedding layers take indices as input (of integer types). And such Tensors cannot track gradients (because they are not contiguous). So I still think that you convert to an integer type between the two layers and thus don’t get gradients for your generator.