Hi,
So I just realized there’s possibly a simple fix for this. If one can represent the nn.Embedding operation as a matrix product, then this can be taken care of.
In particular, the embedding can be thought of as product between a hot-vector and the matrix of all embeddings. This formulation permits non one-hot vectors as input as well.