It seems that the embedding layer is raising this error.
This is often the case, if you pass invalid indices to this layer.
Could you add assert statements and check the inputs?
E.g.:
num_embeddings = 10
emb = nn.Embedding(num_embeddings=num_embeddings, embedding_dim=100).cuda()
input = torch.randint(0, num_embeddings, (5,)).cuda()

assert (input >= 0).all() and (input < num_embeddings).all(), 'INVALID INPUT'
out = emb(input)


input[0] = num_embeddings # will trigger assert
assert (input >= 0).all() and (input < num_embeddings).all(), 'INVALID INPUT'
out = emb(input)