Hi ! Thx for your code 
Currently I’m ordering it using pytorch’s torch.sort function.
But I’m not sure if the gradients are correct, because my grad_output contains the original data ordering (I need to maintain the original order to compare the distance of two sequences), but each model receives sequences in a different ordering.
So I assume that I’ll need to reorder the grad_output (loss derivatives) to match the forward order.
But I’m not sure if I really need to do it, or if pytorch does it internally. And if necessary, I don’t know how to reorder the grad_output