From what I can tell, you are packing the input indices rather than the embeddings. The output of your self.encoder is not used in your second example.
I think you need to do something like sorted_inputs = emb.gather(0, index_sorted_idx.long()).
Hope this helps!