The requirement is used in the implementation of RNN. I doubt that removing those would be easy and the correct thing to do. What I would do is to sort the input sequence by length, store the indices, and use the indices to retrieve corresponding target to compare.