I’m not exactly sure whether my method does the padding on gpu or not, but what I used to do was to pass a list of gpu tensors to the model and pad them inside the model. Also, I feel like that the model would be faster if u do not use a PackedSequence as input; instead, u could use a padded tensor as the input(my feeling came from the “note” under the LSTM part in the doc).