Hi @apaszke, Thanks for the quick reply.
Sorry for not being clearer, I think I am already doing what you are describing–I have a custom class that inherits from nn.Module, and it is this custom class that I wrap with DataParallel. Then inside the forward() method of my custom class, I do the pack + lstm + unpack steps.
But the problem is that if I pass in a 100x2xF array with input_lengths = [100, 50], what happens is that GPU 1 unpacks the LSTM output into a 100x1xF tensor, and GPU 2 unpacks the LSTM output into a 50x1xF tensor, so that when DataParallel tries to gather them together along the batch dimension, it complains about the sizes not matching up.  (Thus the fix in my code to add padding to the output tensors on all GPUs, e.g. in this example I would pad GPU 2’s output to be 100x1xF).