I think both batch-first and batch-second modes are compatible with DataParallel (it assumes batch-first by default, since that’s true of all non-RNN-related tensors, but it has a keyword argument to split over a different dimension). Both modes are definitely compatible with the rest of the RNN infrastructure, including pack_padded_sequence.