There isn’t a way using PyTorch’s default nn.DataParallel class to do this, unfortunately.
You can try to do this by using the lower level constructs yourself, as the implementation is not that complicated: https://github.com/pytorch/pytorch/blob/master/torch/nn/parallel/data_parallel.py#L174-L189