You could check the internal implementation and probably use the functional distributed API to send the appropriate splits to the desired device.
From a general point of view: nn.DataParallel will split the tensors in dim0, so would it be possible to make sure the data and target tensors are constructed in this way?
Also, you might take a look at DistributedDataParallel using one process per GPU, which should be faster, and could even simplify your use case, as each process would load its data.