I don’t believe there is any such option. In DataParallel (and DistributedDataParallel), every GPU will have a replica of the model for local training, and every GPU will see input batches of the same size. Thus the amount of memory used by each rank (each GPU) in distributed training is approximately the same. If one of your GPUs is OOMing, you can try to:

reduce the batch size (though this will reduce the batch size for every rank, even on the GPU with enough memory)
use an optimizer that stores less local state (such as SGD as opposed to Adam)