DataParallel would run the model on three batches in parallel, that won’t help with out of memory errors.
It would be possible to run different parts of a single model on several GPUs using part1.cuda(0), part2.cuda(1) etc. but then you would have to edit the forward method in order to shift the output of part1 to cuda(1) in order to input it to part2.
That would slow the model down considerably, and I am not certain that it would help with out of memory errors.