Hey @TaranRai I think you need do call  model.to(device) before calling DataParallel ctor.

Would this get the most out of the GPUS?

DistributedDataParallel is expected to be faster than DataParallel. See this example 3