DataParallel().
If we use two GPUs, we know model will replicate on both GPUs, but I would like to know while saving model in checkpoint ---- Does checkpoint save the both replicated model?
I believe, the model which is present on default device (GPU-0) will be saved in the checkpoint, Right?