Is there any difference between, saving checkpoint when training with a single GPU and saving checkpoint with 2 GPUs?
Example: If I use DataParallel to train on 2 GPUs, if I save checkpoint after each epoch, which parameters will be saved? GPU1 info saved or GPU-2 info saved in checkpoint ?
How to check while tranining