In https://pytorch.org/tutorials/intermediate/ddp_tutorial.html 51, the code demo_checkpoint really confused me. There are two main quetions:
(1) In demo_checkpoint, all processes need to be synchronized by loading the same checkponts saved by process-0. If it is necessary when we train model with  DistributedDataParallel.?
(2) If it is necessary to synchronize model across multi nodes and gpus by loading the same checkpoint, how can node-B load checkpoints saved in node-A?
I don’t know if I understand demo_checkpoint in a right way. Could you please help answer this question? Thanks !