Hi @Brando_Miranda,
Yes, your understanding is correct. Save the model (or any other sync’ed artifacts) to a permanent store on rank 0 during checkpointing, and load it on all ranks while resuming. What our documentation describes is the common idiom for saving/loading checkpoints, but if you have different requirements, you don’t have to follow it. For instance if your underlying store requires some form of sequential access, you can coordinate your workers using collective calls or coordinate them via a Store (e.g. TCPStore). In our example we have a basic demonstration of this coordination via a barrier call.
Do you mind explaining why the doc did not make sense to you? We always aim to improve our documentation, so any concrete feedback would be greatly appreciated.