You should be able to load a CPU state_dict to each device using DDP.
However, if the original state_dict was stored on a specific GPU, you might run into the mentioned issue.
Let me know, if this would be the case, as I haven’t seen it yet.