@ mrshenli thanks for your reply. I tried your method after a few minor correction but it still gives me the same erroneous result. I use this resnet 2 script to call the model.
I trained it on a large dataset and decided to save it periodically during training. Due to testing slowing down the training I decided to test it later using the saved models. When I train DDP with 8 gpus and test DDP with 8 gpus later, there is no issue. However, when I train DDP with 8 gpus  and test DDP with 2 gpus later the problem occurs.
Also I only want to save and load the state_dict and not the entire model since it takes a lot of space.
I will create a working example for mnist shortly.