PyTorch DDP requires the params in all GPUs follow the same order. e.g.
GPU0: weight [4,4], bias [4]
GPU1: bias[4], weight [4,4]
This is invalid.
Try print all the parameters names and theirs sizes to text files. Just make sure they follow the same order.
with open(f"params_{args.rank}.txt", "w") as fo:
    for name, param in model.parameters():
        fo.write(f"{name}\t{param.size()}\n")