yamand16:

Edit: When I used model.module.module.optimizer_G, it worked.


This is what I suspected was going to be the answer.
You could carefully go through all of your code (or at least the parts where the model gets used), because if this happens:



 yamand16:

Here is the output of the print(type(model.module))

<class ‘torch.nn.parallel.data_parallel.DataParallel’>



It means that the model is being wrapped into the DataParallel class before, and I don’t know if wrapping it twice will cause any bugs or unexpected behavior.
Also, the recommended way is to use DistributedDataParallel whenever possible instead of DataParallel, since it creates one process per GPU. Here is a link 10 to a tutorial that I found very helpful for learning how to use DistributedDataParallel.
I hope it helps!