Hi @Janine,
This is not related to PyTorch version but the DataParallel (also DistributedDataParallel) class wrapper of PyTorch nn class models.
DataParallel encloses the original model as it member variable, self.module.
In case you need both single-GPU and multi-GPU model training, you can change saving/loading behavior with if statements.
For example,
if isinstance(G, nn.DataParallel):
    torch.save(G.module.state_dict(), model_save_name)
else:
    torch.save(G.state_dict(), model_save_name)

If the current model class is DataParallel, you can save G.module.state_dict() otherwise save G.state_dict()
Also, at loading pretrained parameters, you could perform
if isinstance(G, nn.DataParallel):
    G.module.load_state_dict(state_dict)
else:
    G.load_state_dict(state_dict)

I would suggest stating a parent class Model that inherits nn.Module that overrides default state_dict function with the above method so that G and D could inherit it and simplify your training code part.
You may want to take a look at my code 30.
state_dict, load_state_dict, save functions are related.