I think the tutorial you linked has a bug when it comes to the loading. You would want to load the state dict back to model.module, i.e.
# Load to whatever device you want 42
might well be amended as
model.module.load_state_dict(torch.load(PATH))

This way, the state dict matches the model without the DataParallel wrapper, and you can also load it to a unwrapped model on a single GPU (use map_location in torch.load if needed).
Best regards
Thomas