You may try the procedure:
define model=some_model(), and device=torch.device(‘cuda: 0,1,2’),
and model=torch.nn.DataParallel(model, device_ids=[0,1,2]), model=model.to(device),
then load the pretrained model:
model.module.load_state_dict(torch.load(‘path/to/model.pth’)