You could just wrap the model in nn.DataParallel and push it to the device:
model = Model(input_size, output_size)
model = nn.DataParallel(model)
model.to(device)

I would not recommend to save the model directly, but instead its state_dict as explained here 131.
Also, after you’ve wrapped the model in nn.DataParallel, the original model will be accessible via model.module, so you might want to store the state_dict via torch.save(model.module.state_dict(), 'file_name.pt').