Hello,
I’m trying to run inference on a MMSR model. The system has two 2080Ti GPUs and I’m running PyTorch 1.1.0 on Ubuntu 18.04 with CUDA 10.
I have wrapped the model around a nn.DataParallel, and have tried setting and ensured that the two GPUs are visible
os.environ["CUDA_VISIBLE_DEVICES"] = "0, 1"
torch.cuda.device_count() returned 2.
Can anyone help me out? The inference script just goes CUDA OOM and doesn’t use the second GPU.
Edit:
Code Snippet
model.load_state_dict(torch.load(model_path), strict=True)
model.eval()  
model = nn.DataParallel(model, devices_ids=["0, 1"])                                                                 
model = model.to(device)

I’ve looked at all tutorials on the website and used it before, haven’t come across this behaviour. Not sure what I’m doing wrong.
For some more context, I’m trying to load one batch at a time of 5 pretty large images (no way around this) I have a vague feeling I can’t use this approach, so any suggestions would be appreicated.
Thank you so much.