Sorry. I am not clarify so clearly, my data is a numpy image which shape is 416x416x3, because my model is so large, it cannot load into one gpu at same time, so I use model1.to(cuda:0), data.to('cuda:0) and rotate models between GPUs by for loop. seem like that:
image891×648 234 KB
All the model run in single processes, and can only inference in one GPU(other will suspend).