Hao-Yan:

created 7 process for each GPU to deal with different inputs


I’m not sure why you need a dedicated process for different inputs.
Could you explain a bit more, in which way the inputs differ?



 Hao-Yan:

our initial thinking is that GPU RAM should be the bottleneck, but reality is that CPU becomes the bottleneck first


That might be the case, e.g. if your data loading is not fast enough.
However, do you see that multiple processes (including recreating the model) on a single device speed things up?
I would assume that multiple workers in a single DataLoader would give a better performance.