Our case is that we have 2 GPUs, created 7 process for each GPU to deal with different inputs. Each process will load the same model into GPU(I know we can optimize here), our initial thinking is that GPU RAM should be the bottleneck, but reality is that CPU becomes the bottleneck first, that’s the reason why I did the investigation and seek help here.