The inputs are different video streams, sorry, I can’t talk about more detail here.
Our current arch is each process deal with each video stream separately, and we have MPS enabled.  Another arch is to only use one process and batch all inputs together. One bottleneck here is the GPU memory for large batch size. So you think the latter one will have better performance? (Let’s just assume after all model initialized)