Hi,
There are things to be careful when using streams (note that I’m not a specialist at all!) but if you do any op on the default stream, it will sync all streams. So you need to make sure no such op is done. the nvidia visual profiler is a great tool to see what runs where and what runs in parallel. https://developer.nvidia.com/nvidia-visual-profiler 8
Also for very small problems, it’s possible that you are limited by how far you can ask for stuff to be done on the gpu, meaning that you are actually bound by the cpu code. To improve this, you would need to use different processes for each model.