Hey @Mamdouh_Aljoud, please use torch.multiprocessing instead. See examples here: Multiprocessing best practices — PyTorch 1.7.0 documentation 7
Besides, I would recommend first try using multiple CUDA streams in the same process, using one stream for each model. See examples here: CUDA semantics — PyTorch 1.7.0 documentation 5