Are you planning to run your model(s) on a GPU?
In general, a CPU model should be thread safe (there are some exceptions though; some people report involving functions that use MKL with multiprocessing causes hanging). If you’re running CUDA models on one GPU, you will get better performance by not running multiple models at the same time, so it would be good to run a mutex here. If you’re running CUDA models on multiple GPUs, that will probably deadlock due to the nccl backend.