Hi @SMB, one thing that might be causing trouble is that PyTorch will attempt to use all cores by default to process each request. You could try setting the number of threads to 1 in your startup_event function with
torch.set_num_threads(1)

See here 8 for more details. I’m not sure this will solve your concurrency problem, but it’s a good place to start