@ptrblck,
I kinda figured out the problem. The thing is, whenever I load the pytorch inference code from cpu-main thread it works, but if I call the same from the cpu-worker thread it hangs in torch.load for all GPU models.
Do you know what is going on?