If your model is small, you can consider about using torch.jit._fork and torch.jit._wait, they can work around the GIL problem, but also requires to convert your models to JitScriptModule.
While pytorch does release GIL, it could still affect model performance if your model consists of many small linear layers like 100(in)->100(out).