Hey @driou
Are you using torch.multiprocessing only or does your code also used any feature from torch.distributed or torch.nn.parallel?
If it just torch.multiprocessing, it might relates to forking the processing. If the parent process used any CUDA-related feature, there will be a CUDA context on it which does not fork. Even if it is CPU only, the fork could have broken the OMP internal states see discussion here: https://github.com/pytorch/pytorch/issues/41197 4
Can you check if using spawn from torch.multiprocessing solves the problem?