You’re right that there can be a bunch of issues in getting CUDA + multiprocessing to work correctly, I’d suggest starting off reading here for more info: Multiprocessing best practices — PyTorch 1.10.0 documentation 5
The main recommendation is to try using the spawn start method via something like multiprocessing.set_start_method(...).
Alternatively, another recommendation is to try using torch.multiprocessing and the mp.spawn method as documented here: Multiprocessing package - torch.multiprocessing — PyTorch 1.10.0 documentation 3 (check torch.multiprocessing.spawn).