CUDA multiprocessing is quite complicated, and I wouldn’t recommend it, unless it would give you huge performance benefits. The two most important things to remember are:

You can’t use the default fork start method. You need to switch multiprocessing to either use spawn (simpler) or forkserver (possible more efficient, but still subtle - server needs to be initialized before CUDA is initialized). The problem is that CUDA doesn’t support forking the process once the context has been initialized. The child will be in an inconistent state and it’s UB.
A shared CUDA storage/tensor from another process mustn’t go out of scope in the process that created it, for at least as long as it’s used in other processes. Otherwise, UB again.