Solved by ptrblck in post #10 
              
              
                Using the model or any CUDA operations in the collate_fn is uncommon, I think (at least I haven’t seen a use case so far) and I guess not well tested. 
The additional CUDA context creation might come from the usage of multiprocessing. 
I also don’t know what kind of 0MB process is created as I haven…