Can you elaborate please ?

I linked a post describing how to wrap an arbitrary loss function via DataParallel so that you can compute it via scattering the dataset onto multiple GPUs. Then you responded

Im using evolution methods, I dont have backpropagation and my loss function is not differentiabe. I want to parallelize sequential basic operations like torch.matmul.

and I mentioned that in that case, if you don’t want to use backpropagation, you don’t need to call .backward() on your loss function. In other words, I also wasn’t sure how your answer was related to wrapping a loss function in DataParallel. I mean, you can use it whether or not you want to do backpropagation, because DataParallel is not tied to backpropagation as far as I know.