Hello, I want to use the multiprocessing module to communicate autograd graphs to multiple nodes so that I can backpropagate the same problem samples at the same time using different loss functions.
Is there a recommended way to do this?
Could I pass the outputs and targets of the model into the args of mp.Process and then have each node perform criterion(outputs, targets).backward() ?
Right now I’m using this method and the first .backward() call works fine, but the second time I call .backward(), my program freezes.
Any suggestions or thoughts are welcome.
Thank you