I’m now reimplementing the “Holistically-nested Edge Detection”(arXiv 15) with pytorch, which has multiple “side-outputs”.
So how can I backward CNN with multiple outputs ? This thread suggested to use torch.autograd.backward([x1, x2], [grad_x1, grad_x2]), but I run into error with this solution, following is my error message:
Traceback (most recent call last):
  File "main.py", line 68, in <module>
    train()
  File "main.py", line 63, in train
    torch.autograd.backward(outputs, grads)
  File "/usr/local/lib/python2.7/dist-packages/torch/autograd/__init__.py", line 98, in backward
    variables, grad_variables, retain_graph)
RuntimeError: CUDNN_STATUS_MAPPING_ERROR

My code can be summarized as follow:
outputs = model(data)
loss, grads = some_numpy_tricks(outputs)
torch.autograd.backward(outputs, grads)
print "now loss=%f"%loss

In which outputs is a list containing multiple outputs of the model, and  grads is the list containing gradients of loss w.r.t each model output.
So why did error occur and what’s the correct solution for my case?