Solved by ptrblck in post #2 
              
              
                I guess the error is raised by using retain_graph=True and trying to update parameters multiple times, which would be wrong, since the gradient would be stale. 
Could you explain why you are using this argument?