Solved by ptrblck in post #2 
              
              
                Both approaches should work. 
The first one would use less memory and more compute, since you would be using two backward calls (more compute), but Autograd will be able to delete the intermediate tensors from the first forward pass, as they are not needed anymore (gradients were already computed us…