Solved by ptrblck in post #2 
              
              
                I think both approaches would yield the same execution as Autograd should be smart enough to stop the backpropagation where no gradients are needed in previous layers (approach 1). 
The difference of passing all parameters or only the subset which requires gradients would be in this check, which che…