Solved by ptrblck in post #4 
              
              
                torch.no_grad() is more aggressive, as it Autograd won’t track any operations in the block, so that out2 won’t have a valid .grad_fn and you thus won’t be able to call backward on the calculated loss. 
.requires_grad will disable the grad calculation for this particular parameter, while the output o…