Solved by ptrblck in post #2 
              
              
                Yes, using other libraries or explicitly detaching the tensors won’t allow Autograd to track these operations and will thus detach the tensor from the computation graph (parameters used in previous operations also won’t gradients). 
To use other libraries you could write a custom autograd.Function a…