If you use non-pytorch operations (like numpy) then the autograd engine will not work.
If you really need these operations, you will have to create a custom autograd.Function for which you have to implement both the forward and the backward pass. In this, you will get Tensors that do not require gradient and you can use numpy. But you will need to implement the backward pass yourself. You can read the doc here 31 on how to do this.