As long as you define the backward manually, the numpy operations will work, as you are responsible to calculate the right gradients.
However, if you could swap all numpy methods to the PyTorch equivalents, Autograd will automatically create the backward pass for you.