I guess you’re implementing the backward by hand? Otherwise it will work with the autograd.
The derivative for tanh is quite common and can be found here 147 for example. The C implementation of the backward pass by pytorch is here 88.