Aloha,
I’m trying to explore alternatives to the Tanh backwards function and I started by setting up a baseline for the experiment by overwriting the Backwards function with 1 −  tanh^2(x)
However, I did not get the same results as when I used the autograd version of tanh’s derivative.  Can anyone shed some light on this?
Mahalo,
Jonathan