Hi Thomas!



 tom:

it appears to be solved in the nightlies


I can confirm that this is fixed in today’s nightly, 1.9.0.dev20210504
(but not yet in my April nightly, 1.9.0.dev20210416):
1.9.0.dev20210504
forward looks okay:                   relerr = tensor(1.0752e-07)
built-in backward looks unstable:     relerrpt = tensor(1.5066e-07)
backward for hand-rolled looks okay:  relerrtt = tensor(1.8833e-07)
sig[:5] = tensor([1.9287e-22, 2.1316e-22, 2.3558e-22, 2.6035e-22, 2.8774e-22])
xpt.grad[:5] = tensor([1.9287e-22, 2.1316e-22, 2.3558e-22, 2.6035e-22, 2.8774e-22])
xtt.grad[:5] = tensor([1.9287e-22, 2.1316e-22, 2.3558e-22, 2.6035e-22, 2.8774e-22])

(The “looks unstable” is just hard-wired text; the result is correct.)
Thanks.
K. Frank