ethan-r-gallup:

So even though it registers bias goes to infinity on the first iteration but it seems like the problem lies with the torch.mm function if you look at the anomalies reported during backpropagation.


I’m not sure whether I understand this correctly, but so basically, once you have inf, you are most likely in trouble:
When you get inf bias, one might expect an inf gradient to also propagate to the earlier operations (because for a + b, both a and b get the same gradients as the sum).
But once you have inf and matrix multiplications with varying signs, you very soonish get +inf and -inf which gets you NaN in further additions (such as in matrix multiplication reductions).
The other bit (you show MacLaurin and use Taylor, but hey) is that 30 elements of a Taylor series is… shall I say ambitious? Quite likely, this has the potential to be tricky w.r.t. avoiding inf and nan.
Best regards
Thomas