I would expect that code replacing the zeroing/relu in your original code with D.threshold(eps, eps) to be simpler than what you did here.

I’m really puzzled that other people don’t seem to have trouble with this?

It is fairly rare to actually evaluate functions at point where they are not differentiable (sqrt at 0). So it does not happen very often.