I don’t exactly know how this works, but if I recall correctly, one is ought to directly use Normal.log_prob(value) instead of manually taking the log of the probability, as it is numerically more stable that way.
Now I need to take the log of the CDF of a normal distribution, and NaNs are appearing somewhere in my training pipeline. I’m guessing it is because of a similar problem (but let me know if that doesn’t make sense).
What would be the best way to compute the log CDF in a numerically stable way? Similarly, I also need log(1 - Normal.cdf(value).
I know SciPy has logcdf() and logsf() for distribution classes, where ‘sf’ is ‘survival function’ i.e. 1 - cdf. In PyTorch, would I do best to just implement those myself?