I see. Yeah, that makes sense. I guess I was confused because I looked at the code in the context of the equation I gave in the original post. Maybe alpha / n is can be treated as some new single constant alpha_prime (equal to alpha in the equation in my question) and that’s why torch uses that formula? Either way, thanks for resolving my doubt!