I can’t say that they’re equivalent for sure, because I can’t see where avg comes from in the Lua version, but hey look the same to me. As @smth said, we’ve tested convergence on many tasks, and we can’t help you with the limited information we have. I’d suggest to look for NaNs or infs in grads. You might also want to compare the gradients you’re getting from both versions.