Yeah, sometimes without dividing may provide you better convergence because gradients would be large. But also if the model is kinda big then there’s high chance for encountering nan. You can choose which works best in your use case.