I am training the ALBERT LM model from scratch.
I have already trained it for Hindi and Bangla and it was working fine but when I am training on Gujarati and Telugu, the loss is becoming zero in 5K steps.
What could be the reason for the sudden drop in the loss? Can anyone suggest what could be the cause or how to debug such issue?
Any suggestions?