Is that the training loss? Does it start at this point or does it increase and stay there? Does it explode after that point?
That can be many things, but try reducing the learning rate first, and see how that goes.