Just as a suggestion for someone that have this problem, make sure you check your inputs thoroughly and make sure they do not contain NAN. I had this same problem, I spent a week trying  all different solutions such as gradient clipping just to figure out that my problem was that one of the input values is NAN which throws off my calculations leading to a loss thats NAN.