I thought the problem pertain the implementation. Have you tried a different system? like google colab?
Actually, in a model of mine, the loss values explodes in my local environemnt(old system) but on colab model trains very well.