learning rate helps is a sense that I don’t get NaN but now the loss is astonishingly large.  After 500 epochs here are the loss:

training loss: 77571602712294141286416384.0000
validation loss: 30277167064551056736256.0000