I print the scale after each step’s update as follows:
print(scaler.get_scale())

...
65536.0
32768.0
32768.0
16384.0
8192.0
4096.0
...
1e-xxx
...
0.0
0.0
0.0

And all the grads and loss after this step became  Nan  (the scale still is 0).
What’s wrong with my model, loss function or training data? How to fix it?