Thanks for replay.
I moved the zero_grad function to the beginning of training loop, but NAN situation has not changed.
For some reason, I need to use two models instead of one big model.
Could you give me some advice?