doing xi[xi != xi] = 0 will zero-out the nans after the op. But that does not change the fact that you have a nan in the op.
The anomaly mode should point you to the forward function that is causing the nan. I guess one of the input is either nan or infinite? You want to prevent such input (not output) to be nan or inf.