@ptrblck, Thank you for reply.

Could you check the inputs for NaNs and Infs, please?
I assume the NaNs are returned during training?

Yes, NaN coming during training. Currently, I have already trained my model with Conv1d --> ReLU --> BatchNorm --> Dropout setup for TDNN block for 6 epochs without any problem. I have also successfully trained another LSTM based architecture on same data. So I believe there will be no NaN and Inf.

Could you store the inputs with the model’s  state_dict , so that we could reproduce it in a single iteration?

You want script and sample data from my side to reproduce the same by you, am I correct?