I checked, the input tensors don’t have nan or inf values. It is composed of sparse signals.
I tried a learning rate between 1e-3 and 1e-5, but I get the same results.
Yes the nan loss begins from the first batch.
I’m using a training dataset of 5000 signals.
I noticed that when I decrease the number of signals to 50, I don’t get nan losses.