Hi @ptrblck, I got a suggestion that changing Adam optimizer may help.
optimizer = torch.optim.RMSprop(model.parameters(), lr=2e-3)

with the optimizer above nan comes after 2 epochs!
Criteria at the end of epoch 1 subject 2 is 0.3492
criteria increased from 0.1109 to 0.3492, saving model ...
Nan found
Criteria at the end of epoch 2 subject 1 is 0.5875

Is it possible to find out what becomes nan first?
Also to avoid zero division error, I included epsilon value in all denominators and numerators as you can see in the custom loss functions.