Hi @ptrblck , thanks for your reply.
I performed the hook thanks to your code and could see that the 1st layer doesn’t get any nan.
The 2nd and the 3rd layer do.
The outcome of the debugging after the “explosion”;

dense1 ; Has any nan: False ; Count nan vals: 0 (torch.Size([1024, 2048]) shape)
dense2 ; Has any nan: True ; Count nan vals: 1024 (torch.Size([1024, 2048]) shape)
dense3 ; Has any nan: True ; Count nan vals: 1023 (torch.Size([1024, 1]) shape)

If I read it correctly means that there’s a column which is making the nan (2º layer). After debugging further I can see that this column is 5th (all values there are nan).
Surprisingly in the (3º layer) we don’t get all nan in the output, despite all 5th columns having nan (item 434/1024 has value 2.0079257.
Can you please collaborate this theory, or tell me if I should look elsewhere and if not asking to much point me to whether this looks like a model construction problem or data.
Thanks once again!