Solved by ptrblck in post #2 
              
              
                If your weights got a NaN value, this might be due to a NaN input or a faulty weight update caused by e.g. a high learning rate. 
Did you observe the loss during training? 
If some weights are exploding, you would usually see a NaN loss.