If I make the network dimension float16, the output is nan (if epoch > 0).
Forward pass is handled correctly (epoch = 0). After I do the backward, on the second epoch nan. Does anyone know why this is?