Thanks you! I am trying to train the network with double now. The training turned slower and harder. The loss was decreased, but it seemed that the loss still would not get lower while in the magnitude of 10^-6. Maybe the learning rate needs to be tuned more carefully.