Thanks a lot! Thanks ptrblck and jpeg729!
The activation function resolved my issue. Now the loss can decreasing.
ps. seems both relu and tanh can work in my test, I not seen much difference in results.