What about the cost function, is it decreasing properly?
I would advise you to use some Tensorboard logging to better visualize these values, it’s quite easy to use and scales better in the long run. Check the tutorial 2.