similar problem faced by me as I was using relu in the layer and then apply softmax on the layer and then cross entropy so make sure that last layer should not contain any activation function if you are using softmax