Hm but log(0) should actually never happen since the probabilities at this actions are zero and therefore they are never selected. Or am I wrong?
I do normalize the new selected states by dividing with the sum. Or do you mean something different?