I think it is the expected behavior. The purpose of introducing  const * tanh(u) is to increase the exploration, as mentioned in the paper. By applying a softmax on these numbers, you get the same A for all of them. Which can motivate the algorithm to explore more.