The problem is that your network has converged and keep on outputing “invalid” actions, setting their probability to be high and “valid” ones to be low. You should check your implementation rather than blaming the distribution itself.
A2C, A3C… these policy based methods relies on sampling from a distribution to calculate the needed log probability, it is not only “ok” but also “must”.