Three solutions:


use a normal distribution, use tanh as mu activation (to keep the center in range, prevent shifting too much) and then clamp, but you should do clamping only on the action sent to the environment, and not actions stored in buffers. In this way, you are not changing the pdf of your action, but changing the reward distribution.


use a normal distribution, use tanh to clamp your output right after sampling, but in order to calculate the log probability of your sample, you need to do this step, which is used in the Soft Actor Critic algorithm 16.

It should be relatively easy to deduce this equation


use a different distribution, but normally, the normal distribution performs the best in all algorithms, I have not tried Beta or Logit-Normal distribution and don’t know their performance. This method is theoretically possible, but result is not garanteed.