Hi, I think the log_prob comes from the policy optimization algorithm (i.e., convert multiply to sum as here 9). The output of the policy network should still be a distribution over your action space.