<p>It depends on how you have modelled the probability distributions that you are sampling from when you turn the neural network output into a specific action.</p>
<p>If your three action dimensions are sampled independently (which would be a standard way to model a policy like this), then the probabilities or probability densities multiply. Which means the log probabilities add.</p>

CC BY-SA 4.0
2021-08-20T17:53:39.640Z
I got my actions from 3 different output layers of actor-network, and each one has its own mean and std. So I must use the sum function to add log_prob and save it in the memory.
m031n
CC BY-SA 4.0
2021-08-20T16:15:15.150Z
2021-08-20T16:15:15.150Z
Neil Slater
<p>I try to solve a navigation problem with PPO; my actions space have three-part:</p>
<ol>
<li>robot linear velocity that is in [-3,3] range (getting from a tanh activation func)</li>
<li>robot linear angular that is in [-pi/6, pi/6] range (getting from a tanh activation func)</li>
<li>robot step-time duration that is from [0.2, 0.5, 0.8] (getting from a softmax activation func)</li>
</ol>
<p>The problem that I face is how to calculate the ratio of probability from this separate disturbing?
Mean or sum? or was there another way to calculate log_prob from different distributions? Something like log_prob from multivariable distribution!</p>

<reinforcement-learning><probability-distribution><proximal-policy-optimization>
How to calculate policy probability ratio in multiple action space
m031n
# How to calculate policy probability ratio in multiple action space

(Asked by: username_0 on 2021-08-20)

I try to solve a navigation problem with PPO; my actions space have three-part:

1. robot linear velocity that is in [-3,3] range (getting from a tanh activation func)
2. robot linear angular that is in [-pi/6, pi/6] range (getting from a tanh activation func)
3. robot step-time duration that is from [0.2, 0.5, 0.8] (getting from a softmax activation func)

The problem that I face is how to calculate the ratio of probability from this separate disturbing?
Mean or sum? or was there another way to calculate log\_prob from different distributions? Something like log\_prob from multivariable distribution!

**Tags:** reinforcement-learning, probability-distribution, proximal-policy-optimization


---

## 1 answers

### Answer by username_1 on 2021-08-20. Score: 0 | Accepted answer

It depends on how you have modelled the probability distributions that you are sampling from when you turn the neural network output into a specific action.

If your three action dimensions are sampled independently (which would be a standard way to model a policy like this), then the probabilities or probability densities multiply. Which means the log probabilities add.

> **Comments:**
> **username_0**: I got my actions from 3 different output layers of actor-network, and each one has its own mean and std. So I must use the sum function to add log_prob and save it in the memory.  

---

