<p>It's because, in the actor-critic algorithm, the objective function is an expectation under the <span class="math-container">$\tau$</span> of the policy. If we want to use off-policy data, we have to resort to <em>importance sampling</em> relative to the other policy.</p>

CC BY-SA 4.0
2020-03-22T10:38:08.390Z
What does the tau value mean?
asaf92
CC BY-SA 4.0
2019-01-08T02:33:33.190Z
2019-02-15T19:43:04.453Z
2019-02-15T19:43:04.453Z
apuffin
<p>Why is the actor-critic algorithm limited to using on-policy data? Or can we use the actor-critic algorithm with off-policy data?</p>

<reinforcement-learning><actor-critic-methods><on-policy-methods><off-policy-methods>
Why is the actor-critic algorithm limited to using on-policy data?
apuffin
# Why is the actor-critic algorithm limited to using on-policy data?

(Asked by: username_0 on 2019-01-06)

Why is the actor-critic algorithm limited to using on-policy data? Or can we use the actor-critic algorithm with off-policy data?

**Tags:** reinforcement-learning, actor-critic-methods, on-policy-methods, off-policy-methods


---

## 1 answers

### Answer by username_0 on 2019-01-08. Score: 1

It's because, in the actor-critic algorithm, the objective function is an expectation under the $\tau$ of the policy. If we want to use off-policy data, we have to resort to *importance sampling* relative to the other policy.

> **Comments:**
> **username_1**: What does the tau value mean?  

---

