<p>Per <a href="http://adp.princeton.edu/Papers/Powell_ADP_2ndEdition_Chapter%206.pdf" rel="nofollow noreferrer">this paper</a> a look ahead policy is a policy that will make decisions based on some 'horizon'. Here horizon means some time steps into the future, and so a finite horizon is simply a finite amount of time steps into the future. For example, as we are typically concerned with maximising returns in RL, a 10-step look ahead policy would choose an action at time <span class="math-container">$t$</span> that maximises the (expected) rewards at time <span class="math-container">$t+1, ... t+10$</span>. </p>

CC BY-SA 4.0
2020-07-28T19:18:34.477Z
Arent all policies finite horizon lookahead policies as per bellman equations where the discount factor determine how far to look ahead?
gfdsal
CC BY-SA 4.0
2020-07-28T19:42:33.743Z
No, for instance you can get some bandit algorithms that assume an infinite horizon. Gittins Index is an example.
David
CC BY-SA 4.0
2020-05-28T11:47:03.580Z
2020-05-28T11:47:03.580Z
David
<p>I was reading the paper <a href="https://www.aaai.org/ojs/index.php/AAAI/article/view/4227" rel="nofollow noreferrer">How to Combine Tree-Search Methods in Reinforcement Learning</a> published in AAAI Conference 2019. It starts with the sentence </p>

<blockquote>
  <p>Finite-horizon lookahead policies are abundantly used in Reinforcement Learning and demonstrate impressive empirical success.</p>
</blockquote>

<p>What is meant by "finite horizon look-ahead"?</p>

<reinforcement-learning><terminology><papers>
What are finite horizon look-ahead policies in reinforcement learning?
Joel Joseph
# What are finite horizon look-ahead policies in reinforcement learning?

(Asked by: username_0 on 2020-05-28)

I was reading the paper [How to Combine Tree-Search Methods in Reinforcement Learning](https://www.aaai.org/ojs/index.php/AAAI/article/view/4227) published in AAAI Conference 2019. It starts with the sentence

> Finite-horizon lookahead policies are abundantly used in Reinforcement Learning and demonstrate impressive empirical success.

What is meant by "finite horizon look-ahead"?

**Tags:** reinforcement-learning, terminology, papers


---

## 1 answers

### Answer by username_1 on 2020-05-28. Score: 1 | Accepted answer

Per [this paper](http://adp.princeton.edu/Papers/Powell_ADP_2ndEdition_Chapter%206.pdf) a look ahead policy is a policy that will make decisions based on some 'horizon'. Here horizon means some time steps into the future, and so a finite horizon is simply a finite amount of time steps into the future. For example, as we are typically concerned with maximising returns in RL, a 10-step look ahead policy would choose an action at time $t$ that maximises the (expected) rewards at time $t+1, ... t+10$.

> **Comments:**
> **username_2**: Arent all policies finite horizon lookahead policies as per bellman equations where the discount factor determine how far to look ahead?  
> **username_1**: No, for instance you can get some bandit algorithms that assume an infinite horizon. Gittins Index is an example.  

---

