Answers.Body: <p>Per <a href="http://adp.princeton.edu/Papers/Powell_ADP_2ndEdition_Chapter%206.pdf" rel="nofollow noreferrer">this paper</a> a look ahead policy is a policy that will make decisions based on some 'horizon'. Here horizon means some time steps into the future, and so a finite horizon is simply a finite amount of time steps into the future. For example, as we are typically concerned with maximising returns in RL, a 10-step look ahead policy would choose an action at time <span class="math-container">$t$</span> that maximises the (expected) rewards at time <span class="math-container">$t+1, ... t+10$</span>. </p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-07-28T19:18:34.477Z
Answers.Text: Arent all policies finite horizon lookahead policies as per bellman equations where the discount factor determine how far to look ahead?
Answers.UserDisplayName: gfdsal
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-07-28T19:42:33.743Z
Answers.Text: No, for instance you can get some bandit algorithms that assume an infinite horizon. Gittins Index is an example.
Answers.UserDisplayName: David
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-05-28T11:47:03.580Z
Answers.LastActivityDate: 2020-05-28T11:47:03.580Z
Answers.UserDisplayName: David
Body: <p>I was reading the paper <a href="https://www.aaai.org/ojs/index.php/AAAI/article/view/4227" rel="nofollow noreferrer">How to Combine Tree-Search Methods in Reinforcement Learning</a> published in AAAI Conference 2019. It starts with the sentence </p>
  
  <blockquote>
    <p>Finite-horizon lookahead policies are abundantly used in Reinforcement Learning and demonstrate impressive empirical success.</p>
  </blockquote>
  
  <p>What is meant by "finite horizon look-ahead"?</p>
  
Tags: <reinforcement-learning><terminology><papers>
Title: What are finite horizon look-ahead policies in reinforcement learning?
UserDisplayName: Joel Joseph
ThreadText: # What are finite horizon look-ahead policies in reinforcement learning?
  
  (Asked by: username_0 on 2020-05-28)
  
  I was reading the paper [How to Combine Tree-Search Methods in Reinforcement Learning](https://www.aaai.org/ojs/index.php/AAAI/article/view/4227) published in AAAI Conference 2019. It starts with the sentence
  
  > Finite-horizon lookahead policies are abundantly used in Reinforcement Learning and demonstrate impressive empirical success.
  
  What is meant by "finite horizon look-ahead"?
  
  **Tags:** reinforcement-learning, terminology, papers
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-05-28. Score: 1 | Accepted answer
  
  Per [this paper](http://adp.princeton.edu/Papers/Powell_ADP_2ndEdition_Chapter%206.pdf) a look ahead policy is a policy that will make decisions based on some 'horizon'. Here horizon means some time steps into the future, and so a finite horizon is simply a finite amount of time steps into the future. For example, as we are typically concerned with maximising returns in RL, a 10-step look ahead policy would choose an action at time $t$ that maximises the (expected) rewards at time $t+1, ... t+10$.
  
  > **Comments:**
  > **username_2**: Arent all policies finite horizon lookahead policies as per bellman equations where the discount factor determine how far to look ahead?  
  > **username_1**: No, for instance you can get some bandit algorithms that assume an infinite horizon. Gittins Index is an example.  
  
  ---
  
  