Answers.Body: <p>The exploration rate, typically parameterized as epsilon / ε, can be changed on every trial. It depends on the complexity of the model and the goals.</p>
  
  <p>The simplest thing to do is keep exploration rate high and fixed. That means the model will continue to explore new options, even at the cost of not "exploiting" the best available option. </p>
  
  <p>Another option is setting the exploration rate high at the beginning of learning so the model will search the space for possible successful solutions. Then as the model creates a set of policies that are successful for given states, the exploration rate can be lowered or decay. Exploration rate decay can be fixed (i.e., over time there is consistently less exploration and more exploitation). Exploration rate decay can be dynamic and learned. This last option is often the best but also the most complex to implement.</p>
  
  <p>"<a href="http://theses.ubn.ru.nl/bitstream/handle/123456789/5216/Nieuwdorp%2C_T._1.pdf?sequence=1" rel="nofollow noreferrer">Dare to Discover: The Effect of the Exploration Strategy on an Agent’s Performance</a>" goes into greater detail on this topic.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-05-13T20:54:02.137Z
Answers.Text: Welcome to AI!  Thanks for contributing!!! *(Hope to see more of you on this Stack.  There's an "ai-basics" tag if you want to scoop up some quick rep;)*
Answers.UserDisplayName: DukeZhou
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-05-13T16:06:04.387Z
Answers.LastActivityDate: 2018-05-14T02:17:02.667Z
Answers.LastEditDate: 2018-05-14T02:17:02.667Z
Answers.UserDisplayName: Brian Spiering
Body: <p>As the title says, should I reset the exploration rate between trials?</p>
  
  <p>I am currently doing the Open AI pendulum task and after a number of trials my model started playing but did not take any actions (i.e. didn't perform any significant swing). The Actor-Critic tutorial I followed did not reset the exploration rate (<a href="https://towardsdatascience.com/reinforcement-learning-w-keras-openai-actor-critic-models-f084612cfd69" rel="nofollow noreferrer">link</a>) but it seems like there are lots of mistakes in general.</p>
  
  <p>I assume that it should be reset since the model might start from a new unknown situation in a different trial and not know what to do without exploring. </p>
  
Tags: <reinforcement-learning><q-learning><open-ai>
Title: Should the exploration rate be reset after each trial in Q-learning?
UserDisplayName: MattSt
ThreadText: # Should the exploration rate be reset after each trial in Q-learning?
  
  (Asked by: username_0 on 2018-05-12)
  
  As the title says, should I reset the exploration rate between trials?
  
  I am currently doing the Open AI pendulum task and after a number of trials my model started playing but did not take any actions (i.e. didn't perform any significant swing). The Actor-Critic tutorial I followed did not reset the exploration rate ([link](https://towardsdatascience.com/reinforcement-learning-w-keras-openai-actor-critic-models-f084612cfd69)) but it seems like there are lots of mistakes in general.
  
  I assume that it should be reset since the model might start from a new unknown situation in a different trial and not know what to do without exploring.
  
  **Tags:** reinforcement-learning, q-learning, open-ai
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2018-05-13. Score: 5 | Accepted answer
  
  The exploration rate, typically parameterized as epsilon / ε, can be changed on every trial. It depends on the complexity of the model and the goals.
  
  The simplest thing to do is keep exploration rate high and fixed. That means the model will continue to explore new options, even at the cost of not "exploiting" the best available option.
  
  Another option is setting the exploration rate high at the beginning of learning so the model will search the space for possible successful solutions. Then as the model creates a set of policies that are successful for given states, the exploration rate can be lowered or decay. Exploration rate decay can be fixed (i.e., over time there is consistently less exploration and more exploitation). Exploration rate decay can be dynamic and learned. This last option is often the best but also the most complex to implement.
  
  "[Dare to Discover: The Effect of the Exploration Strategy on an Agent’s Performance](http://theses.ubn.ru.nl/bitstream/handle/123456789/5216/Nieuwdorp%2C_T._1.pdf?sequence=1)" goes into greater detail on this topic.
  
  > **Comments:**
  > **username_2**: Welcome to AI!  Thanks for contributing!!! *(Hope to see more of you on this Stack.  There's an "ai-basics" tag if you want to scoop up some quick rep;)*  
  
  ---
  
  