Answers.Body: <p>Once you have estimated the <span class="math-container">$Q$</span> function, you can derive the policy from it in different ways. For example, you can act greedily with respect to it (see <a href="https://ai.stackexchange.com/a/13363/2444">this answer</a>), which can be formally denoted as</p>
  
  <p><span class="math-container">$$
  \pi(s) = \operatorname{argmax}_{a^*}Q(s, a), \; \forall s \in \mathcal{S}
  $$</span>
  where <span class="math-container">$Q(s, a)$</span> is your estimated value function and <span class="math-container">$\pi$</span> the policy greedily derived from it.</p>
  
  <p>This means that you would just exploit your current knowledge of the return. This is probably a good thing to do if you believe your value function is optimal and the dynamics of the environment don't change.</p>
  
  <p>Of course, if your policy is not optimal, you may not want to always execute the greedy action. In that case, you could still perform some form of exploration (e.g. with the <a href="https://ai.stackexchange.com/a/10495/2444"><span class="math-container">$\epsilon$</span>-greedy policy</a>).</p>
  
  <p>Moreover, if the dynamics (e.g. the reward function) of your environment change over time, you could continually train your RL agent. If you are interested in continual RL, the paper <a href="https://arxiv.org/pdf/1802.07239.pdf" rel="nofollow noreferrer">Continual Reinforcement Learning with Complex Synapses</a> (2018) could potentially be useful.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-05-01T23:35:54.573Z
Answers.LastActivityDate: 2020-05-01T23:42:21.007Z
Answers.LastEditDate: 2020-05-01T23:42:21.007Z
Answers.UserDisplayName: nbro
Body: <p>When using a trained Q-learning algorithm in an actual game, would I just use exploitation and no longer use exploration? Should I use exploration only during the training phase?</p>
  
Tags: <reinforcement-learning><training><q-learning><exploration-exploitation-tradeoff>
Title: Should I just use exploitation after I have trained the Q agent?
UserDisplayName: mason7663
ThreadText: # Should I just use exploitation after I have trained the Q agent?
  
  (Asked by: username_0 on 2020-05-01)
  
  When using a trained Q-learning algorithm in an actual game, would I just use exploitation and no longer use exploration? Should I use exploration only during the training phase?
  
  **Tags:** reinforcement-learning, training, q-learning, exploration-exploitation-tradeoff
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-05-01. Score: 1
  
  Once you have estimated the $Q$ function, you can derive the policy from it in different ways. For example, you can act greedily with respect to it (see [this answer](https://ai.stackexchange.com/a/13363/2444)), which can be formally denoted as
  
  $$
  \pi(s) = \operatorname{argmax}\_{a^\*}Q(s, a), \; \forall s \in \mathcal{S}
  $$
  where $Q(s, a)$ is your estimated value function and $\pi$ the policy greedily derived from it.
  
  This means that you would just exploit your current knowledge of the return. This is probably a good thing to do if you believe your value function is optimal and the dynamics of the environment don't change.
  
  Of course, if your policy is not optimal, you may not want to always execute the greedy action. In that case, you could still perform some form of exploration (e.g. with the [$\epsilon$-greedy policy](https://ai.stackexchange.com/a/10495/2444)).
  
  Moreover, if the dynamics (e.g. the reward function) of your environment change over time, you could continually train your RL agent. If you are interested in continual RL, the paper [Continual Reinforcement Learning with Complex Synapses](https://arxiv.org/pdf/1802.07239.pdf) (2018) could potentially be useful.
  
  
  ---
  
  