Answers.Body: <p>Here are two very related interesting papers:</p>
  
  <ol>
  <li><a href="https://openai.com/blog/deep-reinforcement-learning-from-human-preferences/" rel="nofollow noreferrer">Learning from Human Preferences</a></li>
  <li><a href="https://www.ijcai.org/Proceedings/2018/0817.pdf" rel="nofollow noreferrer">Improving Reinforcement Learning with Human Input</a></li>
  </ol>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-04-18T14:34:59.583Z
Answers.LastActivityDate: 2020-04-18T14:34:59.583Z
Answers.UserDisplayName: Ray Walker
Answers.Body: <p>The programmer already guides the RL algorithm (or agent) by specifying the reward function. However, the reward function alone may not be sufficient to learn efficiently and fast, as you correctly noticed.</p>
  
  <p>To attempt to solve this <em>inefficiency problem</em>, one solution is to combine reinforcement learning with supervised learning. For example, the paper <a href="https://arxiv.org/pdf/1704.03732.pdf" rel="nofollow noreferrer">Deep Q-learning from Demonstrations</a> (2017) by Todd Hester et al. describes an approach to achieve this. </p>
  
  <p>The paper <a href="https://cs.stanford.edu/people/acvogel/papers/290.pdf" rel="nofollow noreferrer">Active Reinforcement Learning</a> (2008) by Arkady Epshteyn et al. also tries to solve this problem but by incorporating approximations (given by domain experts) of the MDP. </p>
  
  <p>There are probably many other possible solutions. In fact, all model-based RL algorithms could probably fall into this category of algorithms that estimate or incorporate the dynamics of the environment to find a policy more efficiently.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-04-18T13:10:50.283Z
Answers.LastActivityDate: 2020-04-18T13:15:57.677Z
Answers.LastEditDate: 2020-04-18T13:15:57.677Z
Answers.UserDisplayName: nbro
Body: <p>I have just started to study reinforcement learning and, as far as I understand, existing algorithms search for the optimal solution/policy, but do not allow the possibility for the programmer to suggest a way to find the solution (to guide their learning process). This would be beneficial for finding the optimal solution faster.</p>
  
  <p>Is it possible to guide the learning process in (deep) reinforcement learning?</p>
  
Tags: <reinforcement-learning><deep-rl><supervised-learning><efficiency><active-learning>
Title: Is it possible to guide a reinforcement learning algorithm?
UserDisplayName: Cristian M
ThreadText: # Is it possible to guide a reinforcement learning algorithm?
  
  (Asked by: username_0 on 2020-04-18)
  
  I have just started to study reinforcement learning and, as far as I understand, existing algorithms search for the optimal solution/policy, but do not allow the possibility for the programmer to suggest a way to find the solution (to guide their learning process). This would be beneficial for finding the optimal solution faster.
  
  Is it possible to guide the learning process in (deep) reinforcement learning?
  
  **Tags:** reinforcement-learning, deep-rl, supervised-learning, efficiency, active-learning
  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2020-04-18. Score: 3 | Accepted answer
  
  The programmer already guides the RL algorithm (or agent) by specifying the reward function. However, the reward function alone may not be sufficient to learn efficiently and fast, as you correctly noticed.
  
  To attempt to solve this *inefficiency problem*, one solution is to combine reinforcement learning with supervised learning. For example, the paper [Deep Q-learning from Demonstrations](https://arxiv.org/pdf/1704.03732.pdf) (2017) by Todd Hester et al. describes an approach to achieve this.
  
  The paper [Active Reinforcement Learning](https://cs.stanford.edu/people/acvogel/papers/290.pdf) (2008) by Arkady Epshteyn et al. also tries to solve this problem but by incorporating approximations (given by domain experts) of the MDP.
  
  There are probably many other possible solutions. In fact, all model-based RL algorithms could probably fall into this category of algorithms that estimate or incorporate the dynamics of the environment to find a policy more efficiently.
  
  
  ---
  
  ### Answer by username_1 on 2020-04-18. Score: 2
  
  Here are two very related interesting papers:
  
  1. [Learning from Human Preferences](https://openai.com/blog/deep-reinforcement-learning-from-human-preferences/)
  2. [Improving Reinforcement Learning with Human Input](https://www.ijcai.org/Proceedings/2018/0817.pdf)
  
  
  ---
  
  