Answers.Body: <p>You can check out <a href="https://arxiv.org/abs/1602.04621" rel="nofollow noreferrer">Bootstrapped DQN</a>, with a demonstration <a href="https://www.youtube.com/watch?v=e3KuV_d0EMk" rel="nofollow noreferrer">video</a>. Without reading much of the paper, it seems the authors use a different sampling strategy and an action-guide for specific instances.</p>
  
  <p>Another way to initially set weights for the network is to create a dataset of moves (correct, incorrect, etc, as long as they are relevant) and have the network initially learn the dataset. This also helps debugging, as you can see whether the network can actually learn the policy used in the dataset. After learning the dataset, use the same learned network with DQN and start with a smaller exploration rate (like 0.5 instead of 1.0).</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-05-31T08:41:03.460Z
Answers.LastActivityDate: 2017-05-31T08:41:03.460Z
Answers.UserDisplayName: BlueMoon93
Body: <p>I am interested in the current state-of-the-art ways to use quick, greedy heuristics in order to speed up the learning in a Deep Q-Network in Reinforcement Learning. In classical RL, I initially set the Q-value for a state-action pair (S,a) based on the result of such a greedy heuristic run from state S with action a. Is this still a good idea in the setting of a neural network for the approximation of the Q-function, and if yes, what are the optimal ways of doing it? What are other ways of aiding the DQN with the knowledge from the greedy heuristics?</p>
  
  <p>References to state-of-the-art papers would be highly appreciated.</p>
  
Tags: <reinforcement-learning><training><intelligent-agent><heuristics>
Title: What are state-of-the-art ways of using greedy heuristics to initially set the weights of a Deep Q-Network in Reinforcement Learning?
UserDisplayName: AlexGuevara
ThreadText: # What are state-of-the-art ways of using greedy heuristics to initially set the weights of a Deep Q-Network in Reinforcement Learning?
  
  (Asked by: username_0 on 2017-05-31)
  
  I am interested in the current state-of-the-art ways to use quick, greedy heuristics in order to speed up the learning in a Deep Q-Network in Reinforcement Learning. In classical RL, I initially set the Q-value for a state-action pair (S,a) based on the result of such a greedy heuristic run from state S with action a. Is this still a good idea in the setting of a neural network for the approximation of the Q-function, and if yes, what are the optimal ways of doing it? What are other ways of aiding the DQN with the knowledge from the greedy heuristics?
  
  References to state-of-the-art papers would be highly appreciated.
  
  **Tags:** reinforcement-learning, training, intelligent-agent, heuristics
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2017-05-31. Score: 2
  
  You can check out [Bootstrapped DQN](https://arxiv.org/abs/1602.04621), with a demonstration [video](https://www.youtube.com/watch?v=e3KuV_d0EMk). Without reading much of the paper, it seems the authors use a different sampling strategy and an action-guide for specific instances.
  
  Another way to initially set weights for the network is to create a dataset of moves (correct, incorrect, etc, as long as they are relevant) and have the network initially learn the dataset. This also helps debugging, as you can see whether the network can actually learn the policy used in the dataset. After learning the dataset, use the same learned network with DQN and start with a smaller exploration rate (like 0.5 instead of 1.0).
  
  
  ---
  
  