Answers.Body: <p>No, mainly because these are all stochastic approximations and may not represent the true values.</p>
  
  <p>Almost nothing good can be said about NN approximations to value and Q functions(at least according to a professor I have had). </p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-05-14T06:57:13.973Z
Answers.Text: I believe the first sentence is correct. However, I also think the answer needs to have a few more details. If nothing else, worth clarifying "nothing good can be said" - it could be read as it is bad choice of solution, but actually many novel results have been produced using deep RL that are not possible with any other technique. So perhaps you are referring to the lack of solid theory, such as a practical loss function that is minimised when agent performance is maximised.
Answers.UserDisplayName: Neil Slater
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-05-14T05:59:24.567Z
Answers.LastActivityDate: 2020-05-14T05:59:24.567Z
Answers.UserDisplayName: FourierFlux
Body: <p>I have retrospective data for a sort of "behaviour policy" which I will use to train a deep q network to learn a target greedy policy. After learning the Q values for this target policy, can we make the conclusion that because the Q value for the target policy, <span class="math-container">$Q(s,\pi_e(s))$</span> is higher than the Q values for the behaviour policy, <span class="math-container">$Q(s,\pi_b(s))$</span> at all states encountered, where <span class="math-container">$\pi_e$</span> is the policy output by deep Q-learning and <span class="math-container">$\pi_b$</span> is the behaviour policy, then this target policy has better performance than the behaviour policy?</p>
  
  <p>I know the proper way is to run the policy and do an empirical comparison of some sort. However, that is not possible in my case. </p>
  
Tags: <reinforcement-learning><q-learning><dqn><deep-rl><off-policy-methods>
Title: Is it possible to prove that the target policy is better than the behavioural policy based on learned Q values?
UserDisplayName: calveeen
ThreadText: # Is it possible to prove that the target policy is better than the behavioural policy based on learned Q values?
  
  (Asked by: username_0 on 2020-05-14)
  
  I have retrospective data for a sort of "behaviour policy" which I will use to train a deep q network to learn a target greedy policy. After learning the Q values for this target policy, can we make the conclusion that because the Q value for the target policy, $Q(s,\pi\_e(s))$ is higher than the Q values for the behaviour policy, $Q(s,\pi\_b(s))$ at all states encountered, where $\pi\_e$ is the policy output by deep Q-learning and $\pi\_b$ is the behaviour policy, then this target policy has better performance than the behaviour policy?
  
  I know the proper way is to run the policy and do an empirical comparison of some sort. However, that is not possible in my case.
  
  **Tags:** reinforcement-learning, q-learning, dqn, deep-rl, off-policy-methods
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-05-14. Score: -1
  
  No, mainly because these are all stochastic approximations and may not represent the true values.
  
  Almost nothing good can be said about NN approximations to value and Q functions(at least according to a professor I have had).
  
  > **Comments:**
  > **username_2**: I believe the first sentence is correct. However, I also think the answer needs to have a few more details. If nothing else, worth clarifying "nothing good can be said" - it could be read as it is bad choice of solution, but actually many novel results have been produced using deep RL that are not possible with any other technique. So perhaps you are referring to the lack of solid theory, such as a practical loss function that is minimised when agent performance is maximised.  
  
  ---
  
  