Body: <p>Problems occur when we combine Q-learning with a function approximator.</p>
  <p>What exactly is the delusional-bias and non-delusional Q-learning? I am talking about the neurIPS 18 best paper <a href="https://papers.nips.cc/paper/2018/hash/5fd0245f6c9ddbdf3eff0f505975b6a7-Abstract.html" rel="nofollow noreferrer">Non-delusional Q-learning and value-iteration</a>.</p>
  <p>I have trouble understand the term &quot;Policy Commitments&quot;, and &quot;consistency&quot;. What are they talking about?</p>
  <p>PS: a related <a href="https://ai.stackexchange.com/questions/11679/why-doesnt-q-learning-converge-when-using-function-approximation?newreg=948de601e746433ca62e5f030a2bcba2">post</a></p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-10-16T10:53:40.997Z
Comments.Text: Unfortunately I don't think I'll have time to write up a comprehensive answer very soon. A video of the authors' own presentation of their paper at the conference is freely available here though: https://www.youtube.com/watch?v=fwSEyyEJmkc 
  
  Hopefully that can already help you, or anyone else who may want to write up a detailed answer
Comments.UserDisplayName: Dennis Soemers
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-10-16T21:10:45.443Z
Comments.Text: Non-delusional Q learning is a type of machine learning that can be used to learn how to optimally make decisions in environments where there is some uncertainty. It is related to reinforcement learning, but differs in that it does not require a model of the environment and can be used in environments where the reward function is unknown.
Comments.UserDisplayName: Faizy
Tags: <reinforcement-learning><q-learning><papers><value-iteration><policy-iteration>
Title: What exactly is non-delusional Q-learning?
UserDisplayName: wrek
ThreadText: # What exactly is non-delusional Q-learning?
  
  (Asked by: username_0 on 2022-10-15)
  
  Problems occur when we combine Q-learning with a function approximator.
  
  What exactly is the delusional-bias and non-delusional Q-learning? I am talking about the neurIPS 18 best paper [Non-delusional Q-learning and value-iteration](https://papers.nips.cc/paper/2018/hash/5fd0245f6c9ddbdf3eff0f505975b6a7-Abstract.html).
  
  I have trouble understand the term "Policy Commitments", and "consistency". What are they talking about?
  
  PS: a related [post](https://ai.stackexchange.com/questions/11679/why-doesnt-q-learning-converge-when-using-function-approximation?newreg=948de601e746433ca62e5f030a2bcba2)
  
  **Tags:** reinforcement-learning, q-learning, papers, value-iteration, policy-iteration
  > **Comments:**
  > **username_1**: Unfortunately I don't think I'll have time to write up a comprehensive answer very soon. A video of the authors' own presentation of their paper at the conference is freely available here though: https://www.youtube.com/watch?v=fwSEyyEJmkc   Hopefully that can already help you, or anyone else who may want to write up a detailed answer  
  > **username_2**: Non-delusional Q learning is a type of machine learning that can be used to learn how to optimally make decisions in environments where there is some uncertainty. It is related to reinforcement learning, but differs in that it does not require a model of the environment and can be used in environments where the reward function is unknown.  
  
  ---
  
  ## 0 answers
  
  