Body: <p>I recently tried to reproduce the results of <a href="https://papers.nips.cc/paper/2010/file/091d584fced301b442654dd8c23b3fc9-Paper.pdf" rel="nofollow noreferrer">double Q-learning</a>. However, the results are not satisfying. I have also tried to compare double Q learning with Q-learning in Taxi-v3, FrozenLake without slippery, Roulette-v0, etc. But Q-learning outperforms double Q-learning in all of these environments.</p>
  <p>I am not sure whether if there is something wrong with my implementation as many materials about double Q actually focus on double DQN. While at the same time of checking, I wonder is there any toy example that can exemplify the performance of double Q-learning?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-12-23T09:31:02.450Z
Comments.Text: Have you used the same parameters and hyper-parameters as the ones in [the paper](https://papers.nips.cc/paper/2010/file/091d584fced301b442654dd8c23b3fc9-Paper.pdf) for the roulette environment?
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-12-23T11:38:26.067Z
Comments.Text: @nbro Thanks for your comment. I haven't tested that in the paper's roulette as I am not very understand the environment settings. Consider giving it a try another day.
Comments.UserDisplayName: Allen_FrCh
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-12-26T23:05:43.527Z
Comments.Text: have you seen the example from the Sutton and Barto book?
Comments.UserDisplayName: David
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-12-27T07:11:43.613Z
Comments.Text: @DavidIreland Haven't yet. Thanks for suggestion.
Comments.UserDisplayName: Allen_FrCh
Tags: <reinforcement-learning><q-learning><double-q-learning>
Title: Is there any toy example that can exemplify the performance of double Q-learning?
UserDisplayName: Allen_FrCh
ThreadText: # Is there any toy example that can exemplify the performance of double Q-learning?
  
  (Asked by: username_0 on 2020-12-23)
  
  I recently tried to reproduce the results of [double Q-learning](https://papers.nips.cc/paper/2010/file/091d584fced301b442654dd8c23b3fc9-Paper.pdf). However, the results are not satisfying. I have also tried to compare double Q learning with Q-learning in Taxi-v3, FrozenLake without slippery, Roulette-v0, etc. But Q-learning outperforms double Q-learning in all of these environments.
  
  I am not sure whether if there is something wrong with my implementation as many materials about double Q actually focus on double DQN. While at the same time of checking, I wonder is there any toy example that can exemplify the performance of double Q-learning?
  
  **Tags:** reinforcement-learning, q-learning, double-q-learning
  > **Comments:**
  > **username_1**: Have you used the same parameters and hyper-parameters as the ones in [the paper](https://papers.nips.cc/paper/2010/file/091d584fced301b442654dd8c23b3fc9-Paper.pdf) for the roulette environment?  
  > **username_0**: @nbro Thanks for your comment. I haven't tested that in the paper's roulette as I am not very understand the environment settings. Consider giving it a try another day.  
  > **username_2**: have you seen the example from the Sutton and Barto book?  
  > **username_0**: @DavidIreland Haven't yet. Thanks for suggestion.  
  
  ---
  
  ## 0 answers
  
  