<p>I recently tried to reproduce the results of <a href="https://papers.nips.cc/paper/2010/file/091d584fced301b442654dd8c23b3fc9-Paper.pdf" rel="nofollow noreferrer">double Q-learning</a>. However, the results are not satisfying. I have also tried to compare double Q learning with Q-learning in Taxi-v3, FrozenLake without slippery, Roulette-v0, etc. But Q-learning outperforms double Q-learning in all of these environments.</p>
<p>I am not sure whether if there is something wrong with my implementation as many materials about double Q actually focus on double DQN. While at the same time of checking, I wonder is there any toy example that can exemplify the performance of double Q-learning?</p>

CC BY-SA 4.0
2020-12-23T09:31:02.450Z
Have you used the same parameters and hyper-parameters as the ones in [the paper](https://papers.nips.cc/paper/2010/file/091d584fced301b442654dd8c23b3fc9-Paper.pdf) for the roulette environment?
nbro
CC BY-SA 4.0
2020-12-23T11:38:26.067Z
@nbro Thanks for your comment. I haven't tested that in the paper's roulette as I am not very understand the environment settings. Consider giving it a try another day.
Allen_FrCh
CC BY-SA 4.0
2020-12-26T23:05:43.527Z
have you seen the example from the Sutton and Barto book?
David
CC BY-SA 4.0
2020-12-27T07:11:43.613Z
@DavidIreland Haven't yet. Thanks for suggestion.
Allen_FrCh
<reinforcement-learning><q-learning><double-q-learning>
Is there any toy example that can exemplify the performance of double Q-learning?
Allen_FrCh
# Is there any toy example that can exemplify the performance of double Q-learning?

(Asked by: username_0 on 2020-12-23)

I recently tried to reproduce the results of [double Q-learning](https://papers.nips.cc/paper/2010/file/091d584fced301b442654dd8c23b3fc9-Paper.pdf). However, the results are not satisfying. I have also tried to compare double Q learning with Q-learning in Taxi-v3, FrozenLake without slippery, Roulette-v0, etc. But Q-learning outperforms double Q-learning in all of these environments.

I am not sure whether if there is something wrong with my implementation as many materials about double Q actually focus on double DQN. While at the same time of checking, I wonder is there any toy example that can exemplify the performance of double Q-learning?

**Tags:** reinforcement-learning, q-learning, double-q-learning

> **Comments:**
> **username_1**: Have you used the same parameters and hyper-parameters as the ones in [the paper](https://papers.nips.cc/paper/2010/file/091d584fced301b442654dd8c23b3fc9-Paper.pdf) for the roulette environment?  
> **username_0**: @username_1 Thanks for your comment. I haven't tested that in the paper's roulette as I am not very understand the environment settings. Consider giving it a try another day.  
> **username_2**: have you seen the example from the Sutton and Barto book?  
> **username_0**: @username_2Ireland Haven't yet. Thanks for suggestion.  

---

## 0 answers

