Thanks for pointing out! But I still can’t make it converge. In my case it just diverges very slowly (in case that my hyperparameters are problematic, I use the original example).
Can you let us know what hyperparameters and other tricks, like fixed Q target, you are using?