Hi @AjayTalati, thanks for the links! I’ll try to have a look at these as another point of reference when I can.
My ACER implementation is open sourced as part of an entry to the Malmo Challenge 5, but we never attempted to use the trust region updates for that. I also have a standalone implementation (for the Gym’s CartPole instead of the Malmo Challenge) which I will open source in the next week or so. My ACER implementation without trust region updates can get good scores, but is quite unstable. Haven’t managed to spend time debugging the trust region updates yet, but @ethancaballero has had a look too. I’m looking forward to opening it up to the PyTorch community!