Feel free to check out my code: https://github.com/MillionIntegrals/vel 80
I’ve tried to make it fast, accurate and readable 
If you have any questions, feel free to ask, open an issue etc.
Not all the methods are implemented yet, but I’m aiming for feature parity with openai baselines, if I have some time I’ll finish implementing ACER this weekend.