Hi @andrewliao11,
great stuff !!! That’s really cool, nice videos 
I never managed to get it working very well, (I tried it on non-mujoco stuff), so went back to experimenting with the discrete actions version. Do you plan on experimenting with shared RMSProp?
A3C is a great tool - you can apply it to a lot of stuff - it should be really helpful to you in the future!
Kind regards,
Ajay