I’ll try ShareRMSProp in the near future!
However, I think continuous a3c is a little unstable (you can refer to the learning curve here 88).
The problem might be the insufficient threads, which causes the async update fails (unable to reduce the correlation btn data)