There is a pytorch model in the RL pytorch folder. There reason I say you want batch a3c because if you are updating parameters individually with gpu. The lock acquiring and releasing will slow it down so that it’s no faster than if done on CPU only if you do a whole bunch of updates together and get benefit of much faster matrix computation of gpu will it be beneficial at all
To do individual updates using gpu will be in most all cases slower except if your model is extremely large