Hi IIya, @Ilya_Kostrikov  I’m experimenting with your wonderful A3C implementation - I’m porting over a custom environment (basically blackbox optimization) I’ve got working in TensorFlow. I keep running int this error though,
AttributeError: ‘float’ object has no attribute ‘backward’
Any guesses what’s caused it? Apart from my custom environment, the only difference to your code is I’ve remove the conv layers, and used a 2 layer LSTM - which is the model I’m using in TensorFlow.
As a performance improvement have you tried concatenating the previous action, reward, and a timestep counter onto the end of the state as 3 scalars - I noticed a significant improvement in my TF implementation when I do this.
@apaszke - would it be possible to have a look at your A3C implementation - I’ve spent nearly a whole day trying to debug mine?
Thanks a lot for your help