PyTorch handles the backprop for you. Your computation graph is build dynamically and as a result in the graph the LSTM is repeated 10 times and you would automatically backprop through all the 10 layers. Sorry man, I don’t have much knowledge about reinforcement learning.