You are currently only calculating the gradients using the last batch via single_loss.
Is this your use case or would you like to call single_loss.backward() and optimizer.step() inside the data loop?
If that’s the case, you might need to call detach() on the states to avoid trying to backpropagate multiple times through the same variables.