Let me explain using an example. You have y_0, h_0 = f(x_0, 0) at time 0 and y_1, h_1 = f(x_1, h_0) at time 1. At time 0, when you call loss(y_0, real_y_0).backward(), it backtracks through the graph, including some computations for h_0. At time 1, you call loss(y_1, real_y_1).backward(), it backtracks through both x_1 and h_0, both of which are necessary to compute y_1. It is at this time that you backtrack through the graph to compute h_0 twice.
The solution is to save hidden.detach()