Solved by ptrblck in post #5 
              
              
                Instead of resetting the states, try to detach them via: 
model.lstm._hidden_state = model.lstm._hidden_state.detach()
model.lstm._hidden_cell = model.lstm._hidden_cell.detach()

to keep the state values, but to detach them from the previous calculations.