Well, for one with this line
last_hidden_cell_out = Variable(last_hidden_cell_out[0].data)

you are detaching the computational graph for everything that happened to compute last_hidden_cell_out. Therefore, the parameters for lstm1 and lstm2 never get updated.
What was the point for this, is this intentional?