The hidden layer is initialized in the constructor of the net. I dont want to zero it out before every timestep as I want to keep temporal information in the hidden state of the lstm. The difference is that in that tutorial it seems like they pass in the whole sequence, and not one step at a time and therefore have to zero the hidden state before each new sequence. I am only training on a single sequence. In my code “values” is the timeseries which this time happens to be (1600,1). Resetting the hidden state at each timestep removes the error but the net no longer works as I intended.