tryingtolearnpytorch:

I don’t like how the hidden state is reset to zero for each training batch, as surely that is getting rid of important information about the previous state of the system?


No, that state encodes “present & future” of timeseries in that batch, state transition rules are encoded in rnn module’s parameters.
New batch usually implies a new bunch of timeseries (or at least non-consecutive [random] input slices), thus old momentary states are not applicable. Instead, it is possible, for some tasks, to use static per timeseries features to initialize hidden states (via encoder MLP).