I’ve been confused by this exact example myself - because init_hidden is on forward, it means that not only during training is the initial state (per batch) random, but also during validation and testing?
It seems to me that it’s something you should call in the training loop (per batch or per epoch), but then I’m not sure what initial state you’d use for inference. You probably want to use the final state from the previous batch if you’re predicting from a windowed time-series?