Thanks. That all sounds like very reasonable advice.
I was currently training on around 5k samples of length 24 each. I guess that it’s not sufficient?
I am intrigued by your discussion on shortness of batches. I’d written an educational post on using RNNs for signal denoising in PyTorch 2. I’d noticed the effect you’re mentioning -  that the predicted time series always used to have higher errors for the initial few points.
I guess one solution could be to use rolling 24 hour windows? Not only will it increase the effective sample space, but also have the advantage that the back propagation won’t be cut off early.