This post 4 explains a simple windowed Dataset, where non-overlapping windows would be created, while this post 6 is a bit more complicated in order to avoid “mixing” data samples from different sources.
Maybe one of these two approaches could be useful for you.