Thanks for the reply!
Re your first point: so in my case (assuming all my data is in RAM), I could create a Dataset object that represents the entire dataset, as well as a DataLoader to generate batches from the dataset. I could set it up so that each batch returned by the DataLoader is a 3D tensor with dimensions (batch_size, n_samples, n_time_series) and each [i, :, :] slice of the batch is a view of a slice [j:j+n_samples, :] of my original 2D array, so data in the original array does not need to be copied to create the batch, and I can pass the batch through my model function as if it were a normal tensor. Is that accurate?
Re your second point, I think in my case disk read speed is more likely to be the bottleneck than the number of CPUs. I guess for e.g. JPEG images that need to be uncompressed, the bottleneck is compute rather than I/O, so more processes would help, but in some cases multiple processes simultaneously trying to load e.g. multiple .npy arrays from a hard disk might actually hurt performance, right?