Re first point:
If you create a Dataset that returns one sample for each index. Then the Dataloader will ask for each element one by one and will then concatenate them. So copies will occur.
You can go around this by making the Dataset return a whole batch every time one sample is required. And make the Dataloader load samples one by one (but one sample from the dataset is actually a batch).
This is a hack around the existing elements for you to avoid copies.
Re second point:
Yes if you don’t do any preprocessing then it might be IO bound. unfortunately, there is not much you can do here but get a faster disk