I don’t think you need to do anything special for this scenario as usually the amount of memory required is some small multiple of the batch_size (due to prefetching). For example ImageNet spans hundreds of gigabytes (even more when converted to uncompressed RGB format) yet only a small fraction of this (host and device) memory is required for training.