Where is the dataset stored currently?
If it’s stored on a network drive, you could load the data lazily from there, which will most likely slow down your overall training due to the latency.
Otherwise, you would somehow have to swap the old data for the new one (in each epoch), which might also be too slow.
Do you have an external SSD (with USB 3) which might be big enough?