My training code is like the following:
for epoch in range(n_epoch):
for i, batch in enumerate(dataset):
train()
dataset is created using Dataloader and I have only one data file. I saw that Dataloader loads everything in this file into memory and then extracts batches. I want to train this data file for multiple epochs. But I noticed that every time a new epoch begins, the data file will be loaded from disk again and loading this file takes a lot of time. How to load this file only once for the first epoch and reuse the dataset without fetching from disk repeatedly for the following epochs? Thanks a lot!