Thank you!
I think I’m on the right track now. I was confused about what to iterate over and if it needed to be re-initialized.
I just am stuck on getting the data loader portion now. I figured it would be pretty easy, but I’m not sure how to create the file load_file function.
My files are stored in a file data where the subsequent files are as follows
/data/images/ images to use
/data/targets.txt
Is this the correct format for a loader to work, or do I need to have each batch be a new set of folders?
data_loc = '.../data/'

def load_file(file):
    
class MyDataset(Data.Dataset):
    def __init__(self, data_files):
        self.data_files = sorted(data_files)

    def __getitem__(self, index):
        return load_file(self.data_files[index])

    def __len__(self):
        return len(self.data_files)
set_test = MyDataset(data_loc)
loader = Data.DataLoader(set_test,batch_size = BATCH_SIZE, num_workers=8)```