Hello. When I try to use iter(dataloader) to define an iterator on my dataloader, it is very slow to define. Can anyone give some suggestions? Here is my code:
class CellDataset(Dataset):
def __init__(self, data_frame):

    self.data_frame = data_frame
    self.length = self.data_frame.shape[0]

def __len__(self):
    
    return self.length

def __getitem__(self, idx):
    
    cell = torch.tensor(self.data_frame.iloc[idx], dtype=torch.float)      
    return cell

dataset_train = CellDataset(data_frame=train)   # train is a large pandas dataframe already in memory
dataloader_train = DataLoader(dataset_train, batch_size=batchsize, shuffle=True, num_workers=48)

dataiter = iter(dataloader_train)   # this line very slow, about 30s
data = next(dataiter)   # this line is normal

My case is a little special since my data is not image files in ssd, but a large pandas dataframe in the memory (about 8GB).  Also I use large number of workers. If I reduce the number of workers, say 8, then the dataiter = iter(dataloader_train) is faster but the data = next(dataiter) is slower. When I try regular image data in ssd, then everything is normal.
Thanks in advance for any suggestion!