I have a dataset which is not big, but the torch.utils.data.DataLoader performance is not good on my system, so, I plan to cache the result of the first dataloader, and then use the cached results.
For speed, I don’t want to do torch.tensor copy during the cache processing, I tried with queue/list, but looks that the tensor copy is always there. Could you help on this? How to cache the result with reference, instead of full data copy? thanks.
The code looks like
val_loader = torch.utils.data.DataLoader(
    val_dataset,
    batch_size=16, shuffle=False,
    num_workers=8, pin_memory=True)

cached = [] #queue.Queue()
for i, (_input, _target) in enumerate(val_loader):
    cached.append((_input, _target))  // the performance drops a lot when this line added
    //do something