I see that the recent version rather seems to load prefetch_factor*num_workers “samples”.
prefetch_factor (int, optional, keyword-only arg): Number of samples loaded
in advance by each worker. 2 means there will be a total of
2 * num_workers samples prefetched across all workers. (default: 2)
However, using different prefetch_factor values did not absolutely change the used GPU memory for my pipeline 1. But not sure if it is due to the customized dataloader or another issue with this newer pytorch functionality (hoping to spend more time on this soon, but would appreciate any feedback if someone happens to stop by to look at this).