Hi! I am working on a simple classification problem. However, in my setup, I would like to create batches smarter than just by uniform sampling. Namely, I am trying to mine hard batches as following:

sample a big batch uniformly (e.g. 1024 samples)
apply my model to the big batch and calculate losses
sample a normal batch (e.g. 128 samples) out of the big batch using multinomial distribution parameterized by the losses from step 2

This procedure depends on the model and the model changes after every batch. Consequently, I do not want to preload future batches. In other words, I want the method  __next__ of dataloader.batch_sampler to be called only when the same method of dataloader itself is explicitly called. For example, in the following loop I would like the next batch to be created only after do returns
for batch in dataloader:
    do(batch)

I know that I can achieve that by setting num_workers=0. But what is the behavior if I set num_workers=some_positive_integer ? Does Dataloader preload future batches? If yes, is it possible to avoid it anyhow?
P.S. My final goal is to create my custom BatchSampler and the question, if answered, will help to understand if it is possible to yield batches instead of precalculating them for the whole epoch.