I hope to switch between two batch sizes for training data during data, for different epochs.
For example,
if epoch%2==0:    train with batch_size 4 (i.e. this is a for loop to grab each batch of size 4)
else:  train with batch_size 8 (i.e. this is a for loop to grab each batch of size 8)

I tried to define two train_loaders at the very beginning like this:
train_small = torch.util.data.Dataloader(.., batch_size=4 ,,,)
train_large = torch.util.data.Dataloader(.., batch_size=8 ,,,)
if epoch%2==0:
    train_loader =  train_small
else 
    train_loader = train_large
for input, target in tqdm(train_loader ...):
    do some training.

But got the “out of cuda memory” error.  However,  if I just use “train_large” for all epochs, it is ok.
Is this the correct approach to switch between 2 batch sizes during training?
The reason I am doing this is, different training epoch consumes different size of memory, so I want to use smaller batch for even epochs but a larger batch size for odd epochs. Could someone provide some guidance on how to achieve that?  Thanks.