Could you try to wrap the dataloader2 loop into a separate function?
Since Python uses function scoping, all references will be deleted once you exit the function.
Would it be possible to move the dataloader2 loop outside of the dataloader1 loop?
This would allow you to write both loops into functions, which might reduce unnecessary storage of tensors.