It’s possible the issue isn’t your dataloader. In nearly every case, operations take place on the CPU, so any leak would appear in your RAM usage. How big are the batches in memory?  And how big is your model?
Can you post the code where you call .backward()?  If you are aggregating the total loss (I.e iterating through batches without calling .step()) gpu memory will increase as the gradient graphs get saved at each backward call.
I believe that most typical pytorch code doesn’t require explicit memory management, real memory leaks are pretty rare these days.