ngxbac:

Issue: Potential memory leak in Tensor.size()


I am facing this issue even with the updated PyTorch nightly version. The Dataloder memory usage continuously increases until it runs of memory. My Dataset size is 26GB when initialized, it contains an ndarray from which I return an element based on index value. After running on 10% of data it ends up using another 30+GB of ram and 40GB + swap space. I tried upgrading the PyTorch with the latest nightly version. Tried both on python3.6 and Python3.7, but the issue persists.
Thanks,
Vishnu