Thanks for your response.
I figured out the problem, I was using list to store the data instead of NumPy Array.
I documented everything I tried in this post, if someone is interested:

  
      
      Medium – 3 Apr 20
  
  
    

Memory Leakage with PyTorch 6

If you’re reading this post, then most probably you’re facing this problem. RAM is full, in the very beginning of the training, my data is…

  Reading time: 3 min read