Pytorch is not quite clear about this part, in fact, in this previously asked question (by me):

  
    
    
    Pytorch cpu memory usage 
  
  
    Hi, 
The allocator I mention here is not from pytorch but from your libc (we have no control over it). These are classic CPU allocators. Famous alternatives include jemalloc or tmalloc. But I haven’t tested them myself.
  


In come cases (my implementation 2), even if the tensors are dereferenced, their memory are still not freed immediately after dereferencing, for more deep down implementation information, maybe you could consider asking the pytorch development member @ptrblck and @albanD.
I have not tried the jemalloc solution yet, but it is easy enough to install it by simply making it like:
wget https://github.com/jemalloc/jemalloc/releases/download/5.0.1/jemalloc-5.0.1.tar.bz2
tar -jxvf jemalloc-5.0.1.tar.bz2
cd jemalloc-5.0.1
sudo apt-get install autogen autoconf

./autogen.sh
make -j2
sudo make install
sudo ldconfig
cd ../
rm -rf jemalloc-5.0.1 jemalloc-5.0.1.tar.bz2

then export LD_PRELOAD environment variable before running your program:
LD_PRELOAD=/home/mingfeim/packages/jemalloc-5.2.0/lib/libjemalloc.so ./your_script.sh

Give it a try?
Reference:

  
      
      cnblogs.com
  
  
    

Linux安装jemalloc笔记 - evenleo - 博客园 2

前言 最近研究一个工具库需要用 jemalloc 做内存分配器，但在 ubuntu 下安装过程中遇到很多问题，故记下安装过程的笔记，避免以后遇到在这上面浪费时间。 安装过程 环境：VMware Ubu


  
  
    
    
  
  



  
      
      知乎专栏
  
  
    

PyTorch在CPU上的一些Performance BKM 3

这里简单介绍一下用PyTorch在CPU上的一些性能相关的BKM。内容以inference为主，毕竟CPU上主要的场景还是inference；另外这里CPU都指的是Intel Xeon.gist里面写了英文版的，内容和这里的基本相当： General guideli…