Hi, anyone who cares.
I was aware of the functionality of torch.cuda.empty_cache() that calling this function can release the GPU memory which is no longer bound to a python variable but still in the memory pool. However, when I place the model in any GPU other than GPU 0 and call torch.cuda.empty_cache(), besides releasing memory on the specified GPU, about 700MB memory of GPU 0 will be occupied. It confused me so much and I can’t figure out why. Does anyone know about the internal machenism of this function and can explain why the strage memory comes out?
FYI, the approach of placing model to a certain GPU was specifying device index in pytorch code, rather than setting environment variable ‘CUDA_VISIBLE_DEVICES’ outside.