Please note in libtorch for tensors on the GPU you may have to call c10::cuda::CUDACachingAllocator::empty_cache() once the tensor goes out of scope if you want pytorch to release this memory. Otherwise, it still frees the memory once tensor goes out of scope but still keeps that memory available in libtorch’s cache. If you want to free that memory for other processes then you must call empty_cache()