I’ve tried to calculate free memory by total_memory - memory_allocated or  total_memory - memory_reserved,  I’ve also tried to do it with pynvml, here is the code snippet:
def remaining_memory():
  pynvml.nvmlInit()
  gpu_handle = pynvml.nvmlDeviceGetHandleByIndex(0)
  info = pynvml.nvmlDeviceGetMemoryInfo(gpu_handle)
  return info.free

and none of them are accurate.
I guess it has something to do with memory fragmentation, if the remaining space is not contiguous, you can’t fit a large tensor into it even if it shows there are enough free memory. I just want to know how pytorch checks whether a tensor will fit in memory.