Hello,
I have a agile algorithm that use a changing amount of GPUs, increasing and decreasing the number of GPUs that are needs.It constantly checked for free GPUs, but it seems that I can not really free a GPU after I already sent an object to the GPU.
For example:
import torch
tm = torch.Tensor([1,2]).to("cuda")
!nvidia-smi

|===============================+======================+======================|
|   0  GeForce RTX 208...  On   | 00000000:3D:00.0 Off |                  N/A |
|  0%   37C    P2    52W / 250W |    730MiB / 10989MiB |      0%      Default

So I use 730MiB…
Now no matter what I try I can not make the 730MiB go to zero:
del tm                                                                                                                                                                                                 
torch.cuda.empty_cache()                                                                                                                                                                               
import sys;sys.modules[__name__].__dict__.clear()                                                                                                                                                      
%reset                                                                                                                                                                                                 
Once deleted, variables cannot be recovered. Proceed (y/[n])? y
!nvidia-smi
|   0  GeForce RTX 208...  On   | 00000000:3D:00.0 Off |                  N/A |
|  0%   35C    P8     1W / 250W |    728MiB / 10989MiB |      0%      Default |

Any help?
Thanks