I have done little more googling based on what I figured from your explanation.
one of the git issue I found says that deleting pytorch cuda context is not going to be supported.
Is this something relevant?
  

      github.com/pytorch/pytorch
  

  
    
  
	  
  

  
    
      [feature request] a function to empty all the cuda buffer 15
    

    
      
        opened 
        
          
        
        May 23, 2019
      
      

        
          closed 
        
          
        
        Jun 24, 2019
      
        

      
        
          
          Stonesjtu
        
      
    

    
        
          module: cuda
        
        
          module: memory usage
        
        
          triage review
        
    
  


  
    ## 🚀 Feature


At present the pytorch process uses about 900MB after `lazy_in…it` cuda context. I was searching on internet whether the CUDA context can be shutdown, but seems like such API is not provided yet. Here I'm asking if we can do a `pytorch` side context clear, so the minimal CUDA memory should be allocated to pytorch runtime.

## Motivation



I'm developing an interesting function that each pytorch worker interacts with a scheduling server, dynamically moving the workload from/to GPU, so that the CUDA memory can be used for tasks with higher priority. 

## Pitch



```python
tensor = torch.Tensor(100,100).cuda()  # lazy_init happens here
del tensor
torch.cuda.de_init()  # clear all the CUDA memory maintained by pytorch runtime
```

## Alternatives



## Additional context