Solved by albanD in post #6 
              
              
                HI, 
You can check the doc about how we manage the CUDA memory here. 
In particular, this will explain why the memory is not returned to the OS when you delete your model. 
For trying batch sizes, there are many things that can change the way the memory is allocated on the GPU and so, because of the…