Solved by albanD in post #2 
              
              
                if then the python script delete this tensor, does it effectively clear the memory? 

Yes 

is it deleted once we exit the function scope like in Python or do we have to delete it manually? 

It is deleted when you exit the scope, unless you return it. 
How do you monitor the cuda memory? Note that…