if then the python script delete this tensor, does it effectively clear the memory?

Yes

is it deleted once we exit the function scope like in Python or do we have to delete it manually?

It is deleted when you exit the scope, unless you return it.
How do you monitor the cuda memory? Note that we have a caching allocator that does not return the memory to the driver. So nvidia-smi memory report will always grow.
You can use torch.cuda.memory_allocated() to see the memory actually used to store Tensors.