To measure the GPU memory usage, I use
stats = torch.cuda.memory_stats()
current_active_byte =  stats["active_bytes.all.current"]

and calculate the difference before and after the statement of interest.
The value of current_active_byte does not decrease with tensor_var.cpu() or del tensor_var if  require_grad=True.