When you work with Variable, some intermediate results are saved to be able to perform the backward pass if you ever call it.
Moreover, pytorch uses a custom allocator on gpu for efficiency reasons, so the memory that is freed is not returned to the gpu. You can use torch.cuda.empty_cache() to remove all unused memory from the custom allocator.