Your approach should work as shown here:
print(torch.cuda.memory_allocated())
> 0
print(torch.cuda.memory_reserved())
> 0

t = torch.zeros((1, 2**6, 2**6)).to(f'cuda')
print(torch.cuda.memory_allocated())
> 16384
print(torch.cuda.memory_reserved())
> 2097152

del t
print(torch.cuda.memory_allocated())
> 0
print(torch.cuda.memory_reserved())
> 2097152

torch.cuda.empty_cache()
print(torch.cuda.memory_allocated())
> 0
print(torch.cuda.memory_reserved())
> 0

Note that the first CUDA operation will create the CUDA context on the device, which will load all kernels, cudnn, etc. onto the device.
This memory is not reported by torch.cuda.memory_allocated() and torch.cuda.memory_reserved() and can be seen via nvidia-smi.