I cannot reproduce the issue using a recent master build:
model = models.resnet18()
print(torch.cuda.memory_allocated())
> 0

model.cuda()
print(torch.cuda.memory_allocated())
> 46861312

del model
print(torch.cuda.memory_allocated())
> 0