Using nvidia-smi. I expanded my script to include output from pytorch own commands for checking memory.
import torch
from torchvision import models

a = models.resnet34(pretrained=False)
a.cuda()
a.load_state_dict(torch.load('./resnet34-333f7ec4.pth'))
a.eval()

print("Max Memory occupied by tensors: ", torch.cuda.max_memory_allocated(device=None))
print("Max Memory Cached: ", torch.cuda.max_memory_cached(device=None))
print("Current Memory occupied by tensors: ", torch.cuda.memory_allocated(device=None))
print("Current Memory cached occupied by tensors: ", torch.cuda.memory_cached(device=None))

input("press any key to continue")

b = torch.rand(1,3,224,224)
b = b.to('cuda')
c = a(b)

print("\n\nMax Memory occupied by tensors: ", torch.cuda.max_memory_allocated(device=None))
print("Max Memory Cached: ", torch.cuda.max_memory_cached(device=None))
print("Current Memory occupied by tensors: ", torch.cuda.memory_allocated(device=None))
print("Current Memory cached occupied by tensors: ", torch.cuda.memory_cached(device=None))

input("Press enter to exit")


Here is the output:
Max Memory occupied by tensors:  87333888
Max Memory Cached:  90570752
Current Memory occupied by tensors:  87333888
Current Memory cached occupied by tensors:  90570752
press any key to continue
Max Memory occupied by tensors:  146021376
Max Memory Cached:  156631040
Current Memory occupied by tensors:  119782400
Current Memory cached occupied by tensors:  156631040
Press enter to exit
Im also attaching the output from nvidia-smi:
Before passing an input

Screenshot from 2019-05-28 13-08-30.png714×134 13.3 KB

After passign an input:

Screenshot from 2019-05-28 13-09-20.png715×141 13.3 KB

There is still a huge discrepancy between max memory cached and the amount of memory occupied as shown by nvidia-smi. Am i using the functions incorrectly?