I cannot reproduce this issue using:
import torch
import psutil

a = torch.randn(256, 5000, 1, device='cuda')

for epoch in range(10):
    for n in range(1, 100):
        b = torch.randn(256, 1, n, device='cuda')
        out = a * b
        print('Epoch {} Iter {}'.format(epoch, n))
        print(psutil.virtual_memory().used)
        print('GPU allocated mem {:.3f}MB'.format(torch.cuda.memory_allocated()/1024**2))

and get:
Epoch 0 Iter 1
16522018816
GPU allocated mem 9.767MB
Epoch 0 Iter 2
16522534912
GPU allocated mem 15.119MB
[...]
Epoch 0 Iter 98
16523194368
GPU allocated mem 483.494MB
Epoch 0 Iter 99
16523194368
Epoch 1 Iter 1
16523194368
GPU allocated mem 9.767MB
Epoch 1 Iter 2
16523194368
GPU allocated mem 15.119MB
[...]
Epoch 9 Iter 98
16523194368
GPU allocated mem 483.494MB
Epoch 9 Iter 99
16523194368

Which shows an overall host memory increase of (16523194368 - 16522018816) / 1024**2 ~= 1.12MB.
EDIT: I would generally recommend to update PyTorch to the latest release especially since you are also using the deprecated Python2.7 version.