I guess you might be using an older PyTorch version, as you would get this error in the latest stable version:
RuntimeError: a view of a leaf Variable that requires grad is being used in an in-place operation.

Wrap your code into a with torch.no_grad() block and check, if the memory is still growing.