@albanD I actually realized that I was doing my calculations on a variable that requires gradient (x). So my code was keeping storing gradient at every iteration.
So I changed my code like this
x_ng = x.detach()
x_ = torch.zeros_like(x_ng)

for i in range(my_range):
	x_ng = do_something(x_ng)
	x_ += x_ng   
	x_ng -= x_   

x.data=x_ng.data


I don’t know if it’s a smart strategy but surely it occupies less memory.