Actually, I need to see the code. Anyway, how did you get the “train_loss”, is it a kind of average? If you get the average by adding the “loss” up something like:
loss = ((y_hat - y) ** 2).mean()
sum_loss += loss
avg_loss = sum_loss / itr
print('train_loss', float(avg_loss))

Memory leaking comes from the sum_loss, since it will hold the graph for each iteration from the first iteration. Using .detach() should help in this case, but frankly you could just use float(loss) altogether.