fabiocapsouza:

you are creating a new graph every iteration


I think thats the part I don’t get.

Actually, to make it clear this is the code I plan to run:
for i in range(nb_iter):
    # Forward pass: compute predicted Y using operations on Variables
    batch_xs, batch_ys = get_batch2(data.X_train,data.Y_train,M,dtype) # [M, D], [M, 1]
    ## FORWARD PASS
    y_pred = mdl.forward(batch_xs)
    ## LOSS
        batch_loss = (1/M)*(y_pred - batch_ys).pow(2).sum()
    ## BACKARD PASS
    batch_loss.backward() # Use autograd to compute the backward pass. Now w will have gradients
    ## SGD update
    for W in mdl.parameters():
        delta = eta*W.grad.data
        W.data.copy_(W.data - delta) # W - eta*g + A*gdl_eps
    ## Manually zero the gradients after updating weights
    mdl.zero_grad()
    ## DO OP that should NOT be part of the computation graph
    if i % 100 == 0:
        Op(mdl)

in the above code when exactly is a new computation graph created?