Hmm your code works great, but although using approach #1 I still get the error:
Trying to backward through the graph a second time (or directly access saved variables after they have already been freed)…
except if I change the second loss function according to the comment
Maybe it’s triggered due to the loss functions that I’m using?
    net.train()
    optimizer.zero_grad()

    pred = net(x)
    loss_nat = nn.CrossEntropyLoss()(pred, y).mean()
    true_prob = nn.Softmax(dim=1)(pred)

    pred = net(adv_x)
    adv_prob = nn.LogSoftmax(dim=1)(adv_prob)
    loss_rob = nn.KLDivLoss()(adv_prob, true_prob).mean() # this doesn't work
    # loss_nat = nn.CrossEntropyLoss()(pred, y).mean() # this works, although not desired

    loss = loss_nat + loss_rob/_lambda
    loss.backward()