kirtanp:

for i in range(10):
            for i in range(30):
                l = self._lagrangian(obj, constr, lmbda, tau, sign)
                optimizer.zero_grad()
                l.backward(retain_graph=True)
                optimizer.step()
                obj = self._get_objective(x, y)
                constr = self._get_constraints(x, y)
            lmbda = self._update_lambda(constr, lmbda, tau)
            print(x, y, obj, constr)



But I’m recomputing the obj in the loop here at each iteration. Or did I misunderstand what you meant?