thank you for your reply and explanation. In fact, I had to go through all this because I need to update the value of y_pred1 based on the parameters update of optimezer2.step before running loss1. All this without altering the computation graph of y_pred1.  more concretely optimizer2.step()  will update the l3 layer. a column of this layer I want to reinject to y_pred1 data and then calculate loss1 based on this new value but with the same gradient history.
thus I cannot use the first and second alternatives that you proposed.
I am doing this:
       loss2.backward(retain_graph=True)
       optimizer2.step()
       y_pred1.data = model.l3.weight[0, :].data.unsqueeze(0) # I am trying this one
       l1.append(loss1)
       loss1.backward()
       optimizer1.step()

is this valid? am I assigning a new value to the tensor y_pred1 and preserving everything else about its gradient?
why should y_pred1 be freed when calling loss2.backward() when the gradient is not a function of neither y_pred1, base_l1 nor base_l2? if retaining the gradient graph is a pain that we must go through could we do this partially? only for a set of parameters and not for the whole model?
thank you