I’m not sure I understand the use case completely, but loading parameters into the model via a new or manipulated state_dict is independent from the computation graph and should be done before or after the training step. I.e. performing the forward pass will use the current parameter set (P0) to compute the intermediate forward activations (A0), which will be used to compute the gradients (G0). Changing the parameters to P1 would try to use A0 to calculate the gradients, which is wrong, so could you explain the use case a bit more?