I  know this doesn‘t make sense usually, but in this case it would be easier, I mean.

Well if you disable these checks, all the gradients the gradient computes in this case will be wrong. So that won’t really help you.
You might want to modify the code to move the load_state_dict() after the backward to make sure that you don’t override values that are required.
Otherwise, you can change your model’s forward to .clone() the parameters before using them so that the value modified inplace by the load_state_dict() is not the one that is needed to compute the backward.