I don’t know which higher-level API you are using, but if I understand your issue correctly you are concerned about the determinism between two different runs using is_forward_pass=True and =False.
If so, I would guess that the pseudorandom number generator is called additionally in the if_forward_pass=True case and you thus won’t get the same values.
This could happen if you are using random transformations inside the DataLoader, shuffling, etc. or if the model itself is using random values e.g. in dropout layers and is not in evaluation mode.