The easiest way would be to set a random seed, initialize the model once and save its state_dict.
Once your experiment is finished, just load the state_dict and your model will have the same weights again.
If you might want to exclude all random effects, you probably want to set shuffle=False in your DataLoader, since this could also change the results.
Also, you could set torch.backends.cudnn.deterministic = True, but that could slow down your code a lot.