It should be sufficient to set the seed after the first import of PyTorch.
As long as the control flow stays the same (i.e. the calls to the pseudo-random number generator are the same), you should get the same outputs (besides of course the limitations mentioned in the reproducibility docs).