There are some points we could check to see why your script is slower than usual:

How did you install PyTorch? Via conda/pip or built from source?
Is your data locally stored on an SSD?
Is cuDNN enabled? (print(torch.backends.cudnn.enabled))
Could you monitor the power usage of your GPU in nvidia-smi? (I hope there is some equivalent for Windows) Maybe your GPU is overheating or your power supply is not sufficient.

Also, since you are using Windows, could you copy all the source code into a new script file and guard it with:
def main():
    # Code from tutorial


if __name__=='__main__':
    main()

This shouldn’t be a problem, because the code runs on your machine, but might be worth a try.