Usually you would apply the transformations on each sample in the __getitem__ method of your Dataset. Have a look at the data loading tutorial 60 for an example.
If you wrap your Dataset in a DataLoader with multiple workers, the loading and processing will be applied using multiprocessing in the background while your GPU is busy with the training.
While a lot of torchvision.transforms are applied on PIL.Images, Normalize is applied on a tensor which is why I suggested to call it after ToTensor.
Note that ToTensor will return a tensor image with values in the range [0, 1]. Your mean and std in Normalize should therefore also be relative to this range.