Arturas_Druteika:

But doesn’t that mean that on the 1-st step I have to create a loader, then on the 2-nd step calculate mean and std and then as a 3-rd step create a new loader with Normalize using mean and std values calculated in a previous step?


Yes, you could use either this workflow (or the Dataset directly as mentioned by @omarfoq).
Note, that calculating the dataset stats would only be needed once (if you have a fixed training, validation, and test split) and you could thus reuse the stats in all future experiments (as is done with the ImageNet stats).