I would recommend to split the initial dataset either manually or with e.g. sklearn.model_selection.train_test_split.
Once you have the splits, you could create 3 separate Dataset objects, each with an own transformation (where the normalization stats were calculated from the training set) as well as three separate DataLoaders.
This would create a clean splitting in my opinion, which would be easily readable and might thus avoid potential data leaking.



 fkempf92:

I have very little experience with the batch normalisation tool that exists within Pytorch. Can I standardise my data ‘inside the network’ so I do not actually need to return standardised data in the first place?


BatchNorm layers normalize the activations inside the model, but usually you would still apply a normalization on the model inputs regardless, if batchnorm is used or not.