You could calculate the mean and std of the entire dataset for each feature and normalize each input by subtracting the mean and dividing by std. torchvision.transforms.Normalize applies the same normalization using the mean and std values for each channel.