Hi,
I do not think the problem is with the normalization. Here is the transform for both set-ups.
transforms.RandomResizedCrop(input_size),
transforms.RandomHorizontalFlip(),
transforms.ToTensor(),
transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])

I also want to note that the CIFAR-10 using ImageFolder works with a network that is manually created but not by torchvision preset network.