Normalization of CIFAR and MNIST datasets
<SCODE># The output of torchvision datasets are PIL Image images of range [0, 1].
# We transform them to Tensors of normalized range [-1, 1]
transform=transforms.Compose([transforms.ToTensor(),
                              transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5)),
                             ])
trainset = torchvision.datasets.CIFAR10(root='./data', train=True, download=True, transform=transform)
trainloader = torch.utils.data.DataLoader(trainset, batch_size=4, 
                                          shuffle=True, num_workers=2)
<ECODE> But the MNIST example (pytorch/examples/mnist) uses values very different from 0.5 to normalize the data: <SCODE>testset = torchvision.datasets.CIFAR10(root='./data', train=False, download=True, transform=transform)
testloader = torch.utils.data.DataLoader(testset, batch_size=4, 
                                          shuffle=False, num_workers=2)

datasets.MNIST('../data', train=True, download=True,
               transform=transforms.Compose([
                   transforms.ToTensor(),
                   transforms.Normalize((0.1307,), (0.3081,))
               ])),
batch_size=args.batch_size, shuffle=True, **kwargs)
<ECODE> Why are we using two different approaches to normalize MINIST and CIFAR? What is the correct approach to use? I guess we are NOT normalizing the dataset in the first case, but only each image. If this is the case, I think we should clarify this in the PyTorch Tutorial. Thanks, David
dlmacedo
MNIST is not natural images, it’s data distribution is quite different.
smth
BarryBA