[resolved] Is there MNIST dataset in torchvision.datasets?
<SCODE>from tensorflow.examples.tutorials.mnist import input_data
mnist = input_data.read_data_sets("./mnist/data/")
x_train = mnist.train.images    # numpy array 
y_train = mnist.train.labels
x_test = mnist.test.images
y_test = mnist.test.labels
<ECODE> Is there any simple way to handle this in PyTorch?
yunjey
yunjey
Yes it already there - see here and here, The code looks something like this, <SCODE>train_loader = torch.utils.data.DataLoader(
    datasets.MNIST('../data', train=True, download=True,
                   transform=transforms.Compose([
                       transforms.ToTensor(),
                       transforms.Normalize((0.1307,), (0.3081,))
                   ])),
    batch_size=args.batch_size, shuffle=True, **kwargs)
test_loader = torch.utils.data.DataLoader(
    datasets.MNIST('../data', train=False, transform=transforms.Compose([
                       transforms.ToTensor(),
                       transforms.Normalize((0.1307,), (0.3081,))
                   ])),
    batch_size=args.batch_size, shuffle=True, **kwargs)
<ECODE>
AjayTalati
How do you subset the MNIST training data? It’s 60,000 images, how can you reduce it to say 2000? Here’s the code <SCODE>>>> from torchvision import datasets, transforms


>>> 
>>> 
>>> train_all_mnist = datasets.MNIST('../data', train=True, download=True,
...                    transform=transforms.Compose([
...                        transforms.ToTensor(),
...                        transforms.Normalize((0.1307,), (0.3081,))
...                    ]))
Files already downloaded
>>> train_all_mnist
<torchvision.datasets.mnist.MNIST object at 0x7f89a150cfd0>
<ECODE> It’s a bit ugly - anyone know a neater way to do this?
AjayTalati
what is your purpose of subsetting the training dataset?
yunjey
I’m interested in Omniglot, which is like an inverse, MNIST, lots of classes, each with a small number of examples. Take a look, here By the way - thank you for your tutorials - they are very clear and helpful to learn from. Best regards, Ajay
AjayTalati
smth
Spent a hour hacking together my own loader -  but this looks better! Seems to be the easiest data set for experimenting with one-shot learning?
AjayTalati
Whats the current best methodology for Omniglot? Who or what’s doing the best at the moment?
AjayTalati
smth
I though you might like this, implementation of a GAN on Omniglot,
AjayTalati
Have you found a better way to do this?
ritchieng
Nope sorry - been totally snowed under the past couple of months - not had any time to work on it. If you’re referring to the alternative cost functions for GANs I don’t think they make much difference? Cheers, Aj
AjayTalati
It includes Omniglot data downloading and iterating scripts along with all the models proposed in the paper (the nets are written and trained with Theano).
pranav