I need to solve an unsupervised problem with images from MNIST and SVHN, in which I have 100 images from MNIST and 10 images from SVHN).  I need a pre-trained net to learn how to classify if a given image is from MNIST or SVHN (the anomaly). Basically, it’s an anomaly detection problem.
I know I’ll have to tackle that later with integration of a clustering technique/SVM/whatever with the net somehow, or AE, and a proper loss, but first things first 
It’s very easy to create  a regular Dataloader for MNIST and SVHN in PyTorch, but in this case I need to create a dataloader that includes a small subset of the two datasets without the labels (that’s the more interesting part here:) ). How can I do that?