Solved by christianperone in post #2 
              
              
                Have you looked at ConcatDataset documentation ? 
I think it would be the best approach, or you can extend the MNIST dataset class and concatenate your data after the constructor, so you’ll have to join your data with current loaded MNIST data.