Solved by ptrblck in post #15 
              
              
                If you use datasets.ImageFolder then yes, your images should be located in separate folders which represent the classes. 
If you don’t want that, you can easily write your own Dataset and load the images using your own logic. 
Here is a good tutorial explaining, how to use Dataset. 
One way would be…