Solved by ptrblck in post #6 
              
              
                Based on your code it looks like you are using ImageFolder on your validation directory, which seem to contain only the images without any subfolders. 
ImageFolder creates the targets based on subfolders, so your current Datasets might contains only a single class label. Could you check that?