Solved by futscdav in post #12 
              
              
                Most likely caused by having grayscale images in the dataset. Add .convert('RGB') after loading your PIL image in the dataloader.