Dont use ImageFolder, just read the images using os.walk for example and store their paths and labels (based on which class they reside in you can give them a number) and then in __getitem__, after you have read the image, based on the label that you also have, you can apply the needed transformation  and then return the pair.