The internal classes and class_to_idx attributes are used in DatasetFolder's __init__ to create the samples as seen in these lines of code 272.
After ImageFolder was initialized (DatasetFolder is the parent class of it), changing these values won’t have any effect on the samples anymore, so you might want to derive your custom class from DatasetFolder and change these attributes in the __init__ method.