If your original Dataset is fine, the Subsets shouldn’t be changed, since only the passed indices are called in this line of code 18.
Could you check which index in your subset is giving the wrong class and then try to get the original index based on this information? Maybe this helps debugging which file seems to be wrong.
Alternatively, you could write your own custom Dataset and return the image names along the data and target, so that debugging might be a bit easier.