I don’t think there is a better generic solution than iterating over the dataset and recording the images per class. That said, you’ll be decoding all images once just for nothing, to a solution tailored to your dataset will be much more efficient.
Best regards
Thomas