You could use class_names param to control order of classes.

  
      

      TensorFlow
  

  
    

tf.keras.utils.image_dataset_from_directory  |  TensorFlow Core v2.7.0 2

  Generates a tf.data.Dataset from image files in a directory.