I haven’t seen this been explicitly defined in the code, although in the “no trainer” script the shuffling is defined. Internally in datasets the shuffling is done or is it missing in the MLM with trainer?
If it’s missing, I can contribute with a PR