Did you make your train_loader with shuffle=True?
Also I ran into problems where fetching all files wasn’t necessarily in the same order every time (i.e. if I have a folder of 10k files, sometimes the order that they are retrieved are different). To avoid this, you can sort your retrieved files (assuming you’re working with files)