Yes, I am shuffling the data. My data loader is defined like so:
train_data = ImageDataset(train_file_list, transform=transform)        
train_data_loader = DataLoader(train_data, batch_size=128, shuffle=True, num_workers=4)