Are you referring to your training loss or validation loss? The training loss should be worse when you are shuffling your data since there are many more combinations of batches. This is why we shuffle, so the model doesn’t overfit as much to the training data