Because the estimate of the expected gradients is biased if your data is ordered.
Have a look at Chapter 8 - Deep Learning book 5:

It is also crucial that the minibatches be selected randomly. Computing an unbiased estimate of the expected gradient from a set of samples requires that those samples be independent. We also wish for two subsequent gradient estimates to be independent from each other, so two subsequent minibatches of examples should also be independent from each other.

I can’t explain it that well as Ian Goodfellow does, but I understand it from a point of view, where you force your model to a minimum for the current class. All weight updates aim towards the minimum of class0, until you finally provide samples for class1. This minimum of class1 might be slightly different. If your parameters are already in a “class0 valley” it might be hard for your model to escape it.
That’s probably a very shallow view on the training and loss function, but it helps me visualizing, what could be happening.