So, when the first half of my EACH and EVERY training mini-batches are from class C0 and the second from class C1, my neural net trains up to 60% accuracy and then crashes to flat 50%! I finally figured out it is because my batches are ordered. For instance, they look like (C0, C0, C0, C1, C1, C1). To resolve this, the input tensor to the model must be shuffled, (C0, C1, C0, C1, C1, C0). Why would this matter? Doesn’t the weight/learning update step stem from the average of the losses of datapoints in a mini-batch? If that’s true, why would the order of classes in the training minibatch matter so much?