Solved by ptrblck in post #4 
              
              
                This could explain the peaks, as some samples might be harder to classify than others. 
If you shuffle the dataset, this cyclic behavior should disappear.