Solved by bhushans23 in post #2 
              
              
                Shuffling is always a good idea. 
For Example, you are writing a classifier for 10 different category and see category 1 as first 1000 examples and so on for rest of the category. In such a cases, networks is first overfitting to category 1 and then to other category. Network in such cases, is not a…